From owner-svn-src-all@freebsd.org Wed Nov 28 19:17:29 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5BBF2113A323; Wed, 28 Nov 2018 19:17:29 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C868182DCA; Wed, 28 Nov 2018 19:17:28 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A67E76A61; Wed, 28 Nov 2018 19:17:28 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id wASJHSJC066550; Wed, 28 Nov 2018 19:17:28 GMT (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id wASJHSXP066547; Wed, 28 Nov 2018 19:17:28 GMT (envelope-from glebius@FreeBSD.org) Message-Id: <201811281917.wASJHSXP066547@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: glebius set sender to glebius@FreeBSD.org using -f From: Gleb Smirnoff Date: Wed, 28 Nov 2018 19:17:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r341162 - head/sys/vm X-SVN-Group: head X-SVN-Commit-Author: glebius X-SVN-Commit-Paths: head/sys/vm X-SVN-Commit-Revision: 341162 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: C868182DCA X-Spamd-Result: default: False [0.65 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_SHORT(-0.17)[-0.165,0]; NEURAL_SPAM_MEDIUM(0.31)[0.308,0]; NEURAL_SPAM_LONG(0.51)[0.511,0] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Nov 2018 19:17:29 -0000 Author: glebius Date: Wed Nov 28 19:17:27 2018 New Revision: 341162 URL: https://svnweb.freebsd.org/changeset/base/341162 Log: For not offpage zones the slab is placed at the end of page. Keg's uk_pgoff is calculated to guarantee that struct uma_slab is placed at pointer size alignment. Calculation of real struct uma_slab size is done in keg_ctor() and yet again in keg_large_init(), to check if we need an extra page. This calculation can actually be performed at compile time. - Add SIZEOF_UMA_SLAB macro to calculate size of struct uma_slab placed at an end of a page with alignment requirement. - Use SIZEOF_UMA_SLAB in keg_ctor() and in keg_large_init(). This is a not a functional change. - Use SIZEOF_UMA_SLAB in UMA_SLAB_SPACE definition and in keg_small_init(). This is a potential bugfix, but in reality I don't think there are any systems affected, since compiler aligns struct uma_slab anyway. Modified: head/sys/vm/uma_core.c head/sys/vm/uma_int.h Modified: head/sys/vm/uma_core.c ============================================================================== --- head/sys/vm/uma_core.c Wed Nov 28 18:42:42 2018 (r341161) +++ head/sys/vm/uma_core.c Wed Nov 28 19:17:27 2018 (r341162) @@ -1453,7 +1453,7 @@ keg_small_init(uma_keg_t keg) if (keg->uk_flags & UMA_ZONE_OFFPAGE) shsize = 0; else - shsize = sizeof(struct uma_slab); + shsize = SIZEOF_UMA_SLAB; if (rsize <= slabsize - shsize) keg->uk_ipers = (slabsize - shsize) / rsize; @@ -1523,7 +1523,6 @@ keg_small_init(uma_keg_t keg) static void keg_large_init(uma_keg_t keg) { - u_int shsize; KASSERT(keg != NULL, ("Keg is null in keg_large_init")); KASSERT((keg->uk_flags & UMA_ZFLAG_CACHEONLY) == 0, @@ -1536,23 +1535,17 @@ keg_large_init(uma_keg_t keg) keg->uk_rsize = keg->uk_size; /* Check whether we have enough space to not do OFFPAGE. */ - if ((keg->uk_flags & UMA_ZONE_OFFPAGE) == 0) { - shsize = sizeof(struct uma_slab); - if (shsize & UMA_ALIGN_PTR) - shsize = (shsize & ~UMA_ALIGN_PTR) + - (UMA_ALIGN_PTR + 1); - - if (PAGE_SIZE * keg->uk_ppera - keg->uk_rsize < shsize) { - /* - * We can't do OFFPAGE if we're internal, in which case - * we need an extra page per allocation to contain the - * slab header. - */ - if ((keg->uk_flags & UMA_ZFLAG_INTERNAL) == 0) - keg->uk_flags |= UMA_ZONE_OFFPAGE; - else - keg->uk_ppera++; - } + if ((keg->uk_flags & UMA_ZONE_OFFPAGE) == 0 && + PAGE_SIZE * keg->uk_ppera - keg->uk_rsize < SIZEOF_UMA_SLAB) { + /* + * We can't do OFFPAGE if we're internal, in which case + * we need an extra page per allocation to contain the + * slab header. + */ + if ((keg->uk_flags & UMA_ZFLAG_INTERNAL) == 0) + keg->uk_flags |= UMA_ZONE_OFFPAGE; + else + keg->uk_ppera++; } if ((keg->uk_flags & UMA_ZONE_OFFPAGE) && @@ -1693,20 +1686,11 @@ keg_ctor(void *mem, int size, void *udata, int flags) /* * If we're putting the slab header in the actual page we need to - * figure out where in each page it goes. This calculates a right - * justified offset into the memory on an ALIGN_PTR boundary. + * figure out where in each page it goes. See SIZEOF_UMA_SLAB + * macro definition. */ if (!(keg->uk_flags & UMA_ZONE_OFFPAGE)) { - u_int totsize; - - /* Size of the slab struct and free list */ - totsize = sizeof(struct uma_slab); - - if (totsize & UMA_ALIGN_PTR) - totsize = (totsize & ~UMA_ALIGN_PTR) + - (UMA_ALIGN_PTR + 1); - keg->uk_pgoff = (PAGE_SIZE * keg->uk_ppera) - totsize; - + keg->uk_pgoff = (PAGE_SIZE * keg->uk_ppera) - SIZEOF_UMA_SLAB; /* * The only way the following is possible is if with our * UMA_ALIGN_PTR adjustments we are now bigger than @@ -1714,13 +1698,10 @@ keg_ctor(void *mem, int size, void *udata, int flags) * mathematically possible for all cases, so we make * sure here anyway. */ - totsize = keg->uk_pgoff + sizeof(struct uma_slab); - if (totsize > PAGE_SIZE * keg->uk_ppera) { - printf("zone %s ipers %d rsize %d size %d\n", - zone->uz_name, keg->uk_ipers, keg->uk_rsize, - keg->uk_size); - panic("UMA slab won't fit."); - } + KASSERT(keg->uk_pgoff + sizeof(struct uma_slab) <= + PAGE_SIZE * keg->uk_ppera, + ("zone %s ipers %d rsize %d size %d slab won't fit", + zone->uz_name, keg->uk_ipers, keg->uk_rsize, keg->uk_size)); } if (keg->uk_flags & UMA_ZONE_HASH) Modified: head/sys/vm/uma_int.h ============================================================================== --- head/sys/vm/uma_int.h Wed Nov 28 18:42:42 2018 (r341161) +++ head/sys/vm/uma_int.h Wed Nov 28 19:17:27 2018 (r341162) @@ -139,9 +139,17 @@ #define UMA_MAX_WASTE 10 /* - * Size of memory in a not offpage slab available for actual items. + * Actual size of uma_slab when it is placed at an end of a page + * with pointer sized alignment requirement. */ -#define UMA_SLAB_SPACE (UMA_SLAB_SIZE - sizeof(struct uma_slab)) +#define SIZEOF_UMA_SLAB ((sizeof(struct uma_slab) & UMA_ALIGN_PTR) ? \ + (sizeof(struct uma_slab) & ~UMA_ALIGN_PTR) + \ + (UMA_ALIGN_PTR + 1) : sizeof(struct uma_slab)) + +/* + * Size of memory in a not offpage single page slab available for actual items. + */ +#define UMA_SLAB_SPACE (PAGE_SIZE - SIZEOF_UMA_SLAB) /* * I doubt there will be many cases where this is exceeded. This is the initial