Date: Mon, 6 Jan 2020 14:48:50 -0800 From: Gleb Smirnoff <glebius@freebsd.org> To: Jeff Roberson <jeff@freebsd.org> Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r356393 - head/sys/vm Message-ID: <20200106224850.GJ3339@FreeBSD.org> In-Reply-To: <202001060251.0062pJe9067924@repo.freebsd.org> References: <202001060251.0062pJe9067924@repo.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Jeff,
On Mon, Jan 06, 2020 at 02:51:19AM +0000, Jeff Roberson wrote:
J> Modified: head/sys/vm/vm_page.c
J> ==============================================================================
J> --- head/sys/vm/vm_page.c Mon Jan 6 01:51:23 2020 (r356392)
J> +++ head/sys/vm/vm_page.c Mon Jan 6 02:51:19 2020 (r356393)
J> @@ -613,10 +613,17 @@ vm_page_startup(vm_offset_t vaddr)
J> slab_ipers(sizeof(struct vm_map), UMA_ALIGN_PTR));
J>
J> /*
J> - * Before going fully functional kmem_init() does allocation
J> - * from "KMAP ENTRY" and vmem_create() does allocation from "vmem".
J> + * Before we are fully boot strapped we need to account for the
J> + * following allocations:
J> + *
J> + * "KMAP ENTRY" from kmem_init()
J> + * "vmem btag" from vmem_startup()
J> + * "vmem" from vmem_create()
J> + * "KMAP" from vm_map_startup()
J> + *
J> + * Each needs at least one page per-domain.
J> */
J> - boot_pages += 2;
J> + boot_pages += 4 * vm_ndomains;
J> #endif
J> /*
J> * CTFLAG_RDTUN doesn't work during the early boot process, so we must
I think this chunk brings many extraneous pages per domain. Allocations
for "vmem btag" from vmem_startup() are covered by vmem_startup_count(),
and allocations for "KMAP" from vm_map_startup() are also counted just a
few lines above:
/* vmem_startup() calls uma_prealloc(). */
boot_pages += vmem_startup_count();
/* vm_map_startup() calls uma_prealloc(). */
boot_pages += howmany(MAX_KMAP,
slab_ipers(sizeof(struct vm_map), UMA_ALIGN_PTR));
The uma_prealloc() allocates as much memory as asked, it may not populate
all domains. So, on NUMA it will allocate same amount of pages. Not sure
if this is a right decision, but at lease the counting is correct. So, change
from 2 pages to 4 isn't needed.
The vm_ndomains multiplicator also seems incorrect, since at early stage
we run with buckets disabled, and this single allocation will end with
calling zone_alloc_item() calling zone_import() with count of 1, which
will short-circuit into keg_fetch_slab() before entering domain aware code.
Then keg_fetch_slab() will short circuit into startup_alloc().
Do machines with NUMA and lack of UMA_MD_SMALL_ALLOC exist? Interesting
to see dmesg with DIAGNOSTIC and this change.
--
Gleb Smirnoff
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20200106224850.GJ3339>