From owner-svn-src-head@freebsd.org Mon Jan 6 22:48:53 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3BCF51F8E14; Mon, 6 Jan 2020 22:48:53 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (glebi.us [162.251.186.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47s9fX6WbPz4MBv; Mon, 6 Jan 2020 22:48:52 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.15.2/8.15.2) with ESMTPS id 006MmoR0034496 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 6 Jan 2020 14:48:50 -0800 (PST) (envelope-from glebius@freebsd.org) Received: (from glebius@localhost) by cell.glebi.us (8.15.2/8.15.2/Submit) id 006MmoBt034495; Mon, 6 Jan 2020 14:48:50 -0800 (PST) (envelope-from glebius@freebsd.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@freebsd.org using -f Date: Mon, 6 Jan 2020 14:48:50 -0800 From: Gleb Smirnoff To: Jeff Roberson Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r356393 - head/sys/vm Message-ID: <20200106224850.GJ3339@FreeBSD.org> References: <202001060251.0062pJe9067924@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <202001060251.0062pJe9067924@repo.freebsd.org> User-Agent: Mutt/1.12.2 (2019-09-21) X-Rspamd-Queue-Id: 47s9fX6WbPz4MBv X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-5.99 / 15.00]; NEURAL_HAM_MEDIUM(-0.99)[-0.995,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jan 2020 22:48:53 -0000 Jeff, On Mon, Jan 06, 2020 at 02:51:19AM +0000, Jeff Roberson wrote: J> Modified: head/sys/vm/vm_page.c J> ============================================================================== J> --- head/sys/vm/vm_page.c Mon Jan 6 01:51:23 2020 (r356392) J> +++ head/sys/vm/vm_page.c Mon Jan 6 02:51:19 2020 (r356393) J> @@ -613,10 +613,17 @@ vm_page_startup(vm_offset_t vaddr) J> slab_ipers(sizeof(struct vm_map), UMA_ALIGN_PTR)); J> J> /* J> - * Before going fully functional kmem_init() does allocation J> - * from "KMAP ENTRY" and vmem_create() does allocation from "vmem". J> + * Before we are fully boot strapped we need to account for the J> + * following allocations: J> + * J> + * "KMAP ENTRY" from kmem_init() J> + * "vmem btag" from vmem_startup() J> + * "vmem" from vmem_create() J> + * "KMAP" from vm_map_startup() J> + * J> + * Each needs at least one page per-domain. J> */ J> - boot_pages += 2; J> + boot_pages += 4 * vm_ndomains; J> #endif J> /* J> * CTFLAG_RDTUN doesn't work during the early boot process, so we must I think this chunk brings many extraneous pages per domain. Allocations for "vmem btag" from vmem_startup() are covered by vmem_startup_count(), and allocations for "KMAP" from vm_map_startup() are also counted just a few lines above: /* vmem_startup() calls uma_prealloc(). */ boot_pages += vmem_startup_count(); /* vm_map_startup() calls uma_prealloc(). */ boot_pages += howmany(MAX_KMAP, slab_ipers(sizeof(struct vm_map), UMA_ALIGN_PTR)); The uma_prealloc() allocates as much memory as asked, it may not populate all domains. So, on NUMA it will allocate same amount of pages. Not sure if this is a right decision, but at lease the counting is correct. So, change from 2 pages to 4 isn't needed. The vm_ndomains multiplicator also seems incorrect, since at early stage we run with buckets disabled, and this single allocation will end with calling zone_alloc_item() calling zone_import() with count of 1, which will short-circuit into keg_fetch_slab() before entering domain aware code. Then keg_fetch_slab() will short circuit into startup_alloc(). Do machines with NUMA and lack of UMA_MD_SMALL_ALLOC exist? Interesting to see dmesg with DIAGNOSTIC and this change. -- Gleb Smirnoff