From owner-freebsd-arch@FreeBSD.ORG Fri Jul 30 21:19:08 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 583211065677; Fri, 30 Jul 2010 21:19:08 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 271C68FC0A; Fri, 30 Jul 2010 21:19:08 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 2876946B7F; Fri, 30 Jul 2010 17:19:07 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 250A18A03C; Fri, 30 Jul 2010 17:19:06 -0400 (EDT) From: John Baldwin To: Alan Cox Date: Fri, 30 Jul 2010 16:14:40 -0400 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20100217; KDE/4.4.5; amd64; ; ) References: <4C4DB2B8.9080404@freebsd.org> <201007270935.52082.jhb@freebsd.org> <4C531ED7.9010601@cs.rice.edu> In-Reply-To: <4C531ED7.9010601@cs.rice.edu> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201007301614.40768.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 30 Jul 2010 17:19:06 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: alc@freebsd.org, Matthew Fleming , Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: amd64: change VM_KMEM_SIZE_SCALE to 1? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jul 2010 21:19:08 -0000 On Friday, July 30, 2010 2:49:59 pm Alan Cox wrote: > John Baldwin wrote: > > I have a strawman of that (relative to 7). It simply adjusts the hardcoded > > maximum to instead be a function of the amount of physical memory. > > > > > > Unless I'm misreading this patch, it would allow "desiredvnodes" to grow > (slowly) on i386/PAE starting at 5GB of RAM until we reach the (too > high) "virt" limit of about 329,000. Yes? For example, an 8GB i386/PAE > machine would have 60% more vnodes than was allowed by MAXVNODE_MAX, and > it would not stop there. I think that we should be concerned about > that, because MAXVNODE_MAX came about because the "virt" limit wasn't > working. Agreed. > As the numbers above show, we could more than halve the growth rate for > "virt" and it would have no effect on either amd64 or i386 machines with > up to 1.5GB of RAM. They would have just as many vnodes. Then, with > that slower growth rate, we could simply eliminate MAXVNODES_MAX (or at > least configure it to some absurdly large value), thereby relieving the > fixed cap on amd64, where it isn't needed. > > With that in mind, the following patch slows the growth of "virt" from > 2/5 of vm_kmem_size to 1/7. This has no effect on amd64. However, on > i386. it allows desiredvnodes to grow slowly for machines with 1.5GB to > about 2.5GB of RAM, ultimately exceeding the old desiredvnodes cap by > about 17%. Once we exceed the old cap, we increase desiredvnodes at a > marginal rate that is almost the same as your patch, about 1% of > physical memory. It's just computed differently. > > Using 1/8 instead of 1/7, amd64 machines with less than about 1.5GB lose > about 7% of their vnodes, but they catch up and pass the old limit by > 1.625GB. Perhaps, more importantly, i386 machines only exceed the old > cap by 3%. > > Thoughts? I think this is much better. My strawman was rather hackish in that it was layering a hack on top of the existing calculations. I prefer your approach. I do not think penalizing amd64 machines with less than 1.5GB is a big worry as most x86 machines with a small amount of memory are probably running as i386 anyway. Given that, I would probably lean towards 1/8 instead of 1/7, but I would be happy with either one. > Index: kern/vfs_subr.c > =================================================================== > --- kern/vfs_subr.c (revision 210504) > +++ kern/vfs_subr.c (working copy) > @@ -284,21 +284,29 @@ SYSCTL_INT(_debug, OID_AUTO, vnlru_nowhere, CTLFLA > * Initialize the vnode management data structures. > */ > #ifndef MAXVNODES_MAX > -#define MAXVNODES_MAX 100000 > +#define MAXVNODES_MAX 8388608 /* Reevaluate when physmem > exceeds 512GB. */ > #endif How is this value computed? I would prefer something like: '512 * 1024 * 1024 * 1024 / (sizeof(struct vnode) + sizeof(struct vm_object) / N' if that is how it is computed. A brief note about the magic number of 393216 would also be nice to have (and if it could be a constant with a similar formula value that would be nice, too.). > static void > vntblinit(void *dummy __unused) > { > + int physvnodes, virtvnodes; > > /* > - * Desiredvnodes is a function of the physical memory size and > - * the kernel's heap size. Specifically, desiredvnodes scales > - * in proportion to the physical memory size until two fifths > - * of the kernel's heap size is consumed by vnodes and vm > - * objects. > + * Desiredvnodes is a function of the physical memory size and the > + * kernel's heap size. Generally speaking, it scales with the > + * physical memory size. The ratio of desiredvnodes to physical > pages > + * is one to four until desiredvnodes exceeds 96K. Thereafter, the > + * marginal ratio of desiredvnodes to physical pages is one to > sixteen. > + * However, desiredvnodes is limited by the kernel's heap size. The > + * memory required by desiredvnodes vnodes and vm objects may not > + * exceed one seventh of the kernel's heap size. > */ > - desiredvnodes = min(maxproc + cnt.v_page_count / 4, 2 * > vm_kmem_size / > - (5 * (sizeof(struct vm_object) + sizeof(struct vnode)))); > + physvnodes = maxproc + cnt.v_page_count / 16 + 3 * min(393216, > + cnt.v_page_count) / 16; > + virtvnodes = vm_kmem_size / (7 * (sizeof(struct vm_object) + > + sizeof(struct vnode))); > + printf("physvnodes = %d\nvirtvnodes = %d\n", physvnodes, > virtvnodes); > + desiredvnodes = min(physvnodes, virtvnodes); > if (desiredvnodes > MAXVNODES_MAX) { > if (bootverbose) > printf("Reducing kern.maxvnodes %d -> %d\n", > > -- John Baldwin