Date: Fri, 30 Jul 2010 16:14:40 -0400 From: John Baldwin <jhb@freebsd.org> To: Alan Cox <alc@cs.rice.edu> Cc: alc@freebsd.org, Matthew Fleming <mdf356@gmail.com>, Andriy Gapon <avg@freebsd.org>, freebsd-arch@freebsd.org Subject: Re: amd64: change VM_KMEM_SIZE_SCALE to 1? Message-ID: <201007301614.40768.jhb@freebsd.org> In-Reply-To: <4C531ED7.9010601@cs.rice.edu> References: <4C4DB2B8.9080404@freebsd.org> <201007270935.52082.jhb@freebsd.org> <4C531ED7.9010601@cs.rice.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Friday, July 30, 2010 2:49:59 pm Alan Cox wrote:
> John Baldwin wrote:
> > I have a strawman of that (relative to 7). It simply adjusts the hardcoded
> > maximum to instead be a function of the amount of physical memory.
> >
> >
>
> Unless I'm misreading this patch, it would allow "desiredvnodes" to grow
> (slowly) on i386/PAE starting at 5GB of RAM until we reach the (too
> high) "virt" limit of about 329,000. Yes? For example, an 8GB i386/PAE
> machine would have 60% more vnodes than was allowed by MAXVNODE_MAX, and
> it would not stop there. I think that we should be concerned about
> that, because MAXVNODE_MAX came about because the "virt" limit wasn't
> working.
Agreed.
> As the numbers above show, we could more than halve the growth rate for
> "virt" and it would have no effect on either amd64 or i386 machines with
> up to 1.5GB of RAM. They would have just as many vnodes. Then, with
> that slower growth rate, we could simply eliminate MAXVNODES_MAX (or at
> least configure it to some absurdly large value), thereby relieving the
> fixed cap on amd64, where it isn't needed.
>
> With that in mind, the following patch slows the growth of "virt" from
> 2/5 of vm_kmem_size to 1/7. This has no effect on amd64. However, on
> i386. it allows desiredvnodes to grow slowly for machines with 1.5GB to
> about 2.5GB of RAM, ultimately exceeding the old desiredvnodes cap by
> about 17%. Once we exceed the old cap, we increase desiredvnodes at a
> marginal rate that is almost the same as your patch, about 1% of
> physical memory. It's just computed differently.
>
> Using 1/8 instead of 1/7, amd64 machines with less than about 1.5GB lose
> about 7% of their vnodes, but they catch up and pass the old limit by
> 1.625GB. Perhaps, more importantly, i386 machines only exceed the old
> cap by 3%.
>
> Thoughts?
I think this is much better. My strawman was rather hackish in that it was
layering a hack on top of the existing calculations. I prefer your approach.
I do not think penalizing amd64 machines with less than 1.5GB is a big worry
as most x86 machines with a small amount of memory are probably running as
i386 anyway. Given that, I would probably lean towards 1/8 instead of 1/7,
but I would be happy with either one.
> Index: kern/vfs_subr.c
> ===================================================================
> --- kern/vfs_subr.c (revision 210504)
> +++ kern/vfs_subr.c (working copy)
> @@ -284,21 +284,29 @@ SYSCTL_INT(_debug, OID_AUTO, vnlru_nowhere, CTLFLA
> * Initialize the vnode management data structures.
> */
> #ifndef MAXVNODES_MAX
> -#define MAXVNODES_MAX 100000
> +#define MAXVNODES_MAX 8388608 /* Reevaluate when physmem
> exceeds 512GB. */
> #endif
How is this value computed? I would prefer something like:
'512 * 1024 * 1024 * 1024 / (sizeof(struct vnode) + sizeof(struct vm_object) / N'
if that is how it is computed. A brief note about the magic number of 393216
would also be nice to have (and if it could be a constant with a similar
formula value that would be nice, too.).
> static void
> vntblinit(void *dummy __unused)
> {
> + int physvnodes, virtvnodes;
>
> /*
> - * Desiredvnodes is a function of the physical memory size and
> - * the kernel's heap size. Specifically, desiredvnodes scales
> - * in proportion to the physical memory size until two fifths
> - * of the kernel's heap size is consumed by vnodes and vm
> - * objects.
> + * Desiredvnodes is a function of the physical memory size and the
> + * kernel's heap size. Generally speaking, it scales with the
> + * physical memory size. The ratio of desiredvnodes to physical
> pages
> + * is one to four until desiredvnodes exceeds 96K. Thereafter, the
> + * marginal ratio of desiredvnodes to physical pages is one to
> sixteen.
> + * However, desiredvnodes is limited by the kernel's heap size. The
> + * memory required by desiredvnodes vnodes and vm objects may not
> + * exceed one seventh of the kernel's heap size.
> */
> - desiredvnodes = min(maxproc + cnt.v_page_count / 4, 2 *
> vm_kmem_size /
> - (5 * (sizeof(struct vm_object) + sizeof(struct vnode))));
> + physvnodes = maxproc + cnt.v_page_count / 16 + 3 * min(393216,
> + cnt.v_page_count) / 16;
> + virtvnodes = vm_kmem_size / (7 * (sizeof(struct vm_object) +
> + sizeof(struct vnode)));
> + printf("physvnodes = %d\nvirtvnodes = %d\n", physvnodes,
> virtvnodes);
> + desiredvnodes = min(physvnodes, virtvnodes);
> if (desiredvnodes > MAXVNODES_MAX) {
> if (bootverbose)
> printf("Reducing kern.maxvnodes %d -> %d\n",
>
>
--
John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201007301614.40768.jhb>