Date: Fri, 21 Oct 2011 20:04:03 -0700 From: Jeremy Chadwick <freebsd@jdc.parodius.com> To: Doug Barton <dougb@FreeBSD.org> Cc: freebsd-fs@freebsd.org, Ivan Voras <ivoras@freebsd.org> Subject: Re: dirhash and dynamic memory allocation Message-ID: <20111022030403.GA176@icarus.home.lan> In-Reply-To: <4EA2277B.5080306@FreeBSD.org> References: <4E97FEDD.7060205@quip.cz> <j7938v$66s$1@dough.gmane.org> <4EA19203.5050503@quip.cz> <4EA2277B.5080306@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Oct 21, 2011 at 07:16:27PM -0700, Doug Barton wrote: > On 10/21/2011 08:38, Miroslav Lachman wrote: > > Hi, I am back on this topic... > > > > Ivan Voras wrote: > >> On 14/10/2011 11:20, Miroslav Lachman wrote: > >>> Hi all, > >>> > >>> I tried some tuning of dirhash on our servers and after googlig a bit, I > >>> found an old GSoC project wiki page about Dynamic Memory Allocation for > >>> Dirhash: http://wiki.freebsd.org/DirhashDynamicMemory > >>> Is there any reason not to use it / not commit it to HEAD? > >> > >> AFAIK it's sort-of already present. In 8-stable and recent kernels you > >> can give huge amounts of memory to dirhash via vfs.ufs.dirhash_maxmem > >> (but except in really large edge cases I don't think you *need* more > >> than 32 MB), and the kernel will scale-down or free the memory if not > >> needed. > >> > >> In effect, vfs.ufs.dirhash_maxmem is the upper limit - the kernel will > >> use less and will free the allocated memory in low memory situations > >> (which I've tried and it works). > > > > So the current behavior is that on 7.3+ and 8.x we have smaller average > > dirhash buffer (by default) than it was initialy 10 years ago. Because > > it starts as 2MB fixed size and now we have 2MB max, which is lowered in > > low mem situations... and sometimes it is set to 0MB! > > > > I caught this 2 days ago: > > > > root@rip ~/# sysctl vfs.ufs > > vfs.ufs.dirhash_reclaimage: 5 > > vfs.ufs.dirhash_lowmemcount: 36953 > > vfs.ufs.dirhash_docheck: 0 > > vfs.ufs.dirhash_mem: 0 > > vfs.ufs.dirhash_maxmem: 8388608 > > vfs.ufs.dirhash_minsize: 2560 > > > > I set maxmem to 8MB in sysctl.conf to increase performance and > > dirhash_mem 0 is really bad surprise! > > > > I am worrying about bad performance in situation where dirhash is > > emptied in situations, where server is already running at maximum > > performance (there is some memory hungry process and system can start > > swapping to disk + dirhash is efectively disabled) > > > > I found a PR kern/145246 > > http://www.freebsd.org/cgi/query-pr.cgi?pr=145246 > > > > Is it possible to add some dirhash_minmem limit to not clear all the > > dirhash memory? > > So I can set dirhash_minmem=2MB dirhash_maxmem=16MB and then dirhash_mem > > will be allways between these two limits? > > Isn't that what vfs.ufs.dirhash_minsize is for? I think given that there > is a lot more memory in modern systems setting that higher by default is > probably a good idea. Or maybe I'm misunderstanding what that knob does? I believe the function of that sysctl is different. It's not the "minimum amount of dirhash memory to retain", it's: $ sysctl -d vfs.ufs.dirhash_minsize vfs.ufs.dirhash_minsize: minimum directory size in bytes for which to use hashed lookup The sysctl should really be named "dirhash_mindirsize". -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111022030403.GA176>