From owner-freebsd-fs@FreeBSD.ORG Fri Oct 14 13:52:23 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 18CD11065672 for ; Fri, 14 Oct 2011 13:52:23 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA11.westchester.pa.mail.comcast.net (qmta11.westchester.pa.mail.comcast.net [76.96.59.211]) by mx1.freebsd.org (Postfix) with ESMTP id BA04E8FC15 for ; Fri, 14 Oct 2011 13:52:22 +0000 (UTC) Received: from omta22.westchester.pa.mail.comcast.net ([76.96.62.73]) by QMTA11.westchester.pa.mail.comcast.net with comcast id kbvv1h0081ap0As5BdsNoV; Fri, 14 Oct 2011 13:52:23 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta22.westchester.pa.mail.comcast.net with comcast id kdsM1h00x1t3BNj3idsN1E; Fri, 14 Oct 2011 13:52:22 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 475C5102C1C; Fri, 14 Oct 2011 06:52:20 -0700 (PDT) Date: Fri, 14 Oct 2011 06:52:20 -0700 From: Jeremy Chadwick To: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <20111014135220.GA73637@icarus.home.lan> References: <4E97FEDD.7060205@quip.cz> <4E982C0E.2060900@quip.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E982C0E.2060900@quip.cz> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org, Ivan Voras Subject: Re: dirhash and dynamic memory allocation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2011 13:52:23 -0000 On Fri, Oct 14, 2011 at 02:33:18PM +0200, Miroslav Lachman wrote: > > > Ivan Voras wrote: > >On 14/10/2011 11:20, Miroslav Lachman wrote: > >>Hi all, > >> > >>I tried some tuning of dirhash on our servers and after googlig a bit, I > >>found an old GSoC project wiki page about Dynamic Memory Allocation for > >>Dirhash: http://wiki.freebsd.org/DirhashDynamicMemory > >>Is there any reason not to use it / not commit it to HEAD? > > > >AFAIK it's sort-of already present. In 8-stable and recent kernels you > >can give huge amounts of memory to dirhash via vfs.ufs.dirhash_maxmem > >(but except in really large edge cases I don't think you *need* more > >than 32 MB), and the kernel will scale-down or free the memory if not > >needed. > > Is this change documented somewhere? Maybe it could be noticed on > DirhashDynamicMemory wiki page. Otherwise it seems as abandoned GSoC > project. There is no real form of "documentation" for this kind of change, but I do remember it being discussed on the mailing list at some point (an announcement or something? I forget -- man it was a while ago). There's probably a commit message. I can dig it up if need be, but commit messages are often "Add Snake Wanglesnort support, I like samgyeopsal" and not much else. I'm not even sure what man page would be relevant to document such pieces. How about this? $ sysctl -a | grep dirhash vfs.ufs.dirhash_reclaimage: 5 vfs.ufs.dirhash_lowmemcount: 0 vfs.ufs.dirhash_docheck: 0 vfs.ufs.dirhash_mem: 1075212 vfs.ufs.dirhash_maxmem: 16777216 vfs.ufs.dirhash_minsize: 2560 $ for i in `sysctl -a | grep dirhash | cut -d: -f1`; do sysctl -d $i; done vfs.ufs.dirhash_reclaimage: max time in seconds of hash inactivity before deletion in low VM events vfs.ufs.dirhash_lowmemcount: number of times low memory hook called vfs.ufs.dirhash_docheck: enable extra sanity tests vfs.ufs.dirhash_mem: current dirhash memory usage vfs.ufs.dirhash_maxmem: maximum allowed dirhash memory usage vfs.ufs.dirhash_minsize: minimum directory size in bytes for which to use hashed lookup No idea what docheck does. But it looks like reclaimage (that's "reclaim age") is a pretty good indicator of what Ivan is describing. So basically, if you're worried about how much memory dirhash is taking up, you could log/graph vfs.ufs.dirhash_mem (unit = bytes). > >In effect, vfs.ufs.dirhash_maxmem is the upper limit - the kernel will > >use less and will free the allocated memory in low memory situations > >(which I've tried and it works). > > > >>And second question - is there any negative impact with higher > >>vfs.ufs.dirhash_maxmem? It stil defaults to 2MB (on FreeBSD 8.2) after > > > >Not that I know of. > > > >>10 years, but I think we all are using bigger FS in these days with lot > >>of files and directories and 2MB is not enough. > > > >AFAIK I've changed it to autotune so it's configured to approximately 4 > >MB on a 4 GB machine (and scales up) in 9. > > I don't have 9 installed to test it. Only 8-STABLE. Hope I will test > it soon. The above data I showed was taken from a RELENG_8 box. The dirhash change in question I believe was either implemented in very early 8.x, or very late 7.x. Again: I can dig up the commit if requested. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |