Date: Wed, 28 Aug 2013 18:40:15 +0200 From: Ivan Voras <ivoras@freebsd.org> To: gljennjohn@googlemail.com Cc: freebsd-fs <freebsd-fs@freebsd.org>, freebsd-hackers <freebsd-hackers@freebsd.org> Subject: Re: Call fo comments - raising vfs.ufs.dirhash_reclaimage? Message-ID: <CAF-QHFU80YC3W-k%2BTKM=y3JiVYi=1fp5CJjbCCk1y0VKXzcRQg@mail.gmail.com> In-Reply-To: <20130828181228.0d3618dd@ernst.home> References: <kvkvi7$iv7$1@ger.gmane.org> <20130828181228.0d3618dd@ernst.home>
next in thread | previous in thread | raw e-mail | index | archive | help
On 28 August 2013 18:12, Gary Jennejohn <gljennjohn@googlemail.com> wrote: > So, if I understand this correctly, a normal desktop user won't > notice any real change, except that buildworld might get faster, > and big servers will benefit? Basically, yes, but read on... > But could this negatively impact small, embedded systems, which > usually have only small memory footprints? Although I suppose > one could argue that they usually don't have large numbers of > files cached in memory at any given time. Unless I'm wrong, the only pathological case coming from this change would be the following sequence of events: 1) Memory is scarce [*] 2) There's a sudden surge of requests for a huge number of different directories 3) There's an urgent lowmem event which is observed by dirhash, which attempts to free memory but is prevented in doing so for the next 60 seconds because all entries are young (the idea behind dirhash being that if a directory is accessed, it will probably soon be accessed again - think "ls" then "fopen", so we won't evict him until reclaimage seconds) 4) the kernel runs out of memory, game over. Note that this sequence of events could still happen right now, only over a span of 5 seconds, not 60 seconds. Note also that all of this has nothing to do with regular file cache, dirhash is a very specific corner-case of UFS. [*] Keep in mind that dirhash cache even on large and busy systems is usually ~~15-25 MB; on 16 GB machines the auto-tuning code caps it at 25 MB. As an illustration on how tiny dirhash is: a "du -c" on /usr/ports increases dirhash_mem on my desktop from 103945 to 501507 bytes. One of the issues raised by davide is that the benefits from this are also miniscule and hard to prove. A simple buildworld is not a big enough load. I've seen on my own skin how increasing reclaimage helped, but that was under such specific circumstances that I'm still trying to figure out how to create a self-sustained benchmark (basically - how to provoke lowmem events?). Basically, this change will have no effect for 99.9% of users, but could save that 0.1% from going crazy.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAF-QHFU80YC3W-k%2BTKM=y3JiVYi=1fp5CJjbCCk1y0VKXzcRQg>