Date: Wed, 28 Aug 2013 20:44:20 -0700 (PDT) From: Robert_Burmeister <robert.burmeister@utoledo.edu> To: freebsd-stable@freebsd.org Subject: RE: Suggest changing dirhash defaults for FreeBSD 9.2. Message-ID: <1377747860047-5839828.post@n5.nabble.com> In-Reply-To: <F4E60E862AF142AFBA86521E79C0BCAA@white> References: <521C9E85.4060801@UToledo.edu> <CAE-mSOLCYRM0LRLRgmaEZN1u5ozttJZC3kWtw3Zarqik1N29zw@mail.gmail.com> <521D7552.5080008@UToledo.edu> <kvkh1j$7fj$1@ger.gmane.org> <F4E60E862AF142AFBA86521E79C0BCAA@white>
next in thread | previous in thread | raw e-mail | index | archive | help
Dewayne Geraghty-4 wrote >> I'll bump vfs.ufs.dirhash_reclaimage to 60, it's worth it. > > From the analysis perforned in 2009, and referenced earlier by Robert, > this > https://wiki.freebsd.org/DirhashDynamicMemory and other material at this > site, > indicates that the reclaimage interval is workload dependent and that 5 to > 8 seconds seems, on average, to be adequate. > > We perform quite a detailed analysis before changing our sysctls for > customers' servers that range from 32bit machines with between > 1G and 4G of memory, and 64bit with 30G of memory, incidentally all are > UFS2. Is the discussion, rather than (synthetic) workload > performance, sufficient to warrant changing the default settings by a > factor of 12? > > Regards, Dewayne. I recompile kernel, world and ~1200 ports for every FreeBSD i386 release candidate to detect regressions. Most ports use less than the 2 Gigs of RAM I have to compile, but FireFox, LibreOffice, Sage and others will dig deep into virtual memory. I have observed how the UFS dirhash cache performs during these tasks by periodically running sysctl -a | grep dirhash and I believe it offers noticeable performance improvements for many common tasks, at no risk of loss of performance in other areas, or risk of a kernel panic. I first noticed the UFS dirhash cache because it emulates an old feature of Novel file servers. Most OSs require one disk IO for file metadata and another to access the file. Novel used to store its file metadata in RAM to provide single disk IO file access, at the cost of long boot times. Likewise, the UFS dirhash cache avoids a disk IO for file access when the metadata is in the cache, but only when there is excess memory available for the cache. Since it is an odd on, it was not necessary to change the file system for this increase in performance. This is beneficial for any process requiring disk IO, but especially compiling, mail servers, etc. I have since migrated to SSDs and found the cache to leverage the low latency of SSDs even more. I have read the source code and observed the cache working with high, low and borderline available RAM. Hence determining a cache of > 13 megs is needed to avoid thrashing when rebuilding the ports tree, for instance. I have observed that the UFS dirhash cache gracefully gives back its allocated memory to the kernel before swapping begins, abet with a noticeable drop in performance from dirhash_mem being reduced to insignificance. When kernel memory is again available, the cache will hold its entries indefinitely. Case: if 90% of the cache contains entries from five minutes ago, why should the entries from the last minute that are being used by an active job be lost when dirhash_reclaimage = 60 would clear 90% of the cache on the first vm_lowmem event? In any case, clearing all cache entries older than 5 seconds on every vm_lowmem event keeps the cache from being useful in precisely the circumstances when it would be of most benefit. I suggested vfs.ufs.dirhash_reclaimage = 60 because most bursts of IO tend to fit within >5 - <60 seconds, and I can't think of a case for <=5 seconds. I am interested in hearing other proposals, but I think it shouldn't be difficult to agree that vfs.ufs.dirhash_reclaimage = 5 is too short to really be useful for either servers or desktop users. (Note that 5 seconds was chosen back when the feature was still experimental in 2001.) -- View this message in context: http://freebsd.1045724.n5.nabble.com/Suggest-changing-dirhash-defaults-for-FreeBSD-9-2-tp5839351p5839828.html Sent from the freebsd-stable mailing list archive at Nabble.com.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1377747860047-5839828.post>