Date: Fri, 31 Oct 2003 08:26:47 -0800 From: Ken Marx <kmarx@sploot.vicor-nb.com> To: Don Lewis <truckman@FreeBSD.org> Cc: mckusick@beastie.mckusick.com Subject: Re: 4.8 ffs_dirpref problem Message-ID: <20031031162647.GB30803@sploot.vicor-nb.com> In-Reply-To: <200310310300.h9V2xueF033759@gw.catspoiler.org> References: <3FA1745A.2090205@vicor.com> <200310310300.h9V2xueF033759@gw.catspoiler.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Kirk McKusick wrote: > I know it takes a lot of time, but I would like to hear of the results > when you do the initial loading of the filesystem using Don's code as > that may well effect the set of choices that it has. Ok. I've done this using 48k avgfilesize, and 1500 filesperdir. I left our hashtable patch in. I can give details, but Don's code seems to average a healthy 64-5sec/1.5gb untar. I.e., basically equivalent to the 4.4 code. But after 90% or so starts consuming more time - up to 90-130 seconds (system time). This increase is not always monotonic. Timings in the 60sec range to occur. (I double-checked this, re-running starting at 97%.) The 4.4 dirpref code seems a bit better in this regime, staying mostly in the 60-70 sec range. I'm now starting from a newfs'd raid with the default settings, and running Don's patch (with hashtable patch). Should be done in 10hrs or so. This matters to us, because we'd like to avoid having to newfs all our production raids. On Thu, Oct 30, 2003 at 06:59:56PM -0800, Don Lewis wrote: > On 30 Oct, Ken Marx wrote: > > > > > > Don Lewis wrote: > >> On 30 Oct, Ken Marx wrote: > > > Prehaps the dirpref patch lowers the frequency of having to search so much, > > and hence exercises the hashtable less. > > I'm pretty sure that is the situation, but I wasn't sure if there was > still enough hash table usage to make balancing it worthwhile. It > sounds like there isn't. In this case at least. But is it still possible that some situation might arise in which the quadratic search fails enough to warrant having the more efficient hash for the multiple linear searches? I guess I'm asking if folks are inclined to go for some hashtable patch (in addition to Don's dirpref patch). I should probably compare hashtable vs. no-hashtable kernels, with NO dirpref patch. I can do (not until tonight) if this would help in deciding. Let me know. > It looks like there is still about a 10% increase in system time for my > modifications to dirpref versus reverting to the old algorithm. It > would be nice to know where the extra time was being spent, but that > would require probably require some kernel profiling. Right. In the high-capacity regime at least. We can profile kernels here, but I'm running low on time I can easily devote to this. Julian is due back in a few weeks though. Let me know though, and I'll do whatever I can. > It might also be interesting to play with the cylinder group selection > criteria. Should minbfree be 75% of avgbfree, 100% of avgbfree, or 125% > of avgbfree? Probably not anything over 100%, since the algorithm would > fail if all the cylinder groups were evenly filled ... > (I'm way out of my depth here...) Thanks again for the continued help with all this, k -- Ken Marx, kmarx@vicor-nb.com Analyze progress on the family jewels!! - http://www.bigshed.com/cgi-bin/speak.cgi
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20031031162647.GB30803>