Date: Sun, 16 Nov 2003 15:51:24 -0800 (PST) From: Don Lewis <truckman@FreeBSD.org> To: kmarx@vicor.com Cc: mckusick@beastie.mckusick.com Subject: Re: 4.8 ffs_dirpref problem Message-ID: <200311162351.hAGNpOeF086319@gw.catspoiler.org>
next in thread | raw e-mail | index | archive | help
On 31 Oct, To: kmarx@vicor.com wrote: > On 31 Oct, Ken Marx wrote: > You may get better results if you change the minbfree calculation from > (avgbfree - avgbfree/4) to just (avgbfree). > > I'm somewhat tempted to change the calculation to: > min(avgbfree, max(1, (avgbfree - avgbfree/4), (dirsize/fs->fs_bsize))) > where the last term works out to 4500 with your tunefs parameters. I tried a variation of this on my -CURRENT box and it benchmarked consistently worse. I've got a "spare' 10 GB partition which first copied my /usr/ports/packages to, and then filled by repeatedly tarring my /usr/ports tree over to it. The partition was 100% full, including the reserve space, after four iterations. With minbfree set to max((avgbfree - avgbfree/4), 1) here are two iterations (the fifth line of timing data is for the 'rm -rf' command): 1310.47 real 5.48 user 141.90 sys 1336.78 real 5.62 user 152.27 sys 1368.84 real 6.02 user 151.75 sys 1359.70 real 5.55 user 154.01 sys 423.44 real 2.25 user 107.26 sys 1300.56 real 5.65 user 148.82 sys 1372.20 real 5.79 user 152.25 sys 1359.01 real 6.03 user 152.63 sys 1380.90 real 5.31 user 153.71 sys 437.22 real 2.20 user 105.61 sys With minbfree set to max(min(max(avgbfree - avgbfree / 4, dirsize / fs->fs_bsize), avgbfree), 1) I get the following: 1314.61 real 5.66 user 175.43 sys 1350.40 real 6.12 user 179.15 sys 1386.86 real 6.32 user 179.12 sys 1418.60 real 5.74 user 181.64 sys 508.67 real 2.67 user 119.66 sys 1361.19 real 5.97 user 176.94 sys 1327.63 real 5.72 user 179.60 sys 1376.16 real 6.33 user 179.72 sys 1356.47 real 6.07 user 180.24 sys 462.67 real 2.30 user 119.18 sys I'm using the newfs defaults, but dirsize is recalculated as the filesystem fills if the appropriate value is larger than what is calculated from the parameters set by newfs. I suspect the problem is the large bimodal distribution in file size in my benchmark, with zillions of little files, but also a number of large package files and source distfiles. The large files muck up the dirsize calculation because they are actually distributed across multiple cylinder groups and only the first maxbpg blocks are allocated in the original cylinder group. This would be easy to account for in the avgfilesize * avgfpdir formula, but I don't know how to handle this in the curdirsize formula (other than the degenerate case where most files are larger than maxbpg). Since I probably won't have time to get anything different tested before the -CURRENT code freeze, do you have any objections if I just MFC the code that I previously committed to -CURRENT? It certainly seems to perform better than the original code which is still in 4-STABLE.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200311162351.hAGNpOeF086319>