Date: Sun, 16 Nov 2003 19:31:47 -0800 (PST) From: Don Lewis <truckman@FreeBSD.org> To: kmarx@vicor.com Cc: mckusick@beastie.mckusick.com Subject: Re: 4.8 ffs_dirpref problem Message-ID: <200311170331.hAH3VleF086693@gw.catspoiler.org> In-Reply-To: <200311162351.hAGNpOeF086319@gw.catspoiler.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 16 Nov, Don Lewis wrote: >> I'm somewhat tempted to change the calculation to: >> min(avgbfree, max(1, (avgbfree - avgbfree/4), (dirsize/fs->fs_bsize))) >> where the last term works out to 4500 with your tunefs parameters. > > I tried a variation of this on my -CURRENT box and it benchmarked > consistently worse. I've got a "spare' 10 GB partition which first > copied my /usr/ports/packages to, and then filled by repeatedly tarring > my /usr/ports tree over to it. The partition was 100% full, including > the reserve space, after four iterations. I just looked again, and it is more than 100% full, but only slightly into the reserve space. > With minbfree set to max((avgbfree - avgbfree/4), 1) here are two > iterations (the fifth line of timing data is for the 'rm -rf' command): > > 1310.47 real 5.48 user 141.90 sys > 1336.78 real 5.62 user 152.27 sys > 1368.84 real 6.02 user 151.75 sys > 1359.70 real 5.55 user 154.01 sys > 423.44 real 2.25 user 107.26 sys > > 1300.56 real 5.65 user 148.82 sys > 1372.20 real 5.79 user 152.25 sys > 1359.01 real 6.03 user 152.63 sys > 1380.90 real 5.31 user 153.71 sys > 437.22 real 2.20 user 105.61 sys > > With minbfree set to > max(min(max(avgbfree - avgbfree / 4, dirsize / fs->fs_bsize), > avgbfree), 1) > I get the following: > > 1314.61 real 5.66 user 175.43 sys > 1350.40 real 6.12 user 179.15 sys > 1386.86 real 6.32 user 179.12 sys > 1418.60 real 5.74 user 181.64 sys > 508.67 real 2.67 user 119.66 sys > > 1361.19 real 5.97 user 176.94 sys > 1327.63 real 5.72 user 179.60 sys > 1376.16 real 6.33 user 179.72 sys > 1356.47 real 6.07 user 180.24 sys > 462.67 real 2.30 user 119.18 sys > > I'm using the newfs defaults, but dirsize is recalculated as the > filesystem fills if the appropriate value is larger than what is > calculated from the parameters set by newfs. I filled up the file system again with the minbree = max((avgbfree - avgbfree/4), 1) version of the code. Based on the output of df and dumpfs, I calculate: avgfilesize = 18K curdirsize = 83K avgbfree = 864 avgifree = 14631 What suprises me is the poor distribution of free space across the cylinder groups in the file system. I now suspect the culprit is minifree. The current code calculates minifree as 75% of avgifree, or about 10973. There are some cylinder groups that are less than half full (capacity is 11761 blocks/group) in this filesystem, but their free inode counts are near the 10K minifree limit. It looks like the free inode count should be de-emphasized if the filesystem will run out of blocks before it runs out of inodes, and vice-versa if inodes are likely to be exhausted first. I now suspect that the other version of the minbfree code was more likely to bail out because it could not find any cylinder groups that met both selection criteria and used the fallback code, which probably selected the cylinder groups that were already full but had a large number of free inodes. Something to ponder ... #df -k /mnt Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/da0s2a 10890186 10057300 -38328 100% /mnt #df -i /mnt Filesystem 1K-blocks Used Avail Capacity iused ifree %iused Mounted on /dev/da0s2a 10890186 10057300 -38328 100% 535236 877882 38% /mnt #/usr/obj/usr/src/sbin/dumpfs/dumpfs /dev/da0s2a | head -20 magic 19540119 (UFS2) time Sun Nov 16 18:34:10 2003 superblock location 65536 id [ 3ec6b0f1 ec1a8944 ] ncg 60 size 5622734 blocks 5445093 bsize 16384 shift 14 mask 0xffffc000 fsize 2048 shift 11 mask 0xfffff800 frag 8 shift 3 fsbtodb 2 minfree 8% optim time symlinklen 120 maxbsize 16384 maxbpg 2048 maxcontig 8 contigsumsize 8 nbfree 51887 ndir 120875 nifree 877882 nffree 1347 bpg 11761 fpg 94088 ipg 23552 nindir 2048 inopb 64 maxfilesize 140806241583103 sbsize 2048 cgsize 16384 csaddr 3000 cssize 2048 sblkno 40 cblkno 48 iblkno 56 dblkno 3000 cgrotor 32 fmod 0 ronly 0 clean 0 avgfilesize 16384 avgfpdir 64 flags soft-updates fsmnt /mnt volname swuid 0 #/usr/obj/usr/src/sbin/dumpfs/dumpfs /dev/da0s2a | grep bfree nbfree 51887 ndir 120875 nifree 877882 nffree 1347 cs[].cs_(nbfree,ndir,nifree,nffree): nbfree 3953 ndir 2859 nifree 10786 nffree 3 nbfree 0 ndir 2114 nifree 14323 nffree 12 nbfree 0 ndir 2833 nifree 10755 nffree 31 nbfree 0 ndir 2866 nifree 10984 nffree 19 nbfree 0 ndir 2810 nifree 10983 nffree 15 nbfree 0 ndir 2122 nifree 14324 nffree 62 nbfree 0 ndir 2853 nifree 10887 nffree 39 nbfree 2 ndir 2872 nifree 10990 nffree 62 nbfree 0 ndir 2851 nifree 10985 nffree 18 nbfree 0 ndir 2787 nifree 11019 nffree 17 nbfree 0 ndir 2898 nifree 10982 nffree 15 nbfree 4 ndir 2769 nifree 10982 nffree 6 nbfree 5 ndir 2889 nifree 10984 nffree 18 nbfree 44 ndir 8 nifree 23544 nffree 21 nbfree 0 ndir 2525 nifree 12424 nffree 22 nbfree 0 ndir 2416 nifree 12654 nffree 5 nbfree 3 ndir 2510 nifree 12652 nffree 16 nbfree 0 ndir 2486 nifree 12655 nffree 56 nbfree 2 ndir 2475 nifree 12654 nffree 1 nbfree 21 ndir 2694 nifree 11847 nffree 44 nbfree 14 ndir 2230 nifree 13649 nffree 83 nbfree 196 ndir 4 nifree 23432 nffree 57 nbfree 16 ndir 587 nifree 20844 nffree 9 nbfree 2 ndir 0 nifree 23552 nffree 3 nbfree 5 ndir 2351 nifree 13182 nffree 15 nbfree 0 ndir 2801 nifree 11026 nffree 8 nbfree 0 ndir 2808 nifree 10985 nffree 12 nbfree 0 ndir 1738 nifree 15765 nffree 37 nbfree 0 ndir 1687 nifree 15991 nffree 24 nbfree 0 ndir 1668 nifree 15973 nffree 17 nbfree 0 ndir 1755 nifree 15992 nffree 47 nbfree 597 ndir 2901 nifree 10985 nffree 7 nbfree 1660 ndir 2945 nifree 10963 nffree 2 nbfree 0 ndir 2798 nifree 10984 nffree 10 nbfree 116 ndir 1736 nifree 15993 nffree 12 nbfree 0 ndir 1755 nifree 15978 nffree 8 nbfree 8344 ndir 2860 nifree 10972 nffree 1 nbfree 780 ndir 2770 nifree 10953 nffree 28 nbfree 0 ndir 2 nifree 23321 nffree 51 nbfree 0 ndir 0 nifree 23552 nffree 18 nbfree 2821 ndir 2791 nifree 10970 nffree 0 nbfree 91 ndir 0 nifree 23552 nffree 46 nbfree 1838 ndir 999 nifree 19117 nffree 2 nbfree 6455 ndir 2807 nifree 11028 nffree 4 nbfree 6232 ndir 2871 nifree 11027 nffree 0 nbfree 5 ndir 0 nifree 23552 nffree 17 nbfree 4 ndir 982 nifree 19170 nffree 58 nbfree 40 ndir 5 nifree 23496 nffree 78 nbfree 3169 ndir 484 nifree 21386 nffree 0 nbfree 36 ndir 3 nifree 23509 nffree 47 nbfree 3136 ndir 2852 nifree 10976 nffree 6 nbfree 4860 ndir 2876 nifree 10981 nffree 2 nbfree 34 ndir 2082 nifree 14219 nffree 57 nbfree 0 ndir 2087 nifree 14362 nffree 14 nbfree 1953 ndir 2867 nifree 10982 nffree 11 nbfree 0 ndir 2115 nifree 14365 nffree 14 nbfree 223 ndir 1997 nifree 14292 nffree 17 nbfree 5226 ndir 2876 nifree 10938 nffree 7 nbfree 0 ndir 2100 nifree 14094 nffree 32 nbfree 0 ndir 2048 nifree 14360 nffree 4
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200311170331.hAH3VleF086693>