Date: Fri, 13 Aug 1999 23:13:21 +1000 From: Bruce Evans <bde@zeta.org.au> To: phk@critter.freebsd.dk, vernick@bell-labs.com Cc: freebsd-fs@FreeBSD.ORG Subject: Re: Help with understand file system performance Message-ID: <199908131313.XAA14647@godzilla.zeta.org.au>
next in thread | raw e-mail | index | archive | help
>>3. On a per process basis, performance increases when the number of
>>files per directory increases/number of subdirs decreases. Is this
>>because there is a better chance the directory information about the
>>file could be in memory?
>
>Yes. The minimum directory size is the fragsize of the filesystem,
>filling the directories better means better performance.
>
>>Our goal, of course, it to maximize performance. So any help in the
>>tuning of our system (i.e. reading lots of ~15KB files) would be
>>appreciated
Try increasing nbuf. I think effective caching of directories still
requires 1 buffer per directory.
>Try fiddling the newfs parameters. I see 17% speedup using:
>
> newfs -b 16384 -f 4096 -c 100
I see a 48% speedup using
linux
mkfs.ext2 -b 4096 $device $size_of_device_in_4k_units
:->. This is despite (or because of) ext2fs's block allocator being
broken (it essentially ignores cylinder groups).
The following times are for `tar zxpf linux-2.2.9.tar.gz', unmount
(to sync), and `tar cf /dev/null linux lost+found' on a new filesystem
on a Quantum KA disk on an overclocked Celeron-366 system:
ffs-4096-512:
41.82 real 3.24 user 3.34 sys
3.05 real 0.00 user 0.07 sys
16.53 real 0.08 user 1.34 sys
ffs-4096-1024:
35.89 real 3.35 user 3.70 sys
2.11 real 0.00 user 0.07 sys
15.53 real 0.13 user 1.39 sys
ffs-4096-2048:
29.32 real 3.24 user 4.36 sys
1.17 real 0.00 user 0.07 sys
12.20 real 0.14 user 1.42 sys
ffs-4096-4096:
28.85 real 3.34 user 4.51 sys
1.12 real 0.00 user 0.07 sys
11.24 real 0.10 user 1.59 sys
ffs-8192-1024:
33.39 real 3.26 user 5.44 sys
2.94 real 0.00 user 0.07 sys
13.40 real 0.12 user 1.18 sys
ffs-8192-2048:
28.08 real 3.29 user 3.01 sys
2.32 real 0.00 user 0.07 sys
11.21 real 0.06 user 1.26 sys
ffs-8192-4096:
25.05 real 3.27 user 2.99 sys
1.87 real 0.00 user 0.07 sys
9.17 real 0.09 user 1.21 sys
ffs-8192-8192:
23.27 real 3.27 user 2.82 sys
1.53 real 0.00 user 0.07 sys
8.94 real 0.10 user 1.23 sys
ffs-16384-2048:
28.22 real 3.43 user 4.78 sys
2.52 real 0.00 user 0.07 sys
12.01 real 0.10 user 1.55 sys
ffs-16384-4096:
24.32 real 3.41 user 3.51 sys
1.97 real 0.00 user 0.07 sys
10.56 real 0.11 user 1.37 sys
ffs-16384-8192:
23.63 real 3.33 user 3.35 sys
2.35 real 0.00 user 0.07 sys
8.66 real 0.09 user 1.15 sys
ffs-16384-16384:
85.41 real 3.33 user 3.28 sys
2.00 real 0.00 user 0.08 sys
9.51 real 0.10 user 1.17 sys
ext2fs-1024-1024:
36.33 real 3.33 user 3.67 sys
1.42 real 0.00 user 0.07 sys
14.49 real 0.10 user 2.28 sys
ext2fs-4096-4096:
20.81 real 3.38 user 3.54 sys
1.01 real 0.00 user 0.07 sys
6.96 real 0.12 user 1.57 sys
Note the anomalously slow times for ffs-16384-16384.
I analyzed why ffs was slow and ext2fs was fast for the `tar cf'
part a year or two ago. It was because ffs handles fragments poorly
and needs many more small (but not small enough to be in the drive's
cache) backwards seeks. Not using fragments reduced ext2fs's advantage
significantly but not completely. ffs-4K-4K is only slightly faster
than ffs-8K-1K now, presumably because drive caches are larger and
command overheads are relatively higher (the KA acts like a slow
SCSI drive in wanting a block size of at least 8K to keep up with
the disk).
This output was produced by the following program:
---
#!/bin/sh
for b in 4096 8192 16384
do
for f in $(($b / 8)) $(($b / 4)) $(($b / 2)) $b
do
echo ffs-$b-$f: >>/tmp/ztimes
newfs -b $b -f $f /dev/rwd2s2a
mount /dev/wd2s2a /d
cd /d
sync
time tar zxpf $loc/z/dist/*2.2.9.tar.gz 2>>/tmp/ztimes
cd /tmp
time umount /d 2>>/tmp/ztimes
mount /dev/wd2s2a /d
cd /d
sync
time tar cf /dev/null * 2>>/tmp/ztimes
cd /tmp
umount /d
done
done
for b in 1024 4096
do
for f in $b
do
echo ext2fs-$b-$f: >>/tmp/ztimes
# linux
mkfs.ext2 -b $b /dev/rwd2s2a $((4819437 / ($b / 512)))
# fsck.ext2 /dev/wd2s2a
mount -t ext2fs /dev/wd2s2a /d
cd /d
sync
time tar zxpf $loc/z/dist/*2.2.9.tar.gz 2>>/tmp/ztimes
cd /tmp
time umount /d 2>>/tmp/ztimes
mount -t ext2fs /dev/wd2s2a /d
cd /d
sync
time tar cf /dev/null * 2>>/tmp/ztimes
cd /tmp
umount /d
done
done
---
Bruce
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-fs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199908131313.XAA14647>