Date: Mon, 24 Sep 2001 01:43:55 -0700 (PDT) From: Matt Dillon <dillon@earth.backplane.com> To: Poul-Henning Kamp <phk@critter.freebsd.dk>, David Greenman <dg@root.com>, Seigo Tanimura <tanimura@r.dl.itc.u-tokyo.ac.jp>, bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Second set of stable buildworld results w/ vmiodirenable & nameileafonly combos Message-ID: <200109240843.f8O8htO91785@earth.backplane.com>
next in thread | raw e-mail | index | archive | help
Ok, here is the second set of results. I didn't run all the tests
because nothing I did appeared to really have much of an effect. In
this set of tests I set MAXMEM to 128M. As you can see the buildworld
took longer verses 512M (no surprise), and vmiodirenable still helped
verses otherwise. If one takes into consideration the standard
deviation, the directory vnode reclamation parameters made absolutely
no difference in the tests.
The primary differentiator in all the tests is 'block input ops'. With
vmiodirenable turned on it sits at around 51000. With it off it sits
at around 56000. In the 512M tests the pass-1 numbers were 26000 with
vmiodirenable turned on and 33000 with it off. Pass-2 numbers were
9000 with it on and 18000 with it off. The directory leaf reuse
parameters had almost no effect on either the 128M or 512M numbers.
I'm not sure why test2 wound up doing a better job then test1 in the
128M tests with vmiodirenable disabled. Both machines are configured
identically with only some extra junk on test1's /usr from prior tests.
In anycase, the differences point to a rather significant error spread
in regards to possible outcomes, at least with vmiodirenable=0.
My conclusion from all of this is:
* vmiodirenable should be turned on by default.
* We should rip out the cache_purgeleafdirs() code entirely and use my
simpler version to fix the vnode-growth problem.
* We can probably also rip out my cache_leaf_test() .. we do not need
to add any sophistication to reuse only directory vnodes without
subdirectories in the cache. If it had been a problem we would have
seen it.
I can leave the sysctl's in place on the commit to allow further testing,
and I can leave it conditional on vmiodirenable. I'll set the default
vmiodirenable to 1 (which will also enable directory vnode reuse) and
the default nameileafonly to 0 (i.e. to use the less sophisticated check).
In a few weeks I will rip-out nameileafonly and cache_leaf_test().
-Matt
WIDE TERMINAL WINDOW REQUIRED! ---------------------------------------------------------------------------------------------------
TEST SUITE 2 (128M ram)
buildworld of -stable. DELL2550 (Duel PIII-1.2GHz / 128M ram (via MAXMEM) / SCSI)
23 September 2001 SMP kernel, softupdates-enabled, dirpref'd local /usr/src (no nfs),
make -j 12 buildworld UFS_DIRHASH. 2 identical machines tested in parallel (test1, test2)
/usr/bin/time -l timings note: atime updates left enabled in all tests
REUSE LEAF DIR VNODES: directory vnodes with no subdirectories in the namei cache can be reused
REUSE ALL DIR VNODES: directory vnodes can be reused (namei cache ignored)
DO NOT REUSE DIR...: (Poul's original 1995 algo) directory vnode can only be reused if no subdirectories or files in the
namei cache
I stopped bothering with pass-2 after it became evident that the numbers
were not changing significantly.
VMIODIRENABLE ENABLED [------------ A ------------] [------------ B ------------] [------------ C ------------]
[BEST CASE ] [BEST CASE ] [BEST CASE ]
machine test1 test2 test1 test2 test1 test2 test1 test2 test1 test2 test1 test2
pass (2) R 1 1 2 2 R 1 1 2 2 R 1 1 2 2
vfs.vmiodirenable E 1 1 1 1 E 1 1 1 1 E 1 1 1 1
vfs.nameileafonly B 1 1 1 1 B 0 0 0 0 B -1 -1 -1 -1
O O O
O REUSE LEAF DIR VNODES O REUSE ALL DIR VNODES O DO NOT REUSE DIR VNODES W/ACTIVE NAMEI
T T T
26:49 26:30 26:41 26:24
real 1609 1590 1601 1584
user 1361 1354 1361 1356
sys 617 615 617 614
max resident 16264 16256 16260 16264
avg shared mem 1030 1030 1030 1030
avg unshared data 1004 1005 1006 1004
avg unshared stack 129 129 129 129
page reclaims 11.16M 11.16M 11.15M 11.15M
page faults 3321 3674 2940 2801
swaps 0 0 0 0
block input ops 51748 51881 50777 50690
block output ops 5532 6497 5680 6089
messages sent 35847 35848 35789 35715
messages received 35848 35852 35792 35721
signals received 8 8 8 8
voluntary ctx sw 634633 637640 633166 630426
invol. ctx switch 389944 391048 389868 390128
desiredvnodes 11993 11993 11993 11993 11993 11993 11993 11993 11993 11993 11993 11993
maxvnodes (sysstat)(1) 10775 10624 10775 10624
VMIODIRENABLE DISABLED [------------ D ------------] [------------ E ------------]
[BEST CASE ] [BEST CASE ]
machine test1 test2 test1 test2 test1 test2 test1 test2
pass (2) R 1 1 2 2 R 1 1 2 2
vfs.vmiodirenable E 0 0 0 0 E 0 0 0 0
vfs.nameileafonly B 1 1 1 1 B <=0 <=0 <=0 <=0
O O
O REUSE LEAF DIR VNODES O DO NOT REUSE DIR VNODES W/ACTIVE NAMEI
T T
27:12 26:42 27:06 26:33 27:16 26:42 27:12 26:35
real 1632 1602 1626 1593 1636 1602 1632 1595
user 1359 1354 1361 1352 1361 1352 1359 1354
sys 616 617 614 616 614 618 617 610
max resident 16268 16268 16264 16252 16272 16260 16248 16264
avg shared mem 1030 1028 1031 1029 1030 1028 1030 1029
avg unshared data 1004 1005 1007 1003 1004 1003 1006 1006
avg unshared stack 129 129 129 128 129 129 129 129
page reclaims 11.15M 11.15M 11.15M 11.15M 11.15M 11.15M 11.15M 11.15M
page faults 3299 3201 2677 2716 3179 3268 3391 2788
swaps 0 0 0 0 0 0 0 0
block input ops 56118 56310 55623 55287 56228 56187 44508 55618
block output ops 6181 5454 6179 5305 7935 5355 7758 5357
messages sent 35927 35945 35814 35731 35946 35866 35813 35846
messages received 35927 35945 35815 35732 35948 35866 35819 35848
signals received 8 8 8 8 8 8 8 8
voluntary ctx sw 643370 640072 641656 637785 644750 638374 641644 637633
invol. ctx switch 392619 392669 393116 392564 392246 390639 390585 391506
desiredvnodes 11993 11993 11993 11993 11993 11993 11993 11993
maxvnodes (sysstat)(1) 10657 10606 10657 10606 10698 10630 10698 10630
note(1):
Again there aren't enough directory vnodes to force maxvnodes into
overflow.
note(2): PASS 1 vs PASS 2.
Here the lack of memory comes into play. The second pass has no
pre-cached advantage over the first pass. The numbers are
essentially the same between pass1 and pass2.
TESTS 1A, 1B, 1C:
I didn't bother with 1B and 1C. See below.
TESTS 1D, 1E:
I'm not sure why the 'test2' host has 800 or so fewer output ops.
I reran the test twice. /usr on test2 has less data then /usr on
test1 (due to other unrelated tests), but I dunno if that could
account for it. In anycase, output ops have no real bearing on
the test.
Focusing on block input ops one can see that only vmiodirenable
makes a real difference in regards to read caching. The difference
between the numbers for 1A verses 1D and 1E is almost certainly
associated with directory data.
nameileafonly seems to have made no difference at all. And if it
made no difference in 1D and 1E, it certainly would not make a
difference in 1B or 1C. So I didn't bother with 1B and 1C.
-Matt
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200109240843.f8O8htO91785>