From owner-freebsd-hackers Mon Sep 24 1:44: 1 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id DE38737B40C for ; Mon, 24 Sep 2001 01:43:55 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8O8htO91785; Mon, 24 Sep 2001 01:43:55 -0700 (PDT) (envelope-from dillon) Date: Mon, 24 Sep 2001 01:43:55 -0700 (PDT) From: Matt Dillon Message-Id: <200109240843.f8O8htO91785@earth.backplane.com> To: Poul-Henning Kamp , David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Second set of stable buildworld results w/ vmiodirenable & nameileafonly combos Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Ok, here is the second set of results. I didn't run all the tests because nothing I did appeared to really have much of an effect. In this set of tests I set MAXMEM to 128M. As you can see the buildworld took longer verses 512M (no surprise), and vmiodirenable still helped verses otherwise. If one takes into consideration the standard deviation, the directory vnode reclamation parameters made absolutely no difference in the tests. The primary differentiator in all the tests is 'block input ops'. With vmiodirenable turned on it sits at around 51000. With it off it sits at around 56000. In the 512M tests the pass-1 numbers were 26000 with vmiodirenable turned on and 33000 with it off. Pass-2 numbers were 9000 with it on and 18000 with it off. The directory leaf reuse parameters had almost no effect on either the 128M or 512M numbers. I'm not sure why test2 wound up doing a better job then test1 in the 128M tests with vmiodirenable disabled. Both machines are configured identically with only some extra junk on test1's /usr from prior tests. In anycase, the differences point to a rather significant error spread in regards to possible outcomes, at least with vmiodirenable=0. My conclusion from all of this is: * vmiodirenable should be turned on by default. * We should rip out the cache_purgeleafdirs() code entirely and use my simpler version to fix the vnode-growth problem. * We can probably also rip out my cache_leaf_test() .. we do not need to add any sophistication to reuse only directory vnodes without subdirectories in the cache. If it had been a problem we would have seen it. I can leave the sysctl's in place on the commit to allow further testing, and I can leave it conditional on vmiodirenable. I'll set the default vmiodirenable to 1 (which will also enable directory vnode reuse) and the default nameileafonly to 0 (i.e. to use the less sophisticated check). In a few weeks I will rip-out nameileafonly and cache_leaf_test(). -Matt WIDE TERMINAL WINDOW REQUIRED! --------------------------------------------------------------------------------------------------- TEST SUITE 2 (128M ram) buildworld of -stable. DELL2550 (Duel PIII-1.2GHz / 128M ram (via MAXMEM) / SCSI) 23 September 2001 SMP kernel, softupdates-enabled, dirpref'd local /usr/src (no nfs), make -j 12 buildworld UFS_DIRHASH. 2 identical machines tested in parallel (test1, test2) /usr/bin/time -l timings note: atime updates left enabled in all tests REUSE LEAF DIR VNODES: directory vnodes with no subdirectories in the namei cache can be reused REUSE ALL DIR VNODES: directory vnodes can be reused (namei cache ignored) DO NOT REUSE DIR...: (Poul's original 1995 algo) directory vnode can only be reused if no subdirectories or files in the namei cache I stopped bothering with pass-2 after it became evident that the numbers were not changing significantly. VMIODIRENABLE ENABLED [------------ A ------------] [------------ B ------------] [------------ C ------------] [BEST CASE ] [BEST CASE ] [BEST CASE ] machine test1 test2 test1 test2 test1 test2 test1 test2 test1 test2 test1 test2 pass (2) R 1 1 2 2 R 1 1 2 2 R 1 1 2 2 vfs.vmiodirenable E 1 1 1 1 E 1 1 1 1 E 1 1 1 1 vfs.nameileafonly B 1 1 1 1 B 0 0 0 0 B -1 -1 -1 -1 O O O O REUSE LEAF DIR VNODES O REUSE ALL DIR VNODES O DO NOT REUSE DIR VNODES W/ACTIVE NAMEI T T T 26:49 26:30 26:41 26:24 real 1609 1590 1601 1584 user 1361 1354 1361 1356 sys 617 615 617 614 max resident 16264 16256 16260 16264 avg shared mem 1030 1030 1030 1030 avg unshared data 1004 1005 1006 1004 avg unshared stack 129 129 129 129 page reclaims 11.16M 11.16M 11.15M 11.15M page faults 3321 3674 2940 2801 swaps 0 0 0 0 block input ops 51748 51881 50777 50690 block output ops 5532 6497 5680 6089 messages sent 35847 35848 35789 35715 messages received 35848 35852 35792 35721 signals received 8 8 8 8 voluntary ctx sw 634633 637640 633166 630426 invol. ctx switch 389944 391048 389868 390128 desiredvnodes 11993 11993 11993 11993 11993 11993 11993 11993 11993 11993 11993 11993 maxvnodes (sysstat)(1) 10775 10624 10775 10624 VMIODIRENABLE DISABLED [------------ D ------------] [------------ E ------------] [BEST CASE ] [BEST CASE ] machine test1 test2 test1 test2 test1 test2 test1 test2 pass (2) R 1 1 2 2 R 1 1 2 2 vfs.vmiodirenable E 0 0 0 0 E 0 0 0 0 vfs.nameileafonly B 1 1 1 1 B <=0 <=0 <=0 <=0 O O O REUSE LEAF DIR VNODES O DO NOT REUSE DIR VNODES W/ACTIVE NAMEI T T 27:12 26:42 27:06 26:33 27:16 26:42 27:12 26:35 real 1632 1602 1626 1593 1636 1602 1632 1595 user 1359 1354 1361 1352 1361 1352 1359 1354 sys 616 617 614 616 614 618 617 610 max resident 16268 16268 16264 16252 16272 16260 16248 16264 avg shared mem 1030 1028 1031 1029 1030 1028 1030 1029 avg unshared data 1004 1005 1007 1003 1004 1003 1006 1006 avg unshared stack 129 129 129 128 129 129 129 129 page reclaims 11.15M 11.15M 11.15M 11.15M 11.15M 11.15M 11.15M 11.15M page faults 3299 3201 2677 2716 3179 3268 3391 2788 swaps 0 0 0 0 0 0 0 0 block input ops 56118 56310 55623 55287 56228 56187 44508 55618 block output ops 6181 5454 6179 5305 7935 5355 7758 5357 messages sent 35927 35945 35814 35731 35946 35866 35813 35846 messages received 35927 35945 35815 35732 35948 35866 35819 35848 signals received 8 8 8 8 8 8 8 8 voluntary ctx sw 643370 640072 641656 637785 644750 638374 641644 637633 invol. ctx switch 392619 392669 393116 392564 392246 390639 390585 391506 desiredvnodes 11993 11993 11993 11993 11993 11993 11993 11993 maxvnodes (sysstat)(1) 10657 10606 10657 10606 10698 10630 10698 10630 note(1): Again there aren't enough directory vnodes to force maxvnodes into overflow. note(2): PASS 1 vs PASS 2. Here the lack of memory comes into play. The second pass has no pre-cached advantage over the first pass. The numbers are essentially the same between pass1 and pass2. TESTS 1A, 1B, 1C: I didn't bother with 1B and 1C. See below. TESTS 1D, 1E: I'm not sure why the 'test2' host has 800 or so fewer output ops. I reran the test twice. /usr on test2 has less data then /usr on test1 (due to other unrelated tests), but I dunno if that could account for it. In anycase, output ops have no real bearing on the test. Focusing on block input ops one can see that only vmiodirenable makes a real difference in regards to read caching. The difference between the numbers for 1A verses 1D and 1E is almost certainly associated with directory data. nameileafonly seems to have made no difference at all. And if it made no difference in 1D and 1E, it certainly would not make a difference in 1B or 1C. So I didn't bother with 1B and 1C. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message