From owner-freebsd-hackers Sun Sep 23 20:58:24 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 1C35237B41B for ; Sun, 23 Sep 2001 20:58:18 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8O3wCb90533; Sun, 23 Sep 2001 20:58:12 -0700 (PDT) (envelope-from dillon) Date: Sun, 23 Sep 2001 20:58:12 -0700 (PDT) From: Matt Dillon Message-Id: <200109240358.f8O3wCb90533@earth.backplane.com> To: Poul-Henning Kamp , David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: stable buildworld results w/ vmiodirenable & nameileafonly combos Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Ok, here are the first set of results. I am going to rerun the entire suite of tests again with the machines limited to 128M of ram to see what happens then. BTW, these are really nice machines! I highly recommend DELL2550's. The results w/ 512M are basically that it doesn't matter what we do with the namei cache. vmiodirenable is the only thing that really makes a difference. I expect we will see more differentiation in the 128M tests. -Matt WIDE TERMINAL WINDOW REQUIRED! --------------------------------------------------------------------------------------------------- TEST SUITE 1 (512M ram) buildworld of -stable. DELL2550 (Duel PIII-1.2GHz / 512M ram / SCSI) 23 September 2001 SMP kernel, softupdates-enabled, dirpref'd local /usr/src (no nfs), make -j 12 buildworld UFS_DIRHASH. 2 identical machines tested in parallel (test1, test2) /usr/bin/time -l timings note: atime updates left enabled in all tests REUSE LEAF DIR VNODES: directory vnodes with no subdirectories in the namei cache can be reused REUSE ALL DIR VNODES: directory vnodes can be reused (namei cache ignored) DO NOT REUSE DIR...: (Poul's original 1995 algo) directory vnode can only be reused if no subdirectories or files in the namei cache VMIODIRENABLE ENABLED [------------ A ------------] [------------ B ------------] [------------ C ------------] [BEST CASE ] [BEST CASE ] [BEST CASE ] machine test1 test2 test1 test2 test1 test2 test1 test2 test1 test2 test1 test2 pass (2) R 1 1 2 2 R 1 1 2 2 R 1 1 2 2 vfs.vmiodirenable E 1 1 1 1 E 1 1 1 1 E 1 1 1 1 vfs.nameileafonly B 1 1 1 1 B 0 0 0 0 B -1 -1 -1 -1 O O O O REUSE LEAF DIR VNODES O REUSE ALL DIR VNODES O DO NOT REUSE DIR VNODES W/ACTIVE NAMEI T T T 25:46 25:44 25:19 25:05 25:49 25:40 25:14 25:04 25:46 25:42 25:07 25:14 real 1546 1544 1519 1505 1549 1540 1514 1504 1546 1542 1507 1514 user 1361 1352 1359 1356 1362 1354 1361 1354 1361 1352 1358 1355 sys 636 637 645 640 632 633 642 641 636 637 642 640 max resident 16292 16276 16268 16288 16284 16288 16280 16280 16288 16288 16284 16288 avg shared mem 1026 1025 1025 1025 1027 1028 1025 1023 1025 1026 1026 1025 avg unshared data 1018 1009 1014 1007 1007 1010 1007 1008 1010 1018 1006 1002 avg unshared stack 129 129 129 129 129 129 129 128 129 129 129 129 page reclaims 11.15M 11.15M 11.16M 11.15M 11.15M 11.16M 11.15M 11.16M 11.15M 11.15M 11.15M 11.15M page faults 1812 1800 1316 1348 1797 1798 1320 1273 1795 1797 1307 1321 swaps 0 0 0 0 0 0 0 0 0 0 0 0 block input ops 26542 26535 9272 9233 26555 26577 9258 8720 26470 26552 8905 9237 block output ops 5400 5217 5248 5266 5450 5109 5328 5389 5332 5371 5300 5341 messages sent 34582 34572 33533 33538 34579 34538 33539 33510 34610 34587 33525 33543 messages received 34582 34578 33533 33539 34580 34540 33546 33524 34610 34589 33525 33543 signals received 8 8 8 8 8 8 8 8 8 8 8 8 voluntary ctx sw 594390 595868 575038 574626 594677 594028 571265 572912 593500 593548 571394 573575 invol. ctx switch 380583 381897 378850 376941 380393 381540 374408 376260 380385 379585 375318 374844 desiredvnodes 36157 36157 36157 36157 36157 36157 36157 36157 36157 36157 36157 36157 maxvnodes (sysstat)(1) 37099 37064 37122 37208 37180 36928 37193 37314 37175 37152 37152 37152 VMIODIRENABLE DISABLED [------------ D ------------] [------------ E ------------] [BEST CASE ] [BEST CASE ] machine test1 test2 test1 test2 test1 test2 test1 test2 pass (2) R 1 1 2 2 R 1 1 2 2 vfs.vmiodirenable E 0 0 0 0 E 0 0 0 0 vfs.nameileafonly B 1 1 1 1 B <=0 <=0 <=0 <=0 O O O REUSE LEAF DIR VNODES O DO NOT REUSE DIR VNODES W/ACTIVE NAMEI T T 26:19 26:03 25:50 25:37 26:16 26:10 25:46 25:43 real 1579 1563 1550 1537 1576 1570 1546 1543 user 1361 1356 1360 1355 1359 1349 1361 1354 sys 634 632 640 637 636 639 641 641 max resident 16264 16276 16276 16280 16292 16280 16284 16283 avg shared mem 1026 1025 1026 1025 1027 1025 1025 1026 avg unshared data 1012 1005 1010 1009 1009 1007 1010 1010 avg unshared stack 129 129 129 129 129 129 129 129 page reclaims 11.15M 11.15M 11.15M 11.15M 11.15M 11.15M 11.15M 11.15M page faults 1716 1720 1272 1263 1717 1758 1264 1244 swaps 0 0 0 0 0 0 0 0 block input ops 33716 33749 18888 18930 33788 33849 18760 18794 block output ops 5283 5304 5305 5365 5525 5411 5423 5465 messages sent 34759 34732 33659 33650 34710 34707 33645 33686 messages received 34760 34738 33660 33651 34711 34707 33649 33693 signals received 8 8 8 8 8 8 8 8 voluntary ctx sw 604747 602379 586746 584227 603949 603830 584766 585322 invol. ctx switch 384576 383500 381032 379730 383183 383798 379336 380705 desiredvnodes 36157 36157 36157 36157 36157 36157 36157 36157 maxvnodes (sysstat)(1) 37096 37062 37488 37231 37107 37228 37107 37228 note(1): there aren't enough discrete directories in /usr/src and /usr/obj to make a difference in the maxvnode numbers no matter what algorithm is chosen when vmiodirenable is turned on. /usr/src has 6607 directories and 62635 files. Running the entire suite of tests again with the machine artificially reduced to 128M of memory should yield more interesting results. note(2): PASS 1 vs PASS 2. A second buildworld pass is taken after the first completes. Considering the amount of ram in the machines the second pass is expected to perform much better due to prior caching of data. TESTS 1A, 1B, 1C: All results are within a standard-deviation of each other. With vmiodirenabled turned on, it does not appear to matter what type of namecache restriction(s) we place on directory vnode reuse. The existance of direcory pages in the VM Page cache appear to override all other considerations (at least on these boxes, with 512M of physical ram). Again, running the entire suite of tests again with the machines artifically reduced to 128M of memory should yield more interesting results. TESTS 1D, 1E: These tests were run with vmiodirenable turned off. This has the effect of reducing (by two orderes of magnitude) the amount of memory available to cache directories. Therefore the namei cache tests in the vnode reuse code become much more important. Turning off vmiodirenable resulted in around 40 second longer buildworlds even in the best case. Again, these numbers should alter more drastically in SUITE 2 when we test with less physical memory. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message