From owner-freebsd-performance@FreeBSD.ORG Sun Apr 11 07:55:13 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AE05A16A4CE for ; Sun, 11 Apr 2004 07:55:13 -0700 (PDT) Received: from mx4.mail.ru (fallback.mail.ru [194.67.57.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id B7A7243D54 for ; Sun, 11 Apr 2004 07:55:12 -0700 (PDT) (envelope-from my-subs@mail.ru) Received: from mx2.mail.ru (unknown [194.67.23.122]) by mx4.mail.ru (mPOP.Fallback_MX) with ESMTP id EF09D2C49B2 for ; Sun, 11 Apr 2004 18:55:10 +0400 (MSD) Received: from [81.218.200.8] (port=1771 helo=old.home) by mx2.mail.ru with smtp id 1BCgML-000ILU-00 for freebsd-performance@freebsd.org; Sun, 11 Apr 2004 18:54:57 +0400 Date: Sun, 11 Apr 2004 17:54:48 +0300 From: Alexander Portnoy To: freebsd-performance@freebsd.org Message-Id: <20040411175448.23c5059f.my-subs@mail.ru> In-Reply-To: References: X-Mailer: Sylpheed (FreeBSD) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spam: Probable Spam Subject: Re: optimization of the system by recompilation X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Apr 2004 14:55:13 -0000 On Sun, 11 Apr 2004 01:29:22 +0200 Patrick Proniewski wrote: > Hello, > > I wonder if tuning the make.conf and the kernel to take into account > specific CPU features/options does really make a difference in terms of > performance on FreeBSD. > I've googled for benches but can't find any interesting papers. > Any info greatly appreciated. > > patpro > -- > je cherche un poste d'admin-sys Mac/UNIX > (ou une jeune et jolie femme riche) > http://patpro.net/cv.php > Yes, It make a difference. But the difference is not always positive. I tested two things: 'make buildworld' and openssl under FreeBSD-5.2.1-RELEASE installed on notebook "HP Pavilion ze4523ea" (AMD Athlon-XP Mobile 2400+ 1.8GHz 512KB L2 cache, 256MB DDR266 RAM, ALi chipset based MB, Seagate ST93012A HDD) and desktop system "Compaq Evo D310" (Pentium-IV 2400MHz 512KB L2 cache 533MHz FSB w/o HTT, 256MB DDR266 RAM, Intel-845 chipset based MB, Maxtor-2F040L0 HDD). The Pentium-IV-based system presented for rough comparison "AMD vs Intel" only. So, when I talk about "two systems", I mean two Athlon-based installations on the HP notebook: optimized and not-optimized ones. On the notebook I executed 'make buildworld' twice: the first time in absolutely clean system with GENERIC kernel and binary-installed world, and the second time in recompiled system and kernel. Each test performed without make.conf in single-user mode. In the second case the installed kernel have support of SSE and AMD processors' specific instructions and the world was built with the following options: CPUTYPE?=athlon-xp CFLAGS= -O -pipe CXXFLAGS+= -fmemoize-lookups -fsave-memoized COPTFLAGS= -O -pipe WANT_FORCE_OPTIMIZATION_DOWNGRADE=1 MAKE_IDEA= YES Of course, the optons used for compiling the test system, but not in the tests. So, we have two systems (optimized and not) running on same hardware and executing same task. The results are as follows: make buildworld on not-optimized Athlon-based system: real:2730.52 user:1980.90 system:459.79 make buildworld on optimized Athlon-based system: real:2840.27 user:2074.22 system:456.88 As You can see, this specific task runs slower as result of the optimization. On Pentium-IV with generic binary installation I got these results: make buildworld on not-optimized P-IV system: real:2591.67 user:1855.77 system:512.19 Currently, this system runs recompiled optimized (in the same manner as the Athlon-based system, but using according CPU type) world and kernel, but I not tested the effect of the optimization on the 'make buildworld'. I'll do It in three days. ================================================================================ There are the results of 'openssl speed' test: -------------------------------------------------------------------------------- The not-optimized Athlon-based system: OpenSSL 0.9.7c 30 Sep 2003 built on: Mon Feb 23 18:18:12 GMT 2004 options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx) compiler: cc available timing options: USE_TOD HZ=128 [sysconf value] timing function used: getrusage The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes md2 1353.84k 2883.85k 4037.49k 4498.69k 4637.52k mdc2 3099.95k 3513.05k 3630.29k 3667.98k 3668.41k md4 11665.58k 41273.75k 121422.56k 236540.37k 328188.46k md5 9178.08k 30801.73k 83847.38k 148018.63k 191651.13k hmac(md5) 5219.81k 18714.10k 58407.60k 124940.65k 187243.86k sha1 9021.76k 27620.49k 65920.32k 100429.75k 118665.95k rmd160 7595.82k 21845.25k 46756.02k 66160.23k 75117.55k rc4 91594.44k 97533.81k 99011.52k 99675.56k 99841.45k des cbc 18603.86k 19156.64k 19291.56k 19336.36k 19348.79k des ede3 6752.80k 6823.83k 6847.10k 6852.97k 6854.59k idea cbc 0.00 0.00 0.00 0.00 0.00 rc2 cbc 16364.82k 16954.69k 17169.81k 17204.85k 17165.59k rc5-32/12 cbc 76003.59k 91265.62k 95365.92k 96674.31k 97055.51k blowfish cbc 45465.01k 50084.68k 51324.54k 51718.48k 51965.94k cast cbc 31093.46k 33235.03k 33800.51k 33974.97k 34024.42k aes-128 cbc 38303.47k 39168.76k 39528.18k 39782.53k 39713.81k aes-192 cbc 33541.45k 34119.22k 34379.99k 34494.11k 34610.47k aes-256 cbc 29583.16k 30149.21k 30502.23k 30512.58k 30532.75k sign verify sign/s verify/s rsa 512 bits 0.0014s 0.0001s 725.1 8380.1 rsa 1024 bits 0.0072s 0.0004s 138.0 2743.8 rsa 2048 bits 0.0437s 0.0012s 22.9 826.9 rsa 4096 bits 0.2889s 0.0041s 3.5 242.7 sign verify sign/s verify/s dsa 512 bits 0.0012s 0.0015s 847.8 688.5 dsa 1024 bits 0.0036s 0.0045s 275.7 223.2 dsa 2048 bits 0.0119s 0.0146s 84.4 68.7 -------------------------------------------------------------------------------- The optimized Athlon-based system: OpenSSL 0.9.7c-p1 30 Sep 2003 built on: Sat Apr 3 19:27:44 IST 2004 options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) idea(int) blowfish(idx) compiler: cc available timing options: USE_TOD HZ=128 [sysconf value] timing function used: getrusage The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes md2 1469.85k 3141.10k 4364.08k 4845.00k 5018.59k mdc2 3422.22k 3961.57k 4106.14k 4148.13k 4168.24k md4 13293.53k 46403.23k 133057.80k 247860.40k 332391.22k md5 9923.45k 32653.90k 87393.52k 150725.62k 191716.54k hmac(md5) 5886.66k 21173.88k 64159.59k 131836.68k 188889.57k sha1 10234.21k 30260.14k 69060.28k 100388.71k 115805.02k rmd160 8126.79k 22809.78k 48072.18k 66651.00k 75161.24k rc4 90439.87k 95741.84k 97001.42k 97528.56k 97796.82k des cbc 18663.73k 19158.23k 19290.86k 19385.42k 19346.18k des ede3 6763.25k 6833.31k 6856.65k 6862.52k 6863.96k idea cbc 25537.24k 26803.52k 27140.75k 27241.67k 27318.54k rc2 cbc 15347.39k 15948.06k 16081.37k 16116.49k 16115.75k rc5-32/12 cbc 76203.31k 90366.69k 94297.41k 95552.81k 95914.76k blowfish cbc 45606.12k 50249.45k 51453.63k 51857.83k 51971.32k cast cbc 31290.53k 33480.59k 34130.29k 34222.75k 34260.36k aes-128 cbc 39364.21k 39919.03k 40532.64k 40556.81k 40592.93k aes-192 cbc 34185.24k 34811.82k 35197.58k 35295.92k 35323.30k aes-256 cbc 30500.45k 30863.06k 31166.10k 31243.22k 31264.33k sign verify sign/s verify/s rsa 512 bits 0.0011s 0.0001s 870.6 9641.5 rsa 1024 bits 0.0062s 0.0003s 161.3 3127.8 rsa 2048 bits 0.0384s 0.0011s 26.1 935.0 rsa 4096 bits 0.2530s 0.0037s 4.0 272.4 sign verify sign/s verify/s dsa 512 bits 0.0010s 0.0012s 993.3 852.1 dsa 1024 bits 0.0031s 0.0037s 319.1 267.1 dsa 2048 bits 0.0103s 0.0124s 97.2 80.5 -------------------------------------------------------------------------------- Optimized Pentium-IV-based system: OpenSSL 0.9.7c-p1 30 Sep 2003 built on: Fri Apr 2 08:27:27 IST 2004 options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) idea(int) blowfish(idx) compiler: cc available timing options: USE_TOD HZ=128 [sysconf value] timing function used: getrusage The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes md2 1726.48k 3776.53k 5370.03k 6055.36k 6271.25k mdc2 3273.95k 3776.35k 3931.20k 3968.70k 3991.80k md4 11715.58k 38997.36k 103675.62k 178255.61k 238275.60k md5 8934.14k 28045.88k 69794.77k 106627.55k 124504.85k hmac(md5) 4645.41k 15848.47k 47652.73k 90719.45k 121526.49k sha1 8252.23k 23922.96k 52550.74k 74687.05k 85330.16k rmd160 6495.77k 17329.79k 34379.13k 45735.61k 50501.56k rc4 77374.43k 85999.31k 88387.11k 89063.26k 88959.69k des cbc 12892.70k 13269.26k 13355.82k 13448.57k 13487.01k des ede3 4735.53k 4773.85k 4785.02k 4799.68k 4794.03k idea cbc 13666.19k 14120.88k 14239.32k 14305.44k 14278.27k rc2 cbc 11083.37k 11796.76k 11756.10k 11896.34k 11602.77k rc5-32/12 cbc 59411.37k 64298.25k 65320.48k 66806.59k 66736.33k blowfish cbc 39247.42k 42614.92k 43648.10k 43835.60k 43885.58k cast cbc 25203.76k 26688.89k 27010.57k 27590.74k 27322.58k aes-128 cbc 65918.18k 66334.91k 68613.90k 70164.26k 69141.19k aes-192 cbc 56768.29k 58742.12k 61008.16k 61665.29k 60882.01k aes-256 cbc 52524.87k 53551.62k 54792.55k 54951.13k 54106.28k sign verify sign/s verify/s rsa 512 bits 0.0015s 0.0001s 667.3 7133.5 rsa 1024 bits 0.0084s 0.0004s 119.5 2324.9 rsa 2048 bits 0.0514s 0.0015s 19.5 679.1 rsa 4096 bits 0.3451s 0.0052s 2.9 193.5 sign verify sign/s verify/s dsa 512 bits 0.0013s 0.0016s 752.3 632.4 dsa 1024 bits 0.0042s 0.0050s 238.6 199.0 dsa 2048 bits 0.0142s 0.0172s 70.2 58.3 -------------------------------------------------------------------------------- In openssl the results are not so bad: some algorithms runs faster and another ones slower in the optimized system. Take a look at the difference between performance of Atlon and Pentium on "idea cbc" test. Another example is "aes-256 cbc". So, as You can see, there is not a simple answer to Your question in the common case. If You interested in high performance in some specific task - You must test the task on different hardware platforms with different optimizations (or without ones ;-) From owner-freebsd-performance@FreeBSD.ORG Wed Apr 14 02:21:44 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 85A7F16A4CE for ; Wed, 14 Apr 2004 02:21:44 -0700 (PDT) Received: from boleskine.patpro.net (boleskine.patpro.net [62.4.20.155]) by mx1.FreeBSD.org (Postfix) with ESMTP id 08AAD43D48 for ; Wed, 14 Apr 2004 02:21:44 -0700 (PDT) (envelope-from patpro@patpro.net) Received: from [192.168.0.1] (cassandre [192.168.0.1]) by boleskine.patpro.net (Postfix) with ESMTP id C849239F for ; Wed, 14 Apr 2004 11:21:45 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v613) In-Reply-To: <20040413190123.D261E16A4E6@hub.freebsd.org> References: <20040413190123.D261E16A4E6@hub.freebsd.org> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <23FFDAB6-8DF5-11D8-8A50-0030654D97EC@patpro.net> Content-Transfer-Encoding: 7bit From: Patrick Proniewski Date: Wed, 14 Apr 2004 11:21:41 +0200 To: freebsd-performance@freebsd.org X-Mailer: Apple Mail (2.613) Subject: Re: optimization of the system by recompilation X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Apr 2004 09:21:44 -0000 On 13 avr. 2004, at 21:01, freebsd-performance-request@freebsd.org wrote: > So, as You can see, there is not a simple answer to Your question in > the common case. > If You interested in high performance in some specific task - You must > test the task > on different hardware platforms with different optimizations (or > without ones ;-) Ok, thank you very much for the comprehensive reply ! OpenSSL results in particular are very interesting. I've noticed that the idea cbc bench will just not run on a non-Athlon optimized binary (on the Athlon of course). Do you have any explanations ? regards, patpro -- je cherche un poste d'admin-sys Mac/UNIX (ou une jeune et jolie femme riche) http://patpro.net/cv.php From owner-freebsd-performance@FreeBSD.ORG Thu Apr 15 09:45:25 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8403216A4CE for ; Thu, 15 Apr 2004 09:45:25 -0700 (PDT) Received: from mx2.mail.ru (mx2.mail.ru [194.67.23.122]) by mx1.FreeBSD.org (Postfix) with ESMTP id 221DB43D2D for ; Thu, 15 Apr 2004 09:45:25 -0700 (PDT) (envelope-from my-subs@mail.ru) Received: from [81.218.200.8] (port=4367 helo=old.home) by mx2.mail.ru with smtp id 1BE9zO-000AVh-00 for freebsd-performance@freebsd.org; Thu, 15 Apr 2004 20:45:23 +0400 Date: Thu, 15 Apr 2004 19:44:46 +0300 From: Alexander Portnoy To: freebsd-performance@freebsd.org Message-Id: <20040415194446.7066c67f.my-subs@mail.ru> In-Reply-To: <23FFDAB6-8DF5-11D8-8A50-0030654D97EC@patpro.net> References: <20040413190123.D261E16A4E6@hub.freebsd.org> <23FFDAB6-8DF5-11D8-8A50-0030654D97EC@patpro.net> X-Mailer: Sylpheed (FreeBSD) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spam: Probable Spam Subject: Re: optimization of the system by recompilation X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Apr 2004 16:45:25 -0000 On Wed, 14 Apr 2004 11:21:41 +0200 Patrick Proniewski wrote: > On 13 avr. 2004, at 21:01, freebsd-performance-request@freebsd.org > wrote: > > > So, as You can see, there is not a simple answer to Your question in > > the common case. > > If You interested in high performance in some specific task - You must > > test the task > > on different hardware platforms with different optimizations (or > > without ones ;-) > > Ok, thank you very much for the comprehensive reply ! > OpenSSL results in particular are very interesting. I've noticed that > the > idea cbc bench will just not run on a non-Athlon optimized binary (on > the > Athlon of course). Do you have any explanations ? > > regards, > > patpro > -- It will run. Just IDEA is not included in binary FreeBSD distribution. In order to use It You must include "MAKE_IDEA= YES" line in /etc/make.conf and rebuild world. >From /etc/make.conf: # The following controls building optional IDEA code in libcrypto and # certain ports. Patents are involved - you must not use this unless # you either have a license or fall within patent 'fair use' # provisions. # # *** It is YOUR RESPONSIBILITY to determine if you can use this! *** # # IDEA is patented in the USA and many European countries - thought to # be OK to use for any non-commercial use. This is optional. #MAKE_IDEA= YES # IDEA (128 bit symmetric encryption) From owner-freebsd-performance@FreeBSD.ORG Thu Apr 15 12:08:19 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 25CBF16A4CE for ; Thu, 15 Apr 2004 12:08:19 -0700 (PDT) Received: from flake.decibel.org (flake.decibel.org [66.143.173.58]) by mx1.FreeBSD.org (Postfix) with SMTP id 9E87A43D48 for ; Thu, 15 Apr 2004 12:08:17 -0700 (PDT) (envelope-from decibel@decibel.org) Received: (qmail 82620 invoked by uid 1001); 15 Apr 2004 19:08:08 -0000 Date: Thu, 15 Apr 2004 14:08:08 -0500 From: "Jim C. Nasby" To: freebsd-performance@freebsd.org Message-ID: <20040415190808.GA74840@nasby.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-Operating-System: FreeBSD 4.9-RELEASE-p3 i386 X-Distributed: Join the Effort! http://www.distributed.net Subject: Disk read v. write stats X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Apr 2004 19:08:19 -0000 Is there any way to get stats on reads v. writes to a filesystem or raw device? Something like vmstat or iostat, but broken down by read and write requests. -- Jim C. Nasby, Database Consultant jim@nasby.net Member: Triangle Fraternity, Sports Car Club of America Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to go today?" Linux: "Where do you want to go tomorrow?" FreeBSD: "Are you guys coming, or what?" From owner-freebsd-performance@FreeBSD.ORG Thu Apr 15 13:18:16 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 51AD716A4CE for ; Thu, 15 Apr 2004 13:18:16 -0700 (PDT) Received: from rms04.rommon.net (rms04.rommon.net [212.54.2.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1E16543D60 for ; Thu, 15 Apr 2004 13:18:15 -0700 (PDT) (envelope-from pete@he.iki.fi) Received: from he.iki.fi (h91.vuokselantie10.fi [193.64.42.145]) by rms04.rommon.net (8.12.10/8.12.9) with ESMTP id i3FKIFmo048314; Thu, 15 Apr 2004 23:18:15 +0300 (EEST) (envelope-from pete@he.iki.fi) Message-ID: <407EEE01.7060102@he.iki.fi> Date: Thu, 15 Apr 2004 23:18:09 +0300 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "Jim C. Nasby" References: <20040415190808.GA74840@nasby.net> In-Reply-To: <20040415190808.GA74840@nasby.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-performance@freebsd.org Subject: Re: Disk read v. write stats X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Apr 2004 20:18:16 -0000 Jim C. Nasby wrote: >Is there any way to get stats on reads v. writes to a filesystem or raw >device? Something like vmstat or iostat, but broken down by read and >write requests. > > See man 8 gstat Pete From owner-freebsd-performance@FreeBSD.ORG Thu Apr 15 15:52:37 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1D8BC16A4CF for ; Thu, 15 Apr 2004 15:52:37 -0700 (PDT) Received: from boleskine.patpro.net (boleskine.patpro.net [62.4.20.155]) by mx1.FreeBSD.org (Postfix) with ESMTP id 75E6843D39 for ; Thu, 15 Apr 2004 15:52:36 -0700 (PDT) (envelope-from patpro@patpro.net) Received: from [192.168.0.1] (cassandre [192.168.0.1]) by boleskine.patpro.net (Postfix) with ESMTP id E075A31 for ; Fri, 16 Apr 2004 00:52:36 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v613) In-Reply-To: <20040415190058.A2B8116A4E5@hub.freebsd.org> References: <20040415190058.A2B8116A4E5@hub.freebsd.org> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <9568CC84-8F2F-11D8-95CB-0030654D97EC@patpro.net> Content-Transfer-Encoding: 7bit From: Patrick Proniewski Date: Fri, 16 Apr 2004 00:52:34 +0200 To: freebsd-performance@freebsd.org X-Mailer: Apple Mail (2.613) Subject: Re: optimization of the system by recompilation X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Apr 2004 22:52:37 -0000 > It will run. Just IDEA is not included in binary FreeBSD distribution. > In order to use It You must include "MAKE_IDEA= YES" line in > /etc/make.conf > and rebuild world. thanx ! patpro -- je cherche un poste d'admin-sys Mac/UNIX (ou une jeune et jolie femme riche) http://patpro.net/cv.php From owner-freebsd-performance@FreeBSD.ORG Thu Apr 15 20:20:08 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0708516A4CF for ; Thu, 15 Apr 2004 20:20:08 -0700 (PDT) Received: from main.gmane.org (main.gmane.org [80.91.224.249]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5048C43D45 for ; Thu, 15 Apr 2004 20:20:07 -0700 (PDT) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from root by main.gmane.org with local (Exim 3.35 #1 (Debian)) id 1BEJtd-0004wH-00 for ; Fri, 16 Apr 2004 05:20:05 +0200 Received: from 66-117-150-96.web.lmi.net ([66.117.150.96]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 16 Apr 2004 05:20:05 +0200 Received: from aditya by 66-117-150-96.web.lmi.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 16 Apr 2004 05:20:05 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: Aditya Date: Thu, 15 Apr 2004 23:07:07 -0400 Organization: Grot Free Lines: 16 Message-ID: References: <20040415190808.GA74840@nasby.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 66-117-150-96.web.lmi.net X-Archive: encrypt User-Agent: Gnus/5.090007 (Oort Gnus v0.07) XEmacs/21.4 (Common Lisp, i386--freebsd) Cancel-Lock: sha1:iHbC3fs9AuiyrfeBvj2gA6/cNlA= Sender: news Subject: Re: Disk read v. write stats X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Apr 2004 03:20:08 -0000 > On Thu, 15 Apr 2004 14:08:08 -0500, "Jim C. Nasby" said: > Is there any way to get stats on reads v. writes to a filesystem or > raw device? Something like vmstat or iostat, but broken down by read > and write requests. I asked the same question on questions a day ago and for FreeBSD 4, Dan Nelson provided some patches for iostat to add the read/write stats: http://lists.freebsd.org/pipermail/freebsd-questions/2004-April/043644.html gstat does work for FreeBSD 5 but unfortunately it doesn't print simple text output that can be used to collect long term stats via a script... Thanks, Adi From owner-freebsd-performance@FreeBSD.ORG Fri Apr 16 09:38:50 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 40F6F16A4CE for ; Fri, 16 Apr 2004 09:38:50 -0700 (PDT) Received: from flake.decibel.org (flake.decibel.org [66.143.173.58]) by mx1.FreeBSD.org (Postfix) with SMTP id BAC0843D55 for ; Fri, 16 Apr 2004 09:38:49 -0700 (PDT) (envelope-from decibel@decibel.org) Received: (qmail 49974 invoked by uid 1001); 16 Apr 2004 16:38:45 -0000 Date: Fri, 16 Apr 2004 11:38:45 -0500 From: "Jim C. Nasby" To: freebsd-performance@freebsd.org Message-ID: <20040416163845.GG87362@nasby.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Operating-System: FreeBSD 4.9-RELEASE-p3 i386 X-Distributed: Join the Effort! http://www.distributed.net User-Agent: Mutt/1.5.6i Subject: How does disk caching work? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Apr 2004 16:38:50 -0000 Is there a document anywhere that describes in detail how FreeBSD handles disk caching? I've read Matt Dillon's description of the VM system, but it deals mostly with programs, other than vague statements such as 'FreeBSD uses all available memory for disk caching'. I think I know how caching memory mapped IO works for the most part, since it should be treated just like program data, but what about files that aren't memory mapped? What impact is there as pages move from active to inactive to cache to free? What role do wired and buffer pages play? -- Jim C. Nasby, Database Consultant jim@nasby.net Member: Triangle Fraternity, Sports Car Club of America Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to go today?" Linux: "Where do you want to go tomorrow?" FreeBSD: "Are you guys coming, or what?" From owner-freebsd-performance@FreeBSD.ORG Fri Apr 16 14:57:23 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 72EF016A4CE for ; Fri, 16 Apr 2004 14:57:23 -0700 (PDT) Received: from f7.mail.ru (f7.mail.ru [194.67.57.37]) by mx1.FreeBSD.org (Postfix) with ESMTP id 29DB843D31 for ; Fri, 16 Apr 2004 14:57:23 -0700 (PDT) (envelope-from shmukler@mail.ru) Received: from mail by f7.mail.ru with local id 1BEbKR-000ISM-00; Sat, 17 Apr 2004 01:56:55 +0400 Received: from [24.184.137.35] by msg.mail.ru with HTTP; Sat, 17 Apr 2004 01:56:55 +0400 From: =?koi8-r?Q?=22?=Igor Shmukler=?koi8-r?Q?=22=20?= To: =?koi8-r?Q?=22?=Jim C.Nasby=?koi8-r?Q?=22=20?= Mime-Version: 1.0 X-Mailer: mPOP Web-Mail 2.19 X-Originating-IP: [24.184.137.35] Date: Sat, 17 Apr 2004 01:56:55 +0400 In-Reply-To: <20040416163845.GG87362@nasby.net> Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 8bit Message-Id: cc: freebsd-performance@freebsd.org Subject: Re: How does disk caching work? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: =?koi8-r?Q?=22?=Igor Shmukler=?koi8-r?Q?=22=20?= List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Apr 2004 21:57:23 -0000 > Is there a document anywhere that describes in detail how FreeBSD > handles disk caching? I've read Matt Dillon's description of the VM > system, but it deals mostly with programs, other than vague statements > such as 'FreeBSD uses all available memory for disk caching'. Well, the statement is not vague. FreeBSD has a unified buffer cache. This means that ALL AVAILABLE MEMORY IS A BUFFER CACHE for all device IO. > I think I know how caching memory mapped IO works for the most part, > since it should be treated just like program data, but what about files > that aren't memory mapped? What impact is there as pages move from > active to inactive to cache to free? What role do wired and buffer pages > play? If file is not memory mapped it is not in memory, is it? Where do you cache it? Maybe I am missing somewhing? Do you maybe want to know about node caching? When pages are rotated from active to inactive and then to cache buckets they is still retains vnode references. Once it is in free queue, there is no way to put it back to cache. Association is lost. Wired pages are to pin memory. So that we do not get situation when fault handling code is paged out. I am not FreeBSD guru so I never heard of BUFFER pages. Is there such a concept? IS From owner-freebsd-performance@FreeBSD.ORG Fri Apr 16 15:06:06 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3D17016A4CE for ; Fri, 16 Apr 2004 15:06:06 -0700 (PDT) Received: from flake.decibel.org (flake.decibel.org [66.143.173.58]) by mx1.FreeBSD.org (Postfix) with SMTP id B176543D41 for ; Fri, 16 Apr 2004 15:06:05 -0700 (PDT) (envelope-from decibel@decibel.org) Received: (qmail 65642 invoked by uid 1001); 16 Apr 2004 22:05:56 -0000 Date: Fri, 16 Apr 2004 17:05:56 -0500 From: "Jim C. Nasby" To: freebsd-performance@freebsd.org Message-ID: <20040416220556.GL87362@nasby.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Operating-System: FreeBSD 4.9-RELEASE-p3 i386 X-Distributed: Join the Effort! http://www.distributed.net User-Agent: Mutt/1.5.6i Subject: command piped into bzip not using all available CPU X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Apr 2004 22:06:06 -0000 As you can see below, a command piped into bzip2 is only effectively using one CPU. It's not disk bound, both systat and gstat report less than 10% disk utilization. Why is this? The command I'm running is: pg_dump -vZ0 ogr | bzip2 > ogr-20040416.sql.bz2 last pid: 18345; load averages: 1.17, 1.09, 0.81 up 8+22:12:27 17:00:56 66 processes: 2 running, 64 sleeping CPU states: 49.4% user, 0.0% nice, 3.7% system, 0.2% interrupt, 46.7% idle Mem: 67M Active, 2935M Inact, 359M Wired, 331M Cache, 255M Buf, 5576K Free Swap: 8192M Total, 64M Used, 8127M Free, 48K Out PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND 17334 decibel 109 0 10856K 7164K CPU0 0 11:05 65.77% 65.77% bzip2 17335 pgsql 4 0 154M 124M sbwait 0 5:54 34.03% 34.03% postgres 17333 decibel -8 0 20128K 3236K pipdwt 0 0:46 2.88% 2.88% pg_dump -- Jim C. Nasby, Database Consultant jim@nasby.net Member: Triangle Fraternity, Sports Car Club of America Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to go today?" Linux: "Where do you want to go tomorrow?" FreeBSD: "Are you guys coming, or what?" From owner-freebsd-performance@FreeBSD.ORG Fri Apr 16 15:12:12 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B7A6B16A4CE for ; Fri, 16 Apr 2004 15:12:12 -0700 (PDT) Received: from flake.decibel.org (flake.decibel.org [66.143.173.58]) by mx1.FreeBSD.org (Postfix) with SMTP id 50C4543D55 for ; Fri, 16 Apr 2004 15:12:12 -0700 (PDT) (envelope-from decibel@decibel.org) Received: (qmail 65825 invoked by uid 1001); 16 Apr 2004 22:12:11 -0000 Date: Fri, 16 Apr 2004 17:12:11 -0500 From: "Jim C. Nasby" To: Igor Shmukler Message-ID: <20040416221211.GM87362@nasby.net> References: <20040416163845.GG87362@nasby.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 4.9-RELEASE-p3 i386 X-Distributed: Join the Effort! http://www.distributed.net User-Agent: Mutt/1.5.6i cc: freebsd-performance@freebsd.org Subject: Re: How does disk caching work? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Apr 2004 22:12:12 -0000 On Sat, Apr 17, 2004 at 01:56:55AM +0400, "Igor Shmukler" wrote: > > Is there a document anywhere that describes in detail how FreeBSD > > handles disk caching? I've read Matt Dillon's description of the VM > > system, but it deals mostly with programs, other than vague statements > > such as 'FreeBSD uses all available memory for disk caching'. > > Well, the statement is not vague. FreeBSD has a unified buffer cache. This means that ALL AVAILABLE > MEMORY IS A BUFFER CACHE for all device IO. > > > I think I know how caching memory mapped IO works for the most part, > > since it should be treated just like program data, but what about files > > that aren't memory mapped? What impact is there as pages move from > > active to inactive to cache to free? What role do wired and buffer pages > > play? > > If file is not memory mapped it is not in memory, is it? Where do you cache it? Maybe I am missing > somewhing? Do you maybe want to know about node caching? What if the file isn't memory mapped? You can access a file without mapping it into memory, right? > When pages are rotated from active to inactive and then to cache buckets they is still retains vnode > references. Once it is in free queue, there is no way to put it back to cache. Association is lost. > > Wired pages are to pin memory. So that we do not get situation when fault handling code is paged out. > > I am not FreeBSD guru so I never heard of BUFFER pages. Is there such a concept? I'm reffering to the 'Buf' column at the top of top. I remember reading something about that being used to cache file descriptors before the files are mapped into memory, but I'm not very clear on what is actually happening. -- Jim C. Nasby, Database Consultant jim@nasby.net Member: Triangle Fraternity, Sports Car Club of America Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to go today?" Linux: "Where do you want to go tomorrow?" FreeBSD: "Are you guys coming, or what?" From owner-freebsd-performance@FreeBSD.ORG Fri Apr 16 15:19:55 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1A8EB16A4CE for ; Fri, 16 Apr 2004 15:19:55 -0700 (PDT) Received: from f13.mail.ru (f13.mail.ru [194.67.57.43]) by mx1.FreeBSD.org (Postfix) with ESMTP id 917EB43D2D for ; Fri, 16 Apr 2004 15:19:54 -0700 (PDT) (envelope-from shmukler@mail.ru) Received: from mail by f13.mail.ru with local id 1BEbgf-0005hv-00; Sat, 17 Apr 2004 02:19:53 +0400 Received: from [24.184.137.35] by msg.mail.ru with HTTP; Sat, 17 Apr 2004 02:19:53 +0400 From: =?koi8-r?Q?=22?=Igor Shmukler=?koi8-r?Q?=22=20?= To: =?koi8-r?Q?=22?=Jim C.Nasby=?koi8-r?Q?=22=20?= Mime-Version: 1.0 X-Mailer: mPOP Web-Mail 2.19 X-Originating-IP: [24.184.137.35] Date: Sat, 17 Apr 2004 02:19:53 +0400 In-Reply-To: <20040416221211.GM87362@nasby.net> Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 8bit Message-Id: cc: freebsd-performance@freebsd.org Subject: Re[2]: How does disk caching work? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: =?koi8-r?Q?=22?=Igor Shmukler=?koi8-r?Q?=22=20?= List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Apr 2004 22:19:55 -0000 > > > Is there a document anywhere that describes in detail how FreeBSD > > > handles disk caching? I've read Matt Dillon's description of the VM > > > system, but it deals mostly with programs, other than vague statements > > > such as 'FreeBSD uses all available memory for disk caching'. > > > > Well, the statement is not vague. FreeBSD has a unified buffer cache. This means that ALL AVAILABLE > > MEMORY IS A BUFFER CACHE for all device IO. > > > > > I think I know how caching memory mapped IO works for the most part, > > > since it should be treated just like program data, but what about files > > > that aren't memory mapped? What impact is there as pages move from > > > active to inactive to cache to free? What role do wired and buffer pages > > > play? > > > > If file is not memory mapped it is not in memory, is it? Where do you cache it? Maybe I am missing > > somewhing? Do you maybe want to know about node caching? > > What if the file isn't memory mapped? You can access a file without > mapping it into memory, right? Read/write/execute require mmap as far as I know. What type of access do you want to perform without the mmap? Not that you asked, but I strongly suggest reading AST's book than Design of 44BSD. > > When pages are rotated from active to inactive and then to cache buckets they is still retains vnode > > references. Once it is in free queue, there is no way to put it back to cache. Association is lost. > > > > Wired pages are to pin memory. So that we do not get situation when fault handling code is paged out. > > > > I am not FreeBSD guru so I never heard of BUFFER pages. Is there such a concept? > > I'm reffering to the 'Buf' column at the top of top. I remember reading > something about that being used to cache file descriptors before the > files are mapped into memory, but I'm not very clear on what is actually > happening. Ok, I thought that there was an unknown concept of buffer pages. :) From owner-freebsd-performance@FreeBSD.ORG Fri Apr 16 15:48:21 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C719016A4CE for ; Fri, 16 Apr 2004 15:48:21 -0700 (PDT) Received: from katmai.eltopia.com (katmai.eltopia.com [64.146.186.25]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4886F43D48 for ; Fri, 16 Apr 2004 15:48:21 -0700 (PDT) (envelope-from aseelye-lists@eltopia.com) Received: from metallus (unverified [68.116.17.36]) by katmai.eltopia.com (Vircom SMTPRS 3.0.286) with SMTP id ; Fri, 16 Apr 2004 15:48:20 -0700 Message-ID: <002701c42404$e9dbecf0$3102a8c0@metallus> From: "Aaron Seelye" To: "Jim C. Nasby" , References: <20040416220556.GL87362@nasby.net> Date: Fri, 16 Apr 2004 15:48:19 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2720.3000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2739.300 Subject: Re: command piped into bzip not using all available CPU X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Apr 2004 22:48:21 -0000 I would venture a guess that bzip is not multi threaded and therefore isn't spreading the load around. -Aaron Seelye ----- Original Message ----- From: "Jim C. Nasby" To: Sent: Friday, April 16, 2004 3:05 PM Subject: command piped into bzip not using all available CPU As you can see below, a command piped into bzip2 is only effectively using one CPU. It's not disk bound, both systat and gstat report less than 10% disk utilization. Why is this? The command I'm running is: pg_dump -vZ0 ogr | bzip2 > ogr-20040416.sql.bz2 last pid: 18345; load averages: 1.17, 1.09, 0.81 up 8+22:12:27 17:00:56 66 processes: 2 running, 64 sleeping CPU states: 49.4% user, 0.0% nice, 3.7% system, 0.2% interrupt, 46.7% idle Mem: 67M Active, 2935M Inact, 359M Wired, 331M Cache, 255M Buf, 5576K Free Swap: 8192M Total, 64M Used, 8127M Free, 48K Out PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND 17334 decibel 109 0 10856K 7164K CPU0 0 11:05 65.77% 65.77% bzip2 17335 pgsql 4 0 154M 124M sbwait 0 5:54 34.03% 34.03% postgres 17333 decibel -8 0 20128K 3236K pipdwt 0 0:46 2.88% 2.88% pg_dump -- Jim C. Nasby, Database Consultant jim@nasby.net Member: Triangle Fraternity, Sports Car Club of America Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to go today?" Linux: "Where do you want to go tomorrow?" FreeBSD: "Are you guys coming, or what?" _______________________________________________ freebsd-performance@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-performance To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" From owner-freebsd-performance@FreeBSD.ORG Fri Apr 16 16:17:58 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 67AD416A4CE for ; Fri, 16 Apr 2004 16:17:58 -0700 (PDT) Received: from cmsrelay03.mx.net (cmsrelay03.mx.net [165.212.11.112]) by mx1.FreeBSD.org (Postfix) with SMTP id 13D7A43D49 for ; Fri, 16 Apr 2004 16:17:58 -0700 (PDT) (envelope-from noackjr@alumni.rice.edu) Received: from uadvg131.cms.usa.net (165.212.11.131) by cmsoutbound.mx.net with SMTP; 16 Apr 2004 23:17:57 -0000 Received: from optimator.noacks.org [66.139.21.136] by uadvg131.cms.usa.net (ASMTP/noackjr@usa.net) via mtad (C8.MAIN.3.13N) with ESMTP id 003iDPXR20313M31; Fri, 16 Apr 2004 23:17:53 GMT X-USANET-Auth: 66.139.21.136 AUTH noackjr@usa.net optimator.noacks.org Received: from localhost (localhost [127.0.0.1]) by optimator.noacks.org (Postfix) with ESMTP id 016E76163; Fri, 16 Apr 2004 18:17:51 -0500 (CDT) Received: from optimator.noacks.org ([127.0.0.1]) by localhost (optimator.noacks.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 83604-03; Fri, 16 Apr 2004 18:17:50 -0500 (CDT) Received: from alumni.rice.edu (optimator [192.168.1.11]) by optimator.noacks.org (Postfix) with ESMTP id 8E3F66165; Fri, 16 Apr 2004 18:17:50 -0500 (CDT) Message-ID: <4080699E.9060307@alumni.rice.edu> Date: Fri, 16 Apr 2004 18:17:50 -0500 From: Jon Noack User-Agent: Mozilla Thunderbird 0.5+ (Windows/20040312) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Aaron Seelye References: <20040416220556.GL87362@nasby.net> <002701c42404$e9dbecf0$3102a8c0@metallus> In-Reply-To: <002701c42404$e9dbecf0$3102a8c0@metallus> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at noacks.org cc: freebsd-performance@freebsd.org cc: "Jim C. Nasby" Subject: Re: command piped into bzip not using all available CPU X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: noackjr@alumni.rice.edu List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Apr 2004 23:17:58 -0000 On 4/16/2004 5:48 PM, Aaron Seelye wrote: > I would venture a guess that bzip is not multi threaded and therefore > isn't spreading the load around. I think you may be right. I tested this on my dual Pentium III 933 running 5.2.1-RELEASE-p5. bzip2 behaves like this while gzip behaves as expected. I know top sucks for measuring performance, but it should be accurate enough for the big picture. Here's the simple test with bzip2 (the rand() is to give perl "work"): perl -e 'do{print rand();}while 1' | bzip2 > /dev/null ********************************************************************** last pid: 83786; load averages: 0.53, 0.31, 0.19 up 4+03:06:50 17:54:52 92 processes: 2 running, 90 sleeping CPU states: 52.0% user, 0.0% nice, 11.1% system, 0.4% interrupt, 36.5% idle Mem: 222M Active, 154M Inact, 99M Wired, 22M Cache, 60M Buf, 2140K Free Swap: 1024M Total, 176K Used, 1024M Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND 83784 noackjr 106 0 8804K 7280K CPU0 0 0:22 60.19% 49.76% bzip2 83783 noackjr -8 0 2756K 1908K pipdwt 0 0:14 38.87% 32.13% perl ********************************************************************** This time with gzip (with -9 to make gzip work harder): perl -e 'do{print rand();}while 1' | gzip -9 > /dev/null ********************************************************************** last pid: 83865; load averages: 1.11, 0.41, 0.28 up 4+03:18:11 18:06:13 91 processes: 4 running, 87 sleeping CPU states: 90.7% user, 0.0% nice, 1.6% system, 0.0% interrupt, 7.7% idle Mem: 215M Active, 155M Inact, 99M Wired, 19M Cache, 60M Buf, 10M Free Swap: 1024M Total, 176K Used, 1024M Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND 83864 noackjr 117 0 2756K 1908K CPU0 0 0:32 88.37% 73.05% perl 83865 noackjr 116 0 1696K 960K RUN 0 0:30 84.65% 69.97% gzip ********************************************************************** On a moderately-related note, -CURRENT recently got some pipe updates (up to 75% bandwidth increase on i386): http://lists.freebsd.org/pipermail/cvs-src/2004-March/021197.html Jon Noack From owner-freebsd-performance@FreeBSD.ORG Sat Apr 17 00:41:24 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 70F6F16A4CE for ; Sat, 17 Apr 2004 00:41:24 -0700 (PDT) Received: from gen129.n001.c02.escapebox.net (gen129.n001.c02.escapebox.net [213.73.91.129]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9D3E243D48 for ; Sat, 17 Apr 2004 00:41:23 -0700 (PDT) (envelope-from gemini@geminix.org) Message-ID: <4080DF9F.3040302@geminix.org> Date: Sat, 17 Apr 2004 09:41:19 +0200 From: Uwe Doering Organization: Private UNIX Site User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.6) Gecko/20040119 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-performance@freebsd.org References: <20040416163845.GG87362@nasby.net> <20040416221211.GM87362@nasby.net> In-Reply-To: <20040416221211.GM87362@nasby.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Received: from gemini by geminix.org with asmtp (TLSv1:AES256-SHA:256) (Exim 3.36 #1) id 1BEkS1-0003F7-00; Sat, 17 Apr 2004 09:41:22 +0200 Subject: Re: How does disk caching work? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Apr 2004 07:41:24 -0000 Jim C. Nasby wrote: > On Sat, Apr 17, 2004 at 01:56:55AM +0400, "Igor Shmukler" wrote: > >>>Is there a document anywhere that describes in detail how FreeBSD >>>handles disk caching? I've read Matt Dillon's description of the VM >>>system, but it deals mostly with programs, other than vague statements >>>such as 'FreeBSD uses all available memory for disk caching'. >> >>Well, the statement is not vague. FreeBSD has a unified buffer cache. This means that ALL AVAILABLE >>MEMORY IS A BUFFER CACHE for all device IO. >> >>>I think I know how caching memory mapped IO works for the most part, >>>since it should be treated just like program data, but what about files >>>that aren't memory mapped? What impact is there as pages move from >>>active to inactive to cache to free? What role do wired and buffer pages >>>play? >> >>If file is not memory mapped it is not in memory, is it? Where do you cache it? Maybe I am missing >>somewhing? Do you maybe want to know about node caching? > > What if the file isn't memory mapped? You can access a file without > mapping it into memory, right? In FreeBSD, file and directory data always exists as VM objects, that is, a collection of virtual memory pages. Those that have been accessed exist in physical memory (if not recycled due to inactivity), the rest is just reservations. That's why it is called "virtual memory". Whether these objects get accessed by read()/write() or mmap() depends on your application. These system calls are just different userland interfaces to the same kernel resource. >>When pages are rotated from active to inactive and then to cache buckets they is still retains vnode >>references. Once it is in free queue, there is no way to put it back to cache. Association is lost. >> >>Wired pages are to pin memory. So that we do not get situation when fault handling code is paged out. >> >>I am not FreeBSD guru so I never heard of BUFFER pages. Is there such a concept? > > I'm reffering to the 'Buf' column at the top of top. I remember reading > something about that being used to cache file descriptors before the > files are mapped into memory, but I'm not very clear on what is actually > happening. The disk i/o buffers you refer to (the 'Buf' column in 'top') are the actual interface between the VM system and the disk device drivers. For file and directory data, sets of VM pages get referred by and assigned to disk i/o buffers. There they are dealt with by a kernel daemon process that does the actual synchronization between VM and disks. That's where the soft updates algorithm is implemented, for instance. In case of file and directory data, once the data has been written out to disk (if the memory pages were "dirty") the respective disk i/o buffer gets released immediately and can be recycled for other purposes, since it just referred to memory pages that continue to exist within the VM system. Meta data (inodes etc.) is a different matter, though. There is no VM representation for this, so for disk i/o they have to be cached in extra memory allocated for this purpose. A disk i/o buffer then refers to this memory range and tries to keep it around for as long as possible. A classical cache algorithm like LRU recycles these buffers and memory allocations eventually. As usual, the actual implementation is even more complex, but I think you got a picture of how it works. Uwe -- Uwe Doering | EscapeBox - Managed On-Demand UNIX Servers gemini@geminix.org | http://www.escapebox.net