From owner-freebsd-performance@FreeBSD.ORG Tue May 3 07:06:18 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1118B16A4D1 for ; Tue, 3 May 2005 07:06:18 +0000 (GMT) Received: from smtp812.mail.sc5.yahoo.com (smtp812.mail.sc5.yahoo.com [66.163.170.82]) by mx1.FreeBSD.org (Postfix) with SMTP id 4E11243D6A for ; Tue, 3 May 2005 07:06:17 +0000 (GMT) (envelope-from noackjr@alumni.rice.edu) Received: from unknown (HELO optimator.noacks.org) (noacks@swbell.net@70.240.205.64 with login) by smtp812.mail.sc5.yahoo.com with SMTP; 3 May 2005 07:06:16 -0000 Received: from localhost (localhost [127.0.0.1]) by optimator.noacks.org (Postfix) with ESMTP id 82522614E; Tue, 3 May 2005 02:06:12 -0500 (CDT) Received: from optimator.noacks.org ([127.0.0.1]) by localhost (optimator.noacks.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 00566-12; Tue, 3 May 2005 02:06:07 -0500 (CDT) Received: from compgeek.noacks.org (compgeek [192.168.1.10]) by optimator.noacks.org (Postfix) with ESMTP id 024E060F7; Tue, 3 May 2005 02:06:06 -0500 (CDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) by compgeek.noacks.org (8.13.3/8.13.3) with ESMTP id j4375vZ7001808; Tue, 3 May 2005 02:06:00 -0500 (CDT) (envelope-from noackjr@alumni.rice.edu) Message-ID: <427722D0.8050200@alumni.rice.edu> Date: Tue, 03 May 2005 02:05:52 -0500 From: Jonathan Noack User-Agent: Mozilla Thunderbird 1.0.2 (X11/20050428) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Petri Helenius References: <19879.1115061648@critter.freebsd.dk> <20050502214208.M87351@fledge.watson.org> <42770026.80901@he.iki.fi> In-Reply-To: <42770026.80901@he.iki.fi> X-Enigmail-Version: 0.91.0.0 OpenPGP: id=991D8195; url=http://www.noacks.org/cert/noackjr.asc Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigAAE6D0C55B37F0B412973BCF" X-Virus-Scanned: amavisd-new at noacks.org cc: Steven Hartland cc: Poul-Henning Kamp cc: Robert Watson cc: freebsd-performance@FreeBSD.org cc: Eric Anderson Subject: Re: Very low disk performance on 5.x X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: noackjr@alumni.rice.edu List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 May 2005 07:06:18 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigAAE6D0C55B37F0B412973BCF Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 05/02/05 23:37, Petri Helenius wrote: > Robert Watson wrote: >> The next thing that would be quite nice to measure is the rate of I/O >> transactions per second we can get to the disk using the disk device >> directly, with a minimal transaction size. I have a vague >> recollection that you have to be careful in Linux because their >> character device nodes for disk devices are buffered, and you really >> want unbuffered I/O. > > I noticed that changing vfs.read_max from the default 8 to 16 has a > dramatic effect on sequential read performance. Increasing it further > did not have measurable effect. Wow. Setting vfs.read_max=16 resulted in a 50+% performance boost when sequentially reading from my 4-disk RAID5 (ida(4)) via the filesystem. Write performance was unchanged. Raw device read performance was also unchanged (~40 MB/s). With vfs.read_max>8, reading through the filesystem is *faster* than reading from the raw device! Note that these tests were run in multi-user on an idle 5.4-RC4 machine with 1 GB of RAM and quad P3 Xeons. The machine is currently using ULE. These tests were repeated 3 times with results in the +-0.1MB/s range. $ sysctl vfs.read_max vfs.read_max: 8 $ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000 100000+0 records in 100000+0 records out 6553600000 bytes transferred in 185.571466 secs (35315774 bytes/sec) 3m5.58s real 0.16s user 1m0.89s sys $ sysctl vfs.read_max=16 vfs.read_max: 8 -> 16 $ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000 100000+0 records in 100000+0 records out 6553600000 bytes transferred in 118.846665 secs (55143323 bytes/sec) 1m58.85s real 0.21s user 1m3.25s sys $ sysctl vfs.read_max=24 vfs.read_max: 16 -> 24 $ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000 100000+0 records in 100000+0 records out 6553600000 bytes transferred in 107.360600 secs (61042878 bytes/sec) 1m47.36s real 0.14s user 1m3.27s sys $ sysctl vfs.read_max=32 vfs.read_max: 24 -> 32 $ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000 100000+0 records in 100000+0 records out 6553600000 bytes transferred in 102.288822 secs (64069562 bytes/sec) 1m42.29s real 0.20s user 1m4.52s sys -- Jonathan Noack | noackjr@alumni.rice.edu | OpenPGP: 0x991D8195 --------------enigAAE6D0C55B37F0B412973BCF Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (FreeBSD) iD8DBQFCdyLVUFz01pkdgZURAlDKAJwP9Q85zVvLODRJrTC25BYo+tdCeQCfYQmu OCQNhEc3+DOVFk90zEmoliw= =WR+k -----END PGP SIGNATURE----- --------------enigAAE6D0C55B37F0B412973BCF--