From owner-freebsd-stable@FreeBSD.ORG Mon Mar 31 19:20:42 2003 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 79A0137B401 for ; Mon, 31 Mar 2003 19:20:42 -0800 (PST) Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by mx1.FreeBSD.org (Postfix) with ESMTP id 758A343FCB for ; Mon, 31 Mar 2003 19:20:40 -0800 (PST) (envelope-from grog@lemis.com) Received: by wantadilla.lemis.com (Postfix, from userid 1004) id 7347451A67; Tue, 1 Apr 2003 12:50:38 +0930 (CST) Date: Tue, 1 Apr 2003 12:50:38 +0930 From: Greg 'groggy' Lehey To: "Michael C. Brenner" Message-ID: <20030401032038.GO34617@wantadilla.lemis.com> References: <3E88AECD.10607@liwing.de> <20030330125138.K23911@leelou.in.tern> <3E870CC7.5000204@mac.com> <20030330175605.E23911@leelou.in.tern> <3E87204C.5060304@ludd.luth.se> <3E88524A.1060600@mitre.org> <3E88AECD.10607@liwing.de> <5.2.0.9.2.20030331214429.02677a90@gw.kaibren.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qSHHer9gQ0dtepKr" Content-Disposition: inline In-Reply-To: <5.2.0.9.2.20030331214429.02677a90@gw.kaibren.com> User-Agent: Mutt/1.4i Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 cc: FreeBSD Stable List Subject: Re: vinum performance X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2003 03:20:42 -0000 --qSHHer9gQ0dtepKr Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Monday, 31 March 2003 at 22:01:25 -0500, Michael C. Brenner wrote: > At 04:41 PM 3/31/2003, Jason Andresen wrote: >>>>>> Ok. But I still don't understand why RAID 5 write performance is _so_ >>>>>> bad. >>>>>> The CPU is not the bottle neck, it's rather bored. And I don't >>>>>> understand >>>>>> why RAID 0 doesn't give a big boost at all. Is the ahc driver known to >>>>>> be >>>>>> slow? >>>>> >>>>> (Both of these were on previously untouched files to prevent any >>>>> caching, and the "write" test is on a new file, not rewriting an old >>>>> one) >> Write speed: >> 81920000 bytes transferred in 3.761307 secs (21779663 bytes/sec) >> Read speed: >> 81920000 bytes transferred in 3.488978 secs (23479655 bytes/sec) >> >> But on the RAID5: >> Write speed: >> 81920000 bytes transferred in 17.651300 secs (4641018 bytes/sec) >> Read speed: >> 81920000 bytes transferred in 4.304083 secs (19033090 bytes/sec) > > Writing to a RAID5 stripe set requires that all disks in the array > successfully report completion before the RAID5 controller's buffer can be > released back to the cache. No, you don't need to involve every disk, only the disk(s) to which the data will be written, and the parity disk. > (Applies to either software or hardware raid.) If you are doing a > large block write (like dd) you can easily fill the cache on most > controllers. You mean drive, right? Most controllers don't have cache. > Once the cache is full, the controller slows each write to the > LONGEST completion time of each spindle in the array. Yes, that's correct. > ECC calculation becomes part of the latency also. Well, yes, but most of the time it's negligible. On modern machines, we're talking in the order of few microseconds. > In a 5 drive system (other than one where the cache is larger than > the largest file being written as in a large EMC array) the writes > are always about 4-5 times longer than the reads. This is pretty much independent of the number of drives: assuming your stripes aren't too small, you'll have two reads, two writes. Yes, they can go in parallel, but while they're doing it, they're blocking other potential transfers. Greg -- See complete headers for address and phone numbers --qSHHer9gQ0dtepKr Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.0 (FreeBSD) iD8DBQE+iQWGIubykFB6QiMRAlsHAJ9rV6mbz62QVBPxt8hiTc0hQzm3RgCfX19+ ZMssAmlchJjaCfJ4ytBb8Ls= =oxsz -----END PGP SIGNATURE----- --qSHHer9gQ0dtepKr--