From owner-freebsd-stable@FreeBSD.ORG Mon Mar 31 13:41:29 2003 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 99DA737B404 for ; Mon, 31 Mar 2003 13:41:29 -0800 (PST) Received: from smtpproxy2.mitre.org (smtpproxy2.mitre.org [192.80.55.70]) by mx1.FreeBSD.org (Postfix) with ESMTP id A44BE43F85 for ; Mon, 31 Mar 2003 13:41:28 -0800 (PST) (envelope-from jandrese@mitre.org) Received: from avsrv2.mitre.org (avsrv2.mitre.org [128.29.154.4]) by smtpproxy2.mitre.org (8.12.9/8.12.8) with ESMTP id h2VLfRLS000444 for ; Mon, 31 Mar 2003 16:41:27 -0500 (EST) Received: from MAILHUB1 (mailhub1.mitre.org [129.83.20.31]) by smtpsrv2.mitre.org (8.12.8/8.12.8) with ESMTP id h2VLfPk2017954 for ; Mon, 31 Mar 2003 16:41:26 -0500 (EST) Received: from mm112324-2k.mitre.org (128.29.3.65) by mailhub1.mitre.org with SMTP id 1734895; Mon, 31 Mar 2003 16:41:22 -0500 Message-ID: <3E88B601.90802@mitre.org> Date: Mon, 31 Mar 2003 16:41:21 -0500 From: Jason Andresen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: FreeBSD Stable List References: <20030330125138.K23911@leelou.in.tern> <3E870CC7.5000204@mac.com> <20030330175605.E23911@leelou.in.tern> <3E87204C.5060304@ludd.luth.se> <3E88524A.1060600@mitre.org> <3E88AECD.10607@liwing.de> In-Reply-To: <3E88AECD.10607@liwing.de> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: vinum performance X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2003 21:41:49 -0000 Jens Rehsack wrote: > Jason Andresen wrote: > >> Mattias Pantzare wrote: >> >>> Lukas Ertl wrote: >>> >>>> Ok. But I still don't understand why RAID 5 write performance is >>>> _so_ bad. >>>> The CPU is not the bottle neck, it's rather bored. And I don't >>>> understand >>>> why RAID 0 doesn't give a big boost at all. Is the ahc driver known >>>> to be >>>> slow? >>> >>> >>> >>> >>> To do a RAID 5 write you do this: >>> 1. Read the old data on the blocks that you will write to. >>> 2. Read the coresponding parity data. >>> 3. Write the new data. >>> 4. Write the new parity. >> >> >> >> Hmm, how about the case where you're writing new data? You shouldn't >> have to do steps 1 & 2, and yet the RAID5 write performance is still >> abysmial. > > > Remember for that case that a block covered by the raid-system may be > larger than 512 bytes. I use 32K for my fileserver, so to skip reading > old data I had to write 32K blocks at once. > > Of course, the system software (either vinum or the controller software) > caches a little bit, so if you write enough small data you may get a 32K > block (or whatever you use), full. > >> I get 4565 K/sec on modern ATA/133 HDDs. >> >> Reading is much better at 91908 K/sec at least. >> Well, I'm writing 200MB files most of the time, so the stripe size is not an issue. I'm just wondering why the reads are *20* times faster than the writes. I think the read performance was CPU limited in this case. While some of this is probably an oddity with bonnie (Bonnie always reports my writes to be about half as fast as the reads, but dd thinks otherwise: (Both of these were on previously untouched files to prevent any caching, and the "write" test is on a new file, not rewriting an old one) Write speed: 81920000 bytes transferred in 3.761307 secs (21779663 bytes/sec) Read speed: 81920000 bytes transferred in 3.488978 secs (23479655 bytes/sec) But on the RAID5: Write speed: 81920000 bytes transferred in 17.651300 secs (4641018 bytes/sec) Read speed: 81920000 bytes transferred in 4.304083 secs (19033090 bytes/sec) -- \ |_ _|__ __|_ \ __| Jason Andresen jandrese@mitre.org |\/ | | | / _| Network and Distributed Systems Engineer _| _|___| _| _|_\___| Office: 703-883-7755