From owner-freebsd-performance@FreeBSD.ORG Wed Sep 3 12:37:52 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4285F16A4BF for ; Wed, 3 Sep 2003 12:37:52 -0700 (PDT) Received: from pop017.verizon.net (pop017pub.verizon.net [206.46.170.210]) by mx1.FreeBSD.org (Postfix) with ESMTP id DE08E43FBD for ; Wed, 3 Sep 2003 12:37:50 -0700 (PDT) (envelope-from cswiger@mac.com) Received: from mac.com ([68.237.14.199]) by pop017.verizon.net (InterMail vM.5.01.05.33 201-253-122-126-133-20030313) with ESMTP id <20030903193750.BTCA27671.pop017.verizon.net@mac.com> for ; Wed, 3 Sep 2003 14:37:50 -0500 Message-ID: <3F5642FF.6060702@mac.com> Date: Wed, 03 Sep 2003 15:37:35 -0400 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 Cc: freebsd-performance@freebsd.org References: <3F5634FE.9080303@mac.com> <20030903184506.GC14797@sgh.waw.pl> In-Reply-To: <20030903184506.GC14797@sgh.waw.pl> X-Enigmail-Version: 0.76.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH at pop017.verizon.net from [68.237.14.199] at Wed, 3 Sep 2003 14:37:49 -0500 Subject: Re: 20TB Storage System (fsck????) X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Sep 2003 19:37:52 -0000 Piotr KUCHARSKI wrote: > On Wed, Sep 03, 2003 at 02:37:50PM -0400, Chuck Swiger wrote: >>I'm wary of the write-performance hit from putting too many drives wide >>in a RAID-5 (or -5,0) configuration. > > How many is "too many"? At one point, the advice used to be to use between four and seven disks for a RAID-5 volume. For example, the Apple XServe RAID box has 14 bays, but Apple seems to recommend configuring it as two 7-drive RAID-5 volumes, rather than a single 14-drive-wide RAID-5 volume. > Or, rather, what are write-performance penalties > when using sixteen disks in one hw raid5 set? (With two raid volumes, > 2TB and 1.75TB available for OS.) Find yourself a buncha small files-- a CVS repository, or /usr/ports will do, and compare write performance to a single drive versus RAID-5. Basicly, you get all of the drives in the RAID-set scribbling away at a fraction of the write speed of a single drive, yes? Three disk transactions per write, versus one? Also note that all this disk activity requires three times the I/O bandwidth, interrupts, and assorted overhead. If the OP has hardware RAID which is designed to support a wide array, OK, but setting up a too-wide a RAID-5 array means that things like the system bus may bottleneck performance, rather than the drives. Normally, disk I/O speed is the limiting factor, and your bus and memory are sitting around waiting for the DMA to complete (well, being used by the CPU to run other processes). Let's put it this way, things don't go faster when the drives are waiting for the bus to become available, rather than vice-versa. :-) -- -Chuck