From owner-freebsd-performance@FreeBSD.ORG Sun Oct 29 17:26:43 2006 Return-Path: X-Original-To: performance@freebsd.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7173A16A416 for ; Sun, 29 Oct 2006 17:26:43 +0000 (UTC) (envelope-from pete@he.iki.fi) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id C3F7843D53 for ; Sun, 29 Oct 2006 17:26:42 +0000 (GMT) (envelope-from pete@he.iki.fi) Received: from localhost (localhost [127.0.0.1]) by silver.he.iki.fi (Postfix) with ESMTP id 2CC61BBFF; Sun, 29 Oct 2006 19:26:41 +0200 (EET) Received: from silver.he.iki.fi ([127.0.0.1]) by localhost (silver.he.iki.fi [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P5+nJqdhWp5w; Sun, 29 Oct 2006 19:26:36 +0200 (EET) Received: from [IPv6:2001:670:84:0:dca1:af59:3d40:6dbd] (unknown [IPv6:2001:670:84:0:dca1:af59:3d40:6dbd]) by silver.he.iki.fi (Postfix) with ESMTP; Sun, 29 Oct 2006 19:26:36 +0200 (EET) Message-ID: <4544E44D.7090906@he.iki.fi> Date: Sun, 29 Oct 2006 19:26:37 +0200 From: Petri Helenius User-Agent: Thunderbird 1.5.0.7 (Windows/20060909) MIME-Version: 1.0 To: Steve Peterson References: <6.2.3.4.0.20061027180329.020bed68@localhost> <4542D941.2070204@centtech.com> <6.2.3.4.0.20061028124559.02105930@localhost> <4543AD35.30205@he.iki.fi> <6.2.3.4.0.20061029095927.05833580@localhost> In-Reply-To: <6.2.3.4.0.20061029095927.05833580@localhost> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: performance@freebsd.org Subject: Re: gvinum raid5 performance seems slow X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Oct 2006 17:26:43 -0000 Steve Peterson wrote: > Petri -- thanks for the idea. > > I ran 2 dds in parallel; they took roughly twice as long in clock > time, and had about 1/2 the throughput of the single dd. On my system > it doesn't look like how the work is offered to the disk subsystem > matters. This is the thing I did with similar results before I abandoned vinum ... The performance from the same disks using either graid3 or a real hardware raid controller is significantly greater. I think there is something in vinum blocking out parallelism. > I guess the fundamental question is this -- if I have a 4 disk > subsystem that supports an aggregate ~100MB/sec transfer raw to the > underlying disks, is it reasonable to expect a ~5MB/sec transfer rate > for a RAID5 hosted on that subsystem -- a 95% overhead. > In my opinion, no. Pete > Steve > > > At 01:19 PM 10/28/2006, Petri Helenius wrote: > >> According to my understanding vinum does not overlap requests to >> multiple disks when running in raid5 configuration so you're not >> going to achieve good numbers with just "single stream" tests. >> >> Pete >> >> >> Steve Peterson wrote: >>> Eric -- thanks for looking at my issue. Here's a dd reading from >>> one of the disks underlying the array (the others have basically the >>> same performance): >>> >>> $ time dd if=/dev/ad10 of=/dev/null bs=1m count=1000 >>> 1000+0 records in >>> 1000+0 records out >>> 1048576000 bytes transferred in 15.322421 secs (68434095 bytes/sec) >>> 0.008u 0.506s 0:15.33 3.2% 20+2715k 0+0io 0pf+0w >>> >>> and here's a dd reading from the raw gvinum device /dev/gvinum/vol1: >>> >>> $ time dd if=/dev/gvinum/vol1 of=/dev/null bs=1m count=1000 >>> 1000+0 records in >>> 1000+0 records out >>> 1048576000 bytes transferred in 25.870684 secs (40531437 bytes/sec) >>> 0.006u 0.552s 0:25.88 2.1% 23+3145k 0+0io 0pf+0w >>> >>> Is there a way to nondestructively write to the raw disk or gvinum >>> device? >>> >>> For comparison, here's a read against the raw PATA device on the >>> machine: >>> >>> $ time dd if=/dev/ad0 of=/dev/null bs=1m count=1000 >>> 1000+0 records in >>> 1000+0 records out >>> 1048576000 bytes transferred in 26.096070 secs (40181376 bytes/sec) >>> 0.013u 0.538s 0:26.10 2.0% 24+3322k 0+0io 0pf+0w >>> >>> Steve >>> >>> >>> At 11:14 PM 10/27/2006, Eric Anderson wrote: >>>> On 10/27/06 18:03, Steve Peterson wrote: >>>>> I recently set up a media server for home use and decided to try >>>>> the gvinum raid5 support to learn about it and see how it >>>>> performs. It seems slower than I'd expect -- a little under >>>>> 6MB/second, with about 50 IOs/drive/second -- and I'm trying to >>>>> understand why. Any assistance/pointers would be appreciated. >>>>> The disk system consists of 4 Seagate NL35 SATA ST3250623NS drives >>>>> connected to a Promise TX4300 (PDC40719) controller, organized as >>>>> a RAID5 volume via gvinum using this configuration: >>>>> drive drive01 device /dev/ad10 >>>>> drive drive02 device /dev/ad6 >>>>> drive drive03 device /dev/ad4 >>>>> drive drive04 device /dev/ad8 >>>>> volume vol1 >>>>> plex org raid5 256k >>>>> sd length 200001m drive drive01 >>>>> sd length 200001m drive drive02 >>>>> sd length 200001m drive drive03 >>>>> sd length 200001m drive drive04 >>>>> dd reports the following performance on a 1G file write to the >>>>> RAID5 hosted volume: >>>>> $ time dd if=/dev/zero of=big.file bs=1m count=1000 >>>>> 1000+0 records in >>>>> 1000+0 records out >>>>> 1048576000 bytes transferred in 179.717742 secs (5834571 bytes/sec) >>>>> 179.76 real 0.02 user 16.60 sys >>>>> By comparison, creating the same file on the system disk (an old >>>>> ATA ST380021A connected via a SIS 730 on the motherboard): >>>>> time dd if=/dev/zero of=big.file bs=1m count=1000 >>>>> 1000+0 records in >>>>> 1000+0 records out >>>>> 1048576000 bytes transferred in 28.264056 secs (37099275 bytes/sec) >>>>> 28.32 real 0.01 user 19.13 sys >>>>> and vmstat reports about 280-300 IOs/second to that drive. >>>>> The CPU is pretty weak -- an Athlon 750. Is that the source of my >>>>> problem? If you look at the vmstat output below the machine is >>>>> busy but not pegged. >>>> >>>> >>>> Try the dd to the raw gvinum device instead of through a >>>> filesystem, and also to the individual disks. That will at least >>>> tell us where to look. >>>> >>>> >>>> Eric >>>> >>>> >>>> >>>> -- >>>> ------------------------------------------------------------------------ >>>> >>>> Eric Anderson Sr. Systems Administrator Centaur >>>> Technology >>>> Anything that works is better than anything that doesn't. >>>> ------------------------------------------------------------------------ >>>> >>>> _______________________________________________ >>>> freebsd-performance@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>>> To unsubscribe, send any mail to >>>> "freebsd-performance-unsubscribe@freebsd.org" >>> >>> >>> _______________________________________________ >>> freebsd-performance@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >>> To unsubscribe, send any mail to >>> "freebsd-performance-unsubscribe@freebsd.org" >> > > >