From owner-freebsd-fs@FreeBSD.ORG Sat Jan 20 18:03:59 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7D2CC16A402 for ; Sat, 20 Jan 2007 18:03:59 +0000 (UTC) (envelope-from ahnjoan@gmail.com) Received: from nz-out-0506.google.com (nz-out-0506.google.com [64.233.162.225]) by mx1.freebsd.org (Postfix) with ESMTP id 4122713C43E for ; Sat, 20 Jan 2007 18:03:59 +0000 (UTC) (envelope-from ahnjoan@gmail.com) Received: by nz-out-0506.google.com with SMTP id i11so227966nzh for ; Sat, 20 Jan 2007 10:03:58 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=F8Il1Os8VCOtGxhiNwu6FlvaYPpvLHEl0o6F65WwZawWvOWaZAoQF0EDYiopKdhE1nD+ug+uE8vY6i3WQH3LhLh3j3HVGDIqGv0Mbk521FnnG7S3QPPuJKcEwB+G4gU0GKgRGveVqBp8Wo/AKtxLN9K/81QefJ2nQA8w42Hh0rc= Received: by 10.65.237.19 with SMTP id o19mr4961495qbr.1169316238459; Sat, 20 Jan 2007 10:03:58 -0800 (PST) Received: by 10.65.181.20 with HTTP; Sat, 20 Jan 2007 10:03:58 -0800 (PST) Message-ID: <5e575c8a0701201003j4de54712h6913f0edd62c4ffd@mail.gmail.com> Date: Sat, 20 Jan 2007 13:03:58 -0500 From: "Ahnjoan Amous" To: "Scott Long" In-Reply-To: <5e575c8a0701130703y6c528cecoed9019472a4956c8@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=WINDOWS-1252; format=flowed Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <5e575c8a0701111155l859d0ecif617dbda43cef842@mail.gmail.com> <45A6A138.8090208@samsco.org> <5e575c8a0701111307s5a839b82ra4ba5c45d554d3b9@mail.gmail.com> <45A7551C.5030006@samsco.org> <5e575c8a0701130703y6c528cecoed9019472a4956c8@mail.gmail.com> Cc: freebsd-fs@freebsd.org Subject: Re: aac scsi raid driver performance X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jan 2007 18:03:59 -0000 Thanks again for all the responses. I have been able to run my tests on the hardware of another customer with significantly different results. (50MB/s writes to hardware RAID, 50MB/s writes to GEOM RAID, 65MB/s to no RAID) The only difference in hardware between the two customers are SCSI disks and CPU speed. (The CPUs in the hardware with significantly better performing write speeds are 3.0G 512L2 1024L3 where the CPUs in the hardware with 20MB/s writes are 2.4G 512L2 (no L3). I'm borrowing a couple of CPUs to test in the slower writing hardware, if that doesn't resolve the problems I'll look in to obtaining alternate SCSI drives for further testing. If anyone has any other suggestions I'd be happy to hear them! I'll post any updates to this thread=85 On 1/13/07, Ahnjoan Amous wrote: > Scott - I disabled read cache on both containers and the end result > for my write tests is still 25M/sec. I have included the output to > aaccli to confirm that I disabled the correct cache. Maybe I'm just > expecting too much from the PE2650 with PERC 3/di. I know there are a > lot of variables in this equation but what speeds would you expect for > this U160 scsi raid controller front ending U320 devices on a box with > no load and no i/o, other than two concurrent dd's to two different > ufs mounted hardware raid 0 volumes? > > Thanks > Ahnjoan > > AAC0> container show cache 2 > Read Cache Setting : DISABLE > Write Cache Setting : ENABLE WHEN PROTECTED > Write Cache Status : Active, protected > > AAC0> container show cache 3 > Read Cache Setting : DISABLE > Write Cache Setting : ENABLE WHEN PROTECTED > Write Cache Status : Active, protected > > > On 1/12/07, Scott Long wrote: > > Turn off read caching > > > > Ahnjoan Amous wrote: > > > I reset the controller to the defaults for everything before I did th= e > > > installation. > > > > > > Each of the containers were created with the following options. > > > Container Type - RAID 0 > > > Container Label - data03 > > > Container Size - 279.396 GB > > > Chunk Size - 64KB > > > Read Caching (Yes/No) - Y > > > Write Caching : Enable when protected > > > Create RAID 5 via - N/A > > > > > > Then the controller itself has a single cache option > > > Drives Write Cache - Disabled > > > > > > Thanks > > > Ahnjoan > > > > > > On 1/11/07, Scott Long wrote: > > >> Did you enable read caching for the arrays? > > >> > > >> Scott > > >> > > >> > > >> Ahnjoan Amous wrote: > > >> > I'm trying to find possible explanations for slow concurrent write= s > > >> > through the > > >> > aac driver. This machine runs under 1% load and has less than 4 > > >> > transfers per > > >> > second to the drives in question when not being used for testing. > > >> > > > >> > When I attempt sequential "dd"s as follow, the results are better = then > > >> > 70MB/sec. > > >> > dd if=3D/dev/zero of=3D/data02/helloworld bs=3D1m count=3D1000 > > >> > 1048576000 bytes transferred in 13.886718 secs (75509274 bytes/= sec) > > >> > dd if=3D/dev/zero of=3D/data03/helloworld bs=3D1m count=3D1000 > > >> > 1048576000 bytes transferred in 14.011323 secs (74837758 bytes/= sec) > > >> > > > >> > When I attempt concurrent "dd"s as follow, with a 1 second sleep > > >> interval > > >> > between starts, the results are better than 40MB/sec > > >> > dd if=3D/dev/zero of=3D/data02/helloworld bs=3D1m count=3D1000 & > > >> > sleep 1 > > >> > dd if=3D/dev/zero of=3D/data03/helloworld bs=3D1m count=3D1000 & > > >> > 1048576000 bytes transferred in 25.269555 secs (41495626 bytes/= sec) > > >> > 1048576000 bytes transferred in 24.935765 secs (42051086 bytes/= sec) > > >> > > > >> > When I attempt concurrent "dd"s as follow, the results are little > > >> better > > >> > than > > >> > 20MB/sec > > >> > dd if=3D/dev/zero of=3D/data02/helloworld bs=3D1m count=3D1000 & > > >> > dd if=3D/dev/zero of=3D/data03/helloworld bs=3D1m count=3D1000 & > > >> > 1048576000 bytes transferred in 44.963408 secs (23320652 bytes/= sec) > > >> > 1048576000 bytes transferred in 45.010065 secs (23296478 bytes/= sec) > > >> > > > >> > I can't account for what causes the huge difference however the > > >> results are > > >> > reproducible. I've run the tests dozens and dozens of times now, = first > > >> > blaming > > >> > the em driver for my slow ggatec/ggated results, then GEOM for my = slow > > >> > local > > >> > mirroring after eliminating the network, and finally blaming the a= ac > > >> driver > > >> > after removing GEOM from the equation. If anyone has ideas on wha= t > > >> I might > > >> > look at or change or test I would love to hear. > > >> > > > >> > > > >> > ****** Misc. Information ****** > > >> > root:somehost:~ > df -k > > >> > Filesystem 1K-blocks Used Avail Capacity Mounted on > > >> > /dev/aacd2s1e 2026030 1024532 839416 55% /data02 > > >> > /dev/aacd3s1e 2026030 1024532 839416 55% /data03 > > >> > Hardware - > > >> > aac - Dell PERC3/Di U160 > > >> > aacd2 - hardware RAID 0, w/1 U320 300G drive > > >> > aacd3 - hardware RAID 0, w/1 U320 300G drive > > >> > _______________________________________________ > > >> > freebsd-fs@freebsd.org mailing list > > >> > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > > >> > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.o= rg" > > >> > > >> > > > > >