From owner-freebsd-questions@FreeBSD.ORG Wed Mar 8 03:03:32 2006 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 03BC516A420 for ; Wed, 8 Mar 2006 03:03:32 +0000 (GMT) (envelope-from beastie@mra.co.id) Received: from mx3.mra.co.id (mx3.mra.co.id [202.51.30.78]) by mx1.FreeBSD.org (Postfix) with ESMTP id 68F9643D46 for ; Wed, 8 Mar 2006 03:03:25 +0000 (GMT) (envelope-from beastie@mra.co.id) Received: from localhost (localhost.mra.co.id [127.0.0.1]) by mx3.mra.co.id (Postfix) with ESMTP id 63CE430FB9; Wed, 8 Mar 2006 09:20:17 +0700 (WIT) Received: from mx3.mra.co.id ([127.0.0.1]) by localhost (mx3.mra.co.id [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 24004-25; Wed, 8 Mar 2006 09:20:16 +0700 (WIT) Received: from mailbox.mra.co.id (unknown [172.16.0.225]) by mx3.mra.co.id (Postfix) with ESMTP id EBC0830FAE; Wed, 8 Mar 2006 09:20:16 +0700 (WIT) Received: from [172.16.0.228] (unknown [172.16.0.228]) by mailbox.mra.co.id (Postfix) with ESMTP id 496C6FDC2; Wed, 8 Mar 2006 09:20:24 +0700 (WIT) Message-ID: <440E45D1.5050803@mra.co.id> Date: Wed, 08 Mar 2006 09:47:45 +0700 From: Beastie User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20051229 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Nikolas Britton References: <61560.207.70.139.52.1139628926.squirrel@www.compedgeracing.com> <44058D9E.3010801@dial.pipex.com> <440675E0.1020204@mra.co.id> <4406CB4D.5050300@dial.pipex.com> <44072515.6080105@dial.pipex.com> <44082439.6070101@dial.pipex.com> <440BA5CA.2070202@mra.co.id> In-Reply-To: Content-Type: multipart/mixed; boundary="------------070603020501090404020407" X-Virus-Scanned: by amavisd-new at mra.co.id X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Liste FreeBSD , Alex Zbyslaw Subject: Re: SATA Raid (stress test..) X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Mar 2006 03:03:32 -0000 This is a multi-part message in MIME format. --------------070603020501090404020407 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Nikolas Britton wrote: >On 3/5/06, Beastie wrote: > > >>Nikolas Britton wrote: >>On 3/3/06, Alex Zbyslaw wrote: >> >> > > > >>Nikolas Britton wrote: >> >> > > > > >>Please can you be careful when you attribute your comments. You've sent >> >> >this > > >>email "to" me, and left only my name in the attributions as if I >> >> >were > > >>someone suggesting either dd or diskinfo as accurate benchmarks, >> >> >when in > > >>fact my contribution was to suggest unixbench and sandra-lite. >> >> >Maybe you > > >>hate those too, in which case you can quote what I said >> >> >in-context and > > >>rubbish that at your pleasure. >> >> > > > > > >>Yes I see your point, it does look like I'm replying to something you >> >> >wrote. > > >>This was a oversight and I am sorry. >> >> > > > > > >>OK. >> >> > > > > >>Remember that 105MB/s number I quoted above?, that's just the >> >> >sustained read > > >>transfer rate for a big ass file, I don't need to work >> >> >with big ass files. I > > >>need to work with 15MB files (+/- 5MB). After >> >> >buying the right disks, > > >>controller, mainboard etc. and lots of tuning >> >> >with the help of iozone I get: > > >>200 - 350MB/s overall (read, write, >> >> >etc.) for files less then or equal to > > >>64MB*. >> >> > >So anyways, that's what iozone can do for you. google it and > > >>you'll >> >> >find out more stuff about it. > > > > > >>Thanks for the info. I think I can only dream about numbers like like >> >> >yours. > > >>Iozone looks to be in the ports so I see some of my weekend >> >> >disappearing > > >>looking at it :-) >> >> > > > > >>It runs on over two dozen operating systems, including windows. Their are >>two primary reasons I can get such high transfer rates from simple SATA >>drives. The first one was the selection of the mainboard that had a PCI-X >>slots, I built this system before PCI-Express mainboards and controllers hit >>the market. The PCI bus is severely restricted and obsolete, I'm simply >>going to post the theoretical maximum throughput in MB/s for the various bus >>standards: f(x,y) = x-bits * y-MHz / 8 = maximum theoretical throughput in >>MB/s PCI: 32 bits * 33 Mhz / 8 = 132 MB/s (standard PCI bus found on every >>pc) PCI: (32bits, 66MHz) = 264MB/s (Cards are commonplace, mainboards >>aren't) PCI-X: (64, 33) = 264MB/s (obsolete, won't find it on new boards.) >>PCI-X: (64, 66) = 528MB/s (Commonplace.) PCI-X: (64, 100) = 800 PCI-X: (64, >>133) = 1064 (Commonplace.) PCI-X: (64, 266) = 2128 PCI-X: (64, 533) = 4264 >>(very hard to find, even on high-end equipment.) PCI-X version 1 (66MHz - >>133MHz) and PCI-X version 2 (266MHz - 533MHz). PCI-X is backwards compatible >>with PCI and slower versions of PCI-X, for example you can put a standard >>PCI card in a PCI-X 533MHz slot and it will simply run at (32, 33) similarly >>a 66 MHz PCI card will run at (32, 66) and so on and so forth. PCI-X is also >>forwards compatible in the fact that you can run a 133MHz PCI-X card in a >>standard (32, 33) PCI slot. Because of the backwards and an forwards >>compatibly I feel that PCI-X is superior to PCI-Express, *BUT* PCI-Express >>moving forwards is far far superior to PCI & PCI-X because it does not have >>13 years of legacy to remain compatible with, it's cheaper to produce, and >>it's already in lower-end desktop systems as a replacement for AGP thanks to >>all the gamers. A few years from now PCI will end up where ISA / EISA are. >>I'm veering way off topic so I will not go into anymore details about PCI, >>PCI-X, and PCI-Express. Google around for the shortcomings of PCI / PCI-X >>and why PCI-Express is the future. PCI-Express: PCIe is not compatible with >>PCI or PCI-X (except for PCIe to PCI bridging) and it's just, well, totally >>different from the PCI spec and I'm already way off topic so again just >>google the details. It's theoretical maximums are expressed in Gigabits per >>second but I will convert them to MB/s for comparison with PCI and PCI-X. >>x1: 2.5Gbps = 312.5MB/s x2: 625MB/s x4: 1250MB/s x8: 2500MB/s x12: 3750MB/s >>x16: 5000MB/s x32: 10000MB/s Anyways back on topic, what was the topic? Oh >>yes, why you won't see 200MB/s - 350MB/s if your using a standard PCI slot. >>If you look back up all the way at the top you will see that the standard >>PCI bus is a crap shoot and that it's limited to a theoretical maximum of >>132 MB/s. What this means is that your RAID controller and the disks >>attached to it and the cache buffers attached to the disks are all capped at >>that theoretical maximum of 132MB/s. Then you have to take into account that >>the PCI bus is shared with other devices such as the network card, video >>card, USB, etc. Your RAID controller has to fight will all these devices and >>a 1Gbit NIC card can eat up 125MB/s (12.5MB/s for a 100Mbit NIC). The next >>reason for those high gains is because I picked drives with 16MB cache >>buffers and that I'm insane enough to run a production server with the >>write-back cache policy enabled on the array controller and enabling the >>write cache on the disks. This is stupidly insane unless you've planned for >>the worsts. The worst case scenario would be that you corrupt the array into >>an unrepairable state and loose everything if you had a power failure. -- >>BSD Podcasts @ http://bsdtalk.blogspot.com/ >>_______________________________________________ >>freebsd-questions@freebsd.org mailing list >>http://lists.freebsd.org/mailman/listinfo/freebsd-questions >>To unsubscribe, send any mail to >>"freebsd-questions-unsubscribe@freebsd.org" >>attach iozone result of amrd0 with 4 spindle Seagate Baracuda 300 Gb SATA II >>(1 hotspare) >>w/ Intel SRCS16 PCI-X >>Is that fast or what ? :) >> >> >> > >I'll have to take a closer look, but the first thing I noticed in your >test report is that you are only using a 1MB test file. You should run >a test that will also max out the on disk / controller buffers. I >think the Baracuda's have a 16MB buffers (16MBx4=64MB) so try a 128MB >test file. Also be nice to see more detailed hardware specs about the >system and what version of FreeBSD are you running. > >Thanks. > >-- >BSD Podcasts @ http://bsdtalk.blogspot.com/ > > > . Second test with 128MB buffers (attach) on - SATA II Seagate Baracuda - PCI-X Intel SRCS16 - Intel Xeon 3.0 with 2 GB DDR RAM - and Intel SE7320EP2 board - FreeBSD-6.1 Pre-RELEASE Thanks before for good review and explanation. I need to be sure that there is no performance issue before i put this machine into production. regards reza --------------070603020501090404020407--