From owner-freebsd-questions@FreeBSD.ORG Fri Mar 2 18:51:01 2007 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8F46416A405; Fri, 2 Mar 2007 18:51:01 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from mh1.centtech.com (moat3.centtech.com [64.129.166.50]) by mx1.freebsd.org (Postfix) with ESMTP id 6BA4913C49D; Fri, 2 Mar 2007 18:51:01 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.8/8.13.8) with ESMTP id l22IowaM063935; Fri, 2 Mar 2007 12:50:58 -0600 (CST) (envelope-from anderson@freebsd.org) Message-ID: <45E87212.1010208@freebsd.org> Date: Fri, 02 Mar 2007 12:50:58 -0600 From: Eric Anderson User-Agent: Thunderbird 1.5.0.9 (X11/20070204) MIME-Version: 1.0 To: Brooks Davis References: <45E7F09B.7070005@zedat.fu-berlin.de> <20070302152832.GA79187@lor.one-eyed-alien.net> In-Reply-To: <20070302152832.GA79187@lor.one-eyed-alien.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.88.4/2702/Fri Mar 2 09:04:51 2007 on mh1.centtech.com X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=8.0 tests=BAYES_00 autolearn=ham version=3.1.6 X-Spam-Checker-Version: SpamAssassin 3.1.6 (2006-10-03) on mh1.centtech.com Cc: freebsd-performance@freebsd.org, "O. Hartmann" , freebsd-questions@freebsd.org Subject: Re: (S)ATA performance in FBSD 6.2/7.0 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2007 18:51:01 -0000 On 03/02/07 09:28, Brooks Davis wrote: > On Fri, Mar 02, 2007 at 10:38:35AM +0100, O. Hartmann wrote: >> The last days I tried to figure out why some of my lab's FreeBSD boxes >> and also mine at home seem to be outperformed by some Linux setups >> around here and I saw something interesting. >> >> On my lab's FreeBSD 6.2/i386 box (ASUS P4P800, ICH5 with two SATA 150 >> ports, two SATA 300 drives attached) I copied big files (~ 5GB) from one >> drive to another while the box didn't do anything else than copying. I >> watched the copy process via 'systat -vmstat 1' and realized, that the >> value of 'KB/t' never go byond 128 (128kb buffer limit?). But more >> frustrating, I never got beyond 33 MB/s transfer rate although >> bonni/bonni++ told me both drives are capable doing much more (~75 MB/s >> each). >> At home, I use a FreeBSD 7.0-CURRENT box on an ASUS >> A8N32-SLI/nForce4-SLI based box, amd64 (no 32Bit compatibility). Two >> Hitachi T7K250 250 GB/SATA II drives build up a RAID 0 (nVidia >> MediaShield), and additionally there is a SAMSUNG Spinpoitn SP2004C >> attached to the controller. bonni results in 55 MB/s for the SP2004C >> alone and gives ~ 65 - 70 MB/s for the Hitachis, each and roughly 115 >> MB/s for the RAID 0. But copying from the single drive to the RAID 0 or >> from the RAID 0 to the single drive also reaches this oscure 33 MB/s >> boundary! >> >> In the first place I thought the older i386 hardware has some >> hard-limits, but we have several boxes of the exact same hardware around >> here and a wide spread Linux and Windows utilization and on those boxes >> equipted with more than one harddrive (PATA or SATA) the effective >> transfer rate shown up is about 50 - 65 MB/s as expected with copying a >> big 5G file from one drive to another. >> >> The hardwrae limit is completely nonsense when it comes to the AMD64 box >> with newer hardware. >> >> Before digging into this problem deeper with benchmarks, could anyone >> explain why FreeBSD reaches this 33 MB/s limit (sounds like UDMA 33 >> defaults, but on both boxes nForce4 and ICH5 controller are recognized >> and show up with SATA300 or SATA150 capabilities, respective)? May I >> have some knobs I'm not aware of to tune disk performance? >> >> I would appreciate any coments on that and if someone has some good >> ideas how to benchmark those subjects, please let me know. > > One thing to keep in mind is that it matters a lot were on the disk you > place the data due to the higher angular density of data at the outside > of the disk. The results you are seeing are close to consistant with > the kind of results I'd expect to see from writing at opposiste edges > of the disk. The 33MB/s is suspious ane may diserve investigation, but > make sure you are writing to the same part of the disk if you want to > compare disk IO rates. > > There's an example of IO rates on recent large SATA disks: > > http://storagereview.com/articles/200607/500_2.html > > Also, you should time the actual copy and do the math to verify that > vmstat is actually producing valid results. It's possible there's a bug > in vmstat or the underlying statistics it uses. I usually use gstat instead, but it might also be off (although my tests in the past have not proven that). Eric