From owner-freebsd-questions@FreeBSD.ORG Fri Mar 2 15:28:36 2007 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 28C1416A402 for ; Fri, 2 Mar 2007 15:28:36 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (grnl-static-02-0046.dsl.iowatelecom.net [69.66.56.110]) by mx1.freebsd.org (Postfix) with ESMTP id 7E2E213C4A3 for ; Fri, 2 Mar 2007 15:28:35 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.13.8/8.13.8) with ESMTP id l22FSX3b079624; Fri, 2 Mar 2007 09:28:33 -0600 (CST) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.13.8/8.13.8/Submit) id l22FSX0d079623; Fri, 2 Mar 2007 09:28:33 -0600 (CST) (envelope-from brooks) Date: Fri, 2 Mar 2007 09:28:33 -0600 From: Brooks Davis To: "O. Hartmann" Message-ID: <20070302152832.GA79187@lor.one-eyed-alien.net> References: <45E7F09B.7070005@zedat.fu-berlin.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="TB36FDmn/VVEgNH/" Content-Disposition: inline In-Reply-To: <45E7F09B.7070005@zedat.fu-berlin.de> User-Agent: Mutt/1.5.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (lor.one-eyed-alien.net [127.0.0.1]); Fri, 02 Mar 2007 09:28:33 -0600 (CST) Cc: freebsd-performance@freebsd.org, freebsd-questions@freebsd.org Subject: Re: (S)ATA performance in FBSD 6.2/7.0 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2007 15:28:36 -0000 --TB36FDmn/VVEgNH/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Mar 02, 2007 at 10:38:35AM +0100, O. Hartmann wrote: > The last days I tried to figure out why some of my lab's FreeBSD boxes=20 > and also mine at home seem to be outperformed by some Linux setups=20 > around here and I saw something interesting. >=20 > On my lab's FreeBSD 6.2/i386 box (ASUS P4P800, ICH5 with two SATA 150=20 > ports, two SATA 300 drives attached) I copied big files (~ 5GB) from one= =20 > drive to another while the box didn't do anything else than copying. I=20 > watched the copy process via 'systat -vmstat 1' and realized, that the=20 > value of 'KB/t' never go byond 128 (128kb buffer limit?). But more=20 > frustrating, I never got beyond 33 MB/s transfer rate although=20 > bonni/bonni++ told me both drives are capable doing much more (~75 MB/s= =20 > each). > At home, I use a FreeBSD 7.0-CURRENT box on an ASUS=20 > A8N32-SLI/nForce4-SLI based box, amd64 (no 32Bit compatibility). Two=20 > Hitachi T7K250 250 GB/SATA II drives build up a RAID 0 (nVidia=20 > MediaShield), and additionally there is a SAMSUNG Spinpoitn SP2004C=20 > attached to the controller. bonni results in 55 MB/s for the SP2004C=20 > alone and gives ~ 65 - 70 MB/s for the Hitachis, each and roughly 115=20 > MB/s for the RAID 0. But copying from the single drive to the RAID 0 or= =20 > from the RAID 0 to the single drive also reaches this oscure 33 MB/s=20 > boundary! >=20 > In the first place I thought the older i386 hardware has some=20 > hard-limits, but we have several boxes of the exact same hardware around= =20 > here and a wide spread Linux and Windows utilization and on those boxes= =20 > equipted with more than one harddrive (PATA or SATA) the effective=20 > transfer rate shown up is about 50 - 65 MB/s as expected with copying a= =20 > big 5G file from one drive to another. >=20 > The hardwrae limit is completely nonsense when it comes to the AMD64 box= =20 > with newer hardware. >=20 > Before digging into this problem deeper with benchmarks, could anyone=20 > explain why FreeBSD reaches this 33 MB/s limit (sounds like UDMA 33=20 > defaults, but on both boxes nForce4 and ICH5 controller are recognized=20 > and show up with SATA300 or SATA150 capabilities, respective)? May I=20 > have some knobs I'm not aware of to tune disk performance? >=20 > I would appreciate any coments on that and if someone has some good=20 > ideas how to benchmark those subjects, please let me know. One thing to keep in mind is that it matters a lot were on the disk you place the data due to the higher angular density of data at the outside of the disk. The results you are seeing are close to consistant with the kind of results I'd expect to see from writing at opposiste edges of the disk. The 33MB/s is suspious ane may diserve investigation, but make sure you are writing to the same part of the disk if you want to compare disk IO rates. There's an example of IO rates on recent large SATA disks: http://storagereview.com/articles/200607/500_2.html Also, you should time the actual copy and do the math to verify that vmstat is actually producing valid results. It's possible there's a bug in vmstat or the underlying statistics it uses. -- Brooks --TB36FDmn/VVEgNH/ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFF6EKgXY6L6fI4GtQRArzYAJ4yi9tMIlmHB54psVhyicUGojfGJwCZARVf riI6eItmgtXAXpgi611e1vM= =iKHp -----END PGP SIGNATURE----- --TB36FDmn/VVEgNH/--