Date: Wed, 05 Mar 2008 16:08:14 +0100 From: Erik Stian Tefre <erik@tefre.com> To: freebsd-stable@freebsd.org Subject: Re: 7.0-Release and 3ware 9550SXU w/BBU - horrible write performance Message-ID: <47CEB75E.7080906@tefre.com> In-Reply-To: <fqm72d$klr$1@ger.gmane.org> References: <915419.88748.qm@web50505.mail.re2.yahoo.com> <47CE816A.6050903@tefre.com> <fqm72d$klr$1@ger.gmane.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Ivan Voras wrote: > Erik Stian Tefre wrote: > >> Results from bonnie++ with default settings on my box (7.0-RELEASE, >> amd64, 480 MB system memory, BBU, write cache enabled, 7 drive raid-5, >> storsave = balance, NCQ on): >> >> Version 1.93d ------Sequential Output------ --Sequential Input- >> --Random- >> Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- >> --Seeks-- >> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP >> /sec %CP >> xxxxxxxxxxxxxxxx 1G 275 83 72440 23 24990 6 644 83 76548 15 >> 460.6 6 > > Was the array busy when you did the test? 72-76 MB MB/s is very slow for > a 7-drive array in RAID-5. FreeBSD's disk IO can be sluggish but I doubt > it would get that slow. On a RAID10 array with 6 drives I can get ~~ 200 > MB/s in both directions (though on a different controller). RAID5 is > basically striped so your setup should on average yield close to 6 times > the single drive performance, something in the area of 350 MB/s. It was completely idle. Changing vfs.read_max to 80 triples the sequential read performance, see bonnie++ output below (run on the same box, nothing changed except vfs.read_max). I tried setting it to 256 as recommended by 3ware, but 80 seems to be the sweet spot. 3ware performance tips for FreeBSD: http://www.3ware.com/KB/article.aspx?id=14852 I have measured similar write performance on 2 other servers with the same controller model and the same number of disks. By the way, the newer 9650SE controllers seem to be a lot faster, I remember getting around 350 MB/s of sequential reads/writes from an idle 16 drive raid-6 array. (Sorry, I have no idle 9650SE to run bonnie++ on at the moment.) bonnie++ with vfs_max=80: Version 1.93d ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP xxxxxxxxxxxxxxxx 1G 285 83 88193 28 26858 6 690 89 218642 59 406.5 5 Latency 44353us 557ms 542ms 90795us 209ms 361ms Version 1.93d ------Sequential Create------ --------Random Create-------- xxxxxxxxxxxxxxxxxxx -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 8204 19 +++++ +++ 22163 36 9925 25 +++++ +++ +++++ +++ Latency 585ms 571us 3844us 257ms 24015us 768us -- Erik
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?47CEB75E.7080906>