From owner-freebsd-geom@FreeBSD.ORG Wed Jan 7 03:37:11 2009 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7FD45106564A for ; Wed, 7 Jan 2009 03:37:11 +0000 (UTC) (envelope-from jjacobson@panasas.com) Received: from laguna.int.panasas.com (gw-ca.panasas.com [66.104.249.162]) by mx1.freebsd.org (Postfix) with ESMTP id 648D38FC17 for ; Wed, 7 Jan 2009 03:37:10 +0000 (UTC) (envelope-from jjacobson@panasas.com) Received: from [172.17.133.249] ([172.17.133.249]) by laguna.int.panasas.com with Microsoft SMTPSVC(6.0.3790.3959); Tue, 6 Jan 2009 19:37:03 -0800 Message-Id: From: Joel Jacobson To: Ivan Voras In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v929.2) Date: Tue, 6 Jan 2009 19:37:07 -0800 References: <4AD370A6-2226-442F-BD80-8CFD4045B094@panasas.com> X-Mailer: Apple Mail (2.929.2) X-OriginalArrivalTime: 07 Jan 2009 03:37:03.0670 (UTC) FILETIME=[34D81560:01C97079] Cc: freebsd-geom@freebsd.org Subject: Re: performance problem with gstripe X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2009 03:37:11 -0000 here's what i did: --------------------------- ca-sbox-2# foreach i (0 1) foreach? dd if=/dev/zero of=/dev/da$i bs=512k count=1024 & foreach? end [1] 5402 [2] 5403 ca-sbox-2# 1024+0 records in 1024+0 records out 536870912 bytes transferred in 4.262723 secs (125945532 bytes/sec) 1024+0 records in 1024+0 records out 536870912 bytes transferred in 4.272499 secs (125657357 bytes/sec) ca-sbox-2# gstripe create -s 262144 d0 /dev/da{0,1} ca-sbox-2# dd if=/dev/zero of=/dev/stripe/d0 bs=512k count=4096 4096+0 records in 4096+0 records out 2147483648 bytes transferred in 34.124683 secs (62930508 bytes/sec) ca-sbox-2# newfs /dev/stripe/d0 > /dev/null ca-sbox-2# mount /dev/stripe/d0 /mnt ca-sbox-2# dd if=/dev/zero of=/mnt/bigfile bs=512k count=4096 && /usr/ bin/time sync 4096+0 records in 4096+0 records out 2147483648 bytes transferred in 11.081184 secs (193795502 bytes/sec) 0.06 real 0.00 user 0.04 sys # sysctl kern.geom kern.geom.collectstats: 1 kern.geom.debugflags: 0 kern.geom.label.debug: 0 kern.geom.stripe.fast_failed: 0 kern.geom.stripe.maxmem: 13107200 kern.geom.stripe.fast: 1 kern.geom.stripe.debug: 0 ---------------------- - j On Jan 6, 2009, at 7:05 PM, Ivan Voras wrote: > Joel Jacobson wrote: >> i have a bit of a weird issue, which i suspect is a configuration >> problem, and was looking for a little advice. i have an LSI JBOD box >> with a bunch of SAS drives that i would like to gstripe together. >> each >> drive individually seems to be able to do about 80 MB/sec streaming >> write, and doing parallel dd's gives me the 160 MB/sec i would >> expect. >> if i gstripe them together with a 256k stripe width, i only see 80 >> MB/sec, though. > > How do you measure this? If with dd, what block size (bs) do you use? > >> >> if, however, i newfs/mount it as ufs and then dd myself a big file, >> that >> gets me about 120-130 MB/sec. >> >> why does mounting matter? > > I'd guess because of write caching that enables you to make use of > multiple stripes at once, if your dd bs is smaller than 2*stripe size. >