From owner-freebsd-isp Tue Mar 7 21:59:20 2000 Delivered-To: freebsd-isp@freebsd.org Received: from saturn.rmci.net (saturn.rmci.net [205.162.184.38]) by hub.freebsd.org (Postfix) with SMTP id 267AB37B59E for ; Tue, 7 Mar 2000 21:59:10 -0800 (PST) (envelope-from webweaver@rmci.net) Received: (qmail 32223 invoked from network); 8 Mar 2000 05:59:04 -0000 Received: from usr-boi-382.rmci.net (HELO hobart) (206.159.112.223) by saturn.rmci.net with SMTP; 8 Mar 2000 05:59:04 -0000 Message-Id: <4.2.0.58.20000307225646.0097adf0@mail.rmci.net> X-Sender: webweaver@mail.rmci.net X-Mailer: QUALCOMM Windows Eudora Pro Version 4.2.0.58 Date: Tue, 07 Mar 2000 23:00:42 -0700 To: Kevin Day , grog@lemis.com (Greg Lehey) From: Ken Subject: Re: Vinum Stripe Size Cc: toasty@dragondata.com (Kevin Day), freebsd-isp@FreeBSD.ORG In-Reply-To: <200003080517.XAA94623@celery.dragondata.com> References: <20000308153114.A52208@freebie.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org At 11:17 PM 3/7/00 -0600, Kevin Day wrote: > > >> Actually, I discovered that with 4 drives, you're much better off > using an > > >> odd stripe size. (not a power of two) This is because of how the > cylinder > > >> groups are laid out, they'll all end up on one drive. > > >> > > >> You may want to ask Greg Lehey (grog@lemis.com) for more info about > this, as > > >> I can't remember exactly what he came up with for an optimum stripe > size. > > > > > > I figured he'd end up seeing this and I didn't want to bug him at > > > his private address;) The docs suggest 256k to 512K. I thing I read > > > somewhere to use 512 with larger drives, but cannot recall precisely > > > where. BTW, this is for a web hosting box. I could probably get by > > > with RAID-1, but figured I might as well go with RAID-10 since I > > > have the drives to spare. > > > > Indeed, Kevin is right. At the FreeBSDCon he showed me some > > interesting results running ufs on striped plexes with different > > stripe sizes. > > > > 2. You want to avoid putting all your superblocks on the same disk. > > Nowadays cylinder groups are almost always 32 MB, so any power of > > 2 stripe size will give rise to this undesirable situation. Kevin > > gave me some interesting input on the effect this has on > > performance, but it's quite a difficult problem. I'm planning to > > provide a program to work out optimal stripe sizes, but it's not > > ready yet. The principle is simple, though: spread the > > superblocks equally across all disks. > > >When I used a stripe size of 256k or 512k, I ended up with about 90% of my >disk accesses going on drive 0, and the remaining 10% spread across the >other three drives. This was mostly because my application created HUGE >directories with tons of very small files, I'm guessing. > >After playing with things for far too long, I ended up with some very very >unusual number, that seemed to work best... It probably works for the wrong >reasons, but a co-worker suggested that I try a (prime number * 512) that >ended up being close to 256k. I think just the fact that it was just an >oddball number worked well, the fact that it was prime has nothing to do >with it. :) > >In short, picking something that doesn't end up as an integer if you do: > > ((32M / stripesize) / numdrives) > >was as close as I came to figuring out how to make it not beat one drive to >death. :) Thanks for the great info and sharing your insights after "playing with things for far too long". Found this post from Kevin Street in the archives: I tried changing the newfs cylinders/group to 19 (from 16) but this had little effect. I then rebuilt my vinum plex with a stripe size of 257k (rather than 256k) and newfs -c 19 and this works MUCH better. I now see very even distribution of io during the restore, directory operations and general file access. Prime numbers == good. Sound good to you? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message