Date: Fri, 19 May 2006 15:14:53 -0500 From: "Rick C. Petty" <rick-freebsd@kiwi-computer.com> To: James Snow <snow@teardrop.org> Cc: freebsd-geom@freebsd.org Subject: Re: Slices + stripes and mirrors Message-ID: <20060519201453.GB37562@megan.kiwi-computer.com> In-Reply-To: <20060519192943.GB54819@teardrop.org> References: <200605161555.08195.darcy@wavefire.com> <20060517171955.GB838@garage.freebsd.pl> <20060519172148.GA54819@teardrop.org> <20060519180604.GA37562@megan.kiwi-computer.com> <20060519192943.GB54819@teardrop.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, May 19, 2006 at 03:29:43PM -0400, James Snow wrote: > On Fri, May 19, 2006 at 01:06:04PM -0500, Rick C. Petty wrote: > > > > If you're not mirroring the entire disk, I think you're halfway down a > > dangerous path. > > I think the original poster's use case is a fairly common one. For > example, you've got two 40GB disks, and you want to use the first 20GB > of each disk for a single 20GB mirror, and the second 20GB for a 40GB > stripe. That's common? I guess I personally would feel more comfortable mirroring the whole disk and buying more disks if I want to stripe. I guess I'm not a fan of striping either. The point of a mirror is to reduce your downtime in case of a disk error.. if one of the mirrored disks fails, you likely will lose the entire stripe, in the above configuration. Disks are cheap! Buy more! > Why does this constitute a dangerous path? For the reason I just stated. If you're mirroring part of the disk, why is the remaining disk useless? And if it is really useless, why do you have it on the same disks you believe are important? Part of the danger that I see is the reconstruction of the label. You're not mirroring the label or the boot blocks, so you have to remember how you setup the disk before.. "I *think* I used 20g on the first slice.. but did I add 1 block, knowing that I'd lose a block for gmirror's metadata, or did I round the number of blocks up so the cylinder groups didn't waste as much" etc. It seems to me to be a lot more work to setup than what is potentially gained or lost. You might as well mirror the whole disk, including the label. And if you're low on space, buy more drives. [subliminal message] You know you *want* to [/subliminal message] > > In any case, I've had better luck using gvinum even for just plain > > mirroring. > > I guess we've had somewhat opposite experiences here. I usually wind up > digging up old documentation on vinum off the web and fiddling with > gvinum until I determine what's still applicable. I found out that I > could use 'sd length 0' to autosize disks from this list rather than > from any of the docs. Maybe I'm suffering from a severe case of the > stupids, but good documentation on gvinum seems to be hard to come by. I completely agree with you here, but you only have to figure it out once. Then you save your configuration into a text file on a gmirror'd file system, if you're really paranoid. However, the documentation leads something to be desired. I think the problem is we're in a transition period, where vinum was deprecated and subsequently removed, but gvinum didn't have all vinum's features implemented or documented in time. My point wasn't comparing the ease of setting up an N-drive mirror (of the entire disks) with using gvinum, but that the flexibility gvinum has and its ability to modify volumes while other volumes are mounted, not to mention the ease of using volume names, the case for using gmirror for anything but trivial disk mirroring is pretty thin. > > IMO, it's easier to use a livecd since you have to reboot the box > > anyway. > > But this assumes you have console access. I've successfully used > gmirror to convert machines on the opposite side of the country to RAID > without any remote-hands or physical access. It works perfectly for me > most of the time. I've little doubt that I'm doing something stupid > this time. Yes and no (about console access).. Mess up even one minor step and without a console, you're boned. Besides, it's better to do this from single-user mode if you're migrating your root file system-- serial consoles rock! [you add drives to a remote box without remote hands or physical access? please, share your secrets! :-P] Still, setting up a mirror from a boot disk to another disk is pretty trivial: gmirror label, fdisk, bsdlabel, newfs, "copy files", reboot. Only one extra step over your standard installation. NOTE: I prefer using rsync to dump/restore -- better handling of hard links and sparse files. > You can hot add/remove drives out of a gvinum partition as well. I'll > grant that it was somewhat ungraceful, but I did this recently when I > needed to duplicate a machine in a hurry. It worked very well. Scary, but yes. > I actually started out playing with gvinum for my setup, but it > didn't seem to perform very well. I freely admit I wasn't being very > scientific in my testing - just simple dd write tests - but, well, see > for yourself: > > 273k stripe size: 7857123 bytes/sec > 8190k stripe size: 8207057 bytes/sec > regular disk: 29639806 bytes/sec > graid3 disk: 54549442 bytes/sec Hmm, something must be strange in your setup. What size blocks and where were you reading from? Are those first two tests raid5? Remember, one write operation in RAID5 turns into two reads and two writes (stupid parity). RAID5 performs better with read. I recall seeing significantly higher numbers of total disk bandwidth using raid5 over the drives separately, but I don't recall what the true throughput was. It also depends on whether you go through the filesystem: dd if=/dev/zero of=/dev/gvinum/testvol bs=64k performs quite differently than: mount /dev/gvinum/testvol /testvol apply "dd if=/dev/zero of=/testvol/%1 bs=64k count=1600" 1 2 3 4 5 -- Rick C. Petty
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060519201453.GB37562>