From owner-freebsd-stable@FreeBSD.ORG Thu Jul 23 23:24:45 2009 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 51940106564A for ; Thu, 23 Jul 2009 23:24:45 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from hamlet.setfilepointer.com (hamlet.SetFilePointer.com [63.224.10.2]) by mx1.freebsd.org (Postfix) with SMTP id 072448FC1E for ; Thu, 23 Jul 2009 23:24:44 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: (qmail 59700 invoked from network); 23 Jul 2009 18:24:44 -0500 Received: from keira.kiwi-computer.com (HELO kiwi-computer.com) (63.224.10.3) by hamlet.setfilepointer.com with SMTP; 23 Jul 2009 18:24:44 -0500 Received: (qmail 72746 invoked by uid 2001); 23 Jul 2009 23:24:44 -0000 Date: Thu, 23 Jul 2009 18:24:44 -0500 From: "Rick C. Petty" To: "Sean C. Farley" Message-ID: <20090723232443.GA72486@keira.kiwi-computer.com> References: <200901232244.n0NMiRmM098646@lurza.secnetix.de> <46acbb3e-71bc-4cff-93d7-59b48a1a9302@exchange01.ecp.noc> <4A68B2A0.8050509@incunabulum.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-stable@FreeBSD.org Subject: Re: ataraid's revenge! (Was: Re: A nasty ataraid experience.) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd2008@kiwi-computer.com List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jul 2009 23:24:45 -0000 This question should have been directed to freebsd-geom. On Thu, Jul 23, 2009 at 03:42:47PM -0500, Sean C. Farley wrote: > > Anyone know if I can boot off of a gvinum partition and/or how it works > (or does not) with various label schemes? http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/vinum-root.html There are some downsides. First of all, when you mount your root partition under vinum, you can't unload the geom_vinum module (for obvious reasons). If you decide to relocate volumes in the future, you won't be able to remove the disks from the vinum. The reason is because you're not allowed to change the bsdlabel if there's a vinum slice while vinum is loaded, and you cannot remove a "device" from vinum even if it has no subdisks. I know lulf@ was looking into the second part. I used to do this on all my machines. Sometime soon after 7.1-release, I noticed a significant slowdown in disk performance, perhaps due to disk scheduling. Basically while any moderate I/O hit one gvinum volume, all other I/O on that disk was essentially suspended until the the heavier I/O completed. What this meant was that if I did an rsync to or from a volume whose subdisk resided on the same physical disk as my root and /usr partitions, I could not run any commands until the rsync finished or was killed (with control-C, since I couldn't even get a "killall" to execute). After that I switched all my machines to use gmirror which helped slightly. During that switch is when I discovered you need to be pretty clever to go away from "vinum root". I don't mean to say not to use gvinum. I put a lot of time into it, submitted some patches, and did a lot of testing with Ulf's deltas. There were still a couple of bugs when I stopped using it, but that doesn't mean they won't be fixed by 8.0. I just couldn't deal with the I/O performance issues which I was never able to track down. I need my workstations to be responsive while I'm rsyncing stuff around, and with 4 cores and 8 GB of RAM, I expected it to behave better. YMMV. FYI, the trick I used when migrating mirrors in vinum to gmirror: 1). Pick one disk to convert to gmirror first. I'll call this disk0. 2). Remove all subdisks which are attached to the device using disk0. 3). Reboot into single-user with vinum unloaded, specifying the raw device root (from disk1) instead of /dev/gvinum/root-volume. 4). Wipe sector 8 of the vinum slice on disk0. 5). Create a gmirror with disk0, re-bsdlabel, create new filesystems, etc. 6). Load geom_vinum kernel module. This may panic the kernel. If so, reboot normally using the vinum root volume. 7). Mount the new filesystems. 8). Mount the volumes you want moved to the mirror and rsync them over to the filesystems mounted on disk0. 9). Unmount all vinum volumes and try to unload the geom_vinum module. If this panics the kernel, or if there was a panic in step 6, you'll need to reboot into single-user again with geom_vinum unloaded. 10). Wipe sector 8 of the vinum slick on disk1. 11). Add disk1 to the gmirror, which should start a sync. 12). Load the geom_vinum module again, if you have other volumes not part of this mirror. If this panics the kernel, or if the kernel panicked after step 11, a normal restart might be successful. -- Rick C. Petty