From owner-freebsd-questions@FreeBSD.ORG Wed Nov 24 19:21:39 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EFF64106566B for ; Wed, 24 Nov 2010 19:21:38 +0000 (UTC) (envelope-from cyberleo@cyberleo.net) Received: from paka.cyberleo.net (paka.cyberleo.net [66.219.31.21]) by mx1.freebsd.org (Postfix) with ESMTP id C3ACB8FC08 for ; Wed, 24 Nov 2010 19:21:38 +0000 (UTC) Received: from [172.16.44.4] (den.cyberleo.net [66.253.36.39]) by paka.cyberleo.net (Postfix) with ESMTPSA id 84365296E4; Wed, 24 Nov 2010 14:21:37 -0500 (EST) Message-ID: <4CED65C1.5040102@cyberleo.net> Date: Wed, 24 Nov 2010 13:21:37 -0600 From: CyberLeo Kitsana User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.9) Gecko/20100911 Lightning/1.0b3pre Thunderbird/3.1.3 MIME-Version: 1.0 To: perryh@pluto.rain.com References: <4ce8b7ce.qnD6CCAK8thDasFl%perryh@pluto.rain.com> <4ceb40c8.IB5c12O9ZJa3abeh%perryh@pluto.rain.com> <4CEC23D9.6090600@cyberleo.net> <4cecee6a.FUrbpNUbt2xJT/Wf%perryh@pluto.rain.com> In-Reply-To: <4cecee6a.FUrbpNUbt2xJT/Wf%perryh@pluto.rain.com> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: 7bit Cc: freebsd-questions@freebsd.org, kraduk@gmail.com Subject: Re: a gmirror disappears after adding gjournals to its partitions X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Nov 2010 19:21:39 -0000 On 11/24/2010 04:52 AM, perryh@pluto.rain.com wrote: > It looks to me as if gjournal is confused: It is not gjournal that is confused; it's bsdlabel. The gjournals lie entirely within the partitions defined within the bsdlabel, and don't care about anything outside of that. The ambiguity here is that the bsdlabel is stored at the beginning of the disk, and is very loose about what it accepts as valid, since there is no direct harm in being eager. The metadata for gmirror is stored at the end. The metadata for the bsdlabel is stored at the beginning. When bsdlabel tastes before gmirror, it sees the same label on the component disks that would be on the gm0 mirror. Moreover, all the partitions it then creates are identically sized, and contain exactly the same data, as they would on the mirror. It will complain that partition 'c' doesn't cover the whole unit, but this is not a fatal error as it doesn't take exclusive access, and so you are always free to use that same bsdlabel through another geom path. The problem arises when bsdlabel tastes ad0 before gmirror, and creates all the partitions thereupon, which triggers a taste of all the newly created devices by gjournal, which opens the devices exclusively once it finds the metadata it needs within the partitions. Now that they're opened exclusively somewhere, all the other paths to that device through the geom graph are withered, and cannot be tasted or used by anything else, including gmirror. Hardcoding provider names into gjournal makes it reject these ambiguously created devices. Since gjournal doesn't take exclusive access, gmirror can now taste the still-available ad0, see that it's a mirror, and launch gm0, which triggers a taste by bsdlabel (and creates the partitions) which triggers a taste by gjournal, which matches the names its expecting. That was difficult to keep clear. I hope it makes sense! >> ... either make the two look different somehow (use a different >> geom that stores its metadata at the beginning of the provider, >> instead of the end, thus eliminating ambiguity in the bsdlabel >> taste), > > When I asked earlier how to subdivide gm0, bsdlabel was recommended. > Is there something else that would work better? (This machine is > likely too old to understand GPT.) The machine's bios does not need to understand GPT to use it on a pure data disk; only as a boot disk. There are a few bioses that throw fits when not all the disks include mbr/slice tables, but those (thankfully) tend to be the minority. Plus, since GPT expects metadata at both the beginning and end of the disk, seeing gmirror metadata instead may prevent it from creating these ambiguous device nodes as well (but test this assumption before relying on it). >> or to make the inner geom avoid the outer devices (hardcode >> provider names in metadata). Since you have an outer geom that >> provides a static name, hardcoding the name of the gmirror into >> the gjournal metadata shouldn't cause anything to break if your >> disks change places, either. > > But I suspect this may not scale well. Suppose I later decide to > mirror the swap instead of using ad0s2b and ad8s2b as separate swap > partitions. Is there not a 50/50 chance of the swap mirror becoming > gm0 and my current gm0 becoming gm1, thereby breaking any metadata > that depends on hard-coded provider names? When you create a mirror, you give it an explicit name, which will not change over the life of the mirror without your explicit action. This name does not have to be 'gm0' or some such. I have named mirrors after the hostname, or 'hostname-purpose', such as 'sc1425-root' and 'sc1425-swap' -- Fuzzy love, -CyberLeo Technical Administrator CyberLeo.Net Webhosting http://www.CyberLeo.Net Furry Peace! - http://wwww.fur.com/peace/