From owner-freebsd-questions@FreeBSD.ORG Tue Oct 13 13:07:25 2009 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 551441065670 for ; Tue, 13 Oct 2009 13:07:25 +0000 (UTC) (envelope-from kraduk@googlemail.com) Received: from gv-out-0910.google.com (gv-out-0910.google.com [216.239.58.190]) by mx1.freebsd.org (Postfix) with ESMTP id D474D8FC1F for ; Tue, 13 Oct 2009 13:07:24 +0000 (UTC) Received: by gv-out-0910.google.com with SMTP id n29so3839gve.39 for ; Tue, 13 Oct 2009 06:07:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=e53rOS1b9UhA3dFzfWEMJv//RsbDhZKyW1IIvlTmy4I=; b=estt6EvFARYjttcKK7hoCf9BO1zrewG9kA9XFCN/tpw3htvGMXSNKuLmJP9PxJYxaO ZD8nhsOZnp3ycr89Nh/EXn+/FF+aWw+iPyF6DWv5UxmkmoqReWE/CIhO39sS0g4alDzc IX/CBQYpav9qbPRjiSn4Eq6Xurg1Qbq26a/u8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=dUGKbt1Vys2V/fTRWnTVTKO0ZJxHJl/YDV9ecRRO7lW0/PA7UtwmZjEjTKaWZw6FU3 KonF/FDjtyJ+ydEGSGkh4cXecBsW4Nji9ZuM0FGS+cUba2TSfRNaUnJx3cHJhw6DLGCb jN5d0EcCTFfFB6vFRuKu5DE8KDLe8eqiaC4VA= MIME-Version: 1.0 Received: by 10.239.139.226 with SMTP id u34mr501749hbu.80.1255439243356; Tue, 13 Oct 2009 06:07:23 -0700 (PDT) In-Reply-To: <4AD47A55.2060707@infracaninophile.co.uk> References: <59889.1255436565@thrush.ravenbrook.com> <4AD47A55.2060707@infracaninophile.co.uk> Date: Tue, 13 Oct 2009 14:07:23 +0100 Message-ID: From: krad To: Matthew Seaman Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Nick Barnes , freebsd-questions@freebsd.org, Ravenbrook SysAdmins Subject: Re: how does gmirror start up? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Oct 2009 13:07:25 -0000 2009/10/13 Matthew Seaman > Nick Barnes wrote: > >> I am running a 6.3 system and have had various problems with disk >> reliability on a key filesystem, probably down to SCSI hardware or >> drivers. I'm intending to replace that SCSI disk with a pair of SATA >> disks ad6/7, using gmirror as gm0 (while keeping ad4 as our boot >> disk). I have set up the mirror and right now I'm part-way through >> using rsync to transfer the data. But I have a question concerning >> the underlying operation of GEOM, which is troubling me. I have read >> the various man pages and handbook pages relating to GEOM and gmirror, >> but they don't seem to answer this. >> >> When I reboot the system, gmirror comes up (because of the line in >> /boot/loader.conf) and gm0 appears, backed by ad6/7. Where is this >> configuration information stored? That is, how does the system know >> to make gm0, with ad6/7 as the backing store. >> >> I would expect there to be a file somewhere in /etc with this config >> information, but I don't see it in the documentation. From reading >> gmirror(8), I understand that there is a label sector at the ends of >> ad6 and ad7, identifying them as parts of gm0. But that config >> information is back-to-front: at boot time the system knows from >> /etc/fstab that it needs gm0; how does it find the underlying disks? >> >> Does the system search the ends of all physically-attached disks, >> looking for GEOM labels, and automatically make any corresponding GEOM >> devices? Surely not (that would mean, for instance, that if I took >> one of these disks out of this machine and put it into another FreeBSD >> system then that machine would automatically set up gm0 with this >> disk). >> >> Possibly I'm just being dense. Can someone enlighten me? >> > > The geom configuration is stored on the disks in question -- I believe it > uses the last cylinder of the drive, but I could be wrong about the > details. > On startup, if the appropriate geom modules are loaded into the kernel, > geom will "taste" the disks discovered by enumerating the contents of all > ATA, SCSI, USB, etc. busses (ie. read any geom metadata) to see if they are > part of a RAID array of some type. The RAID will be automatically > recreated, > and appears to the system as a new device. If there are appropriate > entries in > /etc/fstab any file systems on it will be mounted. > > There's no data stored on the filesystem describing the RAID setup. Well, > unless you make a backup of it yourself by dumping the output of 'gmirror > status' > somewhere. > > This means that you can move the disks from one server to another and all > the > RAIDs will survive. Actually, it's good enough that it can cope with you > shuffling the disks or installing them with completely different underlying > device names. You would certainly have to avoid a clash of geom device > names > when doing that -- the names are entirely arbitrary and 'gm0' is only the > de-facto default because it was used in one of the most popular tutorials on > the subject. If you think it necessary, you could include the host name in > the geom label or otherwise take action to make them globally unique across > all your systems. > > You can achieve a similar effect for plain disks by using glabel -- this is > a > handy way of avoiding the foot-shooting potential of a USB thumb drive that > maps > as da0 on reboot, in place of the real system disk. > > Cheers, > > Matthew > > -- > Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard > Flat 3 > PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate > Kent, CT11 9PW > > Also worth mentioning zfs config is stored on disk as well in a similar manner albeit a bit more complicated.