From owner-freebsd-geom@FreeBSD.ORG Thu Nov 18 20:22:05 2010 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 229061065672 for ; Thu, 18 Nov 2010 20:22:05 +0000 (UTC) (envelope-from fb-geom@psconsult.nl) Received: from mx1.psconsult.nl (unknown [IPv6:2001:7b8:30f:e0::5059:ee8a]) by mx1.freebsd.org (Postfix) with ESMTP id A81178FC0A for ; Thu, 18 Nov 2010 20:22:04 +0000 (UTC) Received: from mx1.psconsult.nl (psc11.adsl.iaf.nl [80.89.238.138]) by mx1.psconsult.nl (8.14.4/8.14.4) with ESMTP id oAIKLumF054359 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 18 Nov 2010 21:22:01 +0100 (CET) (envelope-from fb-geom@psconsult.nl) Received: (from paul@localhost) by mx1.psconsult.nl (8.14.4/8.14.4/Submit) id oAIKLuPG054358 for freebsd-geom@freebsd.org; Thu, 18 Nov 2010 21:21:56 +0100 (CET) (envelope-from fb-geom@psconsult.nl) X-Authentication-Warning: mx1.psconsult.nl: paul set sender to fb-geom@psconsult.nl using -f Date: Thu, 18 Nov 2010 21:21:56 +0100 From: Paul Schenkeveld To: freebsd-geom@freebsd.org Message-ID: <20101118202156.GA45692@psconsult.nl> References: <20101118163323.GA2524@gvr.gvr.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20101118163323.GA2524@gvr.gvr.org> User-Agent: Mutt/1.5.19 (2009-01-05) Subject: Re: Booting from a gmirror always leads to a broken gmirror X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2010 20:22:05 -0000 Hi Guido, On Thu, Nov 18, 2010 at 05:33:23PM +0100, Guido van Rooij wrote: > Hi, > > I am running 8.1-RELEASE with the folloing mirror setup: > Geom name: gm0 > State: DEGRADED > Components: 2 > Balance: round-robin > Slice: 4096 > Flags: NONE > GenID: 3 > SyncID: 2 > ID: 2610005691 > Providers: > 1. Name: mirror/gm0 > Mediasize: 500107861504 (466G) > Sectorsize: 512 > Mode: r3w2e5 > Consumers: > 1. Name: da1 > Mediasize: 500107862016 (466G) > Sectorsize: 512 > Mode: r1w1e1 > State: ACTIVE > Priority: 0 > Flags: DIRTY > GenID: 3 > SyncID: 2 > ID: 3263791510 > 2. Name: da0 > Mediasize: 500107862016 (466G) > Sectorsize: 512 > Mode: r1w1e1 > State: SYNCHRONIZING > Priority: 0 > Flags: DIRTY, SYNCHRONIZING > GenID: 3 > SyncID: 2 > Synchronized: 15% > ID: 4162979509 > > I have a dedicated slice to boot from and one GELI slice, both on the mirror. > Now when I reboot this system (at a time when the mirror is 100% fine, unlike above), the following happens: > Nov 18 15:36:01 gvr kernel: Root mount waiting for: usbus4 > Nov 18 15:36:01 gvr kernel: ugen4.2: at usbus4 > Nov 18 15:36:01 gvr kernel: umass0: v 2.00/1.75, addr 2> on usbus4 > Nov 18 15:36:01 gvr kernel: ugen1.2: at usbus1 > Nov 18 15:36:01 gvr kernel: uhub6: 1.10/1.10, addr 2> on usbus1 > Nov 18 15:36:01 gvr kernel: uhub5: 4 ports with 1 removable, self powered > Nov 18 15:36:01 gvr kernel: Root mount waiting for: usbus4 > Nov 18 15:36:01 gvr kernel: uhub6: 3 ports with 2 removable, bus powered > Nov 18 15:36:01 gvr kernel: da0 at umass-sim0 bus 0 scbus0 target 0 lun 0 > Nov 18 15:36:01 gvr kernel: da0: Fixed Direct Access > SCSI-4 device > Nov 18 15:36:01 gvr kernel: da0: 40.000MB/s transfers > Nov 18 15:36:01 gvr kernel: da0: 476940MB (976773168 512 byte sectors: 255H 63S/ > T 60801C) > Nov 18 15:36:01 gvr kernel: ugen1.3: at usbus1 > Nov 18 15:36:01 gvr kernel: ugen4.3: at usbus4 > Nov 18 15:36:01 gvr kernel: umass1: v 2.00/1.75, addr 3> on usbus4 > Nov 18 15:36:01 gvr kernel: da1 at umass-sim1 bus 1 scbus1 target 0 lun 0 > Nov 18 15:36:01 gvr kernel: da1: Fixed Direct Access > SCSI-4 device > Nov 18 15:36:01 gvr kernel: da1: 40.000MB/s transfers > Nov 18 15:36:01 gvr kernel: da1: 476940MB (976773168 512 byte sectors: 255H 63S/ > T 60801C) > Nov 18 15:36:01 gvr kernel: GEOM_MIRROR: Component da0 (device gm0) broken, skipping. > Nov 18 15:36:01 gvr kernel: GEOM_MIRROR: Device mirror/gm0 launched (1/2). > Nov 18 15:36:01 gvr kernel: Enter passphrase for da0s2: Your mirror covers the whole drives da0 amd da1. Geli discovers a encrypted provider on da0s2, which sounds odd. Is geli supposed to find gm0s2 instead? GEOM_MIRROR reports component da0 as broken, could it be that its metadata is not updated correctly at shutdown? Your disks are connected thru USB, which is a bit tricky. I've some experience with a machine that once had a three disk graid3 connected through USB. After a normal shutdown (or reboot) graid3 came up fine but I remember that at some occasions graid3 found one of its providers broken and atarted a rebuild. Could you try the following: 1 Shutdown the computer, watch carefully for any errors regarding gm0 or the physical disks. 2 Boot a kernel without geom_mirror and geom_eli classes compiled in (e.g. a GENERIC kernel) from CDROM or USB stick. 3 Look at the gmirror metadata with gmirror dump da0 and gmirror dump da1 to see if they are in sync. Did differs, priority for one disk should be 0 the other 1, the md5 hash differs but all other fields should be equal for both disks. > Now, the passphrase the geli asks, is for the so-called broken disk. But it is not broken. > I can always rebuild it fine when I rebooted and re-insert da0. > My question: why does gmirror mark da0 as broken? > > -Guido > > _______________________________________________ > freebsd-geom@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-geom > To unsubscribe, send any mail to "freebsd-geom-unsubscribe@freebsd.org"