From owner-freebsd-geom@FreeBSD.ORG  Tue Dec 13 13:16:28 2005
Return-Path: <owner-freebsd-geom@FreeBSD.ORG>
X-Original-To: freebsd-geom@freebsd.org
Delivered-To: freebsd-geom@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 1B81E16A41F
	for <freebsd-geom@freebsd.org>; Tue, 13 Dec 2005 13:16:28 +0000 (GMT)
	(envelope-from mark@gaiahost.coop)
Received: from biodiesel.gaiahost.coop (biodiesel.gaiahost.coop [64.95.78.120])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 3277F43D62
	for <freebsd-geom@freebsd.org>; Tue, 13 Dec 2005 13:16:27 +0000 (GMT)
	(envelope-from mark@gaiahost.coop)
Received: from localhost (host-64-65-195-19.spr.choiceone.net
	[::ffff:64.65.195.19]) (AUTH: LOGIN mark@hubcapconsulting.com)
	by biodiesel.gaiahost.coop with esmtp; Tue, 13 Dec 2005 08:16:24 -0500
	id 006FC054.439EC9AA.00005433
Received: by localhost (sSMTP sendmail emulation);
	Tue, 13 Dec 2005 08:16:38 -0500
Date: Tue, 13 Dec 2005 08:16:37 -0500
From: Mark Bucciarelli <mark@gaiahost.coop>
To: freebsd-geom@freebsd.org
Message-ID: <20051213131637.GC2188@rabbit>
Mail-Followup-To: freebsd-geom@freebsd.org
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
User-Agent: Mutt/1.4.2.1i
Subject: How to recover a gmirror from a disaster
X-BeenThere: freebsd-geom@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: GEOM-specific discussions and implementations
	<freebsd-geom.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-geom>,
	<mailto:freebsd-geom-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-geom>
List-Post: <mailto:freebsd-geom@freebsd.org>
List-Help: <mailto:freebsd-geom-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-geom>,
	<mailto:freebsd-geom-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 13 Dec 2005 13:16:28 -0000

The disaster:

    1. gmirror setup (full disk mirror, da0 + da1)

    2. disk head crash into platter on da1

    3. reboot

    4. mirror starts to sync, hits bad spot on da1, halts

    5. restart (can work for a while b/f bad spot hit)

    6. Enter command "gmirror remove gm0 da1"

    7. Now on restart, da0 will not boot.

On attempting boot on da0, this is the last output b/f system hangs:

    GEOM_MIRROR: Device gm0 created (id=...)
    GEOM_MIRROR: Device gm0: provider da0 detected
    GEOM_MIRROR: Device gm0 destroyed
    GEOM_MIRROR: Device gm0 created (id=...)
    GEOM_MIRROR: Device gm0: provider da0c detected
    GEOM_MIRROR: Device gm0 destroyed
    GEOM_MIRROR: Device gm0 created (id=...)
    GEOM_MIRROR: Device gm0: provider da0 detected
    GEOM_MIRROR: Device gm0 destroyed
    SMP: AP CPU #1 Launched!
    Mounting root from ufs:/dev/mirror/gm0s1a
    setrootbyname failed
    ffs_mountroot: can't find rootvp
    Root mount failed: 6

This is 5.4-RELEASE.

The da0 disk is good (I did a surface scan) and I can mount it and 
backup the data when I put it in bay #2.

So ...

    Did my command in step #6 cause this mess?   If so, what should I 
    have used--forget first?

    How can I get this disk to boot again?

m