From owner-freebsd-fs@FreeBSD.ORG Mon Jan 2 05:18:07 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 62C631065673 for ; Mon, 2 Jan 2012 05:18:07 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 19DF88FC0A for ; Mon, 2 Jan 2012 05:18:06 +0000 (UTC) Received: by vcbfk1 with SMTP id fk1so20614494vcb.13 for ; Sun, 01 Jan 2012 21:18:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=Rp0WFlFc6Bzgk1RqCol6B9lynbLmlLlDjGFuexOYxEc=; b=pUc0rfRaGLREMrwe/kbOOTtyl1dqD1VmlYDyrj1D9bgl3aYmVtbE4y5lc6yy3IU1HP R+zXbC1V3CNegX6Dmfa+PjF8lmb90s1D0WlVhBrkCZc9qoBdzx+7qWPMfqwZD185WYnH zrPPp90CVUdxd6gipLJ++qoVVSbNnFBP4pdK8= MIME-Version: 1.0 Received: by 10.220.147.197 with SMTP id m5mr29184111vcv.48.1325481486124; Sun, 01 Jan 2012 21:18:06 -0800 (PST) Received: by 10.220.194.131 with HTTP; Sun, 1 Jan 2012 21:18:06 -0800 (PST) In-Reply-To: References: <8EA721E0-977D-483C-AC06-1040B87E0AA7@deman.com> <20120101195411.GA73487@icarus.home.lan> Date: Sun, 1 Jan 2012 21:18:06 -0800 Message-ID: From: Freddie Cash To: Michael DeMan Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: zfs detach/replace X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jan 2012 05:18:07 -0000 On Sun, Jan 1, 2012 at 9:09 PM, Michael DeMan wrote: > Sounds realistic to me that the 'replace' command works after a 'detach' = is properly done. > > In my case, I forgot to do the 'detach' - just did the 'replace' after ch= anging the drive (and after a reboot). =C2=A0From there, ZFS gets in the st= ate below - where it automagically created label/ada5LABEL/old and appears = to be wanting to do the rebuild onto label/ada5LABEL as if it is part of a = mirrored pair? "zpool detach" and "zpool replace" are two very different, and totally unconnected, things. "zpool detach" is used to remove drives from mirror vdevs. The vdev then has one less drive in it. The beauty of the way mirror vdevs work in ZFS is that you can "zpool attach" a drive to an existing 2-way mirror vdev, thus turning it into a 3-way mirror. Then "zpool detach" the extra drive. Makes upgrading drive capacity a cinch, as you never lose any redundancy in the vdev. "zpool replace" is used to do an in-place replacement of a drive in a vdev, and can be used with mirror, raidz1, raidz2, and raidz3 vdevs. I think the part you may be getting confused with is the "zpool offline" step that should be done before doing a "zpool replace". That way, if anything goes wrong, you can always put back the old drive, do a "zpool online" to it, and then do a "zpool remove" or "zpool detach" to the "new" drive. That's saved my bacon a couple of times. :) Learnt the hard way about that, losing a 20 TB pool (only a couple TB of data in it) due to "insufficient replicas" and "missing drives" and whatnot. > I know for a fact, back with FreeBSD 7.x, that this same scenario could o= ccur. =C2=A0Basically on below - my old notes show there is no way to fix t= he situation with ada5LABEL and ada5LABEL/old without destroying and rebuil= ding the pool. =C2=A0Any attempts to 'detach', 'offline' or anything else o= n either of those two logical entries fails with a 'no valid replicas'. If you have the original drive (the /old one), put it back in the system, "zpool online" it, then you should be able to "zpool remove" or "zpool detach" the other drive. --=20 Freddie Cash fjwcash@gmail.com