From owner-freebsd-fs@FreeBSD.ORG Mon Jan 2 05:09:40 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E84F8106566B for ; Mon, 2 Jan 2012 05:09:39 +0000 (UTC) (envelope-from freebsd@deman.com) Received: from plato.corp.nas.com (plato.corp.nas.com [66.114.32.138]) by mx1.freebsd.org (Postfix) with ESMTP id A78F28FC0A for ; Mon, 2 Jan 2012 05:09:39 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by plato.corp.nas.com (Postfix) with ESMTP id 1C8161049404B for ; Sun, 1 Jan 2012 21:09:39 -0800 (PST) X-Virus-Scanned: amavisd-new at corp.nas.com Received: from plato.corp.nas.com ([127.0.0.1]) by localhost (plato.corp.nas.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6mTNdApm46d0 for ; Sun, 1 Jan 2012 21:09:38 -0800 (PST) Received: from [192.168.2.247] (mono-sis1.s.bli.openaccess.org [66.114.32.149]) by plato.corp.nas.com (Postfix) with ESMTPSA id 5D0591049403F for ; Sun, 1 Jan 2012 21:09:38 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1084) From: Michael DeMan In-Reply-To: <20120101195411.GA73487@icarus.home.lan> Date: Sun, 1 Jan 2012 21:09:37 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <8EA721E0-977D-483C-AC06-1040B87E0AA7@deman.com> <20120101195411.GA73487@icarus.home.lan> To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.1084) Subject: Re: zfs detach/replace X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jan 2012 05:09:40 -0000 Hi, Sounds realistic to me that the 'replace' command works after a 'detach' = is properly done. In my case, I forgot to do the 'detach' - just did the 'replace' after = changing the drive (and after a reboot). =46rom there, ZFS gets in the = state below - where it automagically created label/ada5LABEL/old and = appears to be wanting to do the rebuild onto label/ada5LABEL as if it is = part of a mirrored pair? This pool was built on FreeBSD 8.0, with an operating system update to = 8.1 after that. Possibly I could try and update the ZFS pool version or = something if this is fixed now? =20 I know for a fact, back with FreeBSD 7.x, that this same scenario could = occur. Basically on below - my old notes show there is no way to fix = the situation with ada5LABEL and ada5LABEL/old without destroying and = rebuilding the pool. Any attempts to 'detach', 'offline' or anything = else on either of those two logical entries fails with a 'no valid = replicas'. $ zpool status pool: zp1rz2 state: DEGRADED status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are = unaffected. action: Determine if the device needs to be replaced, and clear the = errors using 'zpool clear' or replace the device with 'zpool replace'. see: http://www.sun.com/msg/ZFS-8000-9P scrub: scrub in progress for 53h42m, 80.53% done, 12h59m to go config: NAME STATE READ WRITE CKSUM zp1rz2 DEGRADED 0 0 0 raidz2 DEGRADED 0 0 0 label/ada0LABEL ONLINE 0 0 0 label/ada1LABEL ONLINE 0 0 0 label/ada2LABEL ONLINE 35 0 0 19.5K = repaired label/ada3LABEL ONLINE 0 0 0 label/ada4LABEL ONLINE 0 0 0 replacing UNAVAIL 0 3.24M 0 = insufficient replicas label/ada5LABEL/old UNAVAIL 0 3.74M 0 cannot = open label/ada5LABEL UNAVAIL 0 3.74M 0 cannot = open label/ada6LABEL ONLINE 0 0 0 label/ada7LABEL ONLINE 0 0 0 errors: No known data errors - Mike On Jan 1, 2012, at 11:54 AM, Jeremy Chadwick wrote: > On Sun, Jan 01, 2012 at 11:03:21AM -0800, Michael DeMan wrote: >> Right, I botched it and did a 'replace' on a drive that is in an >> raidz2 pool. replace is for replacing a mirrored drive. >=20 > I just want to clarify for readers: this statement is inaccurate. >=20 > "zpool replace" can (should) be used on anything (mirrors, raidzX, > etc.). >=20 > I have used "zpool replace" myself on FreeBSD on raidz1 without = problems > for years. Here's valid proof, and this is not "simulated" or done on = a > VM or anything like that -- this is bare metal. >=20 > = http://koitsu.wordpress.com/2010/07/22/freebsd-and-zfs-hot-swapping-sata-d= isks-with-ahci/ >=20 > Full commands/output are provided there as well (there's a link within > my blog post providing full output). >=20 > --=20 > | Jeremy Chadwick jdc at parodius.com | > | Parodius Networking http://www.parodius.com/ | > | UNIX Systems Administrator Mountain View, CA, US | > | Making life hard for others since 1977. PGP 4BD6C0CB | >=20