From owner-freebsd-fs@FreeBSD.ORG Sun Jan 1 19:03:24 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B83D2106564A for ; Sun, 1 Jan 2012 19:03:24 +0000 (UTC) (envelope-from freebsd@deman.com) Received: from plato.corp.nas.com (plato.corp.nas.com [66.114.32.138]) by mx1.freebsd.org (Postfix) with ESMTP id 71DBF8FC12 for ; Sun, 1 Jan 2012 19:03:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by plato.corp.nas.com (Postfix) with ESMTP id D633910488269 for ; Sun, 1 Jan 2012 11:03:23 -0800 (PST) X-Virus-Scanned: amavisd-new at corp.nas.com Received: from plato.corp.nas.com ([127.0.0.1]) by localhost (plato.corp.nas.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uCO8eKZvWD5L for ; Sun, 1 Jan 2012 11:03:22 -0800 (PST) Received: from [192.168.2.247] (mono-sis1.s.bli.openaccess.org [66.114.32.149]) by plato.corp.nas.com (Postfix) with ESMTPSA id 8904610488257 for ; Sun, 1 Jan 2012 11:03:22 -0800 (PST) From: Michael DeMan Mime-Version: 1.0 (Apple Message framework v1084) Date: Sun, 1 Jan 2012 11:03:21 -0800 In-Reply-To: To: freebsd-fs@freebsd.org References: <8EA721E0-977D-483C-AC06-1040B87E0AA7@deman.com> Message-Id: X-Mailer: Apple Mail (2.1084) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: zfs detach/replace X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jan 2012 19:03:24 -0000 Right, I botched it and did a 'replace' on a drive that is in an raidz2 = pool. replace is for replacing a mirrored drive. This happened to us a couple years ago during early testing, we noticed = it, and at that time were unable to get the machine out of its funky = state without destroying and rebuilding the pool. Basically drive #5 in = the raidz2 pool thinks that it is supposed to be working on a = replacement for a pair of mirrored physical drives. We have shutdown and started the machine without the drive installed, = done all sorts of things, etc. We can not even offline it because: # zpool offline zp1rz2 label/ada5LABEL cannot offline label/ada5LABEL: no valid replicas On Dec 31, 2011, at 4:08 PM, Joshua Boyd wrote: > On Sat, Dec 31, 2011 at 1:58 AM, Michael DeMan = wrote: > Hi All, >=20 > The origination of the problem is entirely my fault on FreeBSD 8.1 = RELEASE #0. We had old notes that attempting a 'replace' (which is = appropriate for a mirror) leaves ZFS in a funky state on BSD. I = inadvertently did just that on a drive swap on a raidz2 pool. My old = notes show the only recovery that we knew of at the time was to rsync or = zfs-send the pool elsewhere, destroy the local and rebuild from scratch. >=20 > I've never had a problem before, and have replaced about 5 drives in = my striped raidz ... >=20 > Usually I'll execute a zpool offline, camcontrol stop, remove the = drive, then zpool replace. >=20 > I am running 8-STABLE though, and not -RELEASE. >=20 > --=20 > Joshua Boyd >=20 > E-mail: boydjd@jbip.net > http://www.jbip.net