From owner-freebsd-fs@FreeBSD.ORG Wed May 19 05:06:07 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2124E106564A for ; Wed, 19 May 2010 05:06:07 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from warped.bluecherry.net (unknown [IPv6:2001:440:eeee:fffb::2]) by mx1.freebsd.org (Postfix) with ESMTP id AAE718FC08 for ; Wed, 19 May 2010 05:06:06 +0000 (UTC) Received: from volatile.chemikals.org (adsl-67-124-127.shv.bellsouth.net [98.67.124.127]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by warped.bluecherry.net (Postfix) with ESMTPSA id 31FC286D9DB2; Wed, 19 May 2010 00:06:05 -0500 (CDT) Received: from localhost (morganw@localhost [127.0.0.1]) by volatile.chemikals.org (8.14.4/8.14.4) with ESMTP id o4J55xkK079568; Wed, 19 May 2010 00:06:00 -0500 (CDT) (envelope-from morganw@chemikals.org) Date: Wed, 19 May 2010 00:05:59 -0500 (CDT) From: Wes Morgan X-X-Sender: morganw@volatile To: Todd Wasson In-Reply-To: <53F15A8B-77DA-4CEF-A790-2902BEC91002@duke.edu> Message-ID: References: <0B97967D-1057-4414-BBD4-4F1AA2659A5D@duke.edu> <4BF0F231.9000706@mapper.nl> <53F15A8B-77DA-4CEF-A790-2902BEC91002@duke.edu> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: clamav-milter 0.95.3 at warped X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: zfs drive replacement issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2010 05:06:07 -0000 On Mon, 17 May 2010, Todd Wasson wrote: > > Hello, > > You could try exporting and importing the pool with three disks. > > Then make sure the "new" drive isn't part of any zpool (low-level format?). > > Then try a "replace" again. > > Have fun! > > > > > Hi Mark, I was about to try this, but I just tried putting the "old" > (damaged) drive back in the pool and detaching the "new" drive from the > pool, which I've tried before, but for some reason this time it > succeeded. I was then able to "zpool offline" the old drive, physically > replace it with the new one, and "zpool replace" the old one with the > new one. It just finished successfully resilvering, and apparently > everything is working well. I'm going to initiate a scrub to be sure > that everything is alright, but I'm fairly sure that the problem is > solved. I didn't do anything that I hadn't already tried, so I don't > know why it worked this time, but I'm not complaining. Thanks to > everyone for your help; at the very least, the idea of putting the > original drive back into the machine and mucking around with it led me > in the right direction. Next time I'll be sure to issue an offline > command before replacing a device! I'm not certain that you really always want to do that. When you offline a device in a redundant pool you lose that redundancy. If you have a drive that is completely dead, it is obviously the right thing to do, but otherwise perhaps not. Were you the have another failure during the rebuild, or if there was another error on a different vdev, you wouldn't be able to recover that data because of the missing device. The same reason why offlining and replacing each device in a raidz1 to "grow" it isn't as safe as you might think -- any error could lead to data loss. Just food for thought.