From owner-freebsd-fs@FreeBSD.ORG Mon May 3 15:48:33 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D123D106564A for ; Mon, 3 May 2010 15:48:33 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-iw0-f180.google.com (mail-iw0-f180.google.com [209.85.223.180]) by mx1.freebsd.org (Postfix) with ESMTP id 93E778FC2A for ; Mon, 3 May 2010 15:48:33 +0000 (UTC) Received: by iwn10 with SMTP id 10so3486739iwn.13 for ; Mon, 03 May 2010 08:48:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=m5HfCq/L5R74kh9O3vtE9gPBOlsb6tf/ujVInXAGLME=; b=Z9RPa1ySEZWebV2jVpqsZMD/03r6uel5Rb9Eq1DwSSEmgUMvEvOcMo0GJiLB9UaoP4 zBIydtb+7m5+8WqZLryVsnvHCUh8DYaup5x1znt8SZy65sk4wZE9djnM4veQngR03wRe tZDOThbQnxbXKkBTalJ11FTMTi1vJSarsXUck= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=GauXr94E4qWtwG861N3irIBp5Z/FJlAJshimGAEaMbeAUamhCCOAH2wvIXIRIJq7ic XsNBx4G2I1LqMkK7Og1RgLRHmIUxn7sYA9ncqiYBNOt+vPpQ9DyZ6TZlXTE8UfiQwT3D to8ByvNtl6PB6T5AvdiqKIu8/6sAF99L0ZF3A= MIME-Version: 1.0 Received: by 10.231.153.130 with SMTP id k2mr1532872ibw.78.1272901663819; Mon, 03 May 2010 08:47:43 -0700 (PDT) Received: by 10.231.11.205 with HTTP; Mon, 3 May 2010 08:47:43 -0700 (PDT) In-Reply-To: References: Date: Mon, 3 May 2010 08:47:43 -0700 Message-ID: From: Freddie Cash To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: ZFS: "Cannot replace a replacing drive" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2010 15:48:33 -0000 On Fri, Apr 30, 2010 at 1:59 PM, Wes Morgan wrote: > On Fri, 30 Apr 2010, Wes Morgan wrote: > > On Thu, 29 Apr 2010, Freddie Cash wrote: > > > > > On Thu, Apr 29, 2010 at 6:06 PM, Wes Morgan > wrote: > > > > > > > On Wed, 28 Apr 2010, Freddie Cash wrote: > > > > > > > > > Going through the archives, I see that others have run into this > issue, > > > > and > > > > > managed to solve it via "zpool detach". However, looking closely > at the > > > > > archived messages, all the successful tests had one thing in > common: 1 > > > > > drive ONLINE, 1 drive FAULTED. If a drive is online, obviously it > can be > > > > > detached. In all the cases where people have been unsuccessful at > fixing > > > > > this situation, 1 drive is OFFLINE, and 1 drive is FAULTED. As is > our > > > > case: > > > > > > > > > > > > > What happened to the drive to fault it? > > > > > > > > Am in the process of replacing 500 GB drives with 1.5 TB drives, to > > > increase the available storage space in the pool (process went > flawlessly on > > > the other storage server). First 3 disks in the vdev replaced without > > > issues. > > Also, where is the original 500gb drive (#4) that you replaced? You should > be able to put that drive back in and let it resilver itself to catch back > up to the other devices, then cancel the replacement. > That worked. :) I pulled the 1.5 TB drive from the system, inserted the original 500 GB drive, and ZFS detected it and marked it as online, with the 1.5 TB as faulted. At that point, I was able to "zpool detach" the 1.5 TB drive, putting the pool back into the original configuration, and online state. After that, I "zpool offline" the 500 GB drive, replace it with a new 1.5 TB drive, and "zpool replace". And it's happily re-silvering the drive. In 35h, it should be complete. Thankfully, the original drive was not faulty and was only being replaced to increase the size. If the original drive was dead, I'd probably be hooped. -- Freddie Cash fjwcash@gmail.com