From owner-freebsd-fs@FreeBSD.ORG Thu Aug 11 16:24:22 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C42A0106564A for ; Thu, 11 Aug 2011 16:24:22 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 82D968FC0C for ; Thu, 11 Aug 2011 16:24:22 +0000 (UTC) Received: by vws18 with SMTP id 18so2430699vws.13 for ; Thu, 11 Aug 2011 09:24:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=WRdMBqB7JTOBQ6v8IS2F8W2PNkJYefQP13LyO9RpJs0=; b=NoY/Pkm88hOBAlH3ALHh3LzCDWVPz1t6x6uglWBqUxcx0dybIgSwyv/RWWEduAkP6x jOyY+jfw8p6LhektQv87RHjNjPu7FlxmTFPH3jzxoEQ6Bddm0ZCMDlME9meKz9mTOh3Q Ek3kvxH0rKDdQ/1o/StWUcZqOSA4e6gwUJOA0= MIME-Version: 1.0 Received: by 10.220.117.136 with SMTP id r8mr1596379vcq.109.1313079861686; Thu, 11 Aug 2011 09:24:21 -0700 (PDT) Received: by 10.220.186.134 with HTTP; Thu, 11 Aug 2011 09:24:21 -0700 (PDT) In-Reply-To: <20110811160314.GA25076@cons.org> References: <20110811160314.GA25076@cons.org> Date: Thu, 11 Aug 2011 09:24:21 -0700 Message-ID: From: Freddie Cash To: Martin Cracauer Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS zpool mirror drive replacement confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Aug 2011 16:24:22 -0000 On Thu, Aug 11, 2011 at 9:03 AM, Martin Cracauer wrote: > My computer keeps doing what I say, not what I want :-) > > This zpool with a 2-drive mirror had a dead drive (ad10) which I > removed. After I tried to put ad2 as a replacement drive I ended up > with the drive not in the mirror, it seems to have extended the zpool > (it's size?). > > NAME STATE READ WRITE CKSUM > cbackup3 DEGRADED 0 0 0 > mirror DEGRADED 0 0 0 > ad6 ONLINE 0 0 0 > ad10 REMOVED 0 0 0 > ad2 ONLINE 0 0 11 > > Unfortunately I didn't keep note of the specific command that I used. > > Mirror vdevs are slightly special in they way they are handled. You don't have to "zpool replace" drives in vdevs. Instead, you "zpool detach" the old/dead drive from the pool (this "breaks" the mirror vdev, leaving you with a single-disk vdev), and then you "zpool attach" the new drive *to the existing drive*. For example, the series of commands you should have used would be: zpool detach cbackup3 ad10 zpool attach cbackup3 ad2 ad6 Or, you don't want to lose any redundancy in your pool, you attach the new drive first, creating a 3-way mirror vdev, and then detach the old drive after the resilver is complete. zpool attach cbackup3 ad2 ad6 zpool detach cbackup3 ad10 Right now, your pool has 2 vdevs in it: 1 mirror (ad6 + ad10) and one single-disk (ad2). You are currently running with a "degraded" pool, as there is no redundancy in it: if you lose disk ad2, you lose the entire pool. Thankfully, if you have the drive bays, this is easy to fix. Just attach another drive to ad2 to create a second mirror vdev. :) Then you will have a pool with 2 mirror vdevs (similar to a RAID10 setup). Of course, you'll need to replace the ad10 disk to make it fully redundant. Something like: zpool attach cbackup3 adXX ad2 zpool detach cbackup3 ad10 zpool attach cbackup3 adYY ad6 > Two questions: > > - Does somebody know offhand what I did and what I should have done > instead to get ad2 into the mirror? > > You used "zpool add" (which adds new vdevs to the pool) instead of "zpool attach" (which attaches drives to an existing mirror vdev). Or, you used "zpool add" instead of "zpool replace". > - Would there have been a way to remove ad2 from this set? Nope. You cannot remove storage vdevs from a pool. You can only remove log and cache vdevs from a pool. -- Freddie Cash fjwcash@gmail.com