Date: Thu, 11 Aug 2011 09:51:40 -0700 From: Jeremy Chadwick <freebsd@jdc.parodius.com> To: Martin Cracauer <cracauer@cons.org> Cc: freebsd-fs@freebsd.org Subject: Re: ZFS zpool mirror drive replacement confusion Message-ID: <20110811165140.GA1421@icarus.home.lan> In-Reply-To: <20110811160314.GA25076@cons.org> References: <20110811160314.GA25076@cons.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Aug 11, 2011 at 12:03:14PM -0400, Martin Cracauer wrote:
> My computer keeps doing what I say, not what I want :-)
First point: OP is using ZFSv15 or earlier. I can tell from the mirror
vdev (not pool) being called "mirror", not "mirror-0". Readers should
take note of that fact.
To the OP: it's very important you disclose exactly what FreeBSD version
you're using when submitting help requests. "I use FreeBSD 8" is
insufficient; providing uname -a is always best.
> This zpool with a 2-drive mirror had a dead drive (ad10) which I
> removed. After I tried to put ad2 as a replacement drive I ended up
> with the drive not in the mirror, it seems to have extended the zpool
> (it's size?).
>
> NAME STATE READ WRITE CKSUM
> cbackup3 DEGRADED 0 0 0
> mirror DEGRADED 0 0 0
> ad6 ONLINE 0 0 0
> ad10 REMOVED 0 0 0
> ad2 ONLINE 0 0 11
The pool "grew" because your pool is now, effectively, a stripe (RAID-0)
of two disks (ad6 and ad2). Keep reading.
> Unfortunately I didn't keep note of the specific command that I used.
>
> Two questions:
>
> - Does somebody know offhand what I did and what I should have done
> instead to get ad2 into the mirror?
I'm pretty sure I know what happened, but the problem is that I'm having
severe difficulty phrasing the explanation. Adding to the annoyance is
the fact that you did not bother to keep track of the commands you
issued.
The problem is something I have run into myself many times. It's a
syntax-specific "quirk" with the zpool command, and ONLY applies when
working with mirrors. Furthermore, the "gotcha" may be gone with ZFSv28
(which you aren't running); I'm not sure.
The "quirk" is that you have to specify the name of an existing
(working) mirror device *before* the new device you're attempting to
integrate. E.g. "zpool {command} pool {existingdev} {newdev}", rather
than "zpool {command} pool {newdev}". Failure to provide the name of
the device of the existing mirror member results in the pool becoming
a stripe, which you have no way to recover from without recreating the
entire pool.
I would show you an example but all my systems run ZFSv28 and I can't
seem to find any online documentation which helps back up my statement.
I hope someone else here can phrase what I'm trying to say in a more
eloquent manner and provide documentation/references confirming it.
> - Would there have been a way to remove ad2 from this set? It didn't
> let me, saying it can only remove drives from mirror or raid sets.
> I think it should have been possible before I mounted the filesystem
> in there read-write for the first time after the drive add. This
> seems like an easy mistake to make.
What you have right now is a pool that consists of both a mirror
(ad6+ad10), as well as a stripe (the mirror + ad2). As you know, stripe
means the equivalent of RAID-0; lose the disk and you lose your pool.
You will need to do this, given that you said "the array doesn't contain
important data":
# zpool destroy cbackup3
# zpool create cbackup3 mirror ad2 ad10
In the future please be sure to take note of all the commands you do, as
well as any physical actions you take (yanking disks, etc.) between
commands.
ZFS is amazing and awesome, and the power of the zpool command is
unmatched. However, I will agree that the likelihood of a mistake being
made is quite high given that its commands use words which could mean
the same thing as one another; "add" vs. "attach", "remove" vs.
"detach", etc.. Yet they differ severely, and issuing the wrong one can
be fatal in some situations.
--
| Jeremy Chadwick jdc at parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, US |
| Making life hard for others since 1977. PGP 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110811165140.GA1421>