Date: Fri, 19 Jun 2015 15:30:51 -0400 From: Emre Gundogan <emre@gundogan.us> To: freebsd-questions@freebsd.org Subject: Re: Expanding zfs+geli pool Message-ID: <55846DEB.6040905@gundogan.us> In-Reply-To: <55843CF1.1070709@gundogan.us> References: <55843CF1.1070709@gundogan.us>
next in thread | previous in thread | raw e-mail | index | archive | help
Thanks a lot, Matthew.I was able to replace two of the disks (disk2 and
disk4) with that method successfully, so I had the pool in the following
state:
pool:
mirror1:
disk1 (2TB)
disk2 (4TB)
mirror2
disk3 (2TB)
disk4 (4TB)
Then, I scrubbed the pool (no errors) and replaced 'disk1'. That's when
the whole pool became 'UNAVAIL' when I brought the machine back up. In
fact regardless of which of the remaining 2TB disks I replaced (disk1 or
disk3) at that point resulted in pool becoming 'UNAVAIL'. I wish I could
tell why, maybe, it's got something to do with the GELI layer, or the
fact that the new disks are 4K advanced format while the old ones are
512-byte logical/physical (although mixed vdevs should be OK from what I
understand), the pool was correctly marked as ashift=12 based on the
output from zdb. Given that I can only afford a limited downtime on this
pool, I went to Plan-B to 'zfs send' the pool snapshot to another
machine, re-create the pool with the 4TB disks, and 'zfs receive' from
the backup machine.So I am back to the state I mentioned above (half
2TB, half 4TB), the pool is 'ONLINE', and the 'zfs send' is going on for
few hours...
Thanks again,
Emre.
> So long as you ensure that only one drive out of each vdev is replaced
> at a time, and that resilvering has completed before you replace the
> other one, then, subject to those constraints, the ordering doesn't
> matter. The order you describe would work fine.
>
> Ideally, yes, adding the new drives to the mirror before removing the
> old ones would be better, but as your hardware doesn't support that,
> you're going to have to accept a period of lower resilience while all of
> the resilvering goes on.
>
> Yes, making each mirrored vdev contain on new and one older drive would
> be sensible.
>
> Check the setting of the 'autoexpand' property on the zpool before you
> begin. I your shoes I'd set autoexpand=off and then issue an explicit
>
> zpool online -e pool disk1 disk2 disk3 disk4
>
> after all the resilvering is done so that you're in control of when the
> expanded space becomes available.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?55846DEB.6040905>