Date: Mon, 18 Jan 2021 14:53:44 +0200 From: Andriy Gapon <avg@FreeBSD.org> To: John Delisle <jdelisle@gmail.com>, freebsd-fs <freebsd-fs@freebsd.org> Subject: Re: zpool remove not working for metadata special devices Message-ID: <92af08f3-342c-c98c-702f-54ee48bb089f@FreeBSD.org> In-Reply-To: <CAMdBLfR8VXAAK4H7UrEAMXaSNUo_55F=Cw17hx0fStd7A3inZQ@mail.gmail.com> References: <CAMdBLfQMdBfbc4qu8LLxL268fXicDiNjnw9FR5Df%2BHJ-GQ31sA@mail.gmail.com> <CAMdBLfR8VXAAK4H7UrEAMXaSNUo_55F=Cw17hx0fStd7A3inZQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2020-12-28 05:04, John Delisle wrote: > To experiment, I've completed this same procedure using Ubuntu 20.10 (on > the same "hardware", both are identically configured VMs in Azure) . > > The steps above work fine with ZFS on Ubuntu, and I can successfully remove > both data vdev mirrors and the special vdev mirror. This makes me think my > syntax is correct at least.. but for whatever reason the exact same > procedure fails on FreeBSD (both removing a mirror and removing the special > mirror fail with the same error on FreeBSD). I don't think that you did anything wrong and the syntax is certainly correct. I can reproduce the problem as well. Note that if you insert a reboot between the add special and the remove, then the latter works fine, at least it does for me. It looks like there is an issue with recording of spa_min_ashift internal parameter when a special device is first added. > On Sun, Dec 27, 2020 at 6:55 PM John Delisle <jdelisle@gmail.com> wrote: > >> I have a pool of mirrors, and added a mirrored special device. Although >> documentation suggests it should be removable, I cannot get zpool to do so. >> >> All top-level vdevs are mirrors, and all have the same sector size and >> ashift. No raidz. >> >> ## Current 12.2 p2 >> root@jmdtest:/ # freebsd-version >> 12.2-RELEASE-p2 >> >> ## Disk info >> root@jmdtest:/ # diskinfo /dev/da[2-9] /dev/da[0-9][0-9] >> /dev/da2 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da3 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da4 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da5 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da6 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da7 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da8 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da9 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da10 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da11 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da12 512 68719476736 134217728 4096 0 >> 8354 255 63 >> /dev/da13 512 68719476736 134217728 4096 0 >> 8354 255 63 >> >> >> ## Create the pool: >> zpool create nebula mirror da2 da3 mirror da4 da5 mirror da6 da7 mirror >> da8 da9 mirror da10 da11 >> >> ## Add the special mirror >> zpool add nebula special mirror da12 da13 >> >> ## zpool status >> root@jmdtest:/ # zpool list -v nebula >> NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP >> HEALTH ALTROOT >> nebula 1.30T 2.57G 1.30T - - 0% 0% 1.00x >> ONLINE - >> mirror 254G 492M 254G - - 0% 0.18% >> da2 - - - - - - - >> da3 - - - - - - - >> mirror 254G 501M 254G - - 0% 0.19% >> da4 - - - - - - - >> da5 - - - - - - - >> mirror 254G 596M 253G - - 0% 0.22% >> da6 - - - - - - - >> da7 - - - - - - - >> mirror 254G 481M 254G - - 0% 0.18% >> da8 - - - - - - - >> da9 - - - - - - - >> mirror 254G 561M 253G - - 0% 0.21% >> da10 - - - - - - - >> da11 - - - - - - - >> special - - - - - - >> mirror 63.5G 2.62M 63.5G - - 0% 0.00% >> da12 - - - - - - - >> da13 - - - - - - - >> root@jmdtest:/ # >> >> ## Remove the special mirror >> root@jmdtest:/ # zpool remove nebula mirror-5 >> cannot remove mirror-5: invalid config; all top-level vdevs must have the >> same sector size and not be raidz. >> >> ## All mirrors, and all the same ashift: >> root@jmdtest:/ # zdb -C | grep -e child -e ashift >> vdev_children: 6 >> children[0]: >> ashift: 12 >> children[0]: >> children[1]: >> children[1]: >> ashift: 12 >> children[0]: >> children[1]: >> children[2]: >> ashift: 12 >> children[0]: >> children[1]: >> children[3]: >> ashift: 12 >> children[0]: >> children[1]: >> children[4]: >> ashift: 12 >> children[0]: >> children[1]: >> children[5]: >> ashift: 12 >> children[0]: >> children[1]: >> root@jmdtest:/ # >> >> What am I doing wrong? -- Andriy Gapon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?92af08f3-342c-c98c-702f-54ee48bb089f>