Date: Sun, 7 Jan 2024 21:56:59 +0100 From: Lev Serebryakov <lev@FreeBSD.org> To: Warner Losh <imp@bsdimp.com> Cc: freebsd-fs <freebsd-fs@freebsd.org>, freebsd-stable <freebsd-stable@freebsd.org> Subject: Re: FreeBSD 13.2-STABLE can not boot from damaged mirror AND pool stuck in "resilver" state even without new devices. Message-ID: <2f91eeb7-430b-49e2-817b-5acd0f445fe9@FreeBSD.org> In-Reply-To: <d11ffb2e-0ee8-4c20-b5d9-5ea63463adba@FreeBSD.org> References: <f97d80ee-0b01-4d68-beb5-53e905f0404c@FreeBSD.org> <e74464be-09b6-43e2-9365-7b0271b2d6eb@FreeBSD.org> <cc136316-f285-41bd-8d59-c5adce06e277@quip.cz> <065f4f5c-f38b-45f4-b7e7-5248f871f7e6@FreeBSD.org> <CANCZdfrYCk7%2B6wCALvszmNZOcZeDxxNp%2Bk5PyH%2BTGJZ%2BovsU=Q@mail.gmail.com> <d11ffb2e-0ee8-4c20-b5d9-5ea63463adba@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 07.01.2024 21:49, Lev Serebryakov wrote: > On 07.01.2024 19:34, Warner Losh wrote: > >> I must have missed it. What were the diagnostics? Oh, and two "nvlist inconsistency" before that vvvv > zio_read error: 5 > zio_read error: 5 > zio_read error: 5 > ZFS: i/o error - all block copies unavailable > ZFS: can't read MOS of pool zroot > > > To be honest, I thinks there is something else. Because sequence of events were (sorry, too long, but I think, tht every detail matters here): > > (1) Update to 13.2 from 12.4. With installation of new gptzfsboot with gpart on both disks. It could place new /boot far away, but see (2) > (2) Reboot, which completed, but showed that ada0 has problems > (3) Replacement of ada0 by DC technicians, new disk is 512/4096, old disk is 512/512, pool has ashift=9 > (4) Server refuses to boot from ada1 (ada0 is empty) with diagnostics (see above) > (5) Linux rescue system, passing 2 devices to qemu with FreeBSD (because Linux shows that ZFS is on whole disk, not on partition!). > (6) Re-creation of GPT on ada0, start of resilver (with sub-optimal ashift!). > (7) Interruption of resilver with reboot, because it is painfully slow under qemu. > (8) Wipe of ada0 (at this point resilver status of pool becomes crazy) to put live FreeBSD image to boot somehow. > (9) Many tries to cancel resilver and boot from single-disk "historical" pool on ada1, no success. I've attributed it to the strange state of pool: one component, no mirrior, but "resilvering". > (10) Boot from small UFS partition (which replaces swap partition). > (11) Pool on ada1 (old, live, 512/512 disk) is still "Reslivering" without any additional components (with zero speed, of course). > (12) Prepare partitions on ada0 again, creating new pool with ashift=12, send|receive. > (13) Removing partition table on ada1 (with old pool, ashift=9, still resilvering after many-many reboots with only one device in it). And pleas note: this pool on ada1 (old, live disk) was NOT upgraded after 12-STABLE. It was old, 12-STABLE "level" pool with all new features disabled. -- // Lev Serebryakov
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2f91eeb7-430b-49e2-817b-5acd0f445fe9>