Date: Fri, 5 Jan 2024 18:28:55 +0100 From: Lev Serebryakov <lev@FreeBSD.org> To: freebsd-fs <freebsd-fs@freebsd.org>, freebsd-stable <freebsd-stable@freebsd.org>, Alexander Motin <mav@FreeBSD.org> Subject: FreeBSD 13.2-STABLE can not boot from damaged mirror AND pool stuck in "resilver" state even without new devices. Message-ID: <f97d80ee-0b01-4d68-beb5-53e905f0404c@FreeBSD.org>
next in thread | raw e-mail | index | archive | help
Hello!
I have (remote) physical server with 2 SATA disks. These disks were partitioned with GPT into "freebsd-boot" (ada{0|1}p1, legacy one, not EFI), "freebsd-swap" (ada{0|1}p2) and "freebsd-zfs" (ada{0|1}p3).
Both disks were 512/512 (it looks important).
I have only one ZFS pool "zroot", mirror of "ada0p3" and "ada1p3".
I have very fresh "gptzfsboot" on both "ada0p1" and "ada1p1".
Now, ada0 failed. It was replaced by DC support with new disk, which is 512/4096.
After that my server fails to boot, gtpzfsboot from second disk (ada1) reports several "zio_read error: 5" and
ZFS: i/o error - all block copies unavailable
ZFS: can't read MOS of pool zroot
after that.
I've booted to rescue Linux (unfortunately, there is NO rescue FreeBSD at Hetzner anymore), and Linux could import (degraded) pool no problem. But Linux has problems with detecting pool on partition, so I don't do nothing under Linux.
I've checked "live" disk under Linux, though: it reads, SMART is clear, everything is Ok.
I've booted FreeBSD 13.2 from installation ISO under qemu with physical devices as disks. Then I partitioned fresh HDD and started disk replacement in mirror. It worked, but resilver was unbearable slow. I stopped VM with FreeBSD to continue process after normal boot.
NO LUCK. "zio_read error: 5", boot failed.
Then I've overwrite ada0 (new disk) with FreeBSD memstick IMG and boot it - it can import pool from ada1p3 but, of course, resilver is stopped.
I've removed all faulted components, effectivly converting mirror to "simple" device. But "zpool status" shows that there is resilver!
And "gptzfsboot" still CAN NOT read this ZFS pool and find loader!
Ok, I've converted swap to UFS boot form UFS. It works. It can use pool as root. But pool still is "reslivering".
Now I have very strange situation:
(1) I have ZFS pool with 1 device which says:
% zpool status -v zroot
pool: zroot
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Fri Jan 5 19:24:07 2024
750G scanned at 472B/s, 40.5G issued at 25B/s, 974G total
0B resilvered, 4.16% done, no estimated completion time
config:
NAME STATE READ WRITE CKSUM
zroot ONLINE 0 0 0
ada1p3 ONLINE 0 0 0
errors: No known data errors
%
(2) gtpzfsboot from very this system version can not read this pool and bot from it
(3) kernel can use this pool as source of root (and all other) filesystems.
--
// Lev Serebryakov
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?f97d80ee-0b01-4d68-beb5-53e905f0404c>