Date: Sun, 7 Jan 2024 16:38:04 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> To: lev@FreeBSD.org, freebsd-fs <freebsd-fs@freebsd.org>, freebsd-stable <freebsd-stable@freebsd.org>, Alexander Motin <mav@FreeBSD.org> Subject: Re: FreeBSD 13.2-STABLE can not boot from damaged mirror AND pool stuck in "resilver" state even without new devices. Message-ID: <cc136316-f285-41bd-8d59-c5adce06e277@quip.cz> In-Reply-To: <e74464be-09b6-43e2-9365-7b0271b2d6eb@FreeBSD.org> References: <f97d80ee-0b01-4d68-beb5-53e905f0404c@FreeBSD.org> <e74464be-09b6-43e2-9365-7b0271b2d6eb@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 07/01/2024 15:49, Lev Serebryakov wrote:
> On 05.01.2024 18:28, Lev Serebryakov wrote:
>
>> After that my server fails to boot, gtpzfsboot from second disk
>> (ada1) reports several "zio_read error: 5" and
>>
>> ZFS: i/o error - all block copies unavailable
>> ZFS: can't read MOS of pool zroot
>>
>> after that.
> I've re-created pool from scratch
>
> zpool create znewroot ada0p3 && zfs send zroot | zfs receive znewroot
> && zpool destroy zroot && zpool attach znewroot ada0p3 ada1p3
>
> but gptzfsboot still can not boot from it with same diagnostics :-(
How large are your disks in a question?
I was bitten by this not a long time ago when migrating my 2TB pool by
zfs send to larger disks (4TB), then I see the error:
ZFS: i/o error - all block copies unavailable
ZFS: can't read MOS of pool zroot
As far as I search the internet it is caused by the boot code (later
stage which is in a file in /boot directory) was moved too far from the
beginning of the disk and some old BIOS cannot allow the system to
continue booting. I am not a boot expert so my words can be wrong but I
hope you get the point. It can be result of the system update, or zfs
send | zfs recv. In my case the pool was unbootable in HP Miniserver Ge
8 but boots perfectly fine in an old Supermicro with X9SCA-F board. The
problem is not in a pool, nor disks, nor FreeBSD but in a BIOS.
I solved it by creating new mirrored pool of the size about 40GB at the
beginning of the disks (40GB GPT partition for freebsd-zfs) where I
installed the FreeBSD system and next freebsd-zfs partition covering the
rest of the 4TB disks for data storage. Everything works fine.
You can also have just a small /boot partition for the boot and later
overlayed by main ZFS pool, but it seems to me as bad for maintaining.
example of my partitions layout
# gpart show -p
=> 40 7814037088 ada0 GPT (3.6T)
40 1024 ada0p1 freebsd-boot (512K)
1064 40960 ada0p2 efi (20M)
42024 83886080 ada0p3 freebsd-zfs (40G)
83928104 20971520 ada0p4 freebsd-swap (10G)
104899624 7707033600 ada0p5 freebsd-zfs (3.6T)
7811933224 2103904 - free - (1.0G)
It can also be avoided if your machine supports EFI boot, but my HP
Microserver Gen 8 does not support it.
Kind regards
Miroslav Lachman
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?cc136316-f285-41bd-8d59-c5adce06e277>