Skip site navigation (1)Skip section navigation (2) 
Date:      Mon, 13 Jul 2020 13:36:46 -0400
From:      Allan Jude <allanjude@freebsd.org>
To:        Christian Kratzer <ck@cksoft.de>
Cc:        freebsd-fs@freebsd.org
Subject:   Re: gptzfsboot targeting wrong vdev
Message-ID:  <d1b537da-8a94-4e12-4d45-b4318db2fbb8@freebsd.org>
In-Reply-To: <alpine.BSF.2.22.395.2007131155560.82939@nocfra1.cksoft.de>
References:  <alpine.BSF.2.22.395.2007061453250.82939@nocfra1.cksoft.de> <9400f5f0-e267-932c-b1ce-8436748cf2c0@FreeBSD.org> <78024f0d-4889-713e-15a5-56ec6d8d82b3@freebsd.org> <alpine.BSF.2.22.395.2007131155560.82939@nocfra1.cksoft.de>
next in thread | previous in thread | raw e-mail | index | archive | help 
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--Upx5m2iDaQcpDAsFa9BiJ76FZdTCNyK6P
Content-Type: multipart/mixed; boundary="ADmQm5kpvKza7Iik7dVjnz5Jx8pwakhwM";
 protected-headers="v1"
From: Allan Jude <allanjude@freebsd.org>
To: Christian Kratzer <ck@cksoft.de>
Cc: freebsd-fs@freebsd.org
Message-ID: <d1b537da-8a94-4e12-4d45-b4318db2fbb8@freebsd.org>
Subject: Re: gptzfsboot targeting wrong vdev
References: <alpine.BSF.2.22.395.2007061453250.82939@nocfra1.cksoft.de>
 <9400f5f0-e267-932c-b1ce-8436748cf2c0@FreeBSD.org>
 <78024f0d-4889-713e-15a5-56ec6d8d82b3@freebsd.org>
 <alpine.BSF.2.22.395.2007131155560.82939@nocfra1.cksoft.de>
In-Reply-To: <alpine.BSF.2.22.395.2007131155560.82939@nocfra1.cksoft.de>

--ADmQm5kpvKza7Iik7dVjnz5Jx8pwakhwM
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable

On 2020-07-13 06:02, Christian Kratzer wrote:
> Hi,
>=20
> On Tue, 7 Jul 2020, Allan Jude wrote:
>=20
>> On 2020-07-07 02:21, Andriy Gapon wrote:
>>> On 06/07/2020 16:10, Christian Kratzer wrote:
> <snipp/>
>>>> When booting from ada0 I get following:
>>>>
>>>> =C2=A0=C2=A0=C2=A0=C2=A0ZFS: i/o error - all block copies unavailabl=
e
>>>> =C2=A0=C2=A0=C2=A0=C2=A0ZFS: can't read MOS of pool zp1
>>>> =C2=A0=C2=A0=C2=A0=C2=A0gptzfsboot: failed to mount default pool zp1=

>>>>
>>>> =C2=A0=C2=A0=C2=A0=C2=A0FreeBSD/x86 boot
>>>>
>>
>> So, just to be clear, at this point you have not loaded the boot loade=
r
>> yet. You are in the bootstrap (gptzfsboot), and it is unable to load t=
he
>> loader.
>=20
>=20
> Sorry for the delay in answering.=C2=A0 I got around to some testing ag=
ain.
>=20
> Thanks for clarifying this. It does make a lot more sense now.
>=20
>=20
>> I think it just looks at the first 'freebsd-zfs' type'd partition.
>> However, if zp1 is GELI encrypted, it shouldn't be able to even tell t=
he
>> name of the pool.
>=20
> That is what I have also been confused about.=C2=A0 It should not have =
access
> to the zpool.cache and and should have no way of seeing zp1 which is
> in geli encrypted da0p1 and da1p2.
>=20
> The loader.conf contains the keys to those which come visbile when the
> kernel starts.
>=20
> So I was pretty confused on where it was getting the name zp1 from whic=
h
> to my undestanding should not be visible anywhere.
>=20
>> You might try changing the partition type of the boots you are not
>> booting from, to 'freebsd-vinum' or something other than 'freebsd-zfs'=

>> so that gptzfsboot only sees 1 'freebsd-zfs' to boot from.
>=20
>=20
> Before trying that I tried setting the bootme property on the root
> partitions
> on ada0 and ada1.
>=20
> That did not help either so I proceeded to set all but the zroot
> partition type to freebsd-vinum
>=20
> =C2=A0=C2=A0=C2=A0=C2=A0root@zfs1:/home/ck # gpart show
> =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
40=C2=A0 7814037088=C2=A0 da0=C2=A0 GPT=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr=
ee -=C2=A0 (2.0M)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2=
=A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8=
M)
>=20
> =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
40=C2=A0 7814037088=C2=A0 da1=C2=A0 GPT=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr=
ee -=C2=A0 (2.0M)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2=
=A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8=
M)
>=20
> =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
40=C2=A0 7814037088=C2=A0 da2=C2=A0 GPT=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr=
ee -=C2=A0 (2.0M)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2=
=A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8=
M)
>=20
> =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
40=C2=A0 7814037088=C2=A0 da3=C2=A0 GPT=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr=
ee -=C2=A0 (2.0M)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2=
=A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8=
M)
>=20
> =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
40=C2=A0 7814037088=C2=A0 da4=C2=A0 GPT=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr=
ee -=C2=A0 (2.0M)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2=
=A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8=
M)
>=20
> =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
40=C2=A0 7814037088=C2=A0 da5=C2=A0 GPT=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr=
ee -=C2=A0 (2.0M)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2=
=A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8=
M)
>=20
> =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
40=C2=A0 7814037088=C2=A0 da6=C2=A0 GPT=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr=
ee -=C2=A0 (2.0M)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2=
=A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8=
M)
>=20
> =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
40=C2=A0 7814037088=C2=A0 da7=C2=A0 GPT=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr=
ee -=C2=A0 (2.0M)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2=
=A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8=
M)
>=20
> =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=
 468862048=C2=A0 ada0=C2=A0 GPT=C2=A0 (224G)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 1024=C2=A0=C2=A0=C2=A0=C2=A0 1=C2=A0 freebsd-boot=C2=A0 (=
512K)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1064=C2=A0=
 134217728=C2=A0=C2=A0=C2=A0=C2=A0 2=C2=A0 freebsd-swap=C2=A0 (64G)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 134218792=C2=A0=C2=A0 33554432=C2=A0=C2=A0=
=C2=A0=C2=A0 3=C2=A0 freebsd-vinum=C2=A0 (16G)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 167773224=C2=A0=C2=A0 33554432=C2=A0=C2=A0=
=C2=A0=C2=A0 4=C2=A0 freebsd-vinum=C2=A0 (16G)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 201327656=C2=A0 267534424=C2=A0=C2=A0=C2=
=A0=C2=A0 5=C2=A0 freebsd-zfs=C2=A0 [bootme]=C2=A0 (128G)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 468862080=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 8=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free =
-=C2=A0 (4.0K)
>=20
> =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=
 468862048=C2=A0 ada1=C2=A0 GPT=C2=A0 (224G)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 1024=C2=A0=C2=A0=C2=A0=C2=A0 1=C2=A0 freebsd-boot=C2=A0 (=
512K)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1064=C2=A0=
 134217728=C2=A0=C2=A0=C2=A0=C2=A0 2=C2=A0 freebsd-swap=C2=A0 (64G)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 134218792=C2=A0=C2=A0 33554432=C2=A0=C2=A0=
=C2=A0=C2=A0 3=C2=A0 freebsd-vinum=C2=A0 (16G)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 167773224=C2=A0=C2=A0 33554432=C2=A0=C2=A0=
=C2=A0=C2=A0 4=C2=A0 freebsd-vinum=C2=A0 (16G)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 201327656=C2=A0 267534424=C2=A0=C2=A0=C2=
=A0=C2=A0 5=C2=A0 freebsd-zfs=C2=A0 [bootme]=C2=A0 (128G)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 468862080=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 8=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free =
-=C2=A0 (4.0K)
>=20
> That helped.=C2=A0 The system now finally boots from the mirror on ada0=
p5 and
> ada1p5.
>=20
>=20
> Unclear why setting the bootme property did not help on its own.
>=20
>=20
> Thanks Christian
>=20

bootme is only used by gptboot, for UFS.

You might want to example your disks with 'zdb -l /dev/ada0' (for each
disk, and each partition). It seems there is something not encrypted
somewhere. It may be an old label from a pool that has been destroyed,
and a new pool with the same name was later created or something.

--=20
Allan Jude


--ADmQm5kpvKza7Iik7dVjnz5Jx8pwakhwM--

--Upx5m2iDaQcpDAsFa9BiJ76FZdTCNyK6P
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (MingW32)

iQIcBAEBAgAGBQJfDJuuAAoJEBmVNT4SmAt+VUAQAJo6R4/1QGtnigzZgZADqqsp
8LD/jBN3CsKmWIOfOLOb777YIlctU12ltxWujGdaPSztpV43mKzfJ4RTrBzVvJzh
KS13naw0jvLxnUrzMsSV55wC9SY2NI0WtSD0ZgKxLeU+YCx+ru7wsex3Ph6eQziL
3qKdInKtjaUU1cp8XteU9zZXS1E1TJj6BHYncTQp2+/fgAUKgAC9Kw61N1vG0P34
d1yAoGa+9yg5KbaceIMDsotHE1KF5unZAofHt1RH43wIJAOcVV5UsImcPF0HEgK+
dB7jZT2k55Pm01YVlhSoVXYBMu5LCZnCiVnDMpXzbqdsSYwamYDv52H0p3dseof6
JVxQNz35ao4FdlkJ+yVxp85HnB9xh+FNsyO4ofZiUxekjyexu3AvDRcF50dJXCL1
HWThCQf0knXSV2aysbCmo8qezDcyZbyd4RgvqnJcJPFt1uJKUcQj4EpXmjIJFOT3
DM9xqcse7cBTEAwdp2693In+F5dAA53HXSa9rub8Z3lNQzXY/RKAI8Eag9ojBK3f
N79JA97bhOgizk1N2zIlQjxIbZouDtWk/wP9bsesaBXQ5FydJwlEvMEUsUHdTz7u
rBMkZK7Dq7XOfbOgQoJCc/g5eEICrWdO+2r/Mfok9a+hj7Mzu5HRuI1Y3Rn4tfte
YPFHEt5rErRTe4vyzVyQ
=2YHU
-----END PGP SIGNATURE-----

--Upx5m2iDaQcpDAsFa9BiJ76FZdTCNyK6P--

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?d1b537da-8a94-4e12-4d45-b4318db2fbb8>