Date: Wed, 3 Jul 2024 16:31:20 -0700 From: Rick Macklem <rick.macklem@gmail.com> To: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net> Cc: FreeBSD CURRENT <freebsd-current@freebsd.org> Subject: Re: Booting in bhyve always sets currdev to ZFS Message-ID: <CAM5tNy7NScnyGyk1b=jnmAVy0waAUMcJ=835-vp5QwYTqJuwYQ@mail.gmail.com> In-Reply-To: <202407021430.462EUI7s017611@gndrsh.dnsmgr.net> References: <CAM5tNy7W71FBm3R10z4RFcb=akqLeCWBSYLbBFWjag-ENgG=JA@mail.gmail.com> <202407021430.462EUI7s017611@gndrsh.dnsmgr.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Jul 2, 2024 at 7:30=E2=80=AFAM Rodney W. Grimes
<freebsd-rwg@gndrsh.dnsmgr.net> wrote:
>
> > On Sun, Jun 30, 2024 at 8:23?AM Rodney W. Grimes
> > <freebsd-rwg@gndrsh.dnsmgr.net> wrote:
> > >
> > > > On Fri, Jun 28, 2024 at 6:26?PM Rick Macklem <rick.macklem@gmail.co=
m> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I've installed FreeBSD current in a bhvye instance. Everything
> > > > > went ok, with UFS as a root partition.
> > > > > Then I created a zpool in another partition...
> > > > > - Now, every time I boot it I have to
> > > > > OK set currdev=3Ddisk0s1a:
> > > > > to get it to boot.
> > > > >
> > > > > What is the trick to keep ZFS from messing up the boot variables?
> > > > I've been poking around, but haven't learned much.
> > > > I think it is userboot (although there are so many boot programs in=
/boot,
> > > > I am not 100% sure?) that sets currdev=3Dzfs:example: since it sees=
there is ZFS
> > > > in a partition on the drive. It is not the boot partition and doesn=
't have
> > > > any boot stuff in it.
> > > >
> > > > When I look at userboot, it appears that it always sets userboot_zf=
s_found
> > > > to 1 whenever userboot_zfs_probe() is called, given that there is a=
ZFS
> > > > partition with a pool on it. This makes extract_currdev get set to =
the
> > > > ZFS stuff,
> > > > assuming I am reading the code correctly.
> > > > What I do not understand is why I have not seen this before?
> > > > (Was the a change to building it with USERBOOT_ZFS_SUPPORT done?)
> > > >
> > > > It seems that userboot_zfs_probe() should check for boot files on t=
he volume
> > > > and not just that a pool exists on the partition, maybe?
> > > >
> > > > Anyhow, manually setting currdev=3Ddisk0s1a: gets around the proble=
m.
> > >
> > > You should be able to set that in your ufs partition /boot/loader.con=
f
> > > file and at least then your not having to drop to the OK prompt and
> > > manually entering this.
> > Doesn't help, because it is not reading files from the boot ufs file sy=
stem.
> > (I could copy all the boot stuff over into the ZFS file system, but tha=
t kinda
> > defeats the purpose of having UFS as the boot fs.)
>
> It had to of loaded userboot from the ufs, right?
I assume it is reading some boot file from UFS. Way back when I thought a
primary boot was read from 8K at the begining of the partition, but that's
how out of date I on this.
-> Put another way, I don't know if "userboot" is the one I am seeing try
to use the ZFS partition.
>
> What type of disk is this? gpt or mbr?
It's old MBR. Although I'll assume bhyve knows other stuff, most of my old
hardware only does MBR and not EFI, so I'll still in the MBR habit.
>
> Can you show the output of gpart show and gpart list?
root@nfsv4-foo:~ # gpart show
=3D> 63 67108801 vtbd0 MBR (32G)
63 1 - free - (512B)
64 67108800 1 freebsd [active] (32G)
=3D> 0 67108800 vtbd0s1 BSD (32G)
0 33554432 1 freebsd-ufs (16G)
33554432 12582912 2 freebsd-swap (6.0G)
46137344 20971456 4 freebsd-zfs (10G)
root@nfsv4-foo:~ # gpart list
Geom name: vtbd0
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 67108863
first: 63
entries: 4
scheme: MBR
Providers:
1. Name: vtbd0s1
Mediasize: 34359705600 (32G)
Sectorsize: 512
Stripesize: 32768
Stripeoffset: 0
Mode: r3w3e5
efimedia: HD(1,MBR,0x90909090,0x40,0x3ffffc0)
attrib: active
rawtype: 165
length: 34359705600
offset: 32768
type: freebsd
index: 1
end: 67108863
start: 64
Consumers:
1. Name: vtbd0
Mediasize: 34359738368 (32G)
Sectorsize: 512
Stripesize: 32768
Stripeoffset: 0
Mode: r3w3e8
Geom name: vtbd0s1
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 67108799
first: 0
entries: 8
scheme: BSD
Providers:
1. Name: vtbd0s1a
Mediasize: 17179869184 (16G)
Sectorsize: 512
Stripesize: 32768
Stripeoffset: 0
Mode: r1w1e1
rawtype: 7
length: 17179869184
offset: 0
type: freebsd-ufs
index: 1
end: 33554431
start: 0
2. Name: vtbd0s1b
Mediasize: 6442450944 (6.0G)
Sectorsize: 512
Stripesize: 32768
Stripeoffset: 0
Mode: r1w1e0
rawtype: 1
length: 6442450944
offset: 17179869184
type: freebsd-swap
index: 2
end: 46137343
start: 33554432
3. Name: vtbd0s1d
Mediasize: 10737385472 (10G)
Sectorsize: 512
Stripesize: 32768
Stripeoffset: 0
Mode: r1w1e1
rawtype: 27
length: 10737385472
offset: 23622320128
type: freebsd-zfs
index: 4
end: 67108799
start: 46137344
Consumers:
1. Name: vtbd0s1
Mediasize: 34359705600 (32G)
Sectorsize: 512
Stripesize: 32768
Stripeoffset: 0
Mode: r3w3e5
>
> >
> > For example, it always fails with:
> > ERROR: cannot open /boot/lua/loader.lua: no such file or directory.
> >
> > When I look in stand/efi/loader/main.c, there is a function that checks
> > for boot files called sanity_check_currdev(). However, userboot does no=
t
> > have a similar function and just seems to assume that the ZFS partition
> > in the bootable one, if it finds one.
>
> That sounds like a wrong assumption by the code.
>
> > --> I am wondering if sanity_check_currdev() shoud be added to userboot
> > to handle this case?
>
> Warner??
>
> > rick
> >
> > >
> > > There is much magic in boot code that bites, and not just in FreeBSD.
> > > Often assumptions are made that "this" boot code, and "this" OS are
> > > the only things on the disk(s).
> > >
> > > The fact that Linux uses a "apple-zfs" uuid, and FreeBSD a "FreeBSD-z=
fs"
> > > drove me nuts for 2 days until I found this little tid bit.
> > >
> > > Zpool import doesnt care, but the boot code does!
> > >
> > > --
> > > Rod Grimes rgrimes@fr=
eebsd.org
> --
> Rod Grimes rgrimes@freebs=
d.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAM5tNy7NScnyGyk1b=jnmAVy0waAUMcJ=835-vp5QwYTqJuwYQ>