Date: Fri, 10 Oct 2025 16:02:20 +0300 From: Daniel Braniss <danny@cs.huji.ac.il> To: Rick Macklem <rick.macklem@gmail.com> Cc: "Bjoern A. Zeeb" <bzeeb-lists@lists.zabbadoz.net>, Freebsd-stable List <stable@freebsd.org>, virtualization@freebsd.org Subject: Re: ALPHA-3 diskless + bhyve issues Message-ID: <C296F2BD-F80C-4798-BFA9-6E3D99C1634D@cs.huji.ac.il> In-Reply-To: <CAM5tNy4TzWw-7A6A4fcRjKkswXcunBVNjw5oZePTVHJ%2BNnj%2BgA@mail.gmail.com> References: <342F2780-9C83-4CEE-BE3E-AE72C9AE12BA@cs.huji.ac.il> <rp3o2r-413n-9p94-656n-32sp812037rn@mnoonqbm.arg> <639210F6-87B6-4B34-AB9B-8EAB46682B80@cs.huji.ac.il> <CAM5tNy4TzWw-7A6A4fcRjKkswXcunBVNjw5oZePTVHJ%2BNnj%2BgA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_B24AFD8D-C67C-4BAD-B1C7-AB53FB33034A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 10 Oct 2025, at 15:44, Rick Macklem <rick.macklem@gmail.com> wrote: >=20 > On Thu, Oct 9, 2025 at 11:15=E2=80=AFPM Daniel Braniss = <danny@cs.huji.ac.il <mailto:danny@cs.huji.ac.il>> wrote: >>=20 >>=20 >>=20 >> On 10 Oct 2025, at 00:33, Bjoern A. Zeeb = <bzeeb-lists@lists.zabbadoz.net> wrote: >>=20 >> On Tue, 23 Sep 2025, Daniel Braniss wrote: >>=20 >> So after many years, I tried to install instead of upgrading, >> and after much fiddling (if anyone is interested I can write how I = did i) I got it to >> install, now here is my problem: >> when the the local disk was empty, loader.efi is called (this via = dhcp) and it boots diskless, >> but now that the disk is populated, it boots from it. >>=20 >> Q: how can I have it boot diskless again? >>=20 >>=20 >> I haven't seen an answer. Have you tried the freebsd-virtualization = list? >>=20 >> I assume you could interrupt the boot process if fast enough (but = it's the blink of an eye); >> so you'll most likely be more successful putting a startup.nsh and = possibly an EFI shell in the EFI System Partition (ESP) and script it? = Could be the bhyve firmware brings the efi shell along already? >>=20 >> /bz >>=20 >> -- >> Bjoern A. Zeeb = r15:7 >>=20 >>=20 >> hi, >> no answer yet, but after some experimenting: >> boots diskless: >> if no local disk - dah >> if local disk and no EFI partition >> if local disk has the above but no valid root partition >> and finally if root partition and efi but no bootx64.efi ( i just = move it to bootx64.efix) > Can you try the case where there is a non-bootable ZFS partition? >=20 > When I have a non-bootable ZFS partition with a zpool in it, bhyve = decides that > is the root and fails to boot the UFS root fs. I need to: >=20 > currdev=3Ddisk0s1 > boot >=20 > to get it to boot. >=20 > I don't know if this is what you might be seeing? rick Not really, in the past I could change from diskless to local root by = changing the filename option in the dhcp, so that if filename is loader.efi it would boot diskless, and if = boot1.efi it would boot off the local disk. Now loader.efi will first try and boot from the local disk. Btw, I think = this is documented. Cheers, Danny >=20 >>=20 >> thanks, >> danny >> ps: i guess i forgot to mention that the host is running in bhyve = (14.3) >> pass: it=E2=80=99s now alpha-5 --Apple-Mail=_B24AFD8D-C67C-4BAD-B1C7-AB53FB33034A Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html aria-label=3D"message body"><head><meta http-equiv=3D"content-type" = content=3D"text/html; charset=3Dutf-8"></head><body = style=3D"overflow-wrap: break-word; -webkit-nbsp-mode: space; = line-break: after-white-space;"><br = id=3D"lineBreakAtBeginningOfMessage"><div><br><blockquote = type=3D"cite"><div>On 10 Oct 2025, at 15:44, Rick Macklem = <rick.macklem@gmail.com> wrote:</div><br = class=3D"Apple-interchange-newline"><div><meta charset=3D"UTF-8"><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;">On Thu, Oct 9, 2025 at 11:15=E2=80=AFPM = Daniel Braniss <</span><a href=3D"mailto:danny@cs.huji.ac.il" = style=3D"font-family: Helvetica; font-size: 14px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px;">danny@cs.huji.ac.il</a><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;">> wrote:</span><br style=3D"caret-color: = rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: = normal; font-variant-caps: normal; font-weight: 400; letter-spacing: = normal; text-align: start; text-indent: 0px; text-transform: none; = white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><blockquote type=3D"cite" style=3D"font-family: = Helvetica; font-size: 14px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; orphans: auto; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><br><br><br>On 10 Oct 2025, at 00:33, Bjoern A. = Zeeb <bzeeb-lists@lists.zabbadoz.net> wrote:<br><br>On Tue, 23 Sep = 2025, Daniel Braniss wrote:<br><br>So after many years, I tried to = install instead of upgrading,<br>and after much fiddling (if anyone is = interested I can write how I did i) I got it to<br>install, now here is = my problem:<br>when the the local disk was empty, loader.efi is called = (this via dhcp) and it boots diskless,<br>but now that the disk is = populated, it boots from it.<br><br>Q: how can I have it boot diskless = again?<br><br><br>I haven't seen an answer. Have you tried the = freebsd-virtualization list?<br><br>I assume you could interrupt the = boot process if fast enough (but it's the blink of an eye);<br>so you'll = most likely be more successful putting a startup.nsh and possibly an EFI = shell in the EFI System Partition (ESP) and script it? Could be the = bhyve firmware brings the efi shell along = already?<br><br>/bz<br><br>--<br>Bjoern A. Zeeb = &n= bsp; &nbs= p; = &n= bsp; r15:7<br><br><br>hi,<br>no answer yet, but after some = experimenting:<br>boots diskless:<br>if no local disk - dah<br>if local = disk and no EFI partition<br>if local disk has the above but no valid = root partition<br>and finally if root partition and efi but no = bootx64.efi ( i just move it to bootx64.efix)<br></blockquote><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;">Can you try the case where there is a = non-bootable ZFS partition?</span><br style=3D"caret-color: rgb(0, 0, = 0); font-family: Helvetica; font-size: 14px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><br style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 14px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 14px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">When I = have a non-bootable ZFS partition with a zpool in it, bhyve decides = that</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 14px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><span style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 14px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none; float: none; display: inline !important;">is the root and fails to = boot the UFS root fs. I need to:</span><br style=3D"caret-color: rgb(0, = 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><br style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 14px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 14px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline = !important;">currdev=3Ddisk0s1</span><br style=3D"caret-color: rgb(0, 0, = 0); font-family: Helvetica; font-size: 14px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 14px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline = !important;">boot</span><br style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 14px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><br style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 14px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 14px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">to get = it to boot.</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 14px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><br style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 14px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><span style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 14px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none; float: none; display: inline !important;">I don't know if this is = what you might be seeing? rick</span><br style=3D"caret-color: rgb(0, 0, = 0); font-family: Helvetica; font-size: 14px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: = none;"></div></blockquote><div><br></div><div><br></div>Not really, in = the past I could change from diskless to local root by changing the = filename option in the dhcp,</div><div>so that if filename is loader.efi = it would boot diskless, and if boot1.efi it would boot off the local = disk.</div><div>Now loader.efi will first try and boot from the local = disk. Btw, I think this is documented.</div><div>Cheers,</div><div><span = class=3D"Apple-tab-span" style=3D"white-space:pre"> = </span>Danny</div><div><br><blockquote type=3D"cite"><div><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><blockquote = type=3D"cite" style=3D"font-family: Helvetica; font-size: 14px; = font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; orphans: auto; text-align: start; text-indent: = 0px; text-transform: none; white-space: normal; widows: auto; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><br>thanks,<br>danny<br>ps: i guess i forgot to mention that the = host is running in bhyve (14.3)<br>pass: it=E2=80=99s now = alpha-5</blockquote></div></blockquote></div><br></body></html>= --Apple-Mail=_B24AFD8D-C67C-4BAD-B1C7-AB53FB33034A--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C296F2BD-F80C-4798-BFA9-6E3D99C1634D>