Date: Fri, 10 Oct 2025 16:02:20 +0300 From: Daniel Braniss <danny@cs.huji.ac.il> To: Rick Macklem <rick.macklem@gmail.com> Cc: "Bjoern A. Zeeb" <bzeeb-lists@lists.zabbadoz.net>, Freebsd-stable List <stable@freebsd.org>, virtualization@freebsd.org Subject: Re: ALPHA-3 diskless + bhyve issues Message-ID: <C296F2BD-F80C-4798-BFA9-6E3D99C1634D@cs.huji.ac.il> In-Reply-To: <CAM5tNy4TzWw-7A6A4fcRjKkswXcunBVNjw5oZePTVHJ%2BNnj%2BgA@mail.gmail.com> References: <342F2780-9C83-4CEE-BE3E-AE72C9AE12BA@cs.huji.ac.il> <rp3o2r-413n-9p94-656n-32sp812037rn@mnoonqbm.arg> <639210F6-87B6-4B34-AB9B-8EAB46682B80@cs.huji.ac.il> <CAM5tNy4TzWw-7A6A4fcRjKkswXcunBVNjw5oZePTVHJ%2BNnj%2BgA@mail.gmail.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] > On 10 Oct 2025, at 15:44, Rick Macklem <rick.macklem@gmail.com> wrote: > > On Thu, Oct 9, 2025 at 11:15 PM Daniel Braniss <danny@cs.huji.ac.il <mailto:danny@cs.huji.ac.il>> wrote: >> >> >> >> On 10 Oct 2025, at 00:33, Bjoern A. Zeeb <bzeeb-lists@lists.zabbadoz.net> wrote: >> >> On Tue, 23 Sep 2025, Daniel Braniss wrote: >> >> So after many years, I tried to install instead of upgrading, >> and after much fiddling (if anyone is interested I can write how I did i) I got it to >> install, now here is my problem: >> when the the local disk was empty, loader.efi is called (this via dhcp) and it boots diskless, >> but now that the disk is populated, it boots from it. >> >> Q: how can I have it boot diskless again? >> >> >> I haven't seen an answer. Have you tried the freebsd-virtualization list? >> >> I assume you could interrupt the boot process if fast enough (but it's the blink of an eye); >> so you'll most likely be more successful putting a startup.nsh and possibly an EFI shell in the EFI System Partition (ESP) and script it? Could be the bhyve firmware brings the efi shell along already? >> >> /bz >> >> -- >> Bjoern A. Zeeb r15:7 >> >> >> hi, >> no answer yet, but after some experimenting: >> boots diskless: >> if no local disk - dah >> if local disk and no EFI partition >> if local disk has the above but no valid root partition >> and finally if root partition and efi but no bootx64.efi ( i just move it to bootx64.efix) > Can you try the case where there is a non-bootable ZFS partition? > > When I have a non-bootable ZFS partition with a zpool in it, bhyve decides that > is the root and fails to boot the UFS root fs. I need to: > > currdev=disk0s1 > boot > > to get it to boot. > > I don't know if this is what you might be seeing? rick Not really, in the past I could change from diskless to local root by changing the filename option in the dhcp, so that if filename is loader.efi it would boot diskless, and if boot1.efi it would boot off the local disk. Now loader.efi will first try and boot from the local disk. Btw, I think this is documented. Cheers, Danny > >> >> thanks, >> danny >> ps: i guess i forgot to mention that the host is running in bhyve (14.3) >> pass: it’s now alpha-5 [-- Attachment #2 --] <html aria-label="message body"><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><br id="lineBreakAtBeginningOfMessage"><div><br><blockquote type="cite"><div>On 10 Oct 2025, at 15:44, Rick Macklem <rick.macklem@gmail.com> wrote:</div><br class="Apple-interchange-newline"><div><meta charset="UTF-8"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">On Thu, Oct 9, 2025 at 11:15 PM Daniel Braniss <</span><a href="mailto:danny@cs.huji.ac.il" style="font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">danny@cs.huji.ac.il</a><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">> wrote:</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><blockquote type="cite" style="font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br><br><br>On 10 Oct 2025, at 00:33, Bjoern A. Zeeb <bzeeb-lists@lists.zabbadoz.net> wrote:<br><br>On Tue, 23 Sep 2025, Daniel Braniss wrote:<br><br>So after many years, I tried to install instead of upgrading,<br>and after much fiddling (if anyone is interested I can write how I did i) I got it to<br>install, now here is my problem:<br>when the the local disk was empty, loader.efi is called (this via dhcp) and it boots diskless,<br>but now that the disk is populated, it boots from it.<br><br>Q: how can I have it boot diskless again?<br><br><br>I haven't seen an answer. Have you tried the freebsd-virtualization list?<br><br>I assume you could interrupt the boot process if fast enough (but it's the blink of an eye);<br>so you'll most likely be more successful putting a startup.nsh and possibly an EFI shell in the EFI System Partition (ESP) and script it? Could be the bhyve firmware brings the efi shell along already?<br><br>/bz<br><br>--<br>Bjoern A. Zeeb r15:7<br><br><br>hi,<br>no answer yet, but after some experimenting:<br>boots diskless:<br>if no local disk - dah<br>if local disk and no EFI partition<br>if local disk has the above but no valid root partition<br>and finally if root partition and efi but no bootx64.efi ( i just move it to bootx64.efix)<br></blockquote><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Can you try the case where there is a non-bootable ZFS partition?</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">When I have a non-bootable ZFS partition with a zpool in it, bhyve decides that</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">is the root and fails to boot the UFS root fs. I need to:</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">currdev=disk0s1</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">boot</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">to get it to boot.</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">I don't know if this is what you might be seeing? rick</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"></div></blockquote><div><br></div><div><br></div>Not really, in the past I could change from diskless to local root by changing the filename option in the dhcp,</div><div>so that if filename is loader.efi it would boot diskless, and if boot1.efi it would boot off the local disk.</div><div>Now loader.efi will first try and boot from the local disk. Btw, I think this is documented.</div><div>Cheers,</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>Danny</div><div><br><blockquote type="cite"><div><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><blockquote type="cite" style="font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br>thanks,<br>danny<br>ps: i guess i forgot to mention that the host is running in bhyve (14.3)<br>pass: it’s now alpha-5</blockquote></div></blockquote></div><br></body></html>help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C296F2BD-F80C-4798-BFA9-6E3D99C1634D>