Date: Sat, 21 Mar 2026 09:18:20 -0600 From: Warner Losh <imp@bsdimp.com> To: Shawn Webb <shawn.webb@hardenedbsd.org> Cc: A FreeBSD User <freebsd@walstatt-de.de>, Charlie Li <vishwin@freebsd.org>, Martin Matuska <mm@freebsd.org>, src-committers <src-committers@freebsd.org>, "<dev-commits-src-all@freebsd.org>" <dev-commits-src-all@freebsd.org>, "<dev-commits-src-main@freebsd.org>" <dev-commits-src-main@freebsd.org> Subject: Re: git: 8a62a2a5659d - main - zfs: merge openzfs/zfs@f8e5af53e Message-ID: <CANCZdfq7YQuzgavRWm=DMKEvxQE9cYOFAEMpgk6faQwOL2tC2Q@mail.gmail.com> In-Reply-To: <qeang5o4tlutdyxu2zfksbpljrrvf22eebf6r7b62umffzaymn@or5qwqnmbtbx> References: <69b561ff.39ea9.b797d91@gitrepo.freebsd.org> <2jb2vg6baofimu5xkxf62o5ogaq7fu5pk4o3vzhpegy446bppf@fzwtj6wtwk53> <CANCZdfpaKyzMDL0%2BMk3tSEeqsSU_C-OoWo1Dwwjg%2BG%2BrxiV2Eg@mail.gmail.com> <hqup3oiorbdgvtputen566fcajjoucurqpk7je6qchixat3i6c@zmfpxouvy7vs> <wryftbwxvsc3mhm6iglbx3ikzentaoewo3ldiw4knyqkckmhmo@hnmn3nosgawc> <becfc71c-2ea1-4a13-a3b1-3235f056badf@freebsd.org> <20260321090444.6f81da15@thor.sb211.local> <qeang5o4tlutdyxu2zfksbpljrrvf22eebf6r7b62umffzaymn@or5qwqnmbtbx>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] On Sat, Mar 21, 2026, 9:08 AM Shawn Webb <shawn.webb@hardenedbsd.org> wrote: > On Sat, Mar 21, 2026 at 09:04:17AM +0100, A FreeBSD User wrote: > > Am Tage des Herren Fri, 20 Mar 2026 23:27:20 -0400 > > Charlie Li <vishwin@freebsd.org> schrieb: > > > > > Shawn Webb wrote: > > > > On Tue, Mar 17, 2026 at 04:52:16PM +0000, Shawn Webb wrote: > > > >> On Tue, Mar 17, 2026 at 10:44:59AM -0600, Warner Losh wrote: > > > >>> On Tue, Mar 17, 2026 at 10:36 AM Shawn Webb < > shawn.webb@hardenedbsd.org> > > > >>> wrote: > > > >>> > > > >>>> Hey Martin, > > > >>>> > > > >>>> On Sat, Mar 14, 2026 at 01:26:23PM +0000, Martin Matuska wrote: > > > >>>>> The branch main has been updated by mm: > > > >>>>> > > > >>>>> URL: > > > >>>> > https://cgit.FreeBSD.org/src/commit/?id=8a62a2a5659d1839d8799b4274c04469d7f17c78 > > > > >>>>> > > > >>>>> commit 8a62a2a5659d1839d8799b4274c04469d7f17c78 > > > >>>>> Merge: f91464171d61 f8e5af53e92f > > > >>>>> Author: Martin Matuska <mm@FreeBSD.org> > > > >>>>> AuthorDate: 2026-03-14 12:14:56 +0000 > > > >>>>> Commit: Martin Matuska <mm@FreeBSD.org> > > > >>>>> CommitDate: 2026-03-14 12:14:56 +0000 > > > >>>>> > > > >>>>> [snip for brevity] > > > >>>>> > > > >>>>> Obtained from: OpenZFS > > > >>>>> OpenZFS commit: f8e5af53e92fa7c03393fbd4922cb9c1d0c15920 > > > >>>> > > > >>>> This commit seems to cause issues when building boot loader > related > > > >>>> code: > > > >>>> > > > >>>> ==== BEGIN LOG ==== > > > >>>> 114232 bytes available > > > >>>> btxld -v -f aout -e 0x200000 -o loader_simp -l > > > >>>> /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btxldr/btxldr -b > > > >>>> /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btx/btx > loader_simp.bin > > > >>>> kernel: ver=1.02 size=690 load=9000 entry=9010 map=16M pgctl=0:58 > > > >>>> client: fmt=elf size=5e2e8 text=57930 data=514c bss=7470 entry=0 > > > >>>> output: fmt=aout size=61000 text=1000 data=5f000 org=200000 > entry=200000 > > > >>>> ===> stand/i386/pxeldr (all) > > > >>>> -560 bytes available > > > >>>> *** Error code 1 > > > >>>> > > > >>> > > > >>> What all do you have enabled? The defaults aren't even close to > running out > > > >>> of space (though I've not looked at this). > > > >> > > > >> Hey Warner, > > > >> > > > >> Thanks for reaching out! I've uploaded `make showconfig` here: > > > >> https://hardenedbsd.org/~shawn/2026-03-17_srcconf-r01.txt > > > >> > > > >> The following options are specific to HardenedBSD (in no particular > > > >> order): > > > >> > > > >> 1. MK_HBSD_UPDATE > > > >> 2. MK_HBSDCONTROL > > > >> 3. MK_PIE > > > >> 4. MK_RELRO > > > >> 5. MK_SHLIBRANDOM > > > >> 6. MK_ZERO_REGS > > > >> 7. MK_SPECTREV1_FIX > > > >> 8. MK_SAFESTACK > > > >> 9. MK_RETPOLINE > > > >> 10. MK_LTOLIB > > > >> 11. MK_CFI > > > > > > > > MK_RETPOLINE was the culprit. Something about this ZFS commit causes > > > > LLVM to emit more retpoline entries than before--too many for a > little > > > > bootloader. That might be something to investigate later, but only to > > > > satisfy a curious mind, not to actuall fix anything (since nothing's > > > > actually broken.) > > > > > > > > Since it doesn't really make sense to apply speculative execution > > > > mitigations to a bootloader, I disabled retpoline for a components > > > > in stand/. > > > > > > > > Good to go. > > > > > > > Also just got bit by this, albeit during the lua loader, since I have > > > WITH_RETPOLINE in my src.conf. > > > > > > > Hello, > > > > I do not have WITH_RETPOLINE in my /etc/src.conf, but since I got this > mysterious error about > > not enough bytes left, I use WITHOUT_LOADER_PXEBOOT= YES (due to issues > with WITH_BEARSSL=YES > > also used). > > Despite not using any WITH_RETPOLINE I also catch the error ... > > Something about this ZFS commit causes the boot laoder to be too big. > I guess the first sign of trouble was with retpolines, but there seem > to now be additional signs. > > What's the process for filing a bug report against OpenzFS for > something like this? (Not asking you directly, just a general question > for the thread.) > That's a good question. We are rapidly aporoaching the day we will have to freeze the set of zfs feature that we can boot with the BIOS path. There's only so much space. Right now, there's little to no bootloader testing, let alone analysis done and that will need to change. Warmer Thanks, > > -- > Shawn Webb > Cofounder / Security Engineer > HardenedBSD > > Signal Username: shawn_webb.74 > Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 > > https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc > [-- Attachment #2 --] <div dir="auto"><div><br><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Sat, Mar 21, 2026, 9:08 AM Shawn Webb <<a href="mailto:shawn.webb@hardenedbsd.org">shawn.webb@hardenedbsd.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Sat, Mar 21, 2026 at 09:04:17AM +0100, A FreeBSD User wrote:<br> > Am Tage des Herren Fri, 20 Mar 2026 23:27:20 -0400<br> > Charlie Li <<a href="mailto:vishwin@freebsd.org" target="_blank" rel="noreferrer">vishwin@freebsd.org</a>> schrieb:<br> > <br> > > Shawn Webb wrote:<br> > > > On Tue, Mar 17, 2026 at 04:52:16PM +0000, Shawn Webb wrote: <br> > > >> On Tue, Mar 17, 2026 at 10:44:59AM -0600, Warner Losh wrote: <br> > > >>> On Tue, Mar 17, 2026 at 10:36 AM Shawn Webb <<a href="mailto:shawn.webb@hardenedbsd.org" target="_blank" rel="noreferrer">shawn.webb@hardenedbsd.org</a>><br> > > >>> wrote:<br> > > >>> <br> > > >>>> Hey Martin,<br> > > >>>><br> > > >>>> On Sat, Mar 14, 2026 at 01:26:23PM +0000, Martin Matuska wrote: <br> > > >>>>> The branch main has been updated by mm:<br> > > >>>>><br> > > >>>>> URL: <br> > > >>>> <a href="https://cgit.FreeBSD.org/src/commit/?id=8a62a2a5659d1839d8799b4274c04469d7f17c78" rel="noreferrer noreferrer" target="_blank">https://cgit.FreeBSD.org/src/commit/?id=8a62a2a5659d1839d8799b4274c04469d7f17c78</a> <br> > > >>>>><br> > > >>>>> commit 8a62a2a5659d1839d8799b4274c04469d7f17c78<br> > > >>>>> Merge: f91464171d61 f8e5af53e92f<br> > > >>>>> Author:   Martin Matuska <mm@FreeBSD.org><br> > > >>>>> AuthorDate: 2026-03-14 12:14:56 +0000<br> > > >>>>> Commit:   Martin Matuska <mm@FreeBSD.org><br> > > >>>>> CommitDate: 2026-03-14 12:14:56 +0000<br> > > >>>>><br> > > >>>>> [snip for brevity]<br> > > >>>>><br> > > >>>>>   Obtained from: OpenZFS<br> > > >>>>>   OpenZFS commit: f8e5af53e92fa7c03393fbd4922cb9c1d0c15920 <br> > > >>>><br> > > >>>> This commit seems to cause issues when building boot loader related<br> > > >>>> code:<br> > > >>>><br> > > >>>> ==== BEGIN LOG ====<br> > > >>>> 114232 bytes available<br> > > >>>> btxld -v -f aout -e 0x200000 -o loader_simp -l<br> > > >>>> /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btxldr/btxldr -b<br> > > >>>> /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btx/btx loader_simp.bin<br> > > >>>> kernel: ver=1.02 size=690 load=9000 entry=9010 map=16M pgctl=0:58<br> > > >>>> client: fmt=elf size=5e2e8 text=57930 data=514c bss=7470 entry=0<br> > > >>>> output: fmt=aout size=61000 text=1000 data=5f000 org=200000 entry=200000 <br> > > >>>> ===> stand/i386/pxeldr (all) <br> > > >>>> -560 bytes available<br> > > >>>> *** Error code 1<br> > > >>>> <br> > > >>><br> > > >>> What all do you have enabled? The defaults aren't even close to running out<br> > > >>> of space (though I've not looked at this). <br> > > >><br> > > >> Hey Warner,<br> > > >><br> > > >> Thanks for reaching out! I've uploaded `make showconfig` here:<br> > > >> <a href="https://hardenedbsd.org/~shawn/2026-03-17_srcconf-r01.txt" rel="noreferrer noreferrer" target="_blank">https://hardenedbsd.org/~shawn/2026-03-17_srcconf-r01.txt</a><br>; > > >><br> > > >> The following options are specific to HardenedBSD (in no particular<br> > > >> order):<br> > > >><br> > > >> 1. MK_HBSD_UPDATE<br> > > >> 2. MK_HBSDCONTROL<br> > > >> 3. MK_PIE<br> > > >> 4. MK_RELRO<br> > > >> 5. MK_SHLIBRANDOM<br> > > >> 6. MK_ZERO_REGS<br> > > >> 7. MK_SPECTREV1_FIX<br> > > >> 8. MK_SAFESTACK<br> > > >> 9. MK_RETPOLINE<br> > > >> 10. MK_LTOLIB<br> > > >> 11. MK_CFI <br> > > > <br> > > > MK_RETPOLINE was the culprit. Something about this ZFS commit causes<br> > > > LLVM to emit more retpoline entries than before--too many for a little<br> > > > bootloader. That might be something to investigate later, but only to<br> > > > satisfy a curious mind, not to actuall fix anything (since nothing's<br> > > > actually broken.)<br> > > > <br> > > > Since it doesn't really make sense to apply speculative execution<br> > > > mitigations to a bootloader, I disabled retpoline for a components<br> > > > in stand/.<br> > > > <br> > > > Good to go.<br> > > >  <br> > > Also just got bit by this, albeit during the lua loader, since I have <br> > > WITH_RETPOLINE in my src.conf.<br> > > <br> > <br> > Hello,<br> > <br> > I do not have WITH_RETPOLINE in my /etc/src.conf, but since I got this mysterious error about<br> > not enough bytes left, I use WITHOUT_LOADER_PXEBOOT= YES (due to issues with WITH_BEARSSL=YES<br> > also used).<br> > Despite not using any WITH_RETPOLINE I also catch the error ...<br> <br> Something about this ZFS commit causes the boot laoder to be too big.<br> I guess the first sign of trouble was with retpolines, but there seem<br> to now be additional signs.<br> <br> What's the process for filing a bug report against OpenzFS for<br> something like this? (Not asking you directly, just a general question<br> for the thread.)<br></blockquote></div></div><div dir="auto"><br></div><div dir="auto">That's a good question. We are rapidly aporoaching the day we will have to freeze the set of zfs feature that we can boot with the BIOS path. There's only so much space.</div><div dir="auto"><br></div><div dir="auto">Right now, there's little to no bootloader testing, let alone analysis done and that will need to change.</div><div dir="auto"><br></div><div dir="auto">Warmer</div><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote gmail_quote_container"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Thanks,<br> <br> -- <br> Shawn Webb<br> Cofounder / Security Engineer<br> HardenedBSD<br> <br> Signal Username: shawn_webb.74<br> Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50<br> <a href="https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc" rel="noreferrer noreferrer" target="_blank">https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc</a><br>; </blockquote></div></div></div>home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfq7YQuzgavRWm=DMKEvxQE9cYOFAEMpgk6faQwOL2tC2Q>