From nobody Sat Mar 21 16:53:00 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4fdQyx4b9Qz6Vkhb; Sat, 21 Mar 2026 17:14:33 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from www541.your-server.de (www541.your-server.de [213.133.107.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4fdQyw6twHz3k6G; Sat, 21 Mar 2026 17:14:32 +0000 (UTC) (envelope-from mm@FreeBSD.org) Authentication-Results: mx1.freebsd.org; none Received: from sslproxy07.your-server.de ([78.47.199.104]) by www541.your-server.de with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.96.2) (envelope-from ) id 1w3zZO-000HJg-19; Sat, 21 Mar 2026 17:53:02 +0100 Received: from localhost ([127.0.0.1]) by sslproxy07.your-server.de with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1w3zZO-0003jY-0Q; Sat, 21 Mar 2026 17:53:01 +0100 Content-Type: multipart/alternative; boundary="------------iHCWzvWUx2Fdz435o2FdY6sf" Message-ID: <1268de25-55c7-4dd8-995c-4521ef0f4c13@FreeBSD.org> Date: Sat, 21 Mar 2026 17:53:00 +0100 List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: git: 8a62a2a5659d - main - zfs: merge openzfs/zfs@f8e5af53e To: Warner Losh , Shawn Webb Cc: A FreeBSD User , Charlie Li , src-committers , "" , "" References: <69b561ff.39ea9.b797d91@gitrepo.freebsd.org> <2jb2vg6baofimu5xkxf62o5ogaq7fu5pk4o3vzhpegy446bppf@fzwtj6wtwk53> <20260321090444.6f81da15@thor.sb211.local> <5eaaokkdmmkcojky7362jvkandaard6eu26n77f5z5q7lqutu5@kywp5prvxozj> From: Martin Matuska Content-Language: en-US In-Reply-To: X-Virus-Scanned: Clear (ClamAV 1.4.3/27947/Sat Mar 21 07:24:23 2026) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:213.133.96.0/19, country:DE] X-Rspamd-Queue-Id: 4fdQyw6twHz3k6G X-Spamd-Bar: ---- This is a multi-part message in MIME format. --------------iHCWzvWUx2Fdz435o2FdY6sf Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit What has changed is bundled ZSTD. Maybe providing a MBR boot loader without ZFS-ZSTD support? Best regards, mm On 21. 3. 2026 16:30, Warner Losh wrote: > > > On Sat, Mar 21, 2026 at 9:23 AM Shawn Webb > wrote: > > On Sat, Mar 21, 2026 at 09:18:20AM -0600, Warner Losh wrote: > > On Sat, Mar 21, 2026, 9:08 AM Shawn Webb > wrote: > > > > > On Sat, Mar 21, 2026 at 09:04:17AM +0100, A FreeBSD User wrote: > > > > Am Tage des Herren Fri, 20 Mar 2026 23:27:20 -0400 > > > > Charlie Li schrieb: > > > > > > > > > Shawn Webb wrote: > > > > > > On Tue, Mar 17, 2026 at 04:52:16PM +0000, Shawn Webb wrote: > > > > > >> On Tue, Mar 17, 2026 at 10:44:59AM -0600, Warner Losh > wrote: > > > > > >>> On Tue, Mar 17, 2026 at 10:36 AM Shawn Webb < > > > shawn.webb@hardenedbsd.org> > > > > > >>> wrote: > > > > > >>> > > > > > >>>> Hey Martin, > > > > > >>>> > > > > > >>>> On Sat, Mar 14, 2026 at 01:26:23PM +0000, Martin > Matuska wrote: > > > > > >>>>> The branch main has been updated by mm: > > > > > >>>>> > > > > > >>>>> URL: > > > > > >>>> > > > > https://cgit.FreeBSD.org/src/commit/?id=8a62a2a5659d1839d8799b4274c04469d7f17c78 > > > > > > > > > >>>>> > > > > > >>>>> commit 8a62a2a5659d1839d8799b4274c04469d7f17c78 > > > > > >>>>> Merge: f91464171d61 f8e5af53e92f > > > > > >>>>> Author:     Martin Matuska > > > > > >>>>> AuthorDate: 2026-03-14 12:14:56 +0000 > > > > > >>>>> Commit:     Martin Matuska > > > > > >>>>> CommitDate: 2026-03-14 12:14:56 +0000 > > > > > >>>>> > > > > > >>>>> [snip for brevity] > > > > > >>>>> > > > > > >>>>>      Obtained from:  OpenZFS > > > > > >>>>>      OpenZFS commit: > f8e5af53e92fa7c03393fbd4922cb9c1d0c15920 > > > > > >>>> > > > > > >>>> This commit seems to cause issues when building boot > loader > > > related > > > > > >>>> code: > > > > > >>>> > > > > > >>>> ==== BEGIN LOG ==== > > > > > >>>> 114232 bytes available > > > > > >>>> btxld -v -f aout -e 0x200000 -o loader_simp -l > > > > > >>>> > /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btxldr/btxldr -b > > > > > >>>> /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btx/btx > > > loader_simp.bin > > > > > >>>> kernel: ver=1.02 size=690 load=9000 entry=9010 > map=16M pgctl=0:58 > > > > > >>>> client: fmt=elf size=5e2e8 text=57930 data=514c > bss=7470 entry=0 > > > > > >>>> output: fmt=aout size=61000 text=1000 data=5f000 > org=200000 > > > entry=200000 > > > > > >>>> ===> stand/i386/pxeldr (all) > > > > > >>>> -560 bytes available > > > > > >>>> *** Error code 1 > > > > > >>>> > > > > > >>> > > > > > >>> What all do you have enabled? The defaults aren't even > close to > > > running out > > > > > >>> of space (though I've not looked at this). > > > > > >> > > > > > >> Hey Warner, > > > > > >> > > > > > >> Thanks for reaching out! I've uploaded `make > showconfig` here: > > > > > >> https://hardenedbsd.org/~shawn/2026-03-17_srcconf-r01.txt > > > > > >> > > > > > >> The following options are specific to HardenedBSD (in > no particular > > > > > >> order): > > > > > >> > > > > > >> 1. MK_HBSD_UPDATE > > > > > >> 2. MK_HBSDCONTROL > > > > > >> 3. MK_PIE > > > > > >> 4. MK_RELRO > > > > > >> 5. MK_SHLIBRANDOM > > > > > >> 6. MK_ZERO_REGS > > > > > >> 7. MK_SPECTREV1_FIX > > > > > >> 8. MK_SAFESTACK > > > > > >> 9. MK_RETPOLINE > > > > > >> 10. MK_LTOLIB > > > > > >> 11. MK_CFI > > > > > > > > > > > > MK_RETPOLINE was the culprit. Something about this ZFS > commit causes > > > > > > LLVM to emit more retpoline entries than before--too > many for a > > > little > > > > > > bootloader. That might be something to investigate > later, but only to > > > > > > satisfy a curious mind, not to actuall fix anything > (since nothing's > > > > > > actually broken.) > > > > > > > > > > > > Since it doesn't really make sense to apply speculative > execution > > > > > > mitigations to a bootloader, I disabled retpoline for a > components > > > > > > in stand/. > > > > > > > > > > > > Good to go. > > > > > > > > > > > Also just got bit by this, albeit during the lua loader, > since I have > > > > > WITH_RETPOLINE in my src.conf. > > > > > > > > > > > > > Hello, > > > > > > > > I do not have WITH_RETPOLINE in my /etc/src.conf, but since > I got this > > > mysterious error about > > > > not enough bytes left, I use WITHOUT_LOADER_PXEBOOT= YES > (due to issues > > > with WITH_BEARSSL=YES > > > > also used). > > > > Despite not using any WITH_RETPOLINE I also catch the error ... > > > > > > Something about this ZFS commit causes the boot laoder to be > too big. > > > I guess the first sign of trouble was with retpolines, but > there seem > > > to now be additional signs. > > > > > > What's the process for filing a bug report against OpenzFS for > > > something like this? (Not asking you directly, just a general > question > > > for the thread.) > > > > > > > That's a good question. We are rapidly aporoaching the day we > will have to > > freeze the set of zfs feature that we can boot with the BIOS > path. There's > > only so much space. > > > > Right now,  there's little to no bootloader testing, let alone > analysis > > done and that will need to change. > > Another question might also be: is it worth the time and effort? Are > people really using a ZFS-enabled traditional MBR BIOS bootloader on > FreeBSD 16-CURRENT (or eventual 16.0-RELEASE)? From my rather naive > (and ignorant, since I don't know FreeBSD's userbase well), it seems > like folks use either loader.efi or u-boot (or both!) > > > Nobody uses MBR BIOS bootloader. It's totally unsupported. They all > use the > GPT one, which is used for all kinds of things, most recently several > light-weight > VM systems that don't have UEFI. BIOS is what imposes the limit, not MBR. > > But there's a huge number of systems that are already in the field > that need > to be upgradeable, but we will eventually need to draw a line in the > sand for > that. > > Not a question I can answer, of course, because everyone likely has a > different take on what "worth the time and effort" means to them. :-) > > > Yea, the BIOS path is stubbornly hanging around. > > Warner > > Thanks, > > -- > Shawn Webb > Cofounder / Security Engineer > HardenedBSD > > Signal Username:  shawn_webb.74 > Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 > https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc > --------------iHCWzvWUx2Fdz435o2FdY6sf Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit

What has changed is bundled ZSTD.

Maybe providing a MBR boot loader without ZFS-ZSTD support?

Best regards,
mm

On 21. 3. 2026 16:30, Warner Losh wrote:


On Sat, Mar 21, 2026 at 9:23 AM Shawn Webb <shawn.webb@hardenedbsd.org> wrote:
On Sat, Mar 21, 2026 at 09:18:20AM -0600, Warner Losh wrote:
> On Sat, Mar 21, 2026, 9:08 AM Shawn Webb <shawn.webb@hardenedbsd.org> wrote:
>
> > On Sat, Mar 21, 2026 at 09:04:17AM +0100, A FreeBSD User wrote:
> > > Am Tage des Herren Fri, 20 Mar 2026 23:27:20 -0400
> > > Charlie Li <vishwin@freebsd.org> schrieb:
> > >
> > > > Shawn Webb wrote:
> > > > > On Tue, Mar 17, 2026 at 04:52:16PM +0000, Shawn Webb wrote:
> > > > >> On Tue, Mar 17, 2026 at 10:44:59AM -0600, Warner Losh wrote:
> > > > >>> On Tue, Mar 17, 2026 at 10:36 AM Shawn Webb <
> > shawn.webb@hardenedbsd.org>
> > > > >>> wrote:
> > > > >>>
> > > > >>>> Hey Martin,
> > > > >>>>
> > > > >>>> On Sat, Mar 14, 2026 at 01:26:23PM +0000, Martin Matuska wrote:
> > > > >>>>> The branch main has been updated by mm:
> > > > >>>>>
> > > > >>>>> URL:
> > > > >>>>
> > https://cgit.FreeBSD.org/src/commit/?id=8a62a2a5659d1839d8799b4274c04469d7f17c78
> >
> > > > >>>>>
> > > > >>>>> commit 8a62a2a5659d1839d8799b4274c04469d7f17c78
> > > > >>>>> Merge: f91464171d61 f8e5af53e92f
> > > > >>>>> Author:     Martin Matuska <mm@FreeBSD.org>
> > > > >>>>> AuthorDate: 2026-03-14 12:14:56 +0000
> > > > >>>>> Commit:     Martin Matuska <mm@FreeBSD.org>
> > > > >>>>> CommitDate: 2026-03-14 12:14:56 +0000
> > > > >>>>>
> > > > >>>>> [snip for brevity]
> > > > >>>>>
> > > > >>>>>      Obtained from:  OpenZFS
> > > > >>>>>      OpenZFS commit: f8e5af53e92fa7c03393fbd4922cb9c1d0c15920
> > > > >>>>
> > > > >>>> This commit seems to cause issues when building boot loader
> > related
> > > > >>>> code:
> > > > >>>>
> > > > >>>> ==== BEGIN LOG ====
> > > > >>>> 114232 bytes available
> > > > >>>> btxld -v -f aout -e 0x200000 -o loader_simp -l
> > > > >>>> /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btxldr/btxldr  -b
> > > > >>>> /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btx/btx
> > loader_simp.bin
> > > > >>>> kernel: ver=1.02 size=690 load=9000 entry=9010 map=16M pgctl=0:58
> > > > >>>> client: fmt=elf size=5e2e8 text=57930 data=514c bss=7470 entry=0
> > > > >>>> output: fmt=aout size=61000 text=1000 data=5f000 org=200000
> > entry=200000
> > > > >>>> ===> stand/i386/pxeldr (all)
> > > > >>>> -560 bytes available
> > > > >>>> *** Error code 1
> > > > >>>>
> > > > >>>
> > > > >>> What all do you have enabled? The defaults aren't even close to
> > running out
> > > > >>> of space (though I've not looked at this).
> > > > >>
> > > > >> Hey Warner,
> > > > >>
> > > > >> Thanks for reaching out! I've uploaded `make showconfig` here:
> > > > >> https://hardenedbsd.org/~shawn/2026-03-17_srcconf-r01.txt
> > > > >>
> > > > >> The following options are specific to HardenedBSD (in no particular
> > > > >> order):
> > > > >>
> > > > >> 1. MK_HBSD_UPDATE
> > > > >> 2. MK_HBSDCONTROL
> > > > >> 3. MK_PIE
> > > > >> 4. MK_RELRO
> > > > >> 5. MK_SHLIBRANDOM
> > > > >> 6. MK_ZERO_REGS
> > > > >> 7. MK_SPECTREV1_FIX
> > > > >> 8. MK_SAFESTACK
> > > > >> 9. MK_RETPOLINE
> > > > >> 10. MK_LTOLIB
> > > > >> 11. MK_CFI
> > > > >
> > > > > MK_RETPOLINE was the culprit. Something about this ZFS commit causes
> > > > > LLVM to emit more retpoline entries than before--too many for a
> > little
> > > > > bootloader. That might be something to investigate later, but only to
> > > > > satisfy a curious mind, not to actuall fix anything (since nothing's
> > > > > actually broken.)
> > > > >
> > > > > Since it doesn't really make sense to apply speculative execution
> > > > > mitigations to a bootloader, I disabled retpoline for a components
> > > > > in stand/.
> > > > >
> > > > > Good to go.
> > > > >
> > > > Also just got bit by this, albeit during the lua loader, since I have
> > > > WITH_RETPOLINE in my src.conf.
> > > >
> > >
> > > Hello,
> > >
> > > I do not have WITH_RETPOLINE in my /etc/src.conf, but since I got this
> > mysterious error about
> > > not enough bytes left, I use WITHOUT_LOADER_PXEBOOT= YES (due to issues
> > with WITH_BEARSSL=YES
> > > also used).
> > > Despite not using any WITH_RETPOLINE I also catch the error ...
> >
> > Something about this ZFS commit causes the boot laoder to be too big.
> > I guess the first sign of trouble was with retpolines, but there seem
> > to now be additional signs.
> >
> > What's the process for filing a bug report against OpenzFS for
> > something like this? (Not asking you directly, just a general question
> > for the thread.)
> >
>
> That's a good question. We are rapidly aporoaching the day we will have to
> freeze the set of zfs feature that we can boot with the BIOS path. There's
> only so much space.
>
> Right now,  there's little to no bootloader testing, let alone analysis
> done and that will need to change.

Another question might also be: is it worth the time and effort? Are
people really using a ZFS-enabled traditional MBR BIOS bootloader on
FreeBSD 16-CURRENT (or eventual 16.0-RELEASE)? From my rather naive
(and ignorant, since I don't know FreeBSD's userbase well), it seems
like folks use either loader.efi or u-boot (or both!)

Nobody uses MBR BIOS bootloader. It's totally unsupported. They all use the
GPT one, which is used for all kinds of things, most recently several light-weight
VM systems that don't have UEFI. BIOS is what imposes the limit, not MBR.

But there's a huge number of systems that are already in the field that need
to be upgradeable, but we will eventually need to draw a line in the sand for
that.
 
Not a question I can answer, of course, because everyone likely has a
different take on what "worth the time and effort" means to them. :-)

Yea, the BIOS path is stubbornly hanging around.

Warner
 
Thanks,

--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Signal Username:  shawn_webb.74
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
--------------iHCWzvWUx2Fdz435o2FdY6sf--