From nobody Sat Mar 21 15:18:20 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4fdNPH6V2Fz6VZG3 for ; Sat, 21 Mar 2026 15:18:43 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4fdNPH0gx3z3SbH for ; Sat, 21 Mar 2026 15:18:38 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x1031.google.com with SMTP id 98e67ed59e1d1-359f35dfef6so791762a91.2 for ; Sat, 21 Mar 2026 08:18:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1774106312; cv=none; d=google.com; s=arc-20240605; b=XIqNH5wtcxfWxqoAm4WwfAwP4mRGwle27eFXL7So5QTAKqYmXtCv2AZ3zMBJpv6eao GP3MERB1IYiAov+V+ui2Se2FxwzfCyxE+W4iC+FCpZ9VzMV+OWN5sTnE3dY6qSYYiyt+ aSBN+M6EBTyjfqZmoctLl93iGhr/5nSt5TV/XDws/hGrafQqecjgABjhOYwtZJrwImXj DpNzg4Rxk7nWJO7fcevGwMJDSUVK4ZFq3ZHx1M7O+k7weFj+X/R5V2LI64HxAiKwR8Qh tXxlbMc+cqwvHmIdLKpn/u7OTE0ut8yTEOmJamKR7UOq7nBktNSzZTvL+G4T3mvXMAbN sNkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=T25g4ScAAc8CyE4pVDio6ZlWtdDZwriXPdwK8wrJaX0=; fh=oIbErIFzn3zteKTtxqRoP3HbfKduu4nLpKj1w48ZWW8=; b=K85x+ACV8jRRHqK1x0oOK5GuqUjIackw8gMWWqk7CRKrngogCo/dsGrTLQA28RU8J5 WQwj+YzG21pcIJKZJKgS51QWCIUfzGQr9e51MAWHnLhIjeIJ4Bu5Ldi0wq9ojSoHBFAz usDYCCtLaqEE5mluGMwSFejuYVnkOHK3SvLkP96eQQtXRTYXQo7JDlhFoOrVTpNUJxDn Tou5FMUuhyntJVVhqwCucMem7ixiCYpJK1yfUTgru2IyeFDOdmIbii8PaHGVzQ25c7kl oie/kMuI+AgfHtpWTIiUr6L7t5Ov40hUCY5F0cr8eClV1usoyBYOUtpUD6V9C33SbrXB hDow==; darn=freebsd.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1774106312; x=1774711112; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=T25g4ScAAc8CyE4pVDio6ZlWtdDZwriXPdwK8wrJaX0=; b=AH2oWQ1pmvIh/9WtO3zQ+GYIgx/NdNhLoLa0EM5kPiidALPKxTRyTxlubNSZL5sgYZ bareVET1eXESudp+HJ5u1capeGx5EkLYrylQhfr/eCJPeKoJLa1z/aQ7H7lrOTH1POzg +Q1rjkScPcX2F7jpCL/LwgMUY6nJ5Zf+8vmsjCaRqqrmJQVm5K50Yk+4Req90EvZJcGE knlmYn4Jjrqrrhv9O2rPXvVrUdzThwLzMeX3hCmPNH0QzV60XW/MC4Yy9PJNDxj/1UIY sCrcLg5fxXpqS316LZMg3pzwVxl6PboO3gEP/ORboky1t3j6BXnbYC1Y7QRxERopYFhg YHmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774106312; x=1774711112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=T25g4ScAAc8CyE4pVDio6ZlWtdDZwriXPdwK8wrJaX0=; b=GGKkJ2s1HLBRyN+T1vMh2fOkL5xnXulwOGVAHWZQvG+EA628f6ViX6ujbnU8bCvTlp itHMz+QVjmiJ70gRUhHD6UbDNkFHAlTxncH4H8cBK+2ERyKGxcT1oddDg512B5QkuAMw EnL5BbWgslLrqpzfS/E1dSbtaKvR0vslSVx1R80BqQMn9UwhKxQJzevFD+31mEzlbU8H rVIdqofXLzkGouytL+t3R0amB6jkpjSRJDNOispO+todxoE6wr6NBcDqxv1GwzDIEesI 1vXqhEqijUME2SIPWEBjxygxClNM2auxOfLohSL6JDp4Lnj+qa6wviBevZGf6qHRN2NR yw7g== X-Forwarded-Encrypted: i=1; AJvYcCWooaiME6gWyMjNf6nxyiVzmcn6I+IsZxJgSqyHvgSdPUMlKf38tU4vLgzJXcpf0987f2T5WlVFGnWihNDC/Je7/lW5UA==@freebsd.org X-Gm-Message-State: AOJu0YzH3frOwfSBy5NjZSIrcfauRp1IhepC+8GHuUZAOADPav76eWQr 2lwaotBVUzscRF5hCnO9kU4phQewhsYQ4Sv50K0WTBjEryuXfp+SthQjcPGqyNzbyBvoDHCe1ui PGftMReyqK4FX2y9jTlVuhL+TafQA+M8un/6TgJmNNw== X-Gm-Gg: ATEYQzwVisCDz1VtDdNn8pxc87+mk5d9v/rIluWFBgYERdf4rXZxuPQdEhvWLOS415o 7JjlC3J3fDSBjr5PDHXNKYoCBiXDSl7cHZ+Qhtr3SSetEnGQdSqnsl+e7+PK0CP+0vbzj13LSZ/ wHNidcQiDJMIt52UbY4B1qHVzze5IpsenTe9ST4gSbyzHojXdGc7pdxoNR3mTc8LqA3z6JKOIL0 73Y029+h+g0rXJMozH6JWrXoGeHUCUhmoneUkunGdGexYg97FEbMWrzyepiFRK5k4zxe6byXvY5 +Nord2YgjavulKcB0x/kYiPrTLS1wXzLrtLehq5h2gLPWznx X-Received: by 2002:a17:903:244b:b0:2b0:775f:febf with SMTP id d9443c01a7336-2b0827e31acmr63966655ad.40.1774106311742; Sat, 21 Mar 2026 08:18:31 -0700 (PDT) List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 References: <69b561ff.39ea9.b797d91@gitrepo.freebsd.org> <2jb2vg6baofimu5xkxf62o5ogaq7fu5pk4o3vzhpegy446bppf@fzwtj6wtwk53> <20260321090444.6f81da15@thor.sb211.local> In-Reply-To: From: Warner Losh Date: Sat, 21 Mar 2026 09:18:20 -0600 X-Gm-Features: AaiRm50jsK9w9KGFmhnyVX868rHtrjXKQvghCz14hpLGiqYfWqe11lUKPWQ9hII Message-ID: Subject: Re: git: 8a62a2a5659d - main - zfs: merge openzfs/zfs@f8e5af53e To: Shawn Webb Cc: A FreeBSD User , Charlie Li , Martin Matuska , src-committers , "" , "" Content-Type: multipart/alternative; boundary="0000000000008161ef064d8a5058" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4fdNPH0gx3z3SbH X-Spamd-Bar: ---- --0000000000008161ef064d8a5058 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Mar 21, 2026, 9:08=E2=80=AFAM Shawn Webb wrote: > On Sat, Mar 21, 2026 at 09:04:17AM +0100, A FreeBSD User wrote: > > Am Tage des Herren Fri, 20 Mar 2026 23:27:20 -0400 > > Charlie Li schrieb: > > > > > Shawn Webb wrote: > > > > On Tue, Mar 17, 2026 at 04:52:16PM +0000, Shawn Webb wrote: > > > >> On Tue, Mar 17, 2026 at 10:44:59AM -0600, Warner Losh wrote: > > > >>> On Tue, Mar 17, 2026 at 10:36=E2=80=AFAM Shawn Webb < > shawn.webb@hardenedbsd.org> > > > >>> wrote: > > > >>> > > > >>>> Hey Martin, > > > >>>> > > > >>>> On Sat, Mar 14, 2026 at 01:26:23PM +0000, Martin Matuska wrote: > > > >>>>> The branch main has been updated by mm: > > > >>>>> > > > >>>>> URL: > > > >>>> > https://cgit.FreeBSD.org/src/commit/?id=3D8a62a2a5659d1839d8799b4274c0446= 9d7f17c78 > > > > >>>>> > > > >>>>> commit 8a62a2a5659d1839d8799b4274c04469d7f17c78 > > > >>>>> Merge: f91464171d61 f8e5af53e92f > > > >>>>> Author: Martin Matuska > > > >>>>> AuthorDate: 2026-03-14 12:14:56 +0000 > > > >>>>> Commit: Martin Matuska > > > >>>>> CommitDate: 2026-03-14 12:14:56 +0000 > > > >>>>> > > > >>>>> [snip for brevity] > > > >>>>> > > > >>>>> Obtained from: OpenZFS > > > >>>>> OpenZFS commit: f8e5af53e92fa7c03393fbd4922cb9c1d0c15920 > > > >>>> > > > >>>> This commit seems to cause issues when building boot loader > related > > > >>>> code: > > > >>>> > > > >>>> =3D=3D=3D=3D BEGIN LOG =3D=3D=3D=3D > > > >>>> 114232 bytes available > > > >>>> btxld -v -f aout -e 0x200000 -o loader_simp -l > > > >>>> /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btxldr/btxldr -b > > > >>>> /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btx/btx > loader_simp.bin > > > >>>> kernel: ver=3D1.02 size=3D690 load=3D9000 entry=3D9010 map=3D16M= pgctl=3D0:58 > > > >>>> client: fmt=3Delf size=3D5e2e8 text=3D57930 data=3D514c bss=3D74= 70 entry=3D0 > > > >>>> output: fmt=3Daout size=3D61000 text=3D1000 data=3D5f000 org=3D2= 00000 > entry=3D200000 > > > >>>> =3D=3D=3D> stand/i386/pxeldr (all) > > > >>>> -560 bytes available > > > >>>> *** Error code 1 > > > >>>> > > > >>> > > > >>> What all do you have enabled? The defaults aren't even close to > running out > > > >>> of space (though I've not looked at this). > > > >> > > > >> Hey Warner, > > > >> > > > >> Thanks for reaching out! I've uploaded `make showconfig` here: > > > >> https://hardenedbsd.org/~shawn/2026-03-17_srcconf-r01.txt > > > >> > > > >> The following options are specific to HardenedBSD (in no particula= r > > > >> order): > > > >> > > > >> 1. MK_HBSD_UPDATE > > > >> 2. MK_HBSDCONTROL > > > >> 3. MK_PIE > > > >> 4. MK_RELRO > > > >> 5. MK_SHLIBRANDOM > > > >> 6. MK_ZERO_REGS > > > >> 7. MK_SPECTREV1_FIX > > > >> 8. MK_SAFESTACK > > > >> 9. MK_RETPOLINE > > > >> 10. MK_LTOLIB > > > >> 11. MK_CFI > > > > > > > > MK_RETPOLINE was the culprit. Something about this ZFS commit cause= s > > > > LLVM to emit more retpoline entries than before--too many for a > little > > > > bootloader. That might be something to investigate later, but only = to > > > > satisfy a curious mind, not to actuall fix anything (since nothing'= s > > > > actually broken.) > > > > > > > > Since it doesn't really make sense to apply speculative execution > > > > mitigations to a bootloader, I disabled retpoline for a components > > > > in stand/. > > > > > > > > Good to go. > > > > > > > Also just got bit by this, albeit during the lua loader, since I have > > > WITH_RETPOLINE in my src.conf. > > > > > > > Hello, > > > > I do not have WITH_RETPOLINE in my /etc/src.conf, but since I got this > mysterious error about > > not enough bytes left, I use WITHOUT_LOADER_PXEBOOT=3D YES (due to issu= es > with WITH_BEARSSL=3DYES > > also used). > > Despite not using any WITH_RETPOLINE I also catch the error ... > > Something about this ZFS commit causes the boot laoder to be too big. > I guess the first sign of trouble was with retpolines, but there seem > to now be additional signs. > > What's the process for filing a bug report against OpenzFS for > something like this? (Not asking you directly, just a general question > for the thread.) > That's a good question. We are rapidly aporoaching the day we will have to freeze the set of zfs feature that we can boot with the BIOS path. There's only so much space. Right now, there's little to no bootloader testing, let alone analysis done and that will need to change. Warmer Thanks, > > -- > Shawn Webb > Cofounder / Security Engineer > HardenedBSD > > Signal Username: shawn_webb.74 > Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 > > https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/0= 3A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc > --0000000000008161ef064d8a5058 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sat, Mar 21, 2026, 9:08=E2=80= =AFAM Shawn Webb <shawn.we= bb@hardenedbsd.org> wrote:
O= n Sat, Mar 21, 2026 at 09:04:17AM +0100, A FreeBSD User wrote:
> Am Tage des Herren Fri, 20 Mar 2026 23:27:20 -0400
> Charlie Li <vishwin@freebsd.org> schrieb:
>
> > Shawn Webb wrote:
> > > On Tue, Mar 17, 2026 at 04:52:16PM +0000, Shawn Webb wrote:= =C2=A0
> > >> On Tue, Mar 17, 2026 at 10:44:59AM -0600, Warner Losh wr= ote:=C2=A0
> > >>> On Tue, Mar 17, 2026 at 10:36=E2=80=AFAM Shawn Webb = <shawn.webb@hardenedbsd.org>
> > >>> wrote:
> > >>>=C2=A0
> > >>>> Hey Martin,
> > >>>>
> > >>>> On Sat, Mar 14, 2026 at 01:26:23PM +0000, Martin= Matuska wrote:=C2=A0
> > >>>>> The branch main has been updated by mm:
> > >>>>>
> > >>>>> URL:=C2=A0
> > >>>> https://cgit.FreeBSD.org/src/commit/?id=3D8a62a2a5659d1= 839d8799b4274c04469d7f17c78=C2=A0
> > >>>>>
> > >>>>> commit 8a62a2a5659d1839d8799b4274c04469d7f17= c78
> > >>>>> Merge: f91464171d61 f8e5af53e92f
> > >>>>> Author:=C2=A0 =C2=A0 =C2=A0Martin Matuska &l= t;mm@FreeBSD.org>
> > >>>>> AuthorDate: 2026-03-14 12:14:56 +0000
> > >>>>> Commit:=C2=A0 =C2=A0 =C2=A0Martin Matuska &l= t;mm@FreeBSD.org>
> > >>>>> CommitDate: 2026-03-14 12:14:56 +0000
> > >>>>>
> > >>>>> [snip for brevity]
> > >>>>>
> > >>>>>=C2=A0 =C2=A0 =C2=A0 Obtained from:=C2=A0 Ope= nZFS
> > >>>>>=C2=A0 =C2=A0 =C2=A0 OpenZFS commit: f8e5af53= e92fa7c03393fbd4922cb9c1d0c15920=C2=A0
> > >>>>
> > >>>> This commit seems to cause issues when building = boot loader related
> > >>>> code:
> > >>>>
> > >>>> =3D=3D=3D=3D BEGIN LOG =3D=3D=3D=3D
> > >>>> 114232 bytes available
> > >>>> btxld -v -f aout -e 0x200000 -o loader_simp -l > > >>>> /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btxl= dr/btxldr=C2=A0 -b
> > >>>> /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btx/= btx loader_simp.bin
> > >>>> kernel: ver=3D1.02 size=3D690 load=3D9000 entry= =3D9010 map=3D16M pgctl=3D0:58
> > >>>> client: fmt=3Delf size=3D5e2e8 text=3D57930 data= =3D514c bss=3D7470 entry=3D0
> > >>>> output: fmt=3Daout size=3D61000 text=3D1000 data= =3D5f000 org=3D200000 entry=3D200000=C2=A0
> > >>>> =3D=3D=3D> stand/i386/pxeldr (all)=C2=A0
> > >>>> -560 bytes available
> > >>>> *** Error code 1
> > >>>>=C2=A0
> > >>>
> > >>> What all do you have enabled? The defaults aren'= t even close to running out
> > >>> of space (though I've not looked at this).=C2=A0=
> > >>
> > >> Hey Warner,
> > >>
> > >> Thanks for reaching out! I've uploaded `make showcon= fig` here:
> > >> https://harde= nedbsd.org/~shawn/2026-03-17_srcconf-r01.txt
> > >>
> > >> The following options are specific to HardenedBSD (in no= particular
> > >> order):
> > >>
> > >> 1. MK_HBSD_UPDATE
> > >> 2. MK_HBSDCONTROL
> > >> 3. MK_PIE
> > >> 4. MK_RELRO
> > >> 5. MK_SHLIBRANDOM
> > >> 6. MK_ZERO_REGS
> > >> 7. MK_SPECTREV1_FIX
> > >> 8. MK_SAFESTACK
> > >> 9. MK_RETPOLINE
> > >> 10. MK_LTOLIB
> > >> 11. MK_CFI=C2=A0
> > >
> > > MK_RETPOLINE was the culprit. Something about this ZFS commi= t causes
> > > LLVM to emit more retpoline entries than before--too many fo= r a little
> > > bootloader. That might be something to investigate later, bu= t only to
> > > satisfy a curious mind, not to actuall fix anything (since n= othing's
> > > actually broken.)
> > >
> > > Since it doesn't really make sense to apply speculative = execution
> > > mitigations to a bootloader, I disabled retpoline for a comp= onents
> > > in stand/.
> > >
> > > Good to go.
> > >=C2=A0 =C2=A0
> > Also just got bit by this, albeit during the lua loader, since I = have
> > WITH_RETPOLINE in my src.conf.
> >
>
> Hello,
>
> I do not have WITH_RETPOLINE in my /etc/src.conf, but since I got this= mysterious error about
> not enough bytes left, I use WITHOUT_LOADER_PXEBOOT=3D YES (due to iss= ues with WITH_BEARSSL=3DYES
> also used).
> Despite not using any WITH_RETPOLINE I also catch the error ...

Something about this ZFS commit causes the boot laoder to be too big.
I guess the first sign of trouble was with retpolines, but there seem
to now be additional signs.

What's the process for filing a bug report against OpenzFS for
something like this? (Not asking you directly, just a general question
for the thread.)

That's a good question. We are rapidly aporoaching the = day we will have to freeze the set of zfs feature that we can boot with the= BIOS path. There's only so much space.

Right now,=C2=A0 there's little to no bootloader te= sting, let alone analysis done and that will need to change.

Warmer

Thanks,

--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Signal Username:=C2=A0 shawn_webb.74
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubk= eys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.as= c
--0000000000008161ef064d8a5058--