From nobody Sat Mar 21 15:18:20 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4fdNPB3djYz6VZFw for ; Sat, 21 Mar 2026 15:18:38 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4fdNPB0nkkz3SB9 for ; Sat, 21 Mar 2026 15:18:38 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x1035.google.com with SMTP id 98e67ed59e1d1-35a1f3f07ebso736384a91.3 for ; Sat, 21 Mar 2026 08:18:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1774106312; cv=none; d=google.com; s=arc-20240605; b=LtEq3JoXV9PCnJTwfVyFWb3I8Q9Bw5Dq2q90RDy53exOU3uwMunvn2e2i/NJmYQbem +JutCwtLEXRS1hOSsywLqI+NFOapoyJNEnLkVxxFbmv1vqEZm+iOdFVg8WCep1epIkBZ 3JN9obrvPFofaJtMHrYz76NBe9wBg/sycXbjzzryLTslsUkeM28mQprqlKgzaO5VnmHV F/8CCVHi1zVlv6niiFUeNaXH4E7Os5zbn+uEB23S8WM3hb13iLrOzWiOF9rfc0UO9iBN kqzp+kNW5SJo3KJ/V+yTFcym+IXS64CuC1Xs/sF2Qtcw9VpJBG0a9fiio478K4p2rsca nfVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=T25g4ScAAc8CyE4pVDio6ZlWtdDZwriXPdwK8wrJaX0=; fh=F40JOpcMkFRxL8G0I9UdLEa+kSNpUsmgdhLN3FmlasQ=; b=R4YvNzUrMcYvCdA0rRCuCkGw1OrYNk/pmfgegZ2yGbnS7cMZkMiCfz5s150fDqaXzn DxZsWlOVq1B1rVmCPJXGWvo+4hHtDiPIsAz4amH4NKUyH8oIZsYySc6OyBJhDHD3BpPj Vxot15G6creOv5J+mUTwJY+X2NKGOlNrWnafpmh5uvEL2HgS4BOPKr5Zf8hGHqHLAq4r BtARFZO2oSDRYMHlvDVTNuAkfRX3B4FosxcBEYkQwXSW3kjMZY7uZkz6meYDhdo8Is2R jSqF5qmOXfV/we3yuqfwXjVsJFWr317tvn058OS1kqK8SHE5ktVveMNtW/hy1/2aukxn B9yQ==; darn=freebsd.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1774106312; x=1774711112; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=T25g4ScAAc8CyE4pVDio6ZlWtdDZwriXPdwK8wrJaX0=; b=AH2oWQ1pmvIh/9WtO3zQ+GYIgx/NdNhLoLa0EM5kPiidALPKxTRyTxlubNSZL5sgYZ bareVET1eXESudp+HJ5u1capeGx5EkLYrylQhfr/eCJPeKoJLa1z/aQ7H7lrOTH1POzg +Q1rjkScPcX2F7jpCL/LwgMUY6nJ5Zf+8vmsjCaRqqrmJQVm5K50Yk+4Req90EvZJcGE knlmYn4Jjrqrrhv9O2rPXvVrUdzThwLzMeX3hCmPNH0QzV60XW/MC4Yy9PJNDxj/1UIY sCrcLg5fxXpqS316LZMg3pzwVxl6PboO3gEP/ORboky1t3j6BXnbYC1Y7QRxERopYFhg YHmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774106312; x=1774711112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=T25g4ScAAc8CyE4pVDio6ZlWtdDZwriXPdwK8wrJaX0=; b=oLOS2/wojTFSDnJfvQsqxGuyxNGXa5Ol3Mvud+LMPXcnAdV4KukCW19QirISMB8Vsr uel6kvVdpJBvvjanNl4z4pk8XWXWqUDO1/C0xfonvULfuLlywn7fdj1YNniuZkNYRvfc xQot2Rtq/0xNhKiHF0JJK0ae9ZIyICg/Q1Bu5YAxwBR0MA2QzbpSHc5DP1FxxLG4oEXj hse2NkUgorwrUQrdo0DQVLP7D/DEgHt28sttsKssjdPu2AqgOjsdNuvWyHd4vKVu3OFo 6eyIxnbwCzym0sdEOjNnteoGFNa4Od7LBiYLEsXLNFG49aJQ8LlDS2He3fZ8/UqJ0+iD K63g== X-Forwarded-Encrypted: i=1; AJvYcCUwnSiR9Ck7rStsTn16FIRdNMVOrwh7pZfim2gyUD7awBzXyPM/9iSCMlTD0tEN3CglxAdTbQgoeOT1WwspLcH0FE34@freebsd.org X-Gm-Message-State: AOJu0YwhrNLRhyS+wzjukPaa/fUiZsOU4zwSbigPvPnL4maNIjm24wbv b87Ft1zg0CuHYcKol6/vjBpooHeIWsA6RuzusuDqIgfECTwBJ1mz5kwxPGOn3dpQnCaS7awiKkT Om2qKZaWH4ItcDdovuWv2JOM7+09yxJjgVgryybqx1w== X-Gm-Gg: ATEYQzzxHquHwy6fpCW6SxOgHFYGHM83woPJgJOJ6ORBRBUd+ftrSEQTwmQFflVFpyj aoc4r/qevZohgmpNAnB5nXX4yvU1Ukg8L6QndjK1hg9ZJL9KMlgCptw/If7Peno+9fkYW95qm75 wtBkRLk5gqM+71szZc1J/gUKa0MvQiIQ7x7zbeOsmC6ikpVy8SNT6I+x39XugABEA+cVUzzpG5s X2qBLAj5Yl9GRbP/6SVJxVsDvZEnnFdIZDyfXg/HSwWcUA0+obluXfAqQ3tCAxKoCVT2zoJ7Upa VBuSLbdgV7e1nCGhR0lr8/JZdPoWoATbWgzUxzHs6LDGPy44 X-Received: by 2002:a17:903:244b:b0:2b0:775f:febf with SMTP id d9443c01a7336-2b0827e31acmr63966655ad.40.1774106311742; Sat, 21 Mar 2026 08:18:31 -0700 (PDT) List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 References: <69b561ff.39ea9.b797d91@gitrepo.freebsd.org> <2jb2vg6baofimu5xkxf62o5ogaq7fu5pk4o3vzhpegy446bppf@fzwtj6wtwk53> <20260321090444.6f81da15@thor.sb211.local> In-Reply-To: From: Warner Losh Date: Sat, 21 Mar 2026 09:18:20 -0600 X-Gm-Features: AaiRm50jsK9w9KGFmhnyVX868rHtrjXKQvghCz14hpLGiqYfWqe11lUKPWQ9hII Message-ID: Subject: Re: git: 8a62a2a5659d - main - zfs: merge openzfs/zfs@f8e5af53e To: Shawn Webb Cc: A FreeBSD User , Charlie Li , Martin Matuska , src-committers , "" , "" Content-Type: multipart/alternative; boundary="0000000000008161ef064d8a5058" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4fdNPB0nkkz3SB9 X-Spamd-Bar: ---- --0000000000008161ef064d8a5058 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Mar 21, 2026, 9:08=E2=80=AFAM Shawn Webb wrote: > On Sat, Mar 21, 2026 at 09:04:17AM +0100, A FreeBSD User wrote: > > Am Tage des Herren Fri, 20 Mar 2026 23:27:20 -0400 > > Charlie Li schrieb: > > > > > Shawn Webb wrote: > > > > On Tue, Mar 17, 2026 at 04:52:16PM +0000, Shawn Webb wrote: > > > >> On Tue, Mar 17, 2026 at 10:44:59AM -0600, Warner Losh wrote: > > > >>> On Tue, Mar 17, 2026 at 10:36=E2=80=AFAM Shawn Webb < > shawn.webb@hardenedbsd.org> > > > >>> wrote: > > > >>> > > > >>>> Hey Martin, > > > >>>> > > > >>>> On Sat, Mar 14, 2026 at 01:26:23PM +0000, Martin Matuska wrote: > > > >>>>> The branch main has been updated by mm: > > > >>>>> > > > >>>>> URL: > > > >>>> > https://cgit.FreeBSD.org/src/commit/?id=3D8a62a2a5659d1839d8799b4274c0446= 9d7f17c78 > > > > >>>>> > > > >>>>> commit 8a62a2a5659d1839d8799b4274c04469d7f17c78 > > > >>>>> Merge: f91464171d61 f8e5af53e92f > > > >>>>> Author: Martin Matuska > > > >>>>> AuthorDate: 2026-03-14 12:14:56 +0000 > > > >>>>> Commit: Martin Matuska > > > >>>>> CommitDate: 2026-03-14 12:14:56 +0000 > > > >>>>> > > > >>>>> [snip for brevity] > > > >>>>> > > > >>>>> Obtained from: OpenZFS > > > >>>>> OpenZFS commit: f8e5af53e92fa7c03393fbd4922cb9c1d0c15920 > > > >>>> > > > >>>> This commit seems to cause issues when building boot loader > related > > > >>>> code: > > > >>>> > > > >>>> =3D=3D=3D=3D BEGIN LOG =3D=3D=3D=3D > > > >>>> 114232 bytes available > > > >>>> btxld -v -f aout -e 0x200000 -o loader_simp -l > > > >>>> /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btxldr/btxldr -b > > > >>>> /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btx/btx > loader_simp.bin > > > >>>> kernel: ver=3D1.02 size=3D690 load=3D9000 entry=3D9010 map=3D16M= pgctl=3D0:58 > > > >>>> client: fmt=3Delf size=3D5e2e8 text=3D57930 data=3D514c bss=3D74= 70 entry=3D0 > > > >>>> output: fmt=3Daout size=3D61000 text=3D1000 data=3D5f000 org=3D2= 00000 > entry=3D200000 > > > >>>> =3D=3D=3D> stand/i386/pxeldr (all) > > > >>>> -560 bytes available > > > >>>> *** Error code 1 > > > >>>> > > > >>> > > > >>> What all do you have enabled? The defaults aren't even close to > running out > > > >>> of space (though I've not looked at this). > > > >> > > > >> Hey Warner, > > > >> > > > >> Thanks for reaching out! I've uploaded `make showconfig` here: > > > >> https://hardenedbsd.org/~shawn/2026-03-17_srcconf-r01.txt > > > >> > > > >> The following options are specific to HardenedBSD (in no particula= r > > > >> order): > > > >> > > > >> 1. MK_HBSD_UPDATE > > > >> 2. MK_HBSDCONTROL > > > >> 3. MK_PIE > > > >> 4. MK_RELRO > > > >> 5. MK_SHLIBRANDOM > > > >> 6. MK_ZERO_REGS > > > >> 7. MK_SPECTREV1_FIX > > > >> 8. MK_SAFESTACK > > > >> 9. MK_RETPOLINE > > > >> 10. MK_LTOLIB > > > >> 11. MK_CFI > > > > > > > > MK_RETPOLINE was the culprit. Something about this ZFS commit cause= s > > > > LLVM to emit more retpoline entries than before--too many for a > little > > > > bootloader. That might be something to investigate later, but only = to > > > > satisfy a curious mind, not to actuall fix anything (since nothing'= s > > > > actually broken.) > > > > > > > > Since it doesn't really make sense to apply speculative execution > > > > mitigations to a bootloader, I disabled retpoline for a components > > > > in stand/. > > > > > > > > Good to go. > > > > > > > Also just got bit by this, albeit during the lua loader, since I have > > > WITH_RETPOLINE in my src.conf. > > > > > > > Hello, > > > > I do not have WITH_RETPOLINE in my /etc/src.conf, but since I got this > mysterious error about > > not enough bytes left, I use WITHOUT_LOADER_PXEBOOT=3D YES (due to issu= es > with WITH_BEARSSL=3DYES > > also used). > > Despite not using any WITH_RETPOLINE I also catch the error ... > > Something about this ZFS commit causes the boot laoder to be too big. > I guess the first sign of trouble was with retpolines, but there seem > to now be additional signs. > > What's the process for filing a bug report against OpenzFS for > something like this? (Not asking you directly, just a general question > for the thread.) > That's a good question. We are rapidly aporoaching the day we will have to freeze the set of zfs feature that we can boot with the BIOS path. There's only so much space. Right now, there's little to no bootloader testing, let alone analysis done and that will need to change. Warmer Thanks, > > -- > Shawn Webb > Cofounder / Security Engineer > HardenedBSD > > Signal Username: shawn_webb.74 > Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 > > https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/0= 3A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc > --0000000000008161ef064d8a5058 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sat, Mar 21, 2026, 9:08=E2=80= =AFAM Shawn Webb <shawn.we= bb@hardenedbsd.org> wrote:
O= n Sat, Mar 21, 2026 at 09:04:17AM +0100, A FreeBSD User wrote:
> Am Tage des Herren Fri, 20 Mar 2026 23:27:20 -0400
> Charlie Li <vishwin@freebsd.org> schrieb:
>
> > Shawn Webb wrote:
> > > On Tue, Mar 17, 2026 at 04:52:16PM +0000, Shawn Webb wrote:= =C2=A0
> > >> On Tue, Mar 17, 2026 at 10:44:59AM -0600, Warner Losh wr= ote:=C2=A0
> > >>> On Tue, Mar 17, 2026 at 10:36=E2=80=AFAM Shawn Webb = <shawn.webb@hardenedbsd.org>
> > >>> wrote:
> > >>>=C2=A0
> > >>>> Hey Martin,
> > >>>>
> > >>>> On Sat, Mar 14, 2026 at 01:26:23PM +0000, Martin= Matuska wrote:=C2=A0
> > >>>>> The branch main has been updated by mm:
> > >>>>>
> > >>>>> URL:=C2=A0
> > >>>> https://cgit.FreeBSD.org/src/commit/?id=3D8a62a2a5659d1= 839d8799b4274c04469d7f17c78=C2=A0
> > >>>>>
> > >>>>> commit 8a62a2a5659d1839d8799b4274c04469d7f17= c78
> > >>>>> Merge: f91464171d61 f8e5af53e92f
> > >>>>> Author:=C2=A0 =C2=A0 =C2=A0Martin Matuska &l= t;mm@FreeBSD.org>
> > >>>>> AuthorDate: 2026-03-14 12:14:56 +0000
> > >>>>> Commit:=C2=A0 =C2=A0 =C2=A0Martin Matuska &l= t;mm@FreeBSD.org>
> > >>>>> CommitDate: 2026-03-14 12:14:56 +0000
> > >>>>>
> > >>>>> [snip for brevity]
> > >>>>>
> > >>>>>=C2=A0 =C2=A0 =C2=A0 Obtained from:=C2=A0 Ope= nZFS
> > >>>>>=C2=A0 =C2=A0 =C2=A0 OpenZFS commit: f8e5af53= e92fa7c03393fbd4922cb9c1d0c15920=C2=A0
> > >>>>
> > >>>> This commit seems to cause issues when building = boot loader related
> > >>>> code:
> > >>>>
> > >>>> =3D=3D=3D=3D BEGIN LOG =3D=3D=3D=3D
> > >>>> 114232 bytes available
> > >>>> btxld -v -f aout -e 0x200000 -o loader_simp -l > > >>>> /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btxl= dr/btxldr=C2=A0 -b
> > >>>> /usr/obj/usr/src/amd64.amd64/stand/i386/btx/btx/= btx loader_simp.bin
> > >>>> kernel: ver=3D1.02 size=3D690 load=3D9000 entry= =3D9010 map=3D16M pgctl=3D0:58
> > >>>> client: fmt=3Delf size=3D5e2e8 text=3D57930 data= =3D514c bss=3D7470 entry=3D0
> > >>>> output: fmt=3Daout size=3D61000 text=3D1000 data= =3D5f000 org=3D200000 entry=3D200000=C2=A0
> > >>>> =3D=3D=3D> stand/i386/pxeldr (all)=C2=A0
> > >>>> -560 bytes available
> > >>>> *** Error code 1
> > >>>>=C2=A0
> > >>>
> > >>> What all do you have enabled? The defaults aren'= t even close to running out
> > >>> of space (though I've not looked at this).=C2=A0=
> > >>
> > >> Hey Warner,
> > >>
> > >> Thanks for reaching out! I've uploaded `make showcon= fig` here:
> > >> https://harde= nedbsd.org/~shawn/2026-03-17_srcconf-r01.txt
> > >>
> > >> The following options are specific to HardenedBSD (in no= particular
> > >> order):
> > >>
> > >> 1. MK_HBSD_UPDATE
> > >> 2. MK_HBSDCONTROL
> > >> 3. MK_PIE
> > >> 4. MK_RELRO
> > >> 5. MK_SHLIBRANDOM
> > >> 6. MK_ZERO_REGS
> > >> 7. MK_SPECTREV1_FIX
> > >> 8. MK_SAFESTACK
> > >> 9. MK_RETPOLINE
> > >> 10. MK_LTOLIB
> > >> 11. MK_CFI=C2=A0
> > >
> > > MK_RETPOLINE was the culprit. Something about this ZFS commi= t causes
> > > LLVM to emit more retpoline entries than before--too many fo= r a little
> > > bootloader. That might be something to investigate later, bu= t only to
> > > satisfy a curious mind, not to actuall fix anything (since n= othing's
> > > actually broken.)
> > >
> > > Since it doesn't really make sense to apply speculative = execution
> > > mitigations to a bootloader, I disabled retpoline for a comp= onents
> > > in stand/.
> > >
> > > Good to go.
> > >=C2=A0 =C2=A0
> > Also just got bit by this, albeit during the lua loader, since I = have
> > WITH_RETPOLINE in my src.conf.
> >
>
> Hello,
>
> I do not have WITH_RETPOLINE in my /etc/src.conf, but since I got this= mysterious error about
> not enough bytes left, I use WITHOUT_LOADER_PXEBOOT=3D YES (due to iss= ues with WITH_BEARSSL=3DYES
> also used).
> Despite not using any WITH_RETPOLINE I also catch the error ...

Something about this ZFS commit causes the boot laoder to be too big.
I guess the first sign of trouble was with retpolines, but there seem
to now be additional signs.

What's the process for filing a bug report against OpenzFS for
something like this? (Not asking you directly, just a general question
for the thread.)

That's a good question. We are rapidly aporoaching the = day we will have to freeze the set of zfs feature that we can boot with the= BIOS path. There's only so much space.

Right now,=C2=A0 there's little to no bootloader te= sting, let alone analysis done and that will need to change.

Warmer

Thanks,

--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Signal Username:=C2=A0 shawn_webb.74
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubk= eys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.as= c
--0000000000008161ef064d8a5058--