Date: Fri, 2 Aug 2024 10:13:51 -0600 From: Warner Losh <imp@bsdimp.com> To: Mark Johnston <markj@freebsd.org> Cc: Warner Losh <imp@freebsd.org>, src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: 46ea2ffc3fbc - main - stand: Reduce limit to 500k for x86 loader Message-ID: <CANCZdfogCrUtSzbTyKvsM14%2B-rSrq_BAuf49t7gUDm_tSNrGrw@mail.gmail.com> In-Reply-To: <ZqztJ2mad4sErULr@nuc> References: <202408012131.471LVOGC039269@gitrepo.freebsd.org> <ZqztJ2mad4sErULr@nuc>
next in thread | previous in thread | raw e-mail | index | archive | help
--0000000000009ec51f061eb59dd1 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Aug 2, 2024 at 8:29=E2=80=AFAM Mark Johnston <markj@freebsd.org> wr= ote: > On Thu, Aug 01, 2024 at 09:31:24PM +0000, Warner Losh wrote: > > The branch main has been updated by imp: > > > > URL: > https://cgit.FreeBSD.org/src/commit/?id=3D46ea2ffc3fbc42089d8322a65fdee84= 76d2b00d6 > > > > commit 46ea2ffc3fbc42089d8322a65fdee8476d2b00d6 > > Author: Warner Losh <imp@FreeBSD.org> > > AuthorDate: 2024-08-01 21:24:51 +0000 > > Commit: Warner Losh <imp@FreeBSD.org> > > CommitDate: 2024-08-01 21:30:26 +0000 > > > > stand: Reduce limit to 500k for x86 loader > > > > The largest loader that works for PXE boot is about 500k. PXE needs > low > > memory for packets and other driver state, so the largest safe size > for > > the loader is about 500k. Reduce the size from 560k to 500k so we > don't > > accidentally break PXE in the future. > > > > Add a comment for people with special needs. If you control the > > hardware, it can be safe to have boot loaders as large as 580k or > 600k > > in some cases. Since the BIOS loader is becoming more and more of a > > legacy item, the build variable LOADERSIZE isn't documented. This > change > > doesn't change that: there's been little demand for this > documentation > > and in general, users shouldn't change it lightly. > > > > PR: 257018 > > Sponsored by: Netflix > > --- > > stand/i386/loader/Makefile | 7 ++++++- > > 1 file changed, 6 insertions(+), 1 deletion(-) > > > > diff --git a/stand/i386/loader/Makefile b/stand/i386/loader/Makefile > > index a4aa3a3c4d45..efd442977780 100644 > > --- a/stand/i386/loader/Makefile > > +++ b/stand/i386/loader/Makefile > > @@ -32,7 +32,12 @@ VERSION_FILE=3D ${.CURDIR}/../loader/version > > # > > # will tell you how many kiB of lomem are available. > > # > > -LOADERSIZE?=3D 560000 # Largest known safe size for loader.bi= n > > +# We further reduce this to 500k, though, to give PXE an additional 64= k > of space > > +# so pxeloader will fit. If you have special needs that do not include > pxeboot, > > +# you can safely set this as high as 560000 generally, or a bit higher > if you > > +# have tight control over the machines you are booting on. > > +# > > +LOADERSIZE?=3D 500000 # Largest known safe size for loader.bi= n > > Hi Warner, > > This breaks the WITH_BEARSSL (which implies WITH_LOADER_VERIEXEC) build. > When enabled, the loader ends up being just slightly larger than the > limit. > I sent another reply, but don't see it now. I hope it doesn't go out since its tone was off. So for me, on main, this combination works. I'm able to build everything with those two options. One can't build with just WITH_LOADER_VERIEXEC, though, you need both. Even if it had been too big, one can fix this with LOADERSIZE=3D510000. Though doing that automatically is tricky, for reasons below. At the very least, I should make a note of it in the WITH_xxx options. So are there other options you are using, or maybe a different clang version than I have (which I think is tip of main, but might be tip of may from early july). However, as the comment states: this is a special needs case. Veriexec for BIOS is a fringe activity compared to PXEBOOT using BIOS, so we have to draw some hard lines in the default settings. This is the first of the hard lines: The boot loader can't be larger than 500k, at least until we can get good data that larger sizes work with pxeboot. So this limit isn't going to change until there's new data that comes in, since there's way more pxeboot users. The build breakage for pxeboot is silent, and its users can be slow to resport it through the right channels. So this is the lessor evil: Strict limits and working pxeboot at the cost (evil) that some people that add in non-default options will need to do other special things. But what to do. Do we warn when these options are enabled? Do we bump the size? Do we disable pxeboot? Do we just do that if it's larger than a certain amount (or fail so unless you've disabled it, you'll see it's too big)? I'm inclined to (1) document you may need to bump LOADERSIZE with these options and (2) fail the pxeboot build if loader_lua is > 500,000 and maybe (3) have a WITHOUT_LOADER_PXEBOOT option as there isn't one already. That way, if you bump the size, you may have to also disable pxeboot to have a successful build, but it will all be explicit and we'll document that sometimes too many options results in size issues that need careful testing after overriding the safety limits. There's also a dark horse in this: There's a bug / pull request (I can't recall which) that uses LTO and other weird, but clever, tricks to reduce the size of the loader. IIRC, it was a big win for loader.efi, but much smaller win (or maybe even a slight pessimization) for /boot/loader. I'll look into that to see if any of the techniques used there can eke us out some space here. Warner --0000000000009ec51f061eb59dd1 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Fri, Aug 2, 2024 at 8:29=E2=80=AFA= M Mark Johnston <<a href=3D"mailto:markj@freebsd.org">markj@freebsd.org<= /a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0= px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">O= n Thu, Aug 01, 2024 at 09:31:24PM +0000, Warner Losh wrote:<br> > The branch main has been updated by imp:<br> > <br> > URL: <a href=3D"https://cgit.FreeBSD.org/src/commit/?id=3D46ea2ffc3fbc= 42089d8322a65fdee8476d2b00d6" rel=3D"noreferrer" target=3D"_blank">https://= cgit.FreeBSD.org/src/commit/?id=3D46ea2ffc3fbc42089d8322a65fdee8476d2b00d6<= /a><br> > <br> > commit 46ea2ffc3fbc42089d8322a65fdee8476d2b00d6<br> > Author:=C2=A0 =C2=A0 =C2=A0Warner Losh <imp@FreeBSD.org><br> > AuthorDate: 2024-08-01 21:24:51 +0000<br> > Commit:=C2=A0 =C2=A0 =C2=A0Warner Losh <imp@FreeBSD.org><br> > CommitDate: 2024-08-01 21:30:26 +0000<br> > <br> >=C2=A0 =C2=A0 =C2=A0stand: Reduce limit to 500k for x86 loader<br> >=C2=A0 =C2=A0 =C2=A0<br> >=C2=A0 =C2=A0 =C2=A0The largest loader that works for PXE boot is about= 500k. PXE needs low<br> >=C2=A0 =C2=A0 =C2=A0memory for packets and other driver state, so the l= argest safe size for<br> >=C2=A0 =C2=A0 =C2=A0the loader is about 500k. Reduce the size from 560k= to 500k so we don't<br> >=C2=A0 =C2=A0 =C2=A0accidentally break PXE in the future.<br> >=C2=A0 =C2=A0 =C2=A0<br> >=C2=A0 =C2=A0 =C2=A0Add a comment for people with special needs. If you= control the<br> >=C2=A0 =C2=A0 =C2=A0hardware, it can be safe to have boot loaders as la= rge as 580k or 600k<br> >=C2=A0 =C2=A0 =C2=A0in some cases. Since the BIOS loader is becoming mo= re and more of a<br> >=C2=A0 =C2=A0 =C2=A0legacy item, the build variable LOADERSIZE isn'= t documented. This change<br> >=C2=A0 =C2=A0 =C2=A0doesn't change that: there's been little de= mand for this documentation<br> >=C2=A0 =C2=A0 =C2=A0and in general, users shouldn't change it light= ly.<br> >=C2=A0 =C2=A0 =C2=A0<br> >=C2=A0 =C2=A0 =C2=A0PR: 257018<br> >=C2=A0 =C2=A0 =C2=A0Sponsored by: Netflix<br> > ---<br> >=C2=A0 stand/i386/loader/Makefile | 7 ++++++-<br> >=C2=A0 1 file changed, 6 insertions(+), 1 deletion(-)<br> > <br> > diff --git a/stand/i386/loader/Makefile b/stand/i386/loader/Makefile<b= r> > index a4aa3a3c4d45..efd442977780 100644<br> > --- a/stand/i386/loader/Makefile<br> > +++ b/stand/i386/loader/Makefile<br> > @@ -32,7 +32,12 @@ VERSION_FILE=3D=C2=A0 =C2=A0 =C2=A0 ${.CURDIR}/../l= oader/version<br> >=C2=A0 #<br> >=C2=A0 # will tell you how many kiB of lomem are available.<br> >=C2=A0 #<br> > -LOADERSIZE?=3D 560000=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 # Largest kno= wn safe size for loader.bin<br> > +# We further reduce this to 500k, though, to give PXE an additional 6= 4k of space<br> > +# so pxeloader will fit. If you have special needs that do not includ= e pxeboot,<br> > +# you can safely set this as high as 560000 generally, or a bit highe= r if you<br> > +# have tight control over the machines you are booting on.<br> > +#<br> > +LOADERSIZE?=3D 500000=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 # Largest kno= wn safe size for loader.bin<br> <br> Hi Warner,<br> <br> This breaks the WITH_BEARSSL (which implies WITH_LOADER_VERIEXEC) build.<br= > When enabled, the loader ends up being just slightly larger than the<br> limit.<br></blockquote><div><br></div><div>I sent another reply, but don= 9;t see it now. I hope it doesn't go out since its tone was off.</div><= div><br></div><div>So for me, on main, this combination works. I'm able= to build everything with those two options.</div><div>One can't build = with just WITH_LOADER_VERIEXEC, though, you need both. Even if it had been<= /div><div>too big, one can fix this with LOADERSIZE=3D510000. Though doing = that automatically is tricky,</div><div>for reasons below. At the very leas= t, I should make a note of it in the WITH_xxx options. So are</div><div>the= re other options you are using, or maybe a different clang version than I h= ave (which I think is</div><div>tip of main, but might be tip of may from e= arly july).</div><div><br></div><div>However, as=C2=A0 the comment states: = this is a special needs case. Veriexec for BIOS is a fringe</div><div>activ= ity compared to PXEBOOT=C2=A0using BIOS, so we have to draw some hard lines= in the default</div><div>settings. This is the first of the hard lines: Th= e boot loader can't be larger than 500k, at least until</div><div>we ca= n get good data that larger sizes work with pxeboot. So this limit isn'= t going to change until</div><div>there's new data that comes in, since= there's way more pxeboot=C2=A0users. The build breakage for</div><div>= pxeboot=C2=A0is silent, and its users can be slow to resport=C2=A0it throug= h the right channels. So this</div><div>is the lessor evil: Strict limits a= nd working pxeboot=C2=A0at the cost (evil) that some people that add</div><= div>in non-default options will need to do other special things.</div><div>= <br></div><div>But what to do. Do we warn when these options are enabled? D= o we bump the size? Do we</div><div>disable pxeboot? Do we just do that if = it's larger than a certain amount (or fail so unless you've</div><d= iv>disabled it, you'll see it's too big)?</div><div><br></div><div>= I'm inclined to (1) document you may need to bump LOADERSIZE with these= options and</div><div>(2) fail the pxeboot=C2=A0 build if loader_lua is &g= t; 500,000 and maybe (3) have a WITHOUT_LOADER_PXEBOOT</div><div>option as = there isn't one already. That way, if you bump the size, you may have t= o also disable</div><div>pxeboot=C2=A0to have a successful build, but it wi= ll all be explicit and we'll document that sometimes</div><div>too many= options results in size issues that need careful testing after overriding = the safety limits.</div><div><br></div><div>There's also a dark horse i= n this: There's a bug / pull request (I can't recall which) that us= es LTO</div><div>and other weird, but clever, tricks to reduce the size of = the loader. IIRC, it was a big win for loader.efi,</div><div>but much small= er win (or maybe even a slight pessimization) for /boot/loader. I'll lo= ok into that to</div><div>see if any of the techniques used there can eke u= s out some space here.=C2=A0</div><div><br></div><div>Warner</div></div></d= iv> --0000000000009ec51f061eb59dd1--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfogCrUtSzbTyKvsM14%2B-rSrq_BAuf49t7gUDm_tSNrGrw>