Date: Fri, 2 Aug 2024 10:35:27 -0600 From: Warner Losh <imp@bsdimp.com> To: Mark Johnston <markj@freebsd.org> Cc: Warner Losh <imp@freebsd.org>, src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: 46ea2ffc3fbc - main - stand: Reduce limit to 500k for x86 loader Message-ID: <CANCZdfquvUPk8TFYNjU76E4G6_rVRjmm0b48wNPbEHYg6moMbw@mail.gmail.com> In-Reply-To: <CANCZdfogCrUtSzbTyKvsM14%2B-rSrq_BAuf49t7gUDm_tSNrGrw@mail.gmail.com> References: <202408012131.471LVOGC039269@gitrepo.freebsd.org> <ZqztJ2mad4sErULr@nuc> <CANCZdfogCrUtSzbTyKvsM14%2B-rSrq_BAuf49t7gUDm_tSNrGrw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000e300dc061eb5eaa8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Aug 2, 2024 at 10:13=E2=80=AFAM Warner Losh <imp@bsdimp.com> wrote: > > > On Fri, Aug 2, 2024 at 8:29=E2=80=AFAM Mark Johnston <markj@freebsd.org> = wrote: > >> On Thu, Aug 01, 2024 at 09:31:24PM +0000, Warner Losh wrote: >> > The branch main has been updated by imp: >> > >> > URL: >> https://cgit.FreeBSD.org/src/commit/?id=3D46ea2ffc3fbc42089d8322a65fdee8= 476d2b00d6 >> > >> > commit 46ea2ffc3fbc42089d8322a65fdee8476d2b00d6 >> > Author: Warner Losh <imp@FreeBSD.org> >> > AuthorDate: 2024-08-01 21:24:51 +0000 >> > Commit: Warner Losh <imp@FreeBSD.org> >> > CommitDate: 2024-08-01 21:30:26 +0000 >> > >> > stand: Reduce limit to 500k for x86 loader >> > >> > The largest loader that works for PXE boot is about 500k. PXE need= s >> low >> > memory for packets and other driver state, so the largest safe siz= e >> for >> > the loader is about 500k. Reduce the size from 560k to 500k so we >> don't >> > accidentally break PXE in the future. >> > >> > Add a comment for people with special needs. If you control the >> > hardware, it can be safe to have boot loaders as large as 580k or >> 600k >> > in some cases. Since the BIOS loader is becoming more and more of = a >> > legacy item, the build variable LOADERSIZE isn't documented. This >> change >> > doesn't change that: there's been little demand for this >> documentation >> > and in general, users shouldn't change it lightly. >> > >> > PR: 257018 >> > Sponsored by: Netflix >> > --- >> > stand/i386/loader/Makefile | 7 ++++++- >> > 1 file changed, 6 insertions(+), 1 deletion(-) >> > >> > diff --git a/stand/i386/loader/Makefile b/stand/i386/loader/Makefile >> > index a4aa3a3c4d45..efd442977780 100644 >> > --- a/stand/i386/loader/Makefile >> > +++ b/stand/i386/loader/Makefile >> > @@ -32,7 +32,12 @@ VERSION_FILE=3D ${.CURDIR}/../loader/version >> > # >> > # will tell you how many kiB of lomem are available. >> > # >> > -LOADERSIZE?=3D 560000 # Largest known safe size for loader.b= in >> > +# We further reduce this to 500k, though, to give PXE an additional >> 64k of space >> > +# so pxeloader will fit. If you have special needs that do not includ= e >> pxeboot, >> > +# you can safely set this as high as 560000 generally, or a bit highe= r >> if you >> > +# have tight control over the machines you are booting on. >> > +# >> > +LOADERSIZE?=3D 500000 # Largest known safe size for loader.b= in >> >> Hi Warner, >> >> This breaks the WITH_BEARSSL (which implies WITH_LOADER_VERIEXEC) build. >> When enabled, the loader ends up being just slightly larger than the >> limit. >> > > I sent another reply, but don't see it now. I hope it doesn't go out sinc= e > its tone was off. > > So for me, on main, this combination works. I'm able to build everything > with those two options. > One can't build with just WITH_LOADER_VERIEXEC, though, you need both. > Even if it had been > too big, one can fix this with LOADERSIZE=3D510000. Though doing that > automatically is tricky, > for reasons below. At the very least, I should make a note of it in the > WITH_xxx options. So are > there other options you are using, or maybe a different clang version tha= n > I have (which I think is > tip of main, but might be tip of may from early july). > > However, as the comment states: this is a special needs case. Veriexec > for BIOS is a fringe > activity compared to PXEBOOT using BIOS, so we have to draw some hard > lines in the default > settings. This is the first of the hard lines: The boot loader can't be > larger than 500k, at least until > we can get good data that larger sizes work with pxeboot. So this limit > isn't going to change until > there's new data that comes in, since there's way more pxeboot users. The > build breakage for > pxeboot is silent, and its users can be slow to resport it through the > right channels. So this > is the lessor evil: Strict limits and working pxeboot at the cost (evil) > that some people that add > in non-default options will need to do other special things. > > But what to do. Do we warn when these options are enabled? Do we bump the > size? Do we > disable pxeboot? Do we just do that if it's larger than a certain amount > (or fail so unless you've > disabled it, you'll see it's too big)? > > I'm inclined to (1) document you may need to bump LOADERSIZE with these > options and > (2) fail the pxeboot build if loader_lua is > 500,000 and maybe (3) have > a WITHOUT_LOADER_PXEBOOT > option as there isn't one already. That way, if you bump the size, you ma= y > have to also disable > pxeboot to have a successful build, but it will all be explicit and we'll > document that sometimes > too many options results in size issues that need careful testing after > overriding the safety limits. > > There's also a dark horse in this: There's a bug / pull request (I can't > recall which) that uses LTO > and other weird, but clever, tricks to reduce the size of the loader. > IIRC, it was a big win for loader.efi, > but much smaller win (or maybe even a slight pessimization) for > /boot/loader. I'll look into that to > see if any of the techniques used there can eke us out some space here. > https://reviews.freebsd.org/D46211 has what I had in mind for (1). It's really the only one of the three I'm not sure what to do about. Warner --000000000000e300dc061eb5eaa8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Fri, Aug 2, 2024 at 10:13=E2=80=AF= AM Warner Losh <<a href=3D"mailto:imp@bsdimp.com">imp@bsdimp.com</a>>= wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px = 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir= =3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote"><div dir= =3D"ltr" class=3D"gmail_attr">On Fri, Aug 2, 2024 at 8:29=E2=80=AFAM Mark J= ohnston <<a href=3D"mailto:markj@freebsd.org" target=3D"_blank">markj@fr= eebsd.org</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style= =3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding= -left:1ex">On Thu, Aug 01, 2024 at 09:31:24PM +0000, Warner Losh wrote:<br> > The branch main has been updated by imp:<br> > <br> > URL: <a href=3D"https://cgit.FreeBSD.org/src/commit/?id=3D46ea2ffc3fbc= 42089d8322a65fdee8476d2b00d6" rel=3D"noreferrer" target=3D"_blank">https://= cgit.FreeBSD.org/src/commit/?id=3D46ea2ffc3fbc42089d8322a65fdee8476d2b00d6<= /a><br> > <br> > commit 46ea2ffc3fbc42089d8322a65fdee8476d2b00d6<br> > Author:=C2=A0 =C2=A0 =C2=A0Warner Losh <imp@FreeBSD.org><br> > AuthorDate: 2024-08-01 21:24:51 +0000<br> > Commit:=C2=A0 =C2=A0 =C2=A0Warner Losh <imp@FreeBSD.org><br> > CommitDate: 2024-08-01 21:30:26 +0000<br> > <br> >=C2=A0 =C2=A0 =C2=A0stand: Reduce limit to 500k for x86 loader<br> >=C2=A0 =C2=A0 =C2=A0<br> >=C2=A0 =C2=A0 =C2=A0The largest loader that works for PXE boot is about= 500k. PXE needs low<br> >=C2=A0 =C2=A0 =C2=A0memory for packets and other driver state, so the l= argest safe size for<br> >=C2=A0 =C2=A0 =C2=A0the loader is about 500k. Reduce the size from 560k= to 500k so we don't<br> >=C2=A0 =C2=A0 =C2=A0accidentally break PXE in the future.<br> >=C2=A0 =C2=A0 =C2=A0<br> >=C2=A0 =C2=A0 =C2=A0Add a comment for people with special needs. If you= control the<br> >=C2=A0 =C2=A0 =C2=A0hardware, it can be safe to have boot loaders as la= rge as 580k or 600k<br> >=C2=A0 =C2=A0 =C2=A0in some cases. Since the BIOS loader is becoming mo= re and more of a<br> >=C2=A0 =C2=A0 =C2=A0legacy item, the build variable LOADERSIZE isn'= t documented. This change<br> >=C2=A0 =C2=A0 =C2=A0doesn't change that: there's been little de= mand for this documentation<br> >=C2=A0 =C2=A0 =C2=A0and in general, users shouldn't change it light= ly.<br> >=C2=A0 =C2=A0 =C2=A0<br> >=C2=A0 =C2=A0 =C2=A0PR: 257018<br> >=C2=A0 =C2=A0 =C2=A0Sponsored by: Netflix<br> > ---<br> >=C2=A0 stand/i386/loader/Makefile | 7 ++++++-<br> >=C2=A0 1 file changed, 6 insertions(+), 1 deletion(-)<br> > <br> > diff --git a/stand/i386/loader/Makefile b/stand/i386/loader/Makefile<b= r> > index a4aa3a3c4d45..efd442977780 100644<br> > --- a/stand/i386/loader/Makefile<br> > +++ b/stand/i386/loader/Makefile<br> > @@ -32,7 +32,12 @@ VERSION_FILE=3D=C2=A0 =C2=A0 =C2=A0 ${.CURDIR}/../l= oader/version<br> >=C2=A0 #<br> >=C2=A0 # will tell you how many kiB of lomem are available.<br> >=C2=A0 #<br> > -LOADERSIZE?=3D 560000=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 # Largest kno= wn safe size for loader.bin<br> > +# We further reduce this to 500k, though, to give PXE an additional 6= 4k of space<br> > +# so pxeloader will fit. If you have special needs that do not includ= e pxeboot,<br> > +# you can safely set this as high as 560000 generally, or a bit highe= r if you<br> > +# have tight control over the machines you are booting on.<br> > +#<br> > +LOADERSIZE?=3D 500000=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 # Largest kno= wn safe size for loader.bin<br> <br> Hi Warner,<br> <br> This breaks the WITH_BEARSSL (which implies WITH_LOADER_VERIEXEC) build.<br= > When enabled, the loader ends up being just slightly larger than the<br> limit.<br></blockquote><div><br></div><div>I sent another reply, but don= 9;t see it now. I hope it doesn't go out since its tone was off.</div><= div><br></div><div>So for me, on main, this combination works. I'm able= to build everything with those two options.</div><div>One can't build = with just WITH_LOADER_VERIEXEC, though, you need both. Even if it had been<= /div><div>too big, one can fix this with LOADERSIZE=3D510000. Though doing = that automatically is tricky,</div><div>for reasons below. At the very leas= t, I should make a note of it in the WITH_xxx options. So are</div><div>the= re other options you are using, or maybe a different clang version than I h= ave (which I think is</div><div>tip of main, but might be tip of may from e= arly july).</div><div><br></div><div>However, as=C2=A0 the comment states: = this is a special needs case. Veriexec for BIOS is a fringe</div><div>activ= ity compared to PXEBOOT=C2=A0using BIOS, so we have to draw some hard lines= in the default</div><div>settings. This is the first of the hard lines: Th= e boot loader can't be larger than 500k, at least until</div><div>we ca= n get good data that larger sizes work with pxeboot. So this limit isn'= t going to change until</div><div>there's new data that comes in, since= there's way more pxeboot=C2=A0users. The build breakage for</div><div>= pxeboot=C2=A0is silent, and its users can be slow to resport=C2=A0it throug= h the right channels. So this</div><div>is the lessor evil: Strict limits a= nd working pxeboot=C2=A0at the cost (evil) that some people that add</div><= div>in non-default options will need to do other special things.</div><div>= <br></div><div>But what to do. Do we warn when these options are enabled? D= o we bump the size? Do we</div><div>disable pxeboot? Do we just do that if = it's larger than a certain amount (or fail so unless you've</div><d= iv>disabled it, you'll see it's too big)?</div><div><br></div><div>= I'm inclined to (1) document you may need to bump LOADERSIZE with these= options and</div><div>(2) fail the pxeboot=C2=A0 build if loader_lua is &g= t; 500,000 and maybe (3) have a WITHOUT_LOADER_PXEBOOT</div><div>option as = there isn't one already. That way, if you bump the size, you may have t= o also disable</div><div>pxeboot=C2=A0to have a successful build, but it wi= ll all be explicit and we'll document that sometimes</div><div>too many= options results in size issues that need careful testing after overriding = the safety limits.</div><div><br></div><div>There's also a dark horse i= n this: There's a bug / pull request (I can't recall which) that us= es LTO</div><div>and other weird, but clever, tricks to reduce the size of = the loader. IIRC, it was a big win for loader.efi,</div><div>but much small= er win (or maybe even a slight pessimization) for /boot/loader. I'll lo= ok into that to</div><div>see if any of the techniques used there can eke u= s out some space here.=C2=A0</div></div></div></blockquote><div><br></div><= div><a href=3D"https://reviews.freebsd.org/D46211">https://reviews.freebsd.= org/D46211</a> has what I had in mind for (1). It's really the only one= of the three</div><div>I'm not sure what to do about.</div><div><br></= div><div>Warner=C2=A0</div></div></div> --000000000000e300dc061eb5eaa8--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfquvUPk8TFYNjU76E4G6_rVRjmm0b48wNPbEHYg6moMbw>