Date: Fri, 2 Aug 2024 10:35:27 -0600 From: Warner Losh <imp@bsdimp.com> To: Mark Johnston <markj@freebsd.org> Cc: Warner Losh <imp@freebsd.org>, src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: 46ea2ffc3fbc - main - stand: Reduce limit to 500k for x86 loader Message-ID: <CANCZdfquvUPk8TFYNjU76E4G6_rVRjmm0b48wNPbEHYg6moMbw@mail.gmail.com> In-Reply-To: <CANCZdfogCrUtSzbTyKvsM14%2B-rSrq_BAuf49t7gUDm_tSNrGrw@mail.gmail.com> References: <202408012131.471LVOGC039269@gitrepo.freebsd.org> <ZqztJ2mad4sErULr@nuc> <CANCZdfogCrUtSzbTyKvsM14%2B-rSrq_BAuf49t7gUDm_tSNrGrw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] On Fri, Aug 2, 2024 at 10:13 AM Warner Losh <imp@bsdimp.com> wrote: > > > On Fri, Aug 2, 2024 at 8:29 AM Mark Johnston <markj@freebsd.org> wrote: > >> On Thu, Aug 01, 2024 at 09:31:24PM +0000, Warner Losh wrote: >> > The branch main has been updated by imp: >> > >> > URL: >> https://cgit.FreeBSD.org/src/commit/?id=46ea2ffc3fbc42089d8322a65fdee8476d2b00d6 >> > >> > commit 46ea2ffc3fbc42089d8322a65fdee8476d2b00d6 >> > Author: Warner Losh <imp@FreeBSD.org> >> > AuthorDate: 2024-08-01 21:24:51 +0000 >> > Commit: Warner Losh <imp@FreeBSD.org> >> > CommitDate: 2024-08-01 21:30:26 +0000 >> > >> > stand: Reduce limit to 500k for x86 loader >> > >> > The largest loader that works for PXE boot is about 500k. PXE needs >> low >> > memory for packets and other driver state, so the largest safe size >> for >> > the loader is about 500k. Reduce the size from 560k to 500k so we >> don't >> > accidentally break PXE in the future. >> > >> > Add a comment for people with special needs. If you control the >> > hardware, it can be safe to have boot loaders as large as 580k or >> 600k >> > in some cases. Since the BIOS loader is becoming more and more of a >> > legacy item, the build variable LOADERSIZE isn't documented. This >> change >> > doesn't change that: there's been little demand for this >> documentation >> > and in general, users shouldn't change it lightly. >> > >> > PR: 257018 >> > Sponsored by: Netflix >> > --- >> > stand/i386/loader/Makefile | 7 ++++++- >> > 1 file changed, 6 insertions(+), 1 deletion(-) >> > >> > diff --git a/stand/i386/loader/Makefile b/stand/i386/loader/Makefile >> > index a4aa3a3c4d45..efd442977780 100644 >> > --- a/stand/i386/loader/Makefile >> > +++ b/stand/i386/loader/Makefile >> > @@ -32,7 +32,12 @@ VERSION_FILE= ${.CURDIR}/../loader/version >> > # >> > # will tell you how many kiB of lomem are available. >> > # >> > -LOADERSIZE?= 560000 # Largest known safe size for loader.bin >> > +# We further reduce this to 500k, though, to give PXE an additional >> 64k of space >> > +# so pxeloader will fit. If you have special needs that do not include >> pxeboot, >> > +# you can safely set this as high as 560000 generally, or a bit higher >> if you >> > +# have tight control over the machines you are booting on. >> > +# >> > +LOADERSIZE?= 500000 # Largest known safe size for loader.bin >> >> Hi Warner, >> >> This breaks the WITH_BEARSSL (which implies WITH_LOADER_VERIEXEC) build. >> When enabled, the loader ends up being just slightly larger than the >> limit. >> > > I sent another reply, but don't see it now. I hope it doesn't go out since > its tone was off. > > So for me, on main, this combination works. I'm able to build everything > with those two options. > One can't build with just WITH_LOADER_VERIEXEC, though, you need both. > Even if it had been > too big, one can fix this with LOADERSIZE=510000. Though doing that > automatically is tricky, > for reasons below. At the very least, I should make a note of it in the > WITH_xxx options. So are > there other options you are using, or maybe a different clang version than > I have (which I think is > tip of main, but might be tip of may from early july). > > However, as the comment states: this is a special needs case. Veriexec > for BIOS is a fringe > activity compared to PXEBOOT using BIOS, so we have to draw some hard > lines in the default > settings. This is the first of the hard lines: The boot loader can't be > larger than 500k, at least until > we can get good data that larger sizes work with pxeboot. So this limit > isn't going to change until > there's new data that comes in, since there's way more pxeboot users. The > build breakage for > pxeboot is silent, and its users can be slow to resport it through the > right channels. So this > is the lessor evil: Strict limits and working pxeboot at the cost (evil) > that some people that add > in non-default options will need to do other special things. > > But what to do. Do we warn when these options are enabled? Do we bump the > size? Do we > disable pxeboot? Do we just do that if it's larger than a certain amount > (or fail so unless you've > disabled it, you'll see it's too big)? > > I'm inclined to (1) document you may need to bump LOADERSIZE with these > options and > (2) fail the pxeboot build if loader_lua is > 500,000 and maybe (3) have > a WITHOUT_LOADER_PXEBOOT > option as there isn't one already. That way, if you bump the size, you may > have to also disable > pxeboot to have a successful build, but it will all be explicit and we'll > document that sometimes > too many options results in size issues that need careful testing after > overriding the safety limits. > > There's also a dark horse in this: There's a bug / pull request (I can't > recall which) that uses LTO > and other weird, but clever, tricks to reduce the size of the loader. > IIRC, it was a big win for loader.efi, > but much smaller win (or maybe even a slight pessimization) for > /boot/loader. I'll look into that to > see if any of the techniques used there can eke us out some space here. > https://reviews.freebsd.org/D46211 has what I had in mind for (1). It's really the only one of the three I'm not sure what to do about. Warner [-- Attachment #2 --] <div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Aug 2, 2024 at 10:13 AM Warner Losh <<a href="mailto:imp@bsdimp.com">imp@bsdimp.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Aug 2, 2024 at 8:29 AM Mark Johnston <<a href="mailto:markj@freebsd.org" target="_blank">markj@freebsd.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Thu, Aug 01, 2024 at 09:31:24PM +0000, Warner Losh wrote:<br> > The branch main has been updated by imp:<br> > <br> > URL: <a href="https://cgit.FreeBSD.org/src/commit/?id=46ea2ffc3fbc42089d8322a65fdee8476d2b00d6" rel="noreferrer" target="_blank">https://cgit.FreeBSD.org/src/commit/?id=46ea2ffc3fbc42089d8322a65fdee8476d2b00d6</a><br>; > <br> > commit 46ea2ffc3fbc42089d8322a65fdee8476d2b00d6<br> > Author: Warner Losh <imp@FreeBSD.org><br> > AuthorDate: 2024-08-01 21:24:51 +0000<br> > Commit: Warner Losh <imp@FreeBSD.org><br> > CommitDate: 2024-08-01 21:30:26 +0000<br> > <br> > stand: Reduce limit to 500k for x86 loader<br> > <br> > The largest loader that works for PXE boot is about 500k. PXE needs low<br> > memory for packets and other driver state, so the largest safe size for<br> > the loader is about 500k. Reduce the size from 560k to 500k so we don't<br> > accidentally break PXE in the future.<br> > <br> > Add a comment for people with special needs. If you control the<br> > hardware, it can be safe to have boot loaders as large as 580k or 600k<br> > in some cases. Since the BIOS loader is becoming more and more of a<br> > legacy item, the build variable LOADERSIZE isn't documented. This change<br> > doesn't change that: there's been little demand for this documentation<br> > and in general, users shouldn't change it lightly.<br> > <br> > PR: 257018<br> > Sponsored by: Netflix<br> > ---<br> > stand/i386/loader/Makefile | 7 ++++++-<br> > 1 file changed, 6 insertions(+), 1 deletion(-)<br> > <br> > diff --git a/stand/i386/loader/Makefile b/stand/i386/loader/Makefile<br> > index a4aa3a3c4d45..efd442977780 100644<br> > --- a/stand/i386/loader/Makefile<br> > +++ b/stand/i386/loader/Makefile<br> > @@ -32,7 +32,12 @@ VERSION_FILE= ${.CURDIR}/../loader/version<br> > #<br> > # will tell you how many kiB of lomem are available.<br> > #<br> > -LOADERSIZE?= 560000 # Largest known safe size for loader.bin<br> > +# We further reduce this to 500k, though, to give PXE an additional 64k of space<br> > +# so pxeloader will fit. If you have special needs that do not include pxeboot,<br> > +# you can safely set this as high as 560000 generally, or a bit higher if you<br> > +# have tight control over the machines you are booting on.<br> > +#<br> > +LOADERSIZE?= 500000 # Largest known safe size for loader.bin<br> <br> Hi Warner,<br> <br> This breaks the WITH_BEARSSL (which implies WITH_LOADER_VERIEXEC) build.<br> When enabled, the loader ends up being just slightly larger than the<br> limit.<br></blockquote><div><br></div><div>I sent another reply, but don't see it now. I hope it doesn't go out since its tone was off.</div><div><br></div><div>So for me, on main, this combination works. I'm able to build everything with those two options.</div><div>One can't build with just WITH_LOADER_VERIEXEC, though, you need both. Even if it had been</div><div>too big, one can fix this with LOADERSIZE=510000. Though doing that automatically is tricky,</div><div>for reasons below. At the very least, I should make a note of it in the WITH_xxx options. So are</div><div>there other options you are using, or maybe a different clang version than I have (which I think is</div><div>tip of main, but might be tip of may from early july).</div><div><br></div><div>However, as the comment states: this is a special needs case. Veriexec for BIOS is a fringe</div><div>activity compared to PXEBOOT using BIOS, so we have to draw some hard lines in the default</div><div>settings. This is the first of the hard lines: The boot loader can't be larger than 500k, at least until</div><div>we can get good data that larger sizes work with pxeboot. So this limit isn't going to change until</div><div>there's new data that comes in, since there's way more pxeboot users. The build breakage for</div><div>pxeboot is silent, and its users can be slow to resport it through the right channels. So this</div><div>is the lessor evil: Strict limits and working pxeboot at the cost (evil) that some people that add</div><div>in non-default options will need to do other special things.</div><div><br></div><div>But what to do. Do we warn when these options are enabled? Do we bump the size? Do we</div><div>disable pxeboot? Do we just do that if it's larger than a certain amount (or fail so unless you've</div><div>disabled it, you'll see it's too big)?</div><div><br></div><div>I'm inclined to (1) document you may need to bump LOADERSIZE with these options and</div><div>(2) fail the pxeboot build if loader_lua is > 500,000 and maybe (3) have a WITHOUT_LOADER_PXEBOOT</div><div>option as there isn't one already. That way, if you bump the size, you may have to also disable</div><div>pxeboot to have a successful build, but it will all be explicit and we'll document that sometimes</div><div>too many options results in size issues that need careful testing after overriding the safety limits.</div><div><br></div><div>There's also a dark horse in this: There's a bug / pull request (I can't recall which) that uses LTO</div><div>and other weird, but clever, tricks to reduce the size of the loader. IIRC, it was a big win for loader.efi,</div><div>but much smaller win (or maybe even a slight pessimization) for /boot/loader. I'll look into that to</div><div>see if any of the techniques used there can eke us out some space here. </div></div></div></blockquote><div><br></div><div><a href="https://reviews.freebsd.org/D46211">https://reviews.freebsd.org/D46211</a>; has what I had in mind for (1). It's really the only one of the three</div><div>I'm not sure what to do about.</div><div><br></div><div>Warner </div></div></div>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfquvUPk8TFYNjU76E4G6_rVRjmm0b48wNPbEHYg6moMbw>