Date: Thu, 11 Aug 2022 12:18:48 -0600 From: Warner Losh <imp@bsdimp.com> To: John Baldwin <jhb@freebsd.org> Cc: Warner Losh <imp@freebsd.org>, src-committers <src-committers@freebsd.org>, "<dev-commits-src-all@freebsd.org>" <dev-commits-src-all@freebsd.org>, dev-commits-src-main@freebsd.org Subject: Re: git: 39fdad34e220 - main - stand: impose 510,000 byte limit for /boot/loader and /boot/pxeldr Message-ID: <CANCZdfo-CyHD0jj==14Cimy9g77MqEBjZV1ho=TC7qgoDz6eCg@mail.gmail.com> In-Reply-To: <727af1f3-7432-c038-a776-682ef161f6f9@FreeBSD.org> References: <202208110331.27B3Va7M007335@gitrepo.freebsd.org> <727af1f3-7432-c038-a776-682ef161f6f9@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--00000000000003f4d305e5fb3340 Content-Type: text/plain; charset="UTF-8" On Thu, Aug 11, 2022 at 10:56 AM John Baldwin <jhb@freebsd.org> wrote: > On 8/10/22 8:31 PM, Warner Losh wrote: > > The branch main has been updated by imp: > > > > URL: > https://cgit.FreeBSD.org/src/commit/?id=39fdad34e220c52a433e78f20c8c39412429014e > > > > commit 39fdad34e220c52a433e78f20c8c39412429014e > > Author: Warner Losh <imp@FreeBSD.org> > > AuthorDate: 2022-08-11 03:19:01 +0000 > > Commit: Warner Losh <imp@FreeBSD.org> > > CommitDate: 2022-08-11 03:29:20 +0000 > > > > stand: impose 510,000 byte limit for /boot/loader and /boot/pxeldr > > > > The BIOS method of booting imposes an absolute limit of 640k for the > > size of the program being run due to btx. In practice, this means > that > > programs larger than about 500kiB will fail in odd ways as the > stack / > > heap will overflow. > > Technically the heap is now always above 1MB, the issue is the stack > growing > down and overwriting .bss. > Fair point. I realized that after I pushed as well... Some compilers I've used in the past have 'stack overflow' checks, but that was done at the userland/kernel boundary and likely would be hard to pull off here... > > Pick 510,000 as the cutoff line semi-arbitrarily. loader_lua is now > > almost too big and we want to break the build when it crosses this > > threshold. In my experience, below 500,000 always works, above > 520,000 > > always seems to fail with things getting bad somewhere between > 512,000 > > to 515,000. 510,000 is as close to the line as I think we can go, > though > > experience may dictate we need to lower this in the future. > > > > This is at-best a stop-breakage until we have a better way to > subset the > > boot loader for BIOS booting to allow better, more fined-tuned > > /boot/loaders for the many different environments they have to run > > in. This likely means we'll have a graphical loader than > understands a > > few filesystmes for installation, and a non-graphical loader that > > understands the most filesystems possible for everything else in the > > future. Our build infrastructure needs some work before we can do > that, > > however. > > > > At this late date, it likely isn't worth the efforts to move parts > of > > the loader into high memory. There's a number of assumptions about > where > > the stack is, where buffers reside, etc that are fulfilled when it > lives > > in the first 640k that would need bounce buffers and/or other > counter > > measures if we were to split it up. All BIOS calls are done in > 16-bit > > mode with SEG:OFF addresses, requiring them to be in the first 640k > of > > RAM. And nearly all machines in the last decade can boot with UEFI > > (though there's some exceptions, so it isn't worth killing outright > > yet). > > Fully agree that we just want to keep the BIOS loader on a sufficient > feature > diet. > Yes. > > Sponsored by: Netflix > > Reviewed by: kevans > > Differential Revision: https://reviews.freebsd.org/D36129 > > You really want to apply the size check to loader.bin, not loader. The > memory > layout down in the first 1MB for boot loaders is roughly: > > 0x0000: real-mode IDT > 0x0400: BIOS data > 0x7c00: where BIOS loads boot loaders such as /boot/mbr, etc. > 0x1000: various BTX global data like GDT, TSS, IDT, stacks > 0x9000: BTX kernel > 0xa000: BTX client (loader.bin) > 0xa0000: top of BTX client stack (though this can be a bit lower for cases > like > PXE booting) > > The real size constraint is on the BTX client (loader.bin) and the fact > that > it's text/data/bss plus stack need to fit into that 576k window (give or > take). > btxldr isn't stored in low memory, so its size isn't relevant, and BTX's > code > always takes up a full page even though it is much smaller. > Where does 576k come from? That's 589824 bytes, but a0000-a000 is 614400 bytes. The delta is 24k (24576). My 'observed' value of about 515,000 is another 75k below that, suggesting we are needing 100k of stack? Can that be right? I knew lua ate a lot of stack, but wow! In theory pxeboot's total size needs to fit into the window at 0x7c00 - > 0xa0000, > but in practice that limit is much larger since the size of pxeldr plus > the BTX > kernel is much smaller than 0xa000 - 0x7c00. > > I would say that you might want the PXE size to be even lower (maybe 4k or > so?) > than the "plain" disk loader as PXE ROMs grab some of the memory ending at > 0xa0000 to use for data buffers. I don't have a firm number I can recall > of how > much they grab hence my guess of about 4k or so. > Hmmm... Good point on that. It likely is better to get well below the limit than to necessarily check for a smaller pxeldr. Or put another way, I'd rather we push the limit down all the time so we don't wind up in that awkward place where /boot/loader fits, but /boot/pxeldr doesn't. Warner --00000000000003f4d305e5fb3340 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Thu, Aug 11, 2022 at 10:56 AM John= Baldwin <<a href=3D"mailto:jhb@freebsd.org">jhb@freebsd.org</a>> wro= te:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px = 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 8/10/22 8= :31 PM, Warner Losh wrote:<br> > The branch main has been updated by imp:<br> > <br> > URL: <a href=3D"https://cgit.FreeBSD.org/src/commit/?id=3D39fdad34e220= c52a433e78f20c8c39412429014e" rel=3D"noreferrer" target=3D"_blank">https://= cgit.FreeBSD.org/src/commit/?id=3D39fdad34e220c52a433e78f20c8c39412429014e<= /a><br> > <br> > commit 39fdad34e220c52a433e78f20c8c39412429014e<br> > Author:=C2=A0 =C2=A0 =C2=A0Warner Losh <imp@FreeBSD.org><br> > AuthorDate: 2022-08-11 03:19:01 +0000<br> > Commit:=C2=A0 =C2=A0 =C2=A0Warner Losh <imp@FreeBSD.org><br> > CommitDate: 2022-08-11 03:29:20 +0000<br> > <br> >=C2=A0 =C2=A0 =C2=A0 stand: impose 510,000 byte limit for /boot/loader = and /boot/pxeldr<br> >=C2=A0 =C2=A0 =C2=A0 <br> >=C2=A0 =C2=A0 =C2=A0 The BIOS method of booting imposes an absolute lim= it of 640k for the<br> >=C2=A0 =C2=A0 =C2=A0 size of the program being run due to btx. In pract= ice, this means that<br> >=C2=A0 =C2=A0 =C2=A0 programs larger than about 500kiB will fail in odd= ways as the stack /<br> >=C2=A0 =C2=A0 =C2=A0 heap will overflow.<br> <br> Technically the heap is now always above 1MB, the issue is the stack growin= g<br> down and overwriting .bss.<br></blockquote><div><br></div><div>Fair point. = I realized that after I pushed as well...</div><div><br></div><div>Some com= pilers I've used in the past have 'stack overflow' checks, but = that was done</div><div>at the userland/kernel boundary and likely would be= hard to pull off here...</div><div>=C2=A0</div><blockquote class=3D"gmail_= quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,= 204);padding-left:1ex"> >=C2=A0 =C2=A0 =C2=A0 Pick 510,000 as the cutoff line semi-arbitrarily. = loader_lua is now<br> >=C2=A0 =C2=A0 =C2=A0 almost too big and we want to break the build when= it crosses this<br> >=C2=A0 =C2=A0 =C2=A0 threshold. In my experience, below 500,000 always = works, above 520,000<br> >=C2=A0 =C2=A0 =C2=A0 always seems to fail with things getting bad somew= here between 512,000<br> >=C2=A0 =C2=A0 =C2=A0 to 515,000. 510,000 is as close to the line as I t= hink we can go, though<br> >=C2=A0 =C2=A0 =C2=A0 experience may dictate we need to lower this in th= e future.<br> >=C2=A0 =C2=A0 =C2=A0 <br> >=C2=A0 =C2=A0 =C2=A0 This is at-best a stop-breakage until we have a be= tter way to subset the<br> >=C2=A0 =C2=A0 =C2=A0 boot loader for BIOS booting to allow better, more= fined-tuned<br> >=C2=A0 =C2=A0 =C2=A0 /boot/loaders for the many different environments = they have to run<br> >=C2=A0 =C2=A0 =C2=A0 in. This likely means we'll have a graphical l= oader than understands a<br> >=C2=A0 =C2=A0 =C2=A0 few filesystmes for installation, and a non-graphi= cal loader that<br> >=C2=A0 =C2=A0 =C2=A0 understands the most filesystems possible for ever= ything else in the<br> >=C2=A0 =C2=A0 =C2=A0 future. Our build infrastructure needs some work b= efore we can do that,<br> >=C2=A0 =C2=A0 =C2=A0 however.<br> >=C2=A0 =C2=A0 =C2=A0 <br> >=C2=A0 =C2=A0 =C2=A0 At this late date, it likely isn't worth the e= fforts to move parts of<br> >=C2=A0 =C2=A0 =C2=A0 the loader into high memory. There's a number = of assumptions about where<br> >=C2=A0 =C2=A0 =C2=A0 the stack is, where buffers reside, etc that are f= ulfilled when it lives<br> >=C2=A0 =C2=A0 =C2=A0 in the first 640k that would need bounce buffers a= nd/or other counter<br> >=C2=A0 =C2=A0 =C2=A0 measures if we were to split it up. All BIOS calls= are done in 16-bit<br> >=C2=A0 =C2=A0 =C2=A0 mode with SEG:OFF addresses, requiring them to be = in the first 640k of<br> >=C2=A0 =C2=A0 =C2=A0 RAM. And nearly all machines in the last decade ca= n boot with UEFI<br> >=C2=A0 =C2=A0 =C2=A0 (though there's some exceptions, so it isn'= ;t worth killing outright<br> >=C2=A0 =C2=A0 =C2=A0 yet).<br> <br> Fully agree that we just want to keep the BIOS loader on a sufficient featu= re<br> diet.<br></blockquote><div><br></div><div>Yes.</div><div>=C2=A0</div><block= quote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1= px solid rgb(204,204,204);padding-left:1ex"> >=C2=A0 =C2=A0 =C2=A0 Sponsored by:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0Netflix<br> >=C2=A0 =C2=A0 =C2=A0 Reviewed by:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 kevans<br> >=C2=A0 =C2=A0 =C2=A0 Differential Revision:=C2=A0 <a href=3D"https://re= views.freebsd.org/D36129" rel=3D"noreferrer" target=3D"_blank">https://revi= ews.freebsd.org/D36129</a><br> <br> You really want to apply the size check to loader.bin, not loader.=C2=A0 Th= e memory<br> layout down in the first 1MB for boot loaders is roughly:<br> <br> 0x0000: real-mode IDT<br> 0x0400: BIOS data<br> 0x7c00: where BIOS loads boot loaders such as /boot/mbr, etc.<br> 0x1000: various BTX global data like GDT, TSS, IDT, stacks<br> 0x9000: BTX kernel<br> 0xa000: BTX client (loader.bin)<br> 0xa0000: top of BTX client stack (though this can be a bit lower for cases = like<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 PXE booting)<br> <br> The real size constraint is on the BTX client (loader.bin) and the fact tha= t<br> it's text/data/bss plus stack need to fit into that 576k window (give o= r take).<br> btxldr isn't stored in low memory, so its size isn't relevant, and = BTX's code<br> always takes up a full page even though it is much smaller.<br></blockquote= ><div><br></div><div>Where does 576k come from? That's 589824 bytes, bu= t a0000-a000 is=C2=A0614400</div><div>bytes. The delta is 24k (24576). My &= #39;observed' value of about 515,000 is another</div><div>75k below tha= t, suggesting we are needing 100k of stack? Can that be right? I knew</div>= <div>lua ate a lot of stack, but wow!</div><div><br></div><blockquote class= =3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rg= b(204,204,204);padding-left:1ex"> In theory pxeboot's total size needs to fit into the window at 0x7c00 -= 0xa0000,<br> but in practice that limit is much larger since the size of pxeldr plus the= BTX<br> kernel is much smaller than 0xa000 - 0x7c00.<br> <br> I would say that you might want the PXE size to be even lower (maybe 4k or = so?)<br> than the "plain" disk loader as PXE ROMs grab some of the memory = ending at<br> 0xa0000 to use for data buffers.=C2=A0 I don't have a firm number I can= recall of how<br>much they grab hence my guess of about 4k or so.<br></blo= ckquote><div><br></div><div>Hmmm... Good point on that. It likely is better= to get well below the limit than to necessarily</div><div>check for a smal= ler pxeldr. Or put another way, I'd rather we push the limit down all</= div><div>the time so we don't wind up in that awkward place where /boot= /loader fits, but /boot/pxeldr</div><div>doesn't.=C2=A0</div><div><br><= /div><div>Warner</div></div></div> --00000000000003f4d305e5fb3340--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfo-CyHD0jj==14Cimy9g77MqEBjZV1ho=TC7qgoDz6eCg>