Date: Fri, 12 Aug 2022 11:18:17 -0600 From: Warner Losh <imp@bsdimp.com> To: John Baldwin <jhb@freebsd.org> Cc: Warner Losh <imp@freebsd.org>, src-committers <src-committers@freebsd.org>, "<dev-commits-src-all@freebsd.org>" <dev-commits-src-all@freebsd.org>, dev-commits-src-main@freebsd.org Subject: Re: git: 39fdad34e220 - main - stand: impose 510,000 byte limit for /boot/loader and /boot/pxeldr Message-ID: <CANCZdfpzpjEK42aBR6xJy7xK9bW1mTNs7VSRez2zv_c-w-uPMg@mail.gmail.com> In-Reply-To: <f1313e78-b745-35da-aa32-f9e0ecdfc07c@FreeBSD.org> References: <202208110331.27B3Va7M007335@gitrepo.freebsd.org> <727af1f3-7432-c038-a776-682ef161f6f9@FreeBSD.org> <CANCZdfo-CyHD0jj==14Cimy9g77MqEBjZV1ho=TC7qgoDz6eCg@mail.gmail.com> <ad8467cd-bbc7-37cf-b085-72bf34256431@FreeBSD.org> <CANCZdfrVB7C=6PMcVxmEtDo92pNNZjbkrNSiTyTTJWFpW1n1nw@mail.gmail.com> <f1313e78-b745-35da-aa32-f9e0ecdfc07c@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--0000000000007c60dd05e60e7846 Content-Type: text/plain; charset="UTF-8" On Fri, Aug 12, 2022 at 11:05 AM John Baldwin <jhb@freebsd.org> wrote: > On 8/11/22 6:45 PM, Warner Losh wrote: > > On Thu, Aug 11, 2022 at 6:24 PM John Baldwin <jhb@freebsd.org> wrote: > > > >> On 8/11/22 11:18 AM, Warner Losh wrote: > >>> On Thu, Aug 11, 2022 at 10:56 AM John Baldwin <jhb@freebsd.org> wrote: > >>>> You really want to apply the size check to loader.bin, not loader. > The > >>>> memory > >>>> layout down in the first 1MB for boot loaders is roughly: > >>>> > >>>> 0x0000: real-mode IDT > >>>> 0x0400: BIOS data > >>>> 0x7c00: where BIOS loads boot loaders such as /boot/mbr, etc. > >>>> 0x1000: various BTX global data like GDT, TSS, IDT, stacks > >>>> 0x9000: BTX kernel > >>>> 0xa000: BTX client (loader.bin) > >>>> 0xa0000: top of BTX client stack (though this can be a bit lower for > >> cases > >>>> like > >>>> PXE booting) > >>>> > >>>> The real size constraint is on the BTX client (loader.bin) and the > fact > >>>> that > >>>> it's text/data/bss plus stack need to fit into that 576k window (give > or > >>>> take). > >>>> btxldr isn't stored in low memory, so its size isn't relevant, and > BTX's > >>>> code > >>>> always takes up a full page even though it is much smaller. > >>>> > >>> > >>> Where does 576k come from? That's 589824 bytes, but a0000-a000 is > 614400 > >>> bytes. The delta is 24k (24576). My 'observed' value of about 515,000 > is > >>> another > >>> 75k below that, suggesting we are needing 100k of stack? Can that be > >> right? > >>> I knew > >>> lua ate a lot of stack, but wow! > >> > >> Hmm, I converted 0xa000 to 64k instead of 40k in my head which is where > >> the 576k came from. Yeah, the total size is 600k. 100k stack does seem > >> like a lot. It is possible that other BIOS ROMs besides just PXE might > >> steal data from the stack top. You could maybe try looking at some of > >> your existing BIOS-boot machines to check the 16-bit word at physical > >> address 0x413. That gives you the actual top of the 640k window. On > >> my current desktop (albeit booted via UEFI and not BIOS) it is 629k: > >> > >> % sudo dd if=/dev/mem bs=1 iseek=0x413 count=2 | hd > >> 2+0 records in > >> 2+0 records out > >> 2 bytes transferred in 0.000026 secs (75643 bytes/sec) > >> 00000000 75 02 |u.| > >> 00000002 > >> % gdb > >> ... > >> (gdb) p/d 0x275 > >> $2 = 629 > >> Still, that's only 11k gone. > >> > > > > So on the machine that I'm having issues with... > > > > # dd if=/dev/mem bs=1 iseek=0x413 count=2 | hd > > 00000000 37 02 > > # echo $((0x237)) > > 567 > > > > Super Yikes! > > Dear goodness what in the world. There must be a 64k bounce buffer or > something weird. Maybe for an HBA ROM of some sort? Oof. That would > explain why you are seeing issues that we aren't normally seeing from > other users though. > Yes. It may be because PXE booting is enabled on these machines or something. I'll have to see if playing around with that makes a difference. I'll also search for an HBA ROM that's stealing space as well. Thankfully it's only on 4 really old revisions of our hardware. But it does mean that I'll have to slowly make things optional so that I can keep these machines working until their retirement date... It's a rather substantial number today, but may be 0 this time next year, so there's a limited time horizon... Warner --0000000000007c60dd05e60e7846 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Fri, Aug 12, 2022 at 11:05 AM John= Baldwin <<a href=3D"mailto:jhb@freebsd.org">jhb@freebsd.org</a>> wro= te:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px = 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 8/11/22 6= :45 PM, Warner Losh wrote:<br> > On Thu, Aug 11, 2022 at 6:24 PM John Baldwin <<a href=3D"mailto:jhb= @freebsd.org" target=3D"_blank">jhb@freebsd.org</a>> wrote:<br> > <br> >> On 8/11/22 11:18 AM, Warner Losh wrote:<br> >>> On Thu, Aug 11, 2022 at 10:56 AM John Baldwin <<a href=3D"m= ailto:jhb@freebsd.org" target=3D"_blank">jhb@freebsd.org</a>> wrote:<br> >>>> You really want to apply the size check to loader.bin, not= loader.=C2=A0 The<br> >>>> memory<br> >>>> layout down in the first 1MB for boot loaders is roughly:<= br> >>>><br> >>>> 0x0000: real-mode IDT<br> >>>> 0x0400: BIOS data<br> >>>> 0x7c00: where BIOS loads boot loaders such as /boot/mbr, e= tc.<br> >>>> 0x1000: various BTX global data like GDT, TSS, IDT, stacks= <br> >>>> 0x9000: BTX kernel<br> >>>> 0xa000: BTX client (loader.bin)<br> >>>> 0xa0000: top of BTX client stack (though this can be a bit= lower for<br> >> cases<br> >>>> like<br> >>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0PXE booting= )<br> >>>><br> >>>> The real size constraint is on the BTX client (loader.bin)= and the fact<br> >>>> that<br> >>>> it's text/data/bss plus stack need to fit into that 57= 6k window (give or<br> >>>> take).<br> >>>> btxldr isn't stored in low memory, so its size isn'= ;t relevant, and BTX's<br> >>>> code<br> >>>> always takes up a full page even though it is much smaller= .<br> >>>><br> >>><br> >>> Where does 576k come from? That's 589824 bytes, but a0000-= a000 is 614400<br> >>> bytes. The delta is 24k (24576). My 'observed' value o= f about 515,000 is<br> >>> another<br> >>> 75k below that, suggesting we are needing 100k of stack? Can t= hat be<br> >> right?<br> >>> I knew<br> >>> lua ate a lot of stack, but wow!<br> >><br> >> Hmm, I converted 0xa000 to 64k instead of 40k in my head which is = where<br> >> the 576k came from.=C2=A0 Yeah, the total size is 600k.=C2=A0 100k= stack does seem<br> >> like a lot.=C2=A0 It is possible that other BIOS ROMs besides just= PXE might<br> >> steal data from the stack top.=C2=A0 You could maybe try looking a= t some of<br> >> your existing BIOS-boot machines to check the 16-bit word at physi= cal<br> >> address 0x413.=C2=A0 That gives you the actual top of the 640k win= dow.=C2=A0 On<br> >> my current desktop (albeit booted via UEFI and not BIOS) it is 629= k:<br> >><br> >> % sudo dd if=3D/dev/mem bs=3D1 iseek=3D0x413 count=3D2 | hd<br> >> 2+0 records in<br> >> 2+0 records out<br> >> 2 bytes transferred in 0.000026 secs (75643 bytes/sec)<br> >> 00000000=C2=A0 75 02=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|u.|<br> >> 00000002<br> >> % gdb<br> >> ...<br> >> (gdb) p/d 0x275<br> >> $2 =3D 629<br> >> Still, that's only 11k gone.<br> >><br> > <br> > So on the machine that I'm having issues with...<br> > <br> > # dd if=3D/dev/mem bs=3D1 iseek=3D0x413 count=3D2 | hd<br> > 00000000=C2=A0 37 02<br> > # echo=C2=A0 $((0x237))<br> > 567<br> > <br> > Super Yikes!<br> <br> Dear goodness what in the world.=C2=A0 There must be a 64k bounce buffer or= <br> something weird.=C2=A0 Maybe for an HBA ROM of some sort?=C2=A0 Oof.=C2=A0 = That would<br> explain why you are seeing issues that we aren't normally seeing from<b= r> other users though.<br></blockquote><div><br></div><div>Yes. It may be beca= use PXE booting is enabled on these machines or something. I'll have</d= iv><div>to see if playing around with that makes a difference. I'll als= o search for an HBA ROM that's</div><div>stealing space as well. Thankf= ully it's only on 4 really old revisions of our hardware. But it</div><= div>does mean that I'll have to slowly make things optional so that I c= an keep these machines</div><div>working until their retirement date... It&= #39;s a rather substantial number today, but may be 0</div><div>this time n= ext year, so there's a limited time horizon...</div><div><br></div><div= >Warner<br></div></div></div> --0000000000007c60dd05e60e7846--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfpzpjEK42aBR6xJy7xK9bW1mTNs7VSRez2zv_c-w-uPMg>