From nobody Fri Aug 12 17:18:17 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4M49PL5bzCz4XwQM for ; Fri, 12 Aug 2022 17:18:30 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ua1-x936.google.com (mail-ua1-x936.google.com [IPv6:2607:f8b0:4864:20::936]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4M49PK609Cz3KRp for ; Fri, 12 Aug 2022 17:18:29 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-ua1-x936.google.com with SMTP id f10so551620uap.2 for ; Fri, 12 Aug 2022 10:18:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=3eMXh/083l2BNn292aACxz1u6Q2a1xSc63ePw4AK2Po=; b=gBkDUto2fHSRcryH+fZgwC0uJE6mxMKgKcWbVD0Np17+koblApdQjLWLNdn6Z+wh05 heU5TBvSW9GYkBQkQ1H/Qkp6Xp5fdwg5K4tLzGHt2noee4vrukYH5N8xF/4dlXPuc+EQ wHx5gwltz7rEizwXFFp2n8ZQ0DJZr7XsoHXyrmsR4j43x4oLSvwHDJqWQPxO2MsBoCa2 1IHHL1x+DNTR6oKVTp6wsvDDh6tcmRTIuZ/0wzbLhtgEUndIHHCup2qbMWDjUGpGHvgi GR0z9/48UpR3+2n04R0yA36T0muRkVCwREAKKTCzoELD9vqveKrjYMk2TmuLrEWNjJGI pSIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=3eMXh/083l2BNn292aACxz1u6Q2a1xSc63ePw4AK2Po=; b=RSiNAYAP/t+EmxugLW7kuPrfa34TRqqRClNVSVl7KHK+KTXNqxWgkPWZM3Pgy8uY8C A7b37eYJyK2lIq7DDjExtPZX0l57OrAx7+2K2Ur7CvV2lkCvnLBBM2a17DCh2XMwhiGU J3FcGhBZk3dfNxnMiVJb4p03ADliMTqUTDTAQ+05UYXVBVDYb3xqYbFS4S8j9J9AH4yW jDwFfwvcBlNoNBqAf56YhP7k5RH5atYUoKh/Z9he24hzZOMBIUDuCFaMeqPR4jaDdYr/ B/d9FKFZdU6SCuAPq0uxhzn/ZJJ4n0a39tWjFx5u4U9xbdQdYgGGTJ3vd2DUls4ZGGWX F9Bw== X-Gm-Message-State: ACgBeo2R9bR5ogPzNb/VyuOggxWPLOGrIGGcNUexyxHhXsN/AUh0h71Q kS35HcI2Pp3RTMioFKizc52C3wetB3gOYRbjL8hp3Q== X-Google-Smtp-Source: AA6agR69vjR8B8N7SCEQ9a/6NuN2/i3iXUh+5CZDurm/FTbTXaHTpVQ5f58zMTQ2xp1fDkdlFbYyM7fEgKtOO65uNek= X-Received: by 2002:ab0:15ed:0:b0:365:f250:7384 with SMTP id j42-20020ab015ed000000b00365f2507384mr2339993uae.44.1660324708894; Fri, 12 Aug 2022 10:18:28 -0700 (PDT) List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 References: <202208110331.27B3Va7M007335@gitrepo.freebsd.org> <727af1f3-7432-c038-a776-682ef161f6f9@FreeBSD.org> In-Reply-To: From: Warner Losh Date: Fri, 12 Aug 2022 11:18:17 -0600 Message-ID: Subject: Re: git: 39fdad34e220 - main - stand: impose 510,000 byte limit for /boot/loader and /boot/pxeldr To: John Baldwin Cc: Warner Losh , src-committers , "" , dev-commits-src-main@freebsd.org Content-Type: multipart/alternative; boundary="0000000000007c60dd05e60e7846" X-Rspamd-Queue-Id: 4M49PK609Cz3KRp X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20210112.gappssmtp.com header.s=20210112 header.b=gBkDUto2; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::936) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-3.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-0.999]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20210112.gappssmtp.com:s=20210112]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::936:from]; MLMMJ_DEST(0.00)[dev-commits-src-all@freebsd.org]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_SPF_NA(0.00)[no SPF record]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TO_DN_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20210112.gappssmtp.com:+]; PREVIOUSLY_DELIVERED(0.00)[dev-commits-src-all@freebsd.org]; RCPT_COUNT_FIVE(0.00)[5]; DMARC_NA(0.00)[bsdimp.com]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com] X-ThisMailContainsUnwantedMimeParts: N --0000000000007c60dd05e60e7846 Content-Type: text/plain; charset="UTF-8" On Fri, Aug 12, 2022 at 11:05 AM John Baldwin wrote: > On 8/11/22 6:45 PM, Warner Losh wrote: > > On Thu, Aug 11, 2022 at 6:24 PM John Baldwin wrote: > > > >> On 8/11/22 11:18 AM, Warner Losh wrote: > >>> On Thu, Aug 11, 2022 at 10:56 AM John Baldwin wrote: > >>>> You really want to apply the size check to loader.bin, not loader. > The > >>>> memory > >>>> layout down in the first 1MB for boot loaders is roughly: > >>>> > >>>> 0x0000: real-mode IDT > >>>> 0x0400: BIOS data > >>>> 0x7c00: where BIOS loads boot loaders such as /boot/mbr, etc. > >>>> 0x1000: various BTX global data like GDT, TSS, IDT, stacks > >>>> 0x9000: BTX kernel > >>>> 0xa000: BTX client (loader.bin) > >>>> 0xa0000: top of BTX client stack (though this can be a bit lower for > >> cases > >>>> like > >>>> PXE booting) > >>>> > >>>> The real size constraint is on the BTX client (loader.bin) and the > fact > >>>> that > >>>> it's text/data/bss plus stack need to fit into that 576k window (give > or > >>>> take). > >>>> btxldr isn't stored in low memory, so its size isn't relevant, and > BTX's > >>>> code > >>>> always takes up a full page even though it is much smaller. > >>>> > >>> > >>> Where does 576k come from? That's 589824 bytes, but a0000-a000 is > 614400 > >>> bytes. The delta is 24k (24576). My 'observed' value of about 515,000 > is > >>> another > >>> 75k below that, suggesting we are needing 100k of stack? Can that be > >> right? > >>> I knew > >>> lua ate a lot of stack, but wow! > >> > >> Hmm, I converted 0xa000 to 64k instead of 40k in my head which is where > >> the 576k came from. Yeah, the total size is 600k. 100k stack does seem > >> like a lot. It is possible that other BIOS ROMs besides just PXE might > >> steal data from the stack top. You could maybe try looking at some of > >> your existing BIOS-boot machines to check the 16-bit word at physical > >> address 0x413. That gives you the actual top of the 640k window. On > >> my current desktop (albeit booted via UEFI and not BIOS) it is 629k: > >> > >> % sudo dd if=/dev/mem bs=1 iseek=0x413 count=2 | hd > >> 2+0 records in > >> 2+0 records out > >> 2 bytes transferred in 0.000026 secs (75643 bytes/sec) > >> 00000000 75 02 |u.| > >> 00000002 > >> % gdb > >> ... > >> (gdb) p/d 0x275 > >> $2 = 629 > >> Still, that's only 11k gone. > >> > > > > So on the machine that I'm having issues with... > > > > # dd if=/dev/mem bs=1 iseek=0x413 count=2 | hd > > 00000000 37 02 > > # echo $((0x237)) > > 567 > > > > Super Yikes! > > Dear goodness what in the world. There must be a 64k bounce buffer or > something weird. Maybe for an HBA ROM of some sort? Oof. That would > explain why you are seeing issues that we aren't normally seeing from > other users though. > Yes. It may be because PXE booting is enabled on these machines or something. I'll have to see if playing around with that makes a difference. I'll also search for an HBA ROM that's stealing space as well. Thankfully it's only on 4 really old revisions of our hardware. But it does mean that I'll have to slowly make things optional so that I can keep these machines working until their retirement date... It's a rather substantial number today, but may be 0 this time next year, so there's a limited time horizon... Warner --0000000000007c60dd05e60e7846 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Fri, Aug 12, 2022 at 11:05 AM John= Baldwin <jhb@freebsd.org> wro= te:
On 8/11/22 6= :45 PM, Warner Losh wrote:
> On Thu, Aug 11, 2022 at 6:24 PM John Baldwin <jhb@freebsd.org> wrote:
>
>> On 8/11/22 11:18 AM, Warner Losh wrote:
>>> On Thu, Aug 11, 2022 at 10:56 AM John Baldwin <jhb@freebsd.org> wrote:
>>>> You really want to apply the size check to loader.bin, not= loader.=C2=A0 The
>>>> memory
>>>> layout down in the first 1MB for boot loaders is roughly:<= br> >>>>
>>>> 0x0000: real-mode IDT
>>>> 0x0400: BIOS data
>>>> 0x7c00: where BIOS loads boot loaders such as /boot/mbr, e= tc.
>>>> 0x1000: various BTX global data like GDT, TSS, IDT, stacks=
>>>> 0x9000: BTX kernel
>>>> 0xa000: BTX client (loader.bin)
>>>> 0xa0000: top of BTX client stack (though this can be a bit= lower for
>> cases
>>>> like
>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0PXE booting= )
>>>>
>>>> The real size constraint is on the BTX client (loader.bin)= and the fact
>>>> that
>>>> it's text/data/bss plus stack need to fit into that 57= 6k window (give or
>>>> take).
>>>> btxldr isn't stored in low memory, so its size isn'= ;t relevant, and BTX's
>>>> code
>>>> always takes up a full page even though it is much smaller= .
>>>>
>>>
>>> Where does 576k come from? That's 589824 bytes, but a0000-= a000 is 614400
>>> bytes. The delta is 24k (24576). My 'observed' value o= f about 515,000 is
>>> another
>>> 75k below that, suggesting we are needing 100k of stack? Can t= hat be
>> right?
>>> I knew
>>> lua ate a lot of stack, but wow!
>>
>> Hmm, I converted 0xa000 to 64k instead of 40k in my head which is = where
>> the 576k came from.=C2=A0 Yeah, the total size is 600k.=C2=A0 100k= stack does seem
>> like a lot.=C2=A0 It is possible that other BIOS ROMs besides just= PXE might
>> steal data from the stack top.=C2=A0 You could maybe try looking a= t some of
>> your existing BIOS-boot machines to check the 16-bit word at physi= cal
>> address 0x413.=C2=A0 That gives you the actual top of the 640k win= dow.=C2=A0 On
>> my current desktop (albeit booted via UEFI and not BIOS) it is 629= k:
>>
>> % sudo dd if=3D/dev/mem bs=3D1 iseek=3D0x413 count=3D2 | hd
>> 2+0 records in
>> 2+0 records out
>> 2 bytes transferred in 0.000026 secs (75643 bytes/sec)
>> 00000000=C2=A0 75 02=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|u.|
>> 00000002
>> % gdb
>> ...
>> (gdb) p/d 0x275
>> $2 =3D 629
>> Still, that's only 11k gone.
>>
>
> So on the machine that I'm having issues with...
>
> # dd if=3D/dev/mem bs=3D1 iseek=3D0x413 count=3D2 | hd
> 00000000=C2=A0 37 02
> # echo=C2=A0 $((0x237))
> 567
>
> Super Yikes!

Dear goodness what in the world.=C2=A0 There must be a 64k bounce buffer or=
something weird.=C2=A0 Maybe for an HBA ROM of some sort?=C2=A0 Oof.=C2=A0 = That would
explain why you are seeing issues that we aren't normally seeing from other users though.

Yes. It may be beca= use PXE booting is enabled on these machines or something. I'll have
to see if playing around with that makes a difference. I'll als= o search for an HBA ROM that's
stealing space as well. Thankf= ully it's only on 4 really old revisions of our hardware. But it
<= div>does mean that I'll have to slowly make things optional so that I c= an keep these machines
working until their retirement date... It&= #39;s a rather substantial number today, but may be 0
this time n= ext year, so there's a limited time horizon...

Warner
--0000000000007c60dd05e60e7846--