Date: Tue, 6 Oct 2020 14:26:24 -0700 From: Mark Millard <marklmi@yahoo.com> To: Robert Crowston <crowston@protonmail.com>, freebsd-arm <freebsd-arm@freebsd.org> Subject: Re: rpi4 FreeBSD vs. ubuntu u-boot fdt print / memereserve difference (lack of reserve in FreeBSD context), 0x3b400000 vs. DMA_HIGH_LIMIT being 0x3c000000 Message-ID: <14D4D816-3CF3-4911-91FB-2AA50A4D1C22@yahoo.com> In-Reply-To: <4D5162C2-592D-439C-9660-33A94DA5C807@yahoo.com> References: <1A13F7B5-F8C3-4022-939C-2992E53D1DF1@yahoo.com> <BDA2D50C-E8C8-4BF2-8C97-C1B2516F06F3@yahoo.com> <01C14FE4-C2A6-4459-A5EE-AE33045FAF42@yahoo.com> <4D5162C2-592D-439C-9660-33A94DA5C807@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2020-Oct-4, at 01:41, Mark Millard <marklmi at yahoo.com> wrote:
>=20
> On 2020-Oct-3, at 20:15, Mark Millard <marklmi at yahoo.com> wrote:
>=20
>> . . .
>=20
> Looking at the FreeBSD boot-time memory map
> on the 8 GiByte RPi4B via a u-boot based boot
> shows that the address ranges
> 0x3b400000..0x3e512fff and
> 0x3ebec000..0x3fffffff are not referenced
> below (I inserted some notes):
>=20
> Type Physical Virtual #Pages Attr
> Reserved 000000000000 0 00000002 WB=20
> ConventionalMemory 000000002000 2000 00007eef WB=20
> ACPIReclaimMemory 000007ef1000 7ef1000 0000001e WB=20
> ConventionalMemory 000007f0f000 7f0f000 00029f71 WB=20
> BootServicesData 000031e80000 31e80000 00000001 WB=20
> LoaderData 000031e81000 31e81000 00008001 WB=20
> LoaderCode 000039e82000 39e82000 000000ad WB=20
> Reserved 000039f2f000 39f2f000 00000007 WB=20
> BootServicesData 000039f36000 39f36000 00000001 WB=20
> RuntimeServicesData 000039f37000 39f37000 00000001 WB RUNTIME
> BootServicesData 000039f38000 39f38000 00000002 WB=20
> Reserved 000039f3a000 39f3a000 00000001 WB=20
> BootServicesData 000039f3b000 39f3b000 00000002 WB=20
> RuntimeServicesData 000039f3d000 39f3d000 00000002 WB RUNTIME
> Reserved 000039f3f000 39f3f000 00000001 WB=20
> BootServicesData 000039f40000 39f40000 00000001 WB=20
> Reserved 000039f41000 39f41000 00000001 WB=20
> BootServicesData 000039f42000 39f42000 00000002 WB=20
> Reserved 000039f44000 39f44000 00000001 WB=20
> LoaderData 000039f45000 39f45000 0000140b WB=20
> RuntimeServicesCode 00003b350000 3b350000 00000010 WB RUNTIME
> LoaderData 00003b360000 3b360000 000000a0 WB=20
> NOTE: Here is where 0x3b400000..0x3e512fff is not mentioned at all =
(even later)
> NOTE: Here is where 0x3ebec000..0x3fffffff is not mentioned at all =
(even later)
> BootServicesData 000040000000 40000000 000bc000 WB=20
> MemoryMappedIO 0000fe100000 fe100000 00000001 RUNTIME
> BootServicesData 000100000000 100000000 00100000 WB=20
> Physical memory chunk(s):
> 0x00002000 - 0x07ef0fff, 126 MB ( 32495 pages)
> 0x07f0f000 - 0x39f2efff, 800 MB ( 204832 pages)
> 0x39f36000 - 0x39f39fff, 0 MB ( 4 pages)
> 0x39f3b000 - 0x39f3efff, 0 MB ( 4 pages)
> 0x39f40000 - 0x39f40fff, 0 MB ( 1 pages)
> 0x39f42000 - 0x39f43fff, 0 MB ( 2 pages)
> 0x39f45000 - 0x3b34ffff, 20 MB ( 5131 pages)
> 0x3b360000 - 0x3b3fffff, 0 MB ( 160 pages)
> NOTE: Here is where 0x3b400000..0x3e512fff is not mentioned at all =
(even later)
> NOTE: Here is where 0x3ebec000..0x3fffffff is not mentioned at all =
(even later)
> 0x40000000 - 0xfbffffff, 3008 MB ( 770048 pages)
> 0x100000000 - 0x1ffffffff, 4096 MB (1048576 pages)
> Excluded memory regions:
> 0x00000000 - 0x00001fff, 0 MB ( 2 pages) NoAlloc=20
> 0x07ef1000 - 0x07f0efff, 0 MB ( 30 pages) NoAlloc=20
> 0x32000000 - 0x33451fff, 20 MB ( 5202 pages) NoAlloc=20
> 0x39f2f000 - 0x39f35fff, 0 MB ( 7 pages) NoAlloc=20
> 0x39f37000 - 0x39f37fff, 0 MB ( 1 pages) NoAlloc=20
> 0x39f3a000 - 0x39f3afff, 0 MB ( 1 pages) NoAlloc=20
> 0x39f3d000 - 0x39f3ffff, 0 MB ( 3 pages) NoAlloc=20
> 0x39f41000 - 0x39f41fff, 0 MB ( 1 pages) NoAlloc=20
> 0x39f44000 - 0x39f44fff, 0 MB ( 1 pages) NoAlloc=20
> 0x3b350000 - 0x3b35ffff, 0 MB ( 16 pages) NoAlloc=20
> NOTE: Here is where 0x3b400000..0x3e512fff is not mentioned at all =
(even above)
> 0x3e513000 - 0x3ebebfff, 6 MB ( 1753 pages) NoAlloc=20
Looks like the above "NoAlloc" line is from:
/* Exclude the EFI framebuffer from our view of physical memory. =
*/
efifb =3D (struct efi_fb *)preload_search_info(kmdp,
MODINFO_METADATA | MODINFOMD_EFI_FB);
if (efifb !=3D NULL)
physmem_exclude_region(efifb->fb_addr, efifb->fb_size,
EXFLAG_NOALLOC);
However, for the RPi4B, it looks like the EFI FrameBuffer is in a
subrange of the more general "gpu_mem" area that can be sized by
doing gpu_mem=3D or gpu_mem_1024=3D via config.txt . The size varies
by changing the smaller memory address and runs to the end of the=20
1 GiByte area on RPi4B's.
At least this gives a reason for the one NoAlloc range that did
show up that is in the gpu_mem=3D/gpu_mem_1024=3D area (but without a
containing "Physical memory chunk").
In the end, I do not expect that this contributes to the issue
at hand: phys_avail and the like seem to omit the whole gpu_mem=3D
area and so it looks like the gpu_mem=3D memory is avoided for
allocation.
> NOTE: Here is where 0x3ebec000..0x3fffffff is not mentioned at all =
(even above)
> 0xfe100000 - 0xfe100fff, 0 MB ( 1 pages) NoAlloc=20
>=20
> Physical memory chunk(s):
> 0x00000000002000 - 0x00000007ef0fff, 133099520 bytes (32495 pages)
> 0x00000007f0f000 - 0x00000031ffffff, 705630208 bytes (172273 pages)
> 0x00000033452000 - 0x00000039f2efff, 112054272 bytes (27357 pages)
> 0x00000039f36000 - 0x00000039f36fff, 4096 bytes (1 pages)
> 0x00000039f38000 - 0x00000039f39fff, 8192 bytes (2 pages)
> 0x00000039f3b000 - 0x00000039f3cfff, 8192 bytes (2 pages)
> 0x00000039f40000 - 0x00000039f40fff, 4096 bytes (1 pages)
> 0x00000039f42000 - 0x00000039f43fff, 8192 bytes (2 pages)
> 0x00000039f45000 - 0x0000003b34ffff, 21016576 bytes (5131 pages)
> 0x0000003b360000 - 0x0000003b3fffff, 655360 bytes (160 pages)
> NOTE: Here is where 0x3b400000..0x3e512fff is not mentioned at all =
(even above)
> NOTE: Here is where 0x3ebec000..0x3fffffff is not mentioned at all =
(even above)
> 0x00000040000000 - 0x000000fbffffff, 3154116608 bytes (770048 pages)
> 0x00000100000000 - 0x000001f3840fff, 4085518336 bytes (997441 pages)
>=20
> So:
>=20
> #define DMA_HIGH_LIMIT 0x3c000000
>=20
> leads to the range for potential DMA use seeming to
> overlap 0x3b400000..0x3bffffff (which is not mentioned
> above).
>=20
> Should the missing ranges have a classification above?
> Is the lack of mention supposed to prevent the range
> of memory's use for DMA activity?
>=20
> I'll note that there are a bunch of explicitly "Excluded
> memory regions" that overlap with the range for potential
> DMA use as well.
>=20
> Note: Use of gpu_mem_1024=3D256 in config.txt instead
> produces (notes added again):
>=20
> Type Physical Virtual #Pages Attr
> Reserved 000000000000 0 00000002 WB=20
> ConventionalMemory 000000002000 2000 00007eef WB=20
> ACPIReclaimMemory 000007ef1000 7ef1000 0000001e WB=20
> ConventionalMemory 000007f0f000 7f0f000 0001eb71 WB=20
> BootServicesData 000026a80000 26a80000 00000001 WB=20
> LoaderData 000026a81000 26a81000 00008001 WB=20
> LoaderCode 00002ea82000 2ea82000 000000ad WB=20
> Reserved 00002eb2f000 2eb2f000 00000007 WB=20
> BootServicesData 00002eb36000 2eb36000 00000001 WB=20
> RuntimeServicesData 00002eb37000 2eb37000 00000001 WB RUNTIME
> BootServicesData 00002eb38000 2eb38000 00000002 WB=20
> Reserved 00002eb3a000 2eb3a000 00000001 WB=20
> BootServicesData 00002eb3b000 2eb3b000 00000002 WB=20
> RuntimeServicesData 00002eb3d000 2eb3d000 00000002 WB RUNTIME
> Reserved 00002eb3f000 2eb3f000 00000001 WB=20
> BootServicesData 00002eb40000 2eb40000 00000001 WB=20
> Reserved 00002eb41000 2eb41000 00000001 WB=20
> BootServicesData 00002eb42000 2eb42000 00000002 WB=20
> Reserved 00002eb44000 2eb44000 00000001 WB=20
> LoaderData 00002eb45000 2eb45000 0000140b WB=20
> RuntimeServicesCode 00002ff50000 2ff50000 00000010 WB RUNTIME
> LoaderData 00002ff60000 2ff60000 000000a0 WB=20
> NOTE: Here is where 0x30000000..0x3e512fff is not mentioned at all =
(even below)
> NOTE: Here is where 0x3ebec000..0x3fffffff is not mentioned at all =
(even below)
> BootServicesData 000040000000 40000000 000bc000 WB=20
> MemoryMappedIO 0000fe100000 fe100000 00000001 RUNTIME
> BootServicesData 000100000000 100000000 00100000 WB=20
> Physical memory chunk(s):
> 0x00002000 - 0x07ef0fff, 126 MB ( 32495 pages)
> 0x07f0f000 - 0x2eb2efff, 620 MB ( 158752 pages)
> 0x2eb36000 - 0x2eb39fff, 0 MB ( 4 pages)
> 0x2eb3b000 - 0x2eb3efff, 0 MB ( 4 pages)
> 0x2eb40000 - 0x2eb40fff, 0 MB ( 1 pages)
> 0x2eb42000 - 0x2eb43fff, 0 MB ( 2 pages)
> 0x2eb45000 - 0x2ff4ffff, 20 MB ( 5131 pages)
> 0x2ff60000 - 0x2fffffff, 0 MB ( 160 pages)
> NOTE: Here is where 0x30000000..0x3e512fff is not mentioned at all =
(even below)
> NOTE: Here is where 0x3ebec000..0x3fffffff is not mentioned at all =
(even below)
> 0x40000000 - 0xfbffffff, 3008 MB ( 770048 pages)
> 0x100000000 - 0x1ffffffff, 4096 MB (1048576 pages)
> Excluded memory regions:
> 0x00000000 - 0x00001fff, 0 MB ( 2 pages) NoAlloc=20
> 0x07ef1000 - 0x07f0efff, 0 MB ( 30 pages) NoAlloc=20
> 0x26c00000 - 0x28051fff, 20 MB ( 5202 pages) NoAlloc=20
> 0x2eb2f000 - 0x2eb35fff, 0 MB ( 7 pages) NoAlloc=20
> 0x2eb37000 - 0x2eb37fff, 0 MB ( 1 pages) NoAlloc=20
> 0x2eb3a000 - 0x2eb3afff, 0 MB ( 1 pages) NoAlloc=20
> 0x2eb3d000 - 0x2eb3ffff, 0 MB ( 3 pages) NoAlloc=20
> 0x2eb41000 - 0x2eb41fff, 0 MB ( 1 pages) NoAlloc=20
> 0x2eb44000 - 0x2eb44fff, 0 MB ( 1 pages) NoAlloc=20
> 0x2ff50000 - 0x2ff5ffff, 0 MB ( 16 pages) NoAlloc=20
(Line that was here moved down to track address-range ordering.)
> NOTE: Here is where 0x30000000..0x3e512fff is not mentioned at all =
(even above)
0x3e513000 - 0x3ebebfff, 6 MB ( 1753 pages) NoAlloc=20
(Again the EFI framebuffer.)
> NOTE: Here is where 0x3ebec000..0x3fffffff is not mentioned at all =
(even above)
> 0xfe100000 - 0xfe100fff, 0 MB ( 1 pages) NoAlloc=20
>=20
> Physical memory chunk(s):
> 0x00000000002000 - 0x00000007ef0fff, 133099520 bytes (32495 pages)
> 0x00000007f0f000 - 0x00000026bfffff, 516886528 bytes (126193 pages)
> 0x00000028052000 - 0x0000002eb2efff, 112054272 bytes (27357 pages)
> 0x0000002eb36000 - 0x0000002eb36fff, 4096 bytes (1 pages)
> 0x0000002eb38000 - 0x0000002eb39fff, 8192 bytes (2 pages)
> 0x0000002eb3b000 - 0x0000002eb3cfff, 8192 bytes (2 pages)
> 0x0000002eb40000 - 0x0000002eb40fff, 4096 bytes (1 pages)
> 0x0000002eb42000 - 0x0000002eb43fff, 8192 bytes (2 pages)
> 0x0000002eb45000 - 0x0000002ff4ffff, 21016576 bytes (5131 pages)
> 0x0000002ff60000 - 0x0000002fffffff, 655360 bytes (160 pages)
> NOTE: Here is where 0x30000000..0x3e512fff is not mentioned at all =
(even above)
> NOTE: Here is where 0x3ebec000..0x3fffffff is not mentioned at all =
(even above)
> 0x00000040000000 - 0x000000fbffffff, 3154116608 bytes (770048 pages)
> 0x00000100000000 - 0x000001f3cb5fff, 4090191872 bytes (998582 pages)
>=20
>=20
> (Note: gpu_mem_1024=3D16 leads to the rpi firmware
> messing up and being unable to read the microsd card
> for later rpi firmware files that it needs. It
> actually ended up doing a USB3 SSD based boot, in
> my context a uefi/ACPI boot.)
>=20
>=20
> gpu_mem_1024=3D32 produced (no notes this time):
>=20
> Type Physical Virtual #Pages Attr
> Reserved 000000000000 0 00000002 WB=20
> ConventionalMemory 000000002000 2000 00007eef WB=20
> ACPIReclaimMemory 000007ef1000 7ef1000 0000001e WB=20
> ConventionalMemory 000007f0f000 7f0f000 0002cb71 WB=20
> BootServicesData 000034a80000 34a80000 00000001 WB=20
> LoaderData 000034a81000 34a81000 00008001 WB=20
> LoaderCode 00003ca82000 3ca82000 000000ad WB=20
> Reserved 00003cb2f000 3cb2f000 00000007 WB=20
> BootServicesData 00003cb36000 3cb36000 00000001 WB=20
> RuntimeServicesData 00003cb37000 3cb37000 00000001 WB RUNTIME
> BootServicesData 00003cb38000 3cb38000 00000002 WB=20
> Reserved 00003cb3a000 3cb3a000 00000001 WB=20
> BootServicesData 00003cb3b000 3cb3b000 00000002 WB=20
> RuntimeServicesData 00003cb3d000 3cb3d000 00000002 WB RUNTIME
> Reserved 00003cb3f000 3cb3f000 00000001 WB=20
> BootServicesData 00003cb40000 3cb40000 00000001 WB=20
> Reserved 00003cb41000 3cb41000 00000001 WB=20
> BootServicesData 00003cb42000 3cb42000 00000002 WB=20
> Reserved 00003cb44000 3cb44000 00000001 WB=20
> LoaderData 00003cb45000 3cb45000 0000140b WB=20
> RuntimeServicesCode 00003df50000 3df50000 00000010 WB RUNTIME
> LoaderData 00003df60000 3df60000 000000a0 WB=20
> BootServicesData 000040000000 40000000 000bc000 WB=20
> MemoryMappedIO 0000fe100000 fe100000 00000001 RUNTIME
> BootServicesData 000100000000 100000000 00100000 WB=20
> Physical memory chunk(s):
> 0x00002000 - 0x07ef0fff, 126 MB ( 32495 pages)
> 0x07f0f000 - 0x3cb2efff, 844 MB ( 216096 pages)
> 0x3cb36000 - 0x3cb39fff, 0 MB ( 4 pages)
> 0x3cb3b000 - 0x3cb3efff, 0 MB ( 4 pages)
> 0x3cb40000 - 0x3cb40fff, 0 MB ( 1 pages)
> 0x3cb42000 - 0x3cb43fff, 0 MB ( 2 pages)
> 0x3cb45000 - 0x3df4ffff, 20 MB ( 5131 pages)
> 0x3df60000 - 0x3dffffff, 0 MB ( 160 pages)
> 0x40000000 - 0xfbffffff, 3008 MB ( 770048 pages)
> 0x100000000 - 0x1ffffffff, 4096 MB (1048576 pages)
> Excluded memory regions:
> 0x00000000 - 0x00001fff, 0 MB ( 2 pages) NoAlloc=20
> 0x07ef1000 - 0x07f0efff, 0 MB ( 30 pages) NoAlloc=20
> 0x34c00000 - 0x36051fff, 20 MB ( 5202 pages) NoAlloc=20
> 0x3cb2f000 - 0x3cb35fff, 0 MB ( 7 pages) NoAlloc=20
> 0x3cb37000 - 0x3cb37fff, 0 MB ( 1 pages) NoAlloc=20
> 0x3cb3a000 - 0x3cb3afff, 0 MB ( 1 pages) NoAlloc=20
> 0x3cb3d000 - 0x3cb3ffff, 0 MB ( 3 pages) NoAlloc=20
> 0x3cb41000 - 0x3cb41fff, 0 MB ( 1 pages) NoAlloc=20
> 0x3cb44000 - 0x3cb44fff, 0 MB ( 1 pages) NoAlloc=20
> 0x3df50000 - 0x3df5ffff, 0 MB ( 16 pages) NoAlloc=20
> 0x3e513000 - 0x3ebebfff, 6 MB ( 1753 pages) NoAlloc=20
(Above: again the EFI framebuffer inside the bigger
gpu_mem area.)
> 0xfe100000 - 0xfe100fff, 0 MB ( 1 pages) NoAlloc=20
>=20
> Physical memory chunk(s):
> 0x00000000002000 - 0x00000007ef0fff, 133099520 bytes (32495 pages)
> 0x00000007f0f000 - 0x00000034bfffff, 751767552 bytes (183537 pages)
> 0x00000036052000 - 0x0000003cb2efff, 112054272 bytes (27357 pages)
> 0x0000003cb36000 - 0x0000003cb36fff, 4096 bytes (1 pages)
> 0x0000003cb38000 - 0x0000003cb39fff, 8192 bytes (2 pages)
> 0x0000003cb3b000 - 0x0000003cb3cfff, 8192 bytes (2 pages)
> 0x0000003cb40000 - 0x0000003cb40fff, 4096 bytes (1 pages)
> 0x0000003cb42000 - 0x0000003cb43fff, 8192 bytes (2 pages)
> 0x0000003cb45000 - 0x0000003df4ffff, 21016576 bytes (5131 pages)
> 0x0000003df60000 - 0x0000003dffffff, 655360 bytes (160 pages)
> 0x00000040000000 - 0x000000fbffffff, 3154116608 bytes (770048 pages)
> 0x00000100000000 - 0x000001f372afff, 4084379648 bytes (997163 pages)
>=20
>=20
> I wonder if such could be contributing to why
> DMA_HIGH_LIMIT had to be set smaller than the
> 3 GiByte limit: to under a 1 GiByte limit.
It looks like my ACPI boots are getting the EFI framebuffer
being more like (default gpu_mem example, as I remember):
0x3e2fe000 - 0x3eae6fff, 7 MB ( 2025 pages) NoAlloc=20
again not having a containing "Physical memory chunk" (and,
again in the gpu_mem=3D/gpu_mem_1024=3D area).
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?14D4D816-3CF3-4911-91FB-2AA50A4D1C22>