Date: Sat, 3 Oct 2020 20:15:26 -0700 From: Mark Millard <marklmi@yahoo.com> To: Robert Crowston <crowston@protonmail.com>, freebsd-arm <freebsd-arm@freebsd.org> Subject: Re: rpi4 FreeBSD vs. ubuntu u-boot fdt print / memereserve difference (lack of reserve in FreeBSD context), 0x3b400000 vs. DMA_HIGH_LIMIT being 0x3c000000 Message-ID: <01C14FE4-C2A6-4459-A5EE-AE33045FAF42@yahoo.com> In-Reply-To: <BDA2D50C-E8C8-4BF2-8C97-C1B2516F06F3@yahoo.com> References: <1A13F7B5-F8C3-4022-939C-2992E53D1DF1@yahoo.com> <BDA2D50C-E8C8-4BF2-8C97-C1B2516F06F3@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2020-Oct-3, at 16:10, Mark Millard <marklmi at yahoo.com> wrote:
> On 2020-Oct-3, at 15:20, Mark Millard <marklmi at yahoo.com> wrote:
>=20
>> Another FreeBSD vs. ubuntu context difference, this time in the
>> fdt print / output . . .
>>=20
>> The ubuntu u-boot has (fdt print / output):
>>=20
>> memreserve =3D <0x3b400000 0x04c00000>;
>> . . . (Note: 0x3b400000+0x04c00000 =3D=3D 0x40000000) . . .
>>=20
>> #address-cells =3D <0x00000002>;
>> #size-cells =3D <0x00000001>;
>> . . .
>> axi {
>> vc_mem {
>> reg =3D <0x3ec00000 0x40000000 0xc0000000>;
>> };
>> Note: "vc_mem is solely used as a mechanism for passing a =
couple
>> of parameters through from the firmware to vcdbg"
>> End note
>> };
>>=20
>> . . . ( boot args has: vc_mem.mem_base=3D0x3ec00000 =
vc_mem.mem_size=3D0x40000000 ) . . .
>>=20
>> reserved-memory {
>> #address-cells =3D <0x00000002>;
>> #size-cells =3D <0x00000001>;
>> ranges;
>> phandle =3D <0x0000003d>;
>> linux,cma {
>> compatible =3D "shared-dma-pool";
>> size =3D <0x04000000>;
>> reusable;
>> linux,cma-default;
>> alloc-ranges =3D <0x00000000 0x00000000 =
0x30000000>;
>> phandle =3D <0x0000003e>;
>> };
>> };
>> . . . (I split the reg into lines below) . . .
>> memory@0 {
>> device_type =3D "memory";
>> reg =3D <0x00000000 0x00000000 0x3b400000
>> 0x00000000 0x40000000 0xbc000000
>> 0x00000001 0x00000000 0x80000000
>> 0x00000001 0x80000000 0x80000000>;
>> };
>> . . . (Note: 0x40000000+0xbc000000 =3D=3D 0xFC000000) . . .
>>=20
>> (I've ignored gpiomem above and below.)
>>=20
>> It appears to be that the memreserve may be important
>> to have. The above may also suggest that FreeBSD's:
>>=20
>> #define DMA_HIGH_LIMIT 0x3c000000
>>=20
>> may be a little too large (< or <=3D 0x3b400000 ?).
>>=20
>> FreeBSD u-boot reports just:
>>=20
>> /memreserve/ 0x0 0x1000;
>> . . .
>> memory@0 {
>>=20
>> device_type =3D "memory";
>> reg =3D <0x0 0x0 0x0>;
>> };
>>=20
>> And so does not indicate anything special for either of
>> (showing begin/end points):
>>=20
>> 0x3b400000..0x3FFFFFFF (in use by the vc?)
>> 0xFC000000..0xFFFFFFFF (I/O peripheral area and such?)
>>=20
>> The context is an 8 GiByte RPi4 in both examples. Various
>> details would vary on 1 GiByte and 2 GiByte RPi4Bs and
>> some in memory@0 on the 4 GiBYTe RPi4B.
>=20
> Turns out that rpi_DATA_2711_1p0.pdf 's "Figure 1. BCM2711
> Address Maps" shows this "SDRAM (for the VC)" area in the
> two 35-bit Address Maps, showing 0x0_4000_0000 as the
> next address after the area in both diagrams --and notes
> for area for each diagram:
>=20
> QUOTE
> Size of VC SDRAM
> determined by
> config.txt
> END QUOTE
>=20
> But ubuntu uses includes and such so overall there
> are the following .txt files ( cmdline.txt being for
> a different purpose ):
>=20
> # ls -ld /boot/firmware/*.txt
> -rwxr-xr-x 1 root root 141 Jul 31 16:48 /boot/firmware/cmdline.txt
> -rwxr-xr-x 1 root root 1117 Jul 31 16:48 /boot/firmware/config.txt
> -rwxr-xr-x 1 root root 327 Jul 31 16:48 /boot/firmware/syscfg.txt
> -rwxr-xr-x 1 root root 251 Sep 26 22:14 /boot/firmware/usercfg.txt
>=20
> config.txt includes syscfg.txt and usercfg.txt .
>=20
> config.txt uses the [pi4], [pi2], [pi3], and [all] section
> notation as well.
>=20
> Looks like one of u-boot's jobs is to figure out the figures
> that go in (base then size):
>=20
> memreserve =3D <0x???????? 0x????????>;
>=20
> (such that the total is 0x40000000 for the RPi4B) in order
> to protect the VC SDRAM area.
>=20
>=20
> The diagrams also indicate that 0xFC000000..0xFFFFFFFF is
> for when "low Perioheral" mode is in use, listing ("up
> to H" not including H):
>=20
> ARM Local peripherals from 0x0_FF80_0000 up to 0x1_0000_0000
> and:
> Main peripherals from 0x0_FC00_0000 up to 0x0_FF80_0000
>=20
> Otherwise they are listed at:
>=20
> ARM Local peripherals from 0x4_C000_0000 up to 0x5_0000_0000
> and:
> Main peripherals from 0x4_7C00_0000 up to 0x4_8000_0000
>=20
> I gather the ubuntu "fdt print /" result implies that ubuntu
> is using "Low Peripheral" mode (or at least allowing for it).
>=20
> There are some other "Reserved" areas and the special areas
> that are "two L2 cache aliases (one allocating, one
> non-allocating) which cache (only) the first 1GB of SDRAM".
> if these are all counted then 0x4_0000_0000 up to
> 0x6_0000_0000 is all some form of special-use area in
> both diagrams and 0x6_0000_0000 through 0x7_FFFF_FFFF is the
> PCIe area in both diagams.
Looks like in ubuntu land the following from the dtb/fdt
work together(?) to cover avoiding memory that is
inappropriate for use for low-RAM DMA activity:
U-Boot> fdt print /
/ {
memreserve =3D <0x3b400000 0x04c00000>;
. . . (Note: 0x3b400000+0x04c00000 =3D=3D 0x40000000) . . .
reserved-memory {
#address-cells =3D <0x00000002>;
#size-cells =3D <0x00000001>;
ranges;
phandle =3D <0x0000003d>;
linux,cma {
compatible =3D "shared-dma-pool";
size =3D <0x04000000>;
reusable;
linux,cma-default;
alloc-ranges =3D <0x00000000 0x00000000 =
0x30000000>;
phandle =3D <0x0000003e>;
};
};
. . .
where:
alloc-ranges =3D <0x00000000 0x00000000 0x30000000> indicates a low
memory range to use for "shared-dma-pool" DMA activity.
memreserve =3D <0x3b400000 0x04c00000> indicates the actual
area in use for gnu_mem=3D/gnu_mem_1024=3D material and so indicates
an area that must be avoided. It appears that this adjusts based
on what config.txt has for (either of) gnu_mem=3D/gnu_mem_1024=3D .
It appears that the alloc-ranges=3D... above allows for the maximum
recommended gpu_mem/gnu_mem_1024 figure for the RPi4B and so,
if that limit is honored, avoids overlapping the memreserve. This
may suggest that your code should change to:
#define DMA_HIGH_LIMIT 0x30000000
XHCI and the type-DMA4 engine use need to explicitly avoid the
memreserve area. The old normal/LITE DMA could be redirected to
use a different 1 GiByte range (shifted above the memreserve
when there is enough RAM to do so).
The more modern dtb/fdt's that I use in some contexts have the
reserved-memory linux,cma alloc-ranges. (I'm not claiming such
information is translated to uefi/ACPI.) But the old dtb/fdt's
used with FreeBSD u-boot do not indicate those alloc-ranges.
The u-boot for FreeBSD that I'm using (from ports) does not
generate the memreserve material so neither the old or new
fdt's end up with such material.
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?01C14FE4-C2A6-4459-A5EE-AE33045FAF42>