Date: Thu, 13 Feb 2020 09:36:59 -0800 From: Mark Millard <marklmi@yahoo.com> To: Kyle Evans <kevans@freebsd.org> Cc: Ralf Wenk <iz-rpi03@hs-karlsruhe.de>, Andrew Turner <andrew@freebsd.org>, Oleksandr Tymoshenko <gonzo@freebsd.org>, freebsd-arm <freebsd-arm@freebsd.org>, Emmanuel Vadot <manu@freebsd.org> Subject: Re: A investigative hack that makes (for example) head -r356529 boot and operate normally an RPi4B (finally!): protect all armstub8-gic.bin's loaded content from replacement by the kernel Message-ID: <36CF6E4B-5607-4752-B2DF-C265BCFB95BA@yahoo.com> In-Reply-To: <B64C6065-D804-4D6F-8660-F8DBB9946DEB@yahoo.com> References: <7E7605DC-021D-448A-8459-8EC26BA9836D.ref@yahoo.com> <7E7605DC-021D-448A-8459-8EC26BA9836D@yahoo.com> <E1j2EmR-000Mwm-ID@iz-wera-new.HS-Karlsruhe.DE> <CACNAnaHVYeuoK=4f7XzokpAVjcqd6ovD58-0LdP_vPGjiXQgdw@mail.gmail.com> <E1j2FRX-000MyR-1O@iz-wera-new.HS-Karlsruhe.DE> <E1j2G3j-000Mzo-Dw@iz-wera-new.HS-Karlsruhe.DE> <CACNAnaHUwu0GmqW9QZRhS2sMK0HpsjdYRz3YouL5FMBdWBCW-Q@mail.gmail.com> <B64C6065-D804-4D6F-8660-F8DBB9946DEB@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2020-Feb-13, at 08:50, Mark Millard <marklmi at yahoo.com> wrote:
> On 2020-Feb-13, at 07:22, Kyle Evans <kevans at freebsd.org> wrote:
>=20
>> On Thu, Feb 13, 2020 at 9:05 AM Ralf Wenk <iz-rpi03@hs-karlsruhe.de> =
wrote:
>>>=20
>>> On 2020-02-13 at 15:26 +0100 Ralf Wenk wrote:
>>>> On 2020-02-13 at 7:49 -0600 Kyle Evans wrote:
>>>>> On Thu, Feb 13, 2020 at 7:43 AM Ralf Wenk =
<iz-rpi03@hs-karlsruhe.de> wrote:
>>>>>>=20
>>>>>> On 2020-02-12 at 18:00 -0800 Mark Millard wrote via freebsd-arm:
>>>>>>> [...]
>>>>>>>=20
>>>>>>> # svnlite diff /usr/src/sys/dev/fdt/fdt_common.c
>>>>>>> Index: /usr/src/sys/dev/fdt/fdt_common.c
>>>>>>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>>>>>>> --- /usr/src/sys/dev/fdt/fdt_common.c (revision 357529)
>>>>>>> +++ /usr/src/sys/dev/fdt/fdt_common.c (working copy)
>>>>>>> @@ -485,7 +485,18 @@
>>>>>>>=20
>>>>>>> tuples =3D res_len / tuple_size;
>>>>>>> reservep =3D (pcell_t *)&reserve;
>>>>>>> +#ifdef __aarch64__
>>>>>>> + //HACK!!!
>>>>>>> + // Reserve the first few pages, for example to
>>>>>>> + // preserve armstub8-gic.bin or armstub.bin
>>>>>>> + // content.
>>>>>>> + mr[0].mr_start=3D 0;
>>>>>>> + mr[0].mr_size=3D 2*4096;
>>>>>>> + tuples++;
>>>>>>> + for (i =3D 1; i < tuples; i++) {
>>>>>>> +#else
>>>>>>> for (i =3D 0; i < tuples; i++) {
>>>>>>> +#endif
>>>>>>>=20
>>>>>>> rv =3D fdt_data_to_res(reservep, addr_cells, =
size_cells,
>>>>>>> (u_long *)&mr[i].mr_start, (u_long =
*)&mr[i].mr_size);
>>>>>>> @@ -512,6 +523,11 @@
>>>>>>>=20
>>>>>>> root =3D OF_finddevice("/reserved-memory");
>>>>>>> if (root =3D=3D -1) {
>>>>>>> + // Fail over to checking for and handling =
memreserve,
>>>>>>> + // such as for a RPi4B.
>>>>>>> + if (0 =3D=3D =
fdt_get_reserved_regions(reserved,mreserved))
>>>>>>> + return (0);
>>>>>>> +
>>>>>>> return (ENXIO);
>>>>>>> }
>>>>>>>=20
>>>>>>=20
>>>>>> I can confirm that with your patch(es) my RPi3 does not freeze =
any more
>>>>>> when loading mac_ntpd.ko. The patches are applied against =
r357853M.
>>>=20
>>> An reboot is working again too.
>>>=20
>>>>> Have you tested the RPi3 with just this second hunk of patch to
>>>>> fallover to memreserve, or is the first hunk definitely required =
as
>>>>> well?
>>>>=20
>>>> Good question. I tested both hunks together.
>>>> Will try what happens when just applying the second and report =
back.
>>>=20
>>> Here it is:
>>> Without the first hunk the system freezes again when loading =
mac_ntpd.ko.
>>> Also the CPU information during boot for CPUs 1 to 3 looks strange =
again.
>>>=20
>>=20
>> Yeah- I see it now; both armstubs are about 5k. I've raised an
>> issue[0] with upstream for armstub/rpi bits to work out the proper
>> solution, because I don't necessarily want to commit the workaround.
>> I'll throw up the second hunk on phabricator for review by =
#arm/#arm64
>> folks, because that seems to me the proper fallback.
>>=20
>> I also discovered some issues when trying to read /memreserve/ with
>> our dtc and filed a PR[1] to fix those.
>>=20
>> Thanks,
>>=20
>> Kyle Evans
>>=20
>> [0] https://github.com/raspberrypi/tools/issues/107
>> [1] https://github.com/davidchisnall/dtc/pull/59
>=20
> The DTB information below is from:
>=20
> U-Boot> fdt addr 0x7ef2000=20
> U-Boot> fdt print / =20
>=20
> on a RPi4B 4 GiByte.
>=20
> On at least the RPi4B memreserve is not what causes
> the first page to be excluded:
>=20
> memreserve =3D <0x3b400000 0x04c00000>;
>=20
> Nor is memory@0 the cause:
>=20
> memory@0 {
> device_type =3D "memory";
> reg =3D <0x00000000 0x00000000 0x3b400000 0x00000000 =
0x40000000 0xbc000000>;
> };
>=20
> (That also skips the memreserve area.)
>=20
> I do not find anything in the DTB that indicates
> to exclude the first page.
>=20
> My hypothesis is that the FreeBSD code excludes
> the page based on some less obvious relationship
> that I'm not identifying.
>=20
> There is the cpu-rlease-addr information that seems
> to refer to some 1st memory page content:
>=20
> cpus {
> #address-cells =3D <0x00000001>;
> #size-cells =3D <0x00000000>;
> enable-method =3D "brcm,bcm2836-smp";
> phandle =3D <0x000000be>;
> cpu@0 {
> device_type =3D "cpu";
> compatible =3D "arm,cortex-a72";
> reg =3D <0x00000000>;
> enable-method =3D "spin-table";
> cpu-release-addr =3D <0x00000000 0x000000d8>;
> phandle =3D <0x0000001d>;
> };
> cpu@1 {
> device_type =3D "cpu";
> compatible =3D "arm,cortex-a72";
> reg =3D <0x00000001>;
> enable-method =3D "spin-table";
> cpu-release-addr =3D <0x00000000 0x000000e0>;
> phandle =3D <0x0000001e>;
> };
> cpu@2 {
> device_type =3D "cpu";
> compatible =3D "arm,cortex-a72";
> reg =3D <0x00000002>;
> enable-method =3D "spin-table";
> cpu-release-addr =3D <0x00000000 0x000000e8>;
> phandle =3D <0x0000001f>;
> };
> cpu@3 {
> device_type =3D "cpu";
> compatible =3D "arm,cortex-a72";
> reg =3D <0x00000003>;
> enable-method =3D "spin-table";
> cpu-release-addr =3D <0x00000000 0x000000f0>;
> phandle =3D <0x00000020>;
> };
> };
Looking at the code there is:
/* Load the physical memory ranges */
efihdr =3D (struct efi_map_header *)preload_search_info(kmdp,
MODINFO_METADATA | MODINFOMD_EFI_MAP);
if (efihdr !=3D NULL)
add_efi_map_entries(efihdr);
#ifdef FDT
else {
/* Grab physical memory regions information from device =
tree. */
if (fdt_get_mem_regions(mem_regions, &mem_regions_sz,
NULL) !=3D 0)
panic("Cannot get physical memory regions");
arm_physmem_hardware_regions(mem_regions, =
mem_regions_sz);
}
if (fdt_get_reserved_mem(mem_regions, &mem_regions_sz) =3D=3D 0)
arm_physmem_exclude_regions(mem_regions, mem_regions_sz,
EXFLAG_NODUMP | EXFLAG_NOALLOC);
#endif
/* Exclude the EFI framebuffer from our view of physical memory. =
*/
efifb =3D (struct efi_fb *)preload_search_info(kmdp,
MODINFO_METADATA | MODINFOMD_EFI_FB);
if (efifb !=3D NULL)
arm_physmem_exclude_region(efifb->fb_addr, =
efifb->fb_size,
EXFLAG_NOALLOC);
. . .
if (boothowto & RB_VERBOSE) {
print_efi_map_entries(efihdr);
arm_physmem_print_tables();
}
It looks to me like the boot -v text:
Type Physical Virtual #Pages Attr
Reserved 000000000000 0 00000001 WB=20
ConventionalMemory 000000001000 1000 00007ef1 WB=20
BootServicesData 000007ef2000 7ef2000 0000001c WB=20
ConventionalMemory 000007f0e000 7f0e000 00029f93 WB=20
BootServicesData 000031ea1000 31ea1000 00000001 WB=20
LoaderData 000031ea2000 31ea2000 00008001 WB=20
LoaderCode 000039ea3000 39ea3000 000000a6 WB=20
Reserved 000039f49000 39f49000 00000007 WB=20
BootServicesData 000039f50000 39f50000 00000001 WB=20
Reserved 000039f51000 39f51000 00000002 WB=20
RuntimeServicesData 000039f53000 39f53000 00000001 WB RUNTIME
Reserved 000039f54000 39f54000 00000001 WB=20
BootServicesData 000039f55000 39f55000 00000002 WB=20
RuntimeServicesData 000039f57000 39f57000 00000001 WB RUNTIME
LoaderData 000039f58000 39f58000 00001408 WB=20
RuntimeServicesCode 00003b360000 3b360000 00000010 WB RUNTIME
LoaderData 00003b370000 3b370000 00000090 WB=20
BootServicesData 000040000000 40000000 000bc000 WB=20
MemoryMappedIO 0000fe100000 fe100000 00000001 RUNTIME
is from print_efi_map_entries via the efihdr instead
of from the FreeBSD FDT code and the DTB.
So is it u-boot that provides the efihdr for which
add_efi_map_entries generated those regions?
That might explain why I do not find matching DTB
material for all of it.
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?36CF6E4B-5607-4752-B2DF-C265BCFB95BA>