Date: Sun, 8 Jan 2023 00:26:48 -0800 From: Mark Millard <marklmi@yahoo.com> To: =?utf-8?Q?Klaus_K=C3=BCchemann?= <maciphone2@googlemail.com> Cc: freebsd-arm@freebsd.org Subject: Re: How to make FreeBSD's kernel boot a RPi4B with modern RPi* firmware Message-ID: <FC75A32C-B839-4699-AA15-5232CF3B1FC1@yahoo.com> In-Reply-To: <3BE72ED7-8787-45E8-8341-FE9CF4CFB84F@googlemail.com> References: <9C037D3F-A440-4708-993D-117F313691BB@yahoo.com> <374EC3E5-4CB4-4336-A8B9-7A9CF6151691@yahoo.com> <BCCBE0D7-8BEB-4D6D-A017-9A59000F1E2B@yahoo.com> <9E9C739E-8308-472A-B797-05A37559DD00@googlemail.com> <EAD84A57-E8F0-4149-BCFC-8A06FF03B11B@yahoo.com> <E1A6057C-4E0F-45B7-9278-86A09426DE49@googlemail.com> <217ACD33-A466-4A01-AD36-5D4A0C1B3CF0@yahoo.com> <3BE72ED7-8787-45E8-8341-FE9CF4CFB84F@googlemail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Jan 7, 2023, at 20:01, Klaus K=C3=BCchemann = <maciphone2@googlemail.com> wrote: >=20 >> Am 25.12.2022 um 04:15 schrieb Mark Millard <marklmi@yahoo.com>: >>=20 >> The example context used below has: serial console with >> USB3 SSD boot media (not requiring a usb_pgood_delay >> setting), booting a stable/13. The RPI4B is a C0T one (no >> 3 GiByte limitation, for example: the PCIe wrapper logic >> has been corrected). >=20 > O.K., Mark, I=C2=B4ve tried to read your 4b-related reports of the = last period more closely..=20 > And now I=E2=80=99m a bit confused : > I tried now : > <<FreeBSD generic 14.0-CURRENT FreeBSD 14.0-CURRENT #0 = main-n259905-231d75568f16: Sun Jan 1 11:28:27 UTC 2023 >> > on USB3 SSD media 4b B0T - device directly from a current img.xz and = couldn=E2=80=99t determine any problems with bcm_dma or pcie, I think you are confusing a failed experimental change with normal-software's operation --and the experimental change only applied to "C0T" parts. C0T RPi4B parts should not normally need bounce buffer activity. But I failed to make that work in my earlier experiment. (The "C0T" parts can do bounce buffer activity just fine but teh activity has the overall performance consequences when there is sustained bouncing.) You do not have my failed experimental code, so you could not test what I originally tested --even if you had a "C0T" RPi4B context available. > I don=E2=80=99t own a 4b C0T, just the C0T CM4, where my = pcie-bug-report stays unpatched since 1 year, > It needs an extended JTAG- investigation, it cannot be fixed by = firmware, it has to happen in the driver logic. > The CM4 is my personal problem, it=E2=80=99s not a supported device = (has also devmatch issues on booting), > but I can live with that on working emmc& working genet0.. I'll note that for snapshot generation there is in: https://cgit.freebsd.org/src/tree/release/arm64/RPI.conf the text: DTB_DIR=3D"/usr/local/share/rpi-firmware" DTB=3D"bcm2710-rpi-2-b.dtb bcm2710-rpi-3-b.dtb bcm2710-rpi-3-b-plus.dtb = bcm2710-rpi-cm3.dtb bcm2711-rpi-4-b.dtb" As near as I can tell, that is the list of what, at one time, was considered supported (tested?) for aarch64 RPi*'s. No CM4. > The 4b(including C0T) is a supported device(is it?) and=20 > if I understand you right : > Your 4b C0T crashes (at least sometimes) by failing in = dma-computation. You are confusing standard-code use with experimental code changes being used, changes related to avoiding unnecessary bounce buffering for "C0T" parts. > And you have a valid fix for the 4b C0T ? =E2=80=A6 I've no clue why my experiment failed. I did want to check if it would fail with modern RPi* firmware (and the matching .dtb files). But that, in turn, requires avoiding the crash for more modern RPi* firmware. Such crash avoidance could be useful to other folks as well (for other purposes than mine). > While I unfortunately cannot test 4b C0T, it=20 > sounds that if you can fix it, it=E2=80=99s a must have fix for that = device? The standard code for RPi4B's that does the bounce buffering does not have the problem that my experiment had --on any RPi4B that I've tested. I was trying to see what some performance was like without bounce buffering being significantly involved but never got such to work reliably. > Side note: have you tried fixing pcie@7d500000 : in the dts without = changing start4.elf and the likes? > Is that 4b C0T bug based on start4.elf or the dts or whatever? I've no clue what lead to the failure of my experiment. Most likely my own mistakes of some kind, given my ignorance of various things. But, that leaves me exploring based on not much understanding of what is likely to prove useful to explore, at least for now. > For my devices the EARLY_DRIVER_ would do nothing but spam dmesg with = new firmware, it can=E2=80=99t fix dma on the CM4 and=20 > Is not needed for the 4b B0T... An RPi4B with EARLY_DRIVER_ + sysutils/rpi-firmware does not produce the "spam" messages and does not crash. An RPi4B with EARLY_DRIVER_ + sufficiently-newer firmware does produce the "spam" messages --because it did not crash first. An RPi4B without EARLY_DRIVER_ but with sufficiently-newer-firmware crashes instead --before it would have generated the many "spam" messages. The difference relative to "spam" messages is the differing live Device Tree content. > but is your 4b C0T fix based on your EARLY_DRIVER_MODULE -patch, even = without touching any firmware? > If so, you have to give it to Phabricator ;-) I have no fix for my failed experiment. I'm still exploring to see what sorts of things might have contributed to the failure. Other folks may have other reasons to want to experiment with more modern firmware. aarch64 or armv7 devices not covered in the snaphots (or even in systutils/rpi-firmware) could be examples that might be involved. Some devices might post-date the sysutils/rpi-firmware content. I listed 4 kinds that sysutils/rpi-firmware does not have .dtb files for. None of the 4 are considered supported at this stage, so far as I know. But someone might work on making one of them somewhat supported. Such a person might like to avoid dealing with the bcm_dma-lack-of-initialization related crashes. > Am 08.01.2023 um 01:59 schrieb Mark Millard <marklmi@yahoo.com>: >>=20 >>=20 >>=20 > Am 25.12.2022 um 04:15 schrieb Mark Millard <marklmi@yahoo.com>: >=20 > The example context used below has: serial console with > USB3 SSD boot media (not requiring a usb_pgood_delay > setting), booting a stable/13. The RPI4B is a C0T one (no > 3 GiByte limitation, for example: the PCIe wrapper logic > has been corrected). >=20 > On Jan 7, 2023, at 10:58, Klaus K=C3=BCchemann = <maciphone2@googlemail.com> wrote: >=20 >>=20 >>> Am 07.01.2023 um 11:18 schrieb Mark Millard <marklmi@yahoo.com>: >>>=20 >>>=20 >>> =E2=80=A6=E2=80=A6=E2=80=A6=E2=80=A6=E2=80=A6=E2=80=A6=E2=80=A6=E2=80=A6= ... >>>>>=20 >>>>>=20 >>>>> stable/13's source code changes are ( similarly for >>>>> releng/13.1 ): >>>>>=20 >>>>> # git -C /usr/13S-src/ diff sys/arm/broadcom/bcm2835/bcm2835_dma.c >>>>> diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c = b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>>>> index cab8639bb607..6d521d6dcace 100644 >>>>> --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>>>> +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>>>> @@ -766,5 +766,6 @@ static driver_t bcm_dma_driver =3D { >>>>>=20 >>>>> static devclass_t bcm_dma_devclass; >>>>>=20 >>>>> -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, = bcm_dma_devclass, 0, 0); >>>>> +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, = bcm_dma_devclass, >>>>> + 0, 0, BUS_PASS_INTERRUPT + BUS_PASS_ORDER_LATE); >>>>> MODULE_VERSION(bcm_dma, 1); >>>>>=20 >>>>>=20 >>>>> main's [so: 14's] source code changes are: >>>>>=20 >>>>> # git -C /usr/main-src/ diff = sys/arm/broadcom/bcm2835/bcm2835_dma.c >>>>> diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c = b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>>>> index 5f9ecb0b7981..d901447df1e9 100644 >>>>> --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>>>> +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>>>> @@ -764,5 +764,6 @@ static driver_t bcm_dma_driver =3D { >>>>> sizeof(struct bcm_dma_softc), >>>>> }; >>>>>=20 >>>>> -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0); >>>>> +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0, >>>>> + BUS_PASS_INTERRUPT + BUS_PASS_ORDER_LATE); >>>>> MODULE_VERSION(bcm_dma, 1); >>>>>=20 >>>=20 >>>=20 >>> =3D=3D=3D >>> Mark Millard >>> marklmi at yahoo.com >>>=20 >>=20 >>=20 >> =E2=80=A6=E2=80=A6.on the other hand : if your = EARLY_DRIVER_MODULE(bcm_dma=E2=80=A6 doesn=E2=80=99t do anything wrong, >> you could give it in phabricator review, why not?!.. >=20 > Yep, once I've better evidence from the RPi*'s that I have > access to. >=20 > I'll note that no vintages of the following .dtb files are > in the current sysutils/rpi-firmware port: >=20 > bcm2709-rpi-cm2.dtb > bcm2710-rpi-zero-2-w.dtb > bcm2710-rpi-zero-2.dtb > bcm2711-rpi-cm4s.dtb >=20 > I've no direct evidence of if any vintage of any of these > would end up hitting the bcm_dma issue or not. But I expect > that the EARLY_DRIVER_MODULE related patching would avoid > (just!) that specific crash problem if it would otherwise > would occur. >=20 > There is: >=20 > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D261147 >=20 > reporting the absence of bcm2710-rpi-zero-2-w.dtb . So > someone might want to experiment with more recent RPi* > firmware, possibly even to develop some level of support > for bcm2710-rpi-zero-2-w.dtb (live Device Tree possibly > adjusted by the RPi* firmware) --if changes are needed. >=20 > The .dtb vintage and the RPi* start*.efi and the like > vintages are not necessarily fully independent. Mixing > and matching could be a problem, independent of any > additional FreeBSD kernel-related issues. (It is another > example of the poor level of documentation for the RPi* > context.) =3D=3D=3D Mark Millard marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?FC75A32C-B839-4699-AA15-5232CF3B1FC1>