Date: Sat, 7 Jan 2023 09:37:55 +0100 From: =?utf-8?Q?Klaus_K=C3=BCchemann?= <maciphone2@googlemail.com> To: Mark Millard <marklmi@yahoo.com>, freebsd-arm@freebsd.org Subject: Re: How to make FreeBSD's kernel boot a RPi4B with modern RPi* firmware Message-ID: <9E9C739E-8308-472A-B797-05A37559DD00@googlemail.com> In-Reply-To: <BCCBE0D7-8BEB-4D6D-A017-9A59000F1E2B@yahoo.com> References: <9C037D3F-A440-4708-993D-117F313691BB@yahoo.com> <374EC3E5-4CB4-4336-A8B9-7A9CF6151691@yahoo.com> <BCCBE0D7-8BEB-4D6D-A017-9A59000F1E2B@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Mark, `ve tested your "early_dma=E2=80=9C patch now on the CM4(not tested on = the 4b)=E2=80=A6 true, it can boot the latest firmware but after investigating in = what=E2=80=99s new=20 in bcm2711-rpi-cm4.dtb I guess new features are mainly related to CPU = (L2-) caches features , That=E2=80=99s why we get things like : "clk_fixed4: <Fixed clock> = disabled on ofwbus0 clk_fixed4: Cannot FDT parameters.=E2=80=9C ... so I think we currently won=E2=80=99t benefit much of the new firmware. I had a little hope that an earlier dma could shine a bit more light on = bugs like this :=20 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D260131 But a tiny little hope can=E2=80=99t of course fix bugs :-) , so I think = we have to focus more on fixing existing bugs and=20 Adding device drivers(of course the ToDo-list is not news but still the = truth;-) Thanks for taking attention the current firmware things, always worth to = take a look into. Regards K. > Am 25.12.2022 um 17:37 schrieb Mark Millard <marklmi@yahoo.com>: >=20 > On Dec 24, 2022, at 20:14, Mark Millard <marklmi@yahoo.com> wrote: >=20 >> On Dec 24, 2022, at 19:15, Mark Millard <marklmi@yahoo.com> wrote: >>=20 >>> I finally ran into EARLY_DRIVER_MODULE, BUS_PASS_RESOURCE, >>> BUS_PASS_ORDER_MIDDLE and the like and they allow being >>> sure that the brcm,bcm2835-dma related setup has been done >>> before any use of it is made, despite the order in the >>> Device Tree: use an earlier pass for brcm,bcm2835-dma >>> related attach. This avoids the kernel crashing during >>> boot. >>>=20 >>> The example context used below has: serial console with >>> USB3 SSD boot media (not requiring a usb_pgood_delay >>> setting), booting a stable/13. The RPI4B is a C0T one (no >>> 3 GiByte limitation, for example: the PCIe wrapper logic >>> has been corrected). >>>=20 >>> stable/13's source code changes ( similarly for >>> releng/13.1 ): >>>=20 >>> diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c = b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>> index cab8639bb607..d8b49acfe332 100644 >>> --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>> +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>> @@ -766,5 +766,6 @@ static driver_t bcm_dma_driver =3D { >>>=20 >>> static devclass_t bcm_dma_devclass; >>>=20 >>> -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, bcm_dma_devclass, = 0, 0); >>> +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, = bcm_dma_devclass, >>> + 0, 0, BUS_PASS_RESOURCE + BUS_PASS_ORDER_MIDDLE); >>> MODULE_VERSION(bcm_dma, 1); >>>=20 >>>=20 >>> For reference, a 13S snapshot with my kernel build replacing >>> the snapshot's kernel, was booted with: >>>=20 >>> # strings /boot/msdos/start4.elf | grep VC_BUILD_ID_ >>> VC_BUILD_ID_USER: dom >>> VC_BUILD_ID_TIME: 11:09:05 >>> VC_BUILD_ID_VARIANT: start >>> VC_BUILD_ID_TIME: Oct 26 2022 >>> VC_BUILD_ID_BRANCH: bcm2711_2 >>> VC_BUILD_ID_HOSTNAME: buildbot >>> VC_BUILD_ID_PLATFORM: raspberrypi_linux >>> VC_BUILD_ID_VERSION: c72ad6b26ff40c91ef776b847436094ee63fabee = (clean) >>>=20 >>> There are new things present that FreeBSD reports >>> but ignores, producing messages like: >>>=20 >>> clk_fixed4: <Fixed clock> disabled on ofwbus0 >>> clk_fixed4: Cannot FDT parameters. >>> device_attach: clk_fixed4 attach returned 6 >>>=20 >>> over and over during part of the boot. It seems to >>> retry as it goes and thus produce so many messages. >>>=20 >>> There was also: >>>=20 >>> fb0: <BCM2835 VT framebuffer driver> on simplebus0 >>> fb0: changing fb bpp from 0 to 24 >>> mbox0: mbox response error >>> fb0: bcm2835_mbox_fb_init failed, err=3D5 >>> device_attach: fb0 attach returned 6 >>>=20 >>> genet0 is working. >>>=20 >>> I've not checked if the microsd card slot can be used. >>>=20 >>> I used the normal FreeBSD U-Boot since I was not booting >>> the NVM3 media that requires extra time (usb_pgood_delay >>> would be assigned in my own U-Boot builds). >>>=20 >>> For reference, I used my typical sort of config.txt : >>>=20 >>> # more /boot/msdos/config.txt >>> [all] >>> arm_64bit=3D1 >>> dtparam=3Daudio=3Don,i2c_arm=3Don,spi=3Don >>> dtoverlay=3Dmmc >>> dtoverlay=3Ddisable-bt >>> device_tree_address=3D0x4000 >>> kernel=3Du-boot.bin >>>=20 >>> [pi4] >>> hdmi_safe=3D1 >>> armstub=3Darmstub8-gic.bin >>>=20 >>> # >>> [all] >>> # >>> # Local addition that avoids USB3 SSD boot failures that look like: >>> # uhub_reattach_port: port ? reset failed, error=3DUSB_ERR_TIMEOUT >>> # uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling = port ? >>> initial_turbo=3D60 >>> # U-Boot that has, for example, a built-in usb_pgood_delay = assignment >>> # for a media specific issue added: >>> #kernel=3Du-boot.bin.2022.10.arm64 >>> # >>> # Local additions: >>> enable_uart=3D1 >>> uart_2ndstage=3D1 >>> dtdebug=3D1 >>> disable_commandline_tags=3D1 >>> disable_overscan=3D1 >>> #gpu_mem_1024=3D32 >>> # >>> #program_usb_boot_mode=3D1 >>> #program_usb_boot_timeout=3D1 >>>=20 >>> # Old RPi3's/RPi2Bv1.2's may ignore [pi4] and the like. >>> # That would make the below inappropriate for such contexts. >>> [pi4] >>> # Locally avoid hdmi_safe's dislay scaling: >>> hdmi_safe=3D0 >>> # >>> armstub=3Darmstub8-gic.bin >>> # >>> # Local additions: >>> over_voltage=3D6 >>> arm_freq=3D2000 >>> sdram_freq_min=3D3200 >>> force_turbo=3D1 >>> # >>> #total_mem=3D1024 >>> #total_mem=3D991 >>> [all] >>>=20 >>> [pi3]=20 >>> armstub=3Darmstub8.bin >>> dtoverlay=3Dpwm >>> audio_pwm_mode=3D0 >>> [all] >>>=20 >>>=20 >>> As for main [so: 14], the devclass_t use is gone, thus >>> the source code changes are: >>>=20 >>> diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c = b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>> index 5f9ecb0b7981..83c4c493a66b 100644 >>> --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>> +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >>> @@ -764,5 +764,6 @@ static driver_t bcm_dma_driver =3D { >>> sizeof(struct bcm_dma_softc), >>> }; >>>=20 >>> -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0); >>> +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0, >>> + BUS_PASS_RESOURCE + BUS_PASS_ORDER_MIDDLE); >>> MODULE_VERSION(bcm_dma, 1); >>>=20 >>>=20 >>=20 >> I should note that the above is not likely to be >> the most appropriate in detail. The boot reports: >>=20 >> # dmesg -a | grep bcm_dma0 >> bcm_dma0: <BCM2835 DMA Controller> mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 >> bcm_dma0: cannot allocate interrupt >> device_attach: bcm_dma0 attach returned 6 >> bcm_dma0: <BCM2835 DMA Controller> mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 >> bcm_dma0: cannot allocate interrupt >> device_attach: bcm_dma0 attach returned 6 >> bcm_dma0: <BCM2835 DMA Controller> mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 >> bcm_dma0: cannot allocate interrupt >> device_attach: bcm_dma0 attach returned 6 >> bcm_dma0: <BCM2835 DMA Controller> mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 >> bcm_dma0: cannot allocate interrupt >> device_attach: bcm_dma0 attach returned 6 >> bcm_dma0: <BCM2835 DMA Controller> mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 >>=20 >> where that last (working) one has the message >> context: >>=20 >> gic0: <ARM Generic Interrupt Controller> mem = 0x40041000-0x40041fff,0x40042000-0x40043fff,0x40044000-0x40045fff,0x400460= 00-0x40047fff irq 30 on simplebus0 >> gic0: pn 0x2, arch 0x2, rev 0x1, implementer 0x43b irqs 256 >> bcm_dma0: <BCM2835 DMA Controller> mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 >>=20 >> So something involving BUS_PASS_INTERRUPT or later >> (but before, say, BUS_PASS_SUPPORTDEV) may be more >> appropriate. Possibly: >>=20 >> BUS_PASS_INTERRUPT + BUS_PASS_ORDER_LATE >>=20 >> (so after gic0). >>=20 >>=20 >=20 > So, I'm now using . . . > (leading whitespace possibly not accurately preserved) >=20 >=20 > stable/13's source code changes are ( similarly for > releng/13.1 ): >=20 > # git -C /usr/13S-src/ diff sys/arm/broadcom/bcm2835/bcm2835_dma.c > diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c = b/sys/arm/broadcom/bcm2835/bcm2835_dma.c > index cab8639bb607..6d521d6dcace 100644 > --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c > +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c > @@ -766,5 +766,6 @@ static driver_t bcm_dma_driver =3D { >=20 > static devclass_t bcm_dma_devclass; >=20 > -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, bcm_dma_devclass, = 0, 0); > +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, = bcm_dma_devclass, > + 0, 0, BUS_PASS_INTERRUPT + BUS_PASS_ORDER_LATE); > MODULE_VERSION(bcm_dma, 1); >=20 >=20 > main's [so: 14's] source code changes are: >=20 > # git -C /usr/main-src/ diff sys/arm/broadcom/bcm2835/bcm2835_dma.c > diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c = b/sys/arm/broadcom/bcm2835/bcm2835_dma.c > index 5f9ecb0b7981..d901447df1e9 100644 > --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c > +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c > @@ -764,5 +764,6 @@ static driver_t bcm_dma_driver =3D { > sizeof(struct bcm_dma_softc), > }; >=20 > -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0); > +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0, > + BUS_PASS_INTERRUPT + BUS_PASS_ORDER_LATE); > MODULE_VERSION(bcm_dma, 1); >=20 > =3D=3D=3D > Mark Millard > marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9E9C739E-8308-472A-B797-05A37559DD00>