From owner-freebsd-arm@freebsd.org Fri Oct 9 10:02:15 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D1F1F43E407 for ; Fri, 9 Oct 2020 10:02:15 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic315-8.consmr.mail.gq1.yahoo.com (sonic315-8.consmr.mail.gq1.yahoo.com [98.137.65.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C73X64DGgz48tX for ; Fri, 9 Oct 2020 10:02:14 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: V3A0FjcVM1lNCuK5d13m1Ny26WZu1j4B1TFwz9AUbwr41I943T9FXFNJMU7tfMD J87pbglq5feJI84u1Rpm3ZPXFq2ISMz2AmaaBWJenf6jWPmtB9Fr7pvECxtee.CbjRMD7aBgjyf5 HFsGFA5_J42l5hhBr8XSPC8k_4Ea53pbIVYbXJRiYNvuMYF5HDRkmzYOsrBm3Tjjj.ny7HTjL8HN MwjCj05lsKf9vRYltHv745rQtAvFVS.9ION2LKYeeD08y5qFU8dY1OqH8I4Wn17KWT4JeO8fbG8L 0PfgWnvuF1Isjh3UzLJTw6mnT.Ka_iTLjbmH4RgBWxYMUn61dru37i3s8Ds3CTg7z5jxgmimXFfq bGeSmN8sRWrKuBEnwm3TQSPtneC0aCBU7eHN4CISI7wYyvhp5.Jk8PrxPUJnGwJUxb8qzMcv54YP hhg9he1bbcF2eK6zdWoQNTuXmOlWKCzph2X2yz3v2qqzWc7S2s1Pz98hy2ifgHUyJTMbhNBig7oo 4tseAkwl9FNfMA98GfwcZpEi3ks0i1YnLgsjnFtRrbxDoiba8M0nwi3gnaW7XDK5av1uXGpB62cP US29g0tT3Rh_93zKG0HZZKzgKf99wTGfzaKd7RIgyGg5hKluBNrmq6pk8F5yJzMXDenx.9aJo400 m7xkspCyoIWHyPzMWqPGCgPYXzvYJhkw1ro5WTSg41RAWZDgWGZCU8U6OF0CSz28fkDzJvDmbJxH iaSo3nnetrG8HH4NNPNLAXP0OaqXA.67_Beh7Xf2JCY.RhhP9jdMx5ztnT_rzs3kkESqk8ZF2aPO ixRupY_3X3uN1BaCRkQcGGd7Kn26QDFRMINBPh7fHw3A5LMC4f9VD43wplXzyULGEIMvXD5LWHU0 wt7hEQfzKHZuFB7r_WepazWfZRxGyBORI1HlqsLLNC0Rzfk.HxxIC7QcH8xBoX36RROcFeDhvhQ5 a.AFUKoC3e4_dASQ315ZVOelO0h561K6HNppn7AxeiHFbKOqlfamiwZcMu9haK4xXFgleA9OMosb 5L_pD9lpF2QZtJ_RS.mar.tRzHYUpsqx30PzxGGR_scPkGlzvVLOUzmK6O9UYYe1jlwFhh35v978 oD20n9.Hs5pEQdcr8pYBVO6ddU1jeEaoDv1c6DAnTqhTJrBWH2TH.zWsR4UT4AWxCh4XKMZ9Fg__ IaJa1zQGSaN0_KTTy5ECsow39rQ.TS5RQ_UTcss5d.m6Rpg4Sb65v5qav1woiLPoG3bL1KUGzzwL W.zIJ6O7kj8Tet.gFn5zh9RoOh0emeb8CjqEdN62YLWRrt9WdoItibQDUyYIq.cDh6a6fAT.AcQt qU7Hb3cuhsfVN6_2XvZesUvxiIOS1OoEcDfsY8N4Cly1t1g1mpnNTg8GRWDae09Wt0oMBHxY3FR. 6N2M6v_vFi0.p2d0rKHQejyv9eQ0vBtq4k_qpAvDV0UIp_XRDaa7FHD0LqI8qdTeKYmZdZhtIYpJ Ha4em.A0oDHG5aXlrVqmH.f1_demme.RsWc39i.s0P1gdRFupQkYxgk7IfQ0Hq.VBkysoa5Ht3K. BmBD5CqNeBKPJRFqQQs2lvVezmu_R0PfJ1rmMKWk- Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.gq1.yahoo.com with HTTP; Fri, 9 Oct 2020 10:02:10 +0000 Received: by smtp423.mail.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 98c16acc2b02c264e7bc5cd030a2d329; Fri, 09 Oct 2020 10:02:06 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: RPi4B: emmc2bus dma-range handling does not track the boot-time-FDT (u-boot based booting) From: Mark Millard In-Reply-To: <2B3F0409-88F2-4EBD-9C39-37929F973C77@yahoo.com> Date: Fri, 9 Oct 2020 03:02:06 -0700 Cc: freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: References: <98BC985D-EAAB-4AFB-AA8F-7391A45C4EBF@yahoo.com> <91324D35-B66A-4674-AE37-45F3DDB736FD@yahoo.com> <2B3F0409-88F2-4EBD-9C39-37929F973C77@yahoo.com> To: Klaus Cucinauomo X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Rspamd-Queue-Id: 4C73X64DGgz48tX X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.31 / 15.00]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.78)[-0.780]; FREEMAIL_TO(0.00)[googlemail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.04)[-1.036]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.998]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.32:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.32:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2020 10:02:15 -0000 On 2020-Oct-8, at 23:01, Mark Millard wrote: > On 2020-Oct-8, at 21:29, Mark Millard wrote: >=20 >> On 2020-Oct-8, at 20:06, Mark Millard wrote: >>=20 >>> On 2020-Oct-8, at 06:27, Klaus Cucinauomo wrote: >>>=20 >>>>> Am 08.10.2020 um 11:01 schrieb Mark Millard via freebsd-arm = : >>>>>=20 >>>>>=20 >>>>> The old u-boot/DTB combination in use does not have emmc2bus. >>>>> And, even if it did, FreeBSD would not use =E2=80=A6=E2=80=A6=E2=80=A6= . >>>>=20 >>>> =E2=80=A6 and the new 2020.10- combinations will even be more = annoying=E2=80=A6 >>>> While I meanwhile e.g. could boot the 4GB off of xhci, it hangs in = DeviceTree, >>>> Depending on GUESSED ;-) combinations and DeviceTree-patches in src = . >>>> There have to be necessary adjustments which are even not yet = upstreamed in u-boot=20 >>>> (depending on hw-revisions)=E2=80=A6and so on ... >>>> I doubt that anyone really will take explicitly time consuming care = of all that annoying RPI4-crap. >>>> I don't see any other way than targeting minimum 1 developer(better = more) ONLY to the RPI-platform=20 >>>> for a time period=E2=80=A6 other than leaving it crappy as is and = to be happy that it nevertheless=20 >>>> is a working fbsd-gadget :-) >>>=20 >>>=20 >>> Summary of probable-cause finding: u-boot 2020.10 is >>> expecting: >>>=20 >>> compatible =3D "brcm,generic-xhci" >>>=20 >>> instead of what seems to be in files that are in use >>> by FreeBSD: >>>=20 >>> compatible =3D "generic-xhci" >>>=20 >>> (I've no clue what all the other differences in the .dtb >>> file contents might be.) >>>=20 >>>=20 >>>=20 >>> The details of what I learned that reached that status . . . >>>=20 >>> I updated my ports environment on a RPi4 to build the final 2020.10 = u-boot >>> for the RPi4. I then built it and replaced the u-boot.bin on the = microsd >>> card that I boot with via the FreeBSD kernel being from that microsd = card >>> but later stages being from the USB3 SSD. I did not update anything = else. >>> So: still an older .dtb file. >>>=20 >>> That 8 GiByte RPi4B context booted just fine. >>>=20 >>> I then rebooted and had it stop in u-boot and looked around just a = little >>> bit. >>>=20 >>> U-Boot 2020.10 (Oct 09 2020 - 00:51:29 +0000) >>>=20 >>> DRAM: 7.9 GiB >>> RPI 4 Model B (0xd03114) >>> MMC: mmc@7e300000: 1, emmc2@7e340000: 0 >>> Loading Environment from FAT... In: serial >>> Out: vidconsole >>> Err: vidconsole >>> Net: eth0: ethernet@7d580000 >>> PCIe BRCM: link up, 5.0 Gbps x1 (SSC) >>> starting USB... >>> Bus xhci_pci: probe failed, error -110 >>> No working controllers found >>> Hit any key to stop autoboot: 0=20 >>>=20 >>> So it reproted the problem above. Yet . . . >>>=20 >>> U-Boot> pci 0 >>> Scanning PCI devices on bus 0 >>> BusDevFun VendorId DeviceId Device Class Sub-Class >>> _____________________________________________________________ >>> 00.00.00 0x14e4 0x2711 Bridge device 0x04 >>>=20 >>> U-Boot> pci 1 =20 >>> Scanning PCI devices on bus 1 >>> BusDevFun VendorId DeviceId Device Class Sub-Class >>> _____________________________________________________________ >>> 01.00.00 0x1106 0x3483 Serial bus controller 0x03 >>>=20 >>> So it does see the xHCI on the pci bus despite the >>> "Bus xhci_pci: probe failed, error -110". -110 looks >>> to be -ETIMEDOUT . >>>=20 >>> Looking at the v2020.10/drivers/usb/host/xhci-pci.c source >>> code shows that the xhci_pci driver has: >>>=20 >>> . . . >>> static const struct udevice_id xhci_pci_ids[] =3D { >>> { .compatible =3D "xhci-pci" }, >>> { } >>> }; >>>=20 >>> U_BOOT_DRIVER(xhci_pci) =3D { >>> .name =3D "xhci_pci", >>> .id =3D UCLASS_USB, >>> .probe =3D xhci_pci_probe, >>> .remove =3D xhci_deregister, >>> .of_match =3D xhci_pci_ids, >>> .ops =3D &xhci_usb_ops, >>> .platdata_auto_alloc_size =3D sizeof(struct usb_platdata), >>> .priv_auto_alloc_size =3D sizeof(struct xhci_ctrl), >>> .flags =3D DM_FLAG_ALLOC_PRIV_DMA, >>> }; >>>=20 >>> static struct pci_device_id xhci_pci_supported[] =3D { >>> { PCI_DEVICE_CLASS(PCI_CLASS_SERIAL_USB_XHCI, ~0) }, >>> {}, >>> }; >>>=20 >>> U_BOOT_PCI_DEVICE(xhci_pci, xhci_pci_supported); >>>=20 >>> (It looks like the above driver is used as the default >>> driver if no explicit match is found. It has been around >>> for years.) >>>=20 >>>=20 >>> But, in the past when I showed the likes of fdt print / >>> or translation to a .dts file, it did not have "xhci-pci" >>> in compatible but had "generic-xhci" instead, such as: >>>=20 >>> xhci@7e9c0000 { >>> compatible =3D "generic-xhci"; >>> status =3D "disabled"; >>> reg =3D <0x00000000 0x7e9c0000 0x00000000 = 0x00100000>; >>> interrupts =3D <0x00000000 0x000000b0 = 0x00000004>; >>> phandle =3D <0x000000d3>; >>> }; >>>=20 >>> For reference, for FreeBSD there is: >>>=20 >>> # grep -ri "xhci-pci" /usr/src/sys/ | more >>> # grep -ri "generic-xhci" /usr/src/sys/ | more >>> /usr/src/sys/dev/usb/controller/generic_xhci_fdt.c: = {"generic-xhci", true}, >>> /usr/src/sys/gnu/dts/arm/bcm-nsp.dtsi: compatible =3D= "generic-xhci"; >>> /usr/src/sys/gnu/dts/arm/bcm5301x.dtsi: = compatible =3D "generic-xhci"; >>> /usr/src/sys/gnu/dts/arm64/broadcom/stingray/stingray-usb.dtsi: = compatible =3D "generic-xhci"; >>> /usr/src/sys/gnu/dts/arm64/broadcom/stingray/stingray-usb.dtsi: = compatible =3D "generic-xhci"; >>> /usr/src/sys/gnu/dts/arm64/marvell/armada-37xx.dtsi: = "generic-xhci"; >>> /usr/src/sys/gnu/dts/arm64/marvell/armada-cp11x.dtsi: = "generic-xhci"; >>> /usr/src/sys/gnu/dts/arm64/marvell/armada-cp11x.dtsi: = "generic-xhci"; >>>=20 >>> That FreeBSD generic_xhci_fdt.c match is from: >>>=20 >>> static struct ofw_compat_data compat_data[] =3D { >>> {"marvell,armada-380-xhci", true}, >>> {"marvell,armada3700-xhci", true}, >>> {"marvell,armada-8k-xhci", true}, >>> {"generic-xhci", true}, >>> {NULL, false} >>> }; >>>=20 >>> (End for-reference.) >>>=20 >>> There is another driver in u-boot 2020.10, one that >>> does mention "generic-" for xhci in >>> drivers/usb/host/xhci-brcm.c : >>>=20 >>> static const struct udevice_id xhci_brcm_ids[] =3D { >>> { .compatible =3D "brcm,generic-xhci" }, >>> { } >>> }; >>>=20 >>> U_BOOT_DRIVER(usb_xhci) =3D { >>> .name =3D "xhci_brcm", >>> .id =3D UCLASS_USB, >>> .probe =3D xhci_brcm_probe, >>> .remove =3D xhci_brcm_deregister, >>> .ops =3D &xhci_usb_ops, >>> .of_match =3D xhci_brcm_ids, >>> .platdata_auto_alloc_size =3D sizeof(struct = brcm_xhci_platdata), >>> .priv_auto_alloc_size =3D sizeof(struct xhci_ctrl), >>> .flags =3D DM_FLAG_ALLOC_PRIV_DMA, >>> }; >>>=20 >>> It is a new driver as of around 6 months ago and so is likely >>> the one created for the RPi4B context. >>>=20 >>> (Note the usb_xhci vs. xhci_brcm distinct namings of >>> the driver.) >>>=20 >>>=20 >>> The .dtb files that I use for uefi/ACPI booting of FreeBSD >>> and for u-boot based booting of ubuntu 2010.04.1 also use >>> "generic-xhci", no "brcm," invovled. >>>=20 >>> It seems that .dtb files that the u-boot folks expect for >>> the RPi4B are vintages/variations that have compatible >>> listing: "brcm,generic-xhci" . >>>=20 >>> If one finds a .dtb with such, it might be close to what >>> they were testing. With such a file, finding other >>> differences with the .dtb files currently in use with >>> FreeBSD could be done. >>>=20 >>> I hope that the above helps. >>=20 >> Looks like the u-boot.bin built includes "xhci-pci" >> but excludes "brcm,generic-xhci": not in the output >> of hd /usr/local/share/u-boot/u-boot-rpi4/u-boot.bin >> after the port is installed. >>=20 >> My initial guess would be that something more needs >> to be specified to cause drivers/usb/host/xhci-brcm.c >> (and possibly more) to be built and included. >=20 > Looks like that was a bad guess, it still gets: >=20 > No working controllers found >=20 > based on instead having the "brcm,generic-xhci" > driver (with a "generic-xhci" added to the > compatibility list). Having reverted back to u-boot 2020.10 without compatibility list adjustments and with xhci-pci in place again, trying a 4 GiByte RPi4 without a microsd card boots as far as the rainbow square on the monitor but hangs there, never displaying even the: U-Boot 2020.10 (Oct 09 2020 - 06:50:04 +0000) notice. By contrast, booted via the microsd card I can stop it in u-boot 2020.10 and see things like: U-Boot> usb tree USB device tree: 1 Hub (5 Gb/s, 0mA) | U-Boot XHCI Host Controller=20 | +-2 Hub (480 Mb/s, 100mA) | USB2.0 Hub=20 | =20 +-3 Vendor specific (5 Gb/s, 72mA) | Realtek USB 10/100/1000 LAN 000000 | =20 +-4 Mass Storage (5 Gb/s, 0mA) OWC Envoy Pro mini So it gets farther with the 4 GiByte than with the 8 GiByte. But, for the "microsd card through kernel load" style of boot, I've discovered that I can use the more modern bcm2711-rpi-4-b.dtb that I use with uefi/ACPI booting for u-boot based booting as well. (This might have been true with the prior 2020.07 u-boot too. I did not go back and try it.) The implication is that the older versions of start4.elf and fixup4.dat are important for u-boot 2020.10 for some reason: having newer ones is what is different for failed microsd card based boots. The start4.elf and fixup4.dat that work with u-boot 2020.10 are too old for USB3 only based booting. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)