From owner-freebsd-arm@freebsd.org Thu Oct 8 19:28:30 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 267D242FF12 for ; Thu, 8 Oct 2020 19:28:30 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic301-22.consmr.mail.gq1.yahoo.com (sonic301-22.consmr.mail.gq1.yahoo.com [98.137.64.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C6h7x1vMVz4Q3D for ; Thu, 8 Oct 2020 19:28:28 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: k3QzSmUVM1kOy9MHBa.xyak_GpBopaC82cTFMGWYGX6O3vLXh3Qx0FozAkj0mn7 840Y5yucg9QW0EFhxriWNUt_qTWFhHzrGFgnTxrE6UvgWjyRdxWa80XU.84d1fO1UUnI69FIibKn NXmBTnRMuoFcxo9JJ.GPSPqWdcmCwqlrY09Qvc25Bb9rRKhIKQwxqs1CDigEk2xH1OP4eU75GPBP mhzjpCFX1GfHBT8WiJxAI_hdtjJfY.KHkl2r3_Ph1eA6kUYzdIeA2dVji4LeRbx7ZBps37.XO4JI _cyRsVTTuqt91GH5PSNWcX29WZsiZigjIddT5dRZCFBYlXApDD1ingK.ynL3aQJjBWLf85VFNT6Q PO4GAWgjpvh.LUPwxPzpWVOjHX5b_8eCNhPZrAq0inqDyzNIWXy6fOAql1blRZaRDQg.L3RqY5Jr PkatB4k0i9ji_8zhqLqSWDWBkvgjMmb3q3IY4jN5kg.e4EjUmn68ax1h3hyf.SshIPXkt2k4xkuh 4Qc7kFIuxKb5WhY29AVju2hI6kpZMw3cqVrHMcylN7mtyuhoWiE_6.xI1P1D8iUvtcL_6JPBUO7E KTqqrq6t6y4OL3XzlTiSMvu98GZRlZNgYQJqI7iLkYQ4VVvoYV1Iv7SdXL3VuMHd.MwYLbgb5pTt AyGGQv9nQ1a0HHM1osMQRI1I793T7CT66XHOptQdzY0Zr1Z2O097iuYOWt.ZuBux4yRHz7s.96.E 7fOTR6IoYl9qJdhIZ39iIksZpR10WYbUdSxSvfTiMicvdLkPa05nl16u1LOWcVE377fRycCrYV5z 0ZJzUADAdt.BQtbAUlEZ4DLr8SkM43QbXj2IRrth0Hacc2Qoe_ul55VXaofiRWDZwpKhFohOHEdd xgv4vGi7IWithRyYopjsx6ChOX8ikFTohSN8Gtq_xGWNLapsa6BeQM9K2A7C.lQgzp78R3S6v0D6 5q8u.ezcd.nTu8lKmSF1e_C9Xi88m058ayhVwIXFt5K.XHhp.yIXYDwbktzhhqSKLCLBSML1pZR6 Jh6iNxCQeS.FvV7enz_nTYPDNpVDXOHt3mNqzoG7tpUL59YvV5IqzEgIFo.Bh.MdMm6cVZH.Rdvt Bqii17sqwSC6l_1FN3u5NIdqXbLxQs2uZVT2oxmE66GUMxs_6E2W2SMA.BzXiSm5fSR6Tp1f1PbM k9QiZEG1QiJqj5.fIbf8tT.uWl7BH3xQFtrAERm4Ly5MapHKBtch9s72MzsUHlo852rgRBhK6Q2R yWUkEE_jmqLUy3ZsInDi0LMeTFkJ_MDa37jCeTD5JzUHuRQzdKzeSQJ.bFecWItCje7h8p44.F3q cclVGeWS59.nN3TdocMvGRK1t6jSzR9AlYxnhNuInGkrZRF4Oex7HruYCt3BA8_ZzwnoQpf8alJg B1ODNxHgQZzWjEf54hA0a8YUXYfZc9.aM3tIuYdCo9apWbtFCBaJ6_RpRirndZS0U1RiRBC6pOIz 9dccojyz9Og4umB5H7ZzFWLFIZEM5KGhK_qjbFNxagqj1dd1CMXga9hi.WgPGxcItlPljcMvR3RU - Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.gq1.yahoo.com with HTTP; Thu, 8 Oct 2020 19:28:26 +0000 Received: by smtp414.mail.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 8e9dce392d602179e56e43acb544c061; Thu, 08 Oct 2020 19:28:23 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: RPi4B: emmc2bus dma-range handling does not track the boot-time-FDT (u-boot based booting) From: Mark Millard In-Reply-To: Date: Thu, 8 Oct 2020 12:28:21 -0700 Cc: freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: <32A8A046-56DF-4998-9C3D-8630ACCE4951@yahoo.com> References: To: Kyle Evans X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Rspamd-Queue-Id: 4C6h7x1vMVz4Q3D X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.81 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.30)[-1.299]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.02)[-1.015]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.996]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.148:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.148:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Oct 2020 19:28:30 -0000 On 2020-Oct-8, at 11:34, Kyle Evans wrote: > On Thu, Oct 8, 2020 at 12:38 PM Kyle Evans wrote: >>=20 >> On Thu, Oct 8, 2020 at 11:33 AM Kyle Evans = wrote: >>>=20 >>> On Thu, Oct 8, 2020 at 4:01 AM Mark Millard via freebsd-arm >>> wrote: >>>>=20 >>>> sys/gnu/dts/arm/bcm2711.dtsi reports: >>>>=20 >>>> /* >>>> * emmc2 has different DMA constraints based on SoC = revisions. It was >>>> * moved into its own bus, so as for RPi4's firmware to = update them. >>>> * The firmware will find whether the emmc2bus alias is = defined, and if >>>> * so, it'll edit the dma-ranges property below accordingly. >>>> */ >>>> [... snip ...] >>>=20 >>> I have no words for how annoying this is. >>>=20 >>=20 >> For a slightly more helpful response: >>=20 >> We can fix this, and it ends up being much cleaner than my current >> hack. Basically, in bcm2835_vcbus.c, we should eradicate the >> busdma_lowaddr from bcm283x_memory_soc_cfg. >>=20 >> bcm283x_dmabus_peripheral_lowaddr should instead take a device_t and >> grab the bus's dma-ranges. It /looks/ to be valid on all the DTS I = see >> for the RPi boards we support, so we can just unconditionally use = that >> and things will just work for the newer RPi4 models. >>=20 >> =46rom my discussion (with an assist Ian on address interpretation) = on >> IRC, so I don't forget: >>=20 >> dma-ranges is three-value: Note: For the below I looked at 3 separate RPi4B examples, using u-boot print fdt / when I could or translating the dtb to a dts otherwise. One of the examples is from one of ubuntu 2020.04.1's RPi4B specific builds. The others I use with FreeBSD, one for u-boot and one for uefi/ACPI. ( cpu_addr is sized via the global: / { #address-cells =3D <0x2>; and the dma_addr and max_len by more local definitions, such as in: #address-cells =3D <0x1>; #size-cells =3D <0x1>; compatible =3D "simple-bus"; dma-ranges =3D <0xc0000000 0x0 0x0 0x40000000>; and: #address-cells =3D <0x2>; #size-cells =3D <0x2>; compatible =3D "simple-bus"; dma-ranges =3D <0x0 0x0 0x0 0x0 0x4 0x0>; Note that the above has #size-cells varying when: 4 <=3D number of cells in dma-range . There will be worse cases later, below. and: #address-cells =3D <0x3>; #interrupt-cells =3D <0x1>; #size-cells =3D <0x2>; . . . dma-ranges =3D <0x2000000 0x0 0x0 0x0 0x0 0x0 = 0xc0000000>; (The #address-cells being 3 indicates a bit mask as the first of the 3, the bit mask indicating extra information about the context.) and: #address-cells =3D <0x2>; #size-cells =3D <0x1>; compatible =3D "simple-bus"; dma-ranges =3D <0x0 0xc0000000 0x0 0x0 0x40000000>; and: #address-cells =3D <0x1>; #size-cells =3D <0x2>; compatible =3D "simple-bus"; dma-ranges =3D <0x0 0x0 0x0 0x4 0x0>; Note that for the last 2 examples above the number of cells in the dma-range (5) is not sufficient to indicate the value for #size-cells or #(dma-addr-cells) without presuming some other context to disambiguate. There is also an example of just: dma-ranges; (in firmware { . . . }). >> We'll see 4 and 5 value variants of this because 64-bit addresses are >> described with pairs of 32-bit values. >>=20 >> 4-value variant: dma_addr will be 32-bit, cpu_addr will be 64-bit >> 5-value variant: both are 64-bit There is an example shown above with 5-value having #size_cells being 1 (32-bit) [and dma-addr being 64 bit (cells 2)] instead of #size_cells being 2 (64-bit) [and dma_addr being 32-bit (cells 1)]. >> Note that bcm283x_dmabus_peripheral_lowaddr() will be returning >> cpu_addr + (max_len - 1) >>=20 >> This won't match perfectly with what we currently return, but it will >> be more accurate. >=20 > Here's a patch that I hacked out and can't test for quite a while yet, > feel free to give it a shot: > https://people.freebsd.org/~kevans/bcm2835_vcbus.diff -- the best > guarantee I can give you is that it builds. We'll need to test it on > both RPi4 models with the separate bus and the original RPi4s, as well > as an RPi3 and RPi2/0w. See above about trying to use the number of cells in a dma-ranges to figure out the sizes of #size-cells or #(dma-addr-cells). =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)