From owner-freebsd-arm@freebsd.org Sun Sep 27 02:01:14 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D80F63E73B5 for ; Sun, 27 Sep 2020 02:01:14 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-22.consmr.mail.gq1.yahoo.com (sonic309-22.consmr.mail.gq1.yahoo.com [98.137.65.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BzTQd6rTRz4LvT for ; Sun, 27 Sep 2020 02:01:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: 43XJ7VIVM1lq0WjhOGAjOqx3fqacAwcI_dpRNzqGQzIXcixqaxB.DF0DR7fQGdm vKEIIWFb.Eqv0zFPnCaslopwrGYt767MIHUGPthmrvOAk2B9vAtbajGrRb1RIxUe22utSXJNBV1f PQAt5uNGQczTdsix765F7dFWe5yHoDwCPTaocfZvDn5TOoxPhmJjMox0q9yLgouwDtXaoObKRC_7 bdnpbZWz2GM4b77GN2z1CbmS2Td_vW3e5UG9XI5QwqWIbEMF0vuHsxEVUbHFTsLGkZ0LcqfzeVds aKjNYSUmbcoADcM0AbQiF7QqhaojAdNhJsrpfSZ8Pu.3EN_rVR1wwSpDvKD29eW_bMQaoMHdZqDM B_0EdwmaooY0St.FqZb.mom3cjRJrkwZAXDCA2H_qASEnj4itc2lSU8C0DmjH5U3fmpgHanxm.en w1.eFE8nKanExUbRaL1T1UgqqLdPKCL37kbmqlAsn8AaNibVuPwg2JK_chwXGgqJDybzxb28Am8R SV.goiltx1_YVer6_sefetLtYsvqPY318QPleFd1nLiz4lRp5DKWHE2WZznME.L7QUXF.TCKIjie eBqE8FSV6PQDGHuIfsQzUu3hQHzKQgDjE1HxvxWZjqCWKzsTs63HLlvW9TP24DWT3tFVp4d36Y3g beKzt0I8K2I5Le3I7iF4py4Morffor67ADljnboyo036JtVlppD5lruFr4v3IPdqps23zl5kzYqC ege3lGlXzO.r0wCwntTEd..WQswZMvT3TLFMMAlbwIu.yA.K8wCcyHmbAx30y6qfHixKVKsySdaj grWWhvHUxwNSKiZDeU.JXmd531qKFvu7PzBbnLlZTVMlh6Sc.lLzxxbef93h01H2Zz5z7tPkw.r7 9AaVJgpH3GvNoHhYc5RVtCHTXs.OwkpWIDeM9Kn_4Sk0NCWR6lBruZ.goujFSMEPn60kuSOAodtH I.052K531vDCwHJJn1v8zq.PFkplTZjN77NUdyie.bF143IKdKevuuovEGTCO89sBf8qAkhwEpZx K4H3Kd2i54KZP7hfJskG58vr0YksM36VbzV4nxuneXNpxAIwAyX_1B32umSmDJ1YWunpzn9SyhQI vif3mMCV7SMG3erruUJHXuowKvgecLLz0L5nj1q5xXZXqjZ99ZNqG1TNOVCrN7HLwPmajuQ7IP47 zkFfQ7f2oQrRSTyNPsbm_zhww5QgN7vFNdFvfdsVZNP00NsGCcVO7t955VfoBaAnVc5xIG.caVaA CQL1jQ01KcNfV6GuEtRjVjJ0eA7uO_0L4UtXpZ5UIo409rVF_NaJ2Pgg97obzaRnJAe2sGK27Mle Ul_QtkgZTMC.ttC7wzPO0qqEQxezNu1cjqvDtr0i6YKvzpMh9diw5DR9cwzf9J8iCKNZRnTxDuyx OX.f7yzMZECbCDFEhidsbLmJVM7.vjAi8.hp0d_Tg32VrTNo5H2sYhd2Ocp1ilp0l2VEMsmzT7fR _d.cl4PZiBoa.donuQnfDccqWD4JeUf_fVtyZi7fSscKu9J5vPfuY.BNIvALX3hNSO251HW7X Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Sun, 27 Sep 2020 02:01:12 +0000 Received: by smtp404.mail.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 3202a82a4abbe471097f6c1431241496; Sun, 27 Sep 2020 02:01:08 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: RPi4B u-boot based booting and hw.cpufreq.voltage_core and dev.cpu.0.freq use: able to use 2000 MHz From: Mark Millard In-Reply-To: <64B39936-7689-4240-A5F9-4DF5EAE4DE42@yahoo.com> Date: Sat, 26 Sep 2020 19:01:06 -0700 Cc: freebsd-arm , Robert Clausecker Content-Transfer-Encoding: quoted-printable Message-Id: <19D27F91-F038-4393-90BF-7A439F8BF4B1@yahoo.com> References: <0578EC2B-D21C-46AA-AD3E-CD13985B18FA.ref@yahoo.com> <0578EC2B-D21C-46AA-AD3E-CD13985B18FA@yahoo.com> <20200926133934.GD54660@bastion.zyxst.net> <64B39936-7689-4240-A5F9-4DF5EAE4DE42@yahoo.com> To: tech-lists X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Rspamd-Queue-Id: 4BzTQd6rTRz4LvT X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.29 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.79)[-0.788]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.987]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.02)[-1.016]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.148:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.148:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Sep 2020 02:01:14 -0000 On 2020-Sep-26, at 13:20, Mark Millard wrote: > On 2020-Sep-26, at 06:39, tech-lists wrote: >=20 >> On Sat, Sep 26, 2020 at 12:37:45AM -0700, Mark Millard via = freebsd-arm wrote: >>> I got access to a 4 GiByte RPi4B that does not have modernized >>> eeprom contents. With it I was able to do a u-boot based boot >>> of head -r365932 based on the msdosfs on a older microsd card >>> that had materials from 2020-Jul-13 (u-boot.bin) and 14 (RPi4B >>> materials). I updated EFI/BOOT/bootaa64.efi . The u-boot.bin >>> is from my build of sysutils/u-boot-rpi4/ (no local changes). >>>=20 >>> In this context . . . >>>=20 >>> I added over_voltage=3D6 and arm_freq=3D2000 to config.txt and it >>> ends up looking like: >>>=20 >>> arm_control=3D0x200 >>> arm_64bit=3D1 >>> dtoverlay=3Ddisable-bt >>> dtoverlay=3Dmmc >>> device_tree_address=3D0x4000 >>> kernel=3Du-boot.bin >>> armstub=3Darmstub8-gic.bin >>> over_voltage=3D6 >>> arm_freq=3D2000 >>>=20 >>>=20 >>> Booting based on that resulted in: >>>=20 >>> dev.bcm2835_cpufreq.0.freq_settings: 2000/-1 600/-1 >>> dev.cpu.0.freq_levels: 2000/-1 600/-1 >>> dev.cpu.0.freq: 600 >>>=20 >>> And: >>>=20 >>> # sysctl dev.cpu.0.freq=3D2000 >>> dev.cpu.0.freq: 600 -> 2000 >>>=20 >>> worked. (Without the over_voltage it reports 600 and then >>> hangs.) Having /etc/sysctl.conf list dev.cpu.0.freq=3D2000 >>> works. >>=20 >> Exellent! On the basis of your post I went ahead and removed from = config.txt >> three lines, then added two and rebooted. >>=20 >> It's now this: >>=20 >> arm_control=3D0x200 >> arm_64bit=3D1 >> dtoverlay=3Ddisable-bt >> dtoverlay=3Dmmc >> device_tree_address=3D0x4000 >> kernel=3Du-boot.bin >> armstub=3Darmstub8-gic.bin >> over_voltage=3D6 >> arm_freq=3D2000 >>=20 >> before, when it didn't work, it had this: >> [same as above], apart from >> gpu_mem=3D16 >> over_voltage=3D6 >> arm_freq=3D2100 >>=20 >> In creating the freebsd instance I followed these instructions for = 8GB >> = https://lists.freebsd.org/pipermail/freebsd-arm/2020-August/022162.html >> and applied D26344. /etc/rc.conf has powerd loaded >>=20 >> I was particularly interested in getting this clocked to a reasonable = speed as >> it's used to build packages with poudriere based on arm7 and aarch64 = for >> -current. >=20 > The cortex-A72 is largely memory subsystem limited on the > RPi4B compared to, say, the MACCHIATObin Double Shot > running at the same clock rate. >=20 > So I've recently experimented with changing the > sdram_freq_min from the default or 400 to 3200 > (to match sdram_freq): >=20 > # more /microsd_efi/config.txt > arm_control=3D0x200 > arm_64bit=3D1 > dtoverlay=3Ddisable-bt > dtoverlay=3Dmmc > device_tree_address=3D0x4000 > kernel=3Du-boot.bin > armstub=3Darmstub8-gic.bin > over_voltage=3D6 > arm_freq=3D2000 > sdram_freq_min=3D3200 >=20 > A benchmark I use suggests I'll see the fastest RPi4B > buildworld buildkernel times that I've seen yet when I=20 > get around to trying that. ( My benchmarking and testing > is based on powerd or the like not being in use but the > kernel or rpi-firmware could be doing its own thing > given the above and sysctl.conf having > dev.cpu.0.freq=3D2000 .) Preiminary benchmarking indicates that rpi4-uefi-devel's uefi/ACPI context effectively ends up with sdram_freq_min=3D3200 ignored (making no difference). (In my context, it is the same FreeBSD root file system for both types of boots, so only boot-specific code varies.) Thus, it looks like the u-boot context can be better tailored for cutting the buildworld buildkernel time required and so it looks like that is the environment that I will be timing a from-scratch build in. The ACPI context does not have a sysctl interface to the values that the u-boot one has. So u-boot based is more direct for validating what is in use. > I have heatsinks and a fan involved, as well as a > (apparently good) 5.1v 3.5A power supply. My experiments > presume such a context. >=20 > (I've sent out a separate list message about hw.cpufreq.sdram_freq > having a value-display problem and the 3 hw.cpufreq.voltage_sdram_* > values looking to be lower than expected. I've not seen evidence > of problems in operation so far.) >=20 > I'll note that: >=20 > = https://www.raspberrypi.org/documentation/configuration/config-txt/boot.md= >=20 > reports: >=20 > QUOTE > arm_control >=20 > DEPRECATED - use arm_64bit to enable 64-bit kernels. >=20 > Sets board-specific control bits. > END QUOTE >=20 > So I'm likely to experiment with removing that line > from config.txt . >=20 >=20 >> seems very stable so far: >>=20 >> 2:36p.m. up 45 mins, 1 user, load averages: 6.51, 3.57, 2.92 >> Sat 26 Sep 2020 14:36:14 BST >> dev.cpu.0.temperature: 65.0C >> dev.cpu.0.freq_levels: 2000/-1 600/-1 >> dev.cpu.0.freq: 2000 >>=20 >> rpi4 is in a FLIRC case for cooling, no fan. 24 degC/44%rh ambient. >>=20 >=20 >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Sun Sep 27 02:53:48 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 17F9F3E9539 for ; Sun, 27 Sep 2020 02:53:48 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-25.consmr.mail.gq1.yahoo.com (sonic312-25.consmr.mail.gq1.yahoo.com [98.137.69.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BzVbG65DSz4PXF for ; Sun, 27 Sep 2020 02:53:46 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: XDp1dE0VM1lWdKlJkGR9aznh69X5qY1uncmyRIKebHvhz3zCQqwFiEpbkyNDYHG MGjE9bDGOipwO73O.QgqBdHcAtaRe.l8cWHa7aqhIoJAAhgYfuxjWvAHtW2b.gRjH2ZWJU5Ls0Ef t4mHK1EIYPsygVPHSWc3XCVnhxC.qWk0c_4hwKgl5X._Lha5ws6regRfvENAXiXj.ZSajO0lKtOc v983fK8zh7uODUxq_iL3WsfV0D5HvSiR4osYXUjSbvDGla.lVeqi7ud65zPiYcKSjIH3dK5Oq_Ha fhT4JPlJLPa_4Dh.q3AYsgUnrmwwV6u46sWvxUDbXs2cD3mDPdvwc26aWcoY9N3R4.gdlLQT138L orZ4wbaSlyX5EHIvyQ9KCpSk2YZUITDGBGrJkAJJ8gpvCAz0HWZvocuBlHVrQQN7AiwG6iOfjjEw ShfY0Ek3vyTfqaIa_6AffhmPaTSVbYjcGwERNEEixdOqvYA9HE54hUW9258AWRAwYDvK_QHCeJuf qPmeaUjClu389_rgQNMX4LkTjehtGvctAXyh_jzwiIWc3BcdD247n9kDKxy4VcMUobLBFgck0mwY e8g.dvZk5l_SVRVtBUaccYvNPmWHt_XVg5nx1uAPQMk8VJkKihtQgZejHsUxCzppo3gaFY0VKf8t 6Hg3v2u.M5Q3Wh7ppTyCXhUdFJKR.zOvRAS.wy2p39oB4x14_W3h.7QqUoOuYJgyDokcT1V4EOxM Bom3oh0fK2tOBMgTX1fxXV1cnMzMFG0ThVn3KgXXOmLxJ_GhzhC0Xq7gdMZGJYqzOGm8HYphV_er lYCf6D3nAsrIF7Iuxaa2VJMxeiOq4UXms9GWdGKnyvHvJC0uDeTByY77QVStdqfX4RpWBrciednR UMiyj5Y7G9JX7zWOx8bMe3XLhpLxubFl4qJ.quw0EUyyGTAz6yFK7ReBUaXof83Brq4wEaj3W1Gl oxM3D1KKHkWi6T3UGPWO.l0FweLLmHjuuofqvAA6cVSEGlYkZcAXqDqCX.nXl7XIiLLhzGtmqBgN VgyBTA00DxMVB_gqvz4S580c0j9tDTPlGXDnZK6_osBV_gwbeqhUsGevV3r4Y9gaadNZRmWfyXhO zuseo8dTJ5gKvotSGvwamcUWUZTTYAn_XQc9WkwQIG46gGU.jxcIdEyDkXe.U_I1GoNVxaMadGo6 99VePni9yANk0JQIhyZb9mNjyRtZGRYyxYetD6VXa.HZYcwcUiBc.LGexdq573aThon8gJvIcQ7I m.co5BgSk_UUFlPmx5tepCyf9T.56CwrHNCJbEc6vA0I0AksVKyi5kN8nVAITWS9FkdidezlgtVD vYI4qXPWR5sn3UmbllRiuhd7FAUcIS2JsNJCVr3n3qDoRHRbsFyJ2jpTWB74F1MRQAXnBZpBITJP .pUUU1JGJbj0Uc_UScukAbrFc3IGNr3gpv6iSD0y0CSCIkmk9wC3.vR63QADHAXgZ_xrpUb4BuWs 4CpSNoUhNbjUvRDVES1ACQwzQLfCgudVXpjv1SgZzp5I3Jgo2yfwuVM9Ea.Z37CCcMwWS.XeebNf kYZ_5DOBMI7UnaX6kBNpOXYkxQRxo5G0- Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Sun, 27 Sep 2020 02:53:44 +0000 Received: by smtp416.mail.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID ab8e29f782089e4600df50d2a77d858d; Sun, 27 Sep 2020 02:53:40 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: head -r365677 and later do not have the xhci related DMA problem fixed From: Mark Millard In-Reply-To: <8084F606-D75B-42B3-A05F-724F7150F304@yahoo.com> Date: Sat, 26 Sep 2020 19:53:39 -0700 Cc: freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: References: <5A60B29E-0D24-480C-807D-4A5E92D9C92A.ref@yahoo.com> <5A60B29E-0D24-480C-807D-4A5E92D9C92A@yahoo.com> <231B8A1B-7F61-4868-B9E6-F8DD824079CA@yahoo.com> <39346C6E-CF29-42E8-BCAB-B04E73F909F7@googlemail.com> <04531BA7-F7A6-498B-BB8E-D3AAA53E15E3@yahoo.com> <20200925063418.GC54660@bastion.zyxst.net> <8084F606-D75B-42B3-A05F-724F7150F304@yahoo.com> To: tech-lists X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Rspamd-Queue-Id: 4BzVbG65DSz4PXF X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.26 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.75)[-0.750]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.988]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.02)[-1.018]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.206:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.206:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Sep 2020 02:53:48 -0000 On 2020-Sep-24, at 23:34, tech-lists wrote: > On Thu, Sep 24, 2020 at 05:53:06PM -0700, Mark Millard via freebsd-arm = wrote: >=20 >> So far I'm unable to figure out how to have a u-boot environment >> that boots the RPi4B's: figuring out what needs to be different >> than the rather modern raspberry pi files that I have in place. >> Also: fully modern eeprom content. As stands I've not been using >> a microsd card at all: it uses the msdos file system from the >> USB3 SSD. >=20 > Mine *will* boot, even after the eeprom upgrade. What it won't do = (what was > always the case) is boot after I modify config.txt in any way at all. . . . Mine too, for how I did things. My previous notes about not being able to boot when modern eeprom content is present and modern rpi firmware is also present on the USB3 SSD did not cover having the indirection of pulling the FreeBSD kernel from the microsd card and having the FreeBSD kernel be what first gets to the USB3 SSD. But what I actually did eventually was have FreeBSD's kernel be the first thing that gets to the USB3 SSD. (Requires that the microsd card with rpi-firmware and u-boot on it has priority when present.) The USB-MSD aspect of the various rpi updates is not required at all for what I did. This also avoids u-boot needing to be able to deal with the xHCI or drives that involve xhCI use generally --but requires the use of a microsd card. (Not that I think u-boot requiring old, beta rpi-firmware is good as a long-term context.) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Sun Sep 27 14:33:03 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 04EE33FD069 for ; Sun, 27 Sep 2020 14:33:03 +0000 (UTC) (envelope-from mw@semihalf.com) Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bzp655CZ3z3ZXG for ; Sun, 27 Sep 2020 14:33:01 +0000 (UTC) (envelope-from mw@semihalf.com) Received: by mail-qk1-x744.google.com with SMTP id w12so7796574qki.6 for ; Sun, 27 Sep 2020 07:33:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to:cc :content-transfer-encoding; bh=MFAf3Hao1Ezm6tfhF1keFgiqXxcIVorvZeKcZ9JgP0I=; b=gmnmklECxuydjXgs5hKbzlx+rYuJx+HKoeQTl7Ra9rg3rx5OnY4m9rKltEZdk1vBD0 jIsJfDB6IrvwZApL5lnoYXDXZg8z/6cGQhcYq+EERd00b4+7gT0JzxOJ0y06oWaiwT5C VydGH33O483xkP3RyKVlMXiwaHOOtDrKoHmw/sTo8Rb7AyUL/0edCoqUfULrozN2/ICP 5JqfHm8wLvR3Be1GeShxpgdIytabjK9ZdE6l1gnkxZ2QhcoINEBEYF41gvIB32ua4fQb O95jM4whtahbM1wDBKY4BA9ganf1SibZfzOsGstEBIdTjDYKooCRXqbsn416Rz5HhfqU Tq1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc :content-transfer-encoding; bh=MFAf3Hao1Ezm6tfhF1keFgiqXxcIVorvZeKcZ9JgP0I=; b=c9c28uui6YIrnVIB5i+/4iT5K4SKf6//f5WdGJlXrsJkb2XcYZ24yHeayt/NLs/VBS gIiVciIHQWx3ie8gcqmU8mmQG6LOw5Dc5THUJCT4hD67ulX6speGrTcQUCDLtJBbnQFL wjyCHOnbfHI/bVRDzpN7EcRzdGzHTALGltf1p6+WA3qgK9fAN6tc+YWW4YP7xYoftXsJ +BlWSbIIBHvb+g+r70J/h7itwMAShKBdir7Pz4OLxfWdGElwSeLaH3BfsU95shmBckEZ 5hspx3JKyUc42q3k0K7Nbut9VhVFrgQvBSVrVcv6z2dwblMZgmHjrR4HWN3+e1/KSOCb oVEw== X-Gm-Message-State: AOAM531hSsXhkTSQx/smKnnwrgMe1YHDOLDJJZwrO1+NmR/oGpo5BWyo Ohr/mRl0SvgCd5XUM78wxx3x7nbJ/PrwTHgjA1p/615YeuoaB9R2 X-Google-Smtp-Source: ABdhPJxGutBhVFVniBrhq3opL/RFXhfqiI+Cxz/NjIC4pAyTX9beBySNmTmUYd1vRR99s/EK83oExvxPSbt8MgcoYLc= X-Received: by 2002:a05:620a:244e:: with SMTP id h14mr8399488qkn.348.1601217180530; Sun, 27 Sep 2020 07:33:00 -0700 (PDT) MIME-Version: 1.0 From: Marcin Wojtas Date: Sun, 27 Sep 2020 16:32:48 +0200 Message-ID: Subject: HEADS UP: NXP LS1046A SoC support in the tree To: freebsd-arm Cc: Rafal Jaworowski , MULLER Laurent , FONTAINE David Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4Bzp655CZ3z3ZXG X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=semihalf-com.20150623.gappssmtp.com header.s=20150623 header.b=gmnmklEC; dmarc=none; spf=none (mx1.freebsd.org: domain of mw@semihalf.com has no SPF policy when checking 2607:f8b0:4864:20::744) smtp.mailfrom=mw@semihalf.com X-Spamd-Result: default: False [-2.16 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[semihalf-com.20150623.gappssmtp.com:s=20150623]; FREEFALL_USER(0.00)[mw]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.005]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; DMARC_NA(0.00)[semihalf.com]; NEURAL_SPAM_SHORT(0.14)[0.145]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[semihalf-com.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::744:from]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Sep 2020 14:33:03 -0000 Hello all, On behalf of the Semihalf team I'd like to announce that starting from r365054 FreeBSD is ready to run NXP LS1046A system-on-a-chip with most of the interfaces supported. LS1046A are quad-core 64-bit ARMv8 Cortex-A72 processors with integrated packet processing acceleration and high speed peripherals including 10 Gb Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 for a wide range of networking, storage, security and industrial applications. It is confirmed that at least some parts of the kernel support (such as SDHCI) was successfully used on the Solid-Run LX2K Honeycomb development platform. Below is the port status together with the features, which were originally developed on top of the release/11.2.0 that have made their way to FreeBSD-HEAD: * LS1046A core support (Quad-CA72 SMP, timers, IRQS, UART) - already working in vanilla FreeBSD-HEAD before * DWC3 USB3.0 host controller driver (FreeBSD-HEAD version developed by manu@) * VF610 I2C controller (adjusted to work with the LS1046A) * TI LM75 temperature sensor (adjusted to work with the LS1046A) * QorIQ platform clockgen * LS1046A CPU clockgen * LS1046A GPIO * RX8803 I2C RTC * TCA6416 I2C GPIO expander * LS1046A AHCI * LS1046A SDHCI The major out-of-tree components are that albeit working on top of release/11.2.0, still require significant effort to adopt to FreeBSD-HEAD: * DPAA network controller support * QSPI controller support This was a joined effort of Semihalf and Alstom Group (main development sponsor), in particular: Laurent Muller David Fontaine Yannis Planus Artur Rojek Dawid G=C3=B3recki Kornel Dul=C4=99ba Kamil Koczurek Micha=C5=82 Stanek Jacek Klimkowicz Best regards, Marcin From owner-freebsd-arm@freebsd.org Sun Sep 27 18:07:57 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1D49642330F for ; Sun, 27 Sep 2020 18:07:57 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-54.consmr.mail.gq1.yahoo.com (sonic316-54.consmr.mail.gq1.yahoo.com [98.137.69.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bztt41RjFz43Td for ; Sun, 27 Sep 2020 18:07:55 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: .EaI3z0VM1ntd_6gzHI1.tXLeDEtMl8quCDW5zxJgZ2kk5r3FnrO.KiWlcxPYgt PMHF7bmVzK225lmHqrO5q_XlwweHKuPteXrz.MO27lj4aDGmYcFzbTQkEe3EFItxEkUahXt_QJNl 8Uv2kOmmj6ne6NUSVmZhJ7rGOs_3girvFQUvRyNqRqGumuqH8jDyWODPL5reDSIExq1hiSx5ppYn oGuJXzSMoHBCHyyKvWci1ZUUdWt9fYFf7wsCi4q3Ei9bJW2NLwCKbvF2YrWgyZoq3JkOdLy89WFZ r62ySWVq0HzRAOIH2.jWaVJQq8zg_YcMHYdxIm8fa5pOOEc685bYdl1CekAKjy_k2hiS53.Rakw7 U99NvZzMzDDqJt3A._SxP0cBMwanyfCZQ5K.fe5lozrnaH1wgjEv9J_iHtHLgsNfFqqMmhNzrmSA BpSiQTeYEyBWkmor.dO8RUD4QbaFyjVyN1cIIgaLv39m2tc1ZHRhxUTtXfWQOwTGvxV7Onerqtl_ QG6khn_Y6Hk_EVq0xbwZ25nnf1twfzz7R5La1ICAJ4BE4f1Nn0LLPw2qGQW2JUheyrQtC3kVtkMP on.UXWbHgcb1PLQLXdeEUu9M1noVxtNdm3c0jXciUvilB6yuN9rNGQJO6nsQ7SPs.Mpy.83ghq28 25eZAReVPHffrMscWb9MtXbLYhJ3n65CmC7CsHC2G92XDprBohsYrpRJb2qtmFwSfq7xZSRjwWSb 1NLok5QLwpBhZh2KE5sU_gk1E57q7JnbThnGFSRG5zUGas0ZwqjQbPFtQd5c7ZzgxplpEXXmXhS1 pB5yUz5sPHDTHUTAsqsW9IJd69UFA9ZB8HUCzih06U.88n7jnEiV5GKZK7kE9IpPEStBVlnf0oZS VkRlvnvCBo5oGBGUOsg5JXkneQDSRW_sHWWbx6JMd1NAF0dhjIaARqs0zUvweNb1ldjoxSgB8baP vbC6mjdrN0BAmetDTH_61ydnnPEeyqQZpR52sesaIVTkJ7xcfE0dSuiBpl6S1Mgry1GrT3FDFhS7 JfMRbxUWN7kNF_58WiTSwZlNzjGJO32XZ75X3ldo2RquUnbDvBnsve8tXFXBYsZy9bH6YFAq.pU_ uI9zBw51aVXpRGwQL.D9GC18N07u285Qwuno6X.HH8.YprWnDZPpqh3_8bBJ9ltFzJLeeGrei8p5 z8CMnhcCWQM_YqHpOiSP2OOVoTlCaYEAr2uMV7Wg90yx2QV8wz.n_vporoG3gxY_FHBlwBBfOp85 sgHK7cIujQnwqjD.yO2niJj0snthaazEzmiX0FKi_28uJkSZ33i.W6E9GNdFL4ZhljTORx.S4doh VfCy9yVTbHCgTuWXP4yEhia7Ce3pPiWaH9Jnuxv7_vybFbFqU5hYY05fSD7XvozWlUDEE3SL3xe8 T8y5QxWqu8.hQgR2McFTBukboGqwzmzJBrVljsiZfUBFt1oF8Yj41_ZDb6JfuF6LuwBo6uRhQnmG Eg1alhu.O6JlAhzjX5PJD.GE9_OhEZqiM1T2gf6TwYzRNo9yYhtExRPTi Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Sun, 27 Sep 2020 18:07:54 +0000 Received: by smtp404.mail.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 1ba1b907f7e1b77d4bf4a358406fd02d; Sun, 27 Sep 2020 18:07:52 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: RPi4B buildworld buildkernel times for already installed system being -mcpu=cortex-a72 vs. -mcpu=cortex-a53 based Date: Sun, 27 Sep 2020 11:07:51 -0700 References: <4E155E94-3AA0-464D-A1E9-45A7827537ED@yahoo.com> To: freebsd-arm In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Rspamd-Queue-Id: 4Bztt41RjFz43Td X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.03 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.51)[-0.510]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.991]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.02)[-1.025]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.30:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.30:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Sep 2020 18:07:57 -0000 On 2020-Sep-20, at 18:40, Mark Millard wrote: > On 2020-Sep-20, at 18:32, Mark Millard wrote: >=20 >> The following are from scratch buildworld buildkernel rebuilds >> on a RPi4B (head -r363590 context). >>=20 >> ENVIRONMENT: -mcpu=3Dcortex-a72 based world and kernel running = already, RPi4B @ 2G Hz, >> Restricted to 3 GiByte RAM, -j3: >>=20 >> World built in 37469 seconds, ncpu: 4, make -j3 >> Kernel(s) GENERIC-NODBG built in 2474 seconds, ncpu: 4, make -j3 >>=20 >> ENVIRONMENT: -mcpu=3Dcortex-a53 based kernel running, RPi4B @ 2G Hz, >> Restricted to 3 GiByte RAM, -j3: >>=20 >> World built in 44034 seconds, ncpu: 4, make -j3 >> Kernel(s) GENERIC-NODBG built in 2895 seconds, ncpu: 4, make -j3 >>=20 >> So a little under 11.1 hr total vs. a little over 13.0 hr total, >> a somewhat over 50 min improvement. >=20 > "a somewhat over 1hr 50 min improvement" is what I should have > managed to type. >=20 >> (A xhci patch finally allowed me to boot -mcpu=3Dcortex-a72 >> based kernel builds on the RPi4B: The xhci event ring >> initialization code was missing a usb_bus_mem_flush_all >> call previously.) >>=20 >>=20 >> Supporting details: >>=20 >> (e-mail based spacing changes expected below) >>=20 >> # diff -u = ~/src.configs/src.conf.cortexA72-clang-bootstrap.aarch64-host = ~/src.configs/src.conf.cortexA53-clang-bootstrap.aarch64-host >> --- /root/src.configs/src.conf.cortexA72-clang-bootstrap.aarch64-host = 2020-03-13 22:29:25.470155000 -0700 >> +++ /root/src.configs/src.conf.cortexA53-clang-bootstrap.aarch64-host = 2020-03-13 22:29:25.469455000 -0700 >> @@ -49,9 +49,9 @@ >> # Use of the .clang 's here avoids >> # interfering with other CFLAGS >> # usage, such as ?=3D usage. >> -CFLAGS.clang+=3D -mcpu=3Dcortex-a72 >> -CXXFLAGS.clang+=3D -mcpu=3Dcortex-a72 >> -CPPFLAGS.clang+=3D -mcpu=3Dcortex-a72 >> -ACFLAGS.arm64cpuid.S+=3D -mcpu=3Dcortex-a72+crypto >> -ACFLAGS.aesv8-armx.S+=3D -mcpu=3Dcortex-a72+crypto >> -ACFLAGS.ghashv8-armx.S+=3D -mcpu=3Dcortex-a72+crypto >> +CFLAGS.clang+=3D -mcpu=3Dcortex-a53 >> +CXXFLAGS.clang+=3D -mcpu=3Dcortex-a53 >> +CPPFLAGS.clang+=3D -mcpu=3Dcortex-a53 >> +ACFLAGS.arm64cpuid.S+=3D -mcpu=3Dcortex-a53+crypto >> +ACFLAGS.aesv8-armx.S+=3D -mcpu=3Dcortex-a53+crypto >> +ACFLAGS.ghashv8-armx.S+=3D -mcpu=3Dcortex-a53+crypto >>=20 >>=20 >> The .amd64-host files are similar for doing cross builds. >>=20 >> I also use +=3D in secure/lib/libcrypto/Makefile : >>=20 >> # svnlite diff /usr/src/secure/lib/libcrypto/Makefile >> Index: /usr/src/secure/lib/libcrypto/Makefile >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- /usr/src/secure/lib/libcrypto/Makefile (revision 365919) >> +++ /usr/src/secure/lib/libcrypto/Makefile (working copy) >> @@ -20,7 +20,7 @@ >> SRCS+=3D o_str.c o_time.c threads_pthread.c uid.c >> .if defined(ASM_aarch64) >> SRCS+=3D arm64cpuid.S armcap.c >> -ACFLAGS.arm64cpuid.S=3D -march=3Darmv8-a+crypto >> +ACFLAGS.arm64cpuid.S+=3D -march=3Darmv8-a+crypto >> .elif defined(ASM_amd64) >> SRCS+=3D x86_64cpuid.S >> .elif defined(ASM_arm) >> @@ -35,7 +35,7 @@ >> SRCS+=3D aes_cbc.c aes_cfb.c aes_ecb.c aes_ige.c aes_misc.c = aes_ofb.c aes_wrap.c >> .if defined(ASM_aarch64) >> SRCS+=3D aes_core.c aesv8-armx.S vpaes-armv8.S >> -ACFLAGS.aesv8-armx.S=3D -march=3Darmv8-a+crypto >> +ACFLAGS.aesv8-armx.S+=3D -march=3Darmv8-a+crypto >> .elif defined(ASM_amd64) >> SRCS+=3D aes_core.c aesni-mb-x86_64.S aesni-sha1-x86_64.S = aesni-sha256-x86_64.S >> SRCS+=3D aesni-x86_64.S vpaes-x86_64.S >> @@ -242,7 +242,7 @@ >> SRCS+=3D ofb128.c wrap128.c xts128.c >> .if defined(ASM_aarch64) >> SRCS+=3D ghashv8-armx.S >> -ACFLAGS.ghashv8-armx.S=3D -march=3Darmv8-a+crypto >> +ACFLAGS.ghashv8-armx.S+=3D -march=3Darmv8-a+crypto >> .elif defined(ASM_amd64) >> SRCS+=3D aesni-gcm-x86_64.S ghash-x86_64.S >> .elif defined(ASM_arm) >>=20 >> The RPi4B is using: >>=20 >> over_voltage=3D6 >> arm_freq=3D2000 >>=20 >> and was booted via uefi/ACPI. >>=20 >> I have not repeated the -j4 or other -jN comparisons that >> I reported in the past. The -mcpu=3Dcortex-a53 figures are >> from the past. The following new timing is based on head -r365932 rebuilding itself where the 8 GiByte RPi4B config.txt ended with: over_voltage=3D6 arm_freq=3D2000 sdram_freq_min=3D3200 and the boot was via u-boot, no RAM restriction. (The sdram_freq_min assignment does not seem to do anything for rpi4-uefi-devel v1.20 uefi/ACPI based booting.) /etc/sysctl.conf has: dev.cpu.0.freq=3D2000 . No use of powerd or other such. ENVIRONMENT: -mcpu=3Dcortex-a72 based world and kernel running already, 8 GiBYte RPi4B @ 2G Hz with sdram_freq_min=3D3200, u-boot style boot, = -j3: World built in 31852 seconds, ncpu: 4, make -j3 Kernel(s) GENERIC-NODBG built in 2059 seconds, ncpu: 4, make -j3 So somewhat under 9.5 hr overall. That means somewhat over 3.5 hours faster than a -mcpu=3Dcortex-a53 based system without sdram_freq_min=3D3200 using 3 GiByte RAM but still RPi4B @ 2G Hz (uefi/ACPI boot): World built in 44034 seconds, ncpu: 4, make -j3 Kernel(s) GENERIC-NODBG built in 2895 seconds, ncpu: 4, make -j3 (Same as reported in prior messages.) But the prior -r362590 vs. the now -r363932 means there is more varying than in my previous comparisons. For example, clang 10 vs. clang 11. I'm probably going to run a -j4 build to see how it compares in this context. I've not run a default arm-freq/sdram_freq_min/dev.cpu.0.freq buildworld buildkernel in a long time and so do not have reasonable comparison figures relative to that type of context. I do not plan on such an experiment. I'll note that I run these tests with a monitor connected that sits with a static login prompt display after booting. I do not not test with X11 or other use that might significantly compete for more power. The serial port console is usually used. I have used ssh sometimes in the past. ~/src.configs/src.conf.cortexA72-clang-bootstrap.aarch64-host is still unchanged: # more ~/src.configs/src.conf.cortexA72-clang-bootstrap.aarch64-host=20 TO_TYPE=3Daarch64 # KERNCONF=3DGENERIC-NODBG TARGET=3Darm64 .if ${.MAKE.LEVEL} =3D=3D 0 TARGET_ARCH=3D${TO_TYPE} .export TARGET_ARCH .endif # #WITH_CROSS_COMPILER=3D WITH_SYSTEM_COMPILER=3D WITH_SYSTEM_LINKER=3D # WITH_LIBCPLUSPLUS=3D #WITH_LLD_BOOTSTRAP=3D WITHOUT_BINUTILS_BOOTSTRAP=3D WITH_ELFTOOLCHAIN_BOOTSTRAP=3D #Disables avoiding bootstrap: WITHOUT_LLVM_TARGET_ALL=3D WITH_LLVM_TARGET_AARCH64=3D WITH_LLVM_TARGET_ARM=3D WITHOUT_LLVM_TARGET_MIPS=3D WITHOUT_LLVM_TARGET_POWERPC=3D WITHOUT_LLVM_TARGET_RISCV=3D WITHOUT_LLVM_TARGET_X86=3D #WITH_CLANG_BOOTSTRAP=3D WITH_CLANG=3D WITH_CLANG_IS_CC=3D WITH_CLANG_FULL=3D WITH_CLANG_EXTRAS=3D WITH_LLD=3D WITH_LLD_IS_LD=3D WITHOUT_BINUTILS=3D WITH_LLDB=3D # WITH_BOOT=3D WITHOUT_LIB32=3D # # NO_WERROR=3D #WERROR=3D MALLOC_PRODUCTION=3D # # Avoid stripping but do not control host -g status as well: DEBUG_FLAGS+=3D # WITH_REPRODUCIBLE_BUILD=3D WITH_DEBUG_FILES=3D # # Use of the .clang 's here avoids # interfering with other CFLAGS # usage, such as ?=3D usage. CFLAGS.clang+=3D -mcpu=3Dcortex-a72 CXXFLAGS.clang+=3D -mcpu=3Dcortex-a72 CPPFLAGS.clang+=3D -mcpu=3Dcortex-a72 ACFLAGS.arm64cpuid.S+=3D -mcpu=3Dcortex-a72+crypto ACFLAGS.aesv8-armx.S+=3D -mcpu=3Dcortex-a72+crypto ACFLAGS.ghashv8-armx.S+=3D -mcpu=3Dcortex-a72+crypto =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Sun Sep 27 21:06:46 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F1E203718B8 for ; Sun, 27 Sep 2020 21:06:46 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f43.google.com (mail-io1-f43.google.com [209.85.166.43]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BzyrQ05hzz4PVN; Sun, 27 Sep 2020 21:06:45 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io1-f43.google.com with SMTP id v8so8998267iom.6; Sun, 27 Sep 2020 14:06:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=yTdXt2uHefwBQJq3Vzm28VCMBGC2g/Cj4n7SCVQl8nA=; b=FvKZLDCsgWIeHRn7VK8brrgMI1hAWP4uy0vGIIDGxs388ieE0viQ2PMz7Sz5Lhrk3Y 14oYe6zQ52WxAZ47Q9S9vMWXQP0MPhPi6fH2pxXdFkrcURVu7zos23LYXSad5oKQqv4L mW4K23LD+5GfCgaLZ3HeYkf5VjtztGQbUYhj79+Gc/Aa9wSTS3+t0WaiD+TBwrEq2Pfd NH/j5O81E9O8Rxn+6I6jTD3xyLbfVg+j+YpajX1tBZvcXxLLWqsqTiqyUjl8AfDaQnuQ NyWdYS14PLYcJEuDhuCTqw2XwIftsPSsr8sZxJOmGx08gDDftfg36/pJz3cL3woDvIg+ oBHg== X-Gm-Message-State: AOAM530HFzjAD5qXncnClxAcarNMd6vgNP96qnbAlv0XXpGFlBiynvpw U1jgdSgmK2/9MFvvCrz2x9g1KZ/eR8n1ApUIoHNq0s6W X-Google-Smtp-Source: ABdhPJx1ccj/iJrFAggeqGIQWDndSHNgSIwtRobu2othoOQAreiyDeRNuLmmAbQ8fPIZJaEFktu8s4yrWpqiJM8VHdc= X-Received: by 2002:a5d:8352:: with SMTP id q18mr5304292ior.31.1601240804371; Sun, 27 Sep 2020 14:06:44 -0700 (PDT) MIME-Version: 1.0 References: <20200925003347.GG60607@FreeBSD.org> <202009250507.08P573Bh045283@mail.karels.net> <20200925093334.GH60607@FreeBSD.org> <97b52e71-9dfe-6b5f-13f0-8dc466ef376f@freebsd.org> <20200925112906.GF26726@FreeBSD.org> <4432eabd-2ef1-c435-085e-11319aec09e0@freebsd.org> In-Reply-To: <4432eabd-2ef1-c435-085e-11319aec09e0@freebsd.org> From: Ed Maste Date: Sun, 27 Sep 2020 17:06:32 -0400 Message-ID: Subject: Re: clock problems with BeagleBone Black on 12.2BETA2 To: Michal Meloun Cc: Glen Barber , "freebsd-arm@freebsd.org" , FreeBSD Release Engineering Team Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4BzyrQ05hzz4PVN X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.43 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-1.78 / 15.00]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEFALL_USER(0.00)[carpeddiem]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.998]; NEURAL_HAM_MEDIUM(-0.97)[-0.966]; NEURAL_SPAM_SHORT(0.19)[0.189]; RCVD_IN_DNSWL_NONE(0.00)[209.85.166.43:from]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.166.43:from]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; MAILMAN_DEST(0.00)[freebsd-arm]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Sep 2020 21:06:47 -0000 On Fri, 25 Sep 2020 at 08:43, Michal Meloun wrote: > > Thanks for response. > I currently preparing (slightly) bigger patchset which should > "normalize" situation about TI clock in CURRENT. These must be MFCed to > STABLE ASAP but I afraid that we miss 12.2 target :( Assuming this issue isn't going to be resolved it looks like we should remove the BBB images from the 12.2 release. From owner-freebsd-arm@freebsd.org Sun Sep 27 22:08:42 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 08C163737B6 for ; Sun, 27 Sep 2020 22:08:42 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C00Cr5xWkz4T69 for ; Sun, 27 Sep 2020 22:08:40 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 08RM8Tv6009290 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 27 Sep 2020 15:08:29 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 08RM8RIW009286; Sun, 27 Sep 2020 15:08:27 -0700 (PDT) (envelope-from jmg) Date: Sun, 27 Sep 2020 15:08:27 -0700 From: John-Mark Gurney To: Emmanuel Vadot Cc: Oskar Holmlund , Oskar Holmlund via freebsd-arm Subject: Re: clock problems with BeagleBone Black on 12.2BETA2 Message-ID: <20200927220826.GH4213@funkthat.com> Mail-Followup-To: Emmanuel Vadot , Oskar Holmlund , Oskar Holmlund via freebsd-arm References: <202009222004.08MK4xFj037249@mail.karels.net> <8217af510a451f10ea173bf1e26d04dcd50e8ca6.camel@freebsd.org> <1072725987.651720.1600964732891@mail.yahoo.com> <1441631630.802580.1600987963534@mail.yahoo.com> <20200925092059.5b9cb22fa87a478e1254b0f4@bidouilliste.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20200925092059.5b9cb22fa87a478e1254b0f4@bidouilliste.com> X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Sun, 27 Sep 2020 15:08:29 -0700 (PDT) X-Rspamd-Queue-Id: 4C00Cr5xWkz4T69 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [0.23 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_CC(0.00)[yahoo.com,freebsd.org]; FREEFALL_USER(0.00)[jmg]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-0.87)[-0.870]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; NEURAL_SPAM_SHORT(0.62)[0.624]; NEURAL_HAM_LONG(-0.73)[-0.728]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Sep 2020 22:08:42 -0000 Emmanuel Vadot wrote this message on Fri, Sep 25, 2020 at 09:20 +0200: > On Thu, 24 Sep 2020 22:52:43 +0000 (UTC) > Oskar Holmlund via freebsd-arm wrote: > > > > Den torsdag 24 september 2020 21:36:38 CEST, Ed Maste skrev: > > > > > > On Thu, 24 Sep 2020 at 12:31, Ian Lepore wrote: > > > > > > > > It was mentioned on irc a few days ago when someone noticed the BBB in > > > > the CI setup was failing... > > > > > > > > hrm, looks like BBB has been broken since end of July > > > > panic: Duplicated clock registration: clk@4_0 > > > > ah, mmel r363700 > > > > emaste: im sorry, but this is (unfortunately) expected > > > > collateral damage. im aware of this problem but right solution is not > > > > trivial and it needs more time. > > > > > > > > (strejda == mmel@) > > > > > > > > Then it was mentioned a while later that that BBB is using a very old > > > > u-boot, so maybe that has something to do with it. > > > > > > I've updated uboot on eMMC on the the CI system's BBB to: > > > U-Boot SPL 2020.07 (Sep 24 2020 - 04:58:48 +0000)^M > > > > > > however it's still failing the same way: > > > > > > clk_fixed7: on ofw_clkbus0^M > > > clk_fixed7: Cannot FDT parameters.^M > > > device_attach: clk_fixed7 attach returned 6^M > > > clk_fixed7: on ofw_clkbus0^M > > > clk_fixed7: Cannot FDT parameters.^M > > > device_attach: clk_fixed7 attach returned 6^M > > > ti_clkctrl0: mem 0x14-0x14f on ti_omap4_cm0^M > > > ti_clkctrl1: mem 0x4-0xd7 on ti_omap4_cm1^M > > > ti_clkctrl2: mem 0x4-0x7 on ti_omap4_cm2^M > > > > > > panic: Duplicated clock registration: clk@4_0 > > > ^M > > > ^M > > > cpuid = 0^M > > > time = 1^M > > > > > > phk@ reported success booting 13-CURRENT snapshots though, so I'm not > > > really sure what's going on. > > > > Well this is u-boot playing a trick on us. > > > > To have a common ground lets assume we use the snapshot from > > https://download.freebsd.org/ftp/snapshots/arm/armv7/ISO-IMAGES/13.0/ > > FreeBSD-13.0-CURRENT-arm-armv7-BEAGLEBONE-20200924-3c514403bef.img.xz > > > > DD to sd-card and press the "sysboot alter"-button it will boot to login prompt. > > > > Hit [Enter] to boot immediately, or any other key for command prompt. > > Booting [/boot/kernel/kernel]...                > > Using DTB provided by EFI at 0x87ee7000. > > Kernel entry at 0x97000200... > > Kernel args: (null) > > ---<>--- > > < remove some 30 lines of output > > > ofwbus0: > > simplebus0: on ofwbus0 > > simplebus1: mem 0x44c00000-0x44c007ff,0x44c00800-0x44c00fff,0x44c01000-0x44c013ff,0x44c01400-0x44c017ff on0 > > simplebus2: on simplebus1 > > simplebus3: on simplebus1 > > simplebus4: on simplebus1 > > ti_sysc0: mem 0-0x3 on simplebus4 > > ti_prcm0: mem 0-0x1fff on ti_sysc0 > > ofw_clkbus0: on ti_prcm0 > > > > Login as root and run: > > root@generic:~ # rm -rf /boot/msdos/dtb/ > > root@generic:~ # reboot > > > > > > Now at boot we got the same problem as Ed Maste described. > > At this boot simplebus1 has lost its ranges and ti_prcm0 are also a little bit different. > > ... > > ofwbus0: > > simplebus0: on ofwbus0 > > simplebus1: on simplebus0 > > ti_prcm0: mem 0x200000-0x203fff on simplebus1 > > ofw_clkbus0: on ti_prcm0 > > clk_fixed0: on ofw_clkbus0 > > clk_fixed1: on ofw_clkbus0 > > clk_fixed2: on ofw_clkbus0 > > clk_fixed3: on ofw_clkbus0 > > clk_fixed4: on ofw_clkbus0 > > clk_fixed5: on ofw_clkbus0 > > .... > > device_attach: clk_fixed7 attach returned 6 > > clk_fixed7: on ofw_clkbus0 > > clk_fixed7: Cannot FDT parameters. > > device_attach: clk_fixed7 attach returned 6 > > ti_clkctrl0: mem 0x14-0x14f on ti_omap4_cm0 > > ti_clkctrl1: mem 0x4-0xd7 on ti_omap4_cm1 > > ti_clkctrl2: mem 0x4-0x7 on ti_omap4_cm2 > > panic: Duplicated clock registration: clk@4_0 > > > > cpuid = 0 > > time = 1 > > KDB: stack backtrace: > > db_trace_self() at db_trace_self > >          pc = 0xc04e25f8  lr = 0xc00704ac (db_trace_self_wrapper+0x30) > >          sp = 0xc0d14810  fp = 0xc0d14928 > > db_trace_self_wrapper() at db_trace_self_wrapper+0x30 > > > > U-Boot have its own version of devicetree used in the SPL. Later in the boot process if "the real" u-boot cant find any devicetree files in its partition it uses its default. The default are from Linux 4.20: > > https://github.com/u-boot/u-boot/blob/v2020.07/arch/arm/dts/am33xx.dtsi > > > > The devicetree expected by the clock implementation are Linux 5.7 and later. > > > > To fix the problem in CI build - just copy devicetrees from base into the FAT partition for uboot to find. > > > > //Oskar > > Thanks for testing on your side Oskar, > > As a reminder, using u-boot embedded dtb will almost always result in > problems. > At best the dtb embedded is the same one from Linux (and so the same > one from our tree) but stripped down, nodes are missing etc ... > At worse it's either from an old version of Linux or a completly > different one specialy made for u-boot. Can we add a FreeBSD specific node, and if it's not present print a warning? I mean, it's less than ideal, but provides some idea. We could even possibly add a FreeBSD kernel version to the node to do a compare against... I understand the idea of unversioned dtb, but the way that Linux changes things enough, it seems like it's not really a valid option in the long run... > There is no real way for us to warn the user as dts aren't versionned > so all we can do is blindly accept the dtb passed to loader(8) via efi > (or LINUX_BOOT_ABI in case of bootm/booti). > > Note that we need u-boot to preload the dtb for us for a few reasons : > - In a EFI based env we cannot query u-boot for the board name, we > only have the full DTB. > - On some boards, like BBB, the same u-boot is used for multiple > boards (BBB, PocketBeagle etc ...) so booting in a generic way is only > possible if u-boot load the dtb for us. > - On some boards u-boot modify the dtb based on the version of the > boards or something and will enable/disable some nodes or add some new > ones. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arm@freebsd.org Sun Sep 27 22:16:10 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0EBDD373B94 for ; Sun, 27 Sep 2020 22:16:10 +0000 (UTC) (envelope-from manu@bidouilliste.com) Received: from mx.blih.net (mx.blih.net [212.83.155.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mx.blih.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C00NS5c80z4TZs for ; Sun, 27 Sep 2020 22:16:08 +0000 (UTC) (envelope-from manu@bidouilliste.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bidouilliste.com; s=mx; t=1601244960; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KvfFyVb9ktwtE1Kpx85lX7WL+Ne4+1soFMjg1sM8hTs=; b=eQkayvf64LoqKfpkx8IJVe+dKeRcPW4Kk3YTFWE1+Ijhh4+xRac/uN8gi1rvoov6s3KIf4 psOyFY4oQQOHiWw3vQWQjkIMC2tVTscGBE3WXr6w4CZqePEM/ZPx8j7VFxQNIFcmI3F+Iq YPiHMLri57Q2CxxvofD36M/d8Q6/Mgk= Received: from amy.home (lfbn-idf2-1-288-247.w82-123.abo.wanadoo.fr [82.123.126.247]) by mx.blih.net (OpenSMTPD) with ESMTPSA id 150ab2c2 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sun, 27 Sep 2020 22:16:00 +0000 (UTC) Date: Mon, 28 Sep 2020 00:16:00 +0200 From: Emmanuel Vadot To: John-Mark Gurney Cc: Oskar Holmlund , Oskar Holmlund via freebsd-arm Subject: Re: clock problems with BeagleBone Black on 12.2BETA2 Message-Id: <20200928001600.a66e86d36b5c94294a98358a@bidouilliste.com> In-Reply-To: <20200927220826.GH4213@funkthat.com> References: <202009222004.08MK4xFj037249@mail.karels.net> <8217af510a451f10ea173bf1e26d04dcd50e8ca6.camel@freebsd.org> <1072725987.651720.1600964732891@mail.yahoo.com> <1441631630.802580.1600987963534@mail.yahoo.com> <20200925092059.5b9cb22fa87a478e1254b0f4@bidouilliste.com> <20200927220826.GH4213@funkthat.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; amd64-portbld-freebsd13.0) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4C00NS5c80z4TZs X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bidouilliste.com header.s=mx header.b=eQkayvf6; dmarc=pass (policy=none) header.from=bidouilliste.com; spf=pass (mx1.freebsd.org: domain of manu@bidouilliste.com designates 212.83.155.74 as permitted sender) smtp.mailfrom=manu@bidouilliste.com X-Spamd-Result: default: False [-2.50 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[bidouilliste.com:s=mx]; FREEFALL_USER(0.00)[manu]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MV_CASE(0.50)[]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; R_SPF_ALLOW(-0.20)[+mx]; ARC_NA(0.00)[]; NEURAL_SPAM_SHORT(0.01)[0.015]; NEURAL_HAM_LONG(-1.01)[-1.014]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bidouilliste.com:+]; DMARC_POLICY_ALLOW(-0.50)[bidouilliste.com,none]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:12876, ipnet:212.83.128.0/19, country:FR]; FREEMAIL_CC(0.00)[yahoo.com,freebsd.org]; MAILMAN_DEST(0.00)[freebsd-arm]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Sep 2020 22:16:10 -0000 On Sun, 27 Sep 2020 15:08:27 -0700 John-Mark Gurney wrote: > Emmanuel Vadot wrote this message on Fri, Sep 25, 2020 at 09:20 +0200: > > On Thu, 24 Sep 2020 22:52:43 +0000 (UTC) > > Oskar Holmlund via freebsd-arm wrote: > >=20 > > > > Den torsdag 24 september 2020 21:36:38 CEST, Ed Maste skrev:=20 > > > > > > > > On Thu, 24 Sep 2020 at 12:31, Ian Lepore wrote: > > > > > > > > > > It was mentioned on irc a few days ago when someone noticed the B= BB in > > > > > the CI setup was failing... > > > > > > > > > > hrm, looks like BBB has been broken since end of July > > > > > panic: Duplicated clock registration: clk@4_0 > > > > > ah, mmel r363700 > > > > > emaste: im sorry, but this is (unfortunately) expected > > > > > collateral damage. im aware of this problem but right solution is= not > > > > > trivial and it needs more time. > > > > > > > > > > (strejda =3D=3D mmel@) > > > > > > > > > > Then it was mentioned a while later that that BBB is using a very= old > > > > > u-boot, so maybe that has something to do with it. > > > >=20 > > > > I've updated uboot on eMMC on the the CI system's BBB to: > > > > U-Boot SPL 2020.07 (Sep 24 2020 - 04:58:48 +0000)^M > > > >=20 > > > > however it's still failing the same way: > > > >=20 > > > > clk_fixed7: on ofw_clkbus0^M > > > > clk_fixed7: Cannot FDT parameters.^M > > > > device_attach: clk_fixed7 attach returned 6^M > > > > clk_fixed7: on ofw_clkbus0^M > > > > clk_fixed7: Cannot FDT parameters.^M > > > > device_attach: clk_fixed7 attach returned 6^M > > > > ti_clkctrl0: mem 0x14-0x14f on ti_omap4_cm0^M > > > > ti_clkctrl1: mem 0x4-0xd7 on ti_omap4_cm1^M > > > > ti_clkctrl2: mem 0x4-0x7 on ti_omap4_cm2^M > > > > > > > > panic: Duplicated clock registration: clk@4_0 > > > > ^M > > > > ^M > > > > cpuid =3D 0^M > > > > time =3D 1^M > > > > > > > > phk@ reported success booting 13-CURRENT snapshots though, so I'm n= ot > > > > really sure what's going on. > > >=20 > > > Well this is u-boot playing a trick on us. > > >=20 > > > To have a common ground lets assume we use the snapshot from > > > https://download.freebsd.org/ftp/snapshots/arm/armv7/ISO-IMAGES/13.0/ > > > FreeBSD-13.0-CURRENT-arm-armv7-BEAGLEBONE-20200924-3c514403bef.img.xz > > >=20 > > > DD to sd-card and press the "sysboot alter"-button it will boot to lo= gin prompt. > > >=20 > > > Hit [Enter] to boot immediately, or any other key for command prompt. > > > Booting [/boot/kernel/kernel]...=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0 =A0 > > > Using DTB provided by EFI at 0x87ee7000. > > > Kernel entry at 0x97000200... > > > Kernel args: (null) > > > ---<>--- > > > < remove some 30 lines of output > > > > ofwbus0: > > > simplebus0: on ofwbus0 > > > simplebus1: mem 0x44c00000-0x44c00= 7ff,0x44c00800-0x44c00fff,0x44c01000-0x44c013ff,0x44c01400-0x44c017ff on0 > > > simplebus2: on simplebus1 > > > simplebus3: on simplebus1 > > > simplebus4: on simplebus1 > > > ti_sysc0: mem 0-0x3 on simplebus4 > > > ti_prcm0: mem 0-0x1fff on ti_sysc0 > > > ofw_clkbus0: on ti_prcm0 > > >=20 > > > Login as root and run: > > > root@generic:~ # rm -rf /boot/msdos/dtb/ > > > root@generic:~ # reboot > > >=20 > > >=20 > > > Now at boot we got the same problem as Ed Maste described. > > > At this boot simplebus1 has lost its ranges and ti_prcm0 are also a l= ittle bit different. > > > ... > > > ofwbus0: > > > simplebus0: on ofwbus0 > > > simplebus1: on simplebus0 > > > ti_prcm0: mem 0x200000-0x203fff on si= mplebus1 > > > ofw_clkbus0: on ti_prcm0 > > > clk_fixed0: on ofw_clkbus0 > > > clk_fixed1: on ofw_clkbus0 > > > clk_fixed2: on ofw_clkbus0 > > > clk_fixed3: on ofw_clkbus0 > > > clk_fixed4: on ofw_clkbus0 > > > clk_fixed5: on ofw_clkbus0 > > > .... > > > device_attach: clk_fixed7 attach returned 6 > > > clk_fixed7: on ofw_clkbus0 > > > clk_fixed7: Cannot FDT parameters. > > > device_attach: clk_fixed7 attach returned 6 > > > ti_clkctrl0: mem 0x14-0x14f on ti_omap4_cm0 > > > ti_clkctrl1: mem 0x4-0xd7 on ti_omap4_cm1 > > > ti_clkctrl2: mem 0x4-0x7 on ti_omap4_cm2 > > > panic: Duplicated clock registration: clk@4_0 > > >=20 > > > cpuid =3D 0 > > > time =3D 1 > > > KDB: stack backtrace: > > > db_trace_self() at db_trace_self > > > =A0=A0=A0=A0=A0=A0=A0=A0 pc =3D 0xc04e25f8=A0 lr =3D 0xc00704ac (db_t= race_self_wrapper+0x30) > > > =A0=A0=A0=A0=A0=A0=A0=A0 sp =3D 0xc0d14810=A0 fp =3D 0xc0d14928 > > > db_trace_self_wrapper() at db_trace_self_wrapper+0x30 > > >=20 > > > U-Boot have its own version of devicetree used in the SPL. Later in t= he boot process if "the real" u-boot cant find any devicetree files in its = partition it uses its default. The default are from Linux 4.20: > > > https://github.com/u-boot/u-boot/blob/v2020.07/arch/arm/dts/am33xx.dt= si > > >=20 > > > The devicetree expected by the clock implementation are Linux 5.7 and= later. > > >=20 > > > To fix the problem in CI build - just copy devicetrees from base into= the FAT partition for uboot to find. > > >=20 > > > //Oskar > >=20 > > Thanks for testing on your side Oskar, > >=20 > > As a reminder, using u-boot embedded dtb will almost always result in > > problems. > > At best the dtb embedded is the same one from Linux (and so the same > > one from our tree) but stripped down, nodes are missing etc ... > > At worse it's either from an old version of Linux or a completly > > different one specialy made for u-boot. >=20 > Can we add a FreeBSD specific node, and if it's not present print a > warning? >=20 > I mean, it's less than ideal, but provides some idea. We could even > possibly add a FreeBSD kernel version to the node to do a compare > against... >=20 > I understand the idea of unversioned dtb, but the way that Linux > changes things enough, it seems like it's not really a valid option > in the long run... Yes we can, see https://github.com/evadot/freebsd/commits/dts-freebsd-branded It's not tested yet. > > There is no real way for us to warn the user as dts aren't versionned > > so all we can do is blindly accept the dtb passed to loader(8) via efi > > (or LINUX_BOOT_ABI in case of bootm/booti). > >=20 > > Note that we need u-boot to preload the dtb for us for a few reasons : > > - In a EFI based env we cannot query u-boot for the board name, we > > only have the full DTB. > > - On some boards, like BBB, the same u-boot is used for multiple > > boards (BBB, PocketBeagle etc ...) so booting in a generic way is only > > possible if u-boot load the dtb for us. > > - On some boards u-boot modify the dtb based on the version of the > > boards or something and will enable/disable some nodes or add some new > > ones. >=20 > --=20 > John-Mark Gurney Voice: +1 415 225 5579 >=20 > "All that I will do, has been done, All that I have, has not." --=20 Emmanuel Vadot From owner-freebsd-arm@freebsd.org Mon Sep 28 00:13:45 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7EB04376E1C for ; Mon, 28 Sep 2020 00:13:45 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C03092CbFz4b2Z; Mon, 28 Sep 2020 00:13:45 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1601252025; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=mjtSAIyJMJGmJFwEwVqTEeG6Stx8cVOylRqpDx9n3Mk=; b=MebhZNh6lMSR6J6ZLAj8wuwu+4fYBGoEi4BsCv9n2Gmb95rb6gtoqGK/v3FF9TADUUyIBD eN5w2Q7dgM1JyVp1pllGIOu8vDQgvCrD1yFC8AnaT9wl/XNXrN9OUULO9CYhg+0fGzEQZA E8GkyHcgaGEmr+usee6e4/qWgP35OCZiwL/HBu+HkUOLB3HUp9Ge8FTMuOm3FuZQtHyPcN cKsLtrhrfeM6/+WX9hqqosLh2t5Ta+sOf0AN7I7Cyn/q4UoRJ8YzFrjzn8fNDpIwftYbaJ Nu0fse/MQDaYfXvaop0ATjyG+U75dznaCxJJXuY49lx3TY5MUP/Xc4MJSjJNCw== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id AE73986B9; Mon, 28 Sep 2020 00:13:44 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Mon, 28 Sep 2020 00:13:42 +0000 From: Glen Barber To: Ed Maste Cc: Michal Meloun , "freebsd-arm@freebsd.org" , FreeBSD Release Engineering Team Subject: Re: clock problems with BeagleBone Black on 12.2BETA2 Message-ID: <20200928001342.GU60607@FreeBSD.org> References: <20200925003347.GG60607@FreeBSD.org> <202009250507.08P573Bh045283@mail.karels.net> <20200925093334.GH60607@FreeBSD.org> <97b52e71-9dfe-6b5f-13f0-8dc466ef376f@freebsd.org> <20200925112906.GF26726@FreeBSD.org> <4432eabd-2ef1-c435-085e-11319aec09e0@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="v77n/kH2jMt1ht8J" Content-Disposition: inline In-Reply-To: ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1601252025; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=mjtSAIyJMJGmJFwEwVqTEeG6Stx8cVOylRqpDx9n3Mk=; b=BR6zxY8aVexrb7BgGSDpE1XDaIU7EY+Gtlj2WW5XxWngDsWlV1XGJ8iBJ07NBQj4GUqbdy A11wQ+mh+hhK1VQvsJn9YoAY/IjVzYlDQJVwlGU1c7E6/pPbTzSgPzywATNei2paXkkjeE HxyaJZCLxXC+Ga5MZdU2Ppo7OqbCLqr3mgKus6wlyI30EuSSgoK/5ooM8kPPm6bejZ7Quw zhNHWsXErVA+B5f7oGbdYS+SMYzbDJGsu++QW9srZPa0SkCYdE5hy2UYAg0P2xRGo+WFGf FofuW0jogDs5V8+y2f1rrEZToLxKYcT4yccwPBNPIe0ZFvwpwgEf15apowXjZA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1601252025; a=rsa-sha256; cv=none; b=c+tqZRBDiFYxe5cKrZJ2AVUqxltam6o3t6n1nlh5A8uNhaoqERhBtTT5r4Go3mtRy8auoR jVGJkuCouqo8v2mhoI4NieWcgraebCsKamcvuRZj1KjFO2gwj7Xxwsl0Cs78spvcIF1pcs rN8XfPMqyiARXcCLRfUJzvqie/fOzdgkUUGrk7rI9xMkL5T5RhxjUM50Tp8m0o6ksyPpvZ lhiewOgeO7tCvXrBcigpvFwZTxA/X2qvvVQOL6fo8lDxqwMAoNSU8t7KMiexFibnsEoMJp hMF+l2o3TTBiV0KoLBoO8utJ8tNFKCSmQeNovCTBylFtDvZ8gSbTi13KJ7826w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Sep 2020 00:13:45 -0000 --v77n/kH2jMt1ht8J Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Sep 27, 2020 at 05:06:32PM -0400, Ed Maste wrote: > On Fri, 25 Sep 2020 at 08:43, Michal Meloun wro= te: > > > > Thanks for response. > > I currently preparing (slightly) bigger patchset which should > > "normalize" situation about TI clock in CURRENT. These must be MFCed to > > STABLE ASAP but I afraid that we miss 12.2 target :( >=20 > Assuming this issue isn't going to be resolved it looks like we should > remove the BBB images from the 12.2 release. >=20 If it isn't resolved by the last RC build, I'll pull it from the final RELEASE builds. Glen --v77n/kH2jMt1ht8J Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAl9xKrYACgkQAxRYpUeP 4pMLvg/+JsRKbX0osXdiTnBc2iUqTclyL3OnCn8y08vDpLxGrFT9Crtib8jhPRgm 1/AqlYWEGnXM7TbGfcLhefL+CK98bhkz6p8I70Y7LhGyp3QfXe4odk6dfDiSk7cl K1Gv3n6KAzulQ2i5YQiTNld94d2L0OENTe6KUXSn0klWQia08VaXYcCJrd6CPeOH 1SVh3tNr8pmFNOWQjSKSnGQ9XzN0nSKIW5mBAGxI7/8cG/8eihoXsDbOiqN4ot8b XCCSNooFjWF5Vy73pInUa/yqv1xcb9C7CAgnmuxOhBO47wXFX/uJ6gV5tZuw/kov ccIiSOVTiztP5dlIe4OKVRhPT9rhAJCjRZVRyrMkLChEDmd5eTR/yVf6peZtQvI+ 4CXxbVtd8FLwS+baUZxBGorNkqmb+A+B2bBZz7WDFXzJibiOSuqjekBpANoE/JAl 6xs9bfbM/CgFUO93knfV6YSMi/yHxPpzfeTDVKj2i9CTzPfff/eSpoXmzn4DCM0j SpK1lifbq5R3eqtLS6eGL3MA47QDkl9v/J9cxTdK9pUJmnTq8C3Qb/SemO/6O6Eg WZPgrNpAQqZhrPnqYbM4SjlrD+tryqMiqHsd2gZokaDWZoJTZF5IBDn+RWDmlP3D S6KhGHBzQFwQyE3sZQgBNMGUagjYQNub5vard+ufPjiPcXaG0os= =B4jU -----END PGP SIGNATURE----- --v77n/kH2jMt1ht8J-- From owner-freebsd-arm@freebsd.org Mon Sep 28 15:36:48 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 48DEB42569C for ; Mon, 28 Sep 2020 15:36:48 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-24.consmr.mail.gq1.yahoo.com (sonic304-24.consmr.mail.gq1.yahoo.com [98.137.68.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C0RTC1jVzz4VNH for ; Mon, 28 Sep 2020 15:36:46 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: XG3ZFZcVM1kS3M3gRpCagH_u1JKF4UjtJLPMcn22BJCal1G0J8Uenxb_INVJMT1 1W0yaG9f8FzVxn63sYmPsJMNFBSCV4wjz4dqmuKoUi7eIaY26VHWYy7bgGqhE78MMNFY4EwxY_GZ G02IjU2Dko_JZ71_limi7X28ylb1u07P2pXVkAWFYruSKxf3jDJAhMq0JZBLUnrGTRWeRZgaIh1D vtfMzqAmn00WJFM2ef0TiZfO20HwHpHiHGhV9B45G6SE2Xq5PJxwgedXHQsXKB0gepljEBEhK8RY CL0FFChmMFtdKzGfyZ1lPUHtPNZNdTaYoHEX2b2s4ihxSGoiytiEVErlis_jL4UlJvjRq7VerXGP M3jgJdeIQ70xdolcEUptfbyzIVVSKqL0wfitpHO5mF4EZZb99T40xgew367Hhi3ZUnf7ppcyJtH1 5Faa0sW725ThxML.bUsTAo7jlFmrVSxo6itdJNAV7YTGcnba9YkErmkujOlcg814lUjPRybVcusr gFrwU_awpQbc4.fFfMlt9j5ITRa5kFcfZaZQ9lHqzpIP6T0PrB298qHWh4H5x9mTWfEjDklDY9DG N3gOmst1thaiWh2JTQIvr0DhacCeQoPPTdLMmcxgDDxFmi4Ky3A5yEXUxhM0O69E2G7nr_XOBVEw kULZVyZ1IlAsjVExhuSQMZwpQAPUqfnbC9cTW7runJhmvaP9neQD04rL6fJDZXlpPIHlcNw09nbn OlBDjqaPrV8mP.vSM0sKaw4J8RKzbJ1tcpR5WHp9ohnQJEPuFDXKcU0tTk.RZzMZ3YIg1MsaaG_S VtISz0dxfqVTc4ncloUwuzFs5GXGET5SbYBrygHn629ehYtp4BnpOZfQJiSbt9vduSR9cQfxlbDn _dLl57oYGnBjo4O8vOGd62I0uoX6jt2NBUKjA3gcsc4sfvekvE5HmeDED2eAeJmDUIKXGTON8PpW i0ib50nWTBX1Nva9ZnkyFAikTXgAbRgnViDoiprwQuOS_DlfOTgknySJx2TT2yR37IdArb8N0HcH Iusv9TuQP2Fsg9._5pldpwI1Iwjlx4IG8ULxw_Js6sygoX9jwINrxSJbIE0Tkh4DpSrPjRkey_Ja k_A9AxKKPH_8MJBtggOFKxCiEqVEQNpfW8mhN315TB.XR8FGs5PhLw1OR0bsi9sB.LYnONzgfyWV bVpV6Xh53jGPcZWuYwNjMKTbUrjfar6_snRwssC_MMfKgs3eeJPZAVJoNBjEXWaKNURvU1tsW16_ g9qKTR4zeiYUjTUtK.ilAzBIG.l8TkdA3MNMRrcVqjU.59iEjZzKFBL4sL1vt0QSahzdJKSJmFwR eiIW12tbhf44ieap2U1vwv4A_Gq3.Jh5Xm__YvbBcwNT1PhV8jn8yyDoCsPyDP6Y6fUNnt3qvtLn 5wVnJqzJ9eDcEynjKMKM3405ykGu_9hTtqxUWUybaxeWrHAc5.Xly5oSiMhrn6_vWJIJzYd6rs_F _MI7GM2lIHy.cAsXnUS5xss8hVARdhLOl8qLN6.PwMkRWZa19o4VbKvu7mtIIhNmuvpaAm_sfMXK jnlkRRiUNeX1L8VYAhA-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Mon, 28 Sep 2020 15:36:45 +0000 Received: by smtp421.mail.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 3fc416d2e491e6b8b37060df97939366; Mon, 28 Sep 2020 15:36:42 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: RPi4B buildworld buildkernel times for already installed system being -mcpu=cortex-a72 vs. -mcpu=cortex-a53 based Date: Mon, 28 Sep 2020 08:36:40 -0700 References: <4E155E94-3AA0-464D-A1E9-45A7827537ED@yahoo.com> To: freebsd-arm In-Reply-To: Message-Id: <9CF3675E-072B-4845-A510-691508DCEF3C@yahoo.com> X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Rspamd-Queue-Id: 4C0RTC1jVzz4VNH X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.54 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.02)[-0.022]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.01)[-1.008]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.011]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.205:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.205:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Sep 2020 15:36:48 -0000 [Turns out, when sdram_freq_min=3D3200 is effective, -j4 builds are = faster than -j3 builds by about an hour (holding other configuration conditions constant).] On 2020-Sep-27, at 11:07, Mark Millard wrote: > On 2020-Sep-20, at 18:40, Mark Millard wrote: >=20 >> On 2020-Sep-20, at 18:32, Mark Millard wrote: >>=20 >>> The following are from scratch buildworld buildkernel rebuilds >>> on a RPi4B (head -r363590 context). >>>=20 >>> ENVIRONMENT: -mcpu=3Dcortex-a72 based world and kernel running = already, RPi4B @ 2G Hz, >>> Restricted to 3 GiByte RAM, -j3: >>>=20 >>> World built in 37469 seconds, ncpu: 4, make -j3 >>> Kernel(s) GENERIC-NODBG built in 2474 seconds, ncpu: 4, make -j3 >>>=20 >>> ENVIRONMENT: -mcpu=3Dcortex-a53 based kernel running, RPi4B @ 2G Hz, >>> Restricted to 3 GiByte RAM, -j3: >>>=20 >>> World built in 44034 seconds, ncpu: 4, make -j3 >>> Kernel(s) GENERIC-NODBG built in 2895 seconds, ncpu: 4, make -j3 >>>=20 >>> So a little under 11.1 hr total vs. a little over 13.0 hr total, >>> a somewhat over 50 min improvement. >>=20 >> "a somewhat over 1hr 50 min improvement" is what I should have >> managed to type. >>=20 >>> (A xhci patch finally allowed me to boot -mcpu=3Dcortex-a72 >>> based kernel builds on the RPi4B: The xhci event ring >>> initialization code was missing a usb_bus_mem_flush_all >>> call previously.) >>>=20 >>>=20 >>> Supporting details: >>>=20 >>> (e-mail based spacing changes expected below) >>>=20 >>> # diff -u = ~/src.configs/src.conf.cortexA72-clang-bootstrap.aarch64-host = ~/src.configs/src.conf.cortexA53-clang-bootstrap.aarch64-host >>> --- = /root/src.configs/src.conf.cortexA72-clang-bootstrap.aarch64-host = 2020-03-13 22:29:25.470155000 -0700 >>> +++ = /root/src.configs/src.conf.cortexA53-clang-bootstrap.aarch64-host = 2020-03-13 22:29:25.469455000 -0700 >>> @@ -49,9 +49,9 @@ >>> # Use of the .clang 's here avoids >>> # interfering with other CFLAGS >>> # usage, such as ?=3D usage. >>> -CFLAGS.clang+=3D -mcpu=3Dcortex-a72 >>> -CXXFLAGS.clang+=3D -mcpu=3Dcortex-a72 >>> -CPPFLAGS.clang+=3D -mcpu=3Dcortex-a72 >>> -ACFLAGS.arm64cpuid.S+=3D -mcpu=3Dcortex-a72+crypto >>> -ACFLAGS.aesv8-armx.S+=3D -mcpu=3Dcortex-a72+crypto >>> -ACFLAGS.ghashv8-armx.S+=3D -mcpu=3Dcortex-a72+crypto >>> +CFLAGS.clang+=3D -mcpu=3Dcortex-a53 >>> +CXXFLAGS.clang+=3D -mcpu=3Dcortex-a53 >>> +CPPFLAGS.clang+=3D -mcpu=3Dcortex-a53 >>> +ACFLAGS.arm64cpuid.S+=3D -mcpu=3Dcortex-a53+crypto >>> +ACFLAGS.aesv8-armx.S+=3D -mcpu=3Dcortex-a53+crypto >>> +ACFLAGS.ghashv8-armx.S+=3D -mcpu=3Dcortex-a53+crypto >>>=20 >>>=20 >>> The .amd64-host files are similar for doing cross builds. >>>=20 >>> I also use +=3D in secure/lib/libcrypto/Makefile : >>>=20 >>> # svnlite diff /usr/src/secure/lib/libcrypto/Makefile >>> Index: /usr/src/secure/lib/libcrypto/Makefile >>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>> --- /usr/src/secure/lib/libcrypto/Makefile (revision 365919) >>> +++ /usr/src/secure/lib/libcrypto/Makefile (working copy) >>> @@ -20,7 +20,7 @@ >>> SRCS+=3D o_str.c o_time.c threads_pthread.c uid.c >>> .if defined(ASM_aarch64) >>> SRCS+=3D arm64cpuid.S armcap.c >>> -ACFLAGS.arm64cpuid.S=3D -march=3Darmv8-a+crypto >>> +ACFLAGS.arm64cpuid.S+=3D -march=3Darmv8-a+crypto >>> .elif defined(ASM_amd64) >>> SRCS+=3D x86_64cpuid.S >>> .elif defined(ASM_arm) >>> @@ -35,7 +35,7 @@ >>> SRCS+=3D aes_cbc.c aes_cfb.c aes_ecb.c aes_ige.c aes_misc.c = aes_ofb.c aes_wrap.c >>> .if defined(ASM_aarch64) >>> SRCS+=3D aes_core.c aesv8-armx.S vpaes-armv8.S >>> -ACFLAGS.aesv8-armx.S=3D -march=3Darmv8-a+crypto >>> +ACFLAGS.aesv8-armx.S+=3D -march=3Darmv8-a+crypto >>> .elif defined(ASM_amd64) >>> SRCS+=3D aes_core.c aesni-mb-x86_64.S aesni-sha1-x86_64.S = aesni-sha256-x86_64.S >>> SRCS+=3D aesni-x86_64.S vpaes-x86_64.S >>> @@ -242,7 +242,7 @@ >>> SRCS+=3D ofb128.c wrap128.c xts128.c >>> .if defined(ASM_aarch64) >>> SRCS+=3D ghashv8-armx.S >>> -ACFLAGS.ghashv8-armx.S=3D -march=3Darmv8-a+crypto >>> +ACFLAGS.ghashv8-armx.S+=3D -march=3Darmv8-a+crypto >>> .elif defined(ASM_amd64) >>> SRCS+=3D aesni-gcm-x86_64.S ghash-x86_64.S >>> .elif defined(ASM_arm) >>>=20 >>> The RPi4B is using: >>>=20 >>> over_voltage=3D6 >>> arm_freq=3D2000 >>>=20 >>> and was booted via uefi/ACPI. >>>=20 >>> I have not repeated the -j4 or other -jN comparisons that >>> I reported in the past. The -mcpu=3Dcortex-a53 figures are >>> from the past. >=20 > The following new timing is based on head -r365932 rebuilding > itself where the 8 GiByte RPi4B config.txt ended with: >=20 > over_voltage=3D6 > arm_freq=3D2000 > sdram_freq_min=3D3200 >=20 > and the boot was via u-boot, no RAM restriction. (The > sdram_freq_min assignment does not seem to do anything > for rpi4-uefi-devel v1.20 uefi/ACPI based booting.) > /etc/sysctl.conf has: dev.cpu.0.freq=3D2000 . No use of > powerd or other such. >=20 >=20 > ENVIRONMENT: -mcpu=3Dcortex-a72 based world and kernel running = already, > 8 GiBYte RPi4B @ 2G Hz with sdram_freq_min=3D3200, u-boot style boot, = -j3: >=20 > World built in 31852 seconds, ncpu: 4, make -j3 > Kernel(s) GENERIC-NODBG built in 2059 seconds, ncpu: 4, make -j3 >=20 > So somewhat under 9.5 hr overall. >=20 >=20 > That means somewhat over 3.5 hours faster than a -mcpu=3Dcortex-a53 > based system without sdram_freq_min=3D3200 using 3 GiByte RAM > but still RPi4B @ 2G Hz (uefi/ACPI boot): >=20 > World built in 44034 seconds, ncpu: 4, make -j3 > Kernel(s) GENERIC-NODBG built in 2895 seconds, ncpu: 4, make -j3 >=20 > (Same as reported in prior messages.) >=20 > But the prior -r362590 vs. the now -r363932 means there is more = varying > than in my previous comparisons. For example, clang 10 vs. clang 11. >=20 > I'm probably going to run a -j4 build to see how it compares in > this context. ENVIRONMENT: -mcpu=3Dcortex-a72 based world and kernel running already, 8 GiBYte RPi4B dev.cpu.0.freq=3D2000 with sdram_freq_min=3D3200, u-boot style boot, -j4: World built in 28526 seconds, ncpu: 4, make -j4 Kernel(s) GENERIC-NODBG built in 1841 seconds, ncpu: 4, make -j4 So somewhat under 8.5 hr overall. That means somewhat over 4.5 hours faster than a -mcpu=3Dcortex-a53 based system without sdram_freq_min=3D3200 using 3 GiByte RAM but still RPi4B @ 2G Hz (uefi/ACPI boot). > I've not run a default arm-freq/sdram_freq_min/dev.cpu.0.freq = buildworld > buildkernel in a long time and so do not have reasonable comparison > figures relative to that type of context. I do not plan on such an > experiment. >=20 >=20 > I'll note that I run these tests with a monitor connected that sits > with a static login prompt display after booting. I do not not test > with X11 or other use that might significantly compete for more power. > The serial port console is usually used. I have used ssh sometimes in > the past. >=20 > ~/src.configs/src.conf.cortexA72-clang-bootstrap.aarch64-host is still > unchanged: >=20 > # more ~/src.configs/src.conf.cortexA72-clang-bootstrap.aarch64-host=20= > TO_TYPE=3Daarch64 > # > KERNCONF=3DGENERIC-NODBG > TARGET=3Darm64 > .if ${.MAKE.LEVEL} =3D=3D 0 > TARGET_ARCH=3D${TO_TYPE} > .export TARGET_ARCH > .endif > # > #WITH_CROSS_COMPILER=3D > WITH_SYSTEM_COMPILER=3D > WITH_SYSTEM_LINKER=3D > # > WITH_LIBCPLUSPLUS=3D > #WITH_LLD_BOOTSTRAP=3D > WITHOUT_BINUTILS_BOOTSTRAP=3D > WITH_ELFTOOLCHAIN_BOOTSTRAP=3D > #Disables avoiding bootstrap: WITHOUT_LLVM_TARGET_ALL=3D > WITH_LLVM_TARGET_AARCH64=3D > WITH_LLVM_TARGET_ARM=3D > WITHOUT_LLVM_TARGET_MIPS=3D > WITHOUT_LLVM_TARGET_POWERPC=3D > WITHOUT_LLVM_TARGET_RISCV=3D > WITHOUT_LLVM_TARGET_X86=3D > #WITH_CLANG_BOOTSTRAP=3D > WITH_CLANG=3D > WITH_CLANG_IS_CC=3D > WITH_CLANG_FULL=3D > WITH_CLANG_EXTRAS=3D > WITH_LLD=3D > WITH_LLD_IS_LD=3D > WITHOUT_BINUTILS=3D > WITH_LLDB=3D > # > WITH_BOOT=3D > WITHOUT_LIB32=3D > # > # > NO_WERROR=3D > #WERROR=3D > MALLOC_PRODUCTION=3D > # > # Avoid stripping but do not control host -g status as well: > DEBUG_FLAGS+=3D > # > WITH_REPRODUCIBLE_BUILD=3D > WITH_DEBUG_FILES=3D > # > # Use of the .clang 's here avoids > # interfering with other CFLAGS > # usage, such as ?=3D usage. > CFLAGS.clang+=3D -mcpu=3Dcortex-a72 > CXXFLAGS.clang+=3D -mcpu=3Dcortex-a72 > CPPFLAGS.clang+=3D -mcpu=3Dcortex-a72 > ACFLAGS.arm64cpuid.S+=3D -mcpu=3Dcortex-a72+crypto > ACFLAGS.aesv8-armx.S+=3D -mcpu=3Dcortex-a72+crypto > ACFLAGS.ghashv8-armx.S+=3D -mcpu=3Dcortex-a72+crypto >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Mon Sep 28 20:08:15 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E5A0342A927 for ; Mon, 28 Sep 2020 20:08:15 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-24.consmr.mail.gq1.yahoo.com (sonic312-24.consmr.mail.gq1.yahoo.com [98.137.69.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C0YVR0GHsz3WXd for ; Mon, 28 Sep 2020 20:08:14 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: VqtsDjQVM1k5zCtijygRp5xZZznP8x_5zLza5xybcXdfEf.prJ.mSG7ppteqJx6 Bp3ZIlf9_saKVv8z_JoxlSjY0mLRLS6CIX13aGNyKYQW7IV1YKc5VgqhEzk8LL6USm_ElXilt_m6 sHfNSUmoaNiGzCMH4rCdjJGaP92sYGdktQYCG6KQNxh0PXr70b5Oq0bUv4Atdvm_2xx9tbMYebFn Sl.pJlHO2saR_OlX1p.Udzch4FOGO3owWURNT_xd1aFRdlQO60jYTa0C.jyWiX7GXNZols4DPnmz 833nnGEbSjWM6P4o_JPNYeTkaoHHjWv9jGmWeR_THXcEHgDSdiMbLIBFjAGpIL1kAolBZTfG3YsX e6PROh4qhftxvpTjzEwzmzi6Ig4owIuO8l8M2Fk01GlTeuS_2Nw6OWhD2c8DKsr77PxnGKZYsKEc ifU9Qt51djBu7Ohk1aQLE3h2l6zsk88Vcvds27a5JWpex6DPgt31yrOOr3dxuBGW_fOi3nJvBEcE sH3m.SIMlyH6.o8a2378ODU61iDzp3cErh_ZD0tQlrxjJxX84aKCH1sdQrNqOvFL7NhBP9CsSxLi .VoBBU9qJBjsfb7IA0VlNvxTlkBYyBr_RIUkj97EQfefnvK2MMxrJGS61D9uDi4LulHugKMJcP37 xwiCGJWdiJj7.fqS.RPDton8HyorX6_QbWkSkDoVut0hI613cBEzC6iGbFbqY9MKKjyJ1D7Ein30 i7459ggarueyCQL57gVezags5WBC15QLRe1MUXjo9Tq0EwUhzZeVDsM0FwFwq3ewBiBvE9pJ8cYX fYAShwFKZQrQ6dkm2QAQk4pDHx51QAwXoQa1fkmkYwdn875KjEjf1GUKc90K5U47rMjkkihPGPxj KAdPM5jg5Fx.i46y6OKrwz9kQEHZFVyLQajl0Q0LTNBlWBRYi7QFBifrKdcyHKJlU5eA1RK4Wnpk Yw_iL8MJW.d5X3HmX4g6Jm8zPAcQJpzvNcKFtXIQGk_jDS2wZbWfbIe5tl_9U4jVI9OWQzdi2Pq7 gzWjaGV4cVvIOkcZ4Q.S_hNWZkQH9PowBLiLmD3PUbV1EB0ALftSXItWauWRix2FeJbhfSkJ1XR0 n3eJpwQdtl1yyFHUbqilvJemVTMhkMY8Oz8ArMiuBk.48FRpN1uLVfbOlyqAlvG9z6BWVpcKKtQA UQ1Mv4igXVT4hADyEQDmdEiPHAQMf48PqG_vcJa13zaAFHqqaoEtBgw7rbqPrJ5p4CJxIjPILIqN EvdssP2WWfRZVPelVLoU6chxopDzz9jg4gAFs_vk0xqJLeozLt9a54qhH7uaD5xCoDt.11Lu1iRP mYJKnhq1QlghUhPLYq93CL9j4rYPyb9X4sDgMaup9q0x.Lzh.6ei2QBLG2LqVcGZzLQsKbIq9Xqi EkgG6yb87qRsIu.N.zTwsR13i_DyTZ2Rs5k56a.pB6vfilNzDDAFtSTSMUp325vByp87RU5T2YuD UfUygMeoNgh5xPjDP.3rrDYKkx.RLqPYPIlW5IYXQU7VBrV5H.TyDxcDhVMM9wPHm68j27t4mJHK ftUyAs_IG7W7plEPZiXE- Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Mon, 28 Sep 2020 20:08:13 +0000 Received: by smtp422.mail.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID eff035c8720ba313c39ea1de7477083e; Mon, 28 Sep 2020 20:08:10 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: FYI: Summary of what it took to get around 8.5 hr head -r365932 buildworld buildkernel times on a RPi4B Message-Id: <1B998B4B-411E-46D9-981D-3197AB1223A3@yahoo.com> Date: Mon, 28 Sep 2020 13:08:09 -0700 To: freebsd-arm X-Mailer: Apple Mail (2.3608.120.23.2.1) References: <1B998B4B-411E-46D9-981D-3197AB1223A3.ref@yahoo.com> X-Rspamd-Queue-Id: 4C0YVR0GHsz3WXd X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.74 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.22)[-0.223]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.01)[-1.005]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.012]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.205:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.205:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Sep 2020 20:08:16 -0000 The major configuration properties contributing to having smaller RPi4B = FreeBSD buildworld buildkernel times have been (not ranked): Cortex-A72 clock rate 2 GHz: over_voltage=3D6 and arm_freq=3D2000 (config.txt) For u-boot based booting: also dev.cpu.0.freq=3D2000 (/etc/sysctl.conf) Always full speed RAM: sdram_freq_min=3D3200 (config.txt) Note: Only effective with u-boot based boot. So uefi/ACPI v1.20 context = is slower. Using Cortex-A72 tuned code ( from -mcpu=3Dcortex-a72 ) in the running = kernel/world When sdram_freq_min=3D3200 is effective: Use of -j4 for doing = buildworld buildkernel When sdram_freq_min=3D3200 is ineffective: Use of -j3 for doing = buildworld buildkernel Use of RPi4B's with sufficient (effective) RAM Heaksinks and fan use, along with 5.1v 3.5A power supply Note: <3072 MiByte effective RAM: not tested. Note: Other types of cooling: not tested. Note: Other types of power supplies: not tested. File system (UFS) on USB3 SSD (vs. microsd card) Note: zfs: not tested Note: Other USB3 media: not tested Other notes: Note: Recent tests were based on head -r365932 . Note: 1024 MiByte RAM would slow things at times (significant paging for the same -jN). Note: Using a kernel built with -mcpu=3Dcortex-a72 requires head = -r365918 ( stable/12: -r366182 ; stable/11: -r366183 ) or later to avoid misconfiguring one of the rings for xHCI. Note: Use of an appropriate u-boot to boot head -r365677 or later allows 4 MiBYte and 8MiByte RPI4Bs to handle xHCI/USB3 reliably (USB2 untested). (No MFC?) Note: Use of uefi/ACPI from https://github.com/pftf/RPi4/releases/ requires use of the uefi 3072 MiByte limit for USB3 I/O to be reliable for 4 GiByte and 8GiBYte RPi4Bs (USB2 untested). The https://reviews.freebsd.org/D25219 patch is involved in having things operate, even for 3072 MiByte. uefi use also ignores (overrides?) sdram_freq_min=3D3200 use. There are also tradeoffs for what devices are supported. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Tue Sep 29 01:30:02 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A80413F0F02 for ; Tue, 29 Sep 2020 01:30:02 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-25.consmr.mail.gq1.yahoo.com (sonic304-25.consmr.mail.gq1.yahoo.com [98.137.68.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C0hdj1nwhz45v6 for ; Tue, 29 Sep 2020 01:30:00 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: zCHFRewVM1mjhotKbwV0L.RQCpHcYhz1p.arvqCRlafr4UdhbAOXRClt8j.X2R0 yqymFRnBemPf86yVoEwVRGd14Ahe22Sz0FNJMrYx5d6Y5A5XseYnGjYffLT5hnjqFY3sC.OZkF67 M2dgoR45oVKybXZmYpnhH9RfwgyZ87MfN5Ug5W3jJgVtMhrbRmXuC3ZqjW0m.Uv5H89cawZH6vxi 97WT9qfVDi8w3JS72p3nVMOLxoKi48Q6HggmO4dTowF5Q0U8BRAd6AhST1H7adq0MrxOziBmNjK1 1oCxjjb.yFLhRY8YJNKRMEpPv5OSjN3o_16ghRhE2gBeEievHSXHQPkM79oReyvf1nt0_A06Sxo_ L9XJY21zrymmlx_VUOp.fQgXEmwOdoQJpHQMsZwDOF3f4JC.uacfVrAh_HZUNlFivU0lqWNnsHjv n578lmWSx5iGMnErOl9ojaiAp3.Xf5I4f14N80u9s6vjIHs4VCsh5ABCThmkXjQv3rQVtdofXczj Nuxvv7lhMbWeRpXcsUThiyxypiR65CNTTKr38xnmYAYzkz6EFW9RFQQCQKe6HC3qL7C9eLJUVomD kHzBDRtthpjqPxC7785YNHfmH89RHc6C.AvfBubOtgvtJkpCTBbPC8SFFwbnzbWp_gL.L6lL8sHF BvxMNz2OkqM_B0f5lpmTsmRKhcG6HkMvkqDddg88G91uM8b6TDeWHTpZpwfeVwVrpmjCyszLskN7 8kEw59HObzcu5dQlXU8zp.M0tkpvmorZdBW6AZuCwD3sbIphjke_RUAdklhFCkNF.U87yjaCBeCp yf4p35hVvdSShxZIsv_fs952tcVPnfzV_EDGrU.PlsdQfJ2QdH9DdLrMwte5hS8heqlPHx8D.Lqz tAG1QFttlHKVwXSX.vQ95AzRSpGwvi7U8Gg0a5DEFNZCVxyp8VBJPaPB8drOYAnSq4cEAZvbA80D ktU5.5i1nrluXgWcTWS2H31ABoLf6wsfnFHEWLJG3QmlKH3pqVouvaaneLNtJ9AkV5QKCkX40TwB F77Iz0k3zcwf4YmEsMUuK7dQg_uBy3Fk.z2Kq2CX3ffyneU3Cw16E8sDcprR2oCrOiYsGWWMyhwO s1Y9_JwLaFo6EM6xS15MtFlevpMmk1FXB5W3JKFqTYLw_V2bEsV_nc994XLaz8F6vcCa85Cw1XVk H_.568dn.5zbAzf_J.k8UTvpyYoylXsxyyNZriVdSmYFGCmVrUU5r3ke9GfIO0xxCrY5TJ3CPvue 1C5f.GOW7331pKD8FmZCetcO9NYPlkQ5IP0MF10kqwUCvS2lopVwi4y8hdKXoIoP_WH1KOH9lrR4 nz1_g5nFo9kv62FlNpJsUoIge99HL3EutujcZwKu6ZSeV_rtGEGa8qlNPfW7NqkLuUbQgjq5KD8f pMHf7NCGmjVbXyBmOzmR4dr5JNyyDvHXR.XuX_Dy5iexp9w4BVvIazu4tyu1GuwRJ4BN.Oo.Uld7 IhAVhRpZK240Fw1FfB1R3EShxR32oGu.xW8SpYjNgVkR58T.N43txHAfqU0X8IsZBDjlYHbTY59N o1wqdn9wv9rRVSUGPHE35hmVGsOKLZM.4 Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Tue, 29 Sep 2020 01:29:59 +0000 Received: by smtp404.mail.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID b76510a732ad7d16c16f646e1deefef4; Tue, 29 Sep 2020 01:29:56 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: RPi4B's DMA11 (DMA4 engine example) vs. xHCI/pcie Message-Id: <8C6DE44F-6CE2-4C74-8748-3BBFB54AE183@yahoo.com> Date: Mon, 28 Sep 2020 18:29:56 -0700 To: Robert Crowston , freebsd-arm X-Mailer: Apple Mail (2.3608.120.23.2.1) References: <8C6DE44F-6CE2-4C74-8748-3BBFB54AE183.ref@yahoo.com> X-Rspamd-Queue-Id: 4C0hdj1nwhz45v6 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.56 / 15.00]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.08)[-1.080]; FREEMAIL_TO(0.00)[protonmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.967]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.015]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.206:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.206:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2020 01:30:02 -0000 [Be warned that the material is not familiar so I may need educating. THis is based ont he example context that I happen to have around.] In the u-boot fdt print / output there are 2 distinct sets of dma = channel information, 1 for soc and 1 for scb, where the dma_tag values for the = two sets should be distinct as far as I can tell: U-Boot> fdt address 0x7ef1000 U-Boot> fdt print / =20 / { . . . soc { dma@7e007000 { compatible =3D "brcm,bcm2835-dma"; reg =3D <0x7e007000 0x00000b00>; interrupts =3D * 0x0000000007ef645c = [0x00000084]; interrupt-names =3D "dma0", "dma1", "dma2", = "dma3", "dma4", "dma5", "dma6", "dma7", "dma8", "dma9", "dma10"; #dma-cells =3D <0x00000001>; brcm,dma-channel-mask =3D <0x000001f5>; phandle =3D <0x0000000b>; }; scb { . . . dma@7e007b00 { compatible =3D "brcm,bcm2711-dma"; reg =3D <0x00000000 0x7e007b00 0x00000000 = 0x00000400>; interrupts =3D <0x00000000 0x00000059 0x00000004 = 0x00000000 0x0000005a 0x00000004 0x00000000 0x0000005b 0x00000004 = 0x00000000 0x0000005c 0x00000004>; interrupt-names =3D "dma11", "dma12", "dma13", = "dma14"; #dma-cells =3D <0x00000001>; brcm,dma-channel-mask =3D <0x00007000>; phandle =3D <0x0000003d>; }; . . . So, 0 through 10 need the soc criteria (mix of DMA and DMA LITE engine = criteria) and 11 through 14 need the scb criteria (DMA4 engine criteria). (I'm = ignore dma-channel-mask's at this point.) I'll here note the code has: #define BCM_DMA_CH_MAX 12 for use in code like: /* setup initial settings */ for (i =3D 0; i < BCM_DMA_CH_MAX; i++) { ch =3D &sc->sc_dma_ch[i]; bzero(ch, sizeof(struct bcm_dma_ch)); ch->ch =3D i; ch->flags =3D BCM_DMA_CH_UNMAP; if ((bcm_dma_channel_mask & (1 << i)) =3D=3D 0) continue; . . . It looks to me like the only scb/DMA4-engine "dma11" is covered by such loops and that the "brcm,dma-channel-mask =3D <0x00007000>" means that dma11 will not be used. So: No scb/DMA4 engine will be used??? (That could explain the 1 GiByte limit?) rpi_DATA_2711_1p0.pdf reports that soc/0-10 have 2 types (0-6 vs. 7-10 as it turns out) as well as the scb/DM4-engines (11-14): QUOTE (with omitted marked by ". . .") . . . The BCM2711 DMA Controller provides a total of 16 DMA channels. Four of = these are DMA Lite channels (with reduced performance and features), and = four of them are DMA4 channels (with increased performance and a wider = address range). . . . 4.5. DMA LITE Engines Several of the DMA engines are of the LITE design. This is a reduced = specification engine designed to save space. The engine behaves in the = same way as a normal DMA engine except for the following differences: . . . =E2=80=A2 The DMA length register is now 16 bits, limiting the = maximum transferable length to 65536 bytes. . . . 4.6. DMA4 Engines Several of the DMA engines are of the DMA4 design. These have higher = performance due to their uncoupled read/write design and can access up = to 40 address bits. Unlike the other DMA engines they are also capable = of performing write bursts. Note that they directly access the full = 35-bit address bus of the BCM2711 and so bypass the paging registers of = the DMA and DMA Lite engines. DMA channel 11 is additionally able to access the PCIe interface. END QUOTE The register map indicates (with some extra notes added): 0-6: DMA 7-10: DMA LITE (65536 bytes limit, for example) 11-14: DMA4 (11 is special relative to "PCIe interface") ("DMA Channel 15 is exclusively used by the VPU.") Yet what I see in the head -r365932 code is: #define BCM_DMA_CH_MAX 12 . . . struct bcm_dma_softc { device_t sc_dev; struct mtx sc_mtx; struct resource * sc_mem; struct resource * sc_irq[BCM_DMA_CH_MAX]; void * sc_intrhand[BCM_DMA_CH_MAX]; struct bcm_dma_ch sc_dma_ch[BCM_DMA_CH_MAX]; bus_dma_tag_t sc_dma_tag; }; . . . err =3D bus_dma_tag_create(bus_get_dma_tag(dev), 1, 0, BUS_SPACE_MAXADDR_32BIT, BUS_SPACE_MAXADDR, NULL, NULL, sizeof(struct bcm_dma_cb), 1, sizeof(struct bcm_dma_cb), BUS_DMA_ALLOCNOW, NULL, NULL, &sc->sc_dma_tag); As an example: does that deal with the likes of DMA LITE (so 7-10) = "limiting the maximum transferable length to 65536 bytes"? As another example: Does it deal with the DMA4 (11-14) distinctions (if such were in use anyway)? For reference from the fdt print / : / { . . . #address-cells =3D <0x00000002>; #size-cells =3D <0x00000001>; . . . soc { compatible =3D "simple-bus"; #address-cells =3D <0x00000001>; #size-cells =3D <0x00000001>; . . . dma-ranges =3D <0xc0000000 0x00000000 0x00000000 = 0x40000000>; . . . firmware { compatible =3D "raspberrypi,bcm2835-firmware", = "simple-bus"; mboxes =3D <0x0000001c>; dma-ranges; . . . emmc2bus { compatible =3D "simple-bus"; #address-cells =3D <0x00000002>; #size-cells =3D <0x00000001>; . . . dma-ranges =3D <0x00000000 0xc0000000 0x00000000 = 0x00000000 0x40000000>; . . . scb { compatible =3D "simple-bus"; #address-cells =3D <0x00000002>; #size-cells =3D <0x00000002>; . . . dma-ranges =3D <0x00000000 0x00000000 0x00000000 = 0x00000000 0x00000000 0xfc000000 0x00000001 0x00000000 0x00000001 = 0x00000000 0x00000001 0x00000000>; . . . pcie@7d500000 { compatible =3D "brcm,bcm2711-pcie"; . . . #address-cells =3D <0x00000003>; . . . #size-cells =3D <0x00000002>; . . . dma-ranges =3D <0x02000000 0x00000000 0x00000000 = 0x00000000 0x00000000 0x00000000 0xc0000000>; . . . v3dbus { compatible =3D "simple-bus"; #address-cells =3D <0x00000001>; #size-cells =3D <0x00000002>; . . . dma-ranges =3D <0x00000000 0x00000000 0x00000000 = 0x00000004 0x00000000>; . . . =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Tue Sep 29 02:04:56 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 629793F2379 for ; Tue, 29 Sep 2020 02:04:56 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-1.consmr.mail.bf2.yahoo.com (sonic303-1.consmr.mail.bf2.yahoo.com [74.6.131.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C0jPz02gXz49F5 for ; Tue, 29 Sep 2020 02:04:54 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: VC_p7JYVM1kVqm.H8XXTSxLnrq0p0H8sR2.HgWjFOgZPcn3M5m1X2JMEZKnEIUx RXfKimzFe0loojkEJw.9Tkfh3cFgJoIeu3kLK_reajyreWvCe0u75CWmPGS7MhEvhnPcimRzENmZ lbTH9_LP1Q6XIzcx0jF9fHE4se1xs81w_8ZpDH5z44oVRF8B.2YdOzdgdtGvWbU6c2.jG9UaYDpD krc83PPoLRof2n6XQWr9SKvAiQkW7duQ6ryax6jI85OlHLhrTkfpwlSuQi5TiImnE6isC3TunXbq pr1455EnFuVkGpJSsYaPZ3H4nt7ARWUxn8irs_8SzaYaKc8IXOQ6KnuRCrrpXm8dFJVvvGrexMbr UeAgRRaxrWc1IjMspEBpIFmY08vqwJk0L4IQ9ZP9a49SgqANIm6DlpxFUl6au4qldGMW.H746UR4 toFD0zmxsOrdrE9rWLoJ.AziU5_g5Rg3jOmCEDCIdZptHSeyX_R0RD8_uxLhElaDiW9QnaPVlGo3 eZdWl8YTomcjb.FfouwM9A07gPilKA.hji_XuBEywOMkDoo.SwgM6eqURuOwH_fuZ4dl1SGI3RGJ OIxwhsh2WnOjIqv0k5._A6LQh8N.iWM6uOCPTgrqC5kkVT2XMxVJzdF39z2wejJk_KnUtJ24p75O sJm6Cq4zvjo5JcuqqHh4q.1P8cj0rrxvLr2htgwafBjuqhXIOTvaBXTKMCGqqY3Fbj8NaHGzIS5u Rq4ob08xv.T7tlF8iKSSzjf4P1XCS1I7ehrs4vPwUkfcXx_yqN_GYc.TLW7vzaQuQxkusfIe9pu4 0s.jqbg6F9eWLhzUMZMH7LnH0mSn5ztPxNnHkx.iTj_OP2XHuRaS9vz.Uy35YTWlpEBStK93fiMc XCIrD86O9QEh2jYfoyZ4uL1z2P6BjOfN9zA9eTwRisp8joI3zNvmmWfX9JD9evIYhsslZtNIfbfc nk0VSFrJN0Z46dxhnHUFW5x.s_rzeQd91PdGX2ZZzGgkl3CWa4IwNwptpyGL4XAohE5reYk0oabP ryNxf8Au_YJZkFqi7FvHJtJkGTokJEKHOR831ElzXyUDcZA74T7le1GSvWoRbIG4QAdk9fXfHC5o MYbVjGKExZP.odfnXHB_fInUfm2oXB98Zs2BBLnA4uW4RHPZ9N2apB6.wG4sz1enjTkbGMFiTAvV 9iX1cr5j5iAre_Bg10hXgHUWFuFOaon0h7mgoghCbXqzCOk0OlaUFdVxx5M7TaANWUTuT_XLaB7Q ckShlQ45gyR1N6_Hb5UB4V_QVSX8JEYDjYUGQIobrHdyS6QfEnGsfHfiRQTUCaZbxPh4h3X4PXNF KF_PN9msohouQZYMkRKDyabKV0l83pVXhfI2MJFItr2js3x_BKBC8JewYSRkYQZhiYvSK73CxgDL vl6bXSlMDUwQRCAaqcLRQvmGXWFZaEiFAaKmKAbgsa2nITmhxwN1GjbRv.1.YNPdg1zlUYnu7HVo XHcL9zK1QsxPYM.Y72KEo6f9CoFNjeUb5LAlyPAXPH_1cI8rR_G.7Nnn9Fo8HOm.eTs1Onccm6Rq xzDgcOkZQOl74ZPk1MQ2ZNrARbfyNTK1XOg-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.bf2.yahoo.com with HTTP; Tue, 29 Sep 2020 02:04:53 +0000 Received: by smtp421.mail.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID eeccf43183dd7720be4fdaa356cc7fa8; Tue, 29 Sep 2020 02:04:50 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: RPi4B's DMA11 (DMA4 engine example) vs. xHCI/pcie Date: Mon, 28 Sep 2020 19:04:48 -0700 References: <8C6DE44F-6CE2-4C74-8748-3BBFB54AE183@yahoo.com> To: Robert Crowston , freebsd-arm In-Reply-To: <8C6DE44F-6CE2-4C74-8748-3BBFB54AE183@yahoo.com> Message-Id: <0FE382AB-8DE3-4467-9CB0-E8582AC70EA2@yahoo.com> X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Rspamd-Queue-Id: 4C0jPz02gXz49F5 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.56 / 15.00]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.07)[-1.074]; FREEMAIL_TO(0.00)[protonmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.970]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.02)[-1.017]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[74.6.131.40:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[74.6.131.40:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2020 02:04:56 -0000 On 2020-Sep-28, at 18:29, Mark Millard wrote: > [Be warned that the material is not familiar so I may need > educating. THis is based ont he example context that I > happen to have around.] >=20 > In the u-boot fdt print / output there are 2 distinct sets of dma = channel > information, 1 for soc and 1 for scb, where the dma_tag values for the = two > sets should be distinct as far as I can tell: >=20 > U-Boot> fdt address 0x7ef1000 > U-Boot> fdt print / =20 > / { > . . . > soc { > dma@7e007000 { > compatible =3D "brcm,bcm2835-dma"; > reg =3D <0x7e007000 0x00000b00>; > interrupts =3D * 0x0000000007ef645c = [0x00000084]; > interrupt-names =3D "dma0", "dma1", "dma2", = "dma3", "dma4", "dma5", "dma6", "dma7", "dma8", "dma9", "dma10"; > #dma-cells =3D <0x00000001>; > brcm,dma-channel-mask =3D <0x000001f5>; > phandle =3D <0x0000000b>; > }; >=20 > scb { > . . . > dma@7e007b00 { > compatible =3D "brcm,bcm2711-dma"; > reg =3D <0x00000000 0x7e007b00 0x00000000 = 0x00000400>; > interrupts =3D <0x00000000 0x00000059 = 0x00000004 0x00000000 0x0000005a 0x00000004 0x00000000 0x0000005b = 0x00000004 0x00000000 0x0000005c 0x00000004>; > interrupt-names =3D "dma11", "dma12", "dma13", = "dma14"; > #dma-cells =3D <0x00000001>; > brcm,dma-channel-mask =3D <0x00007000>; > phandle =3D <0x0000003d>; > }; > . . . >=20 > So, 0 through 10 need the soc criteria (mix of DMA and DMA LITE = engine criteria) > and 11 through 14 need the scb criteria (DMA4 engine criteria). (I'm = ignore > dma-channel-mask's at this point.) >=20 >=20 > I'll here note the code has: >=20 > #define BCM_DMA_CH_MAX 12 >=20 > for use in code like: >=20 > /* setup initial settings */ > for (i =3D 0; i < BCM_DMA_CH_MAX; i++) { > ch =3D &sc->sc_dma_ch[i]; >=20 > bzero(ch, sizeof(struct bcm_dma_ch)); > ch->ch =3D i; > ch->flags =3D BCM_DMA_CH_UNMAP; >=20 > if ((bcm_dma_channel_mask & (1 << i)) =3D=3D 0) > continue; > . . . >=20 > It looks to me like the only scb/DMA4-engine "dma11" is covered > by such loops and that the "brcm,dma-channel-mask =3D <0x00007000>" > means that dma11 will not be used. >=20 > So: No scb/DMA4 engine will be used??? (That could explain the > 1 GiByte limit?) >=20 >=20 > rpi_DATA_2711_1p0.pdf reports that soc/0-10 have 2 types (0-6 vs. 7-10 > as it turns out) as well as the scb/DM4-engines (11-14): >=20 > QUOTE (with omitted marked by ". . .") > . . . > The BCM2711 DMA Controller provides a total of 16 DMA channels. Four = of these are DMA Lite channels (with reduced performance and features), = and four of them are DMA4 channels (with increased performance and a = wider address range). > . . . > 4.5. DMA LITE Engines >=20 > Several of the DMA engines are of the LITE design. This is a reduced = specification engine designed to save space. The engine behaves in the = same way as a normal DMA engine except for the following differences: > . . . > =E2=80=A2 The DMA length register is now 16 bits, limiting the = maximum transferable length to 65536 bytes. > . . . > 4.6. DMA4 Engines >=20 > Several of the DMA engines are of the DMA4 design. These have higher = performance due to their uncoupled read/write design and can access up = to 40 address bits. Unlike the other DMA engines they are also capable = of performing write bursts. Note that they directly access the full = 35-bit address bus of the BCM2711 and so bypass the paging registers of = the DMA and DMA Lite engines. >=20 > DMA channel 11 is additionally able to access the PCIe interface. > END QUOTE >=20 > The register map indicates (with some extra notes added): >=20 > 0-6: DMA > 7-10: DMA LITE (65536 bytes limit, for example) > 11-14: DMA4 (11 is special relative to "PCIe interface") > ("DMA Channel 15 is exclusively used by the VPU.") >=20 > Yet what I see in the head -r365932 code is: >=20 > #define BCM_DMA_CH_MAX 12 > . . . > struct bcm_dma_softc { > device_t sc_dev; > struct mtx sc_mtx; > struct resource * sc_mem; > struct resource * sc_irq[BCM_DMA_CH_MAX]; > void * sc_intrhand[BCM_DMA_CH_MAX]; > struct bcm_dma_ch sc_dma_ch[BCM_DMA_CH_MAX]; > bus_dma_tag_t sc_dma_tag; > }; > . . . > err =3D bus_dma_tag_create(bus_get_dma_tag(dev), > 1, 0, BUS_SPACE_MAXADDR_32BIT, > BUS_SPACE_MAXADDR, NULL, NULL, > sizeof(struct bcm_dma_cb), 1, > sizeof(struct bcm_dma_cb), > BUS_DMA_ALLOCNOW, NULL, NULL, > &sc->sc_dma_tag); >=20 > As an example: does that deal with the likes of DMA LITE (so 7-10) = "limiting > the maximum transferable length to 65536 bytes"? >=20 > As another example: Does it deal with the DMA4 (11-14) distinctions = (if > such were in use anyway)? >=20 > For reference from the fdt print / : >=20 > / { > . . . > #address-cells =3D <0x00000002>; > #size-cells =3D <0x00000001>; > . . . > soc { > compatible =3D "simple-bus"; > #address-cells =3D <0x00000001>; > #size-cells =3D <0x00000001>; > . . . > dma-ranges =3D <0xc0000000 0x00000000 0x00000000 = 0x40000000>; > . . . > firmware { > compatible =3D "raspberrypi,bcm2835-firmware", = "simple-bus"; > mboxes =3D <0x0000001c>; > dma-ranges; > . . . > emmc2bus { > compatible =3D "simple-bus"; > #address-cells =3D <0x00000002>; > #size-cells =3D <0x00000001>; > . . . > dma-ranges =3D <0x00000000 0xc0000000 0x00000000 = 0x00000000 0x40000000>; > . . . > scb { > compatible =3D "simple-bus"; > #address-cells =3D <0x00000002>; > #size-cells =3D <0x00000002>; > . . . > dma-ranges =3D <0x00000000 0x00000000 0x00000000 = 0x00000000 0x00000000 0xfc000000 0x00000001 0x00000000 0x00000001 = 0x00000000 0x00000001 0x00000000>; > . . . > pcie@7d500000 { > compatible =3D "brcm,bcm2711-pcie"; > . . . > #address-cells =3D <0x00000003>; > . . . > #size-cells =3D <0x00000002>; > . . . > dma-ranges =3D <0x02000000 0x00000000 = 0x00000000 0x00000000 0x00000000 0x00000000 0xc0000000>; > . . . > v3dbus { > compatible =3D "simple-bus"; > #address-cells =3D <0x00000001>; > #size-cells =3D <0x00000002>; > . . . > dma-ranges =3D <0x00000000 0x00000000 0x00000000 = 0x00000004 0x00000000>; > . . . rpi_DATA_2711_1p0.pdf reports: (I ignore 2D DMA transfer mode here.) For DMA engines 0-6: XLENGTH has bits 29:0 bits 31:30 are write as 0, read as do not care. That would put maxsegsz as 2**30 =3D=3D 1,073,741,824 which matches a 1 GiByte space. For DMA LITE engines 7-10: XLENGTH has bit 15:0 bits 31:16 are write as 0, read as do not care. That would put maxsegsz as 2**16 =3D=3D 65,536. For DMA4 engines 11-14: XLENGTH has bits 29:0 bits 31:30 are write as 0, read as do not care. That would put maxsegsz as 2**30 =3D=3D 1,073,741,824 which is smaller than the 3 GiByte space associated with xHCI. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Tue Sep 29 04:45:37 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A205E3F589D for ; Tue, 29 Sep 2020 04:45:37 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-56.consmr.mail.bf2.yahoo.com (sonic304-56.consmr.mail.bf2.yahoo.com [74.6.128.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C0mzN1qXHz4J0d for ; Tue, 29 Sep 2020 04:45:36 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: hLfSroEVM1ky0wBinR6T.SemDAP6x383zNai7Yc9098d7gGbBb7RPGfTw3W46_4 _.7sXsCM2TJ8j7OTz.zMq9T98D85rOOhNBbG3iI03sioJ4Sogd2fqBSo9MlUNs3re1ChAVXrzH5. raZhyiY1fknJXze5ybrx4CTsAjsW8B_ZHHVVl3ijWIRrNmUZLCtbF_JPk0u_btPv8Kuov4vPCXUN f_d484s4e6aGR1OYKRzjj8WHN_cA0h1ZZ14KwrQjMZ1GHXlizPIS8tKIyuXGQ72S4xB_xjCzzmxE YZYHddYDK2.9Nk0KdhDGhLQdidbWp2O4uVNIsXDHpFJFIwflgwb90PkTJUHM6L9CtLMY2cyQlDfy 43Cnh_DnYlt4ZpkH1sgwSbQ3sz.soCS.tYVU9ZR76hvqGjZAEYV5jenwhxXgVv0DgpTkHlzH5Y02 GvuRbWIss8II5pdMDvDBGV6l6P5AZfd3x96IBCvl7mK5i7_Nzz8xmUQzh5MnbtNIL9OTuDqi1hdF yRv5CkBNeG0iDTU7pEBSuPApPZL.qb3dCFTH8H5tJrwDdx5DQin8tCw.S8HWZInJrWjBzmMgcOJJ STmkoMGRUe7s_z0cqKEJ58hXznLGHu62.pDxc4ttK_9X.tcWiErNJtOA3nQDVneNUGLY1Zv_qkDg 4OU4MeaG242g6dbZKzIoCkLp3_4ALoGzFZw6LmRkuev6r_9mBR9FyojgnxLDwLqFmBwnzrCby_vX ErIo1Uh99RknvUX5Sujo1H_K92Zd_KYeMHzOOISPukKNDcvAPjDwnr5P2eHO7C9k3jdMAwcLQnyc eWucMWvrAf6eJupK_eocgzm.y49mzK7TtA4IDNFQ6xzuIjeXqa0JLOXenXwk4IBFVMnspblS7TX6 _DoZnfAYaWQ_5LVAKWzMIgFyyEEHRgI4jPLsyssh2lmx2AyAb1XIOE3xJBf_XiNqMJ0DBpfUZbQ4 .Wh8.rxnaTmzV1R31pfl9ZxDhO.b7VX3JjSw_n5YvrBWfdxC_k5mAfy5yLZ94emmho9ni_TcocoS e4PgsC3hYJqiBfMqCMAyn6oNdJN24BWuJkndSHyZHDEXPgFNRTlyTlF.VnP4JQBH8sGVi2MJiGD7 4iVqenv_Xm2bpodSu7EtZJ5dx7I4.g0EpeUxGJbNHodRL1yK1mXFOmYjpZD2Yuz1ksP0OyhvpZQp RFqOB2owTOpdeMiS6LQpKi4myoPZ6gm.M8s5lyL8Bz.FLnVBtM6e0qlcqeU8Fdv2lp9MAXtGuh1T 7JSVJdKVFZHC.TZ6OeLCMWvfGncohlvGj5b.emeGwSrSoyn9Ikyeplgxe2_RV3l9s5pTTbBnGesi AAsDvAT_cxKojrquD.woTfuiDXEiu3rvrhKYzBMTxnYtEtB9xZDv39VTmrzSu5JyIIspKSQN8Oge e_2bWB1wFSamgJL9G4jmCOAgB7sJmmZASNar25iYyURHJghmgiVUtfwNkgkQyMLmrmyE_NeczENF .DNTPgGyHyc6YMYtPcb9TfIibi3GQVb4tfKJzfvG1makcWLwl1q_41.OMnY399H6BN__pDSzsNpw 5fVCUGTb.DTekidPkvDAWoO0J_W9JiCsB Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.bf2.yahoo.com with HTTP; Tue, 29 Sep 2020 04:45:34 +0000 Received: by smtp417.mail.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 12e0997a853a7607a1ca8210cb24f76e; Tue, 29 Sep 2020 04:45:28 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: RPi4B's DMA11 (DMA4 engine example) vs. xHCI/pcie Date: Mon, 28 Sep 2020 21:45:27 -0700 References: <8C6DE44F-6CE2-4C74-8748-3BBFB54AE183@yahoo.com> <0FE382AB-8DE3-4467-9CB0-E8582AC70EA2@yahoo.com> To: Robert Crowston , freebsd-arm In-Reply-To: <0FE382AB-8DE3-4467-9CB0-E8582AC70EA2@yahoo.com> Message-Id: <85FEDC51-B5B0-4ED4-A5ED-62B63EF9D5A8@yahoo.com> X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Rspamd-Queue-Id: 4C0mzN1qXHz4J0d X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.17 / 15.00]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.68)[-0.684]; FREEMAIL_TO(0.00)[protonmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.968]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.02)[-1.015]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[74.6.128.31:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[74.6.128.31:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2020 04:45:37 -0000 On 2020-Sep-28, at 19:04, Mark Millard wrote: > On 2020-Sep-28, at 18:29, Mark Millard wrote: >>=20 >>> [Be warned that the material is not familiar so I may need >>> educating. THis is based ont he example context that I >>> happen to have around.] >>>=20 >>> In the u-boot fdt print / output there are 2 distinct sets of dma = channel >>> information, 1 for soc and 1 for scb, where the dma_tag values for = the two >>> sets should be distinct as far as I can tell: >>>=20 >>> U-Boot> fdt address 0x7ef1000 >>> U-Boot> fdt print / =20 >>> / { >>> . . . >>> soc { >>> dma@7e007000 { >>> compatible =3D "brcm,bcm2835-dma"; >>> reg =3D <0x7e007000 0x00000b00>; >>> interrupts =3D * 0x0000000007ef645c = [0x00000084]; >>> interrupt-names =3D "dma0", "dma1", "dma2", = "dma3", "dma4", "dma5", "dma6", "dma7", "dma8", "dma9", "dma10"; >>> #dma-cells =3D <0x00000001>; >>> brcm,dma-channel-mask =3D <0x000001f5>; >>> phandle =3D <0x0000000b>; >>> }; >>>=20 >>> scb { >>> . . . >>> dma@7e007b00 { >>> compatible =3D "brcm,bcm2711-dma"; >>> reg =3D <0x00000000 0x7e007b00 0x00000000 = 0x00000400>; >>> interrupts =3D <0x00000000 0x00000059 = 0x00000004 0x00000000 0x0000005a 0x00000004 0x00000000 0x0000005b = 0x00000004 0x00000000 0x0000005c 0x00000004>; >>> interrupt-names =3D "dma11", "dma12", "dma13", = "dma14"; >>> #dma-cells =3D <0x00000001>; >>> brcm,dma-channel-mask =3D <0x00007000>; >>> phandle =3D <0x0000003d>; >>> }; >>> . . . >>>=20 >>> So, 0 through 10 need the soc criteria (mix of DMA and DMA LITE = engine criteria) >>> and 11 through 14 need the scb criteria (DMA4 engine criteria). (I'm = ignore >>> dma-channel-mask's at this point.) >>>=20 >>>=20 >>> I'll here note the code has: >>>=20 >>> #define BCM_DMA_CH_MAX 12 >>>=20 >>> for use in code like: >>>=20 >>> /* setup initial settings */ >>> for (i =3D 0; i < BCM_DMA_CH_MAX; i++) { >>> ch =3D &sc->sc_dma_ch[i]; >>>=20 >>> bzero(ch, sizeof(struct bcm_dma_ch)); >>> ch->ch =3D i; >>> ch->flags =3D BCM_DMA_CH_UNMAP; >>>=20 >>> if ((bcm_dma_channel_mask & (1 << i)) =3D=3D 0) >>> continue; >>> . . . >>>=20 >>> It looks to me like the only scb/DMA4-engine "dma11" is covered >>> by such loops and that the "brcm,dma-channel-mask =3D <0x00007000>" >>> means that dma11 will not be used. >>>=20 >>> So: No scb/DMA4 engine will be used??? (That could explain the >>> 1 GiByte limit?) >>>=20 >>>=20 >>> rpi_DATA_2711_1p0.pdf reports that soc/0-10 have 2 types (0-6 vs. = 7-10 >>> as it turns out) as well as the scb/DM4-engines (11-14): >>>=20 >>> QUOTE (with omitted marked by ". . .") >>> . . . >>> The BCM2711 DMA Controller provides a total of 16 DMA channels. Four = of these are DMA Lite channels (with reduced performance and features), = and four of them are DMA4 channels (with increased performance and a = wider address range). >>> . . . >>> 4.5. DMA LITE Engines >>>=20 >>> Several of the DMA engines are of the LITE design. This is a reduced = specification engine designed to save space. The engine behaves in the = same way as a normal DMA engine except for the following differences: >>> . . . >>> =E2=80=A2 The DMA length register is now 16 bits, limiting the = maximum transferable length to 65536 bytes. >>> . . . >>> 4.6. DMA4 Engines >>>=20 >>> Several of the DMA engines are of the DMA4 design. These have higher = performance due to their uncoupled read/write design and can access up = to 40 address bits. Unlike the other DMA engines they are also capable = of performing write bursts. Note that they directly access the full = 35-bit address bus of the BCM2711 and so bypass the paging registers of = the DMA and DMA Lite engines. >>>=20 >>> DMA channel 11 is additionally able to access the PCIe interface. >>> END QUOTE >>>=20 >>> The register map indicates (with some extra notes added): >>>=20 >>> 0-6: DMA >>> 7-10: DMA LITE (65536 bytes limit, for example) >>> 11-14: DMA4 (11 is special relative to "PCIe interface") >>> ("DMA Channel 15 is exclusively used by the VPU.") >>>=20 >>> Yet what I see in the head -r365932 code is: >>>=20 >>> #define BCM_DMA_CH_MAX 12 >>> . . . >>> struct bcm_dma_softc { >>> device_t sc_dev; >>> struct mtx sc_mtx; >>> struct resource * sc_mem; >>> struct resource * sc_irq[BCM_DMA_CH_MAX]; >>> void * sc_intrhand[BCM_DMA_CH_MAX]; >>> struct bcm_dma_ch sc_dma_ch[BCM_DMA_CH_MAX]; >>> bus_dma_tag_t sc_dma_tag; >>> }; >>> . . . >>> err =3D bus_dma_tag_create(bus_get_dma_tag(dev), >>> 1, 0, BUS_SPACE_MAXADDR_32BIT, >>> BUS_SPACE_MAXADDR, NULL, NULL, >>> sizeof(struct bcm_dma_cb), 1, >>> sizeof(struct bcm_dma_cb), >>> BUS_DMA_ALLOCNOW, NULL, NULL, >>> &sc->sc_dma_tag); >>>=20 >>> As an example: does that deal with the likes of DMA LITE (so 7-10) = "limiting >>> the maximum transferable length to 65536 bytes"? >>>=20 >>> As another example: Does it deal with the DMA4 (11-14) distinctions = (if >>> such were in use anyway)? >>>=20 >>> For reference from the fdt print / : >>>=20 >>> / { >>> . . . >>> #address-cells =3D <0x00000002>; >>> #size-cells =3D <0x00000001>; >>> . . . >>> soc { >>> compatible =3D "simple-bus"; >>> #address-cells =3D <0x00000001>; >>> #size-cells =3D <0x00000001>; >>> . . . >>> dma-ranges =3D <0xc0000000 0x00000000 0x00000000 = 0x40000000>; >>> . . . >>> firmware { >>> compatible =3D "raspberrypi,bcm2835-firmware", = "simple-bus"; >>> mboxes =3D <0x0000001c>; >>> dma-ranges; >>> . . . >>> emmc2bus { >>> compatible =3D "simple-bus"; >>> #address-cells =3D <0x00000002>; >>> #size-cells =3D <0x00000001>; >>> . . . >>> dma-ranges =3D <0x00000000 0xc0000000 0x00000000 = 0x00000000 0x40000000>; >>> . . . >>> scb { >>> compatible =3D "simple-bus"; >>> #address-cells =3D <0x00000002>; >>> #size-cells =3D <0x00000002>; >>> . . . >>> dma-ranges =3D <0x00000000 0x00000000 0x00000000 = 0x00000000 0x00000000 0xfc000000 0x00000001 0x00000000 0x00000001 = 0x00000000 0x00000001 0x00000000>; >>> . . . >>> pcie@7d500000 { >>> compatible =3D "brcm,bcm2711-pcie"; >>> . . . >>> #address-cells =3D <0x00000003>; >>> . . . >>> #size-cells =3D <0x00000002>; >>> . . . >>> dma-ranges =3D <0x02000000 0x00000000 = 0x00000000 0x00000000 0x00000000 0x00000000 0xc0000000>; >>> . . . >>> v3dbus { >>> compatible =3D "simple-bus"; >>> #address-cells =3D <0x00000001>; >>> #size-cells =3D <0x00000002>; >>> . . . >>> dma-ranges =3D <0x00000000 0x00000000 0x00000000 = 0x00000004 0x00000000>; >>> . . . >>=20 >> rpi_DATA_2711_1p0.pdf reports: >> (I ignore 2D DMA transfer mode here.) >>=20 >> For DMA engines 0-6: XLENGTH has bits 29:0 >> bits 31:30 are write as 0, read as do not care. >> That would put maxsegsz as 2**30 =3D=3D 1,073,741,824 >> which matches a 1 GiByte space. >>=20 >> For DMA LITE engines 7-10: XLENGTH has bit 15:0 >> bits 31:16 are write as 0, read as do not care. >> That would put maxsegsz as 2**16 =3D=3D 65,536. >>=20 >> For DMA4 engines 11-14: XLENGTH has bits 29:0 >> bits 31:30 are write as 0, read as do not care. >> That would put maxsegsz as 2**30 =3D=3D 1,073,741,824 >> which is smaller than the 3 GiByte space associated >> with xHCI. rpi_DATA_2711_1p0.pdf reports the following specifically for DMA11-DMA14 (so the DMA4 engines) for what goes in the CB and NEXT_CB ADDR fields: QUOTE The address must be 256-bit aligned and so the bottom 5 bits of the byte = address are discarded, i.e. write cb_byte_address[39:0]>>5 into the CB END QUOTE This is not true for DMA0-DMA10 (DMA and DMA LITE). The following is extracted from various places to bring them together. I do not see evidence of handling the cb_byte_address[39:0]>>5 involved for DMA11-DMA14: #define ARMC_TO_VCBUS(pa) bcm283x_armc_to_vcbus(pa) vm_paddr_t bcm283x_armc_to_vcbus(vm_paddr_t pa) { struct bcm283x_memory_soc_cfg *cfg; struct bcm283x_memory_mapping *map, *ment; =20 /* Guaranteed not NULL if we haven't panicked yet. */ cfg =3D bcm283x_get_current_memcfg(); map =3D cfg->memmap; for (ment =3D map; !BCM283X_MEMMAP_ISTERM(ment); ++ment) { if (pa >=3D ment->armc_start && pa < ment->armc_start + ment->armc_size) { return (pa - ment->armc_start) + = ment->vcbus_start; } } /* * Assume 1:1 mapping for anything else, but complain about it = on * verbose boots. */ if (bootverbose) printf("bcm283x_vcbus: No armc -> vcbus mapping found: = %jx\n", (uintmax_t)pa); return (pa); } static void bcm_dmamap_cb(void *arg, bus_dma_segment_t *segs, int nseg, int err) { bus_addr_t *addr; if (err) return; addr =3D (bus_addr_t*)arg; *addr =3D ARMC_TO_VCBUS(segs[0].ds_addr); } Note ds_addr assignments in: static bus_size_t _bus_dmamap_addseg(bus_dma_tag_t dmat, bus_dmamap_t map, bus_addr_t = curaddr, bus_size_t sgsize, bus_dma_segment_t *segs, int *segp) { bus_addr_t baddr, bmask; int seg; =20 /* * Make sure we don't cross any boundaries. */ bmask =3D ~(dmat->common.boundary - 1); if (dmat->common.boundary > 0) { baddr =3D (curaddr + dmat->common.boundary) & bmask; if (sgsize > (baddr - curaddr)) sgsize =3D (baddr - curaddr); } =20 /* * Insert chunk into a segment, coalescing with * previous segment if possible. */ seg =3D *segp; if (seg =3D=3D -1) { seg =3D 0; segs[seg].ds_addr =3D curaddr; segs[seg].ds_len =3D sgsize; } else { if (curaddr =3D=3D segs[seg].ds_addr + segs[seg].ds_len = && (segs[seg].ds_len + sgsize) <=3D = dmat->common.maxsegsz && (dmat->common.boundary =3D=3D 0 || (segs[seg].ds_addr & bmask) =3D=3D (curaddr & = bmask))) segs[seg].ds_len +=3D sgsize; else { if (++seg >=3D dmat->common.nsegments) return (0); segs[seg].ds_addr =3D curaddr; segs[seg].ds_len =3D sgsize; } } *segp =3D seg; return (sgsize); } Note cb_phys and ch->vc_cb in: static int bcm_dma_init(device_t dev) { . . . /* setup initial settings */ for (i =3D 0; i < BCM_DMA_CH_MAX; i++) { . . . err =3D bus_dmamap_load(sc->sc_dma_tag, ch->dma_map, = cb_virt, sizeof(struct bcm_dma_cb), bcm_dmamap_cb, &cb_phys, BUS_DMA_WAITOK); if (err) { device_printf(dev, "cannot load DMA memory\n"); break; } ch->cb =3D cb_virt; ch->vc_cb =3D cb_phys; . . . int bcm_dma_start(int ch, vm_paddr_t src, vm_paddr_t dst, int len) { struct bcm_dma_softc *sc =3D bcm_dma_sc; struct bcm_dma_cb *cb; if (ch < 0 || ch >=3D BCM_DMA_CH_MAX) return (-1); =20 if (!(sc->sc_dma_ch[ch].flags & BCM_DMA_CH_USED)) return (-1); =20 cb =3D sc->sc_dma_ch[ch].cb; cb->src =3D ARMC_TO_VCBUS(src); cb->dst =3D ARMC_TO_VCBUS(dst); =20 cb->len =3D len; =20 bus_dmamap_sync(sc->sc_dma_tag, sc->sc_dma_ch[ch].dma_map, BUS_DMASYNC_PREWRITE); =20 bus_write_4(sc->sc_mem, BCM_DMA_CBADDR(ch), sc->sc_dma_ch[ch].vc_cb); bus_write_4(sc->sc_mem, BCM_DMA_CS(ch), CS_ACTIVE); =20 #ifdef DEBUG bcm_dma_cb_dump(sc->sc_dma_ch[ch].cb); bcm_dma_reg_dump(ch); #endif return (0); } It looks to me like FreeBSD is not set up to use the DMA4 engines (DMA11-DMA14) and happens to not use them for the DTB that I get from u-boot.bin in my context. Of course, I may just have missed something in looking around at the unfamiliar material. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Tue Sep 29 17:35:33 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4B2F542AF49 for ; Tue, 29 Sep 2020 17:35:33 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-14.consmr.mail.bf2.yahoo.com (sonic309-14.consmr.mail.bf2.yahoo.com [74.6.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C163m4qMxz3f0Y for ; Tue, 29 Sep 2020 17:35:32 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: TiOkI4UVM1lLEP5EfwISj0beynCX9G9dMC.VW3yM9usR2md5x_wAqk67vJfjy_H SWLD_mf6hhRV4hF955jm9Wz6tDBxKnyJcOSqoWcolqxErKlkc2ou8JJmVwM1ZzMjQ.VcWktK.2SG Yhi2VDSlYYdner.9__OT3FpfUZAdq6pK1fOrBy43DOcyMj9XlaJCI4x.rcpyk7uhsXyPy7MjPFkr cfoFAsdqgw7RPn8rDhOmDS8lzrYEivjSwRFNa7eX5PIsqmILuuT8z1H2hMp.EaZf340BoIK3oUZN jCIjdkiZNA_7.HrnXbwHpFp1.FvXLFxTCrTkywI0GlQfSaPrsyFYO.G2fzp3W8RjImALQu9tMDTA puwtfEqQ9MIAh9xSbUeqskzk.g8NN7ivF4eFB9AS5bnL1UacnVkHt74ooWlD8xNojQaPHBZQXDNk u8c1Hwj.5J1iSkCuwn6Iaz9zgykGG33ldakOvaASMQBY4BX6Pc8qbpI4VRxGQLV4mcLFNFRhyBZl fGSfed.m4tnuYfZwX6ILmfyLIkYBsqpYH404ppbMLZAs3LPlDzHUHjPw3wo0PVC6fE56Yac..KVq lHUFwdtjS7Ohxt6OBybaSThna.yUMf8C4QrJOvpodCP0LXXaCWTsIHoVN2n6DiE2UVQU1VcxiOUV ZKaFzWm6iqZtScxmf3RiEUhClV9tIGfw0UKRKY4xGx7.u1zHN.1KujUjrm.tRu6oSstTd_mpbt__ L.V7YywiAQJctDeoXJ0nNOqr1.BcZwnXLNVhcUOUwYbiURhvzuJmQ88DcbRysbXcujZmdn2s7EKX vvvqnj0v1xJ8F94mG1PmQ2HsVJ3qZjIsSqHcOICP4hlJ6C3BLvOKGWf0l7XKbm7UXMXj0hqEuQb4 lpJwWomHZoLwLqhoXO26r9wTykOENp51EBaHl04GO83sXazTTCAItp2mbobzGEvj5JLWG_3lEeRc 1BFkXs74NuyuNH3nNppexQpdvEXKOxx6ruV9PriJQQLdJJ6RQKMG1Vkw9lXn40wwNtA6tnXDZbQc GL7glJcUasoXfFDGOKtCcIw5_iQL7DCCmR0ooUC.nvqfNtYZ343Yp29bZkOzTO_79X.tnPgguIaI X8S2PCVP3mQddVM.wF7Ts_5Z3GSOMAuhV7OfzcaQxTqYR2NqVEmzfBcj8qJr31xcxiJlaTVwEM7F 1Iir9hZ.c_Kj7_f5FsbyJrXtCncGVUzPLITemU4PC5kDhK547AFHO0pmyxzS0WSEMSFnBX2kxBI8 UapMK2R7n2og9yvOTzWfm1aBWYzkXvoKjW9g95b1iCr4g5wXxmlFP6Hc_EuxbMbD066jDV1fCjg4 0_yPvP93640TK1OUdnIAZDDFdLdS3mktMblO1M73eUTD0phXcXyHv8XhQfgXvUM.vCHlgmmDaM8Z Yn9n8kFZeIO6miVjGxAyi3pIoo2nLmqu2sEI0EplPDdPro0hWz7R5uk4pR5VFhpzjrDFC5QMYnZG Y90X8R.pp0fuoPG.1XK2VD4PdCoNXLyuiHmLJeXM66UZfyu8nzTVjjysAi2WZjVMG4lxRez8Hg48 - Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.bf2.yahoo.com with HTTP; Tue, 29 Sep 2020 17:35:31 +0000 Received: by smtp411.mail.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID d47a0348dcb3560f51bc7ef5290366a6; Tue, 29 Sep 2020 17:35:29 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: RPi4B's DMA11 (DMA4 engine example) vs. xHCI/pcie Date: Tue, 29 Sep 2020 10:35:28 -0700 References: <8C6DE44F-6CE2-4C74-8748-3BBFB54AE183@yahoo.com> <0FE382AB-8DE3-4467-9CB0-E8582AC70EA2@yahoo.com> <85FEDC51-B5B0-4ED4-A5ED-62B63EF9D5A8@yahoo.com> To: Robert Crowston , freebsd-arm In-Reply-To: <85FEDC51-B5B0-4ED4-A5ED-62B63EF9D5A8@yahoo.com> Message-Id: X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Rspamd-Queue-Id: 4C163m4qMxz3f0Y X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.80 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.26)[-1.262]; FREEMAIL_TO(0.00)[protonmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.02)[-1.019]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.02)[-1.016]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[74.6.129.124:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[74.6.129.124:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2020 17:35:33 -0000 On 2020-Sep-28, at 21:45, Mark Millard wrote: > On 2020-Sep-28, at 19:04, Mark Millard wrote: >=20 >> On 2020-Sep-28, at 18:29, Mark Millard wrote: >>>=20 >>>> [Be warned that the material is not familiar so I may need >>>> educating. THis is based ont he example context that I >>>> happen to have around.] >>>>=20 >>>> In the u-boot fdt print / output there are 2 distinct sets of dma = channel >>>> information, 1 for soc and 1 for scb, where the dma_tag values for = the two >>>> sets should be distinct as far as I can tell: >>>>=20 >>>> U-Boot> fdt address 0x7ef1000 >>>> U-Boot> fdt print / =20 >>>> / { >>>> . . . >>>> soc { >>>> dma@7e007000 { >>>> compatible =3D "brcm,bcm2835-dma"; >>>> reg =3D <0x7e007000 0x00000b00>; >>>> interrupts =3D * 0x0000000007ef645c = [0x00000084]; >>>> interrupt-names =3D "dma0", "dma1", "dma2", = "dma3", "dma4", "dma5", "dma6", "dma7", "dma8", "dma9", "dma10"; >>>> #dma-cells =3D <0x00000001>; >>>> brcm,dma-channel-mask =3D <0x000001f5>; >>>> phandle =3D <0x0000000b>; >>>> }; >>>>=20 >>>> scb { >>>> . . . >>>> dma@7e007b00 { >>>> compatible =3D "brcm,bcm2711-dma"; >>>> reg =3D <0x00000000 0x7e007b00 0x00000000 = 0x00000400>; >>>> interrupts =3D <0x00000000 0x00000059 = 0x00000004 0x00000000 0x0000005a 0x00000004 0x00000000 0x0000005b = 0x00000004 0x00000000 0x0000005c 0x00000004>; >>>> interrupt-names =3D "dma11", "dma12", "dma13", = "dma14"; >>>> #dma-cells =3D <0x00000001>; >>>> brcm,dma-channel-mask =3D <0x00007000>; >>>> phandle =3D <0x0000003d>; >>>> }; >>>> . . . I had presumed that the dma@7e007b00 would be processed. But I finally happened to search for "bcm2711-dma" in FreeBSD and it does not occur. That appears to mean that BCM_DMA_CH_MAX being 12 is depending on dma@7e007000's brcm,dma-channel-mask to avoid referencing number 11 that does not exist in that bcm2835-dma context. I think this makes what I wrote about DMA4 engines (the most capable ones) somewhat incoherent in the details but the basic not-supported-in-the-code and not-used status appears to be true. As for DMA0-DMA10 (bcm2835-dma), some DMA (0-6) vs. DMA LITE (7-10) distinctions not being handled (for example 65536 maxsegsz for DMA LITE) still looks to be true to me. >>>> So, 0 through 10 need the soc criteria (mix of DMA and DMA LITE = engine criteria) >>>> and 11 through 14 need the scb criteria (DMA4 engine criteria). = (I'm ignore >>>> dma-channel-mask's at this point.) >>>>=20 >>>>=20 >>>> I'll here note the code has: >>>>=20 >>>> #define BCM_DMA_CH_MAX 12 >>>>=20 >>>> for use in code like: >>>>=20 >>>> /* setup initial settings */ >>>> for (i =3D 0; i < BCM_DMA_CH_MAX; i++) { >>>> ch =3D &sc->sc_dma_ch[i]; >>>>=20 >>>> bzero(ch, sizeof(struct bcm_dma_ch)); >>>> ch->ch =3D i; >>>> ch->flags =3D BCM_DMA_CH_UNMAP; >>>>=20 >>>> if ((bcm_dma_channel_mask & (1 << i)) =3D=3D 0) >>>> continue; >>>> . . . >>>>=20 >>>> It looks to me like the only scb/DMA4-engine "dma11" is covered >>>> by such loops and that the "brcm,dma-channel-mask =3D <0x00007000>" >>>> means that dma11 will not be used. >>>>=20 >>>> So: No scb/DMA4 engine will be used??? (That could explain the >>>> 1 GiByte limit?) >>>>=20 >>>>=20 >>>> rpi_DATA_2711_1p0.pdf reports that soc/0-10 have 2 types (0-6 vs. = 7-10 >>>> as it turns out) as well as the scb/DM4-engines (11-14): >>>>=20 >>>> QUOTE (with omitted marked by ". . .") >>>> . . . >>>> The BCM2711 DMA Controller provides a total of 16 DMA channels. = Four of these are DMA Lite channels (with reduced performance and = features), and four of them are DMA4 channels (with increased = performance and a wider address range). >>>> . . . >>>> 4.5. DMA LITE Engines >>>>=20 >>>> Several of the DMA engines are of the LITE design. This is a = reduced specification engine designed to save space. The engine behaves = in the same way as a normal DMA engine except for the following = differences: >>>> . . . >>>> =E2=80=A2 The DMA length register is now 16 bits, limiting the = maximum transferable length to 65536 bytes. >>>> . . . >>>> 4.6. DMA4 Engines >>>>=20 >>>> Several of the DMA engines are of the DMA4 design. These have = higher performance due to their uncoupled read/write design and can = access up to 40 address bits. Unlike the other DMA engines they are also = capable of performing write bursts. Note that they directly access the = full 35-bit address bus of the BCM2711 and so bypass the paging = registers of the DMA and DMA Lite engines. >>>>=20 >>>> DMA channel 11 is additionally able to access the PCIe interface. >>>> END QUOTE >>>>=20 >>>> The register map indicates (with some extra notes added): >>>>=20 >>>> 0-6: DMA >>>> 7-10: DMA LITE (65536 bytes limit, for example) >>>> 11-14: DMA4 (11 is special relative to "PCIe interface") >>>> ("DMA Channel 15 is exclusively used by the VPU.") >>>>=20 >>>> Yet what I see in the head -r365932 code is: >>>>=20 >>>> #define BCM_DMA_CH_MAX 12 >>>> . . . >>>> struct bcm_dma_softc { >>>> device_t sc_dev; >>>> struct mtx sc_mtx; >>>> struct resource * sc_mem; >>>> struct resource * sc_irq[BCM_DMA_CH_MAX]; >>>> void * sc_intrhand[BCM_DMA_CH_MAX]; >>>> struct bcm_dma_ch sc_dma_ch[BCM_DMA_CH_MAX]; >>>> bus_dma_tag_t sc_dma_tag; >>>> }; >>>> . . . >>>> err =3D bus_dma_tag_create(bus_get_dma_tag(dev), >>>> 1, 0, BUS_SPACE_MAXADDR_32BIT, >>>> BUS_SPACE_MAXADDR, NULL, NULL, >>>> sizeof(struct bcm_dma_cb), 1, >>>> sizeof(struct bcm_dma_cb), >>>> BUS_DMA_ALLOCNOW, NULL, NULL, >>>> &sc->sc_dma_tag); >>>>=20 >>>> As an example: does that deal with the likes of DMA LITE (so 7-10) = "limiting >>>> the maximum transferable length to 65536 bytes"? >>>>=20 >>>> As another example: Does it deal with the DMA4 (11-14) distinctions = (if >>>> such were in use anyway)? >>>>=20 >>>> For reference from the fdt print / : >>>>=20 >>>> / { >>>> . . . >>>> #address-cells =3D <0x00000002>; >>>> #size-cells =3D <0x00000001>; >>>> . . . >>>> soc { >>>> compatible =3D "simple-bus"; >>>> #address-cells =3D <0x00000001>; >>>> #size-cells =3D <0x00000001>; >>>> . . . >>>> dma-ranges =3D <0xc0000000 0x00000000 0x00000000 = 0x40000000>; >>>> . . . >>>> firmware { >>>> compatible =3D "raspberrypi,bcm2835-firmware", = "simple-bus"; >>>> mboxes =3D <0x0000001c>; >>>> dma-ranges; >>>> . . . >>>> emmc2bus { >>>> compatible =3D "simple-bus"; >>>> #address-cells =3D <0x00000002>; >>>> #size-cells =3D <0x00000001>; >>>> . . . >>>> dma-ranges =3D <0x00000000 0xc0000000 0x00000000 = 0x00000000 0x40000000>; >>>> . . . >>>> scb { >>>> compatible =3D "simple-bus"; >>>> #address-cells =3D <0x00000002>; >>>> #size-cells =3D <0x00000002>; >>>> . . . >>>> dma-ranges =3D <0x00000000 0x00000000 0x00000000 = 0x00000000 0x00000000 0xfc000000 0x00000001 0x00000000 0x00000001 = 0x00000000 0x00000001 0x00000000>; >>>> . . . >>>> pcie@7d500000 { >>>> compatible =3D "brcm,bcm2711-pcie"; >>>> . . . >>>> #address-cells =3D <0x00000003>; >>>> . . . >>>> #size-cells =3D <0x00000002>; >>>> . . . >>>> dma-ranges =3D <0x02000000 0x00000000 = 0x00000000 0x00000000 0x00000000 0x00000000 0xc0000000>; >>>> . . . >>>> v3dbus { >>>> compatible =3D "simple-bus"; >>>> #address-cells =3D <0x00000001>; >>>> #size-cells =3D <0x00000002>; >>>> . . . >>>> dma-ranges =3D <0x00000000 0x00000000 0x00000000 = 0x00000004 0x00000000>; >>>> . . . >>>=20 >>> rpi_DATA_2711_1p0.pdf reports: >>> (I ignore 2D DMA transfer mode here.) >>>=20 >>> For DMA engines 0-6: XLENGTH has bits 29:0 >>> bits 31:30 are write as 0, read as do not care. >>> That would put maxsegsz as 2**30 =3D=3D 1,073,741,824 >>> which matches a 1 GiByte space. >>>=20 >>> For DMA LITE engines 7-10: XLENGTH has bit 15:0 >>> bits 31:16 are write as 0, read as do not care. >>> That would put maxsegsz as 2**16 =3D=3D 65,536. >>>=20 >>> For DMA4 engines 11-14: XLENGTH has bits 29:0 >>> bits 31:30 are write as 0, read as do not care. >>> That would put maxsegsz as 2**30 =3D=3D 1,073,741,824 >>> which is smaller than the 3 GiByte space associated >>> with xHCI. >=20 > rpi_DATA_2711_1p0.pdf reports the following specifically for > DMA11-DMA14 (so the DMA4 engines) for what goes in the CB and > NEXT_CB ADDR fields: >=20 > QUOTE > The address must be 256-bit aligned and so the bottom 5 bits of the = byte address are discarded, i.e. write cb_byte_address[39:0]>>5 into the = CB > END QUOTE >=20 > This is not true for DMA0-DMA10 (DMA and DMA LITE). >=20 > The following is extracted from various places to > bring them together. I do not see evidence of handling > the cb_byte_address[39:0]>>5 involved for DMA11-DMA14: >=20 > #define ARMC_TO_VCBUS(pa) bcm283x_armc_to_vcbus(pa) >=20 > vm_paddr_t > bcm283x_armc_to_vcbus(vm_paddr_t pa) > { > struct bcm283x_memory_soc_cfg *cfg; > struct bcm283x_memory_mapping *map, *ment; >=20 > /* Guaranteed not NULL if we haven't panicked yet. */ > cfg =3D bcm283x_get_current_memcfg(); > map =3D cfg->memmap; > for (ment =3D map; !BCM283X_MEMMAP_ISTERM(ment); ++ment) { > if (pa >=3D ment->armc_start && > pa < ment->armc_start + ment->armc_size) { > return (pa - ment->armc_start) + = ment->vcbus_start; > } > } >=20 > /* > * Assume 1:1 mapping for anything else, but complain about it = on > * verbose boots. > */ > if (bootverbose) > printf("bcm283x_vcbus: No armc -> vcbus mapping found: = %jx\n", > (uintmax_t)pa); > return (pa); > } >=20 > static void > bcm_dmamap_cb(void *arg, bus_dma_segment_t *segs, > int nseg, int err) > { > bus_addr_t *addr; >=20 > if (err) > return; >=20 > addr =3D (bus_addr_t*)arg; > *addr =3D ARMC_TO_VCBUS(segs[0].ds_addr); > } >=20 > Note ds_addr assignments in: >=20 > static bus_size_t > _bus_dmamap_addseg(bus_dma_tag_t dmat, bus_dmamap_t map, bus_addr_t = curaddr, > bus_size_t sgsize, bus_dma_segment_t *segs, int *segp) > { > bus_addr_t baddr, bmask; > int seg; >=20 > /* > * Make sure we don't cross any boundaries. > */ > bmask =3D ~(dmat->common.boundary - 1); > if (dmat->common.boundary > 0) { > baddr =3D (curaddr + dmat->common.boundary) & bmask; > if (sgsize > (baddr - curaddr)) > sgsize =3D (baddr - curaddr); > } >=20 > /* > * Insert chunk into a segment, coalescing with > * previous segment if possible. > */ > seg =3D *segp; > if (seg =3D=3D -1) { > seg =3D 0; > segs[seg].ds_addr =3D curaddr; > segs[seg].ds_len =3D sgsize; > } else { > if (curaddr =3D=3D segs[seg].ds_addr + segs[seg].ds_len = && > (segs[seg].ds_len + sgsize) <=3D = dmat->common.maxsegsz && > (dmat->common.boundary =3D=3D 0 || > (segs[seg].ds_addr & bmask) =3D=3D (curaddr & = bmask))) > segs[seg].ds_len +=3D sgsize; > else { > if (++seg >=3D dmat->common.nsegments) > return (0); > segs[seg].ds_addr =3D curaddr; > segs[seg].ds_len =3D sgsize; > } > } > *segp =3D seg; > return (sgsize); > } >=20 >=20 > Note cb_phys and ch->vc_cb in: >=20 > static int > bcm_dma_init(device_t dev) > { > . . . > /* setup initial settings */ > for (i =3D 0; i < BCM_DMA_CH_MAX; i++) { > . . . > err =3D bus_dmamap_load(sc->sc_dma_tag, ch->dma_map, = cb_virt, > sizeof(struct bcm_dma_cb), bcm_dmamap_cb, &cb_phys, > BUS_DMA_WAITOK); > if (err) { > device_printf(dev, "cannot load DMA memory\n"); > break; > } >=20 > ch->cb =3D cb_virt; > ch->vc_cb =3D cb_phys; > . . . >=20 > int > bcm_dma_start(int ch, vm_paddr_t src, vm_paddr_t dst, int len) > { > struct bcm_dma_softc *sc =3D bcm_dma_sc; > struct bcm_dma_cb *cb; >=20 > if (ch < 0 || ch >=3D BCM_DMA_CH_MAX) > return (-1); >=20 > if (!(sc->sc_dma_ch[ch].flags & BCM_DMA_CH_USED)) > return (-1); >=20 > cb =3D sc->sc_dma_ch[ch].cb; > cb->src =3D ARMC_TO_VCBUS(src); > cb->dst =3D ARMC_TO_VCBUS(dst); >=20 > cb->len =3D len; >=20 > bus_dmamap_sync(sc->sc_dma_tag, > sc->sc_dma_ch[ch].dma_map, BUS_DMASYNC_PREWRITE); >=20 > bus_write_4(sc->sc_mem, BCM_DMA_CBADDR(ch), > sc->sc_dma_ch[ch].vc_cb); > bus_write_4(sc->sc_mem, BCM_DMA_CS(ch), CS_ACTIVE); >=20 > #ifdef DEBUG > bcm_dma_cb_dump(sc->sc_dma_ch[ch].cb); > bcm_dma_reg_dump(ch); > #endif >=20 > return (0); > } >=20 > It looks to me like FreeBSD is not set up to use the DMA4 > engines (DMA11-DMA14) and happens to not use them for the > DTB that I get from u-boot.bin in my context. >=20 > Of course, I may just have missed something in looking > around at the unfamiliar material. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Wed Sep 30 11:16:16 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9EA34424B53 for ; Wed, 30 Sep 2020 11:16:16 +0000 (UTC) (envelope-from dan.kotowski@a9development.com) Received: from mail-40131.protonmail.ch (mail-40131.protonmail.ch [185.70.40.131]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "SwissSign Server Gold CA 2014 - G22" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C1Ybg55tMz46nx for ; Wed, 30 Sep 2020 11:16:15 +0000 (UTC) (envelope-from dan.kotowski@a9development.com) Date: Wed, 30 Sep 2020 11:16:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=a9development.com; s=protonmail; t=1601464573; bh=GWnrqt/fWrhHJ9iuj6SLj3X87S0/R9d1IAgVWDEpcEc=; h=Date:To:From:Reply-To:Subject:In-Reply-To:References:From; b=B1zH3o54THKQhG658jFC0z5UvnON0w2+IWR9gUa/SvK+nnN9ORbkLQGCERxwYH+B0 /UYZmrY9SsCAm6Kv1Diw5t58/PVs1YeiUaOweB3bv5jYfr5/ozJmEKcGD5GejGrHjW sQfEjW0g65fht5tkW+onfLuLgcVOEFoeMXuY9xHw= To: freebsd-arm From: Dan Kotowski Reply-To: Dan Kotowski Subject: Re: HEADS UP: NXP LS1046A SoC support in the tree Message-ID: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.2 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mailout.protonmail.ch X-Rspamd-Queue-Id: 4C1Ybg55tMz46nx X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=a9development.com header.s=protonmail header.b=B1zH3o54; dmarc=pass (policy=none) header.from=a9development.com; spf=none (mx1.freebsd.org: domain of dan.kotowski@a9development.com has no SPF policy when checking 185.70.40.131) smtp.mailfrom=dan.kotowski@a9development.com X-Spamd-Result: default: False [-2.75 / 15.00]; HAS_REPLYTO(0.00)[dan.kotowski@a9development.com]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[a9development.com:s=protonmail]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.73)[-0.728]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_MEDIUM(-0.77)[-0.770]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[a9development.com:+]; DMARC_POLICY_ALLOW(-0.50)[a9development.com,none]; RCVD_IN_DNSWL_NONE(0.00)[185.70.40.131:from]; NEURAL_HAM_SHORT(-0.45)[-0.454]; R_SPF_NA(0.00)[no SPF record]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arm]; RWL_MAILSPIKE_POSSIBLE(0.00)[185.70.40.131:from] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Sep 2020 11:16:16 -0000 > Hello all, > > On behalf of the Semihalf team I'd like to announce that starting from > r365054 FreeBSD is ready to run NXP LS1046A system-on-a-chip with most > of the interfaces supported. > > LS1046A are quad-core 64-bit ARMv8 Cortex-A72 processors with > integrated packet processing acceleration and high speed peripherals > including 10 Gb Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 for a wide > range of networking, storage, security and industrial applications. It > is confirmed that at least some parts of the kernel support (such as > SDHCI) was successfully used on the Solid-Run LX2K Honeycomb > development platform. > > Below is the port status together with the features, which were > originally developed on top of the release/11.2.0 that have made their > way to FreeBSD-HEAD: > > - LS1046A core support (Quad-CA72 SMP, timers, IRQS, UART) > - already working in vanilla FreeBSD-HEAD before > - DWC3 USB3.0 host controller driver (FreeBSD-HEAD > version developed by manu@) > > - VF610 I2C controller (adjusted to work with the LS1046A) > - TI LM75 temperature sensor (adjusted to work with the LS1046A) > - QorIQ platform clockgen > - LS1046A CPU clockgen > - LS1046A GPIO > - RX8803 I2C RTC > - TCA6416 I2C GPIO expander > - LS1046A AHCI > - LS1046A SDHCI > > The major out-of-tree components are that albeit working on top of > release/11.2.0, still require significant effort to adopt to > FreeBSD-HEAD: > > - DPAA network controller support > - QSPI controller support > > This was a joined effort of Semihalf and Alstom Group (main > development sponsor), in particular: > Laurent Muller > David Fontaine > Yannis Planus > Artur Rojek > Dawid G=C3=B3recki > Kornel Dul=C4=99ba > Kamil Koczurek > Micha=C5=82 Stanek > Jacek Klimkowicz > > Best regards, > Marcin Excellent work, all! Do you expect to port up the DPAA controller support or will you be leaving= that for others? And have you looked into what it would take to add DPAA2 support as well? From owner-freebsd-arm@freebsd.org Wed Sep 30 18:13:18 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3E8E742D7C4 for ; Wed, 30 Sep 2020 18:13:18 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic313-20.consmr.mail.gq1.yahoo.com (sonic313-20.consmr.mail.gq1.yahoo.com [98.137.65.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C1krr5TQ8z4c16 for ; Wed, 30 Sep 2020 18:13:16 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: bX9h4xkVM1mwj7v8XvIN3jwJjGhgqRqfjb2zr1NcSgo6AizBMPCl4n90UaEuS48 D8UvC.0Lna2NdCTrkwb2yBiGH28ceZCfC4OAgrrJK8x2LcuaGSMxy1HPx0W88QGtjSFWK0rhSfg4 ZRV7qpsP4H0LAXRGaESDwBai2MQBnBo_SY29ImxCDr208ARUmiyr2yR3rbSqe0u_Yb_PLVLK28hi 2wju3nudUC2LZZIdcvH.2yjTsy7mdONtw2mExrpev2s5Yh62sebNwUwyoKjBhNBRwP2TLwPd1Ex4 20XbMaAAaSMR15JV81I1mtBuvjzz5qa2u94gwr9mAqKKnJBAlZaj2pbppfXxnbUd.0SRVSqiBzWE 4yjhESX.QOPutBK5jxWaakmL_ZCASaQiQ5M5mTVpdbykKkc5ljZtSDa4Xs1y0txXxV6gPAmivDRs gQLzzcPhffaFJ63wNKsiJ4d3j2C7CIrG9Eb5SiQs1n4Xj4CO2ros49_9t_.jz2QKBdUG.rBnBoQw 9caAc8dFkwzrsRiuTDf.uNlgWU1syJWAp2GOrS1QomiUmStjvv7L8WEihIG.J4kks51r15TtNHuP .8OX9g8yLHf25BT.Quuj198hAmB8nCul4kG7PBqzzSdFv.Gy4kq93moSy6c5A5dk5ovY6rTuc8KP tVOkKD_lHJ_oN.6tmIbZ.YNutdCbTDlp8gLahC3pnbNPZO2WgFXPFPtV.2zwrZoJYme56s3QAic3 NjV5.g2RJM.fc2hyhw2HeIwIUawz7FY8geQNWfgl7phWssoSZIM4ItNK1F1V4E0YzOpe9AMHaJX. z_BmnWJ.i87.IDhiM3.gTow0nvG0kw54AofZrSjjLjTxTTeP_uoGZDiw5d1wjZjVjgSH4Lmq1mzB 98ip5WENy0u5JdnaM1ingR0J65KJicDsFwyO0uu8ch311vkfEG1xRVnsab5zlko7aB_P7ZpvspkB O0BAXkb8OnH7U5xBDtUMeG8sleBEJdOHCUU0ody660ohu5uqHqW1QJDhOWiKVWlsXZlm0gSPyRqb Bn6I0HcGzyvCZOKxcpLtGSt2ooT3cZJCD7D8TyNRba_KskLH0BOFNmxcPUnuMHSudnsX6BjYa938 5g8zkyc9MhxGTcYzMRrFkGKU4Oj3ZV6PQa1rj58DnG50pJ5YrgB.5LI4tucQWuzy88wcszDL0fJu eVlVWJjfKIId92dolt.3pHGG9zWb.2u2sfae2jBtXHjTxmY7Jfvn82ltZsObYASmSp453sFrRNJH JPpmyIry62JUC3ObCWAdEg61ati.6uWO1tcVCP1ELbPQOy.iisuGRmHPl534.dIFluAEJWNw5C2c 8ha7u.lBt3QarsTFRyjVRKmUuvA9uhmcRSiKthQ.yTnB3aGltGaIvdTw29.CbxWnPms3s_1fWrP3 _O0K13tMyx.i3NscOPtipk98gHEBV.1oSBmgo6aJHQgJ5O5YbWJnl7mIbvSgGUAuGMMx2.yAAgfZ 1ztVyZr7MREWlh7i2WaFjDVse7MzNMKOUxUYODv7938wJLXVQeZ.xeIlwc5eFm4hlpFpc3csodQ- - Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.gq1.yahoo.com with HTTP; Wed, 30 Sep 2020 18:13:13 +0000 Received: by smtp416.mail.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 0bfda7268b402c4b01cb748ba6763ec8; Wed, 30 Sep 2020 18:13:08 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: RPi4B's DMA11 (DMA4 engine example) vs. xHCI/pcie Date: Wed, 30 Sep 2020 11:13:06 -0700 References: <8C6DE44F-6CE2-4C74-8748-3BBFB54AE183@yahoo.com> <0FE382AB-8DE3-4467-9CB0-E8582AC70EA2@yahoo.com> <85FEDC51-B5B0-4ED4-A5ED-62B63EF9D5A8@yahoo.com> To: Robert Crowston , freebsd-arm In-Reply-To: Message-Id: <903FE769-ED46-4FBC-A272-4D2C89A9CD7A@yahoo.com> X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Rspamd-Queue-Id: 4C1krr5TQ8z4c16 X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.97 / 15.00]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.43)[-0.430]; FREEMAIL_TO(0.00)[protonmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.02)[-1.025]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.011]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.83:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.83:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Sep 2020 18:13:18 -0000 On 2020-Sep-29, at 10:35, Mark Millard wrote: > On 2020-Sep-28, at 21:45, Mark Millard wrote: >=20 >> On 2020-Sep-28, at 19:04, Mark Millard wrote: >>=20 >>> On 2020-Sep-28, at 18:29, Mark Millard wrote: >>>>=20 >>>>> [Be warned that the material is not familiar so I may need >>>>> educating. THis is based ont he example context that I >>>>> happen to have around.] >>>>>=20 >>>>> In the u-boot fdt print / output there are 2 distinct sets of dma = channel >>>>> information, 1 for soc and 1 for scb, where the dma_tag values for = the two >>>>> sets should be distinct as far as I can tell: >>>>>=20 >>>>> U-Boot> fdt address 0x7ef1000 >>>>> U-Boot> fdt print / =20 >>>>> / { >>>>> . . . >>>>> soc { >>>>> dma@7e007000 { >>>>> compatible =3D "brcm,bcm2835-dma"; >>>>> reg =3D <0x7e007000 0x00000b00>; >>>>> interrupts =3D * 0x0000000007ef645c = [0x00000084]; >>>>> interrupt-names =3D "dma0", "dma1", "dma2", = "dma3", "dma4", "dma5", "dma6", "dma7", "dma8", "dma9", "dma10"; >>>>> #dma-cells =3D <0x00000001>; >>>>> brcm,dma-channel-mask =3D <0x000001f5>; >>>>> phandle =3D <0x0000000b>; >>>>> }; >>>>>=20 >>>>> scb { >>>>> . . . >>>>> dma@7e007b00 { >>>>> compatible =3D "brcm,bcm2711-dma"; >>>>> reg =3D <0x00000000 0x7e007b00 0x00000000 = 0x00000400>; >>>>> interrupts =3D <0x00000000 0x00000059 = 0x00000004 0x00000000 0x0000005a 0x00000004 0x00000000 0x0000005b = 0x00000004 0x00000000 0x0000005c 0x00000004>; >>>>> interrupt-names =3D "dma11", "dma12", "dma13", = "dma14"; >>>>> #dma-cells =3D <0x00000001>; >>>>> brcm,dma-channel-mask =3D <0x00007000>; >>>>> phandle =3D <0x0000003d>; >>>>> }; >>>>> . . . >=20 > I had presumed that the dma@7e007b00 would be processed. But > I finally happened to search for "bcm2711-dma" in FreeBSD and > it does not occur. >=20 > That appears to mean that BCM_DMA_CH_MAX being 12 is depending > on dma@7e007000's brcm,dma-channel-mask to avoid referencing > number 11 that does not exist in that bcm2835-dma context. >=20 > I think this makes what I wrote about DMA4 engines (the most > capable ones) somewhat incoherent in the details but the basic > not-supported-in-the-code and not-used status appears to be > true. >=20 > As for DMA0-DMA10 (bcm2835-dma), some DMA (0-6) vs. DMA LITE > (7-10) distinctions not being handled (for example 65536 > maxsegsz for DMA LITE) still looks to be true to me. Looks like FreeBSD is limited to 32-bit via = usb/controller/generic_xhci.c has nothing explicit for other than 32 address lines (and overall the only alternative is 64 address lines): #define IS_DMA_32B 1 int generic_xhci_attach(device_t dev) { . . . err =3D xhci_init(sc, dev, IS_DMA_32B); if (err !=3D 0) { device_printf(dev, "Failed to init XHCI, with error = %d\n", err); generic_xhci_detach(dev); return (ENXIO); } . . . /* * The following structure describes the parent USB DMA tag. */ #if USB_HAVE_BUSDMA struct usb_dma_parent_tag { . . . uint8_t dma_bits; /* number of DMA address lines = */ . . . }; #else struct usb_dma_parent_tag {}; /* empty struct */ #endif . . . usb_error_t xhci_init(struct xhci_softc *sc, device_t self, uint8_t dma32) { . . . /* get DMA bits */ sc->sc_bus.dma_bits =3D (XHCI_HCS0_AC64(temp) && xhcidma32 =3D=3D 0 && dma32 =3D=3D 0) ? 64 : 32; . . . Overall it looks like a bunch of places would need changes to support the RPi4B's 3 GiByte capability. (Probably more than I've discovered, ignoring things like DMA4 engine use to get write bursts and the like.) I will note that I found code in NetBSD that classifies "normal" DMA engines vs. DMA LITE engines (via testing a debug register) for bcm2835-dma and only requests normal DMA engines be used, skipping DMA LITE. (This is for DTB/fdt contexts I think. I've not done as well figuring out even such narrow aspects of ACPI handling of things.) This tends to confirm my worries over FreeBSD's bcm2835-dma handling of the DMA LITE engines existing but being less capable. >>>>> So, 0 through 10 need the soc criteria (mix of DMA and DMA LITE = engine criteria) >>>>> and 11 through 14 need the scb criteria (DMA4 engine criteria). = (I'm ignore >>>>> dma-channel-mask's at this point.) >>>>>=20 >>>>>=20 >>>>> I'll here note the code has: >>>>>=20 >>>>> #define BCM_DMA_CH_MAX 12 >>>>>=20 >>>>> for use in code like: >>>>>=20 >>>>> /* setup initial settings */ >>>>> for (i =3D 0; i < BCM_DMA_CH_MAX; i++) { >>>>> ch =3D &sc->sc_dma_ch[i]; >>>>>=20 >>>>> bzero(ch, sizeof(struct bcm_dma_ch)); >>>>> ch->ch =3D i; >>>>> ch->flags =3D BCM_DMA_CH_UNMAP; >>>>>=20 >>>>> if ((bcm_dma_channel_mask & (1 << i)) =3D=3D 0) >>>>> continue; >>>>> . . . >>>>>=20 >>>>> It looks to me like the only scb/DMA4-engine "dma11" is covered >>>>> by such loops and that the "brcm,dma-channel-mask =3D = <0x00007000>" >>>>> means that dma11 will not be used. >>>>>=20 >>>>> So: No scb/DMA4 engine will be used??? (That could explain the >>>>> 1 GiByte limit?) >>>>>=20 >>>>>=20 >>>>> rpi_DATA_2711_1p0.pdf reports that soc/0-10 have 2 types (0-6 vs. = 7-10 >>>>> as it turns out) as well as the scb/DM4-engines (11-14): >>>>>=20 >>>>> QUOTE (with omitted marked by ". . .") >>>>> . . . >>>>> The BCM2711 DMA Controller provides a total of 16 DMA channels. = Four of these are DMA Lite channels (with reduced performance and = features), and four of them are DMA4 channels (with increased = performance and a wider address range). >>>>> . . . >>>>> 4.5. DMA LITE Engines >>>>>=20 >>>>> Several of the DMA engines are of the LITE design. This is a = reduced specification engine designed to save space. The engine behaves = in the same way as a normal DMA engine except for the following = differences: >>>>> . . . >>>>> =E2=80=A2 The DMA length register is now 16 bits, limiting the = maximum transferable length to 65536 bytes. >>>>> . . . >>>>> 4.6. DMA4 Engines >>>>>=20 >>>>> Several of the DMA engines are of the DMA4 design. These have = higher performance due to their uncoupled read/write design and can = access up to 40 address bits. Unlike the other DMA engines they are also = capable of performing write bursts. Note that they directly access the = full 35-bit address bus of the BCM2711 and so bypass the paging = registers of the DMA and DMA Lite engines. >>>>>=20 >>>>> DMA channel 11 is additionally able to access the PCIe interface. >>>>> END QUOTE >>>>>=20 >>>>> The register map indicates (with some extra notes added): >>>>>=20 >>>>> 0-6: DMA >>>>> 7-10: DMA LITE (65536 bytes limit, for example) >>>>> 11-14: DMA4 (11 is special relative to "PCIe interface") >>>>> ("DMA Channel 15 is exclusively used by the VPU.") >>>>>=20 >>>>> Yet what I see in the head -r365932 code is: >>>>>=20 >>>>> #define BCM_DMA_CH_MAX 12 >>>>> . . . >>>>> struct bcm_dma_softc { >>>>> device_t sc_dev; >>>>> struct mtx sc_mtx; >>>>> struct resource * sc_mem; >>>>> struct resource * sc_irq[BCM_DMA_CH_MAX]; >>>>> void * sc_intrhand[BCM_DMA_CH_MAX]; >>>>> struct bcm_dma_ch sc_dma_ch[BCM_DMA_CH_MAX]; >>>>> bus_dma_tag_t sc_dma_tag; >>>>> }; >>>>> . . . >>>>> err =3D bus_dma_tag_create(bus_get_dma_tag(dev), >>>>> 1, 0, BUS_SPACE_MAXADDR_32BIT, >>>>> BUS_SPACE_MAXADDR, NULL, NULL, >>>>> sizeof(struct bcm_dma_cb), 1, >>>>> sizeof(struct bcm_dma_cb), >>>>> BUS_DMA_ALLOCNOW, NULL, NULL, >>>>> &sc->sc_dma_tag); >>>>>=20 >>>>> As an example: does that deal with the likes of DMA LITE (so 7-10) = "limiting >>>>> the maximum transferable length to 65536 bytes"? >>>>>=20 >>>>> As another example: Does it deal with the DMA4 (11-14) = distinctions (if >>>>> such were in use anyway)? >>>>>=20 >>>>> For reference from the fdt print / : >>>>>=20 >>>>> / { >>>>> . . . >>>>> #address-cells =3D <0x00000002>; >>>>> #size-cells =3D <0x00000001>; >>>>> . . . >>>>> soc { >>>>> compatible =3D "simple-bus"; >>>>> #address-cells =3D <0x00000001>; >>>>> #size-cells =3D <0x00000001>; >>>>> . . . >>>>> dma-ranges =3D <0xc0000000 0x00000000 0x00000000 = 0x40000000>; >>>>> . . . >>>>> firmware { >>>>> compatible =3D "raspberrypi,bcm2835-firmware", = "simple-bus"; >>>>> mboxes =3D <0x0000001c>; >>>>> dma-ranges; >>>>> . . . >>>>> emmc2bus { >>>>> compatible =3D "simple-bus"; >>>>> #address-cells =3D <0x00000002>; >>>>> #size-cells =3D <0x00000001>; >>>>> . . . >>>>> dma-ranges =3D <0x00000000 0xc0000000 0x00000000 = 0x00000000 0x40000000>; >>>>> . . . >>>>> scb { >>>>> compatible =3D "simple-bus"; >>>>> #address-cells =3D <0x00000002>; >>>>> #size-cells =3D <0x00000002>; >>>>> . . . >>>>> dma-ranges =3D <0x00000000 0x00000000 0x00000000 = 0x00000000 0x00000000 0xfc000000 0x00000001 0x00000000 0x00000001 = 0x00000000 0x00000001 0x00000000>; >>>>> . . . >>>>> pcie@7d500000 { >>>>> compatible =3D "brcm,bcm2711-pcie"; >>>>> . . . >>>>> #address-cells =3D <0x00000003>; >>>>> . . . >>>>> #size-cells =3D <0x00000002>; >>>>> . . . >>>>> dma-ranges =3D <0x02000000 0x00000000 = 0x00000000 0x00000000 0x00000000 0x00000000 0xc0000000>; >>>>> . . . >>>>> v3dbus { >>>>> compatible =3D "simple-bus"; >>>>> #address-cells =3D <0x00000001>; >>>>> #size-cells =3D <0x00000002>; >>>>> . . . >>>>> dma-ranges =3D <0x00000000 0x00000000 0x00000000 = 0x00000004 0x00000000>; >>>>> . . . >>>>=20 >>>> rpi_DATA_2711_1p0.pdf reports: >>>> (I ignore 2D DMA transfer mode here.) >>>>=20 >>>> For DMA engines 0-6: XLENGTH has bits 29:0 >>>> bits 31:30 are write as 0, read as do not care. >>>> That would put maxsegsz as 2**30 =3D=3D 1,073,741,824 >>>> which matches a 1 GiByte space. >>>>=20 >>>> For DMA LITE engines 7-10: XLENGTH has bit 15:0 >>>> bits 31:16 are write as 0, read as do not care. >>>> That would put maxsegsz as 2**16 =3D=3D 65,536. >>>>=20 >>>> For DMA4 engines 11-14: XLENGTH has bits 29:0 >>>> bits 31:30 are write as 0, read as do not care. >>>> That would put maxsegsz as 2**30 =3D=3D 1,073,741,824 >>>> which is smaller than the 3 GiByte space associated >>>> with xHCI. >>=20 >> rpi_DATA_2711_1p0.pdf reports the following specifically for >> DMA11-DMA14 (so the DMA4 engines) for what goes in the CB and >> NEXT_CB ADDR fields: >>=20 >> QUOTE >> The address must be 256-bit aligned and so the bottom 5 bits of the = byte address are discarded, i.e. write cb_byte_address[39:0]>>5 into the = CB >> END QUOTE >>=20 >> This is not true for DMA0-DMA10 (DMA and DMA LITE). >>=20 >> The following is extracted from various places to >> bring them together. I do not see evidence of handling >> the cb_byte_address[39:0]>>5 involved for DMA11-DMA14: >>=20 >> #define ARMC_TO_VCBUS(pa) bcm283x_armc_to_vcbus(pa) >>=20 >> vm_paddr_t >> bcm283x_armc_to_vcbus(vm_paddr_t pa) >> { >> struct bcm283x_memory_soc_cfg *cfg; >> struct bcm283x_memory_mapping *map, *ment; >>=20 >> /* Guaranteed not NULL if we haven't panicked yet. */ >> cfg =3D bcm283x_get_current_memcfg(); >> map =3D cfg->memmap; >> for (ment =3D map; !BCM283X_MEMMAP_ISTERM(ment); ++ment) { >> if (pa >=3D ment->armc_start && >> pa < ment->armc_start + ment->armc_size) { >> return (pa - ment->armc_start) + = ment->vcbus_start; >> } >> } >>=20 >> /* >> * Assume 1:1 mapping for anything else, but complain about it = on >> * verbose boots. >> */ >> if (bootverbose) >> printf("bcm283x_vcbus: No armc -> vcbus mapping found: = %jx\n", >> (uintmax_t)pa); >> return (pa); >> } >>=20 >> static void >> bcm_dmamap_cb(void *arg, bus_dma_segment_t *segs, >> int nseg, int err) >> { >> bus_addr_t *addr; >>=20 >> if (err) >> return; >>=20 >> addr =3D (bus_addr_t*)arg; >> *addr =3D ARMC_TO_VCBUS(segs[0].ds_addr); >> } >>=20 >> Note ds_addr assignments in: >>=20 >> static bus_size_t >> _bus_dmamap_addseg(bus_dma_tag_t dmat, bus_dmamap_t map, bus_addr_t = curaddr, >> bus_size_t sgsize, bus_dma_segment_t *segs, int *segp) >> { >> bus_addr_t baddr, bmask; >> int seg; >>=20 >> /* >> * Make sure we don't cross any boundaries. >> */ >> bmask =3D ~(dmat->common.boundary - 1); >> if (dmat->common.boundary > 0) { >> baddr =3D (curaddr + dmat->common.boundary) & bmask; >> if (sgsize > (baddr - curaddr)) >> sgsize =3D (baddr - curaddr); >> } >>=20 >> /* >> * Insert chunk into a segment, coalescing with >> * previous segment if possible. >> */ >> seg =3D *segp; >> if (seg =3D=3D -1) { >> seg =3D 0; >> segs[seg].ds_addr =3D curaddr; >> segs[seg].ds_len =3D sgsize; >> } else { >> if (curaddr =3D=3D segs[seg].ds_addr + segs[seg].ds_len = && >> (segs[seg].ds_len + sgsize) <=3D = dmat->common.maxsegsz && >> (dmat->common.boundary =3D=3D 0 || >> (segs[seg].ds_addr & bmask) =3D=3D (curaddr & = bmask))) >> segs[seg].ds_len +=3D sgsize; >> else { >> if (++seg >=3D dmat->common.nsegments) >> return (0); >> segs[seg].ds_addr =3D curaddr; >> segs[seg].ds_len =3D sgsize; >> } >> } >> *segp =3D seg; >> return (sgsize); >> } >>=20 >>=20 >> Note cb_phys and ch->vc_cb in: >>=20 >> static int >> bcm_dma_init(device_t dev) >> { >> . . . >> /* setup initial settings */ >> for (i =3D 0; i < BCM_DMA_CH_MAX; i++) { >> . . . >> err =3D bus_dmamap_load(sc->sc_dma_tag, ch->dma_map, = cb_virt, >> sizeof(struct bcm_dma_cb), bcm_dmamap_cb, &cb_phys, >> BUS_DMA_WAITOK); >> if (err) { >> device_printf(dev, "cannot load DMA memory\n"); >> break; >> } >>=20 >> ch->cb =3D cb_virt; >> ch->vc_cb =3D cb_phys; >> . . . >>=20 >> int >> bcm_dma_start(int ch, vm_paddr_t src, vm_paddr_t dst, int len) >> { >> struct bcm_dma_softc *sc =3D bcm_dma_sc; >> struct bcm_dma_cb *cb; >>=20 >> if (ch < 0 || ch >=3D BCM_DMA_CH_MAX) >> return (-1); >>=20 >> if (!(sc->sc_dma_ch[ch].flags & BCM_DMA_CH_USED)) >> return (-1); >>=20 >> cb =3D sc->sc_dma_ch[ch].cb; >> cb->src =3D ARMC_TO_VCBUS(src); >> cb->dst =3D ARMC_TO_VCBUS(dst); >>=20 >> cb->len =3D len; >>=20 >> bus_dmamap_sync(sc->sc_dma_tag, >> sc->sc_dma_ch[ch].dma_map, BUS_DMASYNC_PREWRITE); >>=20 >> bus_write_4(sc->sc_mem, BCM_DMA_CBADDR(ch), >> sc->sc_dma_ch[ch].vc_cb); >> bus_write_4(sc->sc_mem, BCM_DMA_CS(ch), CS_ACTIVE); >>=20 >> #ifdef DEBUG >> bcm_dma_cb_dump(sc->sc_dma_ch[ch].cb); >> bcm_dma_reg_dump(ch); >> #endif >>=20 >> return (0); >> } >>=20 >> It looks to me like FreeBSD is not set up to use the DMA4 >> engines (DMA11-DMA14) and happens to not use them for the >> DTB that I get from u-boot.bin in my context. >>=20 >> Of course, I may just have missed something in looking >> around at the unfamiliar material. >=20 >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Wed Sep 30 21:15:47 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 30949431518 for ; Wed, 30 Sep 2020 21:15:47 +0000 (UTC) (envelope-from crowston@protonmail.com) Received: from mail-40136.protonmail.ch (mail-40136.protonmail.ch [185.70.40.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "SwissSign Server Gold CA 2014 - G22" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C1pvQ0crFz3Ybb for ; Wed, 30 Sep 2020 21:15:45 +0000 (UTC) (envelope-from crowston@protonmail.com) Date: Wed, 30 Sep 2020 21:15:30 +0000 To: Mark Millard From: Robert Crowston Cc: freebsd-arm Reply-To: Robert Crowston Subject: Re: RPi4B's DMA11 (DMA4 engine example) vs. xHCI/pcie Message-ID: In-Reply-To: <903FE769-ED46-4FBC-A272-4D2C89A9CD7A@yahoo.com> References: <8C6DE44F-6CE2-4C74-8748-3BBFB54AE183@yahoo.com> <0FE382AB-8DE3-4467-9CB0-E8582AC70EA2@yahoo.com> <85FEDC51-B5B0-4ED4-A5ED-62B63EF9D5A8@yahoo.com> <903FE769-ED46-4FBC-A272-4D2C89A9CD7A@yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.2 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mailout.protonmail.ch X-Rspamd-Queue-Id: 4C1pvQ0crFz3Ybb X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.72 / 15.00]; HAS_REPLYTO(0.00)[crowston@protonmail.com]; FREEMAIL_FROM(0.00)[protonmail.com]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[protonmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[protonmail.com,quarantine]; NEURAL_HAM_SHORT(-0.72)[-0.717]; FREEMAIL_TO(0.00)[yahoo.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[protonmail.com]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.02)[-1.021]; R_DKIM_ALLOW(-0.20)[protonmail.com:s=protonmail]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.985]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_REPLYTO(0.00)[protonmail.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[185.70.40.136:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[185.70.40.136:from]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Sep 2020 21:15:47 -0000 Very interesting analysis. Certainly uncovered a few things I wasn't aware = of. By default sc->sc_bus.dma_bits in xhci_init is 64 bits; I toggle it back to= 32 bits in the xhci shim I wrote for the Pi 4. You can see that output in = a verbose dmesg. =E2=80=94 RHC. =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 Original Me= ssage =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 On Wednesday, 30 September 2020 19:13, Mark Millard wro= te: > > > On 2020-Sep-29, at 10:35, Mark Millard wrote: > > > On 2020-Sep-28, at 21:45, Mark Millard wrote: > > > > > On 2020-Sep-28, at 19:04, Mark Millard wrote: > > > > > > > On 2020-Sep-28, at 18:29, Mark Millard wrote= : > > > > > > > > > > [Be warned that the material is not familiar so I may need > > > > > > educating. THis is based ont he example context that I > > > > > > happen to have around.] > > > > > > In the u-boot fdt print / output there are 2 distinct sets of d= ma channel > > > > > > information, 1 for soc and 1 for scb, where the dma_tag values = for the two > > > > > > sets should be distinct as far as I can tell: > > > > > > U-Boot> fdt address 0x7ef1000 > > > > > > U-Boot> fdt print / > > > > > > / { > > > > > > . . . > > > > > > soc { > > > > > > dma@7e007000 { > > > > > > compatible =3D "brcm,bcm2835-dma"; > > > > > > reg =3D <0x7e007000 0x00000b00>; > > > > > > interrupts =3D * 0x0000000007ef645c [0x00000084]; > > > > > > interrupt-names =3D "dma0", "dma1", "dma2", "dma3", "dma4", "dm= a5", "dma6", "dma7", "dma8", "dma9", "dma10"; > > > > > > #dma-cells =3D <0x00000001>; > > > > > > brcm,dma-channel-mask =3D <0x000001f5>; > > > > > > phandle =3D <0x0000000b>; > > > > > > }; > > > > > > > > > > > > scb { > > > > > > > > > > > > > > > > > > . . . > > > > > > dma@7e007b00 { > > > > > > compatible =3D "brcm,bcm2711-dma"; > > > > > > reg =3D <0x00000000 0x7e007b00 0x00000000 0x00000400>; > > > > > > interrupts =3D <0x00000000 0x00000059 0x00000004 0x00000000 0x0= 000005a 0x00000004 0x00000000 0x0000005b 0x00000004 0x00000000 0x0000005c 0= x00000004>; > > > > > > interrupt-names =3D "dma11", "dma12", "dma13", "dma14"; > > > > > > #dma-cells =3D <0x00000001>; > > > > > > brcm,dma-channel-mask =3D <0x00007000>; > > > > > > phandle =3D <0x0000003d>; > > > > > > }; > > > > > > . . . > > > > I had presumed that the dma@7e007b00 would be processed. But > > I finally happened to search for "bcm2711-dma" in FreeBSD and > > it does not occur. > > That appears to mean that BCM_DMA_CH_MAX being 12 is depending > > on dma@7e007000's brcm,dma-channel-mask to avoid referencing > > number 11 that does not exist in that bcm2835-dma context. > > I think this makes what I wrote about DMA4 engines (the most > > capable ones) somewhat incoherent in the details but the basic > > not-supported-in-the-code and not-used status appears to be > > true. > > As for DMA0-DMA10 (bcm2835-dma), some DMA (0-6) vs. DMA LITE > > (7-10) distinctions not being handled (for example 65536 > > maxsegsz for DMA LITE) still looks to be true to me. > > Looks like FreeBSD is limited to 32-bit via usb/controller/generic_xhci.c > has nothing explicit for other than 32 address lines (and overall the > only alternative is 64 address lines): > > #define IS_DMA_32B 1 > > int > generic_xhci_attach(device_t dev) > { > . . . > err =3D xhci_init(sc, dev, IS_DMA_32B); > if (err !=3D 0) { > device_printf(dev, "Failed to init XHCI, with error %d\n", err); > generic_xhci_detach(dev); > return (ENXIO); > } > . . . > /* > > - The following structure describes the parent USB DMA tag. > / > #if USB_HAVE_BUSDMA > struct usb_dma_parent_tag { > . . . > uint8_t dma_bits; / number of DMA address lines / > . . . > }; > #else > struct usb_dma_parent_tag {}; / empty struct */#endif > . . . > usb_error_t > xhci_init(struct xhci_softc sc, device_t self, uint8_t dma32) > { > . . . > / get DMA bits */sc->sc_bus.dma_bits =3D (XHCI_HCS0_AC64(temp) && > > xhcidma32 =3D=3D 0 && dma32 =3D=3D 0) ? 64 : 32; > > > > . . . > > Overall it looks like a bunch of places would need changes to > support the RPi4B's 3 GiByte capability. (Probably more than > I've discovered, ignoring things like DMA4 engine use to get > write bursts and the like.) > > I will note that I found code in NetBSD that classifies "normal" > DMA engines vs. DMA LITE engines (via testing a debug register) > for bcm2835-dma and only requests normal DMA engines be used, > skipping DMA LITE. (This is for DTB/fdt contexts I think. I've > not done as well figuring out even such narrow aspects of ACPI > handling of things.) This tends to confirm my worries over > FreeBSD's bcm2835-dma handling of the DMA LITE engines existing > but being less capable. > > > > > > > So, 0 through 10 need the soc criteria (mix of DMA and DMA LITE= engine criteria) > > > > > > and 11 through 14 need the scb criteria (DMA4 engine criteria).= (I'm ignore > > > > > > dma-channel-mask's at this point.) > > > > > > I'll here note the code has: > > > > > > #define BCM_DMA_CH_MAX 12 > > > > > > for use in code like: > > > > > > > > > > > > /* setup initial settings */ > > > > > > for (i =3D 0; i < BCM_DMA_CH_MAX; i++) { > > > > > > ch =3D &sc->sc_dma_ch[i]; > > > > > > > > > > > > bzero(ch, sizeof(struct bcm_dma_ch)); > > > > > > ch->ch =3D i; > > > > > > ch->flags =3D BCM_DMA_CH_UNMAP; > > > > > > > > > > > > if ((bcm_dma_channel_mask & (1 << i)) =3D=3D 0) > > > > > > continue; > > > > > > > > > > > > > > > > > > . . . > > > > > > It looks to me like the only scb/DMA4-engine "dma11" is covered > > > > > > by such loops and that the "brcm,dma-channel-mask =3D <0x000070= 00>" > > > > > > means that dma11 will not be used. > > > > > > So: No scb/DMA4 engine will be used??? (That could explain the > > > > > > 1 GiByte limit?) > > > > > > rpi_DATA_2711_1p0.pdf reports that soc/0-10 have 2 types (0-6 v= s. 7-10 > > > > > > as it turns out) as well as the scb/DM4-engines (11-14): > > > > > > QUOTE (with omitted marked by ". . .") > > > > > > . . . > > > > > > The BCM2711 DMA Controller provides a total of 16 DMA channels.= Four of these are DMA Lite channels (with reduced performance and features= ), and four of them are DMA4 channels (with increased performance and a wid= er address range). > > > > > > . . . > > > > > > 4.5. DMA LITE Engines > > > > > > Several of the DMA engines are of the LITE design. This is a re= duced specification engine designed to save space. The engine behaves in th= e same way as a normal DMA engine except for the following differences: > > > > > > . . . > > > > > > =E2=80=A2 The DMA length register is now 16 bits, limiting the = maximum transferable length to 65536 bytes. > > > > > > . . . > > > > > > 4.6. DMA4 Engines > > > > > > Several of the DMA engines are of the DMA4 design. These have h= igher performance due to their uncoupled read/write design and can access u= p to 40 address bits. Unlike the other DMA engines they are also capable of= performing write bursts. Note that they directly access the full 35-bit ad= dress bus of the BCM2711 and so bypass the paging registers of the DMA and = DMA Lite engines. > > > > > > DMA channel 11 is additionally able to access the PCIe interfac= e. > > > > > > END QUOTE > > > > > > The register map indicates (with some extra notes added): > > > > > > 0-6: DMA > > > > > > 7-10: DMA LITE (65536 bytes limit, for example) > > > > > > 11-14: DMA4 (11 is special relative to "PCIe interface") > > > > > > ("DMA Channel 15 is exclusively used by the VPU.") > > > > > > Yet what I see in the head -r365932 code is: > > > > > > #define BCM_DMA_CH_MAX 12 > > > > > > . . . > > > > > > struct bcm_dma_softc { > > > > > > device_t sc_dev; > > > > > > struct mtx sc_mtx; > > > > > > struct resource * sc_mem; > > > > > > struct resource * sc_irq[BCM_DMA_CH_MAX]; > > > > > > void * sc_intrhand[BCM_DMA_CH_MAX]; > > > > > > struct bcm_dma_ch sc_dma_ch[BCM_DMA_CH_MAX]; > > > > > > bus_dma_tag_t sc_dma_tag; > > > > > > }; > > > > > > . . . > > > > > > err =3D bus_dma_tag_create(bus_get_dma_tag(dev), > > > > > > 1, 0, BUS_SPACE_MAXADDR_32BIT, > > > > > > BUS_SPACE_MAXADDR, NULL, NULL, > > > > > > sizeof(struct bcm_dma_cb), 1, > > > > > > sizeof(struct bcm_dma_cb), > > > > > > BUS_DMA_ALLOCNOW, NULL, NULL, > > > > > > &sc->sc_dma_tag); > > > > > > As an example: does that deal with the likes of DMA LITE (so 7-= 10) "limiting > > > > > > the maximum transferable length to 65536 bytes"? > > > > > > As another example: Does it deal with the DMA4 (11-14) distinct= ions (if > > > > > > such were in use anyway)? > > > > > > For reference from the fdt print / : > > > > > > / { > > > > > > . . . > > > > > > #address-cells =3D <0x00000002>; > > > > > > #size-cells =3D <0x00000001>; > > > > > > . . . > > > > > > soc { > > > > > > compatible =3D "simple-bus"; > > > > > > #address-cells =3D <0x00000001>; > > > > > > #size-cells =3D <0x00000001>; > > > > > > . . . > > > > > > dma-ranges =3D <0xc0000000 0x00000000 0x00000000 0x40000000>; > > > > > > . . . > > > > > > firmware { > > > > > > compatible =3D "raspberrypi,bcm2835-firmware", "simple-bus"; > > > > > > mboxes =3D <0x0000001c>; > > > > > > dma-ranges; > > > > > > . . . > > > > > > emmc2bus { > > > > > > compatible =3D "simple-bus"; > > > > > > #address-cells =3D <0x00000002>; > > > > > > #size-cells =3D <0x00000001>; > > > > > > . . . > > > > > > dma-ranges =3D <0x00000000 0xc0000000 0x00000000 0x00000000 0x4= 0000000>; > > > > > > . . . > > > > > > scb { > > > > > > compatible =3D "simple-bus"; > > > > > > #address-cells =3D <0x00000002>; > > > > > > #size-cells =3D <0x00000002>; > > > > > > . . . > > > > > > dma-ranges =3D <0x00000000 0x00000000 0x00000000 0x00000000 0x0= 0000000 0xfc000000 0x00000001 0x00000000 0x00000001 0x00000000 0x00000001 0= x00000000>; > > > > > > . . . > > > > > > pcie@7d500000 { > > > > > > compatible =3D "brcm,bcm2711-pcie"; > > > > > > . . . > > > > > > #address-cells =3D <0x00000003>; > > > > > > . . . > > > > > > #size-cells =3D <0x00000002>; > > > > > > . . . > > > > > > dma-ranges =3D <0x02000000 0x00000000 0x00000000 0x00000000 0x0= 0000000 0x00000000 0xc0000000>; > > > > > > . . . > > > > > > v3dbus { > > > > > > compatible =3D "simple-bus"; > > > > > > #address-cells =3D <0x00000001>; > > > > > > #size-cells =3D <0x00000002>; > > > > > > . . . > > > > > > dma-ranges =3D <0x00000000 0x00000000 0x00000000 0x00000004 0x0= 0000000>; > > > > > > . . . > > > > > > > > > > rpi_DATA_2711_1p0.pdf reports: > > > > > (I ignore 2D DMA transfer mode here.) > > > > > For DMA engines 0-6: XLENGTH has bits 29:0 > > > > > bits 31:30 are write as 0, read as do not care. > > > > > That would put maxsegsz as 2**30 =3D=3D 1,073,741,824 > > > > > which matches a 1 GiByte space. > > > > > For DMA LITE engines 7-10: XLENGTH has bit 15:0 > > > > > bits 31:16 are write as 0, read as do not care. > > > > > That would put maxsegsz as 2**16 =3D=3D 65,536. > > > > > For DMA4 engines 11-14: XLENGTH has bits 29:0 > > > > > bits 31:30 are write as 0, read as do not care. > > > > > That would put maxsegsz as 2**30 =3D=3D 1,073,741,824 > > > > > which is smaller than the 3 GiByte space associated > > > > > with xHCI. > > > > > > rpi_DATA_2711_1p0.pdf reports the following specifically for > > > DMA11-DMA14 (so the DMA4 engines) for what goes in the CB and > > > NEXT_CB ADDR fields: > > > QUOTE > > > The address must be 256-bit aligned and so the bottom 5 bits of the b= yte address are discarded, i.e. write cb_byte_address[39:0]>>5 into the CB > > > END QUOTE > > > This is not true for DMA0-DMA10 (DMA and DMA LITE). > > > The following is extracted from various places to > > > bring them together. I do not see evidence of handling > > > the cb_byte_address[39:0]>>5 involved for DMA11-DMA14: > > > #define ARMC_TO_VCBUS(pa) bcm283x_armc_to_vcbus(pa) > > > vm_paddr_t > > > bcm283x_armc_to_vcbus(vm_paddr_t pa) > > > { > > > struct bcm283x_memory_soc_cfg *cfg; > > > struct bcm283x_memory_mapping *map, *ment; > > > > > > /* Guaranteed not NULL if we haven't panicked yet. */ > > > cfg =3D bcm283x_get_current_memcfg(); > > > map =3D cfg->memmap; > > > for (ment =3D map; !BCM283X_MEMMAP_ISTERM(ment); ++ment) { > > > if (pa >=3D ment->armc_start && > > > pa < ment->armc_start + ment->armc_size) { > > > return (pa - ment->armc_start) + ment->vcbus_st= art; > > > } > > > } > > > > > > /* > > > * Assume 1:1 mapping for anything else, but complain about it = on > > > * verbose boots. > > > */ > > > if (bootverbose) > > > printf("bcm283x_vcbus: No armc -> vcbus mapping found: = %jx\\n", > > > (uintmax_t)pa); > > > return (pa); > > > > > > > > > } > > > static void > > > bcm_dmamap_cb(void *arg, bus_dma_segment_t *segs, > > > int nseg, int err) > > > { > > > bus_addr_t *addr; > > > > > > if (err) > > > return; > > > > > > addr =3D (bus_addr_t*)arg; > > > *addr =3D ARMC_TO_VCBUS(segs[0].ds_addr); > > > > > > > > > } > > > Note ds_addr assignments in: > > > static bus_size_t > > > _bus_dmamap_addseg(bus_dma_tag_t dmat, bus_dmamap_t map, bus_addr_t c= uraddr, > > > bus_size_t sgsize, bus_dma_segment_t *segs, int *segp) > > > { > > > bus_addr_t baddr, bmask; > > > int seg; > > > > > > /* > > > * Make sure we don't cross any boundaries. > > > */ > > > bmask =3D ~(dmat->common.boundary - 1); > > > if (dmat->common.boundary > 0) { > > > baddr =3D (curaddr + dmat->common.boundary) & bmask; > > > if (sgsize > (baddr - curaddr)) > > > sgsize =3D (baddr - curaddr); > > > } > > > > > > /* > > > * Insert chunk into a segment, coalescing with > > > * previous segment if possible. > > > */ > > > seg =3D *segp; > > > if (seg =3D=3D -1) { > > > seg =3D 0; > > > segs[seg].ds_addr =3D curaddr; > > > segs[seg].ds_len =3D sgsize; > > > } else { > > > if (curaddr =3D=3D segs[seg].ds_addr + segs[seg].ds_len= && > > > (segs[seg].ds_len + sgsize) <=3D dmat->common.maxse= gsz && > > > (dmat->common.boundary =3D=3D 0 || > > > (segs[seg].ds_addr & bmask) =3D=3D (curaddr & bmas= k))) > > > segs[seg].ds_len +=3D sgsize; > > > else { > > > if (++seg >=3D dmat->common.nsegments) > > > return (0); > > > segs[seg].ds_addr =3D curaddr; > > > segs[seg].ds_len =3D sgsize; > > > } > > > } > > > *segp =3D seg; > > > return (sgsize); > > > > > > > > > } > > > Note cb_phys and ch->vc_cb in: > > > static int > > > bcm_dma_init(device_t dev) > > > { > > > . . . > > > /* setup initial settings */ > > > for (i =3D 0; i < BCM_DMA_CH_MAX; i++) { > > > . . . > > > err =3D bus_dmamap_load(sc->sc_dma_tag, ch->dma_map, cb_virt, > > > sizeof(struct bcm_dma_cb), bcm_dmamap_cb, &cb_phys, > > > BUS_DMA_WAITOK); > > > if (err) { > > > device_printf(dev, "cannot load DMA memory\n"); > > > break; > > > } > > > > > > ch->cb =3D cb_virt; > > > ch->vc_cb =3D cb_phys; > > > > > > > > > . . . > > > int > > > bcm_dma_start(int ch, vm_paddr_t src, vm_paddr_t dst, int len) > > > { > > > struct bcm_dma_softc *sc =3D bcm_dma_sc; > > > struct bcm_dma_cb *cb; > > > > > > if (ch < 0 || ch >=3D BCM_DMA_CH_MAX) > > > return (-1); > > > > > > if (!(sc->sc_dma_ch[ch].flags & BCM_DMA_CH_USED)) > > > return (-1); > > > > > > cb =3D sc->sc_dma_ch[ch].cb; > > > cb->src =3D ARMC_TO_VCBUS(src); > > > cb->dst =3D ARMC_TO_VCBUS(dst); > > > > > > cb->len =3D len; > > > > > > bus_dmamap_sync(sc->sc_dma_tag, > > > sc->sc_dma_ch[ch].dma_map, BUS_DMASYNC_PREWRITE); > > > > > > bus_write_4(sc->sc_mem, BCM_DMA_CBADDR(ch), > > > sc->sc_dma_ch[ch].vc_cb); > > > bus_write_4(sc->sc_mem, BCM_DMA_CS(ch), CS_ACTIVE); > > > > > > > > > #ifdef DEBUG > > > bcm_dma_cb_dump(sc->sc_dma_ch[ch].cb); > > > bcm_dma_reg_dump(ch); > > > #endif > > > > > > return (0); > > > > > > > > > } > > > It looks to me like FreeBSD is not set up to use the DMA4 > > > engines (DMA11-DMA14) and happens to not use them for the > > > DTB that I get from u-boot.bin in my context. > > > Of course, I may just have missed something in looking > > > around at the unfamiliar material. > > =3D=3D > > Mark Millard > marklmi at yahoo.com > ( dsl-only.net went > away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Wed Sep 30 22:40:12 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C9294432B0F for ; Wed, 30 Sep 2020 22:40:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-25.consmr.mail.gq1.yahoo.com (sonic303-25.consmr.mail.gq1.yahoo.com [98.137.64.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C1rmq6PkFz3d4w for ; Wed, 30 Sep 2020 22:40:11 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: RiZbZswVM1kqYZpqFPhhDKN1gxC8UCuj31_crPr7AciRa0rGhimaLynmi0SLpu3 SJpjONPNdbo0d4kKC8y_tEGccnuVkTjG8ldr8nd1dZ0u6pVn1kwekQwTBVzWnX9yhE3c8zeD49K3 tnErHEvsax9VO4GX7dEMBoYYY1YkOhEn7_nkiutQMetD8GtCybEwLWvwchKfpGCFpKF5Ro3xbvLg LrlhsUDty_YMgtZ9Byi1IfswUlmJm3G_n4OBlB3HCSjgceQ3GhzKG8T5N2XpZ2Z6LrXhF_Zz1RBW H4HIpffNgV6HDgp0KWTLHtsixvrja61c6od33TfUYEn3kZ3TraIoPKAr4nuSo1fMZWf2bSL7F0EB UsbnHC5AmQIPF4P7EZJgU6WX882sC_GPgffy6EW8okJLn9WXyfN6nE5P8oB09HYNBeDEbn5Q0775 pK7G0wy.NGSSKw72h4xP8tX0ctZoZu9YG0RbpEOvK0SFJU_CYvxzuoBUCl3CErqMW8VVcQY_xDWM DRZLTY3pv5QlylJt_jkMoFwt8fQN_3UEGpm5pLr0OQmUuAS7ptmrh4HEmCcomQxPOjzBJEas8OoB 8ErJRy5w1d9Vn7YBhWZy2b664IgYm6PEUKvyC_IZl1npbi0BHZSjX6eUYiLv8qbA7svIxpXUnw.v zp6AYvfCVDe.NctRWcWH7NTH7aOZbj8ywR.mzjsCc70DEiL9.smTZ9qS98rCBbHqkorCpnjTjKoR FbYWybv1.5WL1ZH4nt4q1Qn8QqoWrZbbGTJ.umB9Sxm_EE1WNW489HhbiAVsYpyj6eHtUGv_kCTW VaY3daPCqLoR2q8EMmOG0QKp4woi3qELm8jDR0KxJVN90TK.Biy9yh5z57xagIS8PQdVYffbOpUu 3GkiPEUg2kgOOo2Q5h.nZS40mCrJuFHBI8bwn.ngFpxY8ZYIikfX2p8ZUR1o3Trn950S6MbyVTR9 EvPn0zUbox.i3sacHgHHh89k7SDUF3ZmUtmexdH2nMhFDZDe815DUDIqE.sv640pXHjP.AJFp_ga U9gXqsIQoYAQKN7c1IFbOWwi8PDu4FlL7QYqopZ2RILJTwcwPgTuYfWDlWN7cfwoGe4360Yd.ge2 7Rq7ij5XhhnEhMHt14X_6UtzErPXM6q9T3QzlO1AYHa45eVuPX137zoECPHNg24CPfPgc2rIF_Ep tczCD.KZnSNhlPW5oZMQdGekFrJGYoyqmf2Q3LQRVpS213gzGeNh2Exf.wsPB8OC6v.8jr65eHJ3 ugzrqvz71CVH5C4lp8Hl3eiEvQRMAtOQLCMEdeLMQyB.MEnzp0FKJinf3QQGm3tZ6aK_6uL1u8fM FGwyxxslcVPzAl72eIdwOUB.g2vKxz7f6i.y_qTZQDnYPggofkPcJklt0uxYjX3mfA9PQ4A7Ddjd 3F_f0GlsTO.T9Rhfu9dXiFv19dMbmuI9itSjdC9bpT11LG8PtHATSzPXoUTiJSHL.heatfVpRgux dJAcE58ZVNCaDJkoOtn_uY3YgNCYrdmF9syiyegRLHYY_qxtmsPfLIWe3XuJeOU5P2FlqsNv7NlG QVYr3PWgYaU4KkZwBLbsnKzmRW678Gd8BdPtT Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Wed, 30 Sep 2020 22:40:10 +0000 Received: by smtp407.mail.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 87a43b783f74bd1860a62052778f512d; Wed, 30 Sep 2020 22:40:07 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: RPi4B's DMA11 (DMA4 engine example) vs. xHCI/pcie From: Mark Millard In-Reply-To: Date: Wed, 30 Sep 2020 15:40:06 -0700 Cc: freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: <93A90975-39F1-4DA3-87E0-89E07505108A@yahoo.com> References: <8C6DE44F-6CE2-4C74-8748-3BBFB54AE183@yahoo.com> <0FE382AB-8DE3-4467-9CB0-E8582AC70EA2@yahoo.com> <85FEDC51-B5B0-4ED4-A5ED-62B63EF9D5A8@yahoo.com> <903FE769-ED46-4FBC-A272-4D2C89A9CD7A@yahoo.com> To: Robert Crowston X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Rspamd-Queue-Id: 4C1rmq6PkFz3d4w X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.28 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.74)[-0.745]; FREEMAIL_TO(0.00)[protonmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.03)[-1.025]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.011]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.206:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.206:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Sep 2020 22:40:12 -0000 On 2020-Sep-30, at 14:15, Robert Crowston = wrote: > Very interesting analysis. Certainly uncovered a few things I wasn't = aware of. >=20 > By default sc->sc_bus.dma_bits in xhci_init is 64 bits; I toggle it = back to 32 bits in the xhci shim I wrote for the Pi 4. You can see that = output in a verbose dmesg. >=20 My biggest worry from all that material, for things as they currently are, is that it appears that FreeBSD could try to use a RPi4B DMA LITE engine but not follow its limitations when doing so. For example, ending up using a smaller size DMA transfer than intended (just 16 bits for size). (Other RPi's might have the same issue?) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Thu Oct 1 03:35:46 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AEE5A3F0D0A for ; Thu, 1 Oct 2020 03:35:46 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-23.consmr.mail.gq1.yahoo.com (sonic304-23.consmr.mail.gq1.yahoo.com [98.137.68.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C1zKs5jPvz49bq for ; Thu, 1 Oct 2020 03:35:45 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: 7amKlssVM1lvycgWVfM_pQ.hAFivUuxVDdCzgnolhcB3IFkK3bnK0nA1XdX_vxH rzQbroZ9Ym2Iz0YWoTxPTtV8ZGcGp84YNk_wv6Dfr4bktcgfg1u4AmR.bokVbQy4v6XAe21CkCYo rDfvD7EOpT3hqd.VBLB7lTDW4WHR6.DaGSnVywbE_K4ywTD_h2z72frw27TuJgCphxc8VI6cG1_O oZF3DFNN91u8NXehwPEFFZB9E1kxVa4HwIMVMz67Ovg6Vvnm09RZkwQ.SngDLfIpdN4lq5pb5.pI odob8CoOGIb7oRzvY1AHNbyGrXNLurvv9izbYZXxUedsguTuy6uw8DmkTlnnTLQ4OWLdan0HPG3m VV1yj0dB6IAHxVi8cFwX3kTXiNtHlY6_dkve2Gzf320HCnf9xCHGB6ctdJhci2bZwsche34fqGNT aSKr1nDVFPNlahjbs_3eYz83..GgWYitTmY3HNiHGZkMSBYs.jwmGWqDP0hwq4eL.Ny8UnrsMyH4 .t9ccp7Ahap.s0U1cNNA3n2VOPE.oKBKk0ARxomU7YZvgaUroY6A0CGxAnSm3Nl.oEI622bz0_Jg vTD23mLKLGZyLZAYg6zcTTgpfOLsgJIb8uNQVbfxxDmifrW7LgWZwy1fI2U3K96JQMy_k1gQBVcO bdO2Ee.fp_qq_2rd4_YU73F4BOycrVhd_ezere5HQWtisymZ9P3vzyiY_LnmjSKXgO2rHQz5udyR CVv70h2amyIwL_43uaX9tFzu.ZL_x6s7x2CBmcEDcRvn31zo0FWpGoqMudjciRu3Jgj3zrIPA1dK izf2NihJVweAIZXV4nXKz9nZRIVmo3fqIjQMbA6FfHm_De.vMJphQvPQbKC0bRYW0k6qMX.jqIbd ieZ4REhe6ZM1ZRkFzGbTS3odvLycSexjVRsav31M48QkFkQzpf00FwXRO1Cgyz3DMKJeeawMcoqf Fnlc755LFUQIahh1NOy8Pyze.4dnvQo2.lOmwasXdGvKBXtZ8Kiucg5rstxSKWHcZTQzUI9lChEH nIDLEgSFbq0wjpxXh9K5sbr1Lp1xDA6wjgxDe_i1jfhRVsre1.S9JIOBczgrnj7MT4TAORKHzI0m cRGBTkOiREnZBv36.5aXTcKwH1Bc_STchDCsuBiO4Cu0bNuxNCMtdAsUtin66Bw57SnJVAQ.qYvc 2Q4lJuhRcyU8q7xgvJE93_vYjXJaMjK71TAVUUS9R_A40mCaFvKw62Tv0U5n_slrjkskph2zcMfQ Z5burahbIzyqcHNrKU.JQywe4fU1XRqtZ4lNt6VBVKlaPgYq1wx9NtNpe77aBJK9Tz5SpB2dCA0e HDcmXaOdExFG_U0LQ03tGTfDyV1qDtptNJbnIieUhhjAWcRGygUJuLaJ7oeZLD0SVLwoN8XZDbBr RxB1Io9aaEjtD4h7kMPkwc._66FXuR3Kd4iZSQdYM2NG16uG53QyMTm0THaKvJmcb99IIlDb9c7h p3sbtnushQok3_N4ZiI5RoLqssafc0xME6v8VtZz1zfAU.CMQjX7qp1G8ZNgy_SKcaulRqj2H1tr _3nuwOigDOkycf2Y05yNjp7HlRUWGmrAGBA-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Thu, 1 Oct 2020 03:35:43 +0000 Received: by smtp424.mail.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 53be656cd6b87014279276bd5c46e95e; Thu, 01 Oct 2020 03:35:42 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: RPi4B's DMA11 (DMA4 engine example) vs. xHCI/pcie From: Mark Millard In-Reply-To: <93A90975-39F1-4DA3-87E0-89E07505108A@yahoo.com> Date: Wed, 30 Sep 2020 20:35:41 -0700 Cc: freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: <658D626D-4307-44F2-9B8A-1CE132EEB0F2@yahoo.com> References: <8C6DE44F-6CE2-4C74-8748-3BBFB54AE183@yahoo.com> <0FE382AB-8DE3-4467-9CB0-E8582AC70EA2@yahoo.com> <85FEDC51-B5B0-4ED4-A5ED-62B63EF9D5A8@yahoo.com> <903FE769-ED46-4FBC-A272-4D2C89A9CD7A@yahoo.com> <93A90975-39F1-4DA3-87E0-89E07505108A@yahoo.com> To: Robert Crowston X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Rspamd-Queue-Id: 4C1zKs5jPvz49bq X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.28 / 15.00]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.74)[-0.743]; FREEMAIL_TO(0.00)[protonmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.03)[-1.027]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.009]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.204:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.204:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Oct 2020 03:35:46 -0000 On 2020-Sep-30, at 15:40, Mark Millard wrote: > On 2020-Sep-30, at 14:15, Robert Crowston = wrote: >=20 >> Very interesting analysis. Certainly uncovered a few things I wasn't = aware of. >>=20 >> By default sc->sc_bus.dma_bits in xhci_init is 64 bits; I toggle it = back to 32 bits in the xhci shim I wrote for the Pi 4. You can see that = output in a verbose dmesg. >>=20 >=20 > My biggest worry from all that material, for things as they currently > are, is that it appears that FreeBSD could try to use a RPi4B DMA LITE > engine but not follow its limitations when doing so. For example, > ending up using a smaller size DMA transfer than intended (just 16 = bits > for size). (Other RPi's might have the same issue?) So I looked at BCM2835-ARM-Peripherals.pdf, comparing to rpi_DATA_2711_1p0.pdf and: BCM2835-ARM-Peripherals.pdf indicated DMA engines 7-14 are all DMA LITE engines for the context it was covering. DMA 0-6 are normal DMA engines. So the RPi4B reduced the number of DMA LITE engines (now only 7-10) and filled the removed slots (11-14) with the new type DMA4 engines. So I learn from this some about how to interpret "brcm,bcm2835-dma" as shown by what the RPi4B is doing: dma@7e007000 { compatible =3D "brcm,bcm2835-dma"; reg =3D <0x7e007000 0x00000b00>; interrupts =3D * 0x0000000007ef645c = [0x00000084]; interrupt-names =3D "dma0", "dma1", "dma2", = "dma3", "dma4", "dma5", "dma6", "dma7", "dma8", "dma9", "dma10"; #dma-cells =3D <0x00000001>; brcm,dma-channel-mask =3D <0x000001f5>; phandle =3D <0x0000000b>; }; The brcm,dma-channel-mask value and name list is different than in prior examples of "brcm,bcm2835-dma" but the subset that is present follows = the original protocol. For example, how to tell DMA (normal) from DMA LITE works the same [a debug register bit to check] --but only within the = subset present. Seeing "brcm,bcm2835-dma" should not lead to presuming some fixed number = of interrupts or dma-channels to process. The protocol for DMA (normal) vs. DMA LITE does not even apply to the new RPi4B DMA4 engines (11-14): dma@7e007b00 { compatible =3D "brcm,bcm2711-dma"; reg =3D <0x00000000 0x7e007b00 0x00000000 = 0x00000400>; interrupts =3D <0x00000000 0x00000059 0x00000004 = 0x00000000 0x0000005a 0x00000004 0x00000000 0x0000005b 0x00000004 = 0x00000000 0x0000005c 0x00000004>; interrupt-names =3D "dma11", "dma12", "dma13", = "dma14"; #dma-cells =3D <0x00000001>; brcm,dma-channel-mask =3D <0x00007000>; phandle =3D <0x0000003d>; }; 11-14 have a new, distinct protocol in various respects. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Fri Oct 2 20:56:13 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0F5A0436C6F for ; Fri, 2 Oct 2020 20:56:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic301-22.consmr.mail.gq1.yahoo.com (sonic301-22.consmr.mail.gq1.yahoo.com [98.137.64.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C32Mv6whCz4fqP for ; Fri, 2 Oct 2020 20:56:11 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: _FUOLVcVM1m21bkLFdi.eF7hzBWi9FhW.BzUIrxrKgXkgOw3ST2axjWBrHlu.4U RQv0p9M.w7_2TmEZhmiwyrGAq7nuu144OKSQq0p1358iE68756mGLuDVvDBAbLqEW1J0EOkhJ12O 3jqUWfi.XWQ50ueY7FBSdoPHdQHgoW1nBss9aFMez3hZu4hXpobmot0G.ZmNJ49xsJG8G9lWFZ67 zbYBOaHHsJZ8J09Jbspim3aPLFUdOkln1txUCHQu3ceUuv9qaKXzsC.LHSodmY4YAPfhc8jnO1Vl 9ChcF4hJA52SJbj2slgIvjQYtbdEzVIP6CohEcf2FWN4iygKUMekJACQ7Mm2jKA2Uu_kj7L3w5nL .6UIG61xBefVMcJ9oiPojQZgw__zRFh9uYy5u_mR3zJTtkfN_MBQoWJ1U9gh7WVKffvLq2qGGyt2 tkQGS84iuNQ9PwHCG8l3gH.X2cbkyHeChYylZ_cArzxKeWYv2xug_7UCYnHSCNh0pZk1YkqGx6RE QAW4AVqbknidqDhMzvfnnjQDawLO6J5OoO7cmNsDH.OFBYbzHFyTtnBpo3QrmGQtTzDXtInT6IYL 6jgcEUt.rQRMUMzHenzAJT65T_Upm9B40stJZmKHpaog.JAJQct6Io9rmjZd6n81zMI8M_3d3r7Y QufM3cYTVk0GZ6t3etgXIgd35VYwnjlEDzVzZPDdJp3Q120wF0wbLOupQs1CfNApJ3v91D1pGYLz VEAUYFqwU2BTR7WwLl37RpLtr0i1.Ublax5SHDRZTdQp0nbyaqtfJgU04ohCj1XZEmtdhNZk7HrD uc8iabkLKa9jJoWxmBFcY.vYGRKCwm0_jzqdqhtbHvLLmyj3ymiMLE9TnnifgFrbncPQVGGR4RXU xDsP.tvzKfuo0BX0B5K8Tb1gk6D.CB8FoXAMk9uY3z.DLpexh_DWfEZX2tAriLtqGTy_I80bfgWz aJZU6hkA1bDZdA2H_gi2ZMFNSgpaIfEgab6atDFTxtOF0I30RO_IM28ICd7xSSfIUqmSwOG2CIFO t6VF9Ojw1NjVMFpcoAgNqGqCCHnwqZE0uAN6IBzEyLwi27w3THSXfpUjBu3RYkM6tmFUwUnOqOYC Q6pOCt8TEbbTOphhHzuGYitYhfV15Xp9u8d8U1UIdnUG.S_n3rz5ovgueO872B9uFnr1CNcrrMUj _Br9cuDZsqzHUMwTzJlyw9.uFMvDGa.17mz7o3XsOr_a0RFI5IU2nxyw9edU1ZmhAKMUSlqShhjC 4VXk8lc40A6teDRzFt95rNqWLLTwofWpNQCjx2zwFOQerNGxcADZCgEv5BAAOegkQKEvXM5v.FGk vN7GWEQh1_st.4gyo4aKSJ0Ldof2qeJ.TwEbH_AZ3AjNbDr8in6EgCahIknF7SUjhimIEyD0pnJn fnTHQkKy_IhKPaTkVhJu8kJCp8aT46L5KvYv6gUi7zu5iIwZw81WFHus1iUvMz_rzyg0WEHRKhQo u7U.m0fW3H_ag1JBRy9gsodxX1SfamP0rZVE5l8YHX9S7jPP.bSajNyUyCnOkX9kCPSZwZxj7Mh_ i Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.gq1.yahoo.com with HTTP; Fri, 2 Oct 2020 20:56:09 +0000 Received: by smtp406.mail.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 8cd7209c751b34c66d37cc3ffba9c419; Fri, 02 Oct 2020 20:56:06 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: RPi4B 3 GiByte pcie limitation: The following change survived my huge-file duplicate-and-diff tests so far Message-Id: <7506FD70-D814-4F64-BFAF-CCCE9A0D71A3@yahoo.com> Date: Fri, 2 Oct 2020 13:56:04 -0700 To: Robert Crowston , freebsd-arm X-Mailer: Apple Mail (2.3608.120.23.2.1) References: <7506FD70-D814-4F64-BFAF-CCCE9A0D71A3.ref@yahoo.com> X-Rspamd-Queue-Id: 4C32Mv6whCz4fqP X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.82 / 15.00]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.29)[-0.287]; FREEMAIL_TO(0.00)[protonmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.04)[-1.044]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.991]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.148:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.148:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Oct 2020 20:56:13 -0000 It appears to me that 3 GiByte works for lowaddr and maxsize but that the maxsegsz needs to be limited to 1 GiByte [or so, maybe (1 Gi - 1) Bytes]. I did the following based on all those notes that I sent out, that included the ?_TXFR_LEN being limited 30 bits for (normal) DMA engines (0-6) and the DMA4 engines (11-14) [but not DMA LITE (7-10)]: # svnlite diff /usr/src/sys/arm/broadcom/bcm2835/bcm2838_pci.c = /usr/src/sys/arm/broadcom/bcm2835/bcm2838_xhci.c Index: /usr/src/sys/arm/broadcom/bcm2835/bcm2838_pci.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/arm/broadcom/bcm2835/bcm2838_pci.c (revision = 365932) +++ /usr/src/sys/arm/broadcom/bcm2835/bcm2838_pci.c (working copy) @@ -105,7 +105,8 @@ * * Whatever the true maximum address, 960 MiB works. */ -#define DMA_HIGH_LIMIT 0x3c000000 +#define DMA_SEG_HIGH_LIMIT 0x3c000000 +#define DMA_TOTAL_HIGH_LIMIT 0xc0000000 #define MAX_MEMORY_LOG2 0x21 #define REG_VALUE_DMA_WINDOW_LOW (MAX_MEMORY_LOG2 - 0xf) #define REG_VALUE_DMA_WINDOW_HIGH 0x0 @@ -642,12 +643,12 @@ */ error =3D bus_dma_tag_create(bus_get_dma_tag(dev), /* parent */ 1, 0, /* alignment, bounds */ - DMA_HIGH_LIMIT, /* lowaddr */ + DMA_TOTAL_HIGH_LIMIT, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ - DMA_HIGH_LIMIT, /* maxsize */ + DMA_TOTAL_HIGH_LIMIT, /* maxsize */ BUS_SPACE_UNRESTRICTED, /* nsegments */ - DMA_HIGH_LIMIT, /* maxsegsize */ + DMA_SEG_HIGH_LIMIT, /* maxsegsize */ 0, /* flags */ NULL, NULL, /* lockfunc, lockarg */ &sc->dmat); Then, with such a kernel installed, I used: -rw-r--r-- 1 root wheel 11570948096 Jul 18 18:32:37 2020 = clang-armv7-on-aarch64.tar (so much larger than the 8 GiByte RAM) and did the following with the file: # cp -aRx clang-armv7-on-aarch64.tar clang-armv7-on-aarch64.alt_tar # diff clang-armv7-on-aarch64.tar clang-armv7-on-aarch64.alt_tar # Such tests have been passing. I do not know what your test procedures were. But if the above is suggestive, you might want to try testing something like the above via your test procedures. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Sat Oct 3 02:35:58 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8EE083F60EA for ; Sat, 3 Oct 2020 02:35:58 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic307-8.consmr.mail.gq1.yahoo.com (sonic307-8.consmr.mail.gq1.yahoo.com [98.137.64.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C39vw0QCdz4vRy for ; Sat, 3 Oct 2020 02:35:55 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: 4vnYlJEVM1kPiEo7DiIgPBvPWU44.vzCWUzkd3Zv7n4Gc6KerPI58X_qpapRdQI fhJndhxPR44gEFQa4qIJG570ZaktbrLLLdnu_yYpMiSTQv0I1Q1sZ1nzuuTDJ78ySz7E9VP15ys2 T0Ss1tUt_7vYjHU19qIue3I0co125BA4KMBsBxQFppcFrAF6kuPIS.5lAVGF4DfxFTuBBI1VDWoz IXGRPbgAg7nkSfzid_67uosJU.ilJv3mt7gtvIqb7V6n0ieIZPyBAxc1tfFhymvEJWQIBu_34WL7 2WRa.fkF5z515kPsEAc.Gz.fY.Xu44SM8dxPvm5pn773idXczrpzxWQnd5hzLW52zJehwI0RcXay MWrsabCh63SP2czjOlLH7vbtxZqn.cehOUpScw_XPpb6npcp68FrxsVeBRZN1.hCDgLwZH8ZCxmP 2a_TksPrEkWaUbsJz3sqaQ1GF3j_R2i1bTxKPJxff_Cvv9if03KjbWw_VZmDgVOiFZp8DXslxQkH WQWy5vmW_9050SGZQEFOuUn3nz0YUqPDlrErsOb4MhP8ddO1LadntUzirKdmBmfKUXlU20MILDLm KnuEhK57ISQT1pYVEPF_TWDlZcpWpSLTx7gCPez5tD3Ugt0s1oCvdySrjpJUAXYblgmbDWw8Cy3F lCHgyb8o7rSEEn4sKQiv9uszqeGcqs0ss5gQSE7PX4rSZxIqHiTewM_nUEb9UvdvIAnRYSvP2R4N fwv4EeCYv0xfnQJCLBUQY1u.PXZ.3.K00TAA5qQDEh.kudYThXg0mQh8drMwCvnCVdPnFAGJWr6r rhCnCxPZwMaHzZ9PSRaCS2GcBoKc82i_rCBGeyNSI.lxWCukuzK3CgHXSovweNlzlbknYuawV.G7 9ijFq091yP3bgysbguMb84H2rj4uyLsgTR7SbaWt9wE7f0ZvCOTOHZ9Dzgs6GECDShQq9CpJuIUl IvEm150FVZU741xP3.SrfDz1UuzUN7dAMFFIBIsq7ttEsSyfvQQrSTz.YU9ZeVCjuNHZRospkhJa taJPWhqlA6lxB1JXKbUevyBAPLljWEer7WKIbxJvwMFqt5sFMDcIdE3pwBkGTOD6nNNXqmhnhcio ynLXrVuyT6nraXDk5t_Yg4riW2ZpKeyUymGIB.nRNrWFYqOrRjCsRPH_z4IzXMG7IQIsc9_BXXWd dPRwmS7c1bUf.eSyO32ePy.mY5THt_6c1mnQkGab2l6IScGxKhF.q09tunWKad4OGw_Hcm.PBdUV POrO.3Kcc4sRO_1rlxBwwK_cfFDacm24HSqSoJOt26jJ2Q4Hjs6JO9.qk7H0WlrFIJ6QJsbd22yU CXO9Ody595kUE0DpqV8xUAqBuEB7f4adv1lHvpsWXSBuoBkWEY.KeDCt__BV0Iy04PGYxSG8_cOH PodzAxxRHUzB1QPpgek.cLlq2eIXUulGM3u43mwki5IzO5xTBhbrN1iZe5kKmRt7VzSi1S.H8bLb 0fbZlLgu289XcEchCjdw.t2yHOjHrgFE34O3dqMJZOQ7Wo2JFa9gxhhG.jskLVhS8Tdean1bb7Wh QOLcvMFAUPQ1bObEmxFaNl6e7V6xTbBBq2c8- Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.gq1.yahoo.com with HTTP; Sat, 3 Oct 2020 02:35:53 +0000 Received: by smtp420.mail.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID b3cf9aca45740e738faced47234fb048; Sat, 03 Oct 2020 02:35:48 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: RPi4B 3 GiByte pcie limitation: The following change survived my huge-file duplicate-and-diff tests so far Date: Fri, 2 Oct 2020 19:35:47 -0700 References: <7506FD70-D814-4F64-BFAF-CCCE9A0D71A3@yahoo.com> To: Robert Crowston , freebsd-arm In-Reply-To: <7506FD70-D814-4F64-BFAF-CCCE9A0D71A3@yahoo.com> Message-Id: <447B28E3-3964-4498-A91D-528EA5923793@yahoo.com> X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Rspamd-Queue-Id: 4C39vw0QCdz4vRy X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.23 / 15.00]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.70)[-0.695]; FREEMAIL_TO(0.00)[protonmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.04)[-1.044]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.991]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.32:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.32:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Oct 2020 02:35:58 -0000 [Operator error note and another note.] On 2020-Oct-2, at 13:56, Mark Millard wrote: > It appears to me that 3 GiByte works for lowaddr and > maxsize but that the maxsegsz needs to be limited to > 1 GiByte [or so, maybe (1 Gi - 1) Bytes]. >=20 > I did the following based on all those notes that I > sent out, that included the ?_TXFR_LEN being limited > 30 bits for (normal) DMA engines (0-6) and the DMA4 > engines (11-14) [but not DMA LITE (7-10)]: Ignore: I had the wrong kernel build installed. In fact the below fails when the intended kernel is tested. > # svnlite diff /usr/src/sys/arm/broadcom/bcm2835/bcm2838_pci.c = /usr/src/sys/arm/broadcom/bcm2835/bcm2838_xhci.c > Index: /usr/src/sys/arm/broadcom/bcm2835/bcm2838_pci.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- /usr/src/sys/arm/broadcom/bcm2835/bcm2838_pci.c (revision = 365932) > +++ /usr/src/sys/arm/broadcom/bcm2835/bcm2838_pci.c (working copy) > @@ -105,7 +105,8 @@ > * > * Whatever the true maximum address, 960 MiB works. > */ > -#define DMA_HIGH_LIMIT 0x3c000000 > +#define DMA_SEG_HIGH_LIMIT 0x3c000000 > +#define DMA_TOTAL_HIGH_LIMIT 0xc0000000 > #define MAX_MEMORY_LOG2 0x21 > #define REG_VALUE_DMA_WINDOW_LOW (MAX_MEMORY_LOG2 - 0xf) > #define REG_VALUE_DMA_WINDOW_HIGH 0x0 > @@ -642,12 +643,12 @@ > */ > error =3D bus_dma_tag_create(bus_get_dma_tag(dev), /* parent */ > 1, 0, /* alignment, bounds */ > - DMA_HIGH_LIMIT, /* lowaddr */ > + DMA_TOTAL_HIGH_LIMIT, /* lowaddr */ > BUS_SPACE_MAXADDR, /* highaddr */ > NULL, NULL, /* filter, filterarg */ > - DMA_HIGH_LIMIT, /* maxsize */ > + DMA_TOTAL_HIGH_LIMIT, /* maxsize */ > BUS_SPACE_UNRESTRICTED, /* nsegments */ > - DMA_HIGH_LIMIT, /* maxsegsize */ > + DMA_SEG_HIGH_LIMIT, /* maxsegsize */ > 0, /* flags */ > NULL, NULL, /* lockfunc, lockarg */ > &sc->dmat); >=20 >=20 > Then, with such a kernel installed, I used: >=20 > -rw-r--r-- 1 root wheel 11570948096 Jul 18 18:32:37 2020 = clang-armv7-on-aarch64.tar >=20 > (so much larger than the 8 GiByte RAM) and did the following with > the file: >=20 > # cp -aRx clang-armv7-on-aarch64.tar clang-armv7-on-aarch64.alt_tar > # diff clang-armv7-on-aarch64.tar clang-armv7-on-aarch64.alt_tar > # >=20 > Such tests have been passing. >=20 > I do not know what your test procedures were. But if the above > is suggestive, you might want to try testing something like the > above via your test procedures. Going in a different direction for potential maxsegsz figures (and the like): xhci instead of bcm2711 material . . . extensible-host-controler-interface-usb-xhci.pdf reports that normal TRB Transfer Length fields have bits 16:0 and "Valid values are 0 to 64K". (Transfer Event TRB Transfer Lengths have 23:0 but above 0x10000 as a value is only for Event Data flag being 1 or for condition code is stopped - short packet.) I've not experimented with this. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Sat Oct 3 20:01:21 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DF71E437090 for ; Sat, 3 Oct 2020 20:01:21 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic302-22.consmr.mail.gq1.yahoo.com (sonic302-22.consmr.mail.gq1.yahoo.com [98.137.68.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C3d685qV1z3Wqp for ; Sat, 3 Oct 2020 20:01:20 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: MAUedysVM1la5.nGZaM2t1XPbH4vWYNfDWT_SUt35jPzRkmNZLFx5zr_2fDjZbL R8_HY9i3nzZaEtra2HfFs3PM7sh_LN3rmgoXbK.VtnJiqELV30JjF4atXpBOVROL08EFZM6gi8y7 CuLhDddUnERmgyaHOfT_PrTSyV4zJj5ls5H7VQLYNrF9hCOvbKJSRSKk0ighmEBhHPN0vKRU2lBC ZGBn1dn_jOqgK9BVF49TiIYO.hw4MD417V4PjH.pfoAzPCg0nsxLZ8X80r2nZ0e_xbAiG9UyazFa 0hMKaRjENw4YuA7CcB8Sp9WpKejXpA3u3wM6W2qsPNA_QbxlybeAQYntcFkAK7Fyw_42PvuekWhn aYAceJPbNMDljN7UfzC.L0_.6VUr6xb9y4OcqGIbxVmEkTgyNIby_98UE2pGeUss188Qc9Wcx1hN mt.S1aHhs6Vl5XSlAMjTaFDqtRhixnjIgTk1J6Ej1e86tsW_TFVY7X4zPp5QlHowfeaXCeVAPnDY AA6.i7XFxkDkdOYTC6N58He6EunAfyeTiSfm0aoSrzWbgMasI1HrDSWHYCIpsb_wA6AXZo2Z6sOH 4GaxsNXbQ8.6hmJyQXhKybMWcwrofYUmd1JU2WZE4qYXHrdjOqkYzFUWoo1Kl1MHNVdGZ7_ZRObB YBPB6SHoAcMEVsq.r8SMRCK64iaNxYsO.xZWnTkM0.A4NsvFAVXjS8NEClDsMCgX2aDjGnGknzQz twDXz40PBz7rFUh2de_SNx6_ArVqQS_xrEOk47.ysKGNMpGKHZUeMQqv9H74SIZ0CLct1MMMNQBB ntk1ruIsJvjEJaL7hg8iOiobYuTfuoO2HelmJSmSBJhwiGhmf7bDUk.Z0LgSg8VbqkQqUM22fLKW _IMsy4WygA7ak8cngpL0CuQRbb6dZn3jiHJ04mhqyBFU3qT5N379elGg8kPtMY3riaUHe0F7z9Eb qr5k8tCbD4vHujRNSdNZ5HMcsjFuvV2c7gnbKgzW4SKVmwPBEgerwFNZztKXhg8OnQFCPN88YDyK o5wlxyWvR_mXID9Di1EO_yYF_jIMAKbm.uu.dh.0rOOzvb0ThdwRhSF4B7_T7xm6QCn12Srdyiv8 AGWbFIk1HZlloetaD689zgHma5iUIr7TZc3Wow8Nu97G0lLbaeYy.MKyAsk6u1EMRLUTJvK5Hs_q poUf.6zJOdJ3O1xJDDyErpWNqGYysiX6Dhy.N3Qj5eTlqQLDBg69pGjTvV1UCWCNgFjax6XlurRV 2p3yKgpEP3W50hzR89.NudBEACX0fhbq3bMuRMN6b1rVnUrxSqOVcfGJ4wPAK4c9VyxV58wX6v5d fiXCv586vgG3JXg8GMXNym0Bq13kZEu_KfYsrAqEMZfn58LYjdp3n_Z8MMJ2r4yMbmJ0HCje1rya ._PMJAMRMbV6nXd3OZQ25OvytsnAsB8Xwwo5uCIKP7PQ.q0fLHplBjsMf6ST5.v1xL5cMrfnhooL BfbwVC.pqIsWVd0i6TOi_SrbvNHJ7cqNmmQB5giKzFoE5IAr2oRlFmT8WkuQvInmp3_McnCH6Tp4 - Received: from sonic.gate.mail.ne1.yahoo.com by sonic302.consmr.mail.gq1.yahoo.com with HTTP; Sat, 3 Oct 2020 20:01:19 +0000 Received: by smtp425.mail.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 990d63517536facca49ec57f6aaee44c; Sat, 03 Oct 2020 20:01:16 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: lspci XHCI "Memory at" for RPi4 (u-boot based context): FreeBSD vs. ubuntu vs. "Memory behind bridge" addresses Message-Id: <8D837FD4-FC25-4D17-B008-141DC6A3D0FC@yahoo.com> Date: Sat, 3 Oct 2020 13:01:15 -0700 To: Robert Crowston , freebsd-arm X-Mailer: Apple Mail (2.3608.120.23.2.1) References: <8D837FD4-FC25-4D17-B008-141DC6A3D0FC.ref@yahoo.com> X-Rspamd-Queue-Id: 4C3d685qV1z3Wqp X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.32 / 15.00]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.74)[-0.737]; FREEMAIL_TO(0.00)[protonmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.04)[-1.045]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.04)[-1.040]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.148:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.148:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Oct 2020 20:01:22 -0000 [Be warned that I'm reporting differences that I do not understand. I may end up being told "nothing of interest in these differences".] In the later lspci -v output diff, there is the following for the XHCI (-: FreeBSD, +: ubuntu): - Memory at f8000000 (64-bit, non-prefetchable) . . . + Memory at 600000000 (64-bit, non-prefetchable) [size=3D4K] Odder(?) is comparison of the FreeBSD address with what is listed by both OS's for "Memory behind bridge": Memory behind bridge: f8000000-f80fffff [size=3D1M] . . . - Memory at f8000000 (64-bit, non-prefetchable) ubuntu gets a distinct address (600000000) and FreeBSD gets an exact match to the start of the "Memory behind bridge". (There may be a possible 32-bit address vs. 64-bit address distinction?) [There are also IRQ differences (81/82 for FreeBSD; 41/42 for ubuntu) and ubuntu lists the kernel drivers used.] For reference: # diff -u ~/rpi4-lspci_v-*.txt | more --- /root/rpi4-lspci_v-fbsd.txt 2020-10-03 12:19:57.162261000 -0700 +++ /root/rpi4-lspci_v-ubuntu.txt 2020-10-03 12:21:01.448646000 = -0700 @@ -1,5 +1,5 @@ -00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2711 (prog-if = 00 [Normal decode]) - Flags: bus master, fast devsel, latency 0, IRQ 81 +00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2711 (rev 10) = (prog-if 00 [Normal decode]) + Flags: bus master, fast devsel, latency 0, IRQ 41 Bus: primary=3D00, secondary=3D01, subordinate=3D01, = sec-latency=3D0 I/O behind bridge: 00000000-00000fff [size=3D4K] Memory behind bridge: f8000000-f80fffff [size=3D1M] @@ -9,13 +9,15 @@ Capabilities: [100] Advanced Error Reporting Capabilities: [180] Vendor Specific Information: ID=3D0000 Rev=3D0= Len=3D028 Capabilities: [240] L1 PM Substates + Kernel driver in use: pcieport =20 01:00.0 USB controller: VIA Technologies, Inc. VL805 USB 3.0 Host = Controller (rev 01) (prog-if 30 [XHCI]) Subsystem: VIA Technologies, Inc. VL805 USB 3.0 Host Controller - Flags: bus master, fast devsel, latency 0, IRQ 82 - Memory at f8000000 (64-bit, non-prefetchable) + Flags: bus master, fast devsel, latency 0, IRQ 42 + Memory at 600000000 (64-bit, non-prefetchable) [size=3D4K] Capabilities: [80] Power Management version 3 Capabilities: [90] MSI: Enable+ Count=3D1/4 Maskable- 64bit+ Capabilities: [c4] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting + Kernel driver in use: xhci_hcd =20 Is this odd? Expected/reasonable? =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Sat Oct 3 22:20:45 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D8A893F1DA6 for ; Sat, 3 Oct 2020 22:20:45 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic302-22.consmr.mail.gq1.yahoo.com (sonic302-22.consmr.mail.gq1.yahoo.com [98.137.68.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C3hC03bPbz3dT1 for ; Sat, 3 Oct 2020 22:20:44 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: tXAvh1oVM1l190kpUPHAy8jJWvLRpwqyN4kx9zXYD31Te9w13MN4uVQ7299ZMQE r3V6J_TOOax6XcPhQ_x3bEw8slKSgXTdsLGemdfmDgVRITSflvpuEhhJoI0YGzLAinfWEyEE9SeB GLx1_T6HIORLH.Tc5Nt2DR4hedLY4vhOg3VzFlL8HZhqquUXYkL0lS_.mlJK.Mlcqp7hch.uOam2 k9SopdItGv7EhOvoFWtsNPqlaDiGGEkRb6QKk3BNbnwELw48T2oRDNIlrgbEhvE3igUQKE8DpfV. o5pXml.Y4Mw7muzTuALeKugceitHKAB8Ic3YFMrdze90Q1L1X6SA6117.KxQbV0ttGIpa41mKYOc vT2YNU9QyWrkCoVPoY8CExbdjlDtzsRL35e6jZVoko7axlqAdmfU6jgmK9hBvA_NNdCGavKI50eC Pn4v9xU3CXaCgWMCYI4kkK10h9OvwAH1ShHAvgNp2cd4zNoHuRx12.gN.cQKChQKip_HfZXULxvu 1fxdThmWAgRR2j2oUd9XKwbrvWP9o7XpqYbTaAusDhXJXYwQzyrLf8Gpxt0xj5WF0XN8BU05LLgZ pYpB.IvSPje2AKVcqOwEB358I0h3RpoAeJ_9M5WMwGcqn8Gm8Mz.CUVMjQL0w4mTlpCla5gAiBK5 BDIfNVqDbiIl8cr_YS4tUEyMoTH7hu0se8S7rJKTZO0gU8ZALie0Om7lb9e5bR0czP4Du1Au1FQt ylLGcif56TSUWyDoqWRAT_9SC_0tdbi3FfpiVZGIGr3pOdw.sc0tsxakcQnC9hwg5rKDfvpfEgaG jQIM8e1M49x4YsvnWJe_vqe75c0iEQ7uLOJ6OWQ9ojaR056Ckms89zzv1Siu06s.KTJKasEFvBP9 lYJVPoCdcPEI0GbOyDzXU8bJK_AahI4ecG.Uwf9tsDuaYpK4vMbZz.50ZXD2dwL_nA9f8nnkmhVY YpoOBh3b.3hrZgAlErGapaN8HKeQHKLatuFC3JjktdWz_pcEdBiHMWegCKHJOeUokdGvqXJooDoF uG9ts6TNLG05uAkFMPf5C4YDTmdayfxCKZShoYFhF3DvFHrqRwnhP4tuT1LK3L2osg_7rADWUENA sC4mcSxNmYfFmzjFfjH313tZC.sg5cRpSyvqYAPYnT6cAk06TDl17hX2pPWF1Q6ei.vAGrkzldCO Vs7_TAncfVTJG_9s8sX3zQVkDcqkp_54zAoKw3huLhziptuKGrKnlekjwQ4v9HxeUDmXKOobdiNS loSzAUomlr3Sh9kOd9Lgb0tmr3QByjGo8MH4FeAmWBrkP1i8j0B5daQDHDgKveeIt5TTesA57Og4 Gy2a2OuV2e4fMIloAHoQmkHfCyBrwbe0lSZ3BsG8CHH5_.rm7geU7LxxpsVmMOKnPoKv2yqbnQo8 Cg3VEy.hZdJs5yFxs2MEoi8KWp5ZEaeniTocVkquN4RlxP2saecyb0VKJhGVzGhtfBs56mTgRoHp DcBH2m6UuvkfcmMzbWPRXtJNtU1Usi20vOS1d1.xvtv5GvpCAnIXD1dOt5hd_srkbelKFjHOInXN iBBsrxq87Ofx3JqfT0t.W8mNupLj6i7B_zYw- Received: from sonic.gate.mail.ne1.yahoo.com by sonic302.consmr.mail.gq1.yahoo.com with HTTP; Sat, 3 Oct 2020 22:20:41 +0000 Received: by smtp412.mail.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 0adeea8854ac835ddb137f94387c48ff; Sat, 03 Oct 2020 22:20:38 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: rpi4 FreeBSD vs. ubuntu u-boot fdt print / memereserve difference (lack of reserve in FreeBSD context), 0x3b400000 vs. DMA_HIGH_LIMIT being 0x3c000000 Message-Id: <1A13F7B5-F8C3-4022-939C-2992E53D1DF1@yahoo.com> Date: Sat, 3 Oct 2020 15:20:37 -0700 To: Robert Crowston , freebsd-arm X-Mailer: Apple Mail (2.3608.120.23.2.1) References: <1A13F7B5-F8C3-4022-939C-2992E53D1DF1.ref@yahoo.com> X-Rspamd-Queue-Id: 4C3hC03bPbz3dT1 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.29 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.70)[-0.703]; FREEMAIL_TO(0.00)[protonmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.05)[-1.045]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; SH_EMAIL_DBL_DONT_QUERY_IPS(0.00)[0.0.0.0:email]; NEURAL_HAM_LONG(-1.04)[-1.040]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DBL_PROHIBIT(0.00)[0.0.0.0:email]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.148:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.148:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Oct 2020 22:20:45 -0000 Another FreeBSD vs. ubuntu context difference, this time in the fdt print / output . . . The ubuntu u-boot has (fdt print / output): memreserve =3D <0x3b400000 0x04c00000>; . . . (Note: 0x3b400000+0x04c00000 =3D=3D 0x40000000) . . . =20 #address-cells =3D <0x00000002>; #size-cells =3D <0x00000001>; . . . axi { vc_mem { reg =3D <0x3ec00000 0x40000000 0xc0000000>; }; Note: "vc_mem is solely used as a mechanism for passing a couple of parameters through from the firmware to vcdbg" End note }; . . . ( boot args has: vc_mem.mem_base=3D0x3ec00000 = vc_mem.mem_size=3D0x40000000 ) . . . reserved-memory { #address-cells =3D <0x00000002>; #size-cells =3D <0x00000001>; ranges; phandle =3D <0x0000003d>; linux,cma { compatible =3D "shared-dma-pool"; size =3D <0x04000000>; reusable; linux,cma-default; alloc-ranges =3D <0x00000000 0x00000000 = 0x30000000>; phandle =3D <0x0000003e>; }; }; . . . (I split the reg into lines below) . . . memory@0 { device_type =3D "memory"; reg =3D <0x00000000 0x00000000 0x3b400000 0x00000000 0x40000000 0xbc000000 0x00000001 0x00000000 0x80000000 0x00000001 0x80000000 0x80000000>; }; . . . (Note: 0x40000000+0xbc000000 =3D=3D 0xFC000000) . . . (I've ignored gpiomem above and below.) It appears to be that the memreserve may be important to have. The above may also suggest that FreeBSD's: #define DMA_HIGH_LIMIT 0x3c000000 may be a little too large (< or <=3D 0x3b400000 ?). FreeBSD u-boot reports just: /memreserve/ 0x0 0x1000; . . . memory@0 { =20 device_type =3D "memory"; reg =3D <0x0 0x0 0x0>; }; And so does not indicate anything special for either of (showing begin/end points): 0x3b400000..0x3FFFFFFF (in use by the vc?) 0xFC000000..0xFFFFFFFF (I/O peripheral area and such?) The context is an 8 GiByte RPi4 in both examples. Various details would vary on 1 GiByte and 2 GiByte RPi4Bs and some in memory@0 on the 4 GiBYTe RPi4B. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-arm@freebsd.org Sat Oct 3 23:10:52 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0117D3F2D46 for ; Sat, 3 Oct 2020 23:10:52 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-20.consmr.mail.gq1.yahoo.com (sonic314-20.consmr.mail.gq1.yahoo.com [98.137.69.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C3jJp5SX2z3gCH for ; Sat, 3 Oct 2020 23:10:50 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: zcPTcywVM1mZddaGTk95H7mZ_EDcYzb4GVlrvH0TpWViQsAd2IKp6Dn7ViKYFBG SUzt7X8ejA1p3r4vQP0uNl2zwIgZLfVHKqBUDmpbkFDSGO30EJ2TPjRqzctArv5ej0Q5GI63oJAq 4JQiKwuJe06.6n4bOnLbABPFYwcUYUkVFmpH1Gj._bXI6SgIt_4HJN_OCrrl7b4OeWzBtlg357pE Gf9eFIW8Ea.7hEIV8MAbK5SPL96DNHP1g5AuViXBrP0NazWPaZo.EON2fiF9PanarmO39O.yKOEO h6ULXNPib_I_nIEe76AIQpXNAHOZy06WPJ.dG6gtiU5mMuGQ04iojx0lqFuL5tlPqAMAM1JKLiu4 UBOAwt6OMa08c93DvwbJW1x685H4pZcxcShXBN5cHwuErGlBtCeWFoAkrmn8nFl49KrXqwq1Yg4k x8Mv8hceErIr.t_hOXEJNjAjwpCfTey4ChfAHKbo_j.LKrFlggztnKWcvItR.jGQHqUGJ79fICPX wTwjVK7zlyyy_7dwrDGT3kgwdw2DrQmGK5viYWJTKOXMZRLZ.RlPfPx9JGvw5_uzyw_b.Lro1GOS NVLORF7KSSaQPZf5U.Dhw8hIwiOt6zwJs.t1iv7BMeJssS.rzSIMsiI_n.pzPI.RXpRncz5ox57F ol9OKJokwi7znWAu1BUVXsvf9won8f82CTdqUu9kHCdGzCF1UZwiWB_a3oetOQNQ2ImreaOaVw70 y_gUnNguBvuICodggt3GxPNe.gXKOvFlSKFy6V_evGCbSLIn.1EQQeQUtWsW1_yGJX8ylp9sztOv srmNHROe0SNeRfh9ITeHwd6Nbvu4HdNZbkGia7sjJIdHt0QoB.p6gxUPimdtcTlDtLzJhKkDLAJA m87x3F32dxHU4oM0JRNo0EoWjUCemv9xGEJKJVNNBeMMZ5h1760HUH.csDAKHwtYvAsDjpivhuaB Fr.RWFQhVsh4Ptf9yX.kTg5VmZUUs7aJrBd0mzNIg9hgqlA4dGH4Q0llDRmHjUmlSxHguN09Y7Sl qQd7NB.VrK.mL6F5uob5NQ3LW8JqqTtrWWBKD4dhfIzU0S84C7aYG0xZru39DepV82v4FRRH4hpI EcjMBb7RZUFF6rRtTBMHkYioLRfApniWt9fcCSXo2RkNbKfCDrECl59Mv8nsmbQABgdcgCjMIml9 4AMHblSAwQVDw9.lzBzybdz.M6h2o9guRwIFwFXGXAVv5Cl.039nWZbdmM0GQgNClnVItPYl2KCR 3_KqzflqTqLsHOyKT1E9twPBGwJ8rm7q7enMVAcddEgo.IzrUdCfnP034f7I8.PhakSQsbFdYVjY piPkMNHPoBko7yHLx9kAjj4RgUQc4_kE2a.lBZsV5SSu9KEXLledh9mHGFg4ji5jwidIacIgZ0PD C31xV.jbTvqyb7xc7eWRFVn_C2d6bKh2T71zpQwIJMNbQN95K4reTYiNESB2ye9rerS4to5ZPoh6 V.KN90TpnnHYBZheyF3AawMcrY4eywxPSlxQsAtDjDm.Uwc26SWeLWSYCuPhXsPT0ajn1XE7gtjE - Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Sat, 3 Oct 2020 23:10:47 +0000 Received: by smtp416.mail.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID ce1b65ba19dc58b0766856db6d841894; Sat, 03 Oct 2020 23:10:41 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Re: rpi4 FreeBSD vs. ubuntu u-boot fdt print / memereserve difference (lack of reserve in FreeBSD context), 0x3b400000 vs. DMA_HIGH_LIMIT being 0x3c000000 Date: Sat, 3 Oct 2020 16:10:40 -0700 References: <1A13F7B5-F8C3-4022-939C-2992E53D1DF1@yahoo.com> To: Robert Crowston , freebsd-arm In-Reply-To: <1A13F7B5-F8C3-4022-939C-2992E53D1DF1@yahoo.com> Message-Id: X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Rspamd-Queue-Id: 4C3jJp5SX2z3gCH X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.06 / 15.00]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.48)[-0.480]; FREEMAIL_TO(0.00)[protonmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.04)[-1.044]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; SH_EMAIL_DBL_DONT_QUERY_IPS(0.00)[0.0.0.0:email]; NEURAL_HAM_LONG(-1.04)[-1.039]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DBL_PROHIBIT(0.00)[0.0.0.0:email]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.83:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.83:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Oct 2020 23:10:52 -0000 On 2020-Oct-3, at 15:20, Mark Millard wrote: > Another FreeBSD vs. ubuntu context difference, this time in the > fdt print / output . . . >=20 > The ubuntu u-boot has (fdt print / output): >=20 > memreserve =3D <0x3b400000 0x04c00000>; > . . . (Note: 0x3b400000+0x04c00000 =3D=3D 0x40000000) . . . >=20 > #address-cells =3D <0x00000002>; > #size-cells =3D <0x00000001>; > . . . > axi { > vc_mem { > reg =3D <0x3ec00000 0x40000000 0xc0000000>; > }; > Note: "vc_mem is solely used as a mechanism for passing a couple > of parameters through from the firmware to vcdbg" > End note > }; >=20 > . . . ( boot args has: vc_mem.mem_base=3D0x3ec00000 = vc_mem.mem_size=3D0x40000000 ) . . . >=20 > reserved-memory { > #address-cells =3D <0x00000002>; > #size-cells =3D <0x00000001>; > ranges; > phandle =3D <0x0000003d>; > linux,cma { > compatible =3D "shared-dma-pool"; > size =3D <0x04000000>; > reusable; > linux,cma-default; > alloc-ranges =3D <0x00000000 0x00000000 = 0x30000000>; > phandle =3D <0x0000003e>; > }; > }; > . . . (I split the reg into lines below) . . . > memory@0 { > device_type =3D "memory"; > reg =3D <0x00000000 0x00000000 0x3b400000 > 0x00000000 0x40000000 0xbc000000 > 0x00000001 0x00000000 0x80000000 > 0x00000001 0x80000000 0x80000000>; > }; > . . . (Note: 0x40000000+0xbc000000 =3D=3D 0xFC000000) . . . >=20 > (I've ignored gpiomem above and below.) >=20 > It appears to be that the memreserve may be important > to have. The above may also suggest that FreeBSD's: >=20 > #define DMA_HIGH_LIMIT 0x3c000000 >=20 > may be a little too large (< or <=3D 0x3b400000 ?). >=20 > FreeBSD u-boot reports just: >=20 > /memreserve/ 0x0 0x1000; > . . . > memory@0 { >=20 > device_type =3D "memory"; > reg =3D <0x0 0x0 0x0>; > }; >=20 > And so does not indicate anything special for either of > (showing begin/end points): >=20 > 0x3b400000..0x3FFFFFFF (in use by the vc?) > 0xFC000000..0xFFFFFFFF (I/O peripheral area and such?) >=20 > The context is an 8 GiByte RPi4 in both examples. Various > details would vary on 1 GiByte and 2 GiByte RPi4Bs and > some in memory@0 on the 4 GiBYTe RPi4B. Turns out that rpi_DATA_2711_1p0.pdf 's "Figure 1. BCM2711 Address Maps" shows this "SDRAM (for the VC)" area in the two 35-bit Address Maps, showing 0x0_4000_0000 as the next address after the area in both diagrams --and notes for area for each diagram: QUOTE Size of VC SDRAM determined by config.txt END QUOTE But ubuntu uses includes and such so overall there are the following .txt files ( cmdline.txt being for a different purpose ): # ls -ld /boot/firmware/*.txt -rwxr-xr-x 1 root root 141 Jul 31 16:48 /boot/firmware/cmdline.txt -rwxr-xr-x 1 root root 1117 Jul 31 16:48 /boot/firmware/config.txt -rwxr-xr-x 1 root root 327 Jul 31 16:48 /boot/firmware/syscfg.txt -rwxr-xr-x 1 root root 251 Sep 26 22:14 /boot/firmware/usercfg.txt config.txt includes syscfg.txt and usercfg.txt . config.txt uses the [pi4], [pi2], [pi3], and [all] section notation as well. Looks like one of u-boot's jobs is to figure out the figures that go in (base then size): memreserve =3D <0x???????? 0x????????>; (such that the total is 0x40000000 for the RPi4B) in order to protect the VC SDRAM area. The diagrams also indicate that 0xFC000000..0xFFFFFFFF is for when "low Perioheral" mode is in use, listing ("up to H" not including H): ARM Local peripherals from 0x0_FF80_0000 up to 0x1_0000_0000 and: Main peripherals from 0x0_FC00_0000 up to 0x0_FF80_0000 Otherwise they are listed at: ARM Local peripherals from 0x4_C000_0000 up to 0x5_0000_0000 and: Main peripherals from 0x4_7C00_0000 up to 0x4_8000_0000 I gather the ubuntu "fdt print /" result implies that ubuntu is using "Low Peripheral" mode (or at least allowing for it). There are some other "Reserved" areas and the special areas that are "two L2 cache aliases (one allocating, one non-allocating) which cache (only) the first 1GB of SDRAM". if these are all counted then 0x4_0000_0000 up to 0x6_0000_0000 is all some form of special-use area in both diagrams and 0x6_0000_0000 through 0x7_FFFF_FFFF is the PCIe area in both diagams. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)