From owner-freebsd-arm@freebsd.org Wed Mar 17 00:33:52 2021 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4F3FB573E50 for ; Wed, 17 Mar 2021 00:33:52 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-25.consmr.mail.gq1.yahoo.com (sonic303-25.consmr.mail.gq1.yahoo.com [98.137.64.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F0WNv0lBqz3vxy for ; Wed, 17 Mar 2021 00:33:50 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1615941229; bh=ENkvYG2Qfw/s+HI51j4vH0vOuV7zN5baJcryMx2WJfT=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=s2Im5+hCSGOUuIAm2AixoSSH3/DqYLKun+PKMEUcoyuqz9L4A+qIyXMx9Oc1Jl42vViGr9hhi/TmMNDFRys5e7tYhkgRddgJvLoyB1xgBbym1+EOWbaIqqZxK5jz0wV1ZcfV05ZAjn0sfetGuhIE3fDE5l8Xc0k+WEjCb6vd66vthgLzOMGTRlskOdCwF2/tG3RiPpDV8pyIVJWQFqU+vRjBgfvWsU6/nWjbNgj1ndaKgvSdiNtpgZfvE4zLuKZdlbov2o02gdd7l9UG7cuZuUEbTLubwQyi6VKh0vZqyKi1Ho/Cbz7ifikGyVD4G5mmWEgvnnuc2OetvsH+MN5CUA== X-YMail-OSG: wdFZek4VM1k.amOtMQPS7OY30lDZ6VwPg1GOB.qtBanb4gMnz1vKwbleLUKXb0m bPwbwRuW7f2kGr7P7jDK6Pb2TiX.9ev.GZ5gQ0BLcuqJl1fKMQnbkXhb1NKfxLbldWN4gWul4lbw RzDXHn9yAWbFBpASs8ZuJsnciKpMpK2cu3tq8kKLhiE6qofJlm0mfu1LiKsywx3GW._BAMUWPh3i ymZTtUhizol1C_BEIF9eIpNjUcgMBMWqFYZUvaGH6ugVUjxa.XMxu3BQmr1IFKXiz.MLtW7zQcPY aowjr.XKXl52LqRuYTseDwFdv1EKM_7_FusPq.iP_yT1xLW5oKwoMozTdHsA90Jxs.boXt0.W7jo YgZl5aKTQQ0KZE23fKmd_FKuybtdlume6ef2Y4ov.EoBb_v6hrxkP4oEXJCB3LknOkpuoZHOTcT9 1oWL2QexjPRsF78njL0jMDMroK435BLc.RE0Vaz9a9JGjOI7C1uxgM_49MHW90bBywqG5vhu1poG paGik4sRv1UoeNVDZX6on394g2PAE6nwTzKV86_2UvqBWTuYg.lOAlrYo_Uw8kHWQs6h3ViLWFHC Qys3xstj1ng.SvkAduf4U0TWd5DtW8T9VwvtXhRtbh6AIjbYbihxz0G0O8CtdOQg2gdQOWzY.y2I 2jI7tcF8woJf1asrmQSdeKf5NrJYbyDZsmJ3BceHfdxS7KYlwyYT0hg_xxfG5Kg_FMIlWkftl2jC sO8YEyUAWijUlxGDFq8Tol0LQWGbMyS1cvD2vlO5_28duqYR66ikqbjqnbDmF.UCJm.KVtEqcIyD NwMpT64ySD_Z7MhsRuRsORWrMJnge_cUzdbHpa15iFIyn2p8YwrNOKsA2YOQ84Ev_FUoP4Vtqi6i 4o0Yr04uZxzj14TI1OU9pnmgElaeA1g0secdAaCCXfClaIYH_SBrIkTa3l.p66rz_a4U16DgMmJx GlZgv24W9xAY_CJ1CPoWogVNzQ5S4XlrYd.VY.DM0MpTVtevnUIkWNMpqKLsZH47LSMxfwF6YDVU 6zgIJEeaVeX8p0WeUNISyE97GE7o.UrjYw3OEXpArLPNXxBLAsefaootGPWKKGwQYSZXKb4nQgKh 6zjG9hGA5.nX1MKtyHPa74OHpsTNVNnqWWEXzswiqxvkVfnp_icbYsuzal22nJfmloSyQ2FS.RMb Nsr8HPKjFRfXj9kFpmHD3vQiuwrl3W2ZdLCl5uyM.rgeMuAjVlXeaFciqkGThMk7QizarMioEZ9t vrKUnQ1d13ZqlXJaBNau3_ozd34qmXqU8S6NK7d1blvorxP6xMpRuYebF8mZFhvp1A6MDR6u36iI 7qocsaIkKgUU8j.RXGUzxgqH0ZHBKb6ySUCYFY_BG6lteq7BYON.DKp3lvE0dbdlxbwc1k.b.GcK ZT5nEwnJbVWYdcN1CKLOeVm5WlCJNE4VZwG7QiBWExABQCSxL8DH1Zsi5JokzV4Emta_BVDbKy7O mR6A0H6voxO4LMMzW1p163ub..dPstAs6sPYHaVeoeJtPUUX7CWehaqFUhTNN8A.ki2.kjA_KGJv r6fQDUMef5QpXka_22L5E9mWCvrn9i5KU7loIFA5JPaIf1k3a4RWH14BaAcZiKVOovY41jNgfPI1 SUjI7LdM61lnWnw8VvL3yXBZKIvdtFdnSolZnTNV6Wz0yUE_YIrefsYgb7zAABCfpUfnHXPTr3P9 KKrGsvkA3MNoWzyA7qYaMnVJXfWO_3dA.jX22adlq3RptEfZR8RGVH2F0jHsKbwhFd66mZ5yFMyD S8TYgNEbsnIIIeEy0afw42mtTfnrtYl_32m1KVJQPmMBOv.RwoTUhbg0y262gfbVTrQFAu6FIfg9 NiCd0JpbgyP9EkoQDnWTNqwplO2vieNkO8wtGJuXN3Oxs26ZZEIgOuC1JJ1K_Cb2ArQNy5coLMba SUsLSN3HGs_wSgED3lqZ.Z4YHV_4CSzO4ZNGv9KAnbvGHOGI6MbQ6lxRPV129.oupvhcPsIllB6D rZxgsjVHcOtRfaIS0FfPJlOgVVhnjpfYb6u6eVS6kphXG8xRE8Ff5VQm5oXERZqzpivWeNb.aWKi s3qFNbf8vzlSAr.u8gWBtzAzXLZWUg3A0DiY4OC5p6gjayfquFR2KM43IsfiZHHvsp6ab6Reu0T1 VOnxmNGnniwtSgKQl6ULmYWkjXyEFkDuTDkBb24SE.itxv8vynNfwOzhgxW4I1bmtJigeQyNfvAl 5b7Hn1nry2VzGXpxz88PtBi9OARuhkcYrnfPOTmluSbrte89.roZRISVYFtmt5oR9qcRAS_hX7WA vw7_sW5KHvid7_FaTt7UZWBuEW_TD1_9tY8QNMfdGUqOaXHMVyfiN3Eb0ypFi0wfYzZni99oJ9VJ 0eS5HtQFW6KKd.nxGK5O9.YESzWJGg6ldNbiKc2KWR8Clfp_L4NX3VkGeB8SFebfOP51RvEco0My Wh0ZD00cpjJaqV2rECR0Nh_CG5VGKLFunjgmD8f6IQjxG3Tzbc3mDk9V9GG.0YVoRyC43siEOmTK rekD75pSsdFUoFYbL_P5gjDrqL_GnsyfEscKFCCvJQnTjDVKYc4xcXLerCUBqPa2B1McYYKK7dEZ oTJFP0xYl1h1ru9A1 X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Wed, 17 Mar 2021 00:33:49 +0000 Received: by smtp406.mail.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 25120c8d3935c49438d44b5523952abf; Wed, 17 Mar 2021 00:33:47 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: rpi4b main-n245392-8423f5d4c12 won't boot due to microsd timeout [FIXED] From: Mark Millard In-Reply-To: <548DA04C-8CF0-4898-9C9A-423B8A400AE5@yahoo.com> Date: Tue, 16 Mar 2021 17:33:45 -0700 Cc: freebsd-arm@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <1E71C78B-98B2-47CE-B3B6-F5503170DA54@yahoo.com> References: <79EB88DA-0144-4A12-B716-3CF5011F16C4@yahoo.com> <548DA04C-8CF0-4898-9C9A-423B8A400AE5@yahoo.com> To: tech-lists X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4F0WNv0lBqz3vxy X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.86 / 15.00]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.36)[-0.361]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.64.206:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.64.206:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.206:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.206:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Porting FreeBSD to ARM processors List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2021 00:33:52 -0000 On 2021-Mar-16, at 15:22, Mark Millard wrote: > On 2021-Mar-16, at 09:44, tech-lists wrote: >=20 >> On Mon, Mar 15, 2021 at 06:50:02PM -0700, Mark Millard wrote: >>=20 >>> So there would seem to be no urgent aspect of >>> existing RPi[34] u-boot ports vs. Klaus K.'s >>> build(s) to lead Klaus to put up reviews on >>> Phabricator for updates to: >>>=20 >>> sysutils/u-boot-rpi-arm64 >>> sysutils/u-boot-rpi3 >>> sysutils/u-boot-rpi4 >>=20 >> I don't know what (version) those ports make, as I've never = tested/used them. My way of working (directly updating firmware files) = is a hangover from the time when there was little support for the = rpi4/8GB and I've just never changed my method. Maybe that needs = changing. >=20 > Where those ports are/were used in official FreeBSD > builds . . . >=20 > sysutils/u-boot-rpi-arm64 (from latest) is what is used > in building the official, weekly snapshots' >=20 > FreeBSD-14.0-CURRENT-arm64-aarch64-RPI-*.img.xz >=20 > images. Some version from quarterly is used > in building the official: >=20 > FreeBSD-13.0-*-arm64-aarch64-RPI.img.xz >=20 > images. >=20 > FreeBSD-13.0-RC3-arm64-aarch64-RPI.img.xz this week > should finally use a sysutils/u-boot-rpi-arm64 version > from quarterly that no longer has the RPi* firmware > based USB problem(s). >=20 > = FreeBSD-14.0-CURRENT-arm64-aarch64-RPI-20210311-15565e0a217-257277.img.xz >=20 > from last week was the first to use a version of > sysutils/u-boot-rpi-arm64 from latest that no > longer had the RPI*firmware USB problem(s) but > it had FreeBSD kernel USB problems reported by > its debug kernel. >=20 > [I'm ignoring u-boot's not handling having multiple > storage LUNs in a USB device here.] >=20 > Back when there were separate snapshots for > RPi4 and RPi3 for 64-bit, sysutils/u-boot-rpi4 > and sysutils/u-boot-rpi3 were used. (The detailed > u-boot options for at least sysutils/u-boot-rpi4 > are somewhat different than for > sysutils/u-boot-rpi-arm64 .) >=20 > Two of the ports are intended to be RPi4 vs. > RPi3 (and RPi2 v1.2) specific. In an RPi4 > context, materials from sysutils/u-boot-rpi3 > should not be used. >=20 > What versions are built . . . >=20 > As far as I know all the u-boot ports are currently > picking up the default 2020.10 assignment that > is in sysutils/u-boot-master plus whatever local > patches that they may have. sysutils/u-boot-master > specifies other details for specific ports as well. >=20 >=20 >>>> [1] how would I "test" the installed[2] u-boot.bin to make sure it = was >>>> working correctly? >>>=20 >>> I was thinking of whatever criteria you used when you >>> wrote: >>>=20 >>> QUOTE >>> . . . >>>> = https://sourceforge.net/projects/fbsd-rpi4-u-boot2021-04-klaus/files/u-boo= t.bin/download >>>=20 >>> Pleased to report this new u-boot works perfectly! thank you >>> END QUOTE >>=20 >> what I was looking for when I asked the question was whether you = might >> know a method of testing it programmatically. I mean, I'm not a = programmer, so I don't know where to look. I don't even know what the >> three firmware files do, apart from leveraging the booting process.=20= >=20 > u-boot is used in booting but (for the most part?) > does not survive the OS finishing its boot: basically > nothing to test for u-boot after booting successfully. >=20 > General testing is just if you are having boot problems. > The mess is if the answer is yes and then the problem > needs to be tracked down to contributions from some > combination of firmware, u-boot, loader.efi (bootaa64.efi), > FreeBSD kernel, or world materials (in part depending > on how far things got for the configuration). >=20 >> By "works perfectly", what I mean is it seems to do everything asked = of it. >> The stable/13 machine, as well as self-hosting, runs poudriere for = about 500 >> packages, runs mail clients, is a web server, uses usb3+zfs for >> additional (2Tb) storage, boots off sdcard, is clocked at 2GHz. >=20 > "Boots off sdcard" and possibly "is clocked at 2GHz" are > the parts of that tied to u-boot. (u-boot could force > the initial conditions for what the later stages see for > cpu clock rates and such.) >=20 > "Booting off USB without use of a microsd card" would > be another form of test tied to u-boot. >=20 >> I don't know what else to test apart from transferring files from = sdmmc to >> usb3 storage and back and checking for crc errors, I mention because = there was an issue with large file transfers a while ago. >=20 > The large file transfer problems that I know of were > for RPi4's with > 3 GiByte RAM configured and the > fix was to the FreeBSD kernel. (uefi/ACPI booting is > still limited to <=3D 3 GiByte for reliable operation: > the kernel fix was only to code not involved for ACPI > handling.) >=20 >>> I had not quizzed you about what it takes to have the >>> status "works perfectly". >>=20 >> Hopefully, the above answers that. If not, please suggest some tests = to >> run? >=20 > I think your activity was fine for the purposes > involved. >=20 > You might want to recheck some official snapshots > and 13.0 releases as they are released, hopefully > not having to make substitutions copied from > elsewhere else once this week's materials are > available (including the debug FreeBSD kernel). >=20 >>> (I would expect that "works perfectly" has to involve how the = combination of u-boot and RPi* firmware operate together in making the >>> judgment.) >>=20 >> All I can say for certain is that this combination >> wget >> = https://sourceforge.net/projects/fbsd-rpi4-u-boot2021-04-klaus/files/u-boo= t.bin/download >> -O u-boot.bin >> fetch >> = https://github.com/raspberrypi/firmware/raw/0591568b29a724de406aa737fc8e13= f68c423f3f/boot/start4.elf >> fetch >> = https://github.com/raspberrypi/firmware/raw/0591568b29a724de406aa737fc8e13= f68c423f3f/boot/fixup4.dat >=20 > Hopefully none of that copying will be needed > to get materials once this week's 13.0-RC3 and > main snapshots are available and materials from > them are put to use (msdos file system materials). >=20 > Getting such materials from the official builds > helps check that the official builds are good. > Getting them from elsewhere does not directly > do so. >=20 >> "work perfectly" (but see [3]), with both a stable/13 (13-n244890) = GENERIC rpi4 and current/14 (main-n245454) rpi4 with a GENERIC-NODEBUG = kernel. >>=20 >>> [2] has both a different u-boot and a different >>> group of RPi* firmware files. >>>=20 >>> There would seem to be no urgent aspect of the >>> existing sysutils/rpi-firmware port vs. the >>> Mar 10 2021 RPI* materials to lead anyone to >>> put up a review on Phabricator for updating: >>=20 >> I'm not sure either way because I don't know if or how any of the = three >> files work when it comes to initialising usb, or sdmmc which is where = I >> started. >=20 > Both the RPi* firmware and u-boot could potentially > mess things up. As things are, absent specific > evidence, the RPi* firmware is more likely to be > what broke things if something is discovered to > be broken. >=20 > The 3 different u-boots are alternatives for each > other, although two are intended to be RPi4 vs. > RPi3(/RPi2 V1.2) specific. >=20 > For helping check modern official-build materials, > sysutils/u-boot-rpi-arm64 is the one to check. >=20 >> Additionally, main-n245454 (generic, so debug kernel) unmodified will = boot if there's nothing usb attached. If my usb3 disk is plugged in = after it boots, the pi will panic. If I reboot replacing just the u-boot = with Klaus's u-boot, I get the same result.=20 >=20 > Until this week's main snapshot build and 13.0-RC3 > build, problems with USB are expected and are from > one or both of 2 places: >=20 > A) Older RPi* firmware that is known-broken >=20 > B) FreeBSD debug kernel panics about inappropriate > sleeping being allowed for memory allocations. > (Recently added validation code reporting > pre-existing problems as I understand.) > NOTE: (B) was not limited to arm or aarch64 > platforms but (A) is so limited. >=20 > Both should be fixed in this week's build materials. >=20 > u-boot was not involved in causing either of those > problems. (The only known u-boot issue for USB seems > to be that it does not support USB devices that have > multiple storage LUNs in the device.) >=20 >> If I replace all 3 files with the latest versions as described in the >> URLs, (again generic kernel so with debug on main/14), it will still = panic when usb is plugged in. Presumably these "latest" files are ahead = of those made by ports. >=20 > It only takes the debug kernel to be involved to have > the panics about inappropriate sleeping being allowed. > This problem was not limited to RPi4 or aarch64 or > even to arm. Even with good RPi* firmware and a > good u-boot, it was a problem made obvious by the > debug FreeBSD kernel. >=20 >> So, the things that are fixed for me are sdmmc initialisation = (presumably >> u-boot fixed this) on stable/13 >=20 > As I remember you initially reported a 2020.07 or > some such older u-boot before you updated it. So > not necessarily surprising that an update helped. >=20 >> and usb initialisation (which making a >> generic kernel on main/14 fixes). >=20 > For debug FreeBSD kernel, it takes both a modern > enough kernel build vintage and an appropriate > RPi* firmware vintage before USB booting and > the like will work well. Neither is sufficient > by itself, both are necessary. >=20 > In other words: There has been more than one > USB-support problem present recently for RPi4's, > problems being from separate pieces of software. > This makes it messier to track the issues. >=20 >> [3] only thing tested on current/14 is booting and buildworld >=20 > Sounds good. Just an FYI: For an RPi* using the likes of: # strings /boot/efi/u-boot.bin | grep 'U-Boot 2' U-Boot 2020.10 (Dec 13 2020 - 08:04:27 +0000) to see an indication of the u-boot version and time frame is sort of like using: # strings /boot/efi/start4.elf | grep VC_BUILD_ID_ VC_BUILD_ID_USER: dom VC_BUILD_ID_TIME: 12:10:40 VC_BUILD_ID_VARIANT: start VC_BUILD_ID_TIME: Feb 25 2021 VC_BUILD_ID_BRANCH: bcm2711_2 VC_BUILD_ID_HOSTNAME: buildbot VC_BUILD_ID_PLATFORM: raspberrypi_linux VC_BUILD_ID_VERSION: 564e5f9b852b23a330b1764bcf0b2d022a20afd0 (clean) to do so for RPi* firmware *.elf file(s). A somewhat analogous sequence for the FreeBSD kernel is to use something like: # strings /boot/kernel/kernel | grep 'FreeBSD 1[0-9]\.[0-9]' WARNING: Device "%s" is Giant locked and may be deleted before FreeBSD = 14.0. @(#)FreeBSD 14.0-CURRENT mm-src-n245445-def0058cc690 GENERIC-NODBG FreeBSD 14.0-CURRENT mm-src-n245445-def0058cc690 GENERIC-NODBG (But it is not as useful by itself for FreeBSD with source file patching maintained in the local git repository.) The kernel of interest might not be the one that is running. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)