From nobody Sat Jan 22 16:07:22 2022 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id DD232197A3A7 for ; Sat, 22 Jan 2022 16:07:38 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-24.consmr.mail.gq1.yahoo.com (sonic312-24.consmr.mail.gq1.yahoo.com [98.137.69.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jh1Nn5Ktjz4lx8 for ; Sat, 22 Jan 2022 16:07:37 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1642867650; bh=G5TeCv0BppGXOIYfUxAuyvoLiPFxNpzqdSVrSuOnPQI=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=rWsNrEctjSy3iw/Hp0lCOsr7y29OyoikZDeI5LubTqGMiSapR1PwqQPspf9Qgo1OT4ZdZO38mfjtRcUzdSWdSp1izFSCwdyDEe5M+xXLGmPhiEeviaOuBkYyH8NLGphI9Kcl19ygU8gureI9r7FRQztw/j76dZpsBHoF2Vlii7hx68AAYqdt9CpR+gnj7eQoTd4qEsesklIoJxbXmyfTdBWS1yHqR+BS7d59SoMjqNAgNwI6VVAiLML4OtOjpY22uy7yEXYNm7fBkiAbOUeSip6RBDWOBJB1Mu9Iojn6jQ0NL28ShUQa7NK33KqLLReKQ8MDgWsMVSZhAsLwhRknIQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1642867650; bh=j+9dvd2cz4UVvsPShQFhv7X6McjnhexgjhwDZ0DnFvv=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=RwBoVrnIrnVIx5mHmwFU9qW4fgejC9zsGq8VogCoX9IeJQxoRTPNwD+gyNQPekIhW/evTMmI+eb+YpUwi1byAFpfBeRUqNzMz1hl9a/km8VoNg5iWb3jZsmtZeek8vB5vrQN0ld+PX0gCfgejJ60w3P7wYuRiYf0ApBpCxFmkQqBgit1W+9c3yMJhObEHJBrTdrudR+7CJR1oHlP4RNW03DKh34Z98RzMG+fdBd6xnxULuIvZh2neDYp4hqJvv2dqoy8UB+kmPKJCIEk0SgHfIjKmHHvCM8VRMwfPq+ajVruqqDZKjoTLwUTO1vUEWxCUQhDeam5IAZVJlWe+y/w4Q== X-YMail-OSG: Ecw6vGAVM1kWnfyy7ccSDlLjyh1Y8f5pbCZhssPdClH_MIIdCx1p0oM4sVWx53c E8fI8BaN3veDrxpEyx23qsld9IgTpz1Gn3YnW2mzkILEkAkctDMZLouLFIL_1yZzNqdOXhJMlmhr bf1Y1cnDxPhLt0szko0PMHruPTrSIj.s5MeisYppZqCpUKNSnrjo1hgGXGPvGA_p0.ICVwOiVqLT T1mHPtz.7wIpBNQeF87JFdUIcaAF3cvjEeVDbleg72o3z_qSEEplZUjP17HTOuEFl7.IxY.Y7vi4 mgMEiD6q9bIHrJ9lk3GZ2dxOWPyuxlE_b0IOSWjWHWCco_QRs_Yys.d6gilr0zeKKtdvnbYiaa7A mqIdkgkq79Ju6AHpt5jHOILygERvpFuO39i4gcVWNsOIq8eSd91cS8qyDxpoWGDvvvF6IRbKhjFT 2DwcS.Drfj91cUr7nWoUxE8jZh8UZsMEnIIWJF7wIOrmRZr0nO_u2FvtpH8Xy8o9nOEBCJJH4NmL qBsTiBdUq9ExY_xtnm96IeESZOsR19Q9.kcNzx6aOpZtawPgZy.S.Ge.6BWdGyr6luzuCsQAzkvV G7_9VQn2HRm4G._KNHmp4laiw5MNcWN9Xnx52myg2aR88j5zAVJTm5IIliLbKaZbOK_euGb8da9s 2plYNM4AK15hC0QyZ086sadR4_NEZlc2bg5ARCXaCchOvMiNTTpspgKqoMUL0h9whJ9DOC74UVxs LxwACgiTyDl_swp3ziubEgipspNzAv2bUDVHtAAq98ri1RWnzy5Q31HK6GVeugd1u9QHb9XbAWj5 n4PEzrdSOIwB_0UCiytBCt3auE9qPAju6_HQEnLhdypvGVxMzka9BruGPCg3xkkrA5g6Q_j6D4X7 Xoh.vECqpXYS8onOS.cY9XOAVQE7GLIum_X4oTkJO8yoPMcQ9r4XX5vOeK.StUZrzGI3TPKy3rBb iAOdtNQijbqKU2mU9oaTeqUTs2_t0j3r25s6qN2HORSUxQvMNGLP0FIp7429RbiRdQLfRQgudVKu O.ssj9nY4RHw24C_BrzMN_p4oZUgYtKZ8DMHe3EcYHCfTipdX2pR36t.wt2mGGnzYhq6uIpzQozT 5tXHZHKsk9HkR.6fKAjgCkqp3_KOknw3sNFQkdSeMVIE94abYB7vSsXXBpU3182u1rbHx5F9mG0c mWp0XD91Ft7U49Y5XQZYbHrd2N429jOzdasgpzGAvsh4DLhspAwUhMFGD5g.9z6.EGLFs0BkoAOu qNfRRp9.Lyc6NvEiThyJheXNfwdmtEUzynF7SAgns977W.aJJtvuiHWSrFv4x57f3pjXzx9Den7H B.IPUWiImjzevrfGPKUA6WHT7GXrF3p0Yt18lVrcE3Vqdwgo_bUfmUZqSSPz0ON6n10dz009SSHG r4F5SP339IVMbhGFrPb5mp2HBZvYnoYXIBn.xT.fMJ9WUTTYCIE8wOFf3nPtfvul3BJ3f.UwWmti fgGU7pRf9FSUqUgmgPjA_0dFDnKMbleQQtKZTc4xl88P.23RR9GcVuL50aNgF94nGf9hSkl3Pv0m Qs8lPNSdu2PopJUyQWexWlHv2M0lo0m1_ew1BwaUw3QetbthP_SWEHRptKZFYUEeCPeTc62Y0QG0 iQqKYsmQ.R5EHbAFm5nw7lJnk5a_tvytFRq2RIORlg4ZMBp1P6e9jfu_0WeEj4yx_UMZl23UdIdt 1N.C8qJ0V8HuuuRT_IPP8cpVv1j0FhLB7pct_qT41vMdWwoargnrfLihmud0GLdNtsiMSe4o7tdD 5boiym6gh6OWHie_BQoJ3rnm6XSYS376i9Ybcz5kGNHjuyFhSmMWMx0_B7_sm8EPdomfew52d5d9 CTy.f5uq.jTn9S.oW.RbTuasWcD8JGil.GpbW1l0Oj3oqT9gpdmYuOFdC7xuC2ijcvdbvwn9IcDO 6sq_nol382hsodWNidlUkwgNS6JL6vypXPerdNWGtVG9VaNexIwj58Xn05ggLjsSNDkl090O8xZh CYutiZLY0KbtLTYTqJGHzX.GEgfmOH9CciBR10gUetQ8PCfFTt2k7M_J46R94MoIoUqZJN1a4y_H qkz5bCC39.sKZpdskNsunfzmVxK1dA1792rbY_bCHJGcK9aVokFsI4C0o8UmtJxDQqjl41eAESgg 0tvQiqW0z2xyml1feje1R5CFQD21t9aLleKyeTmXd5b2AY6vqOAQgtLIe_Pheqb3qZtSHke8gjPs J4n17qPLnJnqxSOu_mxvl.SsHg_iNRRvxLBjwvnCmHKxS0wlnYAhJplpIBdxE0Rm0ulCkAlIZRul AXu7Yy9mxtbvR7cEB51Z3flga.v3m6KYmjeV_5w-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Sat, 22 Jan 2022 16:07:30 +0000 Received: by kubenode521.mail-prod1.omega.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID ff5b4e3e9f2400b9d4f2378b4ac7cefd; Sat, 22 Jan 2022 16:07:24 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: Dealing with slow USB disks, was: Re: Saving environment variables in u-boot From: Mark Millard In-Reply-To: <8551EF27-BF69-44E5-BC32-FB7E1752FDD6@yahoo.com> Date: Sat, 22 Jan 2022 08:07:22 -0800 Cc: Free BSD Content-Transfer-Encoding: quoted-printable Message-Id: References: <20211218223543.GA9484@www.zefox.net> <772E3794-B762-429F-B2A5-F504EA293C59@yahoo.com> <20211219043422.GA12811@www.zefox.net> <288258B0-40B0-44EC-B449-7A8FB81575F8@yahoo.com> <20211219192854.GB14873@www.zefox.net> <28D85D32-7C9A-45FC-8965-DBE8E0DF5A5D@yahoo.com> <20211219235409.GA15576@www.zefox.net> <20211220043956.GA16208@www.zefox.net> <85C29A39-824B-4FFF-8EE4-70CECE7AD09D@yahoo.com> <20220102025818.GA42622@www.zefox.net> <073F48C5-4F96-496C-B23F-607752312072@yahoo.com> <8551EF27-BF69-44E5-BC32-FB7E1752FDD6@yahoo.com> To: bob prohaska X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Queue-Id: 4Jh1Nn5Ktjz4lx8 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=rWsNrEct; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.205 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.47 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.97)[-0.972]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.205:from]; MLMMJ_DEST(0.00)[freebsd-arm]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.205:from]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N Status: O Content-Length: 9471 Lines: 246 On 2022-Jan-4, at 01:31, Mark Millard wrote: > On 2022-Jan-4, at 01:03, Mark Millard via freebsd-arm = wrote: >=20 >> On 2022-Jan-1, at 18:58, bob prohaska wrote: >>=20 >>> Coming back to the saving of u-boot environment variables, >>> backing down to a much older set of FAT files seems to help. >>>=20 >>> The machine now has a DOS-only microSD with an old set of >>> files: >>>=20 >>> root@pelorus:/mnt # strings start.elf | grep "VC_BUILD_ID_" >>> VC_BUILD_ID_USER: dc4 >>> VC_BUILD_ID_TIME: 15:31:38 >>> VC_BUILD_ID_BRANCH: master >>> VC_BUILD_ID_TIME: Jun 7 2018 >>=20 >> I've no clue what the status is for this vintage of RPi* firmware. >> But it is not obvious that the RPi* firmware is what enabled >> saving u-boot environment variables: the U-Boot vintage may be >> what made the difference. (It would not be the FreeeBSD loader >> that makes the difference: not involved until too late to >> contribute to the issue.) >>=20 >>> With this suite of bootfiles it's possible to save a value for >>> usb_pgood_delay of 19000, which in most cases results in a hands- >>> off detection of the usb hard disk (via a powered hub): >>=20 >> I do not see anything here identifying the U-Boot vintage used. >> But the vintage may be tied to the save working. >=20 > I may have implicitly presumed that the U-Boot vintage supports > presenting a UEFI interface. I've no clue what vintage such was > first good in. >=20 > Thus, I may be implicitly indicating requirements on the U-Boot > vintage used when I suggest FreeBSD loader related material. >=20 >>> MMC: mmc@7e300000: 1 >>> Loading Environment from FAT... OK >>> In: serial >>> Out: vidconsole >>> Err: vidconsole >>> Bus usb@7e980000: scanning bus usb@7e980000 for devices... 6 USB = Device(s) found >>> scanning usb for storage devices... 1 Storage Device(s) found >>> Hit any key to stop autoboot: 0 >>>=20 >>> Intervention with run bootcmd_usb0 gives a successful boot from USB.=20= >>> If nothing is touched, the console displays:=20 >>>=20 >>> MMC Device 0 not found >>> no mmc device at slot 0 >>> switch to partitions #0, OK >>> mmc1 is current device >>> Scanning mmc 1:1... >>> Found EFI removable media binary efi/boot/bootaa64.efi >>> BootOrder not defined >>> libfdt fdt_check_header(): FDT_ERR_BADMAGIC >>=20 >> I do not see anything here identifying the FreeBSD loader.efi >> vintage used. Likely it need not be old? >>=20 >>> Consoles: EFI console >>> Reading loader env vars from /efi/freebsd/loader.env >>=20 >> It might be possible to tell the FreeBSD loader to use the >> USB drive via the content of the /efi/freebsd/loader.env >> file on the microsd card's msdos file system. Likely this >> file would need to be created. >>=20 >> For all I know setting rootdev ( or currdev ?) to the >> appropriate disk?p?: text might do such. (That notation >> presumes UFS, ZFS. ZFS uses zfs:DATASETNOTATION: I've >> never tried using /efi/freebsd/loader.env and I've never >> found documentation in the system. But there are notes >> in: >>=20 >> https://reviews.freebsd.org/rS346879 >>=20 >> Given that wording, I'm not sure just where >> /efi/freebsd/loader.env ends up being relative to in the >> msdos file system. I've not done much analysis of the >> code that is also listed. >>=20 >> There also is a loaddev that seems to be involved. >>=20 >> I expect that in some respects part of what is described >> in: >>=20 >> # man loader_simp >>=20 >> applies to the contents of /efi/freebsd/loader.env . But >> Other things may not, such as rootdev being used to >> initialize currdev . >>=20 >>> Setting currdev to disk0p1: >>> FreeBSD/arm64 EFI loader, Revision 1.1 >>>=20 >>> Command line arguments: loader.efi >>> Image base: 0x39e91000 >>> EFI version: 2.80 >>> EFI Firmware: Das U-Boot (rev 8217.4096) >>> Console: comconsole (0) >>> Load Path: /efi\boot\bootaa64.efi >>> Load Device: = /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(1)/SD(0)/HD(1,0x >>> 01,0,0x81f,0x18fa8) >>> Trying ESP: = /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(1)/SD(0)/HD(1,0x01,0 >>> ,0x81f,0x18fa8) >>> Setting currdev to disk0p1: >>=20 >> /efi/freebsd/loader.env content might be able to control the above? >>=20 >>> Trying: = /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(1)/SD(0)/HD(2,0x01,0,0x1 >>> 97c7,0x7726839) >>> Setting currdev to disk0p2: >>=20 >> /efi/freebsd/loader.env content might be able to control the above? >>=20 >>> Startup error in /boot/lua/loader.lua: >>> LUA ERROR: cannot open /boot/lua/loader.lua: no such file or = directory. >>>=20 >>> Type '?' for a list of commands, 'help' for more detailed help. >>>=20 >>> It looks like the root device is still the microSD card, >>=20 >> /efi/freebsd/loader.env content might be able to control the >> kernel load device (via indirectly or directly controlling >> the currdev value), possibly via assigning the rootdev value >> so that it will be copied to currdev ? >>=20 >>> even though >>> the USB disk has been found. Any suggestions appreciated. There's a >>> complete and recent ports tree if it's helpful, attempts to use it=20= >>> on my own have caused more trouble than they've solved, in = particular >>> that's what lead to the "saving to FAT....FAILED" messages. . =20 >=20 >=20 > I'll note that the code from https://reviews.freebsd.org/rS346879 > that added reading /efi/freebsd/loader.env if it exists was > not committed until 2019-Apr-28, well after "Jun 7 2018" (the only > date I have for your context). The FreeBSD loader in the msdos file > system would need to be from after the 2019-Apr-28 commit involved. > My guess is that a modern one would be fine, but it is just a guess. >=20 > Setting rootdev via U-Boot may be another alternative that might not > have such a vintage-tie involved. In testing if I could reproduce your system-clang++ buildworld failure, I got ahold of the old RPi3B and tested on it. (FYI: No failure.) But, to get there, I ended up facing the usb_pgood_delay issue for booting the type of USB3 NVMe now in use. Based on my ports tree having sysutils/u-boot-rpi-arm64/ that is based on 2021.07 , the following patch causes the u-boot.bin built to use 2000 for usb_pgood_delay by default: # more files/patch-include_configs_rpi.h=20 --- include/configs/rpi.h.orig 2022-01-22 06:03:55.862541000 -0800 +++ include/configs/rpi.h 2022-01-22 06:03:05.435341000 -0800 @@ -210,6 +210,7 @@ ENV_DEVICE_SETTINGS \ ENV_DFU_SETTINGS \ ENV_MEM_LAYOUT_SETTINGS \ + "usb_pgood_delay=3D2000\0" \ BOOTENV =20 =20 (Some whitespace details might not be preserved via email.) I did not manage to get having just bootcode.bin and config.txt to deal with the USB3 NVMe drive directly. (This might be because of a GPT style of partitioning on the USB3 NVMe drive.) So a full set of RPi* firmware and u-boot was required on the microsd card. Basically U-Boot was the first stage able to deal with the USB3 device (GPT partitioned) --but the FreeBSD loader needed to run on/from the USB3 NVMe drive. I also faced the "automatically find and use the UFS-root-on-USB-drive" issue overall. To make it boot as desired: A) The microsd card msdosfs does not have any of part of the path: EFI/BOOT/bootaa64.efi B) The microsd card msdosfs does have the RPi* firmware files required C) The microsd card msdosfs does have a copy of the u-boot.bin file from share/u-boot/u-boot-rpi-arm64/u-boot.bin (as updated via use of the patch) D) The USB3 NVMe drive msdosfs does have the EFI/BOOT/bootaa64.efi path and content E) The USB3 NVMe drive msdosfs does not need the RPi* firmware F) The USB3 NVMe drive msdosfs does not need the u-boot.bin G) The USB3 NVMe drive UFS has normal content H) Note: The USB3 NVMe drive is GPT partitioned but the microsd card is MBR based: # gpart show -p =3D> 63 62333889 mmcsd0 MBR (30G) 63 8129 - free - (4.0M) 8192 62325760 mmcsd0s1 fat32lba (30G) =3D> 40 1953525088 da0 GPT (932G) 40 532480 da0p1 efi (260M) 532520 2008 - free - (1.0M) 534528 7340032 da0p2 freebsd-swap (3.5G) 7874560 1048576 - free - (512M) 8923136 23068672 da0p3 freebsd-swap (11G) 31991808 2097152 - free - (1.0G) 34088960 33554432 da0p4 freebsd-swap (16G) 67643392 1740636160 da0p5 freebsd-ufs (830G) 1808279552 145245576 - free - (69G) (The odd freebsd-swap's are tied to sometimes booting various machines from the same USB3 NVMe drive and the amount of RAM varying widely.) This results in U-Boot looking in partitions with msdosfs file systems until it finds a EFI/BOOT/bootaa64.efi (if it does). By only having that on the USB3 NVMe drive, that drive is where the FreeBSD loader is run from and then it finds the FreeBSD kernel on that same drive (different partition). In turn, the kernel finds the FreeBSD world on that same drive (and the same partition as it found the FreeBSD kernel). Note: This works without involving a powered hub. The USB3 NVMe drive is bus powered, no separate power source. So far, there has been no evidence of power problems. Note: The same files/patch-include_configs_rpi.h content would work in/for sysutils/u-boot-rpi4/ and sysutils/u-boot-rpi3/ (as long as the are all based on 2021.07). =3D=3D=3D Mark Millard marklmi at yahoo.com