Date: Fri, 17 Dec 2021 19:12:32 -0800 From: Mark Millard via freebsd-arm <freebsd-arm@freebsd.org> To: bob prohaska <fbsd@www.zefox.net> Cc: Free BSD <freebsd-arm@freebsd.org> Subject: Re: Saving environment variables in u-boot Message-ID: <9D416106-660F-40BB-98D2-1354B53D2FEF@yahoo.com> In-Reply-To: <5C44D0E6-2FF1-4EEB-B21A-83333D6FCF46@yahoo.com> References: <20211216180704.GA4173@www.zefox.net> <214132DD-A095-4349-BB81-B79CB8CF6B0C@yahoo.com> <20211217013613.GA4452@www.zefox.net> <E2F6D50B-694A-4108-BD84-C85BC96AD832@yahoo.com> <20211218005946.GA7670@www.zefox.net> <5C44D0E6-2FF1-4EEB-B21A-83333D6FCF46@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2021-Dec-17, at 19:02, Mark Millard via freebsd-arm = <freebsd-arm@freebsd.org> wrote: > On 2021-Dec-17, at 16:59, bob prohaska <fbsd@www.zefox.net> wrote: >=20 >> On Thu, Dec 16, 2021 at 11:00:46PM -0800, Mark Millard wrote: >>> On 2021-Dec-16, at 17:36, bob prohaska <fbsd@www.zefox.net> wrote: >>>=20 >>>>=20 >>>> Yes, the microSD contains a dd of=20 >>>> FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img >>>> The DOS partition is writeable, AFAIK. >>>=20 >>> Hmm. That means there is also a UFS root with >>> a kernel and world on the microsd card. Both >>> the microsd card and the USB drive contain >>> contain such, as well as each having an >>> /etc/fstab (and so on). >>>=20 >>> Having multiple such makes for a mess for >>> controlling which is used (and knowing which >>> is used). What is the first stage that you >>> made sure the USB drive was in use and how >>> did you make sure? >>>=20 >>=20 >> I installed a microSD containing only the FAT slice from=20 >> FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img, the FreeBSD >> slice being deleted using gpart. There is a timeout file. >=20 > Ultimately, you may need to publish a full capture of the > debug output that goes to the serial console when such > debug output has been fully enabled. This could end up > being for each of several variations tried. >=20 > Did you still have the MSDOSFS material on the USB in a > viable form as well? >=20 > https://www.raspberrypi.com/documentation/computers/config_txt.html > reports that there are boot options that can go in config.txt : >=20 > bootcode_delay > boot_delay > boot_delay_ms >=20 > Quoting . . . >=20 > QUOTE > bootcode_delay >=20 > The bootcode_delay command delays for a given number of seconds in = bootcode.bin before loading start.elf: the default value is 0. > END QUOTE >=20 > QUOTE > boot_delay >=20 > The boot_delay command instructs to wait for a given number of seconds = in start.elf before loading the kernel: the default value is 1. The = total delay in milliseconds is calculated as (1000 x boot_delay) + = boot_delay_ms. This can be useful if your SD card needs a while to get = ready before Linux is able to boot from it. >=20 > boot_delay_ms >=20 > The boot_delay_ms command means wait for a given number of = milliseconds in start.elf, together with boot_delay, before loading the = kernel. The default value is 0. > END QUOTE >=20 > (So boot_delay_ms is only for smaller scale adjustments and can > be ignored here.) >=20 > Here "loading the kernel" means what config.txt was told was > the kernel: some *u-boot*.bin for the FreeBSD context. >=20 >=20 > So one experiment is to have only bootcode.bin and a config.txt > on the microsd card's MSDOSFS, specifying a significantly > large value for bootcode_delay . The hope would be to be able > to get the start*.elf file and the rest from the USB drive. >=20 > If that does not work, then the next is to have the RPi* firmware > only on the microsd card's MSDOSFS and have config.txt also > specify a significantly large value for boot_delay . This > first variation would not have a *u-boot*.bin or the FreeBSD > loader on the microsd card's MSDOSFS but only on the USB > drive's MSDOSFS. >=20 > If that does not work, then the next variation moves just the > *u-boot*.bin file to the microsd card. >=20 > (If that fails then controlling U-Boot's delays can be involved. > For now I only list the path of leaving U-Boot's delays as they > are: one short branch of the exploration.) >=20 > If that does not work, then the next variation moves just the > FreeBSD loader file to the microsd card. For the above one I missed a word, making things unclear. So, instead: If that does not work, then the next variation ALSO moves just the FreeBSD loader file to the microsd card. ( *u-boot*.bin stays on the microsd card from the prior step. ) > With that I'll stop listing things to test, pending results on > what I've listed. >=20 >> A hands-off warm boot reports >> resetting system ...=20 >>=20 >> U-Boot 2020.10 (Apr 09 2021 - 03:55:54 +0000) >>=20 >> DRAM: 948 MiB >> RPI 3 Model B (0xa02082) >> MMC: mmc@7e300000: 0 >> Loading Environment from FAT... In: serial >> Out: vidconsole >> Err: vidconsole >> Net: No ethernet found. >> starting USB... >> Bus usb@7e980000: USB DWC2 >> scanning bus usb@7e980000 for devices... 5 USB Device(s) found >> scanning usb for storage devices... 0 Storage Device(s) found >=20 > I would need to also see the RPi* firmware's prior > debug output to be able to interpret this. I can not > tell for sure which device's U-Boot copy is operating > for the above. I'd guess that is the the micrsd card's > MSDOSFS one. >=20 >> Hit any key to stop autoboot: 0=20 >> switch to partitions #0, OK >> mmc0 is current device >> Scanning mmc 0:1... >=20 > This looks to be getting a copy of the FreeBSD loader > from the microsd card's MSDOSFS. >=20 > The FreeBSD loader will only see the same Storage Devices > that U-Boot did: it gets the information from U-Boot. >=20 >> Found EFI removable media binary efi/boot/bootaa64.efi >=20 > There was a FreeBSD loader in the microsd card's > MSDOSFS. >=20 >> libfdt fdt_check_header(): FDT_ERR_BADMAGIC >> Scanning disk mmc@7e300000.blk... >> Found 2 disks >> No EFI system partition >> BootOrder not defined >> EFI boot manager: Cannot load any image >> 1259212 bytes read in 125 ms (9.6 MiB/s) >> libfdt fdt_check_header(): FDT_ERR_BADMAGIC >> Booting /efi\boot\bootaa64.efi >=20 > That FreeBSD loader is what it is using, not one > from the USB drive. >=20 >> [whitespace deleted] >> Consoles: EFI console =20 >> Reading loader env vars from /efi/freebsd/loader.env >> Setting currdev to disk0p1: >=20 > "disk0p1" is U-Boot terminology for a drive, despite > this being the FreeBSD loader's output. The FreeBSD > loader uses the U-Boot information, not FreeBSD's > information. >=20 >> FreeBSD/arm64 EFI loader, Revision 1.1 >>=20 >> Command line arguments: loader.efi >> Image base: 0x39e03000 >> EFI version: 2.80 >> EFI Firmware: Das U-Boot (rev 8224.4096) >> Console: comconsole (0) >> Load Path: /efi\boot\bootaa64.efi >> Load Device: = /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(2)/SD(0)/HD(1,0x01,0,0x81f= ,0x18fa8) >> Trying ESP: = /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(2)/SD(0)/HD(1,0x01,0,0x81f= ,0x18fa8) >=20 > This is reporting where the FreeBSD loader was found. > It indicates that disk0p1 was the microsd card's > MSDOSFS. >=20 >> Setting currdev to disk0p1: >=20 > The microsd card's MSDOSFS again. >=20 >> ERROR: cannot open /boot/lua/loader.lua: no such file or directory. >=20 > There is no FreeBSD directory-tree there to find things > like /boot/lua/loader.lua in. >=20 > Currently we are trying to have the /boot/lua/loader.lua come > from the USB drive's UFS file system, not the microsd card. >=20 >> Type '?' for a list of commands, 'help' for more detailed help. >> OK=20 >>=20 >> AIUI, disk0p1 is the microSD, which has no /boot/lua/loader.lua >> That makes sense. If u-boot finds the USB mass storage device >> then running=20 >> run bootcmd_usb0 starts the system successfully. >>=20 >> It does appear that the FAT partition on the USB disk is involved. >=20 > I see no evidence of that above. (Below is a different context.) >=20 >> If I hide the contents of da0s1 by placing them in a subdirectory >> the boot fails, even if u-boot has found the USB disk: >>=20 >> Hit any key to stop autoboot: 0=20 >> U-Boot> usb reset >> resetting USB... >> Bus usb@7e980000: USB DWC2 >> scanning bus usb@7e980000 for devices... 6 USB Device(s) found >> scanning usb for storage devices... 0 Storage Device(s) found >>=20 >> [Note that the six devices include the disk, it can be seen in usb = tree. >> For some reason it isn't recognized as a mass storage device.] >>=20 >> U-Boot> editenv usb_pgood_delay >> edit: 2 >> U-Boot> usb reset >> resetting USB... >> Bus usb@7e980000: USB DWC2 >> scanning bus usb@7e980000 for devices... 6 USB Device(s) found >> scanning usb for storage devices... 1 Storage Device(s) found >> U-Boot> run bootcmd_usb0 >>=20 >> Device 0: Vendor: SABRENT Rev: 1214 Prod:=20 >> Type: Hard Disk >> Capacity: 953869.7 MB =3D 931.5 GB (1953525168 x 512) >> ... is now current device >> Scanning usb 0:1... >> U-Boot>=20 >> and there it stops. Looks like both FAT partitions have a role to = play. >=20 > That is not clear to me. But I'd rather be lookning into > earlier stages at this point, so that "1 Storage Device(s) found" > hopefully ends up being automatic. >=20 > If we get that to be automatic, then what is happening here > becomes the next area to investigate. But the investigation > is not far enough along to justify investigating this now. >=20 >> An earlier experiment tried booting the microSD card in a USB = adapter, >> That worked correctly with nothing in the microSD slot, so the = internal >> "boot from USB" feature might work if it could be slowed down = sufficiently.=20 >=20 > Agreed, that the first thing is to get it to start up > the disk and wait for it. There may be more at issue > after that. >=20 >>> The order of activity is: >>>=20 >>> MSDOSFS: (all on one of: microsd card vs. usb drive) >>> RPi* firmware (I do not report the file-level sequencing) >>> U-Boot >>> FreeBSD loader (which uses information from U-Boot) >>>=20 >>> UFS: (both: together on one of: microsd card vs. usb drive) >>> FreeBSD kernel >>> FreeBSD world >>>=20 >>> (I do not specify which MSDOSFS is used when >>> there are multiple viable ones in separate places. >>> Similarly for UFS.) >>>=20 >> =46rom the experiment today it seems both are required. >=20 > I do not get such an interpretation from your experiment. > Until "1 Storage Device(s) found" happens automatically > the context is not nicely comparable for the later stages. >=20 >> The first discovers the USB disk, the second uses that >> information to proceed further.=20 >>=20 >>>=20 >>> (It is technically possible to have kernel vs. >>> world in separate UFS partitions, possibly on >>> separate media. I've an example context that I >>> deal with that way to avoid a U-boot limitation >>> for the device: the kernel can find the world >>> where the U-Boot/FreeBSD-Loader can not. (The >>> FreeBSD loader depends on what USB can find: it >>> does not look elsewhere.) I only mention this >>> in case I need to reference it in the future, >>> avoiding a surprise in such a case. Otherwise >>> ignore the complication.) >>>=20 >>=20 >> Are you referring to a case where the root is on >> microSD but /usr and friends is on a USB device?=20 >=20 > Again: I suggest ignoring the details of this for > now. It would just confuse things. But if you > insist: >=20 > On the Rock64 I have a e.MMC in use instead of a > microsd card, and in the end the world is on a > USB3 SSD. But the U-Boot involved can not access > the USB3 --os neither can the FreeBSD loader. > The FreeBSD kernel is the first stage that can > access USB3. >=20 > The e.MMC has the MSDOSFS. It has an partial > copy of what would normally be some of the USB3 > drive's content. (I named the mount point > microsd_ufs before I switched to the e.MMC .) > It has only (much hidden in ". . ."s). >=20 > # find /microsd_ufs -print | sort | less > /microsd_ufs > /microsd_ufs/.snap > /microsd_ufs/COPYRIGHT > /microsd_ufs/boot > /microsd_ufs/boot/beastie.4th > /microsd_ufs/boot/boot1.efi > /microsd_ufs/boot/brand-fbsd.4th > /microsd_ufs/boot/brand.4th > /microsd_ufs/boot/check-password.4th > /microsd_ufs/boot/color.4th > /microsd_ufs/boot/copy_boot_to_microsd.sh > /microsd_ufs/boot/defaults > /microsd_ufs/boot/defaults/loader.conf > /microsd_ufs/boot/delay.4th > /microsd_ufs/boot/dtb > . . . > /microsd_ufs/boot/kernel > /microsd_ufs/boot/kernel/accf_data.ko > /microsd_ufs/boot/kernel/accf_dns.ko > . . . > /microsd_ufs/boot/kernel/kernel > . . . > /microsd_ufs/boot/loader.4th > /microsd_ufs/boot/loader.conf > /microsd_ufs/boot/loader.conf.d > /microsd_ufs/boot/loader.efi > /microsd_ufs/boot/loader.help > /microsd_ufs/boot/loader.rc > /microsd_ufs/boot/loader_4th.efi > /microsd_ufs/boot/loader_lua.efi > /microsd_ufs/boot/loader_simp.efi > /microsd_ufs/boot/logo-beastie.4th > /microsd_ufs/boot/logo-beastiebw.4th > /microsd_ufs/boot/logo-fbsdbw.4th > /microsd_ufs/boot/logo-orb.4th > /microsd_ufs/boot/logo-orbbw.4th > /microsd_ufs/boot/lua > /microsd_ufs/boot/lua/cli.lua > /microsd_ufs/boot/lua/color.lua > /microsd_ufs/boot/lua/config.lua > /microsd_ufs/boot/lua/core.lua > /microsd_ufs/boot/lua/drawer.lua > /microsd_ufs/boot/lua/gfx-beastie.lua > /microsd_ufs/boot/lua/gfx-beastiebw.lua > /microsd_ufs/boot/lua/gfx-fbsdbw.lua > /microsd_ufs/boot/lua/gfx-orb.lua > /microsd_ufs/boot/lua/gfx-orbbw.lua > /microsd_ufs/boot/lua/hook.lua > /microsd_ufs/boot/lua/loader.lua > /microsd_ufs/boot/lua/menu.lua > /microsd_ufs/boot/lua/password.lua > /microsd_ufs/boot/lua/screen.lua > /microsd_ufs/boot/menu-commands.4th > /microsd_ufs/boot/menu.4th > /microsd_ufs/boot/menu.rc > /microsd_ufs/boot/menusets.4th > /microsd_ufs/boot/modules > /microsd_ufs/boot/screen.4th > /microsd_ufs/boot/shortcuts.4th > /microsd_ufs/boot/support.4th > /microsd_ufs/boot/uboot > /microsd_ufs/boot/version.4th > /microsd_ufs/boot/zfs > /microsd_ufs/etc > /microsd_ufs/etc/hostid > /microsd_ufs/lost+found >=20 > Almost no world content. /microsd_ufs/boot/loader.conf > has (in part): >=20 > vfs.root.mountfrom=3D"ufs:/dev/gpt/Rock64root" > kern.cam.boot_delay=3D10000 > vfs.mountroot.timeout=3D10 > vfs.root_mount_always_wait=3D1 >=20 > That vfs.root.mountfrom notation redirects to the > USB3 drive for world but the /boot/loader.conf and > /boot/kernel/kernel and /etc/hostid were first > loaded from the e.MMC media, not the USB drive. >=20 > As I have it, the USB3 drive also has a copy of > the same /boot/... and such materials and load > module activity is from the USB3 copy. I have to > keep the two places tracking so nothing ends up > mismatching. >=20 >>> You might move (not copy) the MSDOSFS material >>> between the microsd card and the usb drive during >>> experiments, avoiding having duplications. There >>> is the possibility of instead renaming things so >>> files are not found on a partition, for example. >>> A similar point goes for UFS materials: avoid >>> having multiple viable-for-booting UFS file >>> systems present. >>>=20 >>> As stands, things seem too hard to track for me >>> to be of much help. Please make things obvious for >>> what was in use by making the material uniquely >>> placed (for the form that can be used). >>>=20 >> I believe the experiment above (deleting UFS on microSD,=20 >> hiding the DOS files on USB) has been an equivalent=20 >> configuration. It looks like control passes in some way=20 >> between the DOS slices, though how is unclear. >=20 > I do not agree to that specific interpretation of the > sequence you did (on the evidence currently available). > Until "1 Storage Device(s) found" is automatic this > later-stage material is too late to yet be relevant > or to have an appropriate context. >=20 > I'm staying focused on getting the "1 Storage Device(s) > found" to be automatic. Absent that you are likely stuck > with doing something similar to be Rock64 e.MMC way of > working --where /boot/loader.conf and /boot/kernel/kernel > and /etc/hostid for early activity is from a UFS > partition on the microsd card. >=20 >>> Separate issues/questions: >>>=20 >>> Do you have the file "timeout" in the MSDOSFS, in >>> addition to config.txt and the like? If not, I >>> recommend including it. >>>=20 >> Yes, it's been tried on both microSD and USB devices.=20 >> Seems like the only one that can matter is on microSD. >>=20 >>=20 >>> What other RPi* firmware controls for having >>> various deliberate RPi* firmware delays do you >>> have set up? >>>=20 >>=20 >> The Raspberry Pi config.txt description lists several delay commands >> that can be placed in config.txt. None seem related to USB discovery, >> might any come into play early enough to be useful? They're named >> bootcode_delay >> boot_delay >> boot_delay_ms >>=20 >> I tried them casually and didn't notice much change. Are they worth >> revisiting more carefully?=20 >=20 > See my earlier notes. >=20 >>> It is not so much that these would be sufficient, >>> but they do establish some context before U-Boot >>> is even active. It could be important to >>> understand that context. (Unsure at this point.) >>>=20 >>>=20 >>>>> But that was for the u-boot-rpi4 or u-boot-rpi-arm64 ports. >>>>> (They also later mentioned using "usb_pgood_delay=3D2000\0" >>>>> instead, a figure they found in a bunch of configrations.) >>>>>=20 >>>> The Pi3 in question is capable of booting from solid-state USB >>>> storage without any microSD card, but fails to detect a mechanical >>>> disk. Which is the appropriate u-boot-rpi3 port to tamper with? I=20= >>>> tried sysutils/u-boot-rpi3 as an upgrade but that simply froze.=20 >>>> The u-boot from FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img finds >>>> the USB mechanical disk, but erratically.=20 >>>=20 >>> FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img uses u-boot-rpi-arm64 >>> as I remember: it is set up for both the RPI3 and RPI4. Given that >>> it works some, that would be the U-Boot port I would experiment >>> with if I was doing the experiments. >>>=20 >>>>> So something somewhat analogous might help if you are willing >>>>> to build and use your own u-boot port variant. >>>>=20 >>>> Obviously, that's a fraught enterprise at my skill level.... >>>> I'm still somewhat hazy on the actual boot sequence when >>>> chaining from microSD to USB.=20 >>>=20 >>> Having the extra text in the U-Boot build does not really >>> depend on the chaining order question. >>>=20 >>> But, to repeat (sticking to the simpler context), >>> the order is: >>>=20 >>> MSDOSFS: (all on one of: microsd card vs. usb drive) >>> RPi* firmware (I do not report the file-level sequencing) >>> U-Boot >>> FreeBSD loader (which uses information from U-Boot) >>>=20 >>> UFS: (both: together on one of: microsd card vs. usb drive) >>> FreeBSD kernel >>> FreeBSD world >>>=20 >>>=20 >>>> Indeed, it's unclear how or if u-boot plays a role in starting=20 >>>> RasPiOS. The term u-boot isn't found on the Raspberry Pi doc=20 >>>> website, and the Pi isn't mentioned in the Denx manuals. Those >>>> discoveries surprised me. >>>=20 >>> RaspiOS and RaspiOS64 do not use U-boot. The RPi*'s can boot some >>> forms of a linux kernel directly and that is what RaspiOS and >>> RaspiOS64 are doing.=20 >>=20 >> That clears up a major misunderstanding, thank you! >>=20 >>> That is why the config.txt type of content >>> makes no mention of u-boot or of any kernel=3D assignment in RaspiOS >>> and RaspiOS64. By contrast, config.txt for FreeBSD lists kernel=3D >>> and an explicit *u-boot*.bin as the value. >>=20 >> Without thinking about it very carefully I tried to use the >> "bootcode.bin-only" method for booting an older Pi2 from a >> usb hard disk, as described in >> = https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#spec= ial-bootcode-bin-only-boot-mode >> It worked on the Pi2 but failed on the Pi3, causing an immediate = hang. >=20 > This gets back into the use of a config.txt with a > bootcode_delay assignment also being in the MSDOSFS > on the microsd card and the file timeout also being > in the MSDOSFS on the microsd card. Only if those > delays together can lead to the USB drive being > accessible will it get any farther. >=20 > I'd suggest such experiments. The vintage of bootcode.bin > matters as I understand. >=20 >> Might that avenue be worth pursuing? In hindsight, that it worked on = a >> Pi2 is quite surprising. =20 >>=20 >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9D416106-660F-40BB-98D2-1354B53D2FEF>