From nobody Sat Dec 18 00:59:46 2021 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2E40A18F44BC for ; Sat, 18 Dec 2021 00:59:49 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (www.zefox.net [50.1.20.27]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "www.zefox.com", Issuer "www.zefox.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JG6vS59NLz4j14 for ; Sat, 18 Dec 2021 00:59:48 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (localhost [127.0.0.1]) by www.zefox.net (8.16.1/8.15.2) with ESMTPS id 1BI0xlqi008840 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 17 Dec 2021 16:59:47 -0800 (PST) (envelope-from fbsd@www.zefox.net) Received: (from fbsd@localhost) by www.zefox.net (8.16.1/8.15.2/Submit) id 1BI0xl90008839; Fri, 17 Dec 2021 16:59:47 -0800 (PST) (envelope-from fbsd) Date: Fri, 17 Dec 2021 16:59:46 -0800 From: bob prohaska To: Mark Millard Cc: freebsd-arm@freebsd.org Subject: Re: Saving environment variables in u-boot Message-ID: <20211218005946.GA7670@www.zefox.net> References: <20211216180704.GA4173@www.zefox.net> <214132DD-A095-4349-BB81-B79CB8CF6B0C@yahoo.com> <20211217013613.GA4452@www.zefox.net> List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4JG6vS59NLz4j14 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N Status: O Content-Length: 9617 Lines: 273 On Thu, Dec 16, 2021 at 11:00:46PM -0800, Mark Millard wrote: > On 2021-Dec-16, at 17:36, bob prohaska wrote: > > > > > Yes, the microSD contains a dd of > > FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img > > The DOS partition is writeable, AFAIK. > > Hmm. That means there is also a UFS root with > a kernel and world on the microsd card. Both > the microsd card and the USB drive contain > contain such, as well as each having an > /etc/fstab (and so on). > > Having multiple such makes for a mess for > controlling which is used (and knowing which > is used). What is the first stage that you > made sure the USB drive was in use and how > did you make sure? > I installed a microSD containing only the FAT slice from FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img, the FreeBSD slice being deleted using gpart. There is a timeout file. A hands-off warm boot reports resetting system ... U-Boot 2020.10 (Apr 09 2021 - 03:55:54 +0000) DRAM: 948 MiB RPI 3 Model B (0xa02082) MMC: mmc@7e300000: 0 Loading Environment from FAT... In: serial Out: vidconsole Err: vidconsole Net: No ethernet found. starting USB... Bus usb@7e980000: USB DWC2 scanning bus usb@7e980000 for devices... 5 USB Device(s) found scanning usb for storage devices... 0 Storage Device(s) found Hit any key to stop autoboot: 0 switch to partitions #0, OK mmc0 is current device Scanning mmc 0:1... Found EFI removable media binary efi/boot/bootaa64.efi libfdt fdt_check_header(): FDT_ERR_BADMAGIC Scanning disk mmc@7e300000.blk... Found 2 disks No EFI system partition BootOrder not defined EFI boot manager: Cannot load any image 1259212 bytes read in 125 ms (9.6 MiB/s) libfdt fdt_check_header(): FDT_ERR_BADMAGIC Booting /efi\boot\bootaa64.efi [whitespace deleted] Consoles: EFI console Reading loader env vars from /efi/freebsd/loader.env Setting currdev to disk0p1: FreeBSD/arm64 EFI loader, Revision 1.1 Command line arguments: loader.efi Image base: 0x39e03000 EFI version: 2.80 EFI Firmware: Das U-Boot (rev 8224.4096) Console: comconsole (0) Load Path: /efi\boot\bootaa64.efi Load Device: /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(2)/SD(0)/HD(1,0x01,0,0x81f,0x18fa8) Trying ESP: /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(2)/SD(0)/HD(1,0x01,0,0x81f,0x18fa8) Setting currdev to disk0p1: ERROR: cannot open /boot/lua/loader.lua: no such file or directory. Type '?' for a list of commands, 'help' for more detailed help. OK AIUI, disk0p1 is the microSD, which has no /boot/lua/loader.lua That makes sense. If u-boot finds the USB mass storage device then running run bootcmd_usb0 starts the system successfully. It does appear that the FAT partition on the USB disk is involved. If I hide the contents of da0s1 by placing them in a subdirectory the boot fails, even if u-boot has found the USB disk: Hit any key to stop autoboot: 0 U-Boot> usb reset resetting USB... Bus usb@7e980000: USB DWC2 scanning bus usb@7e980000 for devices... 6 USB Device(s) found scanning usb for storage devices... 0 Storage Device(s) found [Note that the six devices include the disk, it can be seen in usb tree. For some reason it isn't recognized as a mass storage device.] U-Boot> editenv usb_pgood_delay edit: 2 U-Boot> usb reset resetting USB... Bus usb@7e980000: USB DWC2 scanning bus usb@7e980000 for devices... 6 USB Device(s) found scanning usb for storage devices... 1 Storage Device(s) found U-Boot> run bootcmd_usb0 Device 0: Vendor: SABRENT Rev: 1214 Prod: Type: Hard Disk Capacity: 953869.7 MB = 931.5 GB (1953525168 x 512) ... is now current device Scanning usb 0:1... U-Boot> and there it stops. Looks like both FAT partitions have a role to play. An earlier experiment tried booting the microSD card in a USB adapter, That worked correctly with nothing in the microSD slot, so the internal "boot from USB" feature might work if it could be slowed down sufficiently. > > The order of activity is: > > MSDOSFS: (all on one of: microsd card vs. usb drive) > RPi* firmware (I do not report the file-level sequencing) > U-Boot > FreeBSD loader (which uses information from U-Boot) > > UFS: (both: together on one of: microsd card vs. usb drive) > FreeBSD kernel > FreeBSD world > > (I do not specify which MSDOSFS is used when > there are multiple viable ones in separate places. > Similarly for UFS.) > >From the experiment today it seems both are required. The first discovers the USB disk, the second uses that information to proceed further. > > (It is technically possible to have kernel vs. > world in separate UFS partitions, possibly on > separate media. I've an example context that I > deal with that way to avoid a U-boot limitation > for the device: the kernel can find the world > where the U-Boot/FreeBSD-Loader can not. (The > FreeBSD loader depends on what USB can find: it > does not look elsewhere.) I only mention this > in case I need to reference it in the future, > avoiding a surprise in such a case. Otherwise > ignore the complication.) > Are you referring to a case where the root is on microSD but /usr and friends is on a USB device? > You might move (not copy) the MSDOSFS material > between the microsd card and the usb drive during > experiments, avoiding having duplications. There > is the possibility of instead renaming things so > files are not found on a partition, for example. > A similar point goes for UFS materials: avoid > having multiple viable-for-booting UFS file > systems present. > > As stands, things seem too hard to track for me > to be of much help. Please make things obvious for > what was in use by making the material uniquely > placed (for the form that can be used). > I believe the experiment above (deleting UFS on microSD, hiding the DOS files on USB) has been an equivalent configuration. It looks like control passes in some way between the DOS slices, though how is unclear. > Separate issues/questions: > > Do you have the file "timeout" in the MSDOSFS, in > addition to config.txt and the like? If not, I > recommend including it. > Yes, it's been tried on both microSD and USB devices. Seems like the only one that can matter is on microSD. > What other RPi* firmware controls for having > various deliberate RPi* firmware delays do you > have set up? > The Raspberry Pi config.txt description lists several delay commands that can be placed in config.txt. None seem related to USB discovery, might any come into play early enough to be useful? They're named bootcode_delay boot_delay boot_delay_ms I tried them casually and didn't notice much change. Are they worth revisiting more carefully? > It is not so much that these would be sufficient, > but they do establish some context before U-Boot > is even active. It could be important to > understand that context. (Unsure at this point.) > > > >> But that was for the u-boot-rpi4 or u-boot-rpi-arm64 ports. > >> (They also later mentioned using "usb_pgood_delay=2000\0" > >> instead, a figure they found in a bunch of configrations.) > >> > > The Pi3 in question is capable of booting from solid-state USB > > storage without any microSD card, but fails to detect a mechanical > > disk. Which is the appropriate u-boot-rpi3 port to tamper with? I > > tried sysutils/u-boot-rpi3 as an upgrade but that simply froze. > > The u-boot from FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img finds > > the USB mechanical disk, but erratically. > > FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img uses u-boot-rpi-arm64 > as I remember: it is set up for both the RPI3 and RPI4. Given that > it works some, that would be the U-Boot port I would experiment > with if I was doing the experiments. > > >> So something somewhat analogous might help if you are willing > >> to build and use your own u-boot port variant. > > > > Obviously, that's a fraught enterprise at my skill level.... > > I'm still somewhat hazy on the actual boot sequence when > > chaining from microSD to USB. > > Having the extra text in the U-Boot build does not really > depend on the chaining order question. > > But, to repeat (sticking to the simpler context), > the order is: > > MSDOSFS: (all on one of: microsd card vs. usb drive) > RPi* firmware (I do not report the file-level sequencing) > U-Boot > FreeBSD loader (which uses information from U-Boot) > > UFS: (both: together on one of: microsd card vs. usb drive) > FreeBSD kernel > FreeBSD world > > > > Indeed, it's unclear how or if u-boot plays a role in starting > > RasPiOS. The term u-boot isn't found on the Raspberry Pi doc > > website, and the Pi isn't mentioned in the Denx manuals. Those > > discoveries surprised me. > > RaspiOS and RaspiOS64 do not use U-boot. The RPi*'s can boot some > forms of a linux kernel directly and that is what RaspiOS and > RaspiOS64 are doing. That clears up a major misunderstanding, thank you! > That is why the config.txt type of content > makes no mention of u-boot or of any kernel= assignment in RaspiOS > and RaspiOS64. By contrast, config.txt for FreeBSD lists kernel= > and an explicit *u-boot*.bin as the value. Without thinking about it very carefully I tried to use the "bootcode.bin-only" method for booting an older Pi2 from a usb hard disk, as described in https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#special-bootcode-bin-only-boot-mode It worked on the Pi2 but failed on the Pi3, causing an immediate hang. Might that avenue be worth pursuing? In hindsight, that it worked on a Pi2 is quite surprising. With my thanks! bob prohaska