From nobody Sun May 30 17:59:52 2021
X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 132D6DF9B3D
	for <freebsd-arm@mlmmj.nyi.freebsd.org>; Sun, 30 May 2021 17:59:57 +0000 (UTC)
	(envelope-from freebsd@dsllsn.net)
Received: from mail.disillusion.net (mail.disillusion.net [96.126.127.161])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512
	 client-signature RSA-PSS (4096 bits) client-digest SHA256)
	(Client CN "mail.disillusion.net", Issuer "R3" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4FtR5m6Ykbz3MXG
	for <freebsd-arm@freebsd.org>; Sun, 30 May 2021 17:59:56 +0000 (UTC)
	(envelope-from freebsd@dsllsn.net)
Received: from roast.disillusion.net (localhost [127.0.0.1])
	by roast.disillusion.net (OpenSMTPD) with ESMTP id 2a7421e1;
	Sun, 30 May 2021 12:59:53 -0500 (CDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=dsllsn.net; h=
	content-type:mime-version:subject:from:in-reply-to:date:cc
	:content-transfer-encoding:message-id:references:to; s=drip; bh=
	egY5K+Q9VPh5mIVy4g1VuBJvzzQg5Vk5rgmshGvK7Qw=; b=Cf2JD+PaET/tt9Ks
	WRGMcIdd5lbiJ2GUxxs/s22zxA1IVA+ZzrDYOnJf6W4Ef7X4h31ic6Y6wFqiOSKz
	fopbU5HuUYGC/vT7IgZmhoC3iUJcCk0HQR9is6CzPQ+91U7UbMJyM4FeGJzfSNC+
	FLx9lQnFZ0PePdYz+V0D8cyzRso=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=dsllsn.net; h=content-type
	:mime-version:subject:from:in-reply-to:date:cc
	:content-transfer-encoding:message-id:references:to; q=dns; s=
	drip; b=SGXufvH5OKsmlHSiDm0dFsMjodlY1Xk0YeBiouaF870dXZMvtTZgIwar
	dm4SsY1y3qEXHjzWay0VYVgAOrZqD7IXNowLiHj6XLPEbzyINRJ0snA+Mo0xV9m3
	LdanYytsUCgzJUx5AHIBI+ZQGAjOCPaAYWte0GUUzm49mswNiI8=
Received: 
	by mail.disillusion.net (OpenSMTPD) with ESMTPSA id 9e0f2f18 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO);
	Sun, 30 May 2021 12:59:53 -0500 (CDT)
Content-Type: text/plain;
	charset=us-ascii
List-Id: Porting FreeBSD to ARM processors <freebsd-arm.freebsd.org>
List-Archive: https://lists.freebsd.org/archives/freebsd-arm
List-Help: <mailto:freebsd-arm+help@freebsd.org>
List-Post: <mailto:freebsd-arm@freebsd.org>
List-Subscribe: <mailto:freebsd-arm+subscribe@freebsd.org>
List-Unsubscribe: <mailto:freebsd-arm+unsubscribe@freebsd.org>
Sender: owner-freebsd-arm@freebsd.org
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.6\))
Subject: Re: Boot from USB on RPi4 8GB?
In-Reply-To: <43FAEEAC-EE36-4810-88AA-FF82AFBCC128@yahoo.com>
Date: Sun, 30 May 2021 12:59:52 -0500
Cc: freebsd-arm@freebsd.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <2F58272B-BD9C-464B-9A98-BF638971BA86@dsllsn.net>
References: <4F3EE8D2-649B-4522-AD5A-7C308291802F@dsllsn.net>
 <43FAEEAC-EE36-4810-88AA-FF82AFBCC128@yahoo.com>
To: marklmi@yahoo.com
X-Mailer: Apple Mail (2.3608.120.23.2.6)
X-Rspamd-Queue-Id: 4FtR5m6Ykbz3MXG
X-Spamd-Bar: ----
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-4.00 / 15.00];
	 REPLY(-4.00)[]
Reply-To: freebsd@dsllsn.net
From: William Carson via freebsd-arm <freebsd-arm@freebsd.org>
X-Original-From: William Carson <freebsd@dsllsn.net>
X-ThisMailContainsUnwantedMimeParts: N

Thank you for the quick and thorough reply!

> On May 30, 2021, at 3:55 AM, Mark Millard via freebsd-arm =
<freebsd-arm@freebsd.org> wrote:
>=20
> On 2021-May-29, at 22:17, William Carson <freebsd@dsllsn.net> wrote:
>=20
>> Hello Mark, sorry to unicast you but I keep getting rejected from =
freebsd-arm@. I was hoping you could give me some guidance, as it seems =
you've made quite a bit of progress in this area.
>>=20
>> Is there any documentation or a howto in order to get FreeBSD to boot =
from a USB3 disk on the RPi4 8GB? I dd'd the latest 13.0-RELEASE image =
(FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img) to the USB3 SSD, but it =
does not boot. (The same image works just fine when written to the =
sdcard.) It looks like it's unable to locate the USB disk and then gets =
stuck in a loop trying for network boot. When I get to U-Boot prompt, it =
says there are no USB storage devices. If I issue "usb start", there are =
still no devices.
>=20
> You report:
>=20
>> If I try "usb reset", it seems to find it, but then it displays an =
error ("scanning bus xhci_pci for devices... Setup ERROR address device =
command for slot 1. USB device not accepting new address =
(error=3D80000000)" and then locks up. I checked that I have the latest =
bootloader:
>=20
> (I assume no microsd card was in the microsd card slot.)

Correct.

> I've not seen or heard of that U-Boot report before.
> But you made it to U-Boot so the RPi4B firmware
> worked for getting U-Boot from the USB3 drive.
>=20
> This suggests the U-Boot stage is having the problem.
>=20
> So I've no specific solutions, not having seen the
> kind of report before. I've just more generic notes
> and questions.
>=20
> [I'll note that I've had no problem with USB3 SSD
> booting the RPi4B's that I have access to (no
> microsd card involved).]

I'm trying to use a SAMSUNG (MZ-V7E500BW) 970 EVO SSD 500GB - M.2 NVMe, =
attached via the Geekworm X872 M.2 NVMe 2280/2260/2242/2230 SSD =
Expansion Board.

> One test is to use the microsd card that boots
> via the microsd card slot and instead put it in
> a USB3 microsd card reader, plug the reader into
> a USB3 port, and try to boot from just that.

Hmm, this worked just fine.

> If that boots, then there would seem to be some
> device incompatibility with your specific USB3
> "boot" drive. If, instead, booting via the reader
> fails, then things are rather odd. (See later
> notes about SOC vintages.)
>=20
>> # rpi-eeprom-update
>> BOOTLOADER: up to date
>> CURRENT: Thu 29 Apr 16:11:25 UTC 2021 (1619712685)
>>  LATEST: Thu 29 Apr 16:11:25 UTC 2021 (1619712685)
>> RELEASE: stable (/lib/firmware/raspberrypi/bootloader/stable)
>>          Use raspi-config to change the release.
>>=20
>> VL805_FW: Using bootloader EEPROM
>>   VL805: up to date
>> CURRENT: 000138a1
>>  LATEST: 000138a1
>=20
> Since you got to U-Boot the RPI4B's bootloader worked.
> The above is what I currently use in the RPi4B's,
> 8 GiBYte and 4 GiByte ones.
>=20
>> If I dd Raspberry Pi OS (2021-05-07-raspios-buster-armhf-lite.img) to =
the same disk, it boots up no problem. I'm thinking this is a =
U-Boot/rpi-firmware problem, but I don't really know where to begin.
>=20
> FYI: if you ever want to use both a 64-bit kernel and
> a 64-bit user space, there are BETA's of such (Debian
> Buster based):
>=20
> https://downloads.raspberrypi.org/raspios_lite_arm64/images/
> https://downloads.raspberrypi.org/raspios_arm64/images/
>=20
> (I'm not claiming any gain from such for the problem at hand.)
>=20
>> I tried building my own image using crochet,
>=20
> https://wiki.freebsd.org/arm/ reports about crochet:
>=20
> "Alternatively, images for many boards can be built by crochet =
(deprecated) or using FreeBSD's release build infrastructure."
>=20
> There were no commits at https://github.com/freebsd/crochet
> between 2019-Oct-06 and 2021-Apr-23. But there are some
> on 2021-Apr-24.

Yes, this is very disappointing, but I was able to copy the =
board/RaspberryPi3 and modify it to use the appropriate ports/firmware =
files. I used a similar process for my ROCKPRO64 and Khadas EDGE-V =
boards and they both worked. I'm happy to confine my testing to an =
official FreeBSD image if it makes things easier to troubleshoot :)

> I do my own builds based on using make commands but I do
> not use the release scripts for building. I build for a
> variety of systems.
>=20
>> but that didn't work either.
>=20
>> I'm also not sure whether I should be using sysutils/u-boot-rpi4 (I =
used this) or sysutils/u-boot-rpi-arm64?
>=20
> Either should generally work. (But see later notes
> about RPi4B SOC vintages.)
>=20
> FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img was based on
> sysutils/u-boot-rpi-arm64 . I've historically built and
> used a variant of sysutils/u-boot-rpi4 but my history
> goes back before sysutils/u-boot-rpi-arm64 existed. I
> do not do anything were the configuration variations
> involved matter so I'm not familiar with the distinctions.
>=20
> I've used both a previous 2020.10 based U-Boot and a
> more recent 2021.04 based U-Boot.
>=20
>> After installing sysutils/rpi-firmware (1.20210303.g20210303), should =
I just copy /usr/local/share/rpi-firmware/* to the boot partition and =
rename config_rpi4.txt to config.txt?
>=20
> I assume you mean the msdos file system partition when
> you reference "boot partition". The msdos file system
> needs to contain:
>=20
> A) copies of /usr/local/share/rpi-firmware/* materials,
>   using config_rpi4.txt content as the config.txt
>   content. The copy should be recursive, to pick up
>   the likes of the overlay/ subdirectory tree.
>=20
> B) a copy of an appropriate u-boot.bin  ( from
>   /usr/local/share/u-boot/u-boot-rpi-arm64/ or
>   /usr/local/share/u-boot/u-boot-rpi4/ ).
>   (See later notes as well.)
>=20
> C) A copy of /boot/loader.efi (the FreeBSD loader)
>   placed at/as efi/boot/bootaa64.efi in teh msdos
>   file system.
>=20
> I use a USB3 SSD that has small enough power requirements
> to not require a powered hub. (I also use a 5.1V 3.5A
> power supply as part of that context.) I've never tried
> spinning rust or higher powered USB3 media.

I can confirm those are the steps I followed. I'm not sure what's =
considered "high powered" but the Samsung tech specs say this particular =
model uses 5.7 W on average and 10.0 W maximum. But it does seem curious =
that the Raspberry PI OS will boot this disk without issue, so I don't =
think it's the drive. I also tried a Samsung 950 PRO using a different =
enclosure (QNINE NVME Enclosure, M.2 PCIe SSD (M Key) to USB 3.0 =
External Case), but it behaved the same.

> It is unclear what the dd command details were like for
> the transfer to the USB3 media. So I just assume that
> it was okay.
>=20
> You may want to have an empty file named timeout in
> the msdos file system. It allows for extra time.
> (I doubt it helps, since U-Boot did load and start.)

Correct - the empty timeout file did not help. However I was able to =
capture the beginning of the boot process:

"scanning bus xhci_pci for devices... Device NOT ready
    Request Sense returned 02 04 01
3 USB Device(s) found
        scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot: 0
Card did not respond to voltage select! : -110

Device 0: unknown device
ethernet@7d580000 Waiting for PHY auto negotiation to complete.."

(This was with the USB3 disk attached and no sdcard in the slot; no =
ethernet attached either.)

> What does:
>=20
> # rpi-eeprom-config=20
> [all]
> BOOT_UART=3D0
> WAKE_ON_GPIO=3D1
> POWER_OFF_ON_HALT=3D0
> DHCP_TIMEOUT=3D45000
> DHCP_REQ_TIMEOUT=3D4000
> TFTP_FILE_TIMEOUT=3D30000
> ENABLE_SELF_UPDATE=3D1
> DISABLE_HDMI=3D0
> BOOT_ORDER=3D0xf41
>=20
> report in your context? (An equivalent command is
> "vcgencmd bootloader_config". I show example output
> above.)

root@raspberrypi:~# rpi-eeprom-config
[all]
BOOT_UART=3D0
WAKE_ON_GPIO=3D1
POWER_OFF_ON_HALT=3D0

> Can you see the top of the SOC? Or is there a
> heatsink or some such on it? (There is something
> there to read if it can be seen.)
>=20
> One possibility is that you have newer hardware that
> the normal U-Boot vintages that FreeBSD has used do
> not handle.
>=20
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D255080
> is about someone that instead of having only:
>=20
> QUOTE
> Old Pi has the following identifying marks on the SOC package:
> BROADCOM
> 2711ZPKFSB06BOT
> TE1919
> 045-23 B3 W
> END QUOTE
>=20
> the person also had an example of a 2 GiByte:
>=20
> QUOTE
> New Pi has the following identifying marks on the SOC package:
> BROADCOM
> 2711ZPKFSB06C0T
> TA2105
> 054-05 B3 W
> END QUOTE
>=20
> The:
>=20
> 2711ZPKFSB06BOT
> vs.
> 2711ZPKFSB06C0T

Mine reads 2711ZPKFSB06B0T.

> is significant. If you have a 8GiByte "C0T" RPi4B it
> would be the first known example. In such a case, you
> might want to try the u-boot referenced in Comment 15 of
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D255080 .
> It got the 2 GiByte "C0T" to boot. (But that context
> could not even boot via the microsd card slot with the
> normal media content from 13.0-RELEASE .)
>=20
> I do not know if the 2021.04 U-Boot that the since
> updated port now provides would work for this context
> or not. I've no access to any "C0T" RPi4B's (or any
> Pi400's or CM4's).
>=20
> There is a category of USB device that U-Boot still does
> not support as far as I know: those where the one device
> has multiple storage LUNs (instead of just one Logical
> Unit Number identifying storage).
>=20
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D253983
> is about such a context, where we eventially figured out
> the "multiple storage LUN" issue.
>=20
> But the error messages from the U-Boot of the time was
> not what you report.
>=20
>=20
> =3D=3D=3D
> Mark Millard
> marklmi at yahoo.com
> ( dsl-only.net went
> away in early 2018-Mar)
>=20
>=20