Date: Fri, 27 Jul 2018 12:02:36 +0200 From: "O. Hartmann" <ohartmann@walstatt.org> To: Toomas Soome <tsoome@me.com> Cc: "O. Hartmann" <ohartmann@walstatt.org>, "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>, freebsd-current <freebsd-current@freebsd.org>, Allan Jude <allanjude@freebsd.org> Subject: Re: [UEFI] Boot issues on some UEFI implementations Message-ID: <20180727120232.270e1d9f@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: <6C5D21D2-59C6-42DB-AC75-79D98BA5E62B@me.com> References: <1E6058D2-5804-480B-B6AF-66AA02CDD7AD@me.com> <201807251430.w6PEUWPn041286@pdx.rh.CN85.dnsmgr.net> <20180726155821.6f9906e9@freyja.zeit4.iv.bundesimmobilien.de> <7FA45CAF-6869-4DF6-AA93-5F96F83EF958@me.com> <20180727074558.75b2d730@freyja.zeit4.iv.bundesimmobilien.de> <6C5D21D2-59C6-42DB-AC75-79D98BA5E62B@me.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 27 Jul 2018 08:54:48 +0300
Toomas Soome <tsoome@me.com> wrote:
> > On 27 Jul 2018, at 08:46, O. Hartmann <ohartmann@walstatt.org> wrote:
> >=20
> > On Thu, 26 Jul 2018 19:23:43 +0300
> > Toomas Soome <tsoome@me.com> wrote:
> >=20
> > (reply inline/at the end)
> >=20
> > =20
> >>> On 26 Jul 2018, at 16:58, O. Hartmann <ohartmann@walstatt.org> wrote:
> >>>=20
> >>> On Wed, 25 Jul 2018 07:30:32 -0700 (PDT)
> >>> "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net
> >>> <mailto:freebsd-rwg@pdx.rh.CN85.dnsmgr.net>> wrote: =20
> >>>>>> On 25 Jul 2018, at 12:10, O. Hartmann <o.hartmann@walstatt.org> wr=
ote:
> >>>>>>=20
> >>>>>> On Wed, 25 Jul 2018 11:46:07 +0300
> >>>>>> Toomas Soome <tsoome@me.com> wrote:
> >>>>>> =20
> >>>>>>>> On 25 Jul 2018, at 10:59, O. Hartmann <ohartmann@walstatt.org> w=
rote:
> >>>>>>>>=20
> >>>>>>>> On Tue, 24 Jul 2018 08:53:36 +0300
> >>>>>>>> Toomas Soome <tsoome@me.com> wrote:
> >>>>>>>>=20
> >>>>>>>>=20
> >>>>>>>> Hello Toomas Soome,
> >>>>>>>>=20
> >>>>>>>> I CC Allan Jude since I discovered something weird today regard=
ing
> >>>>>>>> the UEFI boot capabilities of USB flash devices and SSDs. See be=
low.
> >>>>>>>> =20
> >>>>>>>>>> On 24 Jul 2018, at 08:16, O. Hartmann <ohartmann@walstatt.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>=20
> >>>>>>>>>> On Mon, 23 Jul 2018 10:56:04 +0300
> >>>>>>>>>> Toomas Soome <tsoome@me.com> wrote:
> >>>>>>>>>> =20
> >>>>>>>>>>>> On 23 Jul 2018, at 10:27, O. Hartmann <ohartmann@walstatt.or=
g>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>=20
> >>>>>>>>>>>> On Fri, 13 Jul 2018 18:44:23 +0300
> >>>>>>>>>>>> Toomas Soome <tsoome@me.com <mailto:tsoome@me.com>> wrote:
> >>>>>>>>>>>> =20
> >>>>>>>>>>>>>> On 13 Jul 2018, at 17:44, O. Hartmann <o.hartmann@walstatt=
.org
> >>>>>>>>>>>>>> <mailto:o.hartmann@walstatt.org>> wrote:
> >>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>> -----BEGIN PGP SIGNED MESSAGE-----
> >>>>>>>>>>>>>> Hash: SHA512
> >>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>> Am Fri, 13 Jul 2018 14:26:51 +0300
> >>>>>>>>>>>>>> Toomas Soome <tsoome@me.com <mailto:tsoome@me.com>
> >>>>>>>>>>>>>> <mailto:tsoome@me.com <mailto:tsoome@me.com>>>
> >>>>>>>>>>>>>> schrieb: =20
> >>>>>>>>>>>>>>>> On 13 Jul 2018, at 14:00, O. Hartmann
> >>>>>>>>>>>>>>>> <ohartmann@walstatt.org> wrote:
> >>>>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>>>> The problem is some kind of weird. I face UEFI boot prob=
lems
> >>>>>>>>>>>>>>>> on GPT drives where the first partition begins at block =
40
> >>>>>>>>>>>>>>>> of the hdd/ssd.
> >>>>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>>>> I have two host in private use based on an
> >>>>>>>>>>>>>>>> outdated ASRock Z77-Pro4-M and Z77-Pro4 mainboard (IvyBr=
idge,
> >>>>>>>>>>>>>>>> Socket LGA1155). Both boards are equipted with the lates
> >>>>>>>>>>>>>>>> official available AMI firmware revision, dating to 2013.
> >>>>>>>>>>>>>>>> This is for the Z77-Pro4-M revision 2.0 (2013/7/23) and =
for
> >>>>>>>>>>>>>>>> the Z77 Pro4 revision 1.8 (2013/7/17). For both boards a
> >>>>>>>>>>>>>>>> BETA revision for the Spectre/Meltdown mitigation is
> >>>>>>>>>>>>>>>> available, but I didn't test that. But please read.
> >>>>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>>>> The third box I realised this problem is a brand new Fuj=
itsu
> >>>>>>>>>>>>>>>> Esprimo Q956, also AMI firmware, at V5.0.0.11 R 1.26.0 f=
or
> >>>>>>>>>>>>>>>> 3413-A1x, date 05/25/2018 (or 20180525).
> >>>>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>>>> Installing on any kind of HDD or SSD manually or via
> >>>>>>>>>>>>>>>> bsdinstall the OS using UEFI-only boot method on a GPT
> >>>>>>>>>>>>>>>> partitioned device fails. The ASRock boards jump immedia=
tely
> >>>>>>>>>>>>>>>> into the firmware, the Fujitsu offers some kind of
> >>>>>>>>>>>>>>>> CPU/Memory/HDD test facility.
> >>>>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>>>> If on both type of vendor/boards CSM is disabled and UEFI
> >>>>>>>>>>>>>>>> boot only is implied, the MBR partitioned FreeBSD
> >>>>>>>>>>>>>>>> installation USB flash device does boot in UEFI! I guess=
I
> >>>>>>>>>>>>>>>> can assume this when the well known clumsy 80x25 char
> >>>>>>>>>>>>>>>> console suddenly gets bright and shiny with a much higher
> >>>>>>>>>>>>>>>> resoltion as long the GPU supports EFI GOP. Looking with
> >>>>>>>>>>>>>>>> gpart at the USB flash drives reveals that the EFI parti=
tion
> >>>>>>>>>>>>>>>> starts at block 1 of the device and the device has a MBR
> >>>>>>>>>>>>>>>> layout. I haven't found a way to force the GPT scheme, w=
hen
> >>>>>>>>>>>>>>>> initialised via gpart, to let the partitions start at bl=
ock
> >>>>>>>>>>>>>>>> 1. This might be a naiv thinking, so please be patient w=
ith
> >>>>>>>>>>>>>>>> me.
> >>>>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>>>> I do not know whether this is a well-known issue. On the
> >>>>>>>>>>>>>>>> ASRock boards, I tried years ago some LinuxRed Hat and S=
use
> >>>>>>>>>>>>>>>> with UEFI and that worked - FreeBSD not. I gave up on th=
at
> >>>>>>>>>>>>>>>> that time. Now, having the very same issues with a new
> >>>>>>>>>>>>>>>> Fujitsu system, leaves me with the impression that FreeB=
SD's
> >>>>>>>>>>>>>>>> UEFI implementation might have problems I'm not aware of.
> >>>>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>>>> Can someone shed some light onto this?=20
> >>>>>>>>>>>>>>>> =20
> >>>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>>> The first thing to check is if the secure boot is disable=
d. We
> >>>>>>>>>>>>>>> do not support secure boot at all at this time. =
=20
> >>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>> Secure boot is in every scenario disabled!
> >>>>>>>>>>>>>> =20
> >>>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>>> If you have efi or bios version running - you can check f=
rom
> >>>>>>>>>>>>>>> either console variable value (it can have efi or vidcons=
ole
> >>>>>>>>>>>>>>> or comconsole) or better yet, see if efi-version is set (=
show
> >>>>>>>>>>>>>>> efi-version) - if efi-version is not set, it is BIOS load=
er
> >>>>>>>>>>>>>>> running. Another indirect way is to see lsdev -v, with de=
vice
> >>>>>>>>>>>>>>> paths present, it is uefi:) =20
> >>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>> What are you talking about?
> >>>>>>>>>>>>>> What is the point of entry - running system, loader?
> >>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>> sysct machdep.bootmethod: BIOS
> >>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>> This makes me quite sure that the system has booted via BI=
OS -
> >>>>>>>>>>>>>> as I'm sure since I've checked that many times. UEFI doesn=
't
> >>>>>>>>>>>>>> work on those systems with FreeBSD. I'm not sure antmore, =
but
> >>>>>>>>>>>>>> I tried also Windows 7 on those mainboards booting via UEF=
I -
> >>>>>>>>>>>>>> and I might recall that they failed also. I also recall th=
at
> >>>>>>>>>>>>>> there were issues with earlier UEFI versions regarding boo=
ting
> >>>>>>>>>>>>>> only Windows 8/8.1 - and nothing else, but the fact that L=
inux
> >>>>>>>>>>>>>> worked confuses me a bit.
> >>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>> If this ASRock crap (never ever again this brand!) doesn't=
work
> >>>>>>>>>>>>>> at all - who cares, I intend to purchase new server grade
> >>>>>>>>>>>>>> hardware. But the more puzzling issue is with the Fujitsu,
> >>>>>>>>>>>>>> which I consider serious and from the behaviour the Fujitsu
> >>>>>>>>>>>>>> failure looks exactly like the ASRock - Windows 7 works,
> >>>>>>>>>>>>>> RedHat 7.5 works (I assume I can trust the Firmware settin=
gs
> >>>>>>>>>>>>>> when I disable CSM support, that the Firmware will only
> >>>>>>>>>>>>>> EFI/UEFI capable loader? Or is there a ghosty override
> >>>>>>>>>>>>>> somwhere to be expected?). Also on ASRock disabling CSM sh=
ould
> >>>>>>>>>>>>>> ensure not booting a dual-bootstrap-capable system. This s=
aid,
> >>>>>>>>>>>>>> on the recent Fujitsu, it seems to boil down to a FreeBSD
> >>>>>>>>>>>>>> UEFI-firmware interaction problem, while the ASRock is sti=
ll
> >>>>>>>>>>>>>> under suspicion to be broken by design. =20
> >>>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>>> GPT partitions can never start from disk absolute sector =
1;
> >>>>>>>>>>>>>>> this is because at sector 0 there is MBR (for compatibili=
ty),
> >>>>>>>>>>>>>>> sector 1 is GPT table and then sectors 2-33 have GPT
> >>>>>>>>>>>>>>> partition table entries, so the first possible data secto=
r is
> >>>>>>>>>>>>>>> 34 (absolute 34). Thats assuming 512B sectors. For details
> >>>>>>>>>>>>>>> see UEFI 2.7 Chapter 5.3.1 page 131. =20
> >>>>>>>>>>>>>>=20
> >>>>>>>>>>>>>> Thanks for the explanation. That implies the installer did
> >>>>>>>>>>>>>> right, gpart did also right and therefore there must be an
> >>>>>>>>>>>>>> issue with the stuff located within the EFI
> >>>>>>>>>>>>>> partition? =20
> >>>>>>>>>>>>>=20
> >>>>>>>>>>>>> Ok, so, it is not about UEFI bootcode but BIOS, and if we r=
each
> >>>>>>>>>>>>> BIOS loader at all or not - that is, if the BIOS bootstrap =
is
> >>>>>>>>>>>>> actually caring to read the MBR code and start it, since on=
ce
> >>>>>>>>>>>>> the MBR code is started, it is all about our code. =
=20
> >>>>>>>>>>>>=20
> >>>>>>>>>>>> I'm getting confused a bit here. Do you mean by "BIOS" the C=
SM?
> >>>>>>>>>>>> or do you mean that specific portion of the UEFI firmware, w=
hich
> >>>>>>>>>>>> looks for the proper UEFI partition?
> >>>>>>>>>>>> =20
> >>>>>>>>>>>=20
> >>>>>>>>>>> BIOS as either native or CSM. Note that from boot code point =
of
> >>>>>>>>>>> view the CSM boot *is* BIOS boot, we have no access to UEFI
> >>>>>>>>>>> features.=20
> >>>>>>>>>>>>=20
> >>>>>>>>>>>> The boxes in question, most notably the more recent Fujitsu
> >>>>>>>>>>>> Esprimo Q956, refuse booting UEFI, even if properly setup (in
> >>>>>>>>>>>> terms of what FreeBSD provides on recent CURRENT) is applied=
and
> >>>>>>>>>>>> CSM is switched off in the firmware. Again: GPT partition sc=
heme.
> >>>>>>>>>>>>=20
> >>>>>>>>>>>> The system boots properly if a second partition of type
> >>>>>>>>>>>> "freebsd-boot" is applied and bootcode is properly applied v=
ia
> >>>>>>>>>>>> "gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 2 ada0" (a=
da0
> >>>>>>>>>>>> is the device). =20
> >>>>>>>>>>>>>=20
> >>>>>>>>>>>>> btw, you can try to validate the installed boot blocks by u=
sing
> >>>>>>>>>>>>> recent enough loader (usb or iso) and then you can use from=
OK
> >>>>>>>>>>>>> prompt: =20
> >>>>>>>>>>>>=20
> >>>>>>>>>>>> lsdev provides me with the follwoing informations (CSM enabl=
ed):
> >>>>>>>>>>>>=20
> >>>>>>>>>>>> OK lsdev
> >>>>>>>>>>>> disk devices:
> >>>>>>>>>>>> disk0: BIOS DRIVE C ...
> >>>>>>>>>>>>=20
> >>>>>>>>>>>> disk0p1: EFI
> >>>>>>>>>>>> disk0p2: FreeBSD BOOT
> >>>>>>>>>>>> disk0p3: FreeBSD SWAP
> >>>>>>>>>>>> disk0p4: FreeBSD ZFS
> >>>>>>>>>>>> zfs devices:
> >>>>>>>>>>>> zfs:zroot
> >>>>>>>>>>>>=20
> >>>>>>>>>>>> OK chain disk0
> >>>>>>>>>>>> open failed (so for disk0p{1-4}.
> >>>>>>>>>>>>=20
> >>>>>>>>>>>> OK chain zroot
> >>>>>>>>>>>> failed to read disk (just for completeness) =20
> >>>>>>>>>>>=20
> >>>>>>>>>>>=20
> >>>>>>>>>>> chain command does use only device name (such as disk0: or
> >>>>>>>>>>> disk0p2: ), but not zfs pool as device. I just found I haven=
?t
> >>>>>>>>>>> ported the code to read the file. =20
> >>>>>>>>>>=20
> >>>>>>>>>> ??
> >>>>>>>>>> =20
> >>>>>>>>>>>=20
> >>>>>>>>>>> the point for chain command test is to see if the normal read=
and
> >>>>>>>>>>> execute would work, so in your case please try:
> >>>>>>>>>>>=20
> >>>>>>>>>>> chain disk0: =20
> >>>>>>>>>>=20
> >>>>>>>>>> As stated above, I did so, and the result is also mentioned ab=
ove,
> >>>>>>>>>> I always get "open failed".
> >>>>>>>>>> This is the same for=20
> >>>>>>>>>>=20
> >>>>>>>>>> chain disk0
> >>>>>>>>>> chain disk0p1
> >>>>>>>>>> chain disk0p2
> >>>>>>>>>> chain disk0p3
> >>>>>>>>>> chain disk0p4
> >>>>>>>>>>=20
> >>>>>>>>>> as already said. CSM is enabled in this case. =20
> >>>>>>>>>=20
> >>>>>>>>> sigh? chain command does take device as argument, device must a=
lways
> >>>>>>>>> end with colon?. in this case, the devil is in details:) as I w=
rote
> >>>>>>>>> above, the command should be:
> >>>>>>>>>=20
> >>>>>>>>> chain disk0:
> >>>>>>>>>=20
> >>>>>>>>> The disk0p1: etc will only work when partition boot code was
> >>>>>>>>> installed (which you most likely do not have - the only possible
> >>>>>>>>> candidate could be FreeBSD ZFS partition). =20
> >>>>>>>>=20
> >>>>>>>> The command "chain disk0:" works as expected (CSM enabled, GPT
> >>>>>>>> partition scheme, but with PMBR bootblock installed and freebsd-=
boot
> >>>>>>>> partition conatining gptzfsboot installed.
> >>>>>>>>=20
> >>>>>>>> =20
> >>>>>>>>> =20
> >>>>>>>>>> =20
> >>>>>>>>>>>=20
> >>>>>>>>>>> to read pmbr (512B) and execute it. The expected outcome woul=
d be
> >>>>>>>>>>> that pmbr boot code would browse the GPT, read stage1 from
> >>>>>>>>>>> disk0p2: and execute it; stage1 would detect FreeBSD ZFS from
> >>>>>>>>>>> disk0p4: and load and execute /boot/loader. If that will happ=
en,
> >>>>>>>>>>> it means the boot code in our stages is just fine, but the bi=
os
> >>>>>>>>>>> (CSM) does not load pmbr?. if thats true, it would mean that=
you
> >>>>>>>>>>> either need to use UEFI boot or need to have some hack to fool
> >>>>>>>>>>> the BIOS or just not use GPT on that machine with CSM. =
=20
> >>>>>>>>>>=20
> >>>>>>>>>> To make it clear here: The only way to boot this box is using =
CSM
> >>>>>>>>>> (as it is the same with the ASRock boards mentioned earlier). =
But
> >>>>>>>>>> my intention is to disable CSM and use a GPT/UEFI environment
> >>>>>>>>>> only! And GPT/UEFI doesn't work with FreeBSD, neither with
> >>>>>>>>>> 12-CURRENT, nor 11.2-RELENG.
> >>>>>>>>>>=20
> >>>>>>>>>> It would be nice if this could be fixed. I'm more interested i=
n the
> >>>>>>>>>> fix on the recent Fujitsu device than the outdated ASRock crap=
, but
> >>>>>>>>>> if the fix for the Fujitsu Firmware could fix older issues as a
> >>>>>>>>>> byproduct, I'd appreciate that.
> >>>>>>>>>>=20
> >>>>>>>>>> Kind regards,
> >>>>>>>>>> =20
> >>>>>>>>>=20
> >>>>>>>>> ok, somehow I have lost that part of the discussion. Well, you =
wrote
> >>>>>>>>> that the UEFI boot fails when the first partition starts from s=
ector
> >>>>>>>>> 40 - does it mean you have boot when the partition will start f=
rom
> >>>>>>>>> some other sector? I think, there is something else going
> >>>>>>>>> on. =20
> >>>>>>>>=20
> >>>>>>>> Well, I simply try to describe what I "see" to make things
> >>>>>>>> disambiguous. I'm not familiar with the deeper insights of disk
> >>>>>>>> layouts on a binary level. So, you explained to me the reason, w=
hy
> >>>>>>>> ESP (EGI partition) starts at block 40. I compared that to the
> >>>>>>>> FreeBSD USB flash image FreeBSD provides, but this is another st=
ory
> >>>>>>>> since the image uses MBR scheme as I figured out.
> >>>>>>>>=20
> >>>>>>>> =20
> >>>>>>>>>=20
> >>>>>>>>> What you can do is to see if that firmware will offer you EFI s=
hell
> >>>>>>>>> option, from there you can try to start the bootx64.efi manuall=
y and
> >>>>>>>>> see what error you will get. However, the number 1 cause for fa=
iling
> >>>>>>>>> to start the bootloader in UEFI is secure boot - we do not supp=
ort
> >>>>>>>>> it and secure boot must be switched off.=20
> >>>>>>>>>=20
> >>>>>>>>> However, they seem to claim "The Secure Boot option is availabl=
e in
> >>>>>>>>> the UEFI/BIOS of most if not all ASRock boards. It is disabled =
by
> >>>>>>>>> default.?=20
> >>>>>>>>>=20
> >>>>>>>>> Still suggest to double check if thats really the case. Also, i=
f the
> >>>>>>>>> bootx64.efi start will fail and no messages are appearing on sc=
reen,
> >>>>>>>>> then either there is something in firmware logs or you could get
> >>>>>>>>> them from trying to start bootx64.efi from UEFI shell. =20
> >>>>>>>>=20
> >>>>>>>> Since I'm with this problem since 2014 and try from time to time=
, be
> >>>>>>>> ausred that I tried every possible permutationof all reasonable
> >>>>>>>> options, even those nonsense, to get rid of that problem.
> >>>>>>>>=20
> >>>>>>>> I never had any problems with any other UEFI capable
> >>>>>>>> server/workstation firmware so far booting FreeBSD off in
> >>>>>>>> UEFI-native (GPT partition scheme, CSM disabled) so far - until =
now,
> >>>>>>>> when I ran into this Fujitsu ESPRIMO Q956 with the most recent
> >>>>>>>> firmware (as of lat week, week 29 of 2018) having the very same
> >>>>>>>> problems.=20
> >>>>>>>>=20
> >>>>>>>>=20
> >>>>>>>>=20
> >>>>>>>> I figured out something strange on the Fujitsu - and that is the=
same
> >>>>>>>> with the ASRock boards.
> >>>>>>>>=20
> >>>>>>>> We/I prepare some USB flash drives to boot a NanoBSD for a very =
small
> >>>>>>>> appliance, but nevertheless, the USB flash device is booted on
> >>>>>>>> Fujitsu servers with UEFI-only configurations. I assume at this
> >>>>>>>> point that disabling on the most recent Fujitsu firmwares on
> >>>>>>>> reasonable "new" hardware (not older than three years) will disa=
ble
> >>>>>>>> any(!) legacy BIOS capabilities. The same is assumed for the Fuj=
itus
> >>>>>>>> ESPRIMO Q956. I can not speak for the ASRock A77 Pro4/m boards
> >>>>>>>> mentioned above/earlier, they are from 2012/2013 and "quite old".
> >>>>>>>>=20
> >>>>>>>> The NanoBSD image of ours doesn't have a "freebsd-boot" partitio=
n.
> >>>>>>>> The partition scheme of the flash device is GPT. The layout looks
> >>>>>>>> like this:
> >>>>>>>>=20
> >>>>>>>> gpart show -l da4 =20
> >>>>>>>> =3D> 40 15425456 da4 GPT (7.4G) =20
> >>>>>>>> 40 2000 1 efiboot0 (1.0M)
> >>>>>>>> 2040 1453584 3 disk1a (710M)
> >>>>>>>> 1455624 4096 5 disk3 (2.0M)
> >>>>>>>> 1459720 13965776 - free - (6.7G)
> >>>>>>>>=20
> >>>>>>>> I created the flash with md, gpart and dd straightforward, efibo=
ot0
> >>>>>>>> is the ESP partition and its format/content is created via dd
> >>>>>>>> if=3D/boot/boot1.efifat of=3D/dev/da4p1 - I presume this is very=
simple.
> >>>>>>>>=20
> >>>>>>>> This USB flash device boots(!) successfully (UEFI!) on both the
> >>>>>>>> ASRock boards and the Esprimo Q956!
> >>>>>>>>=20
> >>>>>>>> But any SSD prepared the same way doesn't. Why?=20
> >>>>>>>>=20
> >>>>>>>> On the ASRock, I recall having fiddled around with HDD also for a
> >>>>>>>> while conatining Windows 7/SP1 and FreeBSD. Windows7 booted, Fre=
eBSD
> >>>>>>>> - I can't remember.=20
> >>>>>>>>=20
> >>>>>>>> In the lack of proper hardware I'm unable to check whether
> >>>>>>>> USB-attached HDD or SSD will boot or HDD will boot (just in case=
the
> >>>>>>>> local SATA has problems booting UEFI and USB not).
> >>>>>>>>=20
> >>>>>>>> Kind regards,
> >>>>>>>>=20
> >>>>>>>> Oliver=20
> >>>>>>>> =20
> >>>>>>>=20
> >>>>>>> Am. well. I think the suggestion to test out FAT32 is still good =
one
> >>>>>>> to test. This is because it is known that some vendors do not sup=
port
> >>>>>>> booting FAT12/FAT16 from HDD (the likely reason is that UEFI
> >>>>>>> specification does not tell which FAT must be supported, and only=
hint
> >>>>>>> about FAT12/FAT16 in context of removable devices). =20
> >>>>>>=20
> >>>>>> I prepared yesterday a GTP/ZFS-only 11.2-RELENG on the ESPRIMO Q95=
6. It
> >>>>>> took me a time to circumvent the installer and I had to install the
> >>>>>> system manually. In that strain, I also "tried" to establish the E=
SP
> >>>>>> with FAT32, as Allen Jude suggested earlier. I didn't find any ad =
hoc
> >>>>>> help how to find out the format (FAT12/16/32) of the ESP, so I ass=
ume
> >>>>>> when using 12-CURRENT's or 11.2-RELENG's installer with AUTO-ZFS a=
nd
> >>>>>> GPT (UEFI) (only!) the resulting ESP is FAT12 or FAT16 (300mb by
> >>>>>> default). I also assume, that when dd'ing the /boo/boot1.efifat im=
age
> >>>>>> to a partition, the format is FAT, but not FAT32. Therefore, I
> >>>>>> refomatted the manually created ESP (using "gpart add -t efi ...")
> >>>>>> using "newfs_msdos -F 32 -b xxx ...". I had to fiddle around a bit
> >>>>>> with option -b to fit in a proper format to meet a 512mb ESP - I'm=
not
> >>>>>> sure whether this is the proper option to deal with. When using the
> >>>>>> default and only -F32, the size of the partition has to be 4G at l=
east
> >>>>>> I assume. Having done that, I copied the the content of boot1.efif=
at
> >>>>>> (mdconfig -t vnode ..., I guess we know the drill ...) to the newly
> >>>>>> formatted ESP to /boot/efi/ ...
> >>>>>>=20
> >>>>>> Having so far no knowledge of how to asure that the created ESP is
> >>>>>> FAT32, I assume it is FAT32.
> >>>>>>=20
> >>>>>> The result is negative on the ESPRIMO Q956. When disabling the CSM=
, the
> >>>>>> box is not willing to boot from SSD with the ESP prepared as decri=
bed.
> >>>>>> So, a chance that this might still be due to a misconfiguration li=
es
> >>>>>> now within the -b option of newfs_msdos - if the -b option is assu=
med
> >>>>>> the proper option?
> >>>>>>=20
> >>>>>> At the moment, the ESP of the Esprimo is subject to changes, if you
> >>>>>> wish, but not in size, since it is limited to 512mb.
> >>>>>>=20
> >>>>>> Thanks and kind regards,
> >>>>>>=20
> >>>>>> Oliver =20
> >>>>>=20
> >>>>> Yea, i was hoping fstyp command would report the FAT type, but it d=
oes
> >>>>> not (request for feature?:) =20
> >>>>=20
> >>>> FYI, the file(1) command is very good at disecting a disk image to t=
ell
> >>>> you what the MBR looks like, and at looking at partitions if pointed=
at
> >>>> them with the -s option. It should be able to detect FAT12/16/32.
> >>>>=20
> >>>> root@x230a:/home/ISO/x # file -s /dev/md2s1
> >>>> /dev/md2s1: DOS/MBR boot sector, code offset 0x3c+2, OEM-ID "BSD4.4 =
",
> >>>> root entries 512, sectors 1600 (volumes <=3D32 MB) , sectors/FAT 5,
> >>>> sectors/track 63, heads 1, serial number 0xbd4111ee, label: "EFISYS
> >>>> ", FAT (12 bit), followed by FAT
> >>>> =20
> >>>>>=20
> >>>>> However, the more annoying idea would be to install some OS which w=
ill
> >>>>> boot with UEFI on this machine, then copy boot1.efi from freebsd to=
it
> >>>>> (the default program the UEFI will load is ESP:EFI/boot/bootx64.efi=
in
> >>>>> case of UEFI64 and ESP:EFI/boot/bootia32.efi for EFI32. However, we=
do
> >>>>> not support EFI32.
> >>>>>=20
> >>>>> note that boot1.efi alone will not do much but printing on screen h=
ow it
> >>>>> will search for freebsd, but for the purpose of the test it would
> >>>>> suffice
> >>>>> - that would give us confirmed working ESP file system (since the o=
ther
> >>>>> os would be able to boot) and then we can confirm if boot1.efi itse=
lf is
> >>>>> OK. =20
> >>>> =20
> >>>=20
> >>> Some new results.
> >>> I installed RedHat 7.5 and inestigated the ESP.
> >>>=20
> >>> - The ESP starts at block 2048, while FreeBSD's ESP starts at block 4=
0.
> >>> - size is both 200mb if installed automatically. I forgit to investig=
ate
> >>> the FAT format, but this might be unnecessary as shown further in this
> >>> post.
> >>> - RedHat's ESP contains ~ 10 MB of data in two
> >>> folders, /efi/boot, /efi/redhat. copying FreeBSD's BOOTX64.efi over
> >>> RedHat's doesn't change anything, also renaming /efi/boot/fbx64.efi of
> >>> RedHat's installation. But renaming /efi/redhat renders RedHat to fai=
l the
> >>> boot process on the Fujitsu with the signs of the built-in testprogra=
m as
> >>> reported.
> >>>=20
> >>> I took the liberty and installed 11.2-RELENG again, ZFS only, UEFI bo=
ot
> >>> only (CSM in firmware disabled, but there is still a gptzfsboot-prepa=
red
> >>> partition for later use, just for the record). Booting UEFI-only fail=
s as
> >>> reported. On this installation I copied the RedHat ESP completely into
> >>> FreeBSD's ESP, renamed /efi/boot/BOOTX64.efi to /efi/boot/BOOTX64.efi=
.rh
> >>> and copied FreeBSD's BOOTX64.efi to /efi/boot.=20
> >>> The Esprimo Q956 tries then to boot(!) RedHat's kernel. It seems, that
> >>> the /efi/redhat folder of the ESP is important, if renamed, the whole
> >>> process dies as I reported earlier.
> >>>=20
> >>> Still unanswered is the question: why is a NanoBSD prepared UEFI-only=
USB
> >>> flash booting with CSM disabled (so asumingly UEFI only then) on both
> >>> ASRock and Fujitsu (as described in more detail initially and earlier=
),
> >>> while SSDs fail? Is there a difference? Since FreeBSD boots in UEFI m=
ode
> >>> from USB flash prepared as described (straight forward, 800k ESP star=
ting
> >>> at block 40, the boot1.efifat image dd'ed onto the partition, UFS
> >>> partition following, no freebsd-boot partition or MBR/PMBR bootcode
> >>> applied ever!), I think BOOTX64.EFI is technically all right. There m=
ust
> >>> be then an issue with the SATA/SSD/HDD boot pathway.
> >>>=20
> >>> Hope I could provide some more details, sorry if it sounds confusing =
or
> >>> way too long, but I try to descibe the situation as thorough as possi=
ble.
> >>> =20
> >>=20
> >> OK, this is already good hint. The thing with ESP is that there is
> >> =E2=80=9Cdefault=E2=80=9D file system tree: EFI/BOOT/BOOT<sysname>.EFI=
, this is noted as
> >> default for *removable* media, fortunately it is usable for hard disks=
as
> >> well, or at least in most cases.
> >>=20
> >> Now, for non-removable media, the UEFI does provide boot manager inter=
face
> >> to define boot entries, and the fact that renaming efi/redhad director=
y did
> >> break the redhat boot, is very loud hint. And this means, this system =
is
> >> probably ignoring efi/boot tree on non-removable media and is expectin=
g the
> >> boot manager entry to be created instead. =20
> >=20
> > This inplication I'd confirm for the recent Fujitsu ESPRIMO Q956 firmwa=
re
> > (not tested on ASRock Z77-Pro4 firmware).
> > =20
> >>=20
> >> UEFI boot manager can be configured /usually/ manually via firmware me=
nu,
> >> or by application, such as efibootmgr. The normal approach is to create
> >> efi/<vendorname> directory and to copy the application there, then cre=
ate
> >> the boot manager configuration.
> >>=20
> >> See UEFI specification v2.7, chapter 3 Boot Manager, page 79.
> >>=20
> >> What is different in FreeBSD case is that the whole interface to progr=
am
> >> the UEFI Boot Manager is currently being developed, so either it has t=
o be
> >> done manually or from some other OS (see
> >> https://wiki.gentoo.org/wiki/Efibootmgr
> >> <https://wiki.gentoo.org/wiki/Efibootmgr>; for example, first hit from
> >> google:D). =20
> >=20
> > Well, thanks for this important hint! FreeBSD 12-CURRENT's and FreeBSD
> > 11.2-RELENG's USB flash devices are capable of booting off on Fujitsu's
> > ESPRIMO and ASRock's boards. As a note: after "kldload efirt.ko" I was =
able
> > to use the already in FreeBSD present toolset efibootmgr(8) and sibblin=
gs
> > (the tools do not do anything useful when booted non-UEFI).
> >=20
> > Mounting the ESP of the harddrive (in my case, ada0p1) to /mnt and foll=
owing
> > the steps in the examples and having created /efi/freebsd/BOOTx64.efi as
> > recommended by copy from /efi/boot, let me create a proper boot variabl=
e.
> >=20
> > To make things sure, I also applied "efibootmgr -a VARIABLENAME".
> >=20
> > And ... it worked! Yes, it worked! The ESP is FAT32 formatted, I do not=
know
> > whether this will also work with FAT12/16, I should test this case, too.
> >=20
> > There is a bug in the manpage of efibootmg(8). It does not explain the
> > options -d and -p, although they could be "implied" by reading carefull=
y.
> > There is now a PR at=20
> >=20
> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230080
> >=20
> > for this issue.
> >=20
> > So, it's apity that the handbook has no note I could easily find on thi=
s;=20
> >=20
> > Thank you very much for your patience and help!
> >=20
> > Kind regards,
> > Oliver =20
>=20
> yep, efibootmgr does call UEFI RuntimeServices to set up the variables, a=
nd
> this is only possible when booted UEFI. But glad we finally found the root
> cause. It would be good to have HW notes for such cases, it is important =
to
> know that those systems wont boot UEFI from HDD unless the boot manager s=
etup
> is done.
>=20
> rgds,
> toomas
This pops up the question how to deal with such HW. We have as a institutio=
n a
lot of Fujitsu hardware her - from larger servers down to Fujitsu ESPRIMO Q=
956.
The latter one is the first (and so far the only) piece of hardware I found
incapable of booting off UEFI within the past 5 years.
If the "standard" for removeable devices is to boot from /efi/boot/bootx64.=
efi,
than I'd guess it is good luck for FreeBSD that the firmware vendors did
fallback to such a mechanism. GRUB/Linux seem to install by default their
bootloader into /efi/something/ I'll check on Debian, Suse and CentOS so far
soon, the latter probably will, since its the "free" RedHat).
Anyway, apart from any criticism, I'm glad to have the tools to make things
work without using alien help (outside FreeBSD's world!). That is a thank y=
ou
towards the developers.
Kind regards,
oh
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180727120232.270e1d9f>