Date: Wed, 25 Jul 2018 12:26:13 +0300 From: Toomas Soome <tsoome@me.com> To: "O. Hartmann" <o.hartmann@walstatt.org> Cc: freebsd-current <freebsd-current@freebsd.org>, Allan Jude <allanjude@freebsd.org> Subject: Re: [UEFI] Boot issues on some UEFI implementations Message-ID: <1E6058D2-5804-480B-B6AF-66AA02CDD7AD@me.com> In-Reply-To: <20180725111032.1632b885@freyja.zeit4.iv.bundesimmobilien.de> References: <20180713130001.219a0a84@freyja.zeit4.iv.bundesimmobilien.de> <68505F98-E840-4148-9E48-BDB350F7431A@me.com> <20180713164447.42430301@thor.intern.walstatt.dynvpn.de> <680FBB42-75BF-427F-AA3B-6D864E83ED1F@me.com> <20180723092735.12a5d2a8@freyja.zeit4.iv.bundesimmobilien.de> <80523BE6-C035-482D-B134-23812183DE83@me.com> <20180724071514.6cb9d111@freyja.zeit4.iv.bundesimmobilien.de> <16439773-3E9A-40FF-99A1-32E97CCEE2C3@me.com> <20180725095926.7d52a15a@freyja.zeit4.iv.bundesimmobilien.de> <2FCF2BCC-05E5-466A-B1AE-90300990ED1A@me.com> <20180725111032.1632b885@freyja.zeit4.iv.bundesimmobilien.de>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 25 Jul 2018, at 12:10, O. Hartmann <o.hartmann@walstatt.org> wrote:
>=20
> On Wed, 25 Jul 2018 11:46:07 +0300
> Toomas Soome <tsoome@me.com> wrote:
>=20
>>> On 25 Jul 2018, at 10:59, O. Hartmann <ohartmann@walstatt.org> =
wrote:
>>>=20
>>> On Tue, 24 Jul 2018 08:53:36 +0300
>>> Toomas Soome <tsoome@me.com> wrote:
>>>=20
>>>=20
>>> Hello Toomas Soome,
>>>=20
>>> I CC Allan Jude since I discovered something weird today regarding =
the UEFI
>>> boot capabilities of USB flash devices and SSDs. See below.
>>>=20
>>>>> On 24 Jul 2018, at 08:16, O. Hartmann <ohartmann@walstatt.org> =
wrote:
>>>>>=20
>>>>> On Mon, 23 Jul 2018 10:56:04 +0300
>>>>> Toomas Soome <tsoome@me.com> wrote:
>>>>>=20
>>>>>>> On 23 Jul 2018, at 10:27, O. Hartmann <ohartmann@walstatt.org> =
wrote:
>>>>>>>=20
>>>>>>> On Fri, 13 Jul 2018 18:44:23 +0300
>>>>>>> Toomas Soome <tsoome@me.com <mailto:tsoome@me.com>> wrote:
>>>>>>>=20
>>>>>>>>> On 13 Jul 2018, at 17:44, O. Hartmann <o.hartmann@walstatt.org
>>>>>>>>> <mailto:o.hartmann@walstatt.org>> wrote:
>>>>>>>>>=20
>>>>>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>>>>>> Hash: SHA512
>>>>>>>>>=20
>>>>>>>>> Am Fri, 13 Jul 2018 14:26:51 +0300
>>>>>>>>> Toomas Soome <tsoome@me.com <mailto:tsoome@me.com>
>>>>>>>>> <mailto:tsoome@me.com <mailto:tsoome@me.com>>> schrieb: =20
>>>>>>>>>>> On 13 Jul 2018, at 14:00, O. Hartmann =
<ohartmann@walstatt.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>=20
>>>>>>>>>>> The problem is some kind of weird. I face UEFI boot problems =
on GPT
>>>>>>>>>>> drives where the first partition begins at block 40 of the =
hdd/ssd.
>>>>>>>>>>>=20
>>>>>>>>>>> I have two host in private use based on an
>>>>>>>>>>> outdated ASRock Z77-Pro4-M and Z77-Pro4 mainboard =
(IvyBridge, Socket
>>>>>>>>>>> LGA1155). Both boards are equipted with the lates official =
available
>>>>>>>>>>> AMI firmware revision, dating to 2013. This is for the =
Z77-Pro4-M
>>>>>>>>>>> revision 2.0 (2013/7/23) and for the Z77 Pro4 revision 1.8
>>>>>>>>>>> (2013/7/17). For both boards a BETA revision for the
>>>>>>>>>>> Spectre/Meltdown mitigation is available, but I didn't test =
that.
>>>>>>>>>>> But please read.
>>>>>>>>>>>=20
>>>>>>>>>>> The third box I realised this problem is a brand new Fujitsu =
Esprimo
>>>>>>>>>>> Q956, also AMI firmware, at V5.0.0.11 R 1.26.0 for 3413-A1x, =
date
>>>>>>>>>>> 05/25/2018 (or 20180525).
>>>>>>>>>>>=20
>>>>>>>>>>> Installing on any kind of HDD or SSD manually or via =
bsdinstall the
>>>>>>>>>>> OS using UEFI-only boot method on a GPT partitioned device =
fails.
>>>>>>>>>>> The ASRock boards jump immediately into the firmware, the =
Fujitsu
>>>>>>>>>>> offers some kind of CPU/Memory/HDD test facility.
>>>>>>>>>>>=20
>>>>>>>>>>> If on both type of vendor/boards CSM is disabled and UEFI =
boot only
>>>>>>>>>>> is implied, the MBR partitioned FreeBSD installation USB =
flash
>>>>>>>>>>> device does boot in UEFI! I guess I can assume this when the =
well
>>>>>>>>>>> known clumsy 80x25 char console suddenly gets bright and =
shiny with
>>>>>>>>>>> a much higher resoltion as long the GPU supports EFI GOP. =
Looking
>>>>>>>>>>> with gpart at the USB flash drives reveals that the EFI =
partition
>>>>>>>>>>> starts at block 1 of the device and the device has a MBR =
layout. I
>>>>>>>>>>> haven't found a way to force the GPT scheme, when =
initialised via
>>>>>>>>>>> gpart, to let the partitions start at block 1. This might be =
a naiv
>>>>>>>>>>> thinking, so please be patient with me.
>>>>>>>>>>>=20
>>>>>>>>>>> I do not know whether this is a well-known issue. On the =
ASRock
>>>>>>>>>>> boards, I tried years ago some LinuxRed Hat and Suse with =
UEFI and
>>>>>>>>>>> that worked - FreeBSD not. I gave up on that that time. Now, =
having
>>>>>>>>>>> the very same issues with a new Fujitsu system, leaves me =
with the
>>>>>>>>>>> impression that FreeBSD's UEFI implementation might have =
problems
>>>>>>>>>>> I'm not aware of.
>>>>>>>>>>>=20
>>>>>>>>>>> Can someone shed some light onto this?=20
>>>>>>>>>>>=20
>>>>>>>>>>=20
>>>>>>>>>> The first thing to check is if the secure boot is disabled. =
We do not
>>>>>>>>>> support secure boot at all at this time. =20
>>>>>>>>>=20
>>>>>>>>> Secure boot is in every scenario disabled!
>>>>>>>>>=20
>>>>>>>>>>=20
>>>>>>>>>> If you have efi or bios version running - you can check from =
either
>>>>>>>>>> console variable value (it can have efi or vidconsole or =
comconsole)
>>>>>>>>>> or better yet, see if efi-version is set (show efi-version) - =
if
>>>>>>>>>> efi-version is not set, it is BIOS loader running. Another =
indirect
>>>>>>>>>> way is to see lsdev -v, with device paths present, it is
>>>>>>>>>> uefi:) =20
>>>>>>>>>=20
>>>>>>>>> What are you talking about?
>>>>>>>>> What is the point of entry - running system, loader?
>>>>>>>>>=20
>>>>>>>>> sysct machdep.bootmethod: BIOS
>>>>>>>>>=20
>>>>>>>>> This makes me quite sure that the system has booted via BIOS - =
as I'm
>>>>>>>>> sure since I've checked that many times. UEFI doesn't work on =
those
>>>>>>>>> systems with FreeBSD. I'm not sure antmore, but I tried also =
Windows 7
>>>>>>>>> on those mainboards booting via UEFI - and I might recall that =
they
>>>>>>>>> failed also. I also recall that there were issues with earlier =
UEFI
>>>>>>>>> versions regarding booting only Windows 8/8.1 - and nothing =
else, but
>>>>>>>>> the fact that Linux worked confuses me a bit.
>>>>>>>>>=20
>>>>>>>>> If this ASRock crap (never ever again this brand!) doesn't =
work at
>>>>>>>>> all - who cares, I intend to purchase new server grade =
hardware. But
>>>>>>>>> the more puzzling issue is with the Fujitsu, which I consider =
serious
>>>>>>>>> and from the behaviour the Fujitsu failure looks exactly like =
the
>>>>>>>>> ASRock - Windows 7 works, RedHat 7.5 works (I assume I can =
trust the
>>>>>>>>> Firmware settings when I disable CSM support, that the =
Firmware will
>>>>>>>>> only EFI/UEFI capable loader? Or is there a ghosty override =
somwhere
>>>>>>>>> to be expected?). Also on ASRock disabling CSM should ensure =
not
>>>>>>>>> booting a dual-bootstrap-capable system. This said, on the =
recent
>>>>>>>>> Fujitsu, it seems to boil down to a FreeBSD UEFI-firmware =
interaction
>>>>>>>>> problem, while the ASRock is still under suspicion to be =
broken by
>>>>>>>>> design. =20
>>>>>>>>>>=20
>>>>>>>>>> GPT partitions can never start from disk absolute sector 1; =
this is
>>>>>>>>>> because at sector 0 there is MBR (for compatibility), sector =
1 is GPT
>>>>>>>>>> table and then sectors 2-33 have GPT partition table entries, =
so the
>>>>>>>>>> first possible data sector is 34 (absolute 34). Thats =
assuming 512B
>>>>>>>>>> sectors. For details see UEFI 2.7 Chapter 5.3.1 page 131. =
=20
>>>>>>>>>=20
>>>>>>>>> Thanks for the explanation. That implies the installer did =
right,
>>>>>>>>> gpart did also right and therefore there must be an issue with =
the
>>>>>>>>> stuff located within the EFI partition? =20
>>>>>>>>=20
>>>>>>>> Ok, so, it is not about UEFI bootcode but BIOS, and if we reach =
BIOS
>>>>>>>> loader at all or not - that is, if the BIOS bootstrap is =
actually
>>>>>>>> caring to read the MBR code and start it, since once the MBR =
code is
>>>>>>>> started, it is all about our code. =20
>>>>>>>=20
>>>>>>> I'm getting confused a bit here. Do you mean by "BIOS" the CSM? =
or do
>>>>>>> you mean that specific portion of the UEFI firmware, which looks =
for the
>>>>>>> proper UEFI partition?
>>>>>>>=20
>>>>>>=20
>>>>>> BIOS as either native or CSM. Note that from boot code point of =
view the
>>>>>> CSM boot *is* BIOS boot, we have no access to UEFI features.
>>>>>>=20
>>>>>>>=20
>>>>>>> The boxes in question, most notably the more recent Fujitsu =
Esprimo
>>>>>>> Q956, refuse booting UEFI, even if properly setup (in terms of =
what
>>>>>>> FreeBSD provides on recent CURRENT) is applied and CSM is =
switched off
>>>>>>> in the firmware. Again: GPT partition scheme.
>>>>>>>=20
>>>>>>> The system boots properly if a second partition of type =
"freebsd-boot"
>>>>>>> is applied and bootcode is properly applied via "gpart bootcode
>>>>>>> -b /boot/pmbr -p /boot/gptboot -i 2 ada0" (ada0 is the device). =
=20
>>>>>>>>=20
>>>>>>>> btw, you can try to validate the installed boot blocks by using =
recent
>>>>>>>> enough loader (usb or iso) and then you can use from OK prompt: =
=20
>>>>>>>=20
>>>>>>> lsdev provides me with the follwoing informations (CSM enabled):
>>>>>>>=20
>>>>>>> OK lsdev
>>>>>>> disk devices:
>>>>>>> disk0: BIOS DRIVE C ...
>>>>>>>=20
>>>>>>> disk0p1: EFI
>>>>>>> disk0p2: FreeBSD BOOT
>>>>>>> disk0p3: FreeBSD SWAP
>>>>>>> disk0p4: FreeBSD ZFS
>>>>>>> zfs devices:
>>>>>>> zfs:zroot
>>>>>>>=20
>>>>>>> OK chain disk0
>>>>>>> open failed (so for disk0p{1-4}.
>>>>>>>=20
>>>>>>> OK chain zroot
>>>>>>> failed to read disk (just for completeness) =20
>>>>>>=20
>>>>>>=20
>>>>>> chain command does use only device name (such as disk0: or =
disk0p2: ),
>>>>>> but not zfs pool as device. I just found I haven=E2=80=99t =
ported the code to
>>>>>> read the file. =20
>>>>>=20
>>>>> ??
>>>>>=20
>>>>>>=20
>>>>>> the point for chain command test is to see if the normal read and =
execute
>>>>>> would work, so in your case please try:
>>>>>>=20
>>>>>> chain disk0: =20
>>>>>=20
>>>>> As stated above, I did so, and the result is also mentioned above, =
I
>>>>> always get "open failed".
>>>>> This is the same for=20
>>>>>=20
>>>>> chain disk0
>>>>> chain disk0p1
>>>>> chain disk0p2
>>>>> chain disk0p3
>>>>> chain disk0p4
>>>>>=20
>>>>> as already said. CSM is enabled in this case. =20
>>>>=20
>>>> sigh=E2=80=A6 chain command does take device as argument, device =
must always end
>>>> with colon=E2=80=A6. in this case, the devil is in details:) as I =
wrote above, the
>>>> command should be:
>>>>=20
>>>> chain disk0:
>>>>=20
>>>> The disk0p1: etc will only work when partition boot code was =
installed
>>>> (which you most likely do not have - the only possible candidate =
could be
>>>> FreeBSD ZFS partition). =20
>>>=20
>>> The command "chain disk0:" works as expected (CSM enabled, GPT =
partition
>>> scheme, but with PMBR bootblock installed and freebsd-boot partition
>>> conatining gptzfsboot installed.
>>>=20
>>>=20
>>>>=20
>>>>>=20
>>>>>>=20
>>>>>> to read pmbr (512B) and execute it. The expected outcome would be =
that
>>>>>> pmbr boot code would browse the GPT, read stage1 from disk0p2: =
and
>>>>>> execute it; stage1 would detect FreeBSD ZFS from disk0p4: and =
load and
>>>>>> execute /boot/loader. If that will happen, it means the boot code =
in our
>>>>>> stages is just fine, but the bios (CSM) does not load pmbr=E2=80=A6=
. if thats
>>>>>> true, it would mean that you either need to use UEFI boot or need =
to have
>>>>>> some hack to fool the BIOS or just not use GPT on that machine =
with
>>>>>> CSM. =20
>>>>>=20
>>>>> To make it clear here: The only way to boot this box is using CSM =
(as it
>>>>> is the same with the ASRock boards mentioned earlier). But my =
intention
>>>>> is to disable CSM and use a GPT/UEFI environment only! And =
GPT/UEFI
>>>>> doesn't work with FreeBSD, neither with 12-CURRENT, nor =
11.2-RELENG.
>>>>>=20
>>>>> It would be nice if this could be fixed. I'm more interested in =
the fix on
>>>>> the recent Fujitsu device than the outdated ASRock crap, but if =
the fix
>>>>> for the Fujitsu Firmware could fix older issues as a byproduct, =
I'd
>>>>> appreciate that.
>>>>>=20
>>>>> Kind regards,
>>>>>=20
>>>>=20
>>>> ok, somehow I have lost that part of the discussion. Well, you =
wrote that
>>>> the UEFI boot fails when the first partition starts from sector 40 =
- does
>>>> it mean you have boot when the partition will start from some other
>>>> sector? I think, there is something else going on. =20
>>>=20
>>> Well, I simply try to describe what I "see" to make things =
disambiguous. I'm
>>> not familiar with the deeper insights of disk layouts on a binary =
level. So,
>>> you explained to me the reason, why ESP (EGI partition) starts at =
block 40.
>>> I compared that to the FreeBSD USB flash image FreeBSD provides, but =
this is
>>> another story since the image uses MBR scheme as I figured out.
>>>=20
>>>=20
>>>>=20
>>>> What you can do is to see if that firmware will offer you EFI shell =
option,
>>>> from there you can try to start the bootx64.efi manually and see =
what error
>>>> you will get. However, the number 1 cause for failing to start the
>>>> bootloader in UEFI is secure boot - we do not support it and secure =
boot
>>>> must be switched off.=20
>>>>=20
>>>> However, they seem to claim "The Secure Boot option is available in =
the
>>>> UEFI/BIOS of most if not all ASRock boards. It is disabled by =
default.=E2=80=9D=20
>>>>=20
>>>> Still suggest to double check if thats really the case. Also, if =
the
>>>> bootx64.efi start will fail and no messages are appearing on =
screen, then
>>>> either there is something in firmware logs or you could get them =
from
>>>> trying to start bootx64.efi from UEFI shell. =20
>>>=20
>>> Since I'm with this problem since 2014 and try from time to time, be =
ausred
>>> that I tried every possible permutationof all reasonable options, =
even those
>>> nonsense, to get rid of that problem.
>>>=20
>>> I never had any problems with any other UEFI capable =
server/workstation
>>> firmware so far booting FreeBSD off in UEFI-native (GPT partition =
scheme,
>>> CSM disabled) so far - until now, when I ran into this Fujitsu =
ESPRIMO Q956
>>> with the most recent firmware (as of lat week, week 29 of 2018) =
having the
>>> very same problems.=20
>>>=20
>>>=20
>>>=20
>>> I figured out something strange on the Fujitsu - and that is the =
same with
>>> the ASRock boards.
>>>=20
>>> We/I prepare some USB flash drives to boot a NanoBSD for a very =
small
>>> appliance, but nevertheless, the USB flash device is booted on =
Fujitsu
>>> servers with UEFI-only configurations. I assume at this point that
>>> disabling on the most recent Fujitsu firmwares on reasonable "new" =
hardware
>>> (not older than three years) will disable any(!) legacy BIOS =
capabilities.
>>> The same is assumed for the Fujitus ESPRIMO Q956. I can not speak =
for the
>>> ASRock A77 Pro4/m boards mentioned above/earlier, they are from =
2012/2013
>>> and "quite old".
>>>=20
>>> The NanoBSD image of ours doesn't have a "freebsd-boot" partition. =
The
>>> partition scheme of the flash device is GPT. The layout looks like =
this:
>>>=20
>>> gpart show -l da4 =20
>>> =3D> 40 15425456 da4 GPT (7.4G) =20
>>> 40 2000 1 efiboot0 (1.0M)
>>> 2040 1453584 3 disk1a (710M)
>>> 1455624 4096 5 disk3 (2.0M)
>>> 1459720 13965776 - free - (6.7G)
>>>=20
>>> I created the flash with md, gpart and dd straightforward, efiboot0 =
is the
>>> ESP partition and its format/content is created via dd =
if=3D/boot/boot1.efifat
>>> of=3D/dev/da4p1 - I presume this is very simple.
>>>=20
>>> This USB flash device boots(!) successfully (UEFI!) on both the =
ASRock
>>> boards and the Esprimo Q956!
>>>=20
>>> But any SSD prepared the same way doesn't. Why?=20
>>>=20
>>> On the ASRock, I recall having fiddled around with HDD also for a =
while
>>> conatining Windows 7/SP1 and FreeBSD. Windows7 booted, FreeBSD - I =
can't
>>> remember.=20
>>>=20
>>> In the lack of proper hardware I'm unable to check whether =
USB-attached HDD
>>> or SSD will boot or HDD will boot (just in case the local SATA has =
problems
>>> booting UEFI and USB not).
>>>=20
>>> Kind regards,
>>>=20
>>> Oliver=20
>>>=20
>>=20
>> Am. well. I think the suggestion to test out FAT32 is still good one =
to test.
>> This is because it is known that some vendors do not support booting
>> FAT12/FAT16 from HDD (the likely reason is that UEFI specification =
does not
>> tell which FAT must be supported, and only hint about FAT12/FAT16 in =
context
>> of removable devices).
>=20
> I prepared yesterday a GTP/ZFS-only 11.2-RELENG on the ESPRIMO Q956. =
It took me
> a time to circumvent the installer and I had to install the system =
manually. In
> that strain, I also "tried" to establish the ESP with FAT32, as Allen =
Jude
> suggested earlier. I didn't find any ad hoc help how to find out the =
format
> (FAT12/16/32) of the ESP, so I assume when using 12-CURRENT's or =
11.2-RELENG's
> installer with AUTO-ZFS and GPT (UEFI) (only!) the resulting ESP is =
FAT12 or
> FAT16 (300mb by default). I also assume, that when dd'ing the =
/boo/boot1.efifat
> image to a partition, the format is FAT, but not FAT32. Therefore, I =
refomatted
> the manually created ESP (using "gpart add -t efi ...") using =
"newfs_msdos -F
> 32 -b xxx ...". I had to fiddle around a bit with option -b to fit in =
a proper
> format to meet a 512mb ESP - I'm not sure whether this is the proper =
option to
> deal with. When using the default and only -F32, the size of the =
partition has
> to be 4G at least I assume. Having done that, I copied the the content =
of
> boot1.efifat (mdconfig -t vnode ..., I guess we know the drill ...) to =
the
> newly formatted ESP to /boot/efi/ ...
>=20
> Having so far no knowledge of how to asure that the created ESP is =
FAT32, I
> assume it is FAT32.
>=20
> The result is negative on the ESPRIMO Q956. When disabling the CSM, =
the box is
> not willing to boot from SSD with the ESP prepared as decribed. So, a =
chance
> that this might still be due to a misconfiguration lies now within the =
-b
> option of newfs_msdos - if the -b option is assumed the proper option?
>=20
> At the moment, the ESP of the Esprimo is subject to changes, if you =
wish, but
> not in size, since it is limited to 512mb.
>=20
> Thanks and kind regards,
>=20
> Oliver
Yea, i was hoping fstyp command would report the FAT type, but it does =
not (request for feature?:)
However, the more annoying idea would be to install some OS which will =
boot with UEFI on this machine, then copy boot1.efi from freebsd to it =
(the default program the UEFI will load is ESP:EFI/boot/bootx64.efi in =
case of UEFI64 and ESP:EFI/boot/bootia32.efi for EFI32. However, we do =
not support EFI32.
note that boot1.efi alone will not do much but printing on screen how it =
will search for freebsd, but for the purpose of the test it would =
suffice - that would give us confirmed working ESP file system (since =
the other os would be able to boot) and then we can confirm if boot1.efi =
itself is OK.
rgds,
toomas
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1E6058D2-5804-480B-B6AF-66AA02CDD7AD>