Date: Wed, 25 Jul 2018 11:10:32 +0200 From: "O. Hartmann" <o.hartmann@walstatt.org> To: Toomas Soome <tsoome@me.com> Cc: "O. Hartmann" <ohartmann@walstatt.org>, freebsd-current <freebsd-current@freebsd.org>, Allan Jude <allanjude@freebsd.org> Subject: Re: [UEFI] Boot issues on some UEFI implementations Message-ID: <20180725111032.1632b885@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: <2FCF2BCC-05E5-466A-B1AE-90300990ED1A@me.com> References: <20180713130001.219a0a84@freyja.zeit4.iv.bundesimmobilien.de> <68505F98-E840-4148-9E48-BDB350F7431A@me.com> <20180713164447.42430301@thor.intern.walstatt.dynvpn.de> <680FBB42-75BF-427F-AA3B-6D864E83ED1F@me.com> <20180723092735.12a5d2a8@freyja.zeit4.iv.bundesimmobilien.de> <80523BE6-C035-482D-B134-23812183DE83@me.com> <20180724071514.6cb9d111@freyja.zeit4.iv.bundesimmobilien.de> <16439773-3E9A-40FF-99A1-32E97CCEE2C3@me.com> <20180725095926.7d52a15a@freyja.zeit4.iv.bundesimmobilien.de> <2FCF2BCC-05E5-466A-B1AE-90300990ED1A@me.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 25 Jul 2018 11:46:07 +0300
Toomas Soome <tsoome@me.com> wrote:
> > On 25 Jul 2018, at 10:59, O. Hartmann <ohartmann@walstatt.org> wrote:
> >=20
> > On Tue, 24 Jul 2018 08:53:36 +0300
> > Toomas Soome <tsoome@me.com> wrote:
> >=20
> >=20
> > Hello Toomas Soome,
> >=20
> > I CC Allan Jude since I discovered something weird today regarding the=
UEFI
> > boot capabilities of USB flash devices and SSDs. See below.
> > =20
> >>> On 24 Jul 2018, at 08:16, O. Hartmann <ohartmann@walstatt.org> wrote:
> >>>=20
> >>> On Mon, 23 Jul 2018 10:56:04 +0300
> >>> Toomas Soome <tsoome@me.com> wrote:
> >>> =20
> >>>>> On 23 Jul 2018, at 10:27, O. Hartmann <ohartmann@walstatt.org> wrot=
e:
> >>>>>=20
> >>>>> On Fri, 13 Jul 2018 18:44:23 +0300
> >>>>> Toomas Soome <tsoome@me.com <mailto:tsoome@me.com>> wrote:
> >>>>> =20
> >>>>>>> On 13 Jul 2018, at 17:44, O. Hartmann <o.hartmann@walstatt.org
> >>>>>>> <mailto:o.hartmann@walstatt.org>> wrote:
> >>>>>>>=20
> >>>>>>> -----BEGIN PGP SIGNED MESSAGE-----
> >>>>>>> Hash: SHA512
> >>>>>>>=20
> >>>>>>> Am Fri, 13 Jul 2018 14:26:51 +0300
> >>>>>>> Toomas Soome <tsoome@me.com <mailto:tsoome@me.com>
> >>>>>>> <mailto:tsoome@me.com <mailto:tsoome@me.com>>> schrieb: =20
> >>>>>>>>> On 13 Jul 2018, at 14:00, O. Hartmann <ohartmann@walstatt.org>
> >>>>>>>>> wrote:
> >>>>>>>>>=20
> >>>>>>>>> The problem is some kind of weird. I face UEFI boot problems on=
GPT
> >>>>>>>>> drives where the first partition begins at block 40 of the hdd/=
ssd.
> >>>>>>>>>=20
> >>>>>>>>> I have two host in private use based on an
> >>>>>>>>> outdated ASRock Z77-Pro4-M and Z77-Pro4 mainboard (IvyBridge, S=
ocket
> >>>>>>>>> LGA1155). Both boards are equipted with the lates official avai=
lable
> >>>>>>>>> AMI firmware revision, dating to 2013. This is for the Z77-Pro4=
-M
> >>>>>>>>> revision 2.0 (2013/7/23) and for the Z77 Pro4 revision 1.8
> >>>>>>>>> (2013/7/17). For both boards a BETA revision for the
> >>>>>>>>> Spectre/Meltdown mitigation is available, but I didn't test tha=
t.
> >>>>>>>>> But please read.
> >>>>>>>>>=20
> >>>>>>>>> The third box I realised this problem is a brand new Fujitsu Es=
primo
> >>>>>>>>> Q956, also AMI firmware, at V5.0.0.11 R 1.26.0 for 3413-A1x, da=
te
> >>>>>>>>> 05/25/2018 (or 20180525).
> >>>>>>>>>=20
> >>>>>>>>> Installing on any kind of HDD or SSD manually or via bsdinstall=
the
> >>>>>>>>> OS using UEFI-only boot method on a GPT partitioned device fail=
s.
> >>>>>>>>> The ASRock boards jump immediately into the firmware, the Fujit=
su
> >>>>>>>>> offers some kind of CPU/Memory/HDD test facility.
> >>>>>>>>>=20
> >>>>>>>>> If on both type of vendor/boards CSM is disabled and UEFI boot =
only
> >>>>>>>>> is implied, the MBR partitioned FreeBSD installation USB flash
> >>>>>>>>> device does boot in UEFI! I guess I can assume this when the we=
ll
> >>>>>>>>> known clumsy 80x25 char console suddenly gets bright and shiny =
with
> >>>>>>>>> a much higher resoltion as long the GPU supports EFI GOP. Looki=
ng
> >>>>>>>>> with gpart at the USB flash drives reveals that the EFI partiti=
on
> >>>>>>>>> starts at block 1 of the device and the device has a MBR layout=
. I
> >>>>>>>>> haven't found a way to force the GPT scheme, when initialised v=
ia
> >>>>>>>>> gpart, to let the partitions start at block 1. This might be a =
naiv
> >>>>>>>>> thinking, so please be patient with me.
> >>>>>>>>>=20
> >>>>>>>>> I do not know whether this is a well-known issue. On the ASRock
> >>>>>>>>> boards, I tried years ago some LinuxRed Hat and Suse with UEFI =
and
> >>>>>>>>> that worked - FreeBSD not. I gave up on that that time. Now, ha=
ving
> >>>>>>>>> the very same issues with a new Fujitsu system, leaves me with =
the
> >>>>>>>>> impression that FreeBSD's UEFI implementation might have proble=
ms
> >>>>>>>>> I'm not aware of.
> >>>>>>>>>=20
> >>>>>>>>> Can someone shed some light onto this?=20
> >>>>>>>>> =20
> >>>>>>>>=20
> >>>>>>>> The first thing to check is if the secure boot is disabled. We d=
o not
> >>>>>>>> support secure boot at all at this time. =20
> >>>>>>>=20
> >>>>>>> Secure boot is in every scenario disabled!
> >>>>>>> =20
> >>>>>>>>=20
> >>>>>>>> If you have efi or bios version running - you can check from eit=
her
> >>>>>>>> console variable value (it can have efi or vidconsole or comcons=
ole)
> >>>>>>>> or better yet, see if efi-version is set (show efi-version) - if
> >>>>>>>> efi-version is not set, it is BIOS loader running. Another indir=
ect
> >>>>>>>> way is to see lsdev -v, with device paths present, it is
> >>>>>>>> uefi:) =20
> >>>>>>>=20
> >>>>>>> What are you talking about?
> >>>>>>> What is the point of entry - running system, loader?
> >>>>>>>=20
> >>>>>>> sysct machdep.bootmethod: BIOS
> >>>>>>>=20
> >>>>>>> This makes me quite sure that the system has booted via BIOS - as=
I'm
> >>>>>>> sure since I've checked that many times. UEFI doesn't work on tho=
se
> >>>>>>> systems with FreeBSD. I'm not sure antmore, but I tried also Wind=
ows 7
> >>>>>>> on those mainboards booting via UEFI - and I might recall that th=
ey
> >>>>>>> failed also. I also recall that there were issues with earlier UE=
FI
> >>>>>>> versions regarding booting only Windows 8/8.1 - and nothing else,=
but
> >>>>>>> the fact that Linux worked confuses me a bit.
> >>>>>>>=20
> >>>>>>> If this ASRock crap (never ever again this brand!) doesn't work at
> >>>>>>> all - who cares, I intend to purchase new server grade hardware. =
But
> >>>>>>> the more puzzling issue is with the Fujitsu, which I consider ser=
ious
> >>>>>>> and from the behaviour the Fujitsu failure looks exactly like the
> >>>>>>> ASRock - Windows 7 works, RedHat 7.5 works (I assume I can trust =
the
> >>>>>>> Firmware settings when I disable CSM support, that the Firmware w=
ill
> >>>>>>> only EFI/UEFI capable loader? Or is there a ghosty override somwh=
ere
> >>>>>>> to be expected?). Also on ASRock disabling CSM should ensure not
> >>>>>>> booting a dual-bootstrap-capable system. This said, on the recent
> >>>>>>> Fujitsu, it seems to boil down to a FreeBSD UEFI-firmware interac=
tion
> >>>>>>> problem, while the ASRock is still under suspicion to be broken by
> >>>>>>> design. =20
> >>>>>>>>=20
> >>>>>>>> GPT partitions can never start from disk absolute sector 1; this=
is
> >>>>>>>> because at sector 0 there is MBR (for compatibility), sector 1 i=
s GPT
> >>>>>>>> table and then sectors 2-33 have GPT partition table entries, so=
the
> >>>>>>>> first possible data sector is 34 (absolute 34). Thats assuming 5=
12B
> >>>>>>>> sectors. For details see UEFI 2.7 Chapter 5.3.1 page 131. =
=20
> >>>>>>>=20
> >>>>>>> Thanks for the explanation. That implies the installer did right,
> >>>>>>> gpart did also right and therefore there must be an issue with the
> >>>>>>> stuff located within the EFI partition? =20
> >>>>>>=20
> >>>>>> Ok, so, it is not about UEFI bootcode but BIOS, and if we reach BI=
OS
> >>>>>> loader at all or not - that is, if the BIOS bootstrap is actually
> >>>>>> caring to read the MBR code and start it, since once the MBR code =
is
> >>>>>> started, it is all about our code. =20
> >>>>>=20
> >>>>> I'm getting confused a bit here. Do you mean by "BIOS" the CSM? or =
do
> >>>>> you mean that specific portion of the UEFI firmware, which looks fo=
r the
> >>>>> proper UEFI partition?
> >>>>> =20
> >>>>=20
> >>>> BIOS as either native or CSM. Note that from boot code point of view=
the
> >>>> CSM boot *is* BIOS boot, we have no access to UEFI features.
> >>>> =20
> >>>>>=20
> >>>>> The boxes in question, most notably the more recent Fujitsu Esprimo
> >>>>> Q956, refuse booting UEFI, even if properly setup (in terms of what
> >>>>> FreeBSD provides on recent CURRENT) is applied and CSM is switched =
off
> >>>>> in the firmware. Again: GPT partition scheme.
> >>>>>=20
> >>>>> The system boots properly if a second partition of type "freebsd-bo=
ot"
> >>>>> is applied and bootcode is properly applied via "gpart bootcode
> >>>>> -b /boot/pmbr -p /boot/gptboot -i 2 ada0" (ada0 is the device). =
=20
> >>>>>>=20
> >>>>>> btw, you can try to validate the installed boot blocks by using re=
cent
> >>>>>> enough loader (usb or iso) and then you can use from OK prompt: =
=20
> >>>>>=20
> >>>>> lsdev provides me with the follwoing informations (CSM enabled):
> >>>>>=20
> >>>>> OK lsdev
> >>>>> disk devices:
> >>>>> disk0: BIOS DRIVE C ...
> >>>>>=20
> >>>>> disk0p1: EFI
> >>>>> disk0p2: FreeBSD BOOT
> >>>>> disk0p3: FreeBSD SWAP
> >>>>> disk0p4: FreeBSD ZFS
> >>>>> zfs devices:
> >>>>> zfs:zroot
> >>>>>=20
> >>>>> OK chain disk0
> >>>>> open failed (so for disk0p{1-4}.
> >>>>>=20
> >>>>> OK chain zroot
> >>>>> failed to read disk (just for completeness) =20
> >>>>=20
> >>>>=20
> >>>> chain command does use only device name (such as disk0: or disk0p2: =
),
> >>>> but not zfs pool as device. I just found I haven=E2=80=99t ported t=
he code to
> >>>> read the file. =20
> >>>=20
> >>> ??
> >>> =20
> >>>>=20
> >>>> the point for chain command test is to see if the normal read and ex=
ecute
> >>>> would work, so in your case please try:
> >>>>=20
> >>>> chain disk0: =20
> >>>=20
> >>> As stated above, I did so, and the result is also mentioned above, I
> >>> always get "open failed".
> >>> This is the same for=20
> >>>=20
> >>> chain disk0
> >>> chain disk0p1
> >>> chain disk0p2
> >>> chain disk0p3
> >>> chain disk0p4
> >>>=20
> >>> as already said. CSM is enabled in this case. =20
> >>=20
> >> sigh=E2=80=A6 chain command does take device as argument, device must =
always end
> >> with colon=E2=80=A6. in this case, the devil is in details:) as I wrot=
e above, the
> >> command should be:
> >>=20
> >> chain disk0:
> >>=20
> >> The disk0p1: etc will only work when partition boot code was installed
> >> (which you most likely do not have - the only possible candidate could=
be
> >> FreeBSD ZFS partition). =20
> >=20
> > The command "chain disk0:" works as expected (CSM enabled, GPT partition
> > scheme, but with PMBR bootblock installed and freebsd-boot partition
> > conatining gptzfsboot installed.
> >=20
> > =20
> >> =20
> >>> =20
> >>>>=20
> >>>> to read pmbr (512B) and execute it. The expected outcome would be th=
at
> >>>> pmbr boot code would browse the GPT, read stage1 from disk0p2: and
> >>>> execute it; stage1 would detect FreeBSD ZFS from disk0p4: and load a=
nd
> >>>> execute /boot/loader. If that will happen, it means the boot code in=
our
> >>>> stages is just fine, but the bios (CSM) does not load pmbr=E2=80=A6.=
if thats
> >>>> true, it would mean that you either need to use UEFI boot or need to=
have
> >>>> some hack to fool the BIOS or just not use GPT on that machine with
> >>>> CSM. =20
> >>>=20
> >>> To make it clear here: The only way to boot this box is using CSM (as=
it
> >>> is the same with the ASRock boards mentioned earlier). But my intenti=
on
> >>> is to disable CSM and use a GPT/UEFI environment only! And GPT/UEFI
> >>> doesn't work with FreeBSD, neither with 12-CURRENT, nor 11.2-RELENG.
> >>>=20
> >>> It would be nice if this could be fixed. I'm more interested in the f=
ix on
> >>> the recent Fujitsu device than the outdated ASRock crap, but if the f=
ix
> >>> for the Fujitsu Firmware could fix older issues as a byproduct, I'd
> >>> appreciate that.
> >>>=20
> >>> Kind regards,
> >>> =20
> >>=20
> >> ok, somehow I have lost that part of the discussion. Well, you wrote t=
hat
> >> the UEFI boot fails when the first partition starts from sector 40 - d=
oes
> >> it mean you have boot when the partition will start from some other
> >> sector? I think, there is something else going on. =20
> >=20
> > Well, I simply try to describe what I "see" to make things disambiguous=
. I'm
> > not familiar with the deeper insights of disk layouts on a binary level=
. So,
> > you explained to me the reason, why ESP (EGI partition) starts at block=
40.
> > I compared that to the FreeBSD USB flash image FreeBSD provides, but th=
is is
> > another story since the image uses MBR scheme as I figured out.
> >=20
> > =20
> >>=20
> >> What you can do is to see if that firmware will offer you EFI shell op=
tion,
> >> from there you can try to start the bootx64.efi manually and see what =
error
> >> you will get. However, the number 1 cause for failing to start the
> >> bootloader in UEFI is secure boot - we do not support it and secure bo=
ot
> >> must be switched off.=20
> >>=20
> >> However, they seem to claim "The Secure Boot option is available in the
> >> UEFI/BIOS of most if not all ASRock boards. It is disabled by default.=
=E2=80=9D=20
> >>=20
> >> Still suggest to double check if thats really the case. Also, if the
> >> bootx64.efi start will fail and no messages are appearing on screen, t=
hen
> >> either there is something in firmware logs or you could get them from
> >> trying to start bootx64.efi from UEFI shell. =20
> >=20
> > Since I'm with this problem since 2014 and try from time to time, be au=
sred
> > that I tried every possible permutationof all reasonable options, even =
those
> > nonsense, to get rid of that problem.
> >=20
> > I never had any problems with any other UEFI capable server/workstation
> > firmware so far booting FreeBSD off in UEFI-native (GPT partition schem=
e,
> > CSM disabled) so far - until now, when I ran into this Fujitsu ESPRIMO =
Q956
> > with the most recent firmware (as of lat week, week 29 of 2018) having =
the
> > very same problems.=20
> >=20
> >=20
> >=20
> > I figured out something strange on the Fujitsu - and that is the same w=
ith
> > the ASRock boards.
> >=20
> > We/I prepare some USB flash drives to boot a NanoBSD for a very small
> > appliance, but nevertheless, the USB flash device is booted on Fujitsu
> > servers with UEFI-only configurations. I assume at this point that
> > disabling on the most recent Fujitsu firmwares on reasonable "new" hard=
ware
> > (not older than three years) will disable any(!) legacy BIOS capabiliti=
es.
> > The same is assumed for the Fujitus ESPRIMO Q956. I can not speak for t=
he
> > ASRock A77 Pro4/m boards mentioned above/earlier, they are from 2012/20=
13
> > and "quite old".
> >=20
> > The NanoBSD image of ours doesn't have a "freebsd-boot" partition. The
> > partition scheme of the flash device is GPT. The layout looks like this:
> >=20
> > gpart show -l da4 =20
> > =3D> 40 15425456 da4 GPT (7.4G) =20
> > 40 2000 1 efiboot0 (1.0M)
> > 2040 1453584 3 disk1a (710M)
> > 1455624 4096 5 disk3 (2.0M)
> > 1459720 13965776 - free - (6.7G)
> >=20
> > I created the flash with md, gpart and dd straightforward, efiboot0 is =
the
> > ESP partition and its format/content is created via dd if=3D/boot/boot1=
.efifat
> > of=3D/dev/da4p1 - I presume this is very simple.
> >=20
> > This USB flash device boots(!) successfully (UEFI!) on both the ASRock
> > boards and the Esprimo Q956!
> >=20
> > But any SSD prepared the same way doesn't. Why?=20
> >=20
> > On the ASRock, I recall having fiddled around with HDD also for a while
> > conatining Windows 7/SP1 and FreeBSD. Windows7 booted, FreeBSD - I can't
> > remember.=20
> >=20
> > In the lack of proper hardware I'm unable to check whether USB-attached=
HDD
> > or SSD will boot or HDD will boot (just in case the local SATA has prob=
lems
> > booting UEFI and USB not).
> >=20
> > Kind regards,
> >=20
> > Oliver=20
> > =20
>=20
> Am. well. I think the suggestion to test out FAT32 is still good one to t=
est.
> This is because it is known that some vendors do not support booting
> FAT12/FAT16 from HDD (the likely reason is that UEFI specification does n=
ot
> tell which FAT must be supported, and only hint about FAT12/FAT16 in cont=
ext
> of removable devices).
I prepared yesterday a GTP/ZFS-only 11.2-RELENG on the ESPRIMO Q956. It too=
k me
a time to circumvent the installer and I had to install the system manually=
. In
that strain, I also "tried" to establish the ESP with FAT32, as Allen Jude
suggested earlier. I didn't find any ad hoc help how to find out the format
(FAT12/16/32) of the ESP, so I assume when using 12-CURRENT's or 11.2-RELEN=
G's
installer with AUTO-ZFS and GPT (UEFI) (only!) the resulting ESP is FAT12 or
FAT16 (300mb by default). I also assume, that when dd'ing the /boo/boot1.ef=
ifat
image to a partition, the format is FAT, but not FAT32. Therefore, I refoma=
tted
the manually created ESP (using "gpart add -t efi ...") using "newfs_msdos =
-F
32 -b xxx ...". I had to fiddle around a bit with option -b to fit in a pro=
per
format to meet a 512mb ESP - I'm not sure whether this is the proper option=
to
deal with. When using the default and only -F32, the size of the partition =
has
to be 4G at least I assume. Having done that, I copied the the content of
boot1.efifat (mdconfig -t vnode ..., I guess we know the drill ...) to the
newly formatted ESP to /boot/efi/ ...
Having so far no knowledge of how to asure that the created ESP is FAT32, I
assume it is FAT32.
The result is negative on the ESPRIMO Q956. When disabling the CSM, the box=
is
not willing to boot from SSD with the ESP prepared as decribed. So, a chance
that this might still be due to a misconfiguration lies now within the -b
option of newfs_msdos - if the -b option is assumed the proper option?
At the moment, the ESP of the Esprimo is subject to changes, if you wish, b=
ut
not in size, since it is limited to 512mb.
Thanks and kind regards,
Oliver
>=20
> There are other possible causes too, for example:
> https://ubuntuforums.org/showthread.php?t=3D2147295=20
>=20
> Also about the ESP sizes: https://www.ctrl.blog/entry/esp-size-guide
>=20
> "The UEFI System Partition should be at least 260 MiB (273 MB) to ensure =
its
> properly formatted with FAT32 so that you avoid UEFI implementation
> compatibility issues. (If you do have incompatible hardware that requires
> FAT16-formatting, then I suggest you move aside the files on the UEFI Sys=
tem
> Partition, convert the partition to FAT16, and copy the files back over to
> it.)=E2=80=9D
>=20
> So, as you see, even just telling =E2=80=9Cuse FAT32=E2=80=9D is not univ=
ersal medicine, but
> I suspect it is still more universal than using FAT12/FAT16:)
>=20
> Just to be clear, there is *no* standard size rule for ESP, there are only
> suggestions from vendors=E2=80=A6=20
>=20
> Yes, this all means that if the solution from default installer does not
> work, the manual work is needed to identify why the default is not working
> and the findings should be reported, so the installer (and possibly other
> parts of the system) could be adjusted. Since this all is vendor specific=
, it
> has to be handled case by case.
>=20
> rgds,
> toomas
>=20
>=20
>=20
>=20
> _______________________________________________
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180725111032.1632b885>