From owner-freebsd-current@freebsd.org Wed Jul 25 09:26:27 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DAD0510457D5 for ; Wed, 25 Jul 2018 09:26:26 +0000 (UTC) (envelope-from tsoome@me.com) Received: from st13p35im-asmtp001.me.com (st13p35im-asmtp001.me.com [17.164.199.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6D991765DC; Wed, 25 Jul 2018 09:26:26 +0000 (UTC) (envelope-from tsoome@me.com) Received: from process-dkim-sign-daemon.st13p35im-asmtp001.me.com by st13p35im-asmtp001.me.com (Oracle Communications Messaging Server 8.0.1.2.20170607 64bit (built Jun 7 2017)) id <0PCF008000IUX000@st13p35im-asmtp001.me.com>; Wed, 25 Jul 2018 09:26:19 +0000 (GMT) Received: from icloud.com ([127.0.0.1]) by st13p35im-asmtp001.me.com (Oracle Communications Messaging Server 8.0.1.2.20170607 64bit (built Jun 7 2017)) with ESMTPSA id <0PCF004JG0VPHD40@st13p35im-asmtp001.me.com>; Wed, 25 Jul 2018 09:26:18 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-07-25_02:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1015 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1807250104 Content-type: text/plain; charset=utf-8 MIME-version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: [UEFI] Boot issues on some UEFI implementations From: Toomas Soome In-reply-to: <20180725111032.1632b885@freyja.zeit4.iv.bundesimmobilien.de> Date: Wed, 25 Jul 2018 12:26:13 +0300 Cc: freebsd-current , Allan Jude Content-transfer-encoding: quoted-printable Message-id: <1E6058D2-5804-480B-B6AF-66AA02CDD7AD@me.com> References: <20180713130001.219a0a84@freyja.zeit4.iv.bundesimmobilien.de> <68505F98-E840-4148-9E48-BDB350F7431A@me.com> <20180713164447.42430301@thor.intern.walstatt.dynvpn.de> <680FBB42-75BF-427F-AA3B-6D864E83ED1F@me.com> <20180723092735.12a5d2a8@freyja.zeit4.iv.bundesimmobilien.de> <80523BE6-C035-482D-B134-23812183DE83@me.com> <20180724071514.6cb9d111@freyja.zeit4.iv.bundesimmobilien.de> <16439773-3E9A-40FF-99A1-32E97CCEE2C3@me.com> <20180725095926.7d52a15a@freyja.zeit4.iv.bundesimmobilien.de> <2FCF2BCC-05E5-466A-B1AE-90300990ED1A@me.com> <20180725111032.1632b885@freyja.zeit4.iv.bundesimmobilien.de> To: "O. Hartmann" X-Mailer: Apple Mail (2.3445.9.1) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jul 2018 09:26:27 -0000 > On 25 Jul 2018, at 12:10, O. Hartmann wrote: >=20 > On Wed, 25 Jul 2018 11:46:07 +0300 > Toomas Soome wrote: >=20 >>> On 25 Jul 2018, at 10:59, O. Hartmann = wrote: >>>=20 >>> On Tue, 24 Jul 2018 08:53:36 +0300 >>> Toomas Soome wrote: >>>=20 >>>=20 >>> Hello Toomas Soome, >>>=20 >>> I CC Allan Jude since I discovered something weird today regarding = the UEFI >>> boot capabilities of USB flash devices and SSDs. See below. >>>=20 >>>>> On 24 Jul 2018, at 08:16, O. Hartmann = wrote: >>>>>=20 >>>>> On Mon, 23 Jul 2018 10:56:04 +0300 >>>>> Toomas Soome wrote: >>>>>=20 >>>>>>> On 23 Jul 2018, at 10:27, O. Hartmann = wrote: >>>>>>>=20 >>>>>>> On Fri, 13 Jul 2018 18:44:23 +0300 >>>>>>> Toomas Soome > wrote: >>>>>>>=20 >>>>>>>>> On 13 Jul 2018, at 17:44, O. Hartmann >>>>>>>> > wrote: >>>>>>>>>=20 >>>>>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>>>>> Hash: SHA512 >>>>>>>>>=20 >>>>>>>>> Am Fri, 13 Jul 2018 14:26:51 +0300 >>>>>>>>> Toomas Soome >>>>>>>>> >> schrieb: =20 >>>>>>>>>>> On 13 Jul 2018, at 14:00, O. Hartmann = >>>>>>>>>>> wrote: >>>>>>>>>>>=20 >>>>>>>>>>> The problem is some kind of weird. I face UEFI boot problems = on GPT >>>>>>>>>>> drives where the first partition begins at block 40 of the = hdd/ssd. >>>>>>>>>>>=20 >>>>>>>>>>> I have two host in private use based on an >>>>>>>>>>> outdated ASRock Z77-Pro4-M and Z77-Pro4 mainboard = (IvyBridge, Socket >>>>>>>>>>> LGA1155). Both boards are equipted with the lates official = available >>>>>>>>>>> AMI firmware revision, dating to 2013. This is for the = Z77-Pro4-M >>>>>>>>>>> revision 2.0 (2013/7/23) and for the Z77 Pro4 revision 1.8 >>>>>>>>>>> (2013/7/17). For both boards a BETA revision for the >>>>>>>>>>> Spectre/Meltdown mitigation is available, but I didn't test = that. >>>>>>>>>>> But please read. >>>>>>>>>>>=20 >>>>>>>>>>> The third box I realised this problem is a brand new Fujitsu = Esprimo >>>>>>>>>>> Q956, also AMI firmware, at V5.0.0.11 R 1.26.0 for 3413-A1x, = date >>>>>>>>>>> 05/25/2018 (or 20180525). >>>>>>>>>>>=20 >>>>>>>>>>> Installing on any kind of HDD or SSD manually or via = bsdinstall the >>>>>>>>>>> OS using UEFI-only boot method on a GPT partitioned device = fails. >>>>>>>>>>> The ASRock boards jump immediately into the firmware, the = Fujitsu >>>>>>>>>>> offers some kind of CPU/Memory/HDD test facility. >>>>>>>>>>>=20 >>>>>>>>>>> If on both type of vendor/boards CSM is disabled and UEFI = boot only >>>>>>>>>>> is implied, the MBR partitioned FreeBSD installation USB = flash >>>>>>>>>>> device does boot in UEFI! I guess I can assume this when the = well >>>>>>>>>>> known clumsy 80x25 char console suddenly gets bright and = shiny with >>>>>>>>>>> a much higher resoltion as long the GPU supports EFI GOP. = Looking >>>>>>>>>>> with gpart at the USB flash drives reveals that the EFI = partition >>>>>>>>>>> starts at block 1 of the device and the device has a MBR = layout. I >>>>>>>>>>> haven't found a way to force the GPT scheme, when = initialised via >>>>>>>>>>> gpart, to let the partitions start at block 1. This might be = a naiv >>>>>>>>>>> thinking, so please be patient with me. >>>>>>>>>>>=20 >>>>>>>>>>> I do not know whether this is a well-known issue. On the = ASRock >>>>>>>>>>> boards, I tried years ago some LinuxRed Hat and Suse with = UEFI and >>>>>>>>>>> that worked - FreeBSD not. I gave up on that that time. Now, = having >>>>>>>>>>> the very same issues with a new Fujitsu system, leaves me = with the >>>>>>>>>>> impression that FreeBSD's UEFI implementation might have = problems >>>>>>>>>>> I'm not aware of. >>>>>>>>>>>=20 >>>>>>>>>>> Can someone shed some light onto this?=20 >>>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> The first thing to check is if the secure boot is disabled. = We do not >>>>>>>>>> support secure boot at all at this time. =20 >>>>>>>>>=20 >>>>>>>>> Secure boot is in every scenario disabled! >>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> If you have efi or bios version running - you can check from = either >>>>>>>>>> console variable value (it can have efi or vidconsole or = comconsole) >>>>>>>>>> or better yet, see if efi-version is set (show efi-version) - = if >>>>>>>>>> efi-version is not set, it is BIOS loader running. Another = indirect >>>>>>>>>> way is to see lsdev -v, with device paths present, it is >>>>>>>>>> uefi:) =20 >>>>>>>>>=20 >>>>>>>>> What are you talking about? >>>>>>>>> What is the point of entry - running system, loader? >>>>>>>>>=20 >>>>>>>>> sysct machdep.bootmethod: BIOS >>>>>>>>>=20 >>>>>>>>> This makes me quite sure that the system has booted via BIOS - = as I'm >>>>>>>>> sure since I've checked that many times. UEFI doesn't work on = those >>>>>>>>> systems with FreeBSD. I'm not sure antmore, but I tried also = Windows 7 >>>>>>>>> on those mainboards booting via UEFI - and I might recall that = they >>>>>>>>> failed also. I also recall that there were issues with earlier = UEFI >>>>>>>>> versions regarding booting only Windows 8/8.1 - and nothing = else, but >>>>>>>>> the fact that Linux worked confuses me a bit. >>>>>>>>>=20 >>>>>>>>> If this ASRock crap (never ever again this brand!) doesn't = work at >>>>>>>>> all - who cares, I intend to purchase new server grade = hardware. But >>>>>>>>> the more puzzling issue is with the Fujitsu, which I consider = serious >>>>>>>>> and from the behaviour the Fujitsu failure looks exactly like = the >>>>>>>>> ASRock - Windows 7 works, RedHat 7.5 works (I assume I can = trust the >>>>>>>>> Firmware settings when I disable CSM support, that the = Firmware will >>>>>>>>> only EFI/UEFI capable loader? Or is there a ghosty override = somwhere >>>>>>>>> to be expected?). Also on ASRock disabling CSM should ensure = not >>>>>>>>> booting a dual-bootstrap-capable system. This said, on the = recent >>>>>>>>> Fujitsu, it seems to boil down to a FreeBSD UEFI-firmware = interaction >>>>>>>>> problem, while the ASRock is still under suspicion to be = broken by >>>>>>>>> design. =20 >>>>>>>>>>=20 >>>>>>>>>> GPT partitions can never start from disk absolute sector 1; = this is >>>>>>>>>> because at sector 0 there is MBR (for compatibility), sector = 1 is GPT >>>>>>>>>> table and then sectors 2-33 have GPT partition table entries, = so the >>>>>>>>>> first possible data sector is 34 (absolute 34). Thats = assuming 512B >>>>>>>>>> sectors. For details see UEFI 2.7 Chapter 5.3.1 page 131. = =20 >>>>>>>>>=20 >>>>>>>>> Thanks for the explanation. That implies the installer did = right, >>>>>>>>> gpart did also right and therefore there must be an issue with = the >>>>>>>>> stuff located within the EFI partition? =20 >>>>>>>>=20 >>>>>>>> Ok, so, it is not about UEFI bootcode but BIOS, and if we reach = BIOS >>>>>>>> loader at all or not - that is, if the BIOS bootstrap is = actually >>>>>>>> caring to read the MBR code and start it, since once the MBR = code is >>>>>>>> started, it is all about our code. =20 >>>>>>>=20 >>>>>>> I'm getting confused a bit here. Do you mean by "BIOS" the CSM? = or do >>>>>>> you mean that specific portion of the UEFI firmware, which looks = for the >>>>>>> proper UEFI partition? >>>>>>>=20 >>>>>>=20 >>>>>> BIOS as either native or CSM. Note that from boot code point of = view the >>>>>> CSM boot *is* BIOS boot, we have no access to UEFI features. >>>>>>=20 >>>>>>>=20 >>>>>>> The boxes in question, most notably the more recent Fujitsu = Esprimo >>>>>>> Q956, refuse booting UEFI, even if properly setup (in terms of = what >>>>>>> FreeBSD provides on recent CURRENT) is applied and CSM is = switched off >>>>>>> in the firmware. Again: GPT partition scheme. >>>>>>>=20 >>>>>>> The system boots properly if a second partition of type = "freebsd-boot" >>>>>>> is applied and bootcode is properly applied via "gpart bootcode >>>>>>> -b /boot/pmbr -p /boot/gptboot -i 2 ada0" (ada0 is the device). = =20 >>>>>>>>=20 >>>>>>>> btw, you can try to validate the installed boot blocks by using = recent >>>>>>>> enough loader (usb or iso) and then you can use from OK prompt: = =20 >>>>>>>=20 >>>>>>> lsdev provides me with the follwoing informations (CSM enabled): >>>>>>>=20 >>>>>>> OK lsdev >>>>>>> disk devices: >>>>>>> disk0: BIOS DRIVE C ... >>>>>>>=20 >>>>>>> disk0p1: EFI >>>>>>> disk0p2: FreeBSD BOOT >>>>>>> disk0p3: FreeBSD SWAP >>>>>>> disk0p4: FreeBSD ZFS >>>>>>> zfs devices: >>>>>>> zfs:zroot >>>>>>>=20 >>>>>>> OK chain disk0 >>>>>>> open failed (so for disk0p{1-4}. >>>>>>>=20 >>>>>>> OK chain zroot >>>>>>> failed to read disk (just for completeness) =20 >>>>>>=20 >>>>>>=20 >>>>>> chain command does use only device name (such as disk0: or = disk0p2: ), >>>>>> but not zfs pool as device. I just found I haven=E2=80=99t = ported the code to >>>>>> read the file. =20 >>>>>=20 >>>>> ?? >>>>>=20 >>>>>>=20 >>>>>> the point for chain command test is to see if the normal read and = execute >>>>>> would work, so in your case please try: >>>>>>=20 >>>>>> chain disk0: =20 >>>>>=20 >>>>> As stated above, I did so, and the result is also mentioned above, = I >>>>> always get "open failed". >>>>> This is the same for=20 >>>>>=20 >>>>> chain disk0 >>>>> chain disk0p1 >>>>> chain disk0p2 >>>>> chain disk0p3 >>>>> chain disk0p4 >>>>>=20 >>>>> as already said. CSM is enabled in this case. =20 >>>>=20 >>>> sigh=E2=80=A6 chain command does take device as argument, device = must always end >>>> with colon=E2=80=A6. in this case, the devil is in details:) as I = wrote above, the >>>> command should be: >>>>=20 >>>> chain disk0: >>>>=20 >>>> The disk0p1: etc will only work when partition boot code was = installed >>>> (which you most likely do not have - the only possible candidate = could be >>>> FreeBSD ZFS partition). =20 >>>=20 >>> The command "chain disk0:" works as expected (CSM enabled, GPT = partition >>> scheme, but with PMBR bootblock installed and freebsd-boot partition >>> conatining gptzfsboot installed. >>>=20 >>>=20 >>>>=20 >>>>>=20 >>>>>>=20 >>>>>> to read pmbr (512B) and execute it. The expected outcome would be = that >>>>>> pmbr boot code would browse the GPT, read stage1 from disk0p2: = and >>>>>> execute it; stage1 would detect FreeBSD ZFS from disk0p4: and = load and >>>>>> execute /boot/loader. If that will happen, it means the boot code = in our >>>>>> stages is just fine, but the bios (CSM) does not load pmbr=E2=80=A6= . if thats >>>>>> true, it would mean that you either need to use UEFI boot or need = to have >>>>>> some hack to fool the BIOS or just not use GPT on that machine = with >>>>>> CSM. =20 >>>>>=20 >>>>> To make it clear here: The only way to boot this box is using CSM = (as it >>>>> is the same with the ASRock boards mentioned earlier). But my = intention >>>>> is to disable CSM and use a GPT/UEFI environment only! And = GPT/UEFI >>>>> doesn't work with FreeBSD, neither with 12-CURRENT, nor = 11.2-RELENG. >>>>>=20 >>>>> It would be nice if this could be fixed. I'm more interested in = the fix on >>>>> the recent Fujitsu device than the outdated ASRock crap, but if = the fix >>>>> for the Fujitsu Firmware could fix older issues as a byproduct, = I'd >>>>> appreciate that. >>>>>=20 >>>>> Kind regards, >>>>>=20 >>>>=20 >>>> ok, somehow I have lost that part of the discussion. Well, you = wrote that >>>> the UEFI boot fails when the first partition starts from sector 40 = - does >>>> it mean you have boot when the partition will start from some other >>>> sector? I think, there is something else going on. =20 >>>=20 >>> Well, I simply try to describe what I "see" to make things = disambiguous. I'm >>> not familiar with the deeper insights of disk layouts on a binary = level. So, >>> you explained to me the reason, why ESP (EGI partition) starts at = block 40. >>> I compared that to the FreeBSD USB flash image FreeBSD provides, but = this is >>> another story since the image uses MBR scheme as I figured out. >>>=20 >>>=20 >>>>=20 >>>> What you can do is to see if that firmware will offer you EFI shell = option, >>>> from there you can try to start the bootx64.efi manually and see = what error >>>> you will get. However, the number 1 cause for failing to start the >>>> bootloader in UEFI is secure boot - we do not support it and secure = boot >>>> must be switched off.=20 >>>>=20 >>>> However, they seem to claim "The Secure Boot option is available in = the >>>> UEFI/BIOS of most if not all ASRock boards. It is disabled by = default.=E2=80=9D=20 >>>>=20 >>>> Still suggest to double check if thats really the case. Also, if = the >>>> bootx64.efi start will fail and no messages are appearing on = screen, then >>>> either there is something in firmware logs or you could get them = from >>>> trying to start bootx64.efi from UEFI shell. =20 >>>=20 >>> Since I'm with this problem since 2014 and try from time to time, be = ausred >>> that I tried every possible permutationof all reasonable options, = even those >>> nonsense, to get rid of that problem. >>>=20 >>> I never had any problems with any other UEFI capable = server/workstation >>> firmware so far booting FreeBSD off in UEFI-native (GPT partition = scheme, >>> CSM disabled) so far - until now, when I ran into this Fujitsu = ESPRIMO Q956 >>> with the most recent firmware (as of lat week, week 29 of 2018) = having the >>> very same problems.=20 >>>=20 >>>=20 >>>=20 >>> I figured out something strange on the Fujitsu - and that is the = same with >>> the ASRock boards. >>>=20 >>> We/I prepare some USB flash drives to boot a NanoBSD for a very = small >>> appliance, but nevertheless, the USB flash device is booted on = Fujitsu >>> servers with UEFI-only configurations. I assume at this point that >>> disabling on the most recent Fujitsu firmwares on reasonable "new" = hardware >>> (not older than three years) will disable any(!) legacy BIOS = capabilities. >>> The same is assumed for the Fujitus ESPRIMO Q956. I can not speak = for the >>> ASRock A77 Pro4/m boards mentioned above/earlier, they are from = 2012/2013 >>> and "quite old". >>>=20 >>> The NanoBSD image of ours doesn't have a "freebsd-boot" partition. = The >>> partition scheme of the flash device is GPT. The layout looks like = this: >>>=20 >>> gpart show -l da4 =20 >>> =3D> 40 15425456 da4 GPT (7.4G) =20 >>> 40 2000 1 efiboot0 (1.0M) >>> 2040 1453584 3 disk1a (710M) >>> 1455624 4096 5 disk3 (2.0M) >>> 1459720 13965776 - free - (6.7G) >>>=20 >>> I created the flash with md, gpart and dd straightforward, efiboot0 = is the >>> ESP partition and its format/content is created via dd = if=3D/boot/boot1.efifat >>> of=3D/dev/da4p1 - I presume this is very simple. >>>=20 >>> This USB flash device boots(!) successfully (UEFI!) on both the = ASRock >>> boards and the Esprimo Q956! >>>=20 >>> But any SSD prepared the same way doesn't. Why?=20 >>>=20 >>> On the ASRock, I recall having fiddled around with HDD also for a = while >>> conatining Windows 7/SP1 and FreeBSD. Windows7 booted, FreeBSD - I = can't >>> remember.=20 >>>=20 >>> In the lack of proper hardware I'm unable to check whether = USB-attached HDD >>> or SSD will boot or HDD will boot (just in case the local SATA has = problems >>> booting UEFI and USB not). >>>=20 >>> Kind regards, >>>=20 >>> Oliver=20 >>>=20 >>=20 >> Am. well. I think the suggestion to test out FAT32 is still good one = to test. >> This is because it is known that some vendors do not support booting >> FAT12/FAT16 from HDD (the likely reason is that UEFI specification = does not >> tell which FAT must be supported, and only hint about FAT12/FAT16 in = context >> of removable devices). >=20 > I prepared yesterday a GTP/ZFS-only 11.2-RELENG on the ESPRIMO Q956. = It took me > a time to circumvent the installer and I had to install the system = manually. In > that strain, I also "tried" to establish the ESP with FAT32, as Allen = Jude > suggested earlier. I didn't find any ad hoc help how to find out the = format > (FAT12/16/32) of the ESP, so I assume when using 12-CURRENT's or = 11.2-RELENG's > installer with AUTO-ZFS and GPT (UEFI) (only!) the resulting ESP is = FAT12 or > FAT16 (300mb by default). I also assume, that when dd'ing the = /boo/boot1.efifat > image to a partition, the format is FAT, but not FAT32. Therefore, I = refomatted > the manually created ESP (using "gpart add -t efi ...") using = "newfs_msdos -F > 32 -b xxx ...". I had to fiddle around a bit with option -b to fit in = a proper > format to meet a 512mb ESP - I'm not sure whether this is the proper = option to > deal with. When using the default and only -F32, the size of the = partition has > to be 4G at least I assume. Having done that, I copied the the content = of > boot1.efifat (mdconfig -t vnode ..., I guess we know the drill ...) to = the > newly formatted ESP to /boot/efi/ ... >=20 > Having so far no knowledge of how to asure that the created ESP is = FAT32, I > assume it is FAT32. >=20 > The result is negative on the ESPRIMO Q956. When disabling the CSM, = the box is > not willing to boot from SSD with the ESP prepared as decribed. So, a = chance > that this might still be due to a misconfiguration lies now within the = -b > option of newfs_msdos - if the -b option is assumed the proper option? >=20 > At the moment, the ESP of the Esprimo is subject to changes, if you = wish, but > not in size, since it is limited to 512mb. >=20 > Thanks and kind regards, >=20 > Oliver Yea, i was hoping fstyp command would report the FAT type, but it does = not (request for feature?:) However, the more annoying idea would be to install some OS which will = boot with UEFI on this machine, then copy boot1.efi from freebsd to it = (the default program the UEFI will load is ESP:EFI/boot/bootx64.efi in = case of UEFI64 and ESP:EFI/boot/bootia32.efi for EFI32. However, we do = not support EFI32. note that boot1.efi alone will not do much but printing on screen how it = will search for freebsd, but for the purpose of the test it would = suffice - that would give us confirmed working ESP file system (since = the other os would be able to boot) and then we can confirm if boot1.efi = itself is OK. rgds, toomas