From owner-freebsd-current@freebsd.org Sun Jan 24 13:35:07 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5BD1D7CB7 for ; Sun, 24 Jan 2016 13:35:07 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from dec.sakura.ne.jp (dec.sakura.ne.jp [210.188.226.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2502A5F8 for ; Sun, 24 Jan 2016 13:35:06 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from fortune.joker.local (123-48-23-227.dz.commufa.jp [123.48.23.227]) (authenticated bits=0) by dec.sakura.ne.jp (8.15.2/8.15.2/[SAKURA-WEB]/20080708) with ESMTPA id u0OCr1j0079967 for ; Sun, 24 Jan 2016 21:53:02 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Sun, 24 Jan 2016 21:53:00 +0900 From: Tomoaki AOKI To: freebsd-current@freebsd.org Subject: Re: ZFSROOT UEFI boot Message-Id: <20160124215300.4cd7f1207f5a4c7b28ef7ffc@dec.sakura.ne.jp> In-Reply-To: <8991747525093115430@unknownmsgid> References: <8991747525093115430@unknownmsgid> Organization: Junchoon corps X-Mailer: Sylpheed 3.4.3 (GTK+ 2.24.29; amd64-portbld-freebsd10.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Jan 2016 13:35:07 -0000 Unfortunately, this (and its committed successor and original for UFS) fails to boot in some situation, like below. OTOH, gptzfsboot (and maybe gptboot for UFS, too) is OK. When I select Disk1 from UEFI firmware, bootx64.efi in Disk1 EFI partition is used and it searches /boot/loader.efi from Disk2 (in ZFS, if none, in UFS) only. And when I select Disk2, bootx64.efi in Disk2 EFI partition is used and it searches /boot/loader.efi from Disk1 only. In fact, this is a long-standing and living problem. At past, USB memstick with head memstick.img (UEFI enabled, but without root-on-ZFS support) booted fine, but after I added UFS2 partition in internal disk, the USB memstick didn't boot anymore. It searches /boot/loader.efi from internal UFS and fails as it was blank (only newfs'ed) at that time. Another USB memstick with stable/10 memstick.img is still fine, as it's still ancient BIOS based. Possibly, it's not a fault of boot1.efi but caused by differense in implementation of UEFI firmware. If that's it, different boot1.efi would be needed for each implementation. A bit more details of tests are as below. Not all combinations are covered, but would be sufficient to determine above conclusion. Common configurations for all tests: *Each disk has one EFI partition (p1), one freebsd-boot partition (p2), one swap partition (p3), one UFS partition (p4), and one ZFS pool (p5) with this order. *Each partition has different GEOM label. *In each disk, FreeBSD is installed as root on ZFS. No other OS. *stable/10 (r294614) is installed in Disk1. *head (r294567) is installed in Disk2. *ZFS-enabled boot1.efi (head r294567) is used as bootx64.efi. Set 1: Boot from Disk1 (select it in UEFI firmware). In all tests, /boot/loader.efi in Disk1 (both UFS and ZFS) are NOT searched at all. 1-1) Both UFS and ZFS has no /boot/loader.efi -> Fail to boot. Fall back to boot1 prompt. 1-2) Disk2 UFS only has /boot/loader.efi, whole /boot of Disk2 ZFS is copied to UFS. -> head in Disk2 boots fine. 1-3) Same as 1-2, except its /boot/loader.efi is overwritten by the one of stable/10. -> head in Disk2 boots fine, as loader.efi loads kernel from /boot/kernel/kernel in UFS and kernel with zfs.ko can mount root on ZFS specified by vfs.root.mountfrom. 1-4) Disk2 UFS only has /boot/loader.efi, whole /boot of Disk1 ZFS is copied to UFS and its /boot/loader.efi is overwritten by the one of head. -> stable/10 in Disk1 ZFS boots fine. 1-5) Disk2 ZFS only has /boot/loader.efi. -> head in Disk2 ZFS boots fine. 1-6) Both UFS and ZFS in Disk2 has /boot/loader.efi. (Mix of 1-4 and 1-5) -> head in Disk2 ZFS boots fine. Set 2: Boot from Disk2 (select it in UEFI firmware). In all tests, /boot/loader.efi in Disk2 (both UFS and ZFS) are NOT searched at all. 2-1) Both UFS and ZFS has no /boot/loader.efi -> Fail to boot. Fall back to boot1 prompt. ZFS pool in Disk2 is shown before one in Disk1. 2-2) Disk1 UFS only has /boot/loader.efi, whole /boot of Disk2 ZFS is copied to UFS. -> head in Disk2 ZFS boots fine. 2-3) Disk1 UFS only has /boot/loader.efi, whole /boot of Disk1 ZFS is copied to UFS. -> stable/10 in Disk1 ZFS boots fine, as loader.efi loads kernel from /boot/kernel/kernel in UFS and kernel with zfs.ko can mount root on ZFS specified by vfs.root.mountfrom. 2-4) Disk1 UFS only has /boot/loader.efi, whole /boot of Disk1 ZFS is copied to UFS and its /boot/loader.efi is overwritten by the one of head. -> stable/10 in Disk1 ZFS boots fine. 2-5) Disk1 ZFS only has /boot/loader.efi of stable/10 itself. -> Fail to boot. Fall back to boot1 prompt. ZFS pool in Disk2 is shown before one in Disk1. 2-6) Disk1 ZFS only has /boot/loader.efi of head. -> stable/10 in Disk1 ZFS boots fine. 2-7) Both UFS and ZFS in Disk1 has /boot/loader.efi of head. (Mix of 2-2 and 2-6) -> stable/10 in Disk1 ZFS boots fine. 2-8) UFS has /boot/loader.efi of head (head kernel copied), but ZFS has /boot/loader.efi of stable/10 itself. (Mix of 2-2 and 2-5) -> Same as 2-5. Fail to boot. Fall back to boot1 prompt. ZFS pool in Disk2 is shown before one in Disk1. Set 3: Disk2 is removed. (Disk1 only environment) 3-1) ZFS only has /boot/loader.efi of head. -> stable/10 in Disk1 ZFS boots fine. 3-2) Same as 2-2 without Disk2. -> Fail to boot. Fall back to loader prompt. (Of course. Specified root device doesn't exists.) 3-3) Same as 2-4 without Disk2. -> stable/10 in Disk1 ZFS boots fine. 3-4) Both UFS and ZFS have /boot/loader.efi of head. -> stable/10 in Disk1 ZFS boots fine. Regards. On Thu, 10 Dec 2015 12:41:15 +0000 Steven Hartland wrote: > Ive literally just got this working on 10.2 after working on the code > posted on the review which you can find here: > https://reviews.freebsd.org/D4104 > > If you're happy running current then the patch file I linked in my comment > should apply cleanly and just work. > > If you want 10.x then there's quite a bit more needed. As I said I do have > this working so can post patches when I'm back in the office. > > Either way once applied a standard efi install just works. Essentially > create efi partition and use gpart to install the efi bootcode and away you > go. > > I've just used this with a custom mfsbsd iso to perform and 10.2-RELEASE > ZFS boot install on some Intel nvme disks setup as raidz2, which only > support efi boot. > > On 10 Dec 2015, at 12:18, krad wrote: > > Hi, I need to get one of my machines converted over from bios GPT zfsroot > boot to efi. I know you can boot freebsd under EFI with a ufs kernel but > this isnt the route i want. There are patches under test for EFI zfs root. > However when I read the thread it was unclear which version of these > patches were needed and where to get them. Does anyone know where they are, > if there are any prebuilt zfsloader etc binaries, or if the patches have > made it to head yet? > > Also does anyone have any pointers or good experience with grub efi and zfs > on root? I'm considering this option as it would make booting into specific > boot environments easier > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > -- Tomoaki AOKI junchoon@dec.sakura.ne.jp