From owner-freebsd-current@freebsd.org Sun Jan 31 11:44:06 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2AF74A741AF for ; Sun, 31 Jan 2016 11:44:06 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from dec.sakura.ne.jp (dec.sakura.ne.jp [210.188.226.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D7693C1C for ; Sun, 31 Jan 2016 11:44:05 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from fortune.joker.local (123-48-23-227.dz.commufa.jp [123.48.23.227]) (authenticated bits=0) by dec.sakura.ne.jp (8.15.2/8.15.2/[SAKURA-WEB]/20080708) with ESMTPA id u0VBhv0Y002961 for ; Sun, 31 Jan 2016 20:43:57 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Sun, 31 Jan 2016 20:43:56 +0900 From: Tomoaki AOKI To: freebsd-current@freebsd.org Subject: Re: ZFSROOT UEFI boot Message-Id: <20160131204356.f863901e7125456b5586b215@dec.sakura.ne.jp> In-Reply-To: <56AD0B22.1050704@multiplay.co.uk> References: <8991747525093115430@unknownmsgid> <20160124215300.4cd7f1207f5a4c7b28ef7ffc@dec.sakura.ne.jp> <56A51A4C.1040808@multiplay.co.uk> <20160129000344.feaf5f828e5d43d5fbbb652a@dec.sakura.ne.jp> <56AAD552.9090202@multiplay.co.uk> <20160130155755.485736c8e12868663b9ccfbc@dec.sakura.ne.jp> <56AD0B22.1050704@multiplay.co.uk> Organization: Junchoon corps X-Mailer: Sylpheed 3.5.0 (GTK+ 2.24.29; amd64-portbld-freebsd10.3) Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="Multipart=_Sun__31_Jan_2016_20_43_56_+0900_bgoZUWsUvKtBDQcK" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jan 2016 11:44:06 -0000 This is a multi-part message in MIME format. --Multipart=_Sun__31_Jan_2016_20_43_56_+0900_bgoZUWsUvKtBDQcK Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit I found Diff6 you uploaded to PHABRICATOR. So my report below is based on it. The patched boot1.efi runs as expected (== as you wrote) for me. *boot1.efi without -DEFI_DEBUG isn't tested. Needed? As I mentioned in my previous post, I have no serial console environment. So I took movie of limited tests and typed it. Which of the tests are typed: 1) Boot from ada0 removing ada1, no USB memstick attached, ZFS in ada0 has loader.efi. (Single drive config.) 2) Boot from ada0, no USB memstick attached, ZFS in ada0 has loader.efi. 3) Boot from ada1, no USB memstick attached, ZFS in ada1 has loader.efi. 4) Boot from ada1, no USB memstick attached, only UFS in ada0 has loader.efi. 5) Boot from da0 (USB memstick with memstick.img of head). UFS in da0 has loader.efi and ZFS isn't present in da0. (3 drives [2 drives + USB memstick] config.) Please see attached text for detail. Are more outputs needed? Thanks in advance! I'm looking forward to see this MFC'ed before releng/10.3 is branched. (Relies on imp@'s test?) *Will need to be in conjunction with changes after r294265 in head, as currently Diff6 doesn't apply to stable/10. Regards. On Sat, 30 Jan 2016 19:12:34 +0000 Steven Hartland wrote: > I believe, based on testing, that the from Diff 5 onwards of > https://reviews.freebsd.org/D5108 this should work as you expect it i.e. > > If boot1 is loaded from a device which has either a UFS or ZFS bootable > install then this is the device that will be used to boot. > > If said device has both then the ZFS setup will still be tried first. > > If you can test in your setup and confirm either way that would be most > appreciated. > > Regards > Steve > > On 30/01/2016 06:57, Tomoaki AOKI wrote: > > Thanks for your quick support! > > I tried your patch [Diff1] (built with head r295032 world/kernel) and > > now have good and bad news. > > > > Good news is that without USB memstick boot1.efi runs as expected. > > Great! > > > > Bad news is that when booting from USB memstick (the one I used my > > previous test, boot1.efi [bootx64.efi] and loader.efi is replaced) and > > whichever of internal disk (ada[01]) have loader.efi in its ZFS pool, > > ada[01] is booted instead of da0 (USB memstick). > > > > *If ada0 has loader.efi, always booted from ada0 (stable/10). > > *If ada0 doesn't have loader.efi and ada1 has, booted from ada1 > > (head). > > *If both ada0 and ada1 don't have loader.efi, da0 (USB memstick) is > > booted (head, installer is invoked). > > > > *Whichever ada[01] has loader.efi in their UFS or not didn't matter. > > > > These behaviour would be because ZFS thoughout all disks is tried > > before trying UFS throughout all disks, if I understood correctly. > > > > Changing boot order (ZFS to UFS per each disk, instead of each > > ZFS to each UFS) would help. > > But providing ZFS-disabled boot1.efi (boot1ufs.efi?) for installation > > media (memstick, dvd, ...) helps, too. I built ZFS-disabled boot1.efi > > and it worked fine for USB memstick for me. > > > > *`make clean && make -DMK_ZFS=no` in sys/boot/efi/boot1 didn't disabled > > ZFS module, so I must edit the definition of *boot_modules[] in > > boot1.c. I'd have been missing something. > > > > Regards. > > > > > > On Fri, 29 Jan 2016 02:58:26 +0000 > > Steven Hartland wrote: > > > >> On 28/01/2016 16:22, Doug Rabson wrote: > >>> On 28 January 2016 at 15:03, Tomoaki AOKI wrote: > >>> > >>>> It's exactly the NO GOOD point. The disk where boot1 is read from > >>>> should be where loader.efi and loader.conf are first read. > >>>> > >>> I just wanted to note that gptzfsboot and zfsboot behaves this way. Boot1 > >>> looks for loader in the pool which contains the disk that the BIOS booted. > >>> It passes through the ID of that pool to loader which uses that pool as the > >>> default for loading kernel and modules. I believe this is the correct > >>> behaviour. For gptzfsboot and zfsboot, it is possible to override by > >>> pressing space at the point where it is about to load loader. > >> I believe I understand at least some of your issue now, could you please > >> test the code on the following review to see if it fixes your issue please: > >> https://reviews.freebsd.org/D5108 > >> > >> Regards > >> Steve > >> _______________________________________________ > >> freebsd-current@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-current > >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > >> > > > > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > -- Tomoaki AOKI junchoon@dec.sakura.ne.jp --Multipart=_Sun__31_Jan_2016_20_43_56_+0900_bgoZUWsUvKtBDQcK Content-Type: text/plain; name="boot1.efi-Diff6-bootlogs.txt" Content-Disposition: attachment; filename="boot1.efi-Diff6-bootlogs.txt" Content-Transfer-Encoding: 7bit For internal disks, ZFS only have loader.efi [Except 4)]. USB memstick has no ZFS and has loader.efi in UFS. ======================================================================== Output of gpart show with USB memstick attached. Note) diskid/DISK-S21MNXAG821947P is ada1. => 34 2000409197 ada0 GPT (954G) 34 2014 - free - (1.0M) 2048 409600 1 efi (200M) 411648 1024 2 freebsd-boot (512K) 412672 1024 - free - (512K) 413696 20971520 3 freebsd-swap (10G) 21385216 10485760 4 freebsd-ufs (5.0G) 31870976 1968537600 5 freebsd-zfs (939G) 2000408576 655 - free - (328K) => 34 488397101 ada1 GPT (233G) 34 2014 - free - (1.0M) 2048 204800 1 efi (100M) 206848 1024 2 freebsd-boot (512K) 207872 1024 - free - (512K) 208896 20971520 3 freebsd-swap (10G) 21180416 16777216 4 freebsd-ufs (8.0G) 37957632 450439168 5 freebsd-zfs (215G) 488396800 335 - free - (168K) => 34 488397101 diskid/DISK-S21MNXAG821947P GPT (233G) 34 2014 - free - (1.0M) 2048 204800 1 efi (100M) 206848 1024 2 freebsd-boot (512K) 207872 1024 - free - (512K) 208896 20971520 3 freebsd-swap (10G) 21180416 16777216 4 freebsd-ufs (8.0G) 37957632 450439168 5 freebsd-zfs (215G) 488396800 335 - free - (168K) => 3 1617794 da0 GPT (1.9G) [CORRUPT] 3 1600 1 efi (800K) 1603 34 2 freebsd-boot (17K) 1637 1614112 3 freebsd-ufs (788M) 1615749 2048 4 freebsd-swap (1.0M) ======================================================================== 1) Boot from ada0 WITHOUT USB memstick and ada1 attached (single drive). >> FreeBSD EFI boot block Loader path: /boot/loader.efi Initializing modules: ZFS UFS Probing 6 block devices... probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(1) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(2) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(3) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(4) probe: * supported (preferred) probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) probe: * supported (preferred) done ZFS found the following pools: zsysS01 UFS found 1 partitions load: '/boot/loader.efi' spa: 'zsysS01', devpath: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) load: '/boot.config' spa: 'zsysS01', devpath: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) Failed to find '/boot.config' on pool 'zsysS01' (2) load: '/boot.config' spa: 'zsysS01', devpath: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) Failed to find '/boot.config' on pool 'zsysS01' (2) Starting '/boot/lodader.efi' in 5 seconds....... ======================================================================== 2) Boot from ada0 without USB memstick attached (2 drives). >> FreeBSD EFI boot block Loader path: /boot/loader.efi Initializing modules: ZFS UFS Probing 12 block devices... probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(1) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(2) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(3) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(4) probe: + supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) probe: + supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(1) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(2) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(3) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(4) probe: * supported (preferred) probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) probe: * supported (preferred) done ZFS found the following pools: zsysS02 zsysS01 UFS found 2 partitions load: '/boot/loader.efi' spa: 'zsysS01', devpath: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) load: '/boot.config' spa: 'zsysS01', devpath: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) Failed to find '/boot.config' on pool 'zsysS01' (2) load: '/boot.config' spa: 'zsysS01', devpath: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) Failed to find '/boot.config' on pool 'zsysS01' (2) Starting '/boot/lodader.efi' in 5 seconds....... ======================================================================== 3) Boot from ada1 without USB memstick attached (2 drives). >> FreeBSD EFI boot block Loader path: /boot/loader.efi Initializing modules: ZFS UFS Probing 12 block devices... probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(1) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(2) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(3) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(4) probe: + supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) probe: + supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(1) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(2) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(3) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(4) probe: * supported (preferred) probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) probe: * supported (preferred) done ZFS found the following pools: zsysS01 zsysS02 UFS found 2 partitions load: '/boot/loader.efi' spa: 'zsysS02', devpath: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) load: '/boot.config' spa: 'zsysS02', devpath: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) Failed to find '/boot.config' on pool 'zsysS02' (2) load: '/boot.config' spa: 'zsysS02', devpath: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) Failed to find '/boot.config' on pool 'zsysS02' (2) Starting '/boot/lodader.efi' in 5 seconds....... ======================================================================== 4) Boot from ada1 without USB memstick attached, and only UFS in ada0 has loader.efi (2 drives). >> FreeBSD EFI boot block Loader path: /boot/loader.efi Initializing modules: ZFS UFS Probing 12 block devices... probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(1) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(2) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(3) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(4) probe: + supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) probe: + supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(1) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(2) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(3) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(4) probe: * supported (preferred) probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) probe: * supported (preferred) done ZFS found the following pools: zsysS01 zsysS02 UFS found 2 partitions load: '/boot/loader.efi' spa: 'zsysS02', devpath: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) Failed to find '/boot/loader.efi' on pool 'zsysS02' (2) Loading '/boot/loader.efi' from pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(4) Failed to lookup '/boot/loader.efi' (file not found?) Failed to load '/boot/loader.efi' load: '/boot/loader.efi' spa: 'zsysS01', devpath: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) Failed to find '/boot/loader.efi' on pool 'zsysS01' (2) Loading '/boot/loader.efi' from pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(4) Load complete Loading '/boot.config' from pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(4) Failed to lookup '/boot.config' (file not found?) Loading '/boot.config' from pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(4) Failed to lookup '/boot.config' (file not found?) Starting '/boot/lodader.efi' in 5 seconds...... ======================================================================== 5) Boot from da0 [USB memstick] (3 drives). >> FreeBSD EFI boot block Loader path: /boot/loader.efi Initializing modules: ZFS UFS Probing 17 block devices... probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(1) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(2) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(3) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(4) probe: + supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) probe: + supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(1) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(2) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(3) probe: . not supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(4) probe: + supported probing: pciroot(0x0):pci(0x1f,0x02):msg(0x12):hd(5) probe: + supported probing: pciroot(0x0):pci(0x1d,0x00):usb(0x00,0x00):usb(0x01,0x00) probe: . not supported probing: pciroot(0x0):pci(0x1d,0x00):usb(0x00,0x00):usb(0x01,0x00):hd(1) probe: . not supported probing: pciroot(0x0):pci(0x1d,0x00):usb(0x00,0x00):usb(0x01,0x00):hd(2) vdev_read: failed dev: 0xdca13d30, id: 1, lba: 32, size: 114688, status: 2 dskread: failed dev: 0xdca13d30, id: 1, lba: 128, size: 8192, status: 2 probe: . not supported probing: pciroot(0x0):pci(0x1d,0x00):usb(0x00,0x00):usb(0x01,0x00):hd(3) probe: * supported (preferred) probing: pciroot(0x0):pci(0x1d,0x00):usb(0x00,0x00):usb(0x01,0x00):hd(4) probe: . not supported done ZFS found the following pools: zsysS01 zsysS02 UFS found 3 partitions Loading '/boot/loader.efi' from pciroot(0x0):pci(0x1d,0x00):usb(0x00,0x00):usb(0x01,0x00):hd(3) Load complete Loading '/boot.config' from pciroot(0x0):pci(0x1d,0x00):usb(0x00,0x00):usb(0x01,0x00):hd(3) Failed to lookup '/boot.config' (file not found?) Loading '/boot.config' from pciroot(0x0):pci(0x1d,0x00):usb(0x00,0x00):usb(0x01,0x00):hd(3) Failed to lookup '/boot.config' (file not found?) Starting '/boot/lodader.efi' in 5 seconds...... --Multipart=_Sun__31_Jan_2016_20_43_56_+0900_bgoZUWsUvKtBDQcK--