From owner-freebsd-fs@freebsd.org Tue Mar 7 05:26:39 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58950D0139E for ; Tue, 7 Mar 2017 05:26:39 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B1F1215A9; Tue, 7 Mar 2017 05:26:38 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lgwl-lstewart2.corp.netflix.com (c110-22-60-167.eburwd6.vic.optusnet.com.au [110.22.60.167]) by lauren.room52.net (Postfix) with ESMTPSA id 465C87E8FF; Tue, 7 Mar 2017 16:26:28 +1100 (EST) Subject: Re: svn commit: r308089 - in head To: Andriy Gapon References: <201610291409.u9TE9WXJ020650@repo.freebsd.org> Cc: freebsd-fs@freebsd.org, Toomas Soome From: Lawrence Stewart Message-ID: Date: Tue, 7 Mar 2017 16:25:23 +1100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <201610291409.u9TE9WXJ020650@repo.freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=2.4 required=5.0 tests=DNS_FROM_AHBL_RHSBL, UNPARSEABLE_RELAY autolearn=no version=3.3.2 X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lauren.room52.net X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Mar 2017 05:26:39 -0000 Hi Andriy, On 30/10/2016 01:09, Andriy Gapon wrote: > Author: avg > Date: Sat Oct 29 14:09:32 2016 > New Revision: 308089 > URL: https://svnweb.freebsd.org/changeset/base/308089 > > Log: > zfsbootcfg: a simple tool to set next boot (one time) options for zfsboot > > (gpt)zfsboot will read one-time boot directives from a special ZFS pool > area. The area was previously described as "Boot Block Header", but > currently it is know as Pad2, marked as reserved and is zeroed out on > pool creation. The new code interprets data in this area, if any, using > the same format as boot.config. The area is immediately wiped out. > Failure to parse the directives results in a reboot right after the > cleanup. Otherwise the boot sequence proceeds as usual. > > zfsbootcfg writes zfsboot arguments specified on its command line to the > Pad2 area of a disk identified by vfs.zfs.boot.primary_pool and > vfs.zfs.boot.primary_vdev kenv variables that are set by loader during > boot. Please see the manual page for more. > > Thanks to all who reviewed, contributed and made suggestions! There are > many potential improvements to the feature, please see the review for > details. > > Reviewed by: wblock (docs) > Discussed with: jhb, tsoome > MFC after: 3 weeks > Relnotes: yes > Differential Revision: https://reviews.freebsd.org/D7612 > > Added: > head/sbin/zfsbootcfg/ > head/sbin/zfsbootcfg/Makefile (contents, props changed) > head/sbin/zfsbootcfg/zfsbootcfg.8 (contents, props changed) > head/sbin/zfsbootcfg/zfsbootcfg.c (contents, props changed) > Modified: > head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h > head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c > head/sbin/Makefile > head/sys/boot/i386/common/drv.c > head/sys/boot/i386/common/drv.h > head/sys/boot/i386/gptzfsboot/Makefile > head/sys/boot/i386/zfsboot/Makefile > head/sys/boot/i386/zfsboot/zfsboot.c > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c > head/sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h > [snip] > @@ -634,7 +712,39 @@ main(void) > primary_spa = spa; > primary_vdev = spa_get_primary_vdev(spa); > > - if (zfs_spa_init(spa) != 0 || zfs_mount(spa, 0, &zfsmount) != 0) { > + nextboot = 0; > + rc = vdev_read_pad2(primary_vdev, cmd, sizeof(cmd)); > + if (vdev_clear_pad2(primary_vdev)) > + printf("failed to clear pad2 area of primary vdev\n"); > + if (rc == 0) { > + if (*cmd) { > + /* > + * We could find an old-style ZFS Boot Block header here. > + * Simply ignore it. > + */ > + if (*(uint64_t *)cmd != 0x2f5b007b10c) { > + /* > + * Note that parse() is destructive to cmd[] and we also want > + * to honor RBX_QUIET option that could be present in cmd[]. > + */ > + nextboot = 1; > + memcpy(cmddup, cmd, sizeof(cmd)); > + if (parse()) { > + printf("failed to parse pad2 area of primary vdev\n"); > + reboot(); > + } > + if (!OPT_CHECK(RBX_QUIET)) > + printf("zfs nextboot: %s\n", cmddup); > + } > + /* Do not process this command twice */ > + *cmd = 0; > + } > + } else > + printf("failed to read pad2 area of primary vdev\n"); > + I've just taken Allan Jude's & co-conspirators' work for a spin that allows gptzfsboot to boot from a geli + ZFS partition. Everything is working amazingly well, but I see the above "failed to read pad2 area of primary vdev" message on every boot. It doesn't appear to cause any problems per se and the system boots/works fine. I assume that message is printed to signal an unexpected situation though, so figured I'd get in touch to get your thoughts. I installed the KVM-based virtual machine system manually from the live shell of: FreeBSD-12.0-CURRENT-amd64-20170301-r314495-disc1.iso The partitioning is very simple: gpart create -s gpt /dev/vtbd0 gpart add -t freebsd-boot -a 8 -b 40 -s 512k vtbd0 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 vtbd0 gpart add -t freebsd-zfs -b 2088 vtbd0 root# gpart show => 40 83886000 vtbd0 GPT (40G) 40 1024 1 freebsd-boot (512K) 1064 1024 - free - (512K) 2088 83883952 2 freebsd-zfs (40G) geli was inited/attached to vtbd0p2 and the zpool was created with command: zpool create -o altroot=/tmp/zroot -o cachefile=/tmp/zpool.cache -O checksum=skein -O compression=lz4 vtbd0p2.eli i.e. the entire pool including bootfs is using skein for checksumming and lz4 for compression. I hit another boot bug using skein previously which Toomas (CCed) fixed, and am wondering if this issue might also be related to the skein implementation. I haven't tested if the zfsbootcfg functionality works for fear that the printf is indicating a low level problem with the zpool. I can test potentially destructive things and break the pool though if that would be helpful. Any thoughts? Cheers, Lawrence