From owner-freebsd-bugs@FreeBSD.ORG Tue Feb 7 17:10:15 2012 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 747601065672 for ; Tue, 7 Feb 2012 17:10:15 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 547F48FC2A for ; Tue, 7 Feb 2012 17:10:15 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q17HAFDU016122 for ; Tue, 7 Feb 2012 17:10:15 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q17HAFP1016121; Tue, 7 Feb 2012 17:10:15 GMT (envelope-from gnats) Resent-Date: Tue, 7 Feb 2012 17:10:15 GMT Resent-Message-Id: <201202071710.q17HAFP1016121@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Chris Jones Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8DED4106566B for ; Tue, 7 Feb 2012 17:09:00 +0000 (UTC) (envelope-from chris@cjones.org) Received: from maxwell.cjones.org (mail.cjones.org [69.146.226.138]) by mx1.freebsd.org (Postfix) with ESMTP id 74FF18FC14 for ; Tue, 7 Feb 2012 17:09:00 +0000 (UTC) Received: by maxwell.cjones.org (Postfix, from userid 1000) id AC9AD4A64D6; Tue, 7 Feb 2012 09:53:12 -0700 (MST) Message-Id: <20120207165312.AC9AD4A64D6@maxwell.cjones.org> Date: Tue, 7 Feb 2012 09:53:12 -0700 (MST) From: Chris Jones To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Cc: Subject: kern/164861: Cannot boot from zfs built in degraded mode X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Chris Jones List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Feb 2012 17:10:15 -0000 >Number: 164861 >Category: kern >Synopsis: Cannot boot from zfs built in degraded mode >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Tue Feb 07 17:10:14 UTC 2012 >Closed-Date: >Last-Modified: >Originator: Chris Jones >Release: FreeBSD 9.0-RELEASE amd64 >Organization: >Environment: System: FreeBSD dee.mt.sri.com 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan 3 07:46:30 UTC 2012 root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 >Description: If a raidz1 or mirror is built in degraded mode, it will be unbootable (using GPT). Adding the missing disk and resilvering will render it bootable, even if the replaced disk is removed again prior to boot. I've reproduced this in various combinations of raidz1 and mirror; USB flash disks and SATA drives; i386 and amd64. Upon booting, these messages are printed by the primary/secondary bootstrap loader: ZFS: can only boot from disk, mirror, raidz1, raidz2 and raidz3 vdevs ZFS: i/o error - all block copies unavailable ZFS: can't read MOS ZFS: unexpected object set type 0 ...then it tries to directly load the kernel a couple of times and gives up. >How-To-Repeat: Using a 4GB USB flash drive da0: gpart create -s gpt da0 gpart add -s 64k -t freebsd-boot -l boot0 da0 gpart add -t freebsd-zfs -l disk0 da0 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0 dd if=/dev/zero of=/usr/bigfile oseek=4G count=1 zpool create -f -o altroot=/mnt zroot raidz1 /dev/gpt/disk0 /usr/bigfile dump 0afL - / | ( cd /mnt && restore rf - ) echo 'zfs_enable="YES"' >> /mnt/etc/rc.conf echo 'zfs_load="YES"' >> /mnt/boot/loader.conf echo 'vfs.root.mountfrom="zfs:zroot"' >> /mnt/boot/loader.conf zpool export zroot zpool import -o altroot=/mnt -o cachefile=/tmp/zpool.cache zroot mv /tmp/zpool.cache /mnt/boot/zfs/zpool.cache zfs unmount -a zpool set bootfs=zroot zroot zpool set cachefile='' zroot zfs set mountpoint=legacy zroot reboot >Fix: Going back to a running system, I inserted the 4GB flash disk da0 plus another 4GB flash disk da1. Then: gpart create -s gpt da1 gpart add -s 64k -t freebsd-boot -l boot1 da1 gpart add -t freebsd-zfs -l disk1 da1 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da1 zpool replace zroot da1p2 reboot It booted just fine. Then I powered off the machine, removed da1, and powered on again. It booted just fine. >Release-Note: >Audit-Trail: >Unformatted: