From owner-freebsd-fs@FreeBSD.ORG Wed Sep 7 04:48:04 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BB8551065670 for ; Wed, 7 Sep 2011 04:48:04 +0000 (UTC) (envelope-from peterjeremy@acm.org) Received: from mail14.syd.optusnet.com.au (mail14.syd.optusnet.com.au [211.29.132.195]) by mx1.freebsd.org (Postfix) with ESMTP id 4EB498FC08 for ; Wed, 7 Sep 2011 04:48:04 +0000 (UTC) Received: from server.vk2pj.dyndns.org (c220-239-116-103.belrs4.nsw.optusnet.com.au [220.239.116.103]) by mail14.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p874m1dY004603 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 7 Sep 2011 14:48:02 +1000 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.4/8.14.4) with ESMTP id p874m0Q3030235; Wed, 7 Sep 2011 14:48:00 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.4/8.14.4/Submit) id p874m0QC030234; Wed, 7 Sep 2011 14:48:00 +1000 (EST) (envelope-from peter) Date: Wed, 7 Sep 2011 14:48:00 +1000 From: Peter Jeremy To: freebsd-fs@freebsd.org Message-ID: <20110907044800.GA96277@server.vk2pj.dyndns.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="h31gzZEtNLTqOjlF" Content-Disposition: inline X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.21 (2010-09-15) Cc: avg@freebsd.org Subject: "can't load 'kernel'" on ZFS root X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Sep 2011 04:48:04 -0000 --h31gzZEtNLTqOjlF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I have a several systems with ZFS on mirrored root and have run into problems during recent 8-stable upgrades with loader reporting it was unable to load the kernel. On the first system, I did an installkernel, rebooted and all came up OK. I then did an installworld, rebooted and got (re-typed): Loading /boot/defaults/loader.conf Unable to load a kernel! | can't load 'kernel' and the loader prompt. After some research, I found bin/144214 which suggested my bootblocks were out of date. I booted from another system and installed new bootblocks on both disks, rebooted and got the same result. I then tried backing up and restoring all the files in /boot that were touched by installworld. This time when I tried to boot, I was greeted with "ZFS i/o error - all block copies unavailable" followed by the bootblock prompt. I tried loading zroot:/boot/zfsloader.old without success. At this point, I gave up and rolled back to a snapshot prior to the installworld (but after the installkernel) and everything worked. I have since verified that the bootblocks match the newly built pmbr and gptzfsboot and that the changes in r208892 are present in the sourcetree. On the second system, I was unable to boot after the installkernel and rolled back - I haven't checked how old the bootblocks are there. Firstly, does anyone have any ideas on how to move forward? I will rebuild both zroots at some point but that's not a fix. (I am reasonably confident I can reproduce the problem if someone has a patch to test). Secondly, I found that ZFS booting is extremely fragile - if your default kernel & modules won't load automatically, there's no way to boot the backup kernel because (as far as I can find) there's no way to manually load /boot/zfs/zpool.cache - without which the kernel can't mount the root FS. In both above cases, recovering the system required booting from recovery media. There needs to be a documented method for booting from a snapshot or clone. --=20 Peter Jeremy --h31gzZEtNLTqOjlF Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk5m94AACgkQ/opHv/APuIeABgCfY1TwefWE71QlhsEaZjjF3klL 9KkAn15FkjhfVQscdgCfvldV3Cu8rXHy =rBSz -----END PGP SIGNATURE----- --h31gzZEtNLTqOjlF--