From owner-freebsd-fs@FreeBSD.ORG Sun May 18 01:51:48 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0300C20E for ; Sun, 18 May 2014 01:51:48 +0000 (UTC) Received: from mailbox.supranet.net (mailbox.supranet.net [IPv6:2607:f4e0:100:111::9]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D83082C87 for ; Sun, 18 May 2014 01:51:47 +0000 (UTC) Received: from [209.204.169.179] (helo=[192.168.1.201]) by mailbox.supranet.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1WlqGP-0008UG-6g; Sat, 17 May 2014 20:51:45 -0500 Date: Sat, 17 May 2014 18:51:54 -0700 From: Jeff Chan Reply-To: Jeff Chan X-Priority: 3 (Normal) Message-ID: <1129127016.20140517185154@supranet.net> To: freebsd-fs@freebsd.org Subject: ZFS snapshot restore not quite working; missing steps? MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 01:51:48 -0000 We're trying to do a ZFS snapshot restore of a ZFS non-RAIDZ* FreeBSD 9.2-RELEASE to a new bare system configured with ZFS RAIDZ, and it's not quite working right. The restore itself seems to complete, but we're not able to successfully boot the resulting system. We can't recall the exact error we got, but think the ZFS cache file was not found. Presumably we're missing some steps, don't have all magic for ZFS booting from a FreeBSD root partition installed/configured correctly, etc. What did we miss? (We have a private network to serve the snapshot over NFS from a third server on 192.168.0.2.) (There may be some line wrap below.) (Comments in (parens)) (Some outputs in [brackets]) On the old system, call it foo: foo: [103]% zfs list NAME USED AVAIL REFER MOUNTPOINT zroot 169G 3.40T 545M legacy zroot/home 15.1G 3.40T 10.3G /home zroot/home/username 4.04G 3.40T 2.74G /home/username zroot/bar 128G 3.40T 97K /bar zroot/bar/prod 126G 3.40T 73.8G /bar/prod zroot/bar/test 2.18G 3.40T 2.18G /bar/test zroot/tmp 1.35G 3.40T 1.35G /tmp zroot/usr 9.60G 3.40T 9.52G /usr zroot/var 14.0G 3.40T 14.0G /var foo: [104]% zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT zroot 3.62T 170G 3.46T 4% 1.00x ONLINE - On the new system: Boot from a FreeBSD 9.2-RELEASE installer USB flash drive into LiveCD, then: (bring up NFS private LAN) ifconfig ix3 192.168.0.3 netmask 255.255.255.0 up [ix2: Could not setup receive structures [ix2: Could not setup receive structures -> sysctl kern.ipc.nmbclusters=3D131072 sysctl kern.ipc.nmbjumbo9=3D38400 (mount remote NFS backup directory) mount -t nfs 192.168.0.2:/home/backup /mnt mkdir /var/mnt sysctl kern.disks [kern.disks: da0 mfid2 mfid1 mfid0] gpart create -s gpt mfid0 gpart create -s gpt mfid1 gpart create -s gpt mfid2 gpart add -s 222 -a 4k -t freebsd-boot -l boot0 mfid0 gpart add -s 222 -a 4k -t freebsd-boot -l boot1 mfid1 gpart add -s 222 -a 4k -t freebsd-boot -l boot2 mfid2 gpart add -s 8g -a 4k -t freebsd-swap -l swap0 mfid0 gpart add -s 8g -a 4k -t freebsd-swap -l swap1 mfid1 gpart add -s 8g -a 4k -t freebsd-swap -l swap2 mfid2 gpart add -a 4k -t freebsd-zfs -l disk0 mfid0 gpart add -a 4k -t freebsd-zfs -l disk1 mfid1 gpart add -a 4k -t freebsd-zfs -l disk2 mfid2 (Clear any old zfs data) dd if=3D/dev/zero of=3D/dev/mfid0p3 count=3D560 bs=3D512 dd if=3D/dev/zero of=3D/dev/mfid1p3 count=3D560 bs=3D512 dd if=3D/dev/zero of=3D/dev/mfid2p3 count=3D560 bs=3D512 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 mfid0 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 mfid1 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 mfid2 zpool create -f -m none -o altroot=3D/var/mnt -o cachefile=3D/tmp/zpool.cac= he zroot raidz mfid0 mfid1 mfid2 gunzip -c /mnt/foo-backup-full/zroot@20140516183425.gz | zfs receive -vdFu = zroot zpool set bootfs=3Dzroot zroot zpool get all zroot zfs get all zroot zfs set mountpoint=3D/var/mnt zroot (add swap to fstab) vi /var/mnt/var/mnt/etc/fstab [/dev/mfid0p2 none swap sw 0 0 [/dev/mfid1p2 none swap sw 0 0 [/dev/mfid2p2 none swap sw 0 0 vi /var/mnt/var/mnt/etc/rc.conf [ change adapter name & IP ] vi /var/mnt/var/mnt/etc/pf.conf [ change adapter name ] [vi /var/mnt/var/mnt/boot/loader.conf] (not performed) cd / zpool export zroot zpool import -o altroot=3D/var/mnt -o cachefile=3D/tmp/zpool.cache zroot [zfs set mountpoint=3D/ zroot] (not performed) cp /tmp/zpool.cache /var/mnt/boot/zfs zpool get all zroot zpool set cachefile=3D/boot/zfs/zpool.cache zroot zfs unmount -a zfs set mountpoint=3Dlegacy zroot zfs set mountpoint=3D/home zroot/home zfs set mountpoint=3D/bar zroot/bar zfs set mountpoint=3D/tmp zroot/tmp zfs set mountpoint=3D/usr zroot/usr zfs set mountpoint=3D/var zroot/var [remove USB] reboot Any hints at what we missed/goofed? Cheers, Jeff C. -- Jeff Chan mailto:jeffc@supranet.net