Date: Mon, 20 May 2019 19:17:31 -0700 From: David Christensen <dpchrist@holgerdanske.com> To: freebsd-questions@freebsd.org Subject: zpool import -- state: UNAVAIL, status: One or more devices are missing Message-ID: <d708f9e9-2b48-0347-61f0-61945cbd3f11@holgerdanske.com>
next in thread | raw e-mail | index | archive | help
freebsd-questions: I have a computer with FreeBSD: 2019-05-20 16:59:02 toor@dipsy ~ # freebsd-version ; uname -a 11.2-RELEASE-p9 FreeBSD dipsy 11.2-RELEASE-p9 FreeBSD 11.2-RELEASE-p9 #0: Tue Feb 5 15:30:36 UTC 2019 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 The computer had a 16 GB USB 3.0 flash drive as the system drive, with ZFS boot, GELI swap, and GELI ZFS root, all created by the FreeBSD 11.2 RELEASE amd64 installer "Auto (ZFS)". The computer has three 1.5 TB HDD's with GELI ZFS RAIDZ1: root@dipsy:~ # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT <snip> tracy_p0 4.06T 1.34T 2.72T - - 0% 33% 1.00x ONLINE - I exported the pool per "Migrating ZFS Storage Pools": https://docs.oracle.com/cd/E19253-01/819-5461/gbchy/index.html root@dipsy:~ # zpool export tracy_p0 Then I copied the raw USB system drive image to an SSD. The machine would not boot -- it appeared that the boot loader could not find the root partition. I booted the FreeBSD 11.2 installer to a rescue shell, mounted the SSD boot partition, and looked around. I found that /boot/loader.conf had some variables containing the USB device node 'da0', so I changed them to the SSD device node 'ada0': 2019-05-20 17:00:19 toor@dipsy ~ # cat /boot/loader.conf | grep ada geli_ada0s1d_keyfile0_load="YES" geli_ada0s1d_keyfile0_type="ada0s1d:geli_keyfile0" geli_ada0s1d_keyfile0_name="/boot/encryption.key" Somehow, I discovered that I also needed to move aside /boot/ssd/zpool.cache. Now the machine can boot from the SSD, but the RAIDZ1 was missing. Looking in /etc/rc.conf, I saw some variables containing the HDD device nodes 'ada0', 'ada1', and 'ada2', so I changed them to the new device nodes 'ada1, 'ada2', and 'ada3': 2019-05-20 17:22:00 toor@dipsy ~ # cat /etc/rc.conf | grep ada geli_ada1s1_flags="-k /root/raidz-geli.key -p" geli_ada2s1_flags="-k /root/raidz-geli.key -p" geli_ada3s1_flags="-k /root/raidz-geli.key -p" geli_devices="ada1s1 ada2s1 ada3s1" After rebooting again, FreeBSD could now see the all the drives, partitions, and GELI encrypted disk devices: 2019-05-20 17:15:01 toor@dipsy ~ # camcontrol devlist | grep ada <SAMSUNG SSD UM410 Series 2.5" 16GB VAM12D1Q> at scbus0 target 0 lun 0 (pass0,ada0) <ST31500341AS CC1H> at scbus2 target 0 lun 0 (pass1,ada1) <ST31500341AS CC1H> at scbus3 target 0 lun 0 (pass2,ada2) <ST31500341AS CC1H> at scbus4 target 0 lun 0 (pass3,ada3) 2019-05-20 17:15:53 toor@dipsy ~ # geli status Name Status Components ada0s1d.eli ACTIVE ada0s1d ada1s1.eli ACTIVE ada1s1 ada2s1.eli ACTIVE ada2s1 ada3s1.eli ACTIVE ada3s1 2019-05-20 17:16:08 toor@dipsy ~ # ls -1 /dev/ada* /dev/ada0 /dev/ada0s1 /dev/ada0s1a /dev/ada0s1b /dev/ada0s1d /dev/ada0s1d.eli /dev/ada1 /dev/ada1s1 /dev/ada1s1.eli /dev/ada2 /dev/ada2s1 /dev/ada2s1.eli /dev/ada3 /dev/ada3s1 /dev/ada3s1.eli But when I try to import the pool: 2019-05-20 17:21:21 toor@dipsy ~ # zpool import pool: tracy_p0 id: 6957586590476290317 state: UNAVAIL status: One or more devices are missing from the system. action: The pool cannot be imported. Attach the missing devices and try again. see: http://illumos.org/msg/ZFS-8000-3C config: tracy_p0 UNAVAIL insufficient replicas raidz1-0 UNAVAIL insufficient replicas 8533131415900712652 UNAVAIL cannot open 11863649002236717295 UNAVAIL cannot open 16410494706036382010 UNAVAIL cannot open Following the 'see' link, STFW, RTFM, etc., I am unable to determine how to attach the drives. Furthermore, if I boot the original USB system drive and try to import the RAIDZ1: 2019-05-20 18:48:43 toor@dipsy ~ # zpool import pool: tracy_p0 id: 6957586590476290317 state: UNAVAIL status: One or more devices are missing from the system. action: The pool cannot be imported. Attach the missing devices and try again. see: http://illumos.org/msg/ZFS-8000-3C config: tracy_p0 UNAVAIL insufficient replicas raidz1-0 UNAVAIL insufficient replicas 8533131415900712652 UNAVAIL cannot open 11863649002236717295 UNAVAIL cannot open 16410494706036382010 UNAVAIL cannot open Any suggestions? David
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?d708f9e9-2b48-0347-61f0-61945cbd3f11>