Date: Sat, 23 Jan 2016 13:54:48 +0000 From: bugzilla-noreply@freebsd.org To: freebsd-bugs@FreeBSD.org Subject: [Bug 206521] Can't decrypt disks on ZFS+Geli installation after order of devices changed Message-ID: <bug-206521-8@https.bugs.freebsd.org/bugzilla/>
next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206521 Bug ID: 206521 Summary: Can't decrypt disks on ZFS+Geli installation after order of devices changed Product: Base System Version: 10.2-RELEASE Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: bin Assignee: freebsd-bugs@FreeBSD.org Reporter: florian.ermisch@alumni.tu-berlin.de (Rendered Markdown here: https://gist.github.com/0xf10e/ddefc6fad77d6b51672= f) ## Preface I added a 256GB mSATA SSD to my Lenovo x220 of which I planned to use a nice chunk as L2ARC for the SATA HDD. Even without a SATA disk=20 present the SSD showed up as HDD2 in the BIOS. Made a fresh installation of FreeBSD 10.2 amd64 with the memstick image and chose ZFS+Geli.=20 Booted, worked, everything just fine. I added a=20 SATA HDD which the BIOS listed as HDD0 just as=20 I expected from the previous SATA-only setup. When I now tried to boot from the SSD I was=20 prompted for the GELI key of a different device than before, `ada1p3` instead of `ada0p3`. But my passphrase wasn't accepted. Never. Re-installed, added HDD, same result.=20 Reinstalled with HDD present, the removed the=20 HDD, same result (then prompted for `ada0p3`=20 instead of `ada1p3`, of course). When I restored the configuration which was present during the installation process (either remove the HDD or add it back in)=20 everything worked fine again. ## Using bhyve to reproduce Created a VM, added more disks: ``` floh@fuchi-cyber220:~:1305% iohyve info Name Size RAM CPU OS Loader fbsd102 - 2G 1 default bhyveload fbsd102/disk0 2G 2G 1 default bhyveload fbsd102/disk1 2G 2G 1 default bhyveload fbsd102/disk2 4G 2G 1 default bhyveload ``` Install FreeBSD 10.2 (the one I used for my laptop): ``` floh@fuchi-cyber220:~:1305% sudo iohyve install fbsd102 FreeBSD-10.2-RELEASE-amd64-disc1.iso Installing fbsd102... floh@fuchi-cyber220:~:1307% sudo iohyve console fbsd102 Starting console on fbsd102... ~~. to escape console [uses cu(1) for console] Connected ``` * Chose Auto (ZFS), added only ada2/disk2 to the rootpool. ``` =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80ZFS Configuration=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=90 =E2=94=82 Configure Options: =E2=94=82 =E2=94=82 =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=90 =E2=94=82 =E2=94=82 =E2=94=82 >>> Install Proceed with Installation =E2=94= =82 =E2=94=82 =E2=94=82 =E2=94=82 T Pool Type/Disks: stripe: 1 disk =E2=94= =82 =E2=94=82 =E2=94=82 =E2=94=82 - Rescan Devices * =E2=94= =82 =E2=94=82 =E2=94=82 =E2=94=82 - Disk Info * =E2=94= =82 =E2=94=82 =E2=94=82 =E2=94=82 N Pool Name zroot =E2=94= =82 =E2=94=82 =E2=94=82 =E2=94=82 4 Force 4K Sectors? YES =E2=94= =82 =E2=94=82 =E2=94=82 =E2=94=82 E Encrypt Disks? YES =E2=94= =82 =E2=94=82 =E2=94=82 =E2=94=82 P Partition Scheme GPT =E2=94= =82 =E2=94=82 =E2=94=82 =E2=94=82 S Swap Size 256m =E2=94= =82 =E2=94=82 =E2=94=82 =E2=94=82 M Mirror Swap? NO =E2=94= =82 =E2=94=82 =E2=94=82 =E2=94=82 W Encrypt Swap? YES =E2=94= =82 =E2=94=82 =E2=94=82 =E2=94=94=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=98 =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=A4 =E2=94=82 <Select> <Cancel> =E2=94=82 =E2=94=94=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=98 ``` * proceed with installation. * remove disk1 from the VM: ``` floh@fuchi-cyber220:~:1313% sudo iohyve remove fbsd102 disk1 Are you sure you want to remove disk1 from fbsd102 [Y/N]? y floh@fuchi-cyber220:~:1314% iohyve info Name Size RAM CPU OS Loader fbsd102 - 2G 1 default bhyveload fbsd102/disk0 2G 2G 1 default bhyveload fbsd102/disk2 4G 2G 1 default bhyveload ``` * shuffle disks around: ``` floh@fuchi-cyber220:~:1340% sudo zfs rename zroot/iohyve/fbsd102/disk{0,1} floh@fuchi-cyber220:~:1342% sudo zfs rename zroot/iohyve/fbsd102/disk{2,0} floh@fuchi-cyber220:~:1342% sudo zfs rename zroot/iohyve/fbsd102/disk{1,2} ``` * now the 4GB one is `disk0`: ``` floh@fuchi-cyber220:~:1343% iohyve info Name Size RAM CPU OS Loader fbsd102 - 2G 1 default bhyveload fbsd102/disk0 4G 2G 1 default bhyveload fbsd102/disk2 2G 2G 1 default bhyveload ``` * boot the VM and try to get the rootpool imported: ``` floh@fuchi-cyber220:~:1344% sudo iohyve start fbsd102 Starting fbsd102... (Takes 15 seconds for FreeBSD guests) floh@fuchi-cyber220:~:1344% floh@fuchi-cyber220:~:1345% sudo iohyve console fbsd102 Starting console on fbsd102... ~~. to escape console [uses cu(1) for console] Connected / ______ ____ _____ _____ | ____| | _ \ / ____| __ \ | |___ _ __ ___ ___ | |_) | (___ | | | | | ___| '__/ _ \/ _ \| _ < \___ \| | | | | | | | | __/ __/| |_) |____) | |__| | | | | | | | || | | | |_| |_| \___|\___||____/|_____/|_____/ ``` ` s` `.....---.......--.``` -/ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3DWelcome to FreeBSD=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D+ +o .--` /y:` +. | | yo`:. :o `+- | 1. Boot Multi User [Enter] | y/ -/` -o/ | 2. Boot [S]ingle User | .- ::/sy+:. | 3. [Esc]ape to loader prompt | / `-- / | 4. Reboot | `: :` | | `: :` | Options: | / / | 5. [K]ernel: kernel (1 of 2) | .- -. | 6. Configure Boot [O]ptions... | -- -. | | `:` `:` | | .-- `--. | | .---.....----. +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ /boot/kernel/kernel text=3D0xfc8de8 data=3D0x1283b0+0x207880 syms=3D[0x8+0x145350+0x8+0x15fe20] /boot/kernel/zfs.ko size 0x2f9b00 at 0x199e000 loading required module 'opensolaris' /boot/kernel/opensolaris.ko size 0x6048 at 0x1c98000 /boot/kernel/geom_eli.ko size 0x21568 at 0x1c9f000 loading required module 'crypto' /boot/kernel/crypto.ko size 0x35318 at 0x1cc1000 /boot/encryption.key size=3D0x1000 /boot/kernel/aesni.ko size 0x5a30 at 0x1cf8000 /boot/zfs/zpool.cache size=3D0x8bc Booting... Copyright (c) 1992-2015 The FreeBSD Project. [... lots kernel messages ...] ada1: <BHYVE SATA DISK 001> ACS-2 ATA SATA 3.x device ada1: Serial Number BHYVE-D12E-A75C-27F5 ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 2048MB (4194304 512 byte sectors: 16H 63S/T 4161C) ada1: Previously was known as ad6 random: unblocking device. Timecounter "TSC-low" frequency 1345403547 Hz quality 1000 Enter passphrase for ada0p4: GEOM_ELI: Wrong key for ada0p4. Tries left: 2. Enter passphrase for ada0p4: GEOM_ELI: Wrong key for ada0p4. Tries left: 1. Enter passphrase for ada0p4: GEOM_ELI: Wrong key for ada0p4. No tries left. Enter passphrase for diskid/DISK-BHYVE-F485-E5AF-7C59p4: GEOM_ELI: Wrong key for diskid/DISK-BHYVE-F485-E5AF-7C59p4. Tries left: 2. Enter passphrase for diskid/DISK-BHYVE-F485-E5AF-7C59p4: GEOM_ELI: Wrong key for diskid/DISK-BHYVE-F485-E5AF-7C59p4. Tries left: 1. Enter passphrase for diskid/DISK-BHYVE-F485-E5AF-7C59p4: GEOM_ELI: Wrong key for diskid/DISK-BHYVE-F485-E5AF-7C59p4. No tries left. Enter passphrase for gpt/zfs0: GEOM_ELI: Wrong key for gpt/zfs0. Tries left: 2. Enter passphrase for gpt/zfs0: GEOM_ELI: Wrong key for gpt/zfs0. Tries left: 1. Enter passphrase for gpt/zfs0: GEOM_ELI: Wrong key for gpt/zfs0. No tries left. Trying to mount root from zfs:zroot/ROOT/default []... Mounting from zfs:zroot/ROOT/default failed with error 2. Loader variables: vfs.root.mountfrom=3Dzfs:zroot/ROOT/default Manual root filesystem specification: <fstype>:<device> [options] Mount <device> using filesystem <fstype> and with the specified (optional) option list. eg. ufs:/dev/da0s1a zfs:tank cd9660:/dev/acd0 ro (which is equivalent to: mount -t cd9660 -o ro /dev/acd0 /) ? List valid disk boot devices . Yield 1 second (for background tasks) <empty line> Abort manual input mountroot> panic: mountroot: unable to (re-)mount root. cpuid =3D 0 KDB: stack backtrace: #0 0xffffffff80984e30 at kdb_backtrace+0x60 #1 0xffffffff809489e6 at vpanic+0x126 #2 0xffffffff809488b3 at panic+0x43 #3 0xffffffff809e7f5f at vfs_mountroot+0x1eaf #4 0xffffffff808f03b3 at start_init+0x53 #5 0xffffffff8091244a at fork_exit+0x9a #6 0xffffffff80d30d2e at fork_trampoline+0xe Uptime: 4m16s ``` And, no, srsly, I don't need more than 9 tries to type "foobar". Changing the order of disks back would fix the problem. As there's no BIOS menu to choose the boot device in bhyve having no bootcode in (the original) disk0 makes this a tiny bit difficult with bhyve. --=20 You are receiving this mail because: You are the assignee for the bug.=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-206521-8>