Skip site navigation (1)Skip section navigation (2) 
Date:      Wed, 01 Oct 2014 16:27:37 -0500
From:      Karl Denninger <karl@denninger.net>
To:        freebsd-stable@freebsd.org
Subject:   Encrypted (GELI) root on ZFS troubles
Message-ID:  <542C71C9.1050907@denninger.net>
next in thread | raw e-mail | index | archive | help 

[-- Attachment #1 --]
So here's the fun part of what I'm trying to do (and getting frustrated
with)

I have set up a GPT disk with the following setup:

=>       34  625142381  da2  GPT  (298G)
         34          6       - free -  (3.0K)
         40       1024    1  freebsd-boot  (512K)
       1064    4194304    2  freebsd-zfs  [bootme]  (2.0G)
    4195368  134217728    3  freebsd-swap  (64G)
  138413096  486729312    4  freebsd-zfs  (232G)
  625142408          7       - free -  (3.5K)

Then on freebsd-boot I have written the bootloaders.

The "bootme" filesystem has *only* the /boot directory copied over from
the rest of the system's root directory (that is, the kernel, loadables,
/boot/loader.conf, etc); that pool is called "zboot"

Partition 4 has the label "root0" on it, and thus shows up in /dev/gpt. 
I have initialized that with geli, set the boot option flag (that is,
prompt on boot) and created a pool called "root" on the resulting .eli
device and then put the system on that.  That's all ok.

Finally, I set the bootfs on that latter pool.  There is no bootfs set
on /zboot:

# zpool get bootfs zboot
NAME   PROPERTY  VALUE   SOURCE
zboot  bootfs    -       default

It is set on the root pool to the proper filesystem:

# zpool get bootfs root
NAME  PROPERTY  VALUE              SOURCE
root  bootfs    root/R/10.1-CLEAN  local

The problem is that when the system boots geli "finds" the raw device
(in this case /dev/da0p4), prompts for the password and attaches there
instead of in /dev/gpt.  The gpt label is missing --- and equally bad
the "root" pool does not appear to import at boot time either.

As a result the system tries to mount root from /zboot (even though it's
not been told to, and HAS been told where to mount off the root pool),
but there's no init in there (or anything else other than the boot
filesystem itself) and as a result I get an immediate panic.

If I boot off a different (working) zfs-based system the probe still
finds the "prompt during boot" flag on that gpt partition and asks for
the password on the device.  I can see the pool; zpool import shows it:

 pool: root
     id: 17719633931604198170
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

        root         ONLINE
          da2p4.eli  ONLINE

Not so good.

If I detach that the device reappears in /dev/gpt; I can then attach
geli and import the pool in either location.  Putting the cache file
from the previous imported state in the zboot/boot/zfs directory doesn't
help (nor does removing the cache file entirely)

More-interestingly if I reboot the cloned system with the root pool
imported it does come back up, even though the device is the base
(da2p4.eli) rather than in the /dev/gpt directory.

Anyone know what's going on here?  And is there a way to have geli
attach during boot-time off the /dev/gpt directory instead of on the
base device partition name?

-- 
Karl Denninger
karl@denninger.net <mailto:karl@denninger.net>
/The Market Ticker/

[-- Attachment #2 --]
0	*H
010	+0	*H
O0K030
	*H
010	UUS10UFlorida10U	Niceville10U
Cuda Systems LLC10UCuda Systems LLC CA1/0-	*H
	 customer-service@cudasystems.net0
130824190344Z
180823190344Z0[10	UUS10UFlorida10UKarl Denninger1!0	*H
	karl@denninger.net0"0
	*H
0
bi՞]MNԿawx?`)'ҴcWgR@BlWh+	u}ApdCFJVй~FOL}EW^bچYp3K&ׂ(R
lxڝ.xz?6&nsJ+1v9v/(kqĪp[vjcK%fϻe?iq]z
lyzFO'ppdX//Lw(3JIA*S#՟H[f|CGqJKooy.oEuOw$/섀$삻J9b|AP~8]D1YI<"""Y^T2iQ2b	yH)]	Ƶ0y$_N6XqMC 9՘	XgώjGTP"#nˋ"Bk100	U00	`HB0U0,	`HB
OpenSSL Generated Certificate0U|8˴d[20U#0]Af4U3x&^"408	`HB+)https://cudasystems.net:11443/revoked.crl0
	*H
gBwH]j\x`(&gW32"Uf^.^Iϱ
k!DQAg{(w/)\N'[oRW@CHO>)XrTNɘ!u`xt5(=f\-l3<@C6mnhv##1ŃbH͍_Nq
aʷ?rk$^9TIa!kh,D-ct1
00010	UUS10UFlorida10U	Niceville10U
Cuda Systems LLC10UCuda Systems LLC CA1/0-	*H
	 customer-service@cudasystems.net0	+;0	*H
	1	*H
0	*H
	1
141001212737Z0#	*H
	1$T>[y|sp7Q
0l	*H
	1_0]0	`He*0	`He0
*H
0*H
0
*H
@0+0
*H
(0	+710010	UUS10UFlorida10U	Niceville10U
Cuda Systems LLC10UCuda Systems LLC CA1/0-	*H
	 customer-service@cudasystems.net0*H
	1010	UUS10UFlorida10U	Niceville10U
Cuda Systems LLC10UCuda Systems LLC CA1/0-	*H
	 customer-service@cudasystems.net0
	*H
VȈ?<_џ3q-pYK);SOs?bl)EGȓc6߾a.S瞧K:߯˜kђw(7gז
ZG|vͶ?
"|+
J6<yyZ/(.?›#p婬XĨRmOԤ{@x~vaفz}4kwS4B!x
`}8WQbrGp6ϨSr:sDZۨf3iM>2a
Je jK>:F2
\˸uE3ħz(Nw	n\$82'ɆkDUJ|D8U=K4\'HC/jR&sUM#`o֧;kEPOѠV![O+!<|B\#k>-yjhTd>
?NDwDM

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?542C71C9.1050907>