Date: Sun, 10 Feb 2019 13:43:13 -0600 From: Karl Denninger <karl@denninger.net> To: freebsd-stable@freebsd.org Subject: Re: Geli prompts on gptzfsboot (Was:: Serious ZFS Bootcode Problem (GPT NON-UEFI -- RESOLVED) Message-ID: <ef143cb8-e674-cc06-4569-e5fa66676f25@denninger.net> In-Reply-To: <2f5e28f7f48fa34e60e6225d63cdb2bf357313aa.camel@freebsd.org> References: <911d001f-9e33-0521-51fe-f7d1383dfc62@denninger.net> <CANCZdfp0QaXodmYBp9Eox9Ca5kyQibCXw5rRTwsO-mCjApYswA@mail.gmail.com> <b11ec38c-1c6a-6e92-810c-4d2fe3e8df3d@freebsd.org> <a107a4f5-2851-191a-5f8c-a4cd44c98458@denninger.net> <16c56c89ff8a3d89164d9152f6c38687dcba99b5.camel@freebsd.org> <3fd7f001-879c-7b1e-3d1a-d2939ac07d9c@denninger.net> <398cae11ff6b81d0bc1dbdcd54f64eb97b2c812a.camel@freebsd.org> <df021c0b-ef2c-df61-7042-303dbadaab75@denninger.net> <2f5e28f7f48fa34e60e6225d63cdb2bf357313aa.camel@freebsd.org>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --]
On 2/10/2019 12:40, Ian Lepore wrote:
> On Sun, 2019-02-10 at 12:35 -0600, Karl Denninger wrote:
>> On 2/10/2019 12:01, Ian Lepore wrote:
>>> On Sun, 2019-02-10 at 11:54 -0600, Karl Denninger wrote:
>>>> On 2/10/2019 11:50, Ian Lepore wrote:
>>>>> On Sun, 2019-02-10 at 11:37 -0600, Karl Denninger wrote:
>>>>>
>>>>>> [...]
>>>>>>
>>>>>> BTW am I correct that gptzfsboot did *not* get the ability to
>>>>>> read
>>>>>> geli-encrypted pools in 12.0? The UEFI loader does know how
>>>>>> (which I'm
>>>>>> using on my laptop) but I was under the impression that for
>>>>>> non-
>>>>>> UEFI
>>>>>> systems you still needed the unencrypted boot partition from
>>>>>> which to
>>>>>> load the kernel.
>>>>>>
>>>>> Nope, that's not correct. GELI support was added to the boot
>>>>> and
>>>>> loader
>>>>> programs for both ufs and zfs in freebsd 12. You must set the
>>>>> geli
>>>>> '-g'
>>>>> option to be prompted for the passphrase while booting (this is
>>>>> separate from the '-b' flag that enables mounting the encrypted
>>>>> partition as the rootfs). You can use "geli configure -g" to
>>>>> turn
>>>>> on
>>>>> the flag on any existing geli partition.
>>>>>
>>>>> -- Ian
>>>> Excellent - this will eliminate the need for me to run down the
>>>> foot-shooting that occurred in my update script since the
>>>> unencrypted
>>>> kernel partition is no longer needed at all. That also
>>>> significantly
>>>> reduces the attack surface on such a machine (although you could
>>>> still
>>>> tamper with the contents of freebsd-boot of course.)
>>>>
>>>> The "-g" flag I knew about from experience in putting 12 on my X1
>>>> Carbon
>>>> (which works really well incidentally; the only issue I'm aware
>>>> of is
>>>> that there's no 5Ghz WiFi support.)
>>>>
>>> One thing that is rather unfortunate... if you have multiple geli
>>> encrypted partitions that all have the same passphrase, you will be
>>> required to enter that passphrase twice while booting -- once in
>>> gpt[zfs]boot, then again during kernel startup when the rest of the
>>> drives/partitions get tasted by geom. This is because APIs within
>>> the
>>> boot process got changed to pass keys instead of the passphrase
>>> itself
>>> from one stage of booting to the next, and the fallout of that is
>>> the
>>> key for the rootfs is available to the kernel for mountroot, but
>>> the
>>> passphrase is not available to the system when geom is probing all
>>> the
>>> devices, so you get prompted for it again.
>>>
>>> -- Ian
>> Let me see if I understand this before I do it then... :-)
>>
>> I have the following layout:
>>
>> 1. Two SSDs that contain the OS as a two-provider ZFS pool, which has
>> "-b" set on both members; I get the "GELI Passphrase:" prompt from
>> the
>> loader and those two providers (along with encrypted swap) attach
>> early
>> in the boot process. The same SSDs contain a mirrored non-encrypted
>> pool that has /boot (and only /boot) on it because previously you
>> couldn't boot from an EFI-encrypted pool at all.
>>
>> Thus:
>>
>> [\u@NewFS /root]# gpart show da1
>> => 34 468862061 da1 GPT (224G)
>> 34 2014 - free - (1.0M)
>> 2048 1024 1 freebsd-boot (512K)
>> 3072 1024 - free - (512K)
>> 4096 20971520 2 freebsd-zfs [bootme] (10G)
>> 20975616 134217728 3 freebsd-swap (64G)
>> 155193344 313667584 4 freebsd-zfs (150G)
>> 468860928 1167 - free - (584K)
>>
>> There is of course a "da2" that is identical. The actual encrypted
>> root
>> pool is on partition 4 with "-b" set at present. I get prompted from
>> loader as a result after the unencrypted partition (#2) boots.
>>
>> 2. Multiple additional "user space" pools on a bunch of other disks.
>>
>> Right now #2 is using geli groups. Prior to 12.0 they were handled
>> using a custom /etc/rc.d script I wrote that did basically the same
>> thing that geli groups does because all use the same passphrase and
>> entering the same thing over and over on a boot was a pain in the
>> butt.
>> It prompted cleanly with no echo, took a password and then iterated
>> over
>> a list of devices attaching them one at a time. That requirement is
>> now
>> gone with geli groups, which is nice since mergemaster always
>> complained
>> about it being a "non-standard" thing; it *had* to go in /etc/rc.d
>> and
>> not in /usr/etc/rc.d else I couldn't get it to run early enough --
>> unfortunately.
>>
>> So if I remove the non-encrypted freebsd-zfs mirror that the system
>> boots from in favor of setting "-g" on the root pool (both providers)
>> gptzfsboot will find and prompt for the password to boot before
>> loader
>> gets invoked at all, much like the EFI loader does. That's good.
>> (My
>> assumption is that the "-g" is sufficient; I don't need (or want)
>> "bootme" set -- correct?)
>>
>> /However, /once the kernel boots somewhere in the mishmash of boot-
>> time
>> messages, and probably not where it's instantly obvious nor where it
>> will halt the cascade display on the console, I'm going to get asked
>> for
>> that passphrase again? I assume I want to remove
>> 'geom_eli_passphrase_prompt="YES"' from loader.conf as well -- or
>> would
>> leaving it in there save me from the prompt that's hard to find in
>> the
>> cascade?
>>
>> Or, even better, would that situation of a double-prompt only apply
>> if I
>> had "-b" set on something /other than /the boot device pool vdevs (I
>> don't -- those are handled by #2 for this exact reason.)
>>
> I think at this point I have to ease out of the conversation, because I
> know almost nothing about zfs, despite having somehow managed to add
> geli support to the zfs code in loader. I did so without understanding
> zfs in any way, because I added the support at a more generic "disk
> drive support" layer in loader, and did all my testing using automated
> scripts Alan and Warner created to test zfs booting using qemu.
>
> -- Ian
I can confirm that this boots and comes up cleanly without re-prompting
for the boot pool password.
The machines I have in the field in this config, during the next upgrade
cycle, are going to get set up this way. When it makes sense to replace
these with UEFI boards (likely when Coffee Lake Xeons and Mobos that can
handle them get a bit more reasonable and start showing up with IPMI/kvm
ports) I'll likely start getting rid of these older devices simply on
the performance-for-power equation, but these are likely to be out there
for me, anyway, for the next few years.
In short very nice work -- and thank you!
--
Karl Denninger
karl@denninger.net <mailto:karl@denninger.net>
/The Market Ticker/
/[S/MIME encrypted email preferred]/
[-- Attachment #2 --]
0 *H
010
`He 0 *H
00 H^Ōc!5
H0
*H
01 0 UUS10U Florida10U Niceville10U
Cuda Systems LLC10U Cuda Systems CA1!0 U Cuda Systems LLC 2017 CA0
170817164217Z
270815164217Z0{1 0 UUS10U Florida10U
Cuda Systems LLC10U Cuda Systems CA1%0#U Cuda Systems LLC 2017 Int CA0"0
*H
0
h-5B>[;olӴ0~͎O9}9Ye*$ g !ukvʶLzN`jL>MD'7U 45CB+kY`bd~b*c3Ny-78ju]9H euέsӬDؽmgwER?&UURj'}9nWD i`XcbGz \gG=u%\Oi13ߝ4
K44pYQr]Ie/r0+eEޝݖ0C15Mݚ@JSZ(zȏ NTa(25DD5.l<g[[Z arQQ%Buȴ~~`IohRbʳڟu2MS8EdFUClCMaѳ !}ș+2k/bųE,n当ꖛ\(8 WV8 d]b yXw ܊:I39
00 U ]^§Q\ӎ0U #0T039N0b01 0 UUS10U Florida10U Niceville10U
Cuda Systems LLC10U Cuda Systems CA1!0 U Cuda Systems LLC 2017 CA @Ui0U 0 0U 0
*H
:P U!>vJ n io -#ן]Wyu jǑR̀Q
nƇ!G ѦF g\ yLx gw=OPycehf[ }ܷ['4ڝ\[p 6\o.B&JF" ZC{;*o*mcCcLY߾`
t*S!(` ]DHP5A~/NPp6=mhk밣'doA$86hm5ӚS@ j ެEgl
)0JG`%k35PaC?σ
׳HEt}!P㏏%*BxbQwaKG$6 h¦Mve; [o-Iی&
I,Tc ߎ#t wPA@l0P+KXBպT zGv;NcI3&J ĬUPNa?/%W6G۟N000 k#Xd\ =0
*H
0{1 0 UUS10U Florida10U
Cuda Systems LLC10U Cuda Systems CA1%0#U Cuda Systems LLC 2017 Int CA0
170817212120Z
220816212120Z0W1 0 UUS10U Florida10U
Cuda Systems LLC10U karl@denninger.net0"0
*H
0
T[I-ΆϏ dn;Å@שy.us~_ZG%<MYd\gvfnsa1'6Egyjs"C [{~ _K Pn+<*pv#Q+H/7[-vqDV^U>f%GX)H.|l`M(Cr >е͇6#odc"YljҦln8@5SA0&ۖ"OGj?UDWZ5 dDB 7k-)9Izs-JAv
J 6L$Ն1SmY.Lqw*SH;EF'DĦH]MOgQQ|Mٙג2Z9y@ y]}6ٽeY9Y2xˆ$T=e CǺǵbn֛{j |@LLt1[Dk5:$= ` M 00<+00.0,+0 http://ocsp.cudasystems.net:88880 U 0 0 `HB0U 0 U %0++03 `HB
&$OpenSSL Generated Client Certificate0 U %՞V =;bzQ0U #0]^§Q\ӎϡ01 0 UUS10U Florida10U Niceville10U
Cuda Systems LLC10U Cuda Systems CA1!0 U Cuda Systems LLC 2017 CA H^Ōc!5
H0 U 0karl@denninger.net0
*H
۠A0-j%--$% g2#ޡ1^>{K+uGE v1ş7Af&b&O; .;A5* U)ND2bF|\= ]<sˋL!wrw٧> YMÄ3\mWR hSv!_zvl? 3_ xU%\^#O*Gk̍YI_&Fꊛ@&1n } ͬ:{hTP3B.;bU8:Z=^Gw8 !k-@xE@i,+'Iᐚ:f hztX7/(hY` O.1}a`%RW^akǂpCAufgDix UTЩ/7}%=jnVZvcF<M=
2^GKH5魉
_O4ެByʈySkw=5@h.0z>
W1000{1 0 UUS10U Florida10U
Cuda Systems LLC10U Cuda Systems CA1%0#U Cuda Systems LLC 2017 Int CA k#Xd\ =0
`He E0 *H
1 *H
0 *H
1
190210194313Z0O *H
1B@Bی
䥨(<&dNCmw[1@n*IL|$bo.zf*0l *H
1_0]0 `He*0 `He0
*H
0*H
0
*H
@0+0
*H
(0 +7100{1 0 UUS10U Florida10U
Cuda Systems LLC10U Cuda Systems CA1%0#U Cuda Systems LLC 2017 Int CA k#Xd\ =0 *H
10{1 0 UUS10U Florida10U
Cuda Systems LLC10U Cuda Systems CA1%0#U Cuda Systems LLC 2017 Int CA k#Xd\ =0
*H
uqwYL4\\s}ߓqcWø"3Aɀ6 &(f |r\ZpzەS7k"UVe IvQh P:*Kwr3}QFHSI,E3,RMb$UKZ*) 2y.ڂ_ZQl*c $P c)l"ްцn jWYk5,-zaS1|=a1Nb2 й-[BA51)Rz-K^n] gVAdǤY]` XS)xVdRKN ;ů
9/OOѝV'hcZIfl!/^ red P ,́ ̶KFQlbR)C
!7f B8 ăsKRgΚS%tFWz7%-nG@;s ~ȉc
home |
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ef143cb8-e674-cc06-4569-e5fa66676f25>