From owner-freebsd-stable@freebsd.org Sun Feb 10 18:40:39 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C77414D6096 for ; Sun, 10 Feb 2019 18:40:39 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound3d.ore.mailhop.org (outbound3d.ore.mailhop.org [54.186.57.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 61EFF8AFBB for ; Sun, 10 Feb 2019 18:40:37 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1549824000; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=GVid3O2oQ5cGNa9U/tnpd6++vY8NJSXkaRiLrEdPuSDYvcYWTcIXZfXQ963d0xmMv9AYMoeSpD8Y0 1mS3CFDxU/xz7/L7i0mpfd9dQB9M+uU1JFdVBHTX1o//L+J924SPCkgBOEYGKbAvkMCBUt2Yj+W6A/ oDrHJLWkpyVbTM4+lxdYwkK9sFPRdJygp4/PrScgCx51BDj2BiplB6q9FaOSAEkvMKJyB1tTaDObYs NfAi3zdJW6htk0hilXKnH54w4BP0OchtZ7BnGD3BFH6RFJKJhHYrErbRdpKHWoouiZktv+nnZx/I6A G4sJmJL5BbwrGNCK5gxb5HP8gvsapKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=AqQeNayZrcpJjzs1VpwudWYGefdIQW7bz4mgMsGMtoE=; b=f3O16CILlknOWmgwpZH0OWibZ30oP+dgHrHnp/CjXcJupUqSTYdFN2bW5m8U5Ttw9WFSrUDbCUrwr KxjKbjJQVTQMBmjE2lRS1qtnKMbqAfvMVsOUBKYOVjSxlgeRPsFWTrHQNvcRO6skCuXaIZ3jIQFV/e YD/dmc5QeLQrIvpA27BdaKMzg7Pn4AxXM83M7HDgq/vhxcLKJFZyKy6E/0MZ1Z0bRT+/G8i3reWQIS Gw7E092ZfEckgzuyCBQpErAiiqTZkSPudssniXYK/+vaz1wZAOOb3Oy5fZZZRquOHskt4L7RoXDuYx EvnXsFNmP3WKgFzJASXOguWuL4kJtmg== ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=AqQeNayZrcpJjzs1VpwudWYGefdIQW7bz4mgMsGMtoE=; b=tFdcgpMx6I18TBVZ8AwCUz69K7M0aTXWueZw20faM3F753JXY+VQUp8KHd7CYY36zb4PACFDB06B5 UZkXUJeExyoLGKYEGRly3DeqVxpBIXDK6EFCSuh6uAzUBcn8LgEZ3efgFUyVKaEgjdhgg0UTDpq2Rh zd3StMBL55zvcdeSiAYqFqiF9n6LuRZxUwelSHpvgs69RyznGL++phW/jOacmP1XmuuyoqED2TEu4f mbbJSpBi4VnG2EEqmeC0ncUM1YJDAVzoCckgU3vWvi23QtW2Tqf/qKs3V5vpPkIh/p5WpE0U3AMhSI WTU0bshD+098/jLTZJv3VZuQmUg0NXw== X-MHO-RoutePath: aGlwcGll X-MHO-User: 444b0273-2d63-11e9-9789-75353a1f43cf X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound3.ore.mailhop.org (Halon) with ESMTPSA id 444b0273-2d63-11e9-9789-75353a1f43cf; Sun, 10 Feb 2019 18:39:59 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x1AIeSQU052770; Sun, 10 Feb 2019 11:40:28 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <2f5e28f7f48fa34e60e6225d63cdb2bf357313aa.camel@freebsd.org> Subject: Re: Geli prompts on gptzfsboot (Was:: Serious ZFS Bootcode Problem (GPT NON-UEFI -- RESOLVED) From: Ian Lepore To: Karl Denninger Cc: freebsd-stable@freebsd.org Date: Sun, 10 Feb 2019 11:40:28 -0700 In-Reply-To: References: <911d001f-9e33-0521-51fe-f7d1383dfc62@denninger.net> <16c56c89ff8a3d89164d9152f6c38687dcba99b5.camel@freebsd.org> <3fd7f001-879c-7b1e-3d1a-d2939ac07d9c@denninger.net> <398cae11ff6b81d0bc1dbdcd54f64eb97b2c812a.camel@freebsd.org> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 61EFF8AFBB X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; ASN(0.00)[asn:16509, ipnet:54.186.0.0/15, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2019 18:40:39 -0000 On Sun, 2019-02-10 at 12:35 -0600, Karl Denninger wrote: > On 2/10/2019 12:01, Ian Lepore wrote: > > On Sun, 2019-02-10 at 11:54 -0600, Karl Denninger wrote: > > > On 2/10/2019 11:50, Ian Lepore wrote: > > > > On Sun, 2019-02-10 at 11:37 -0600, Karl Denninger wrote: > > > > > > > > > [...] > > > > > > > > > > BTW am I correct that gptzfsboot did *not* get the ability to > > > > > read > > > > > geli-encrypted pools in 12.0? The UEFI loader does know how > > > > > (which I'm > > > > > using on my laptop) but I was under the impression that for > > > > > non- > > > > > UEFI > > > > > systems you still needed the unencrypted boot partition from > > > > > which to > > > > > load the kernel. > > > > > > > > > > > > > Nope, that's not correct. GELI support was added to the boot > > > > and > > > > loader > > > > programs for both ufs and zfs in freebsd 12. You must set the > > > > geli > > > > '-g' > > > > option to be prompted for the passphrase while booting (this is > > > > separate from the '-b' flag that enables mounting the encrypted > > > > partition as the rootfs). You can use "geli configure -g" to > > > > turn > > > > on > > > > the flag on any existing geli partition. > > > > > > > > -- Ian > > > > > > Excellent - this will eliminate the need for me to run down the > > > foot-shooting that occurred in my update script since the > > > unencrypted > > > kernel partition is no longer needed at all. That also > > > significantly > > > reduces the attack surface on such a machine (although you could > > > still > > > tamper with the contents of freebsd-boot of course.) > > > > > > The "-g" flag I knew about from experience in putting 12 on my X1 > > > Carbon > > > (which works really well incidentally; the only issue I'm aware > > > of is > > > that there's no 5Ghz WiFi support.) > > > > > > > One thing that is rather unfortunate... if you have multiple geli > > encrypted partitions that all have the same passphrase, you will be > > required to enter that passphrase twice while booting -- once in > > gpt[zfs]boot, then again during kernel startup when the rest of the > > drives/partitions get tasted by geom. This is because APIs within > > the > > boot process got changed to pass keys instead of the passphrase > > itself > > from one stage of booting to the next, and the fallout of that is > > the > > key for the rootfs is available to the kernel for mountroot, but > > the > > passphrase is not available to the system when geom is probing all > > the > > devices, so you get prompted for it again. > > > > -- Ian > > Let me see if I understand this before I do it then... :-) > > I have the following layout: > > 1. Two SSDs that contain the OS as a two-provider ZFS pool, which has > "-b" set on both members; I get the "GELI Passphrase:" prompt from > the > loader and those two providers (along with encrypted swap) attach > early > in the boot process. The same SSDs contain a mirrored non-encrypted > pool that has /boot (and only /boot) on it because previously you > couldn't boot from an EFI-encrypted pool at all. > > Thus: > > [\u@NewFS /root]# gpart show da1 > => 34 468862061 da1 GPT (224G) > 34 2014 - free - (1.0M) > 2048 1024 1 freebsd-boot (512K) > 3072 1024 - free - (512K) > 4096 20971520 2 freebsd-zfs [bootme] (10G) > 20975616 134217728 3 freebsd-swap (64G) > 155193344 313667584 4 freebsd-zfs (150G) > 468860928 1167 - free - (584K) > > There is of course a "da2" that is identical. The actual encrypted > root > pool is on partition 4 with "-b" set at present. I get prompted from > loader as a result after the unencrypted partition (#2) boots. > > 2. Multiple additional "user space" pools on a bunch of other disks. > > Right now #2 is using geli groups. Prior to 12.0 they were handled > using a custom /etc/rc.d script I wrote that did basically the same > thing that geli groups does because all use the same passphrase and > entering the same thing over and over on a boot was a pain in the > butt. > It prompted cleanly with no echo, took a password and then iterated > over > a list of devices attaching them one at a time. That requirement is > now > gone with geli groups, which is nice since mergemaster always > complained > about it being a "non-standard" thing; it *had* to go in /etc/rc.d > and > not in /usr/etc/rc.d else I couldn't get it to run early enough -- > unfortunately. > > So if I remove the non-encrypted freebsd-zfs mirror that the system > boots from in favor of setting "-g" on the root pool (both providers) > gptzfsboot will find and prompt for the password to boot before > loader > gets invoked at all, much like the EFI loader does. That's good. > (My > assumption is that the "-g" is sufficient; I don't need (or want) > "bootme" set -- correct?) > > /However, /once the kernel boots somewhere in the mishmash of boot- > time > messages, and probably not where it's instantly obvious nor where it > will halt the cascade display on the console, I'm going to get asked > for > that passphrase again? I assume I want to remove > 'geom_eli_passphrase_prompt="YES"' from loader.conf as well -- or > would > leaving it in there save me from the prompt that's hard to find in > the > cascade? > > Or, even better, would that situation of a double-prompt only apply > if I > had "-b" set on something /other than /the boot device pool vdevs (I > don't -- those are handled by #2 for this exact reason.) > I think at this point I have to ease out of the conversation, because I know almost nothing about zfs, despite having somehow managed to add geli support to the zfs code in loader. I did so without understanding zfs in any way, because I added the support at a more generic "disk drive support" layer in loader, and did all my testing using automated scripts Alan and Warner created to test zfs booting using qemu. -- Ian