From owner-freebsd-geom@FreeBSD.ORG Wed Sep 3 20:00:24 2014 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 30753AFD for ; Wed, 3 Sep 2014 20:00:24 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 08EBB1CB6 for ; Wed, 3 Sep 2014 20:00:23 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s83K0EAG082844 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 3 Sep 2014 13:00:15 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s83K0E81082843; Wed, 3 Sep 2014 13:00:14 -0700 (PDT) (envelope-from jmg) Date: Wed, 3 Sep 2014 13:00:14 -0700 From: John-Mark Gurney To: Karl Denninger Subject: Re: Attempt to add multiple device attachment to "geli attach" Message-ID: <20140903200014.GB82175@funkthat.com> Mail-Followup-To: Karl Denninger , freebsd-geom@freebsd.org References: <54076871.5010405@denninger.net> <54076CFE.5010308@denninger.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54076CFE.5010308@denninger.net> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 03 Sep 2014 13:00:15 -0700 (PDT) Cc: freebsd-geom@freebsd.org X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Sep 2014 20:00:24 -0000 Karl Denninger wrote this message on Wed, Sep 03, 2014 at 14:33 -0500: > Never mind... I know what I missed -- the key generation that is passed > in is dependent on the metadata read from the userspace. > > More work to do here.... will have to pass a separate key structure for > each disk and it will also require some more work in the userspace > command area so it doesn't prompt a second time for a password. > > I'll post the completed patch set once I have it if people here think it > would be interesting. Just some comments on this as I've thought about this issue... There are two issues here, one is for root and one is for geli volume mounted later... For the later, I personally use a key volume that is encrypted, and uses that "key store" for my large 8 disk raidz pool.. This is less of an issue, but still requires me to type in the password twice... It basicly boils down to: (cd /zkeys && for i in *.key; do geli attach -p -k "$i" "label/${i%.key}"; geli attach -p -k "$i" "gpt/${i%.key}"; done) || exit 5 I have to do both label and gpt since disks are labeled, but things like zlog are on gpt partitions... I haven't reviewed your patch, nor have I looked at how geli keys volumes upon init, but make sure that you have each volume's master key salted seperately... This way if the volumes get seperated from your system, it won't leak that they use the same key... Yes, it'll take a bit more cpu time to unlock, but not that big of an issue IMO... Handling unlocking mirrored roots is a bit more interesting as you now have to touch the geli kernel code... btw, reattaching a single disk that was previously part of a pool is fast... I've done this on more than one occasion where one disk drops out of the raidz and then shortly after I reattach it... It will recognize the original data, so only if new data that got written can't be read will you suffer a loss, but that would be a double failure case, and known limitation of raidz... Thanks for looking at this... I'm definately interested in making multi disk geli more usable... $find /dev -name "*.eli" | wc -l 17 :) 8 (raidz data disks) + 2 (mirrored root) + 1 (swap) + 2 (cache) + 2 (log) + 2 (duplicates from root ada vs ad) > On 9/3/2014 14:13, Karl Denninger wrote: > >I'm aware of the potential issues here in terms of keying risks, but > >there are plenty of reasons to support this capability with the > >largest one being ZFS volumes that you wish to run encrypted. > > > >Take the following: > > > >label/pool0 > >label/pool1 > >label/pool2 > >label/pool3 > > > >(all relative to /dev, of course) > > > >These are all gpt partitions on different devices (typically full > >disks less labels.) You "geli init" them and then attach them and > >build a raidz2 pool on that. > > > >OK, now the system is rebooted. If you use the rc.conf file's option > >to request geli passwords during the boot you had better not screw up > >three times for only ONE of these volumes or the pool WILL come up > >degraded! Needless to say that's not nice. It's even worse if it's a > >raidz pool, you blow it, you reattach that disk and allow it to > >resilver *and take a disk error on the remaining drives during the > >resilver* -- now you're completely hosed. > > > >So, here's the idea -- if you use the same password and/or keyfile for > >ALL of the volumes then either they ALL mount (if you get it right) or > >NONE of them mount (if you get it wrong.) Now the pool won't import > >if you get it wrong and you're safe from the risk of taking a forced > >resilver and potential data loss. > > > >The geom subclass command has a simple "nargs" test (must be "1") in > >the attach command; I replaced that with "nargs < 1" for the error > >case. Now I can pass multiple devices to the kernel's geom handler > >and they get passed to the kernel ctl handler. > > > >The following patch should, I believe, work -- but it doesn't. The > >first disk attaches but the second one that was init'd with the same > >passphrase fails. > > > >As near as I can tell the key components are not picked up off the > >metadata until the geom driver gets ahold of it -- and thus the second > >decryption attempt should work since on the second iteration through > >the code grabs the key parameters off the request a second time. > > > >But I'm obviously missing something because the second volume returns > >"Wrong key for ...." > > > >Ideas? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."