From nobody Fri Mar 10 12:09:32 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PY4dk6ll9z3xSV9 for ; Fri, 10 Mar 2023 12:11:06 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from weser.webweaving.org (weser.webweaving.org [148.251.234.232]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "weser.webweaving.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PY4dk3Dbpz3LN6 for ; Fri, 10 Mar 2023 12:11:06 +0000 (UTC) (envelope-from dirkx@webweaving.org) Authentication-Results: mx1.freebsd.org; none Received: from smtpclient.apple (217-102-131-103.cable.dynamic.v4.ziggo.nl [217.102.131.103]) (authenticated bits=0) by weser.webweaving.org (8.17.1/8.17.1) with ESMTPSA id 32AC9XmS059116 (version=TLSv1.2 cipher=ECDHE-ECDSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 10 Mar 2023 13:09:34 +0100 (CET) (envelope-from dirkx@webweaving.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=webweaving.org; s=shared; t=1678450174; bh=nRpFlcOkZH6HfBF2NB9vOsZbwk7WCiO5kPrKrVuIYAo=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=pCBuj/cOL6aMuJYNImOzBB4C2+K+MjvDLU2XnY5s0zFSl7mMssG/hRWMfgxAzTRba c1oqKL1yienuCD3SiV8yxzkGcj++tAUkjcvcg2uSWsiRR4tOYXZkYMqsHFgQqx6QOO P3alfZEU5DznM3+YU8g/IKdJR00eDjud2vVjn1Cc= X-Authentication-Warning: weser.webweaving.org: Host 217-102-131-103.cable.dynamic.v4.ziggo.nl [217.102.131.103] claimed to be smtpclient.apple Content-Type: text/plain; charset=us-ascii List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: ZFS and Encryption at dataset level From: Dirk-Willem van Gulik In-Reply-To: Date: Fri, 10 Mar 2023 13:09:32 +0100 Cc: freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <8BDBF2BE-77BB-481A-BFB0-2488A8FFA118@webweaving.org> References: <8BAC3BC0-63B2-449D-BF0E-8E5A0A42F1E0@webweaving.org> To: ltning-freebsd-hackers@anduin.net X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.4 (weser.webweaving.org [148.251.234.232]); Fri, 10 Mar 2023 13:09:34 +0100 (CET) X-Rspamd-Queue-Id: 4PY4dk3Dbpz3LN6 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:148.251.0.0/16, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N > On 10 Mar 2023, at 12:18, ltning-freebsd-hackers@anduin.net wrote: >=20 > On 3/8/23 12:50, Dirk-Willem van Gulik wrote: >> Just some words of thanks & praise -- compliments to those & all the = hard work of getting ZFS encryption at data set into 13.1. >> Was an absolute breeze to configure & install - and very easy to = manage ! And flexible enough to integrate with PKCS11 and HSM's. >=20 > This sounds really interesting! We've done some hackery to achieve = something similar, but I'd be really interested in knowing what you've = done in this respect, and how! Hope it's something you can share = publicly :) It is a bit hard to share the HSM integration; as that relies on their = pkcs#11 tooling, the rather specific key references and the way their = callback for N from M works. But in short; we boot the machine without mounting the encrypted = partions. The basic assumption (with some measures like trusted boot, = etc) is that this environment is sufficiently safe (i.e. shells and = share libraries cannot easily be hijacked) - also relative to the issues = with a root empowered entity their access to kernel and the keys stored = there once you have loaded them. We keep a ZFS encryption key on the machine as a file.=20 Depending on risk of loss v.s. v.s. pain of not getting up after a = reboot v.s. risk of leak v.s. risk of ops-errors/process-errors and the = level of 4-eye needed; we then have a few different options. =20 In general, this file is then encrypted with a (second) file-key. This = file key is stored encrypted against a set of public keys. At the most = strict end - we then have an HSM decrypt this key - with a 2 of M check = on chipcard access & pin entry on a desktop device that is integrated = with the HSM remotey. At the simplest case - we just use a OpenPGP = chipcard in a reader with a pincard on a normal desktop with just GPG = and OpenSC and SSH.=20 Two of these below (one plain & direct; when you essentially trust the = user; one with the file-key separation -- which lets us rotate that file = with ease as long as there is enough 4 eye/no backup/separation-of-roles = on the encrypted key file). Below is simplified/edited a bit - we have some sudo-glue in place as = `load-keys' is under zfs; we basically made a 'zfs-load-key' to curtail = 'zfs' power. Dw=20 #!/bin/sh # # Authorized keys in ~/.ssh/authorized_keys on target # # command=3D"cat /.zfs-key; /sbin/zfs load-key -L prompt tank/enc && = zfs mount -a" ssh-rsa AAAAB.... # # Initial generate & encrypt key with: =20 # # openssl rand -base64 128 | |tr -d '\n' |\ # gpg -R xx -R xx -R .... -a -e --no-encrypt-to > /.zfs-key #=20 # Assumption: Unlocking user `trusted` as root; no 4 eyes, no role = separation, etc # set -e if [ $# =3D 0 ]; then echo Syntax: $0 hostname ... exit 1 fi if ! /usr/local/bin/gpg --card-status 2> /dev/null; then echo Seat your personal/npi chipcard first exit 2 fi FIFO=3D/tmp/fifo.$$ KEYID=3D${KEYID:-XXXXXXXXXX} /usr/bin/mkfifo $FIFO E=3D0 ( for host in $* do cat $FIFO |\ /usr/bin/ssh -F /dev/null -i $SSHKEY $SSH_EXTRA_ARGS -T = -l root "$host" |\ /usr/local/bin/gpg --quiet --default-key "$KEYID" >> = $FIFO done ) || E=3D$? rm -f $FIFO exit $E #!/bin/sh # # Authorized keys in ~/.ssh/authorized_keys on target # # command=3D"/usr/local/sbin/zfs-hsm-load-key && zfs mount -a" ssh-rsa = AAAAB.... # # zfs-hsm-load-key essentially does # get /.zfs-key-to-keyfile decrypted via the hardware # use it to decrypt /.zfs-key # sudo zfs load-key the output # remount and restart jails as needed. # # Initial generate & encrypt key with: =20 # # openssl rand -base64 128 | |tr -d '\n' |\ # hsm-file -enc -k - > /.zfs-key-to-keyfile # # openssl rand -base64 128 | |tr -d '\n' |\ # hsm-file -enc -k - -f /.zfs-key-to-keyfile -k /.zfs-key #=20 set -e if [ $# !=3D 1 ]; then echo Syntax: $0 hostname ... exit 1 fi if ! hsmlnk -p 9731 -lrp; then echo Make sure the extender connected to USB exit 1 fi ssh -L 9731:localhost:9731 -F /dev/null -i $SSHKEY $SSH_EXTRA_ARGS -T -l = hunclk $host exit $E