From owner-freebsd-geom@freebsd.org Sun Nov 11 21:01:34 2018 Return-Path: Delivered-To: freebsd-geom@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A742D1124487 for ; Sun, 11 Nov 2018 21:01:34 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 2CC066DEB2 for ; Sun, 11 Nov 2018 21:01:34 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id E62A81124475; Sun, 11 Nov 2018 21:01:33 +0000 (UTC) Delivered-To: geom@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C3F501124474 for ; Sun, 11 Nov 2018 21:01:33 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4EF236DEA2 for ; Sun, 11 Nov 2018 21:01:33 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 599BBE42A for ; Sun, 11 Nov 2018 21:01:32 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id wABL1WWC058847 for ; Sun, 11 Nov 2018 21:01:32 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id wABL1WWd058840 for geom@FreeBSD.org; Sun, 11 Nov 2018 21:01:32 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201811112101.wABL1WWd058840@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: geom@FreeBSD.org Subject: Problem reports for geom@FreeBSD.org that need special attention Date: Sun, 11 Nov 2018 21:01:32 +0000 MIME-Version: 1.0 X-Rspamd-Queue-Id: 2CC066DEB2 X-Spamd-Result: default: False [-105.89 / 200.00]; FORGED_RECIPIENTS_FORWARDING(0.00)[]; ALLOW_DOMAIN_WHITELIST(-100.00)[FreeBSD.org]; FORWARDED(0.00)[geom@mailman.ysv.freebsd.org]; SPF_FAIL_FORWARDING(0.00)[]; TO_DN_NONE(0.00)[]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; URI_COUNT_ODD(1.00)[3]; RCVD_IN_DNSWL_MED(-0.20)[5.0.0.0.0.5.0.0.0.0.0.0.0.0.0.0.a.6.0.2.4.5.2.2.0.0.9.1.1.0.0.2.list.dnswl.org : 127.0.9.2]; MX_GOOD(-0.01)[cached: mx1.FreeBSD.org]; NEURAL_HAM_SHORT(-1.00)[-1.000,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; IP_SCORE(-3.68)[ip: (-9.84), ipnet: 2001:1900:2254::/48(-4.76), asn: 10310(-3.69), country: US(-0.09)]; ASN(0.00)[asn:10310, ipnet:2001:1900:2254::/48, country:US]; FORGED_RECIPIENTS(0.00)[geom@FreeBSD.org,freebsd-geom@freebsd.org]; TO_DOM_EQ_FROM_DOM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[FreeBSD.org]; RCPT_COUNT_ONE(0.00)[1]; FROM_NO_DN(0.00)[]; RCVD_COUNT_SEVEN(0.00)[7] X-Rspamd-Server: mx1.freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Nov 2018 21:01:34 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- In Progress | 218679 | [geli] add a verify command 1 problems total for which you should take action. From owner-freebsd-geom@freebsd.org Fri Nov 16 22:35:40 2018 Return-Path: Delivered-To: freebsd-geom@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4FEF11106904 for ; Fri, 16 Nov 2018 22:35:40 +0000 (UTC) (envelope-from coco@executive-computing.de) Received: from mail.moehre.org (mail.moehre.org [195.96.35.7]) by mx1.freebsd.org (Postfix) with ESMTP id 739736C876 for ; Fri, 16 Nov 2018 22:35:39 +0000 (UTC) (envelope-from coco@executive-computing.de) Received: from mail.moehre.org (unknown [195.96.35.7]) by mail.moehre.org (Postfix) with ESMTP id 4F46F38D23 for ; Fri, 16 Nov 2018 23:18:03 +0100 (CET) X-Spam-Flag: NO X-Spam-Score: -100.918 X-Spam-Level: X-Spam-Status: No, score=-100.918 tagged_above=-999 required=5 tests=[ALL_TRUSTED=-1, AWL=-0.072, TW_GP=0.077, TW_ZF=0.077, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mail.moehre.org ([195.96.35.7]) by mail.moehre.org (mail.moehre.org [195.96.35.7]) (amavisd-new, port 10024) with ESMTP id G_iBr2qK4yBL for ; Fri, 16 Nov 2018 23:18:03 +0100 (CET) Received: from bsdbuch.c0c0.intra (p54BEC3DB.dip0.t-ipconnect.de [84.190.195.219]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: coco@executive-computing.de) by mail.moehre.org (Postfix) with ESMTPSA id E2ACD38D1C for ; Fri, 16 Nov 2018 23:18:02 +0100 (CET) Date: Fri, 16 Nov 2018 23:18:09 +0100 From: Marco Steinbach To: freebsd-geom@freebsd.org Subject: eli encrypted providers for zfs raidz1 Message-ID: <20181116231809.40a8f74c@bsdbuch.c0c0.intra> X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.32; amd64-portbld-freebsd11.2) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 739736C876 X-Spamd-Result: default: False [-3.04 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[mail.moehre.org]; NEURAL_HAM_SHORT(-0.92)[-0.916,0]; RCVD_NO_TLS_LAST(0.10)[]; RECEIVED_SPAMHAUS_PBL(0.00)[219.195.190.84.zen.spamhaus.org : 127.0.0.10]; R_DKIM_NA(0.00)[]; IP_SCORE(-0.00)[country: DE(-0.01)]; ASN(0.00)[asn:8354, ipnet:195.96.32.0/19, country:DE]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.960,0]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.95)[-0.951,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-geom@freebsd.org]; DMARC_NA(0.00)[executive-computing.de]; RCPT_COUNT_ONE(0.00)[1] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Nov 2018 22:35:40 -0000 Hi. I'm using 11.2-RELEASE r335510 amd64 GENERIC in an Oracle VirtualBox setup on FreeBSD, which is what comes out of the box, when installing 11.2 from the distribution media. I'm trying to wrap my head around on how to avoid a zpool resilver on a non-booting ZFS raidz1 of off four equally sized (GPT) partitions on four distinct drives using eli for encyption. IOW: I do struggle with finding a way to attach all the providers such, that ZFS does not initiate a resilver due to the providers being attached sequentially. I've created and initialized the partitions as follows (generic notation, comments on chosen encryption algo welome, since this testing setup lacks AES-NI): # gpart create -s gpt /dev/ada[2-5] # gpart add -t freebsd-zfs /dev/ada[2-5] # geli init -e AES-CBC -l 128 /dev/ada[2-5]p1 Then I attached the geli partitions like so: # geli attach /dev/ada[2-5]p1 And finally created a raidz1 spanning all four partitions: # zpool create u0001 raidz1 /dev/ada[2-5]p1.eli That works flawlessly. And naturally, after a reboot none of the encrypted devices is available to the zpool then, unless I attach them. Doing so using geli attach requires me to attach them sequentially, which then results in ZFS resilvering the pool. So, here's my questions: 1. Is the inavoidable resilver intended behaviour based on current implementation, or am I missing something ? 2. In case I use a bootable zfsroot (cudos to allanjude@, I highly recommend his BSDCan presentations on the matter), is there a way to hand over the zfsroot passphrase to eli for automatically attaching other providers ? Please note, that I'd like to stick as close as possible to what the base system offers for this use-case. MfG CoCo From owner-freebsd-geom@freebsd.org Sat Nov 17 00:42:31 2018 Return-Path: Delivered-To: freebsd-geom@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 679D41109C49 for ; Sat, 17 Nov 2018 00:42:31 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: from mail-ua1-x929.google.com (mail-ua1-x929.google.com [IPv6:2607:f8b0:4864:20::929]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BAD077147F for ; Sat, 17 Nov 2018 00:42:30 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: by mail-ua1-x929.google.com with SMTP id z11so8894861uaa.10 for ; Fri, 16 Nov 2018 16:42:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=k7qLTVFzV9pKc9OayGTtCUecEgYLFUwe2hitxu45nVE=; b=cSsewII4VlnFdgeWtt1nlGsYnDzj7AgBttbpNGo8DLgUSpZjnpx9KCUS5Z/ge0pkYI SQTsE39Tsp7D8U49dWzR6vSa9fn7Jj7P9jqUgdACVK3NBw1LyC/g/VtvNOQ83DAKkIj2 O0IHTj1GDAx7wOORmkvuw4KeRiA3rxDD/s9zZNwg3QYiPLmf0BAs1Qb90ysKgJ2lsvUm T2h5q0ZKUhbCHs/wtZZTMxRruzTeeLWujh5NdXHGKuHsV+upYbxeIiovsiBgF73apG8i TkGNhaFcR3mqUIgf/JhtYDzg876+PYtyi+tBwICfXtI5nF+XRlhnnwwQK/xGUPQDAzZA enMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=k7qLTVFzV9pKc9OayGTtCUecEgYLFUwe2hitxu45nVE=; b=KbsrUutyt3dcIVDYmkr1Pf8m5oQcsiExAcfyeXt4606sJYocjojykWA53QzpxBz8Jc EXThzMM2/TZk1UYgBF2tZZ0VO4b9XRdM6nd0kWgnAAn6jxF1nb1kxDCqbNpJGpMyaccQ B1uDR3Cmm545OgwiQE9jLJbhJOlKyYD1LoVzufs7STudqQL5QPAwtetFoGt6ImY7/+Dd JB7wTLgijBM9pPNIM9T4RQXbh7vO1+Oiy/o4IWBhyhO5G12wbiOc8SZaGpPmfVSySQ4c uBYKVGxglILCeYBcSfkIN+AZyA3BMe/ERYuPmohMOiYqbhOk4KCuJLbGDRMuJJ3q7O2G 2OHQ== X-Gm-Message-State: AGRZ1gKvq42u/JxMPmpPT3tSOm+Js9jTNO5t3s/cK9WU7pSjBRH/ddDY LmFYWsrHq7Q4bnDdrTjFRO8nj+zj9l005t1myC2drm6k X-Google-Smtp-Source: AJdET5fUQJHKf3ev818jg6SUHNoem/JIzE/u+J6CYBwU9Ew4d26h2eO15YNpF+6+t5WDvd/OPwCYLM6cLaUzcj+U1Gk= X-Received: by 2002:ab0:490f:: with SMTP id z15mr1393484uac.81.1542415350214; Fri, 16 Nov 2018 16:42:30 -0800 (PST) MIME-Version: 1.0 References: <20181116231809.40a8f74c@bsdbuch.c0c0.intra> In-Reply-To: <20181116231809.40a8f74c@bsdbuch.c0c0.intra> From: Ben Woods Date: Sat, 17 Nov 2018 08:42:18 +0800 Message-ID: Subject: Re: eli encrypted providers for zfs raidz1 To: coco@executive-computing.de Cc: freebsd-geom@freebsd.org X-Rspamd-Queue-Id: BAD077147F X-Spamd-Result: default: False [-1.92 / 15.00]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-geom@freebsd.org]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCVD_IN_DNSWL_NONE(0.00)[9.2.9.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-0.91)[ipnet: 2607:f8b0::/32(-2.64), asn: 15169(-1.81), country: US(-0.10)] X-Rspamd-Server: mx1.freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Nov 2018 00:42:31 -0000 On Sat, 17 Nov 2018 at 06:36, Marco Steinbach wrote: > I'm trying to wrap my head around on how to avoid a zpool resilver on a > non-booting ZFS raidz1 of off four equally sized (GPT) partitions on > four distinct drives using eli for encyption. > > IOW: I do struggle with finding a way to attach all the > providers such, that ZFS does not initiate a resilver due to the > providers being attached sequentially. > ZFS doesn't auto-mount by itself - there are other elements at work doing this. 1. During boot, the /etc/rc.d/zfs script will call "zfs mount -va" to automount any available datasets. Note this only happens once during boot (not continuously after boot), and it happens *after* the /etc/rc.d/geli script has already attached any geli providers. 2. The daemon zfsd(8) was newly introduced in FreeBSD 11.0 will online any vdevs as new GEOM devices appear. It also handles other devctl(4) events. This daemon is not enabled by default - you have to specify zfsd_enable="YES" in /etc/rc.conf if you want this behaviour. I suspect your problem is the second items here. Have you enabled it? I've created and initialized the partitions as follows (generic > notation, comments on chosen encryption algo welome, since this > testing setup lacks AES-NI): > # gpart create -s gpt /dev/ada[2-5] > # gpart add -t freebsd-zfs /dev/ada[2-5] > # geli init -e AES-CBC -l 128 /dev/ada[2-5]p1 > I suspect you are just using the [2-5] as shorthand in your email here, and not running this exact command on FreeBSD 11.2. For reference, geli has been updated on FreeBSD 12.0 such that you could run this exact command to init multiple providers in a single command (they will all use the same password/file with unique salt). > Then I attached the geli partitions like so: > # geli attach /dev/ada[2-5]p1 > Same for geli attach - this could all be done in the one command like this on FreeBSD 12.0. > And finally created a raidz1 spanning all four partitions: > # zpool create u0001 raidz1 /dev/ada[2-5]p1.eli > > That works flawlessly. And naturally, after a reboot none of the > encrypted devices is available to the zpool then, unless I attach them. > You can configure the /etc/rc.d/geli script to attach additional devices (e.g. non-root ZFS) during userland boot. You can add the following to /etc/rc.conf: geli_devices="ada2p1 ada3p1 ada4p1 ada5p1" This will prompt you to type your password in 4 times during userland boot, and each would be attached one at a time. In FreeBSD 12.0, you can make this only prompt you for your password once by putting the following in /etc/rc.conf: geli_groups="u0001" geli_u0001_devices="ada2p1 ada3p1 ada4p1 ada5p1" > > Doing so using geli attach requires me to attach them sequentially, > which then results in ZFS resilvering the pool. > This would only occur if there was something automatically onlining the vdevs and mounting the datasets, such as zfsd. > So, here's my questions: > > 1. Is the inavoidable resilver intended behaviour based on current > implementation, or am I missing something ? > I suspect this is zfsd working against you in this instance. 2. In case I use a bootable zfsroot (cudos to allanjude@, I highly > recommend his BSDCan presentations on the matter), is there a way to > hand over the zfsroot passphrase to eli for automatically attaching > other providers ? > The geli attach for root ZFS devices would be handled by the boot loader, whilst the geli attach for non-root ZFS devices should be handled by the userland /etc/rc.d/geli script. Whilst you will need to type in your password separately for root and non-root devices, FreeBSD 12.0 at least makes it so you don't have to type the password multiple times for non-root devices. There might be a way to get the boot loader to handle the non-root devices also - I don't have experience with this. > Please note, that I'd like to stick as close as possible to what the > base system offers for this use-case. > > MfG CoCo > Good luck! -- From: Benjamin Woods woodsb02@gmail.com From owner-freebsd-geom@freebsd.org Sat Nov 17 11:03:23 2018 Return-Path: Delivered-To: freebsd-geom@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54EFF1130ADD for ; Sat, 17 Nov 2018 11:03:23 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A0F3785841 for ; Sat, 17 Nov 2018 11:03:21 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from moby.local ([79.107.11.8]) by mail.gmx.com (mrgmx002 [212.227.17.184]) with ESMTPSA (Nemesis) id 0MbPPQ-1g5Cq32utY-00IphS; Sat, 17 Nov 2018 11:50:15 +0100 From: Nikos Vassiliadis Subject: Re: eli encrypted providers for zfs raidz1 To: Marco Steinbach , freebsd-geom@freebsd.org References: <20181116231809.40a8f74c@bsdbuch.c0c0.intra> Message-ID: <0824ef45-642d-d8ff-c5e6-e627f9f18e0d@gmx.com> Date: Sat, 17 Nov 2018 12:50:09 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.3.0 MIME-Version: 1.0 In-Reply-To: <20181116231809.40a8f74c@bsdbuch.c0c0.intra> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K1:5L7X0wGkS5PQ36hjctLNvQ3eZ5KiETuVvhzgtFAYtistAnm8Kuj AfJeXIdhukIhsUFhl08DSE+2A23qOYzgpiDoNXagP0BAVwq7rIZ73eHx7I6i0Pu75peNd1v dlHciVQLeZsLzH02XxUR3WmIquprBNlNpG/JveBO+WJPiwbWPjOeC1CGBaB0HnVM18e8ISP k9/m6E4LNRFRcwFfjPi7g== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V01:K0:n9MCjWyk/Ak=:bDgwcFmyH6qQ3tKb5iaNeM IPfdC1KTl6WafpsX2TJU0LmI6C6bOwe/fpJxj2BClrq1Smhz7Fm+E54HOTkEuVJoZ5+MK5gL5 vOTdI8k9UIfEcjLYMmwAZyxLNgXYMSDX5zhzAgeUMYNTS3aHNrKfJQt18AnRlC0doc9clKzie Eje48H9/G6bI5eLjC5ry1n339hnEf7UGa+W8cuFkne+1nF62g19W89k9ensDOSEZtQ8zbzJo7 j403x0Kt8XpCGq2A/Px0JRk8oVElV2slesKJ1uD5mV8xqi9smmPhP6mHotW5hkMkz16+oBwbQ KF8VV5bwuN1Z1V7mZxrzUi5ZSLnr2WrbS8vS+zE7tYdCJsmMv4Bm+5yrZRNTROwkLHAwJj9w2 o8kc/lccebToNgQiM5gzmYjaULxU2xOPhX5XR5APkqY6oiY6QCeDgvtDHft2clJAuz+WZteae nJ/c3+DIU794hLofBogKzB5uuRiUT3xS22zVIVMs4evEkq3ZpDlRmFbLBP/0lZ+XTOk/vqClK Yv7bswUGk8jUv0bw42if1KytNv3FnYfvD+/+9MyXyu1YTqV55XBSdSSw13uS0W/Zi5W+bx2jw wyOrCgYYj7x68glDeshXs3DyYHMK2hXZ0/Wbgt/9zp6UFr/95BG13GiL7eATBSZCe+lq3ueHm pDT0cMkKBSpy9+zB7bcoK736Ap5tovMkiVGqRLcy4tpZP5TjrCwPdyVlfiILBzLqUSvwid+XG yQWcMBt+dJGKcg3p9IeXJi9LEWHSlW+VyVyPsywfjJH9kYOHoHA9I31RGUqoNKdwiqn4Gg1xh EnxR6Y3U8s+W6Y0HFe8deUleYr4afr/4cO7pQ0XsRWGZ4vcLWwcRfztLkOrcUgaJLU45aTCJ4 MUXsgJYuIKRbUByanDu0hZRSFQedorXrLvQ5RbHiEsxZfPSoqJ3R7SdkmSCWNv X-Rspamd-Queue-Id: A0F3785841 X-Spamd-Result: default: False [-0.97 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:212.227.15.0/24]; FREEMAIL_FROM(0.00)[gmx.com]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[gmx.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[mx00.gmx.net,mx01.gmx.net]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.52)[-0.515,0]; IP_SCORE(-0.05)[ipnet: 212.227.0.0/16(-0.56), asn: 8560(0.33), country: DE(-0.01)]; RCVD_IN_DNSWL_LOW(-0.10)[19.15.227.212.list.dnswl.org : 127.0.3.1]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmx.com]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[8.11.107.79.zen.spamhaus.org : 127.0.0.11] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Nov 2018 11:03:23 -0000 Hi Marco, On 11/17/18 12:18 AM, Marco Steinbach wrote: > Hi. > > I'm using 11.2-RELEASE r335510 amd64 GENERIC in an Oracle VirtualBox > setup on FreeBSD, which is what comes out of the box, when installing > 11.2 from the distribution media. > > > I'm trying to wrap my head around on how to avoid a zpool resilver on a > non-booting ZFS raidz1 of off four equally sized (GPT) partitions on > four distinct drives using eli for encyption. > > IOW: I do struggle with finding a way to attach all the > providers such, that ZFS does not initiate a resilver due to the > providers being attached sequentially. > > I've created and initialized the partitions as follows (generic > notation, comments on chosen encryption algo welome, since this > testing setup lacks AES-NI): > # gpart create -s gpt /dev/ada[2-5] > # gpart add -t freebsd-zfs /dev/ada[2-5] > # geli init -e AES-CBC -l 128 /dev/ada[2-5]p1 > > Then I attached the geli partitions like so: > # geli attach /dev/ada[2-5]p1 > > And finally created a raidz1 spanning all four partitions: > # zpool create u0001 raidz1 /dev/ada[2-5]p1.eli > > That works flawlessly. And naturally, after a reboot none of the > encrypted devices is available to the zpool then, unless I attach them. > > Doing so using geli attach requires me to attach them sequentially, > which then results in ZFS resilvering the pool. Why don't you just export the pool before shutting down? Since you already attach GELI manually, it'd make sense to import the pool manually as well. You could automate the import using devd and some scripting, that is, detect when all GELIs are there and then run zpool import. > So, here's my questions: > > 1. Is the inavoidable resilver intended behaviour based on current > implementation, or am I missing something ? It makes sense to resilver, given that ZFS will try to import the pool as soon as enough devices appear. I am not sure whether it is unavoidable though. > 2. In case I use a bootable zfsroot (cudos to allanjude@, I highly > recommend his BSDCan presentations on the matter), is there a way to > hand over the zfsroot passphrase to eli for automatically attaching > other providers ? > > Please note, that I'd like to stick as close as possible to what the > base system offers for this use-case. > > MfG CoCo > > _______________________________________________ > freebsd-geom@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-geom > To unsubscribe, send any mail to "freebsd-geom-unsubscribe@freebsd.org" >