From owner-freebsd-fs@freebsd.org Mon Apr 25 14:51:33 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC535B1BF35 for ; Mon, 25 Apr 2016 14:51:33 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay03.ispgateway.de (smtprelay03.ispgateway.de [80.67.31.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7B3D419A2 for ; Mon, 25 Apr 2016 14:51:32 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from [78.35.175.160] (helo=fabiankeil.de) by smtprelay03.ispgateway.de with esmtpsa (TLSv1.2:AES128-GCM-SHA256:128) (Exim 4.84) (envelope-from ) id 1auhMQ-000185-9i for freebsd-fs@freebsd.org; Mon, 25 Apr 2016 16:19:38 +0200 Date: Mon, 25 Apr 2016 16:06:25 +0200 From: Fabian Keil To: freebsd-fs@freebsd.org Subject: Re: GELI + Zpool Scrub Results in GELI Device Destruction (and Later a Corrupt Pool) Message-ID: <20160425160625.5b585c8d@fabiankeil.de> In-Reply-To: <1461587321.22294.85.camel@michaeleichorn.com> References: <1461560445.22294.53.camel@michaeleichorn.com> <20160425101124.068c8333@fabiankeil.de> <1461587321.22294.85.camel@michaeleichorn.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/Awxbp/X4B=qp6AnH7BO5GJy"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Apr 2016 14:51:33 -0000 --Sig_/Awxbp/X4B=qp6AnH7BO5GJy Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable "Michael B. Eichorn" wrote: > On Mon, 2016-04-25 at 10:11 +0200, Fabian Keil wrote: > > "Michael B. Eichorn" wrote: > > =20 > > >=20 > > > I just ran into something rather unexpected. I have a pool > > > consisting > > > of a mirrored pair of geli encrypted partitions on WD Red 3TB > > > disks. > > >=20 > > > The machine is running 10.3-RELEASE, the root zpool was setup with > > > GELI > > > encryption from the installer, the pool that is acting up was setup > > > per > > > the handbook. =20 > > [...] =20 > > >=20 > > > I had just noticed that I had failed to enable the zpool scrub > > > periodic > > > on this machine. So I began to run zpool scrub by hand. It > > > succeeded > > > for the root pool which is also geli encrypted, but when I ran it > > > against my primary data pool I encountered: > > >=20 > > > Apr 24 23:18:23 terra kernel: GEOM_ELI: Device ada3p1.eli > > > destroyed. > > > Apr 24 23:18:23 terra kernel: GEOM_ELI: Detached ada3p1.eli on last > > > close. > > > Apr 24 23:18:23 terra kernel: GEOM_ELI: Device ada2p1.eli > > > destroyed. > > > Apr 24 23:18:23 terra kernel: GEOM_ELI: Detached ada2p1.eli on last > > > close. =20 > > Did you attach the devices using geli's -d (auto-detach) flag? >=20 > I am using whatever the default setup comes out of the rc.d scripts. > My rc.conf was: >=20 > geli_devices=3D"ada2p1 ada3p1" > geli_default_flags=3D"-k /root/encryption.key" > zfs_enable=3D"YES" >=20 > I will try adding geli_autodetach=3D"NO" and scubbing in about 9 hours. On FreeBSD the default (set in /etc/defaults/rc.conf) is YES. > > If yes, this is a known issue: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D117158 > > =20 >=20 > Reading that bug in detail it appears to be *specifically* for the > kernel panic and that zfs closing and reopening providers is expected > behavior, and that if geli has autodetach configured that it would > detach. >=20 > It stikes me that even though this is expected behavior it should not > be. Is there a way we could prevent the detach when zfs does closes and > reopens providers? I cannnot think of a case where the desired behavior > is for the pool to detach when zfs wants to reopen it. I suppose geli could delay the detachment a bit to give the consumer a chance to reopen it. Fabian --Sig_/Awxbp/X4B=qp6AnH7BO5GJy Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEUEARECAAYFAlceJGEACgkQBYqIVf93VJ0dIACgto8ufhTsp8yx2efZjeN5a0ze 3u0AmPLxV2QspWG57OTC1ADtc9/YZ0U= =7XHt -----END PGP SIGNATURE----- --Sig_/Awxbp/X4B=qp6AnH7BO5GJy--