Date: Mon, 25 Apr 2016 18:51:05 -0400 From: "Michael B. Eichorn" <ike@michaeleichorn.com> To: Fabian Keil <freebsd-listen@fabiankeil.de>, freebsd-fs@freebsd.org Subject: Re: GELI + Zpool Scrub Results in GELI Device Destruction (and Later a Corrupt Pool) Message-ID: <1461624665.22294.103.camel@michaeleichorn.com> In-Reply-To: <20160425160625.5b585c8d@fabiankeil.de> References: <1461560445.22294.53.camel@michaeleichorn.com> <20160425101124.068c8333@fabiankeil.de> <1461587321.22294.85.camel@michaeleichorn.com> <20160425160625.5b585c8d@fabiankeil.de>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] On Mon, 2016-04-25 at 16:06 +0200, Fabian Keil wrote: > "Michael B. Eichorn" <ike@michaeleichorn.com> wrote: > > > > > On Mon, 2016-04-25 at 10:11 +0200, Fabian Keil wrote: > > > > > > "Michael B. Eichorn" <ike@michaeleichorn.com> wrote: > > > > > > > > > > > > > > > I just ran into something rather unexpected. I have a pool > > > > consisting > > > > of a mirrored pair of geli encrypted partitions on WD Red 3TB > > > > disks. > > > > > > > > The machine is running 10.3-RELEASE, the root zpool was setup > > > > with > > > > GELI > > > > encryption from the installer, the pool that is acting up was > > > > setup > > > > per > > > > the handbook. > > > [...] > > > > > > > > > > > > I had just noticed that I had failed to enable the zpool scrub > > > > periodic > > > > on this machine. So I began to run zpool scrub by hand. It > > > > succeeded > > > > for the root pool which is also geli encrypted, but when I ran > > > > it > > > > against my primary data pool I encountered: > > > > > > > > Apr 24 23:18:23 terra kernel: GEOM_ELI: Device ada3p1.eli > > > > destroyed. > > > > Apr 24 23:18:23 terra kernel: GEOM_ELI: Detached ada3p1.eli on > > > > last > > > > close. > > > > Apr 24 23:18:23 terra kernel: GEOM_ELI: Device ada2p1.eli > > > > destroyed. > > > > Apr 24 23:18:23 terra kernel: GEOM_ELI: Detached ada2p1.eli on > > > > last > > > > close. > > > Did you attach the devices using geli's -d (auto-detach) flag? > > I am using whatever the default setup comes out of the rc.d > > scripts. > > My rc.conf was: > > > > geli_devices="ada2p1 ada3p1" > > geli_default_flags="-k /root/encryption.key" > > zfs_enable="YES" > > > > I will try adding geli_autodetach="NO" and scubbing in about 9 > > hours. > On FreeBSD the default (set in /etc/defaults/rc.conf) is YES. Ah, I forgot about that file. For the record geli_autodetach="NO" did work properly. Interestingly, when I rebooted to test that rc.d modification when the pool came up it resilvered for a second time and now reports no errors at all. > > > > > > If yes, this is a known issue: > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=117158 > > > > > Reading that bug in detail it appears to be *specifically* for the > > kernel panic and that zfs closing and reopening providers is > > expected > > behavior, and that if geli has autodetach configured that it would > > detach. > > > > It stikes me that even though this is expected behavior it should > > not > > be. Is there a way we could prevent the detach when zfs does closes > > and > > reopens providers? I cannnot think of a case where the desired > > behavior > > is for the pool to detach when zfs wants to reopen it. > I suppose geli could delay the detachment a bit to give the consumer > a chance to reopen it. That would probably work, but my inner engineer is dissatisfied with the idea of slowing down all detachments to solve a single case. What about a new feature whereby zfs (or any other consumer) can inhibit the autodetach for a brief period? I don't really know if this is feasible, but I thought I would ask. Anything like this is probably too major to implement without some thought and probably consensus building. So in the meantime I will file my bug against the handbook to make sure geli_autodetach and zfs are mentioned. [-- Attachment #2 --] 0 *H 010 `He 0 *H 000]0 *H 01 0 UIL10U StartCom Ltd.1+0)U "Secure Digital Certificate Signing1806U/StartCom Class 1 Primary Intermediate Client CA0 150613202446Z 160614003550Z0H1 0 U ike@michaeleichorn.com1%0# *H ike@michaeleichorn.com0"0 *H 0 UՀ,k9D %Z|Y6J<rrK g;&|uNlUE9)V.[ט̊:qS](#v SYDz* CpugYݔ,v<`j(w aS#ڒ6n(K5'KVLåErv< J=[}W bLA%gޭnVb| I?M7D :$׃bM_T[,ƃ\ 00 U 0 0 U 0 U %0++0 U jj: γ+39啖0 U #0Sr풜\|~5NԸQ0!U 0ike@michaeleichorn.com0LU C0?0; +70*0.+"http://www.startssl.com/policy.pdf0+00' StartCom Certification Authority0This certificate was issued according to the Class 1 Validation requirements of the StartCom CA policy, reliance only for the intended purpose in compliance of the relying party obligations.06U /0-0+)'%http://crl.startssl.com/crtu1-crl.crl0+009+0-http://ocsp.startssl.com/sub/class1/client/ca0B+06http://aia.startssl.com/certs/sub.class1.client.ca.crt0#U 0http://www.startssl.com/0 *H x+ȐF}pw.X vF?rg P]EOp)L˻yA ;hi0u2]m [Sbp$_ gr Xm*YP3 #H>mKAǠt)HO|=@} 3ӝ'iO81> 03 v'h5U "H;ECZtpҗ4rWHu^6+i*kJL8shAV|5;?HMc\ j[j|+000]0 *H 01 0 UIL10U StartCom Ltd.1+0)U "Secure Digital Certificate Signing1806U/StartCom Class 1 Primary Intermediate Client CA0 150613202446Z 160614003550Z0H1 0 U ike@michaeleichorn.com1%0# *H ike@michaeleichorn.com0"0 *H 0 UՀ,k9D %Z|Y6J<rrK g;&|uNlUE9)V.[ט̊:qS](#v SYDz* CpugYݔ,v<`j(w aS#ڒ6n(K5'KVLåErv< J=[}W bLA%gޭnVb| I?M7D :$׃bM_T[,ƃ\ 00 U 0 0 U 0 U %0++0 U jj: γ+39啖0 U #0Sr풜\|~5NԸQ0!U 0ike@michaeleichorn.com0LU C0?0; +70*0.+"http://www.startssl.com/policy.pdf0+00' StartCom Certification Authority0This certificate was issued according to the Class 1 Validation requirements of the StartCom CA policy, reliance only for the intended purpose in compliance of the relying party obligations.06U /0-0+)'%http://crl.startssl.com/crtu1-crl.crl0+009+0-http://ocsp.startssl.com/sub/class1/client/ca0B+06http://aia.startssl.com/certs/sub.class1.client.ca.crt0#U 0http://www.startssl.com/0 *H x+ȐF}pw.X vF?rg P]EOp)L˻yA ;hi0u2]m [Sbp$_ gr Xm*YP3 #H>mKAǠt)HO|=@} 3ӝ'iO81> 03 v'h5U "H;ECZtpҗ4rWHu^6+i*kJL8shAV|5;?HMc\ j[j|+040 0 *H 0}1 0 UIL10U StartCom Ltd.1+0)U "Secure Digital Certificate Signing1)0'U StartCom Certification Authority0 071024210155Z 171024210155Z01 0 UIL10U StartCom Ltd.1+0)U "Secure Digital Certificate Signing1806U/StartCom Class 1 Primary Intermediate Client CA0"0 *H 0 -).2AUGo#G B|NDRpM-B=o -we5JQpa>O.#._<V [~**pz~3WG .ᘟMlr[<Ce6fqO"uxfWN#uicgkv$Lb%y`_{`xK'GN 00U 00U 0 U Sr풜\|~5NԸQ0 U #0N @[i04hCA0f+Z0X0'+0http://ocsp.startssl.com/ca0-+0!http://www.startssl.com/sfsca.crt0[U T0R0'%#!http://www.startssl.com/sfsca.crl0'%#!http://crl.startssl.com/sfsca.crl0U y0w0u +70f0.+"http://www.startssl.com/policy.pdf04+(http://www.startssl.com/intermediate.pdf0 *H }x,\c^ #wMq}>UK/^yX֏y frMIŲB61ymQҨݬZ0& ;@#13qۑ& ̢o 6r_;GO>*I( 74XS1r3)!LJy6Kotˆ# _wSr ;B ADp(fs 䰷6%.W0J3:bC<8t X1<Cn=t==wST~\wkBf|15zUP)(IjVB!OfI=bb\4-*em/нSJm7N[]'@ڽ D9Kr>R7/|o^I@ټ'Pa $ z9a'L)( I}vcH]۸D*W} m>Q|C.(,lQ10{001 0 UIL10U StartCom Ltd.1+0)U "Secure Digital Certificate Signing1806U/StartCom Class 1 Primary Intermediate Client CA]0 `He 0 *H 1 *H 0 *H 1 160425225105Z0/ *H 1" bʙt 7''&qTz]iG0 +71001 0 UIL10U StartCom Ltd.1+0)U "Secure Digital Certificate Signing1806U/StartCom Class 1 Primary Intermediate Client CA]0 *H 101 0 UIL10U StartCom Ltd.1+0)U "Secure Digital Certificate Signing1806U/StartCom Class 1 Primary Intermediate Client CA]0 *H pUU* [~= zs dxll;p$O 3`Y pvg Zˤ bԔ+ ,]mb YҽOdYgZXՔ6=0K Z`rU>f?[3HB c#~2EQ<D@pkLa:$Yk5#?L-M '*V
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1461624665.22294.103.camel>