Date: Thu, 11 Oct 2007 19:38:46 +0200 From: Pawel Jakub Dawidek <pjd@FreeBSD.org> To: Fabian Keil <freebsd-listen@fabiankeil.de> Cc: freebsd-current@FreeBSD.org Subject: Re: zpool scrub causes geli vdev to detach Message-ID: <20071011173846.GB52831@garage.freebsd.pl> In-Reply-To: <20071011190517.770fd3ab@fabiankeil.de> References: <20071011190517.770fd3ab@fabiankeil.de>
next in thread | previous in thread | raw e-mail | index | archive | help
--1LKvkjL3sHcu1TtY
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Thu, Oct 11, 2007 at 07:05:17PM +0200, Fabian Keil wrote:
> On:
>=20
> FreeBSD TP51.local 7.0-CURRENT FreeBSD 7.0-CURRENT
> #0: Thu Oct 11 09:21:19 CEST 2007
> fk@TP51.local:/usr/obj/usr/src/sys/THINKPAD i386
>=20
> this zpool:
>=20
> fk@TP51 ~ $sudo zpool status
> pool: tank
> state: ONLINE
> scrub: scrub completed with 0 errors on Thu Oct 11 13:56:28 2007
> config:
>=20
> NAME STATE READ WRITE CKSUM
> tank ONLINE 0 0 0
> ad0s3f.eli ONLINE 0 0 0
> ad0s2.eli ONLINE 0 0 0
>=20
> errors: No known data errors
>=20
> worked without issues when it only consisted of ad0s3f.eli,
> but then I added ad0s2.eli and now scrubbing the pool leads to:
>=20
> Unread portion of the kernel message buffer:
> GEOM_ELI: Detached ad0s2.eli on last close.
> GEOM_LABEL: Label for provider ad0s2 is msdosfs/?A.?,{(#0.
> panic: Function g_eli_orphan_spoil_assert() called for ad0s3f.eli.
> KDB: enter: panic
> panic: from debugger
> Uptime: 5m27s
> Physical memory: 1014 MB
> Dumping 120 MB: 105 89 73 57 41 25 9
>=20
> #0 doadump () at pcpu.h:195
> 195 pcpu.h: No such file or directory.
> in pcpu.h
> (kgdb) bt =20
> #0 doadump () at pcpu.h:195
> #1 0xc05db8f3 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c=
:409
> #2 0xc05dbb1c in panic (fmt=3DVariable "fmt" is not available.
> ) at /usr/src/sys/kern/kern_shutdown.c:563
> #3 0xc04a40c7 in db_panic (addr=3DCould not find the frame base for "db_=
panic".
> ) at /usr/src/sys/ddb/db_command.c:433
> #4 0xc04a4825 in db_command_loop () at /usr/src/sys/ddb/db_command.c:401
> #5 0xc04a6255 in db_trap (type=3D3, code=3D0) at /usr/src/sys/ddb/db_mai=
n.c:222
> #6 0xc06014a4 in kdb_trap (type=3D3, code=3D0, tf=3D0xf4bcac1c) at /usr/=
src/sys/kern/subr_kdb.c:502
> #7 0xc0838d2b in trap (frame=3D0xf4bcac1c) at /usr/src/sys/i386/i386/tra=
p.c:621
> #8 0xc082120b in calltrap () at /usr/src/sys/i386/i386/exception.s:139
> #9 0xc0601602 in kdb_enter (msg=3D0xc0889693 "panic") at cpufunc.h:60
> #10 0xc05dbb05 in panic (fmt=3D0xc3e9a0e8 "Function %s() called for %s.")=
at /usr/src/sys/kern/kern_shutdown.c:547
> #11 0xc3e92d65 in ?? ()
> #12 0xc3e9a0e8 in ?? ()
> #13 0xc3e99d84 in ?? ()
> #14 0xc3e0d890 in ?? ()
> #15 0xf4bcacac in ?? ()
> #16 0xc058eff5 in g_spoil_event (arg=3D0xc3c92940, flag=3D-945983104) at =
/usr/src/sys/geom/geom_subr.c:903
> (kgdb) f 16
> #16 0xc058eff5 in g_spoil_event (arg=3D0xc3c92940, flag=3D-945983104) at =
/usr/src/sys/geom/geom_subr.c:903
> 903 cp->geom->spoiled(cp);
> (kgdb) l
> 898 if (!cp->spoiled)
> 899 continue;
> 900 cp->spoiled =3D 0;
> 901 if (cp->geom->spoiled =3D=3D NULL)
> 902 continue;
> 903 cp->geom->spoiled(cp);
> 904 g_topology_assert();
> 905 }
> 906 }
> 907 =20
>=20
> As a workaround I'm currently using:
>=20
> --- sys/geom/eli/g_eli.c.orig 2007-10-11 13:35:53.744592827 +0200
> +++ sys/geom/eli/g_eli.c 2007-10-11 13:38:41.687509241 +0200
> @@ -451,10 +451,14 @@
> gp =3D sc->sc_geom;
> pp =3D LIST_FIRST(&gp->provider);
> strlcpy(ppname, pp->name, sizeof(ppname));
> +
> + G_ELI_DEBUG(0, "Ignored request to detach %s on last close.", ppn=
ame);
> + /*
> error =3D g_eli_destroy(sc, 1);
> KASSERT(error =3D=3D 0, ("Cannot detach %s on last close (error=
=3D%d).",
> ppname, error));
> G_ELI_DEBUG(0, "Detached %s on last close.", ppname);
> + */
> }
> =20
> int
>=20
> and with that scrubbing finishes without errors (see status above),
> but triggers the G_ELI_DEBUG message once or twice:
>=20
> Oct 11 13:46:30 TP51 sudo: fk : TTY=3Dttyv0 ; PWD=3D/home/fk ; USER=
=3Droot ; COMMAND=3D/etc/rc.d/zfs start
> Oct 11 13:46:31 TP51 kernel: WARNING: ZFS is considered to be an experime=
ntal feature in FreeBSD.
> Oct 11 13:46:31 TP51 kernel: ZFS filesystem version 6
> Oct 11 13:46:31 TP51 kernel: ZFS storage pool version 6
> Oct 11 13:46:32 TP51 kernel: GEOM_ELI: Ignored request to detach ad0s3f.e=
li on last close.
> Oct 11 13:46:32 TP51 last message repeated 3 times
> Oct 11 13:46:32 TP51 kernel: GEOM_ELI: Ignored request to detach ad0s2.el=
i on last close.
> Oct 11 13:46:44 TP51 sudo: fk : TTY=3Dttyv0 ; PWD=3D/home/fk ; USER=
=3Droot ; COMMAND=3D/sbin/zpool status
> Oct 11 13:47:12 TP51 sudo: fk : TTY=3Dttyv0 ; PWD=3D/home/fk ; USER=
=3Droot ; COMMAND=3D/sbin/zpool scrub tank
> Oct 11 13:47:13 TP51 kernel: GEOM_ELI: Ignored request to detach ad0s3f.e=
li on last close.
> Oct 11 13:47:13 TP51 kernel: GEOM_ELI: Ignored request to detach ad0s3f.e=
li on last close.
>=20
> Not sure if it helps, but the zdevs are:
>=20
> Geom name: ad0s3f.eli
> EncryptionAlgorithm: AES-CBC
> KeyLength: 128
> Crypto: software
> UsedKey: 0
> Flags: RW-DETACH, W-OPEN
> Providers:
> 1. Name: ad0s3f.eli
> Mediasize: 8881561600 (8.3G)
> Sectorsize: 4096
> Mode: r1w1e1
> Consumers:
> 1. Name: ad0s3f
> Mediasize: 8881565184 (8.3G)
> Sectorsize: 512
> Mode: r1w1e1
>=20
> Geom name: ad0s2.eli
> EncryptionAlgorithm: AES-CBC
> KeyLength: 128
> Crypto: software
> UsedKey: 0
> Flags: RW-DETACH, W-OPEN
> Providers:
> 1. Name: ad0s2.eli
> Mediasize: 9932263424 (9.3G)
> Sectorsize: 4096
> Mode: r1w1e1
> Consumers:
> 1. Name: ad0s2
> Mediasize: 9932267520 (9.3G)
> Sectorsize: 512
> Mode: r1w1e1
>=20
>=20
> Is there an obvious solution, or should I file a PR about this?
Please file PR, but let me explain.
GELI's detach-on-last-close mechanism is a general purpose mechanism, it
may not work correctly with ZFS, because ZFS sometimes closes and reopen
providers, which will make GELI to detach. In other words you shouldn't
configure detach-on-last-close for ZFS components. It shouldn't panic
still.
--=20
Pawel Jakub Dawidek http://www.wheel.pl
pjd@FreeBSD.org http://www.FreeBSD.org
FreeBSD committer Am I Evil? Yes, I Am!
--1LKvkjL3sHcu1TtY
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4 (FreeBSD)
iD8DBQFHDl+lForvXbEpPzQRAshYAJ9pD710CNquZxAXbJat2euiJ9a3vwCgtHSi
6BzACCky85JWyCU9EgmpPzE=
=/NZ4
-----END PGP SIGNATURE-----
--1LKvkjL3sHcu1TtY--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20071011173846.GB52831>