From owner-freebsd-current@FreeBSD.ORG Thu Oct 11 17:39:12 2007 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 401B016A417 for ; Thu, 11 Oct 2007 17:39:12 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id 6E1AB13C4A7 for ; Thu, 11 Oct 2007 17:39:11 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 4C9D245EE5; Thu, 11 Oct 2007 19:39:09 +0200 (CEST) Received: from localhost (154.81.datacomsa.pl [195.34.81.154]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id B7CCD45CD9; Thu, 11 Oct 2007 19:39:02 +0200 (CEST) Date: Thu, 11 Oct 2007 19:38:46 +0200 From: Pawel Jakub Dawidek To: Fabian Keil Message-ID: <20071011173846.GB52831@garage.freebsd.pl> References: <20071011190517.770fd3ab@fabiankeil.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="1LKvkjL3sHcu1TtY" Content-Disposition: inline In-Reply-To: <20071011190517.770fd3ab@fabiankeil.de> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-current@FreeBSD.org Subject: Re: zpool scrub causes geli vdev to detach X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Oct 2007 17:39:12 -0000 --1LKvkjL3sHcu1TtY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 11, 2007 at 07:05:17PM +0200, Fabian Keil wrote: > On: >=20 > FreeBSD TP51.local 7.0-CURRENT FreeBSD 7.0-CURRENT > #0: Thu Oct 11 09:21:19 CEST 2007 > fk@TP51.local:/usr/obj/usr/src/sys/THINKPAD i386 >=20 > this zpool: >=20 > fk@TP51 ~ $sudo zpool status > pool: tank > state: ONLINE > scrub: scrub completed with 0 errors on Thu Oct 11 13:56:28 2007 > config: >=20 > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 0 > ad0s3f.eli ONLINE 0 0 0 > ad0s2.eli ONLINE 0 0 0 >=20 > errors: No known data errors >=20 > worked without issues when it only consisted of ad0s3f.eli, > but then I added ad0s2.eli and now scrubbing the pool leads to: >=20 > Unread portion of the kernel message buffer: > GEOM_ELI: Detached ad0s2.eli on last close. > GEOM_LABEL: Label for provider ad0s2 is msdosfs/?A.?,{(#0. > panic: Function g_eli_orphan_spoil_assert() called for ad0s3f.eli. > KDB: enter: panic > panic: from debugger > Uptime: 5m27s > Physical memory: 1014 MB > Dumping 120 MB: 105 89 73 57 41 25 9 >=20 > #0 doadump () at pcpu.h:195 > 195 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) bt =20 > #0 doadump () at pcpu.h:195 > #1 0xc05db8f3 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c= :409 > #2 0xc05dbb1c in panic (fmt=3DVariable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:563 > #3 0xc04a40c7 in db_panic (addr=3DCould not find the frame base for "db_= panic". > ) at /usr/src/sys/ddb/db_command.c:433 > #4 0xc04a4825 in db_command_loop () at /usr/src/sys/ddb/db_command.c:401 > #5 0xc04a6255 in db_trap (type=3D3, code=3D0) at /usr/src/sys/ddb/db_mai= n.c:222 > #6 0xc06014a4 in kdb_trap (type=3D3, code=3D0, tf=3D0xf4bcac1c) at /usr/= src/sys/kern/subr_kdb.c:502 > #7 0xc0838d2b in trap (frame=3D0xf4bcac1c) at /usr/src/sys/i386/i386/tra= p.c:621 > #8 0xc082120b in calltrap () at /usr/src/sys/i386/i386/exception.s:139 > #9 0xc0601602 in kdb_enter (msg=3D0xc0889693 "panic") at cpufunc.h:60 > #10 0xc05dbb05 in panic (fmt=3D0xc3e9a0e8 "Function %s() called for %s.")= at /usr/src/sys/kern/kern_shutdown.c:547 > #11 0xc3e92d65 in ?? () > #12 0xc3e9a0e8 in ?? () > #13 0xc3e99d84 in ?? () > #14 0xc3e0d890 in ?? () > #15 0xf4bcacac in ?? () > #16 0xc058eff5 in g_spoil_event (arg=3D0xc3c92940, flag=3D-945983104) at = /usr/src/sys/geom/geom_subr.c:903 > (kgdb) f 16 > #16 0xc058eff5 in g_spoil_event (arg=3D0xc3c92940, flag=3D-945983104) at = /usr/src/sys/geom/geom_subr.c:903 > 903 cp->geom->spoiled(cp); > (kgdb) l > 898 if (!cp->spoiled) > 899 continue; > 900 cp->spoiled =3D 0; > 901 if (cp->geom->spoiled =3D=3D NULL) > 902 continue; > 903 cp->geom->spoiled(cp); > 904 g_topology_assert(); > 905 } > 906 } > 907 =20 >=20 > As a workaround I'm currently using: >=20 > --- sys/geom/eli/g_eli.c.orig 2007-10-11 13:35:53.744592827 +0200 > +++ sys/geom/eli/g_eli.c 2007-10-11 13:38:41.687509241 +0200 > @@ -451,10 +451,14 @@ > gp =3D sc->sc_geom; > pp =3D LIST_FIRST(&gp->provider); > strlcpy(ppname, pp->name, sizeof(ppname)); > + > + G_ELI_DEBUG(0, "Ignored request to detach %s on last close.", ppn= ame); > + /* > error =3D g_eli_destroy(sc, 1); > KASSERT(error =3D=3D 0, ("Cannot detach %s on last close (error= =3D%d).", > ppname, error)); > G_ELI_DEBUG(0, "Detached %s on last close.", ppname); > + */ > } > =20 > int >=20 > and with that scrubbing finishes without errors (see status above), > but triggers the G_ELI_DEBUG message once or twice: >=20 > Oct 11 13:46:30 TP51 sudo: fk : TTY=3Dttyv0 ; PWD=3D/home/fk ; USER= =3Droot ; COMMAND=3D/etc/rc.d/zfs start > Oct 11 13:46:31 TP51 kernel: WARNING: ZFS is considered to be an experime= ntal feature in FreeBSD. > Oct 11 13:46:31 TP51 kernel: ZFS filesystem version 6 > Oct 11 13:46:31 TP51 kernel: ZFS storage pool version 6 > Oct 11 13:46:32 TP51 kernel: GEOM_ELI: Ignored request to detach ad0s3f.e= li on last close. > Oct 11 13:46:32 TP51 last message repeated 3 times > Oct 11 13:46:32 TP51 kernel: GEOM_ELI: Ignored request to detach ad0s2.el= i on last close. > Oct 11 13:46:44 TP51 sudo: fk : TTY=3Dttyv0 ; PWD=3D/home/fk ; USER= =3Droot ; COMMAND=3D/sbin/zpool status > Oct 11 13:47:12 TP51 sudo: fk : TTY=3Dttyv0 ; PWD=3D/home/fk ; USER= =3Droot ; COMMAND=3D/sbin/zpool scrub tank > Oct 11 13:47:13 TP51 kernel: GEOM_ELI: Ignored request to detach ad0s3f.e= li on last close. > Oct 11 13:47:13 TP51 kernel: GEOM_ELI: Ignored request to detach ad0s3f.e= li on last close. >=20 > Not sure if it helps, but the zdevs are: >=20 > Geom name: ad0s3f.eli > EncryptionAlgorithm: AES-CBC > KeyLength: 128 > Crypto: software > UsedKey: 0 > Flags: RW-DETACH, W-OPEN > Providers: > 1. Name: ad0s3f.eli > Mediasize: 8881561600 (8.3G) > Sectorsize: 4096 > Mode: r1w1e1 > Consumers: > 1. Name: ad0s3f > Mediasize: 8881565184 (8.3G) > Sectorsize: 512 > Mode: r1w1e1 >=20 > Geom name: ad0s2.eli > EncryptionAlgorithm: AES-CBC > KeyLength: 128 > Crypto: software > UsedKey: 0 > Flags: RW-DETACH, W-OPEN > Providers: > 1. Name: ad0s2.eli > Mediasize: 9932263424 (9.3G) > Sectorsize: 4096 > Mode: r1w1e1 > Consumers: > 1. Name: ad0s2 > Mediasize: 9932267520 (9.3G) > Sectorsize: 512 > Mode: r1w1e1 >=20 >=20 > Is there an obvious solution, or should I file a PR about this? Please file PR, but let me explain. GELI's detach-on-last-close mechanism is a general purpose mechanism, it may not work correctly with ZFS, because ZFS sometimes closes and reopen providers, which will make GELI to detach. In other words you shouldn't configure detach-on-last-close for ZFS components. It shouldn't panic still. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --1LKvkjL3sHcu1TtY Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHDl+lForvXbEpPzQRAshYAJ9pD710CNquZxAXbJat2euiJ9a3vwCgtHSi 6BzACCky85JWyCU9EgmpPzE= =/NZ4 -----END PGP SIGNATURE----- --1LKvkjL3sHcu1TtY--