From owner-freebsd-stable@FreeBSD.ORG Mon Jul 21 14:28:01 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E73B01065677 for ; Mon, 21 Jul 2008 14:28:01 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206045140.chello.pl [87.206.45.140]) by mx1.freebsd.org (Postfix) with ESMTP id 7341F8FC0C for ; Mon, 21 Jul 2008 14:28:00 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id CDF3545B36; Mon, 21 Jul 2008 16:27:59 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id EDC6345683; Mon, 21 Jul 2008 16:27:54 +0200 (CEST) Date: Mon, 21 Jul 2008 16:28:00 +0200 From: Pawel Jakub Dawidek To: Daniel Eriksson Message-ID: <20080721142800.GB6310@garage.freebsd.pl> References: <4F9C9299A10AE74E89EA580D14AA10A61A197A@royal64.emp.zapto.org> <20080719221813.GC4733@garage.freebsd.pl> <4F9C9299A10AE74E89EA580D14AA10A61A197E@royal64.emp.zapto.org> <20080721090235.GA2953@garage.freebsd.pl> <4F9C9299A10AE74E89EA580D14AA10A61A197F@royal64.emp.zapto.org> <20080721135156.GA6310@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="rS8CxjVDS/+yyDmU" Content-Disposition: inline In-Reply-To: <20080721135156.GA6310@garage.freebsd.pl> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-stable@freebsd.org Subject: Re: Panic on ZFS startup after crash X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2008 14:28:02 -0000 --rS8CxjVDS/+yyDmU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 21, 2008 at 03:51:56PM +0200, Pawel Jakub Dawidek wrote: > On Mon, Jul 21, 2008 at 03:49:24PM +0200, Daniel Eriksson wrote: > > Pawel Jakub Dawidek wrote: > >=20 > > > I'm afraid your pool's metadata is > > > somehow corrupted that ZFS can't handle that. > >=20 > > Yes, that's my conclusion also. It looks like the intent log is messed > > up enough to trigger an assert while ZFS tries to parse/replay it. > >=20 > > > I saw warnings in your > > > first e-mail about ZFS not beeing able to replay ZIL. Can you try > > > disabling ZIL? Something like: > >=20 > > I've already tried this, and it made no difference. When the box crashed > > ZIL was enabled, and for some reason garbage got written into the ZIL. > > Now whenever ZFS tries to import the pool it sees a non-empty ZIL and > > tries to parse/replay it. > >=20 > > Is there an easy way to trick ZFS into thinking the ZIL is empty? >=20 > I'll check that. Ok. We may try not to replay the ZIL, but leave it there and see what will happen. We can also try to destroy the ZIL without replaying it. What we do from now on can mess up your pool even further, so you may want to backup entire disks if you want. To skip replaying the ZIL you need to edit /sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c file, find zil_replay() function and make the head of it looks like this: void zil_replay(objset_t *os, void *arg, uint64_t *txgp, zil_replay_func_t *replay_func[TX_MAX_TYPE]) { zilog_t *zilog =3D dmu_objset_zil(os); const zil_header_t *zh =3D zilog->zl_header; zil_replay_arg_t zr; /* XXX: Try to skip the ZIL replay. */ return; if (zil_empty(zilog)) { zil_destroy(zilog, B_TRUE); return; } [...] If that won't work, we can try to destroy the ZIL without replaying it: void zil_replay(objset_t *os, void *arg, uint64_t *txgp, zil_replay_func_t *replay_func[TX_MAX_TYPE]) { zilog_t *zilog =3D dmu_objset_zil(os); const zil_header_t *zh =3D zilog->zl_header; zil_replay_arg_t zr; /* XXX: Destroy the ZIL without replaying it. */ zil_destroy(zilog, B_FALSE); return; if (zil_empty(zilog)) { zil_destroy(zilog, B_TRUE); return; } [...] --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --rS8CxjVDS/+yyDmU Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFIhJzvForvXbEpPzQRArjZAKCnJagjVv6KQPxySdMmtTefeL8/0gCgwXcv Nkz76RFnstg5jhQjieJQ+2E= =iDjn -----END PGP SIGNATURE----- --rS8CxjVDS/+yyDmU--