From owner-freebsd-fs@FreeBSD.ORG Fri Dec 26 14:49:46 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FB11F71 for ; Fri, 26 Dec 2014 14:49:46 +0000 (UTC) Received: from smtprelay02.ispgateway.de (smtprelay02.ispgateway.de [80.67.18.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 176663609 for ; Fri, 26 Dec 2014 14:49:45 +0000 (UTC) Received: from [78.35.187.37] (helo=fabiankeil.de) by smtprelay02.ispgateway.de with esmtpsa (TLSv1.2:AES128-GCM-SHA256:128) (Exim 4.84) (envelope-from ) id 1Y4W6h-0000Ij-Gb for freebsd-fs@freebsd.org; Fri, 26 Dec 2014 15:43:11 +0100 Date: Fri, 26 Dec 2014 15:43:12 +0100 From: Fabian Keil To: Subject: Panic after vdev loss: assert: zap_update([...]) == 0 (0x6 == 0x0), [...]/zfs/dsl_scan.c, line: 41 Message-ID: <51ee5a33.776435f0@fabiankeil.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/xZAQEilkmebo3PF2ju4WqC4"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Dec 2014 14:49:46 -0000 --Sig_/xZAQEilkmebo3PF2ju4WqC4 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Yesterday I got a panic after a zpool that was being scrubbed lost its (only) vdev: [6507] GEOM_ELI: g_eli_read_done() failed (error=3D5) label/extreme.eli[REA= D(offset=3D2507528704, length=3D12800)] [6507] GEOM_ELI: g_eli_read_done() failed (error=3D5) label/extreme.eli[REA= D(offset=3D2507494912, length=3D33792)] [6507] (da1:umass-sim1:1:0:0): READ(10). CDB: 28 00 00 4a bb 06 00 00 05 00= =20 [6507] (da1:umass-sim1:1:0:0): CAM status: CCB request completed with an er= ror [6507] (da1:umass-sim1:1:0:0): Retrying command [6507] (da1:umass-sim1:1:0:0): READ(10). CDB: 28 00 00 4a bb 06 00 00 05 00= =20 [6507] (da1:umass-sim1:1:0:0): CAM status: CCB request completed with an er= ror [6507] (da1:umass-sim1:1:0:0): Retrying command [6507] (da1:umass-sim1:1:0:0): READ(10). CDB: 28 00 00 4a bb 06 00 00 05 00= =20 [6507] (da1:umass-sim1:1:0:0): CAM status: CCB request completed with an er= ror [6507] (da1:umass-sim1:1:0:0): Retrying command [6507] (da1:umass-sim1:1:0:0): READ(10). CDB: 28 00 00 4a bb 06 00 00 05 00= =20 [6507] (da1:umass-sim1:1:0:0): CAM status: CCB request completed with an er= ror [6507] (da1:umass-sim1:1:0:0): Retrying command [6507] (da1:umass-sim1:1:0:0): READ(10). CDB: 28 00 00 4a bb 06 00 00 05 00= =20 [6507] (da1:umass-sim1:1:0:0): CAM status: CCB request completed with an er= ror [6507] (da1:umass-sim1:1:0:0): Error 5, Retries exhausted [6507] GEOM_ELI: g_eli_read_done() failed (error=3D5)(da1:umass-sim1:1:0:0)= : READ(10). CDB: 28 00 00 37 25 ad 00 00 6a 00=20 [6507] label/extreme.eli[READ(offset=3D4022607872, length=3D8192)](da1:uma= ss-sim1:1:0:0): CAM status: CCB request completed with an error [6507]=20 [6507] (da1:GEOM_ELIumass-sim1:1:: g_eli_read_done() failed (error=3D5)0: 0= ): label/extreme.eli[READ(offset=3D4022870016, length=3D8192)]Retrying comm= and [6507]=20 [6507] GEOM_ELI: g_eli_read_done() failed (error=3D5) label/extreme.eli[REA= D(offset=3D270336, length=3D8192)](da1:umass-sim1:1:0:0): READ(10). CDB: 28= 00 00 37 25 ad 00 00 6a 00=20 [6507]=20 [6507] GEOM_ELI(da1:umass-sim1:1:0:0): CAM status: CCB request completed wi= th an error [6507] : g_eli_read_done() failed (error=3D5)(da1: umass-sim1:1:label/extre= me.eli[READ(offset=3D2507541504, length=3D2560)]0: [6507] 0): Retrying command [6507] (da1:umass-sim1:1:0:0): READ(10). CDB: 28 00 00 37 25 ad 00 00 6a 00= =20 [6507] (da1:umass-sim1:1:0:0): CAM status: CCB request completed with an er= ror [6507] (da1:umass-sim1:1:0:0): Retrying command [6507] (da1:umass-sim1:1:0:0): READ(10). CDB: 28 00 00 37 25 ad 00 00 6a 00= =20 [6507] (da1:umass-sim1:1:0:0): CAM status: CCB request completed with an er= ror [6507] (da1:umass-sim1:1:0:0): Retrying command [6507] (da1:umass-sim1:1:0:0): READ(10). CDB: 28 00 00 37 25 ad 00 00 6a 00= =20 [6507] (da1:umass-sim1:1:0:0): CAM status: CCB request completed with an er= ror [6507] (da1:umass-sim1:1:0:0): Error 5, Retries exhausted [6507] GEOM_ELI: g_eli_read_done() failed (error=3D5) label/extreme.eli[REA= D(offset=3D1850432000, length=3D54272)] [6507] da1 at umass-sim1 bus 1 scbus3 target 0 lun 0 [6507] da1: s/n 8123201007= 08 detached [6507] pass3 at umass-sim1 bus 1 scbus3 target 0 lun 0 [6507] pass3: s/n 81232010= 0708 detached [6507] (pass3:umass-sim1:1:0:0): Periph destroyed [6507] panic: solaris assert: zap_update(scn->scn_dp->dp_meta_objset, 1, "s= can", sizeof (uint64_t), (sizeof (dsl_scan_phys_t) / sizeof (uint64_t)), &s= cn->scn_phys, tx) =3D=3D 0 (0x6 =3D=3D 0x0), file: /usr/src/sys/cddl/contri= b/opensolaris/uts/common/fs/zfs/dsl_scan.c, line: 41 [6507] cpuid =3D 0 [6507] KDB: stack backtrace: [6507] db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe= 0095157520 [6507] kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe00951575d0 [6507] panic() at panic+0x1c1/frame 0xfffffe0095157690 [6507] assfail3() at assfail3+0x2f/frame 0xfffffe00951576b0 [6507] dsl_scan_sync() at dsl_scan_sync+0xa83/frame 0xfffffe0095157a00 [6507] spa_sync() at spa_sync+0x5c1/frame 0xfffffe0095157ae0 [6507] txg_sync_thread() at txg_sync_thread+0x3a6/frame 0xfffffe0095157bb0 [6507] fork_exit() at fork_exit+0x9a/frame 0xfffffe0095157bf0 [6507] fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0095157bf0 [6507] --- trap 0, rip =3D 0, rsp =3D 0xfffffe0095157cb0, rbp =3D 0 --- [6507] KDB: enter: panic The assertion in dsl_scan_sync_state() seems to expect that the pool is available and apparently spa->spa_state was still POOL_STATE_ACTIVE. Additional details: http://www.fabiankeil.de/bilder/freebsd/kernel-panic-r275748-zfs/ Fabian --Sig_/xZAQEilkmebo3PF2ju4WqC4 Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlSddAAACgkQBYqIVf93VJ22ugCfcO8r7FQhgz2c0yvRlh1V5SC9 YegAoK7ghP6yhMubF4bpQFHxJPEch34x =R2Jn -----END PGP SIGNATURE----- --Sig_/xZAQEilkmebo3PF2ju4WqC4--