From owner-freebsd-fs@FreeBSD.ORG Tue Sep 9 19:04:44 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A23CA100 for ; Tue, 9 Sep 2014 19:04:44 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 303EC89F for ; Tue, 9 Sep 2014 19:04:44 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 01FA820E7088E; Tue, 9 Sep 2014 19:04:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: * X-Spam-Status: No, score=1.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTP id C465720E7088B; Tue, 9 Sep 2014 19:04:39 +0000 (UTC) Message-ID: <55240AD4BA84415EBCE5B5B2F9DC22F3@multiplay.co.uk> From: "Steven Hartland" To: "Mike Carlson" , References: Subject: Re: ZFS Panic (3rd time) Date: Tue, 9 Sep 2014 20:04:45 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 19:04:44 -0000 You panic is being cause by the dereference of a null vdev in vdev_rele but the issue seems to start earlier as the zio value passed to vdev_mirror_scrub_done also looks invalid. The call point for this should be in zio_done specifically if (zio->io_done) zio->io_done(zio); So if zio is actually invalid then something is really wrong which is unfortunatelly not much help :( With regards zdb try not passing a pool. ----- Original Message ----- From: "Mike Carlson" snip... > > #7 0xffffffff81860336 in vdev_rele (vd=0x0) at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:1556 > #8 0xffffffff81866800 in vdev_mirror_scrub_done (zio=0x3) at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c:193 > #9 0xffffffff81866344 in vdev_mirror_io_start (zio=0xfffff80142733d00) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c:286 > #10 0xffffffff818803c4 in zio_vdev_io_start (zio=0xfffff8013eb20b10) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2571 > #11 0xffffffff8187d796 in zio_suspend (spa=0xfffff8000e122000, > zio=0xfffff8013eb20b10) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1456 > #12 0xffffffff8180ddec in arc_read (pio=0x0, spa=0xfffff8001e37d000, > bp=, done=0x2, private=0x0, priority=6, zio_flags=0, > arc_flags=, zb=0xfffff8001ed06558) at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3371 > #13 0xffffffff818268f1 in traverse_prefetcher (spa=0xfffff8001e37d000, > zilog=0xf01ff, bp=, zb=, > dnp=0xfffff80142733d00, arg=) at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:451 > #14 0xffffffff81825d14 in traverse_visitbp (td=0xfffffe104c763900, > dnp=0xfffffe000fe64800, bp=0xfffffe000fe64980, zb=0xfffffe104c762e88) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:250 > #15 0xffffffff8182677f in traverse_dnode (td=0xfffffe104c763900, > dnp=0xfffffe000fe64800, objset=110, object=26823324) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:417 > #16 0xffffffff81826487 in traverse_visitbp (td=0xfffffe104c763900, > dnp=0xfffffe000fe61000, bp=0xfffffe001285ea00, zb=0xfffffe104c7630a8) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:309 > #17 0xffffffff81825ee3 in traverse_visitbp (td=0xfffffe104c763900, > dnp=0xfffff8013ed8f000, bp=0xfffffe0012867a00, zb=0xfffffe104c7631d8) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:284 > #18 0xffffffff81825ee3 in traverse_visitbp (td=0xfffffe104c763900, > dnp=0xfffff8013ed8f000, bp=0xfffffe0012842980, zb=0xfffffe104c763308) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:284 > #19 0xffffffff81825ee3 in traverse_visitbp (td=0xfffffe104c763900, > dnp=0xfffff8013ed8f000, bp=0xfffffe0012848000, zb=0xfffffe104c763438) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:284 > #20 0xffffffff81825ee3 in traverse_visitbp (td=0xfffffe104c763900, > dnp=0xfffff8013ed8f000, bp=0xfffffe000fe6d000, zb=0xfffffe104c763568) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:284 > #21 0xffffffff81825ee3 in traverse_visitbp (td=0xfffffe104c763900, > dnp=0xfffff8013ed8f000, bp=0xfffffe001282d000, zb=0xfffffe104c763698) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:284 > #22 0xffffffff81825ee3 in traverse_visitbp (td=0xfffffe104c763900, > dnp=0xfffff8013ed8f000, bp=0xfffff8013ed8f040, zb=0xfffffe104c763758) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:284 > #23 0xffffffff81826714 in traverse_dnode (td=0xfffffe104c763900, > dnp=0xfffff8013ed8f000, objset=110, object=0) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:407 > #24 0xffffffff81826190 in traverse_visitbp (td=0xfffffe104c763900, dnp=0x0, > bp=0xfffff8013ed7ea80, zb=0xfffffe104c7638e0) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:338 > #25 0xffffffff81825af6 in traverse_prefetch_thread (arg=0xfffffe104cd2f0e0) > at > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:470 > #26 0xffffffff817fcc00 in taskq_run (arg=0xfffff801445bef30, pending=983551) > at > /usr/src/sys/modules/zfs/../../cddl/compat/opensolaris/kern/opensolaris_taskq.c:109 > #27 0xffffffff808f5c26 in taskqueue_run_locked (queue=0xfffff8000e0eba00) > at /usr/src/sys/kern/subr_taskqueue.c:333 > #28 0xffffffff808f64a8 in taskqueue_thread_loop (arg=) > at /usr/src/sys/kern/subr_taskqueue.c:535 > #29 0xffffffff80881a4a in fork_exit (callout=0xffffffff808f6400 > , arg=0xfffff8000e10aac0, frame=0xfffffe104c763a40) > at /usr/src/sys/kern/kern_fork.c:995 > #30 0xffffffff80c75a6e in fork_trampoline () at > /usr/src/sys/amd64/amd64/exception.S:606 > #31 0x0000000000000000 in ?? () > Current language: auto; currently minimal > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >