From owner-freebsd-current@freebsd.org Fri Nov 6 18:15:51 2015 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A88DA27C17 for ; Fri, 6 Nov 2015 18:15:51 +0000 (UTC) (envelope-from lidl@pix.net) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.pix.net", Issuer "Pix.Com Technologies, LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E840C1C85 for ; Fri, 6 Nov 2015 18:15:50 +0000 (UTC) (envelope-from lidl@pix.net) Received: from torb.pix.net (vzw.pix.net [71.178.232.3]) (authenticated bits=0) by hydra.pix.net (8.15.2/8.15.2) with ESMTPA id tA6IFnZT085800; Fri, 6 Nov 2015 13:15:49 -0500 (EST) (envelope-from lidl@pix.net) X-Authentication-Warning: hydra.pix.net: Host vzw.pix.net [71.178.232.3] claimed to be torb.pix.net Subject: Re: panic in zfs on reboot To: FreeBSD-Current References: <563CED7C.4050904@pix.net> From: Kurt Lidl Message-ID: <563CEE55.1060800@pix.net> Date: Fri, 6 Nov 2015 13:15:49 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <563CED7C.4050904@pix.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 18:15:51 -0000 On 11/6/15 1:12 PM, Kurt Lidl wrote: > I updated my machine that tracks head and rebooted it, > and it panic'd during the 'shutdown -r' execution. > > Panic String: solaris assert: zrl->zr_refcount == 0 (0x2 == 0x0), > file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zrlock.c, > line: 64 > > (kgdb) bt > #0 doadump (textdump=1) at pcpu.h:221 > #1 0xffffffff80708ec5 in kern_reboot (howto=260) > at /usr/src/sys/kern/kern_shutdown.c:364 > #2 0xffffffff8070949b in vpanic (fmt=, > ap=) at /usr/src/sys/kern/kern_shutdown.c:757 > #3 0xffffffff807094e3 in panic (fmt=0x0) > at /usr/src/sys/kern/kern_shutdown.c:688 > #4 0xffffffff81b3c25f in assfail3 (a=, > lv=, op=, > rv=, f=, l= optimized out>) > at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:91 > #5 0xffffffff8187fe54 in zrl_destroy (zrl=0xfffff8001d429820) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zrlock.c:64 > #6 0xffffffff81804432 in dnode_special_close (dnh=0xfffff8001d429820) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c:1005 > #7 0xffffffff817fa608 in dmu_objset_evict_done (os=0xfffff8001d429800) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:726 > #8 0xffffffff8180f191 in dsl_dataset_evict (dbu=0xfffff8000f957000) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c:287 > #9 0xffffffff80756ae0 in taskqueue_run_locked (queue=0xfffff8000d00cd00) > at /usr/src/sys/kern/subr_taskqueue.c:430 > #10 0xffffffff807575f8 in taskqueue_thread_loop (arg=) > at /usr/src/sys/kern/subr_taskqueue.c:683 > #11 0xffffffff806cf6b4 in fork_exit ( > callout=0xffffffff80757570 , > arg=0xfffff8000d01e020, frame=0xfffffe085c058ac0) > at /usr/src/sys/kern/kern_fork.c:1011 > #12 0xffffffff809ae70e in fork_trampoline () > at /usr/src/sys/amd64/amd64/exception.S:609 > #13 0x0000000000000000 in ?? () > > I have a vmcore for this panic. I have saved off the installed kernel > directory for future examination. > > -Kurt I should have copied in the whole info.2 file: Dump header from device: /dev/gpt/swap0 Architecture: amd64 Architecture Version: 2 Dump Length: 1404231680 Blocksize: 512 Dumptime: Fri Nov 6 12:10:25 2015 Hostname: busybox.pix.net Magic: FreeBSD Kernel Dump Version String: FreeBSD 11.0-CURRENT #86 r290447M: Fri Nov 6 11:00:18 EST 2015 lidl@busybox.pix.net:/usr/obj/usr/src/sys/BUSYBOX Panic String: solaris assert: zrl->zr_refcount == 0 (0x2 == 0x0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zrlock.c, line: 64 Dump Parity: 375889703 Bounds: 2 Dump Status: good The only delta in my source tree on this machine is a patch to the pagezero.S file, which I've been running for several months, so I know it's not that. (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199151 for the delta, if you care. I need to benchmark this more fully, but just haven't gotten around to doing it yet.) -Kurt