From owner-freebsd-current@FreeBSD.ORG Mon Aug 18 20:47:40 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8456B234 for ; Mon, 18 Aug 2014 20:47:40 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 507063592 for ; Mon, 18 Aug 2014 20:47:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=8i3aLsFoUbUkWU+nAlhpraqSTDm2RqWV0FhkVJZW7Sw=; b=Fd/xQBifRJusr8ZGUV/NrvwrTtf3RGkw+kdbtbbSw97s8QYtqf/0XsVbiSDs7ZZEzhq5aW7CuGI7uQl1v5dmxiNu1zLUkbrfwiW8A49AapJFBqbon54tYkNLlqnFCeamNnLdPBw6Vqe/yM0XpUpeKCHO5Wyrapb7NrsQRfbKH68=; Received: from localhost.lerctr.org ([127.0.0.1]:18736 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.84 (FreeBSD)) (envelope-from ) id 1XJTq5-000K6L-8N; Mon, 18 Aug 2014 15:47:39 -0500 Received: from host.alcatel.com ([198.205.55.139]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Mon, 18 Aug 2014 15:47:37 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Mon, 18 Aug 2014 15:47:37 -0500 From: Larry Rosenman To: Ryan Stone Subject: Re: DEADLKRES crash In-Reply-To: References: <20140818152138.GA3481@borg.lerctr.org> Message-ID: <0915cd35b52b70eb2fb20765223fc7f1@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.2 X-Spam-Score: -3.6 (---) X-LERCTR-Spam-Score: -3.6 (---) X-Spam-Report: SpamScore (-3.6/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.668 X-LERCTR-Spam-Report: SpamScore (-3.6/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.668 Cc: FreeBSD Current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Aug 2014 20:47:40 -0000 On 2014-08-18 15:45, Ryan Stone wrote: > On Mon, Aug 18, 2014 at 11:21 AM, Larry Rosenman > wrote: >> I got the following: >> >> borg.lerctr.org dumped core - see /var/crash/vmcore.8 >> >> Mon Aug 18 07:30:42 CDT 2014 >> >> FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #63 >> r269784M: Sun Aug 10 12:33:07 CDT 2014 >> root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 >> >> panic: deadlkres: possible deadlock detected for 0xfffff8002abeb000, >> blocked for 1800926 ticks >> >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and >> you are >> welcome to change it and/or distribute copies of it under certain >> conditions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for >> details. >> This GDB was configured as "amd64-marcel-freebsd"... >> >> Unread portion of the kernel message buffer: >> panic: deadlkres: possible deadlock detected for 0xfffff8002abeb000, >> blocked for 1800926 ticks >> >> cpuid = 3 >> KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >> 0xfffffe100bff1a10 >> kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe100bff1ac0 >> vpanic() at vpanic+0x126/frame 0xfffffe100bff1b00 >> panic() at panic+0x43/frame 0xfffffe100bff1b60 >> deadlkres() at deadlkres+0x35c/frame 0xfffffe100bff1bb0 >> fork_exit() at fork_exit+0x84/frame 0xfffffe100bff1bf0 >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe100bff1bf0 >> --- trap 0, rip = 0, rsp = 0xfffffe100bff1cb0, rbp = 0 --- >> Uptime: 7d14h14m38s > > The first thing that I'd like to see is (in kgdb): > > set $td=(struct thread)0xfffff8002abeb000 > tid $td->td_tid > bt > > That will show us the backtrace of the thread that was blocked for so > long. 0 doadump (textdump=1) at pcpu.h:219 219 pcpu.h: No such file or directory. in pcpu.h (kgdb) set $td=(struct thread)0xfffff8002abeb000 Invalid cast. (kgdb) set $td=(struct thread*)0xfffff8002abeb000 Current language: auto; currently minimal (kgdb) tid $td->td_tid [Switching to thread 469 (Thread 100681)]#0 sched_switch ( td=0xfffff8002abeb000, newtd=, flags=) at /usr/src/sys/kern/sched_ule.c:1931 1931 cpuid = PCPU_GET(cpuid); (kgdb) bt #0 sched_switch (td=0xfffff8002abeb000, newtd=, flags=) at /usr/src/sys/kern/sched_ule.c:1931 #1 0xffffffff80a107d9 in mi_switch (flags=260, newtd=0x0) at /usr/src/sys/kern/kern_synch.c:493 #2 0xffffffff80a4c442 in sleepq_switch (wchan=, pri=) at /usr/src/sys/kern/subr_sleepqueue.c:552 #3 0xffffffff80a4c2a3 in sleepq_wait (wchan=0xfffff80070a4dd50, pri=96) at /usr/src/sys/kern/subr_sleepqueue.c:631 #4 0xffffffff809eb1fa in sleeplk (lk=, flags=, ilk=, wmesg=, pri=, timo=) at /usr/src/sys/kern/kern_lock.c:225 #5 0xffffffff809eaa06 in __lockmgr_args (lk=0xfffff80070a4dd50, flags=, ilk=0xfffff80070a4dd80, wmesg=, pri=, timo=) at /usr/src/sys/kern/kern_lock.c:931 #6 0xffffffff8092e092 in nfs_lock1 (ap=) at lockmgr.h:97 #7 0xffffffff80f2d57c in VOP_LOCK1_APV (vop=, a=) at vnode_if.c:2082 #8 0xffffffff80abd22a in _vn_lock (vp=0xfffff80070a4dce8, flags=, file=0xffffffff8110db88 "/usr/src/sys/kern/vfs_subr.c", line=2137) at vnode_if.h:859 #9 0xffffffff80aad4e7 in vget (vp=0xfffff80070a4dce8, flags=524544, ---Type to continue, or q to quit--- td=0xfffff8002abeb000) at /usr/src/sys/kern/vfs_subr.c:2137 #10 0xffffffff80aa1491 in vfs_hash_get (mp=0xfffff8002aa1e990, hash=1741450670, flags=, td=0xfffff8002abeb000, vpp=0xfffffe100c75c670, fn=0xffffffff80935820 ) at /usr/src/sys/kern/vfs_hash.c:88 #11 0xffffffff809314bd in ncl_nget (mntp=0xfffff8002aa1e990, fhp=0xfffff80070ccf4a4 "\001", fhsize=12, npp=0xfffffe100c75c6e0, lkflags=) at /usr/src/sys/fs/nfsclient/nfs_clnode.c:114 #12 0xffffffff809340fd in nfs_statfs (mp=0xfffff8002aa1e990, sbp=0xfffff8002aa1ea48) at /usr/src/sys/fs/nfsclient/nfs_clvfsops.c:288 #13 0xffffffff80aa7ade in __vfs_statfs (mp=0x0, sbp=0xfffff8002aa1ea48) at /usr/src/sys/kern/vfs_mount.c:1706 #14 0xffffffff80ab4f5e in kern_getfsstat (td=0xfffff8002abeb000, buf=, bufsize=, bufseg=UIO_USERSPACE, flags=) at /usr/src/sys/kern/vfs_syscalls.c:511 #15 0xffffffff80e1625a in amd64_syscall (td=0xfffff8002abeb000, traced=0) at subr_syscall.c:133 #16 0xffffffff80df760b in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:390 #17 0x00000008010fc83a in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688