From owner-freebsd-stable@FreeBSD.ORG Wed Sep 11 21:18:36 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AD9E3834 for ; Wed, 11 Sep 2013 21:18:36 +0000 (UTC) (envelope-from olgeni@olgeni.com) Received: from olgeni.olgeni.com (host-156-246-171-31.cloudsigma.com [31.171.246.156]) by mx1.freebsd.org (Postfix) with ESMTP id 73FEA291C for ; Wed, 11 Sep 2013 21:18:35 +0000 (UTC) Received: from olgeni.olgeni (unknown [82.84.82.178]) by olgeni.olgeni.com (Postfix) with ESMTPSA id C6B60174482; Wed, 11 Sep 2013 23:18:34 +0200 (CEST) Date: Wed, 11 Sep 2013 23:18:34 +0200 (CEST) From: Jimmy Olgeni X-X-Sender: olgeni@olgeni.olgeni To: Konstantin Belousov Subject: Re: Possible kqueue related issue on STABLE/RC. In-Reply-To: <20130911171913.GG41229@kib.kiev.ua> Message-ID: References: <20130911171913.GG41229@kib.kiev.ua> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Sep 2013 21:18:36 -0000 Hi, On Wed, 11 Sep 2013, Konstantin Belousov wrote: > Also, do you have all options listed at > http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kerneldebug-deadlocks.html > enabled ? This time I tried with clang + these options and I got something more interesting. All works fine until the lock violation below: acquiring duplicate lock of same type: "os.lock_mtx" 1st os.lock_mtx @ nvidia_os.c:748 2nd os.lock_mtx @ nvidia_os.c:748 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffff8360e1e2f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xffffff8360e1e3a0 witness_checkorder() at witness_checkorder+0xc0a/frame 0xffffff8360e1e420 _mtx_lock_flags() at _mtx_lock_flags+0x74/frame 0xffffff8360e1e460 os_acquire_spinlock() at os_acquire_spinlock+0x17/frame 0xffffff8360e1e470 _nv012281rm() at _nv012281rm+0x9/frame 0xffffff800cfadec0 lock order reversal: 1st 0xfffffe003603c098 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:1240 2nd 0xfffffe003603b848 syncer (syncer) @ /usr/src/sys/kern/vfs_subr.c:2335 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffff8360bf3660 kdb_backtrace() at kdb_backtrace+0x39/frame 0xffffff8360bf3710 witness_checkorder() at witness_checkorder+0xc0a/frame 0xffffff8360bf3790 __lockmgr_args() at __lockmgr_args+0x744/frame 0xffffff8360bf38b0 vop_stdlock() at vop_stdlock+0x3c/frame 0xffffff8360bf38d0 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xbe/frame 0xffffff8360bf3900 _vn_lock() at _vn_lock+0x63/frame 0xffffff8360bf3960 vputx() at vputx+0x34b/frame 0xffffff8360bf39c0 dounmount() at dounmount+0x282/frame 0xffffff8360bf3a30 sys_unmount() at sys_unmount+0x3a6/frame 0xffffff8360bf3b20 amd64_syscall() at amd64_syscall+0x259/frame 0xffffff8360bf3c30 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xffffff8360bf3c30 --- syscall (22, FreeBSD ELF64, sys_unmount), rip = 0x801918a7c, rsp = 0x7fffffffbf18, rbp = 0x802818800 --- KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffff83611df6b0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xffffff83611df760 assert_vop_elocked() at assert_vop_elocked+0x6a/frame 0xffffff83611df790 fifo_open() at fifo_open+0x38/frame 0xffffff83611df810 VOP_OPEN_APV() at VOP_OPEN_APV+0xd1/frame 0xffffff83611df840 vn_open_cred() at vn_open_cred+0x532/frame 0xffffff83611df9b0 kern_openat() at kern_openat+0x1c1/frame 0xffffff83611dfb20 amd64_syscall() at amd64_syscall+0x259/frame 0xffffff83611dfc30 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xffffff83611dfc30 Here it goes: --- syscall (5, FreeBSD ELF64, sys_open), rip = 0x800db3d3c, rsp = 0x7fffffffc968, rbp = 0 --- fifo_open: 0xfffffe0063251cd0 is not exclusive locked but should be KDB: enter: lock violation 0xfffffe0063251cd0: tag zfs, type VFIFO usecount 2, writecount 0, refcount 2 mountedhere 0xfffffe005048cc00 flags (VI_ACTIVE) lock type zfs: SHARED (count 1) #0 0xffffffff808b22fa at __lockmgr_args+0xeba #1 0xffffffff8095df4c at vop_stdlock+0x3c #2 0xffffffff80d7f6ae at VOP_LOCK1_APV+0xbe #3 0xffffffff80980153 at _vn_lock+0x63 #4 0xffffffff8096e321 at vget+0xa1 #5 0xffffffff80959921 at cache_lookup_times+0x591 #6 0xffffffff8095aa7d at vfs_cache_lookup+0x9d #7 0xffffffff80d7c881 at VOP_LOOKUP_APV+0xd1 #8 0xffffffff80963aff at lookup+0x6bf #9 0xffffffff80963029 at namei+0x589 #10 0xffffffff8097fa0f at vn_open_cred+0x27f #11 0xffffffff80978031 at kern_openat+0x1c1 #12 0xffffffff80ccce49 at amd64_syscall+0x259 #13 0xffffffff80cb5e9b at Xfast_syscall+0xfb , fifo with 0 readers and 1 writers 0xfffffe00a21d3000: tag zfs, type VREG usecount 4, writecount 0, refcount 5 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffffe00a0f71bc8 ref 3 pages 150 lock type zfs: SHARED (count 1) #0 0xffffffff808b22fa at __lockmgr_args+0xeba #1 0xffffffff8095df4c at vop_stdlock+0x3c #2 0xffffffff80d7f6ae at VOP_LOCK1_APV+0xbe #3 0xffffffff80980153 at _vn_lock+0x63 #4 0xffffffff8096e321 at vget+0xa1 #5 0xffffffff80b4e1de at vm_fault_hold+0x5ee #6 0xffffffff80b4dba7 at vm_fault+0x77 #7 0xffffffff80ccc85b at trap_pfault+0x1bb #8 0xffffffff80ccbf92 at trap+0x512 #9 0xffffffff80cb5bb2 at calltrap+0x8 0xfffffe0195389290: tag zfs, type VREG usecount 4, writecount 0, refcount 5 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffffe000f2df740 ref 3 pages 223 lock type zfs: SHARED (count 1) #0 0xffffffff808b22fa at __lockmgr_args+0xeba #1 0xffffffff8095df4c at vop_stdlock+0x3c #2 0xffffffff80d7f6ae at VOP_LOCK1_APV+0xbe #3 0xffffffff80980153 at _vn_lock+0x63 #4 0xffffffff8096e321 at vget+0xa1 #5 0xffffffff80b4e1de at vm_fault_hold+0x5ee #6 0xffffffff80b4dba7 at vm_fault+0x77 #7 0xffffffff80ccc85b at trap_pfault+0x1bb #8 0xffffffff80ccbf92 at trap+0x512 #9 0xffffffff80cb5bb2 at calltrap+0x8 0xfffffe00a2207000: tag zfs, type VREG usecount 4, writecount 0, refcount 5 mountedhere 0 flags (VI_ACTIVE) v_object 0xfffffe000f0b8d98 ref 3 pages 201 lock type zfs: SHARED (count 1) #0 0xffffffff808b22fa at __lockmgr_args+0xeba #1 0xffffffff8095df4c at vop_stdlock+0x3c #2 0xffffffff80d7f6ae at VOP_LOCK1_APV+0xbe #3 0xffffffff80980153 at _vn_lock+0x63 #4 0xffffffff8096e321 at vget+0xa1 #5 0xffffffff80b4e1de at vm_fault_hold+0x5ee #6 0xffffffff80b4dba7 at vm_fault+0x77 #7 0xffffffff80ccc85b at trap_pfault+0x1bb #8 0xffffffff80ccbf92 at trap+0x512 #9 0xffffffff80cb5bb2 at calltrap+0x8 -- jimmy