From owner-freebsd-current@FreeBSD.ORG Thu Jan 29 22:32:38 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 13242106567D for ; Thu, 29 Jan 2009 22:32:38 +0000 (UTC) (envelope-from nakal@web.de) Received: from fmmailgate01.web.de (fmmailgate01.web.de [217.72.192.221]) by mx1.freebsd.org (Postfix) with ESMTP id 7FD398FC25 for ; Thu, 29 Jan 2009 22:32:37 +0000 (UTC) (envelope-from nakal@web.de) Received: from smtp05.web.de (fmsmtp05.dlan.cinetic.de [172.20.4.166]) by fmmailgate01.web.de (Postfix) with ESMTP id 16048FC75BAD; Thu, 29 Jan 2009 23:32:36 +0100 (CET) Received: from [217.236.2.27] (helo=zelda.local) by smtp05.web.de with asmtp (TLSv1:AES128-SHA:128) (WEB.DE 4.110 #277) id 1LSfR5-00069m-00; Thu, 29 Jan 2009 23:32:35 +0100 Date: Thu, 29 Jan 2009 23:32:20 +0100 From: Martin To: sbruno@freebsd.org Message-ID: <20090129233220.1ed64e6d@zelda.local> In-Reply-To: <1233007263.9302.2.camel@localhost.localdomain> References: <1233007263.9302.2.camel@localhost.localdomain> X-Mailer: Claws Mail 3.7.0 (GTK+ 2.14.7; amd64-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: nakal@web.de X-Sender: nakal@web.de X-Provags-ID: V01U2FsdGVkX19Pxn9kt1aPsbb4dTQzMpnUXa6XDz3ler2d6mt/ 560sHPLbWZPhk7xCx7Vx97Y0DssmoQwUdyORFG2pFpcqJpiGDg J/09KJkWU= Cc: sean.bruno@dsl-only.net, freebsd-current@freebsd.org Subject: Re: UFS Witness LoR + 5 other LoRs X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jan 2009 22:32:41 -0000 Am Mon, 26 Jan 2009 14:01:03 -0800 schrieb Sean Bruno : > Anyone else seeing this LoR today? > > lock order reversal: > 1st 0xc456b044 user map (user map) > @ /root/bsd/head/sys/vm/vm_map.c:3198 > 2nd 0xc4911ad0 ufs (ufs) @ /root/bsd/head/sys/kern/vfs_subr.c:2071 > KDB: stack backtrace: > db_trace_self_wrapper(c0bf0913,c417990c,c087a4b5,c0c1458c,0,...) at > db_trace_self_wrapper+0x26 > kdb_backtrace(c0c1458c,0,c4521728,c4526730,c4179968,...) at > kdb_backtrace+0x29 > _witness_debugger(c0bf3680,c4911ad0,c0be6d88,c4526730,c0bfa429,...) at > _witness_debugger+0x25 > witness_checkorder(c4911ad0,1,c0bfa429,817,0,...) at > witness_checkorder +0x839 > __lockmgr_args(c4911ad0,200501,c4911aec,0,0,...) at > __lockmgr_args+0x237 > ffs_lock(c4179a78,c087a25b,c0c1697a,200501,c4911a78,...) at ffs_lock > +0x8a VOP_LOCK1_APV(c0cf7ca0,c4179a78,c4567e24,c0d0bdc0,c4911a78,...) > at VOP_LOCK1_APV+0xb5 > _vn_lock(c4911a78,200501,c0bfa429,817,4,...) at _vn_lock+0x5e > vget(c4911a78,200501,c4567d80,4b4,0,...) at vget+0xc9 > vnode_pager_lock(c187c1f0,0,c0c13f07,127,c4179c18,...) at > vnode_pager_lock+0x1e0 > vm_fault(c456b000,80db000,2,8,80db620,...) at vm_fault+0x1df > trap_pfault(5,0,c0c241d1,2e7,c4565d34,...) at trap_pfault+0x118 > trap(c4179d38) at trap+0x289 > calltrap() at calltrap+0x6 > --- trap 0xc, eip = 0x80480e5, esp = 0xbfbfeef0, ebp = 0xbfbfef10 --- Hi, I have similar LORs. One of them might be affecting Gnome2. I cannot even save anything once the LOR happened. nautilus is also hanging around. I cannot even kill it. This occurs since I migrated my /usr/home to ZFS (see fourth, fifth LoR). Here one like yours: lock order reversal: 1st 0xffffff0002402070 user map (user map) @ /usr/src/sys/vm/vm_map.c:3198 2nd 0xffffff0002b8a7f8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2071 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e __lockmgr_args() at __lockmgr_args+0xca8 ffs_lock() at ffs_lock+0x8c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x47 vget() at vget+0x8b vnode_pager_lock() at vnode_pager_lock+0x1d0 vm_fault() at vm_fault+0x1e2 trap_pfault() at trap_pfault+0x128 trap() at trap+0x51c calltrap() at calltrap+0x8 --- trap 0xc, rip = 0x40014f, rsp = 0x7fffffffee70, rbp = 0x7fffffffee90 --- Here a second one: lock order reversal: 1st 0xfffffffe803eb610 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2443 2nd 0xffffff00080d2000 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:275 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 ufsdirhash_acquire() at ufsdirhash_acquire+0x33 ufsdirhash_add() at ufsdirhash_add+0x19 ufs_direnter() at ufs_direnter+0x88b ufs_mkdir() at ufs_mkdir+0x633 VOP_MKDIR_APV() at VOP_MKDIR_APV+0x93 kern_mkdirat() at kern_mkdirat+0x2b1 syscall() at syscall+0x1bf Xfast_syscall() at Xfast_syscall+0xab --- syscall (136, FreeBSD ELF64, mkdir), rip = 0x800719e9c, rsp = 0x7fffffffed08, rbp = 0x7fffffffef76 --- Here the third: lock order reversal: 1st 0xffffff0008648248 filedesc structure (filedesc structure) @ /usr/src/sys/kern/kern_descrip.c:1076 2nd 0xffffff0019110ba8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:4057 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e __lockmgr_args() at __lockmgr_args+0xc2a ffs_lock() at ffs_lock+0x8c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x47 knlist_remove_kq() at knlist_remove_kq+0x73 knote_fdclose() at knote_fdclose+0x177 kern_close() at kern_close+0xe6 syscall() at syscall+0x1bf Xfast_syscall() at Xfast_syscall+0xab --- syscall (6, FreeBSD ELF64, close), rip = 0x800e35e8c, rsp = 0x7fffffffe678, rbp = 0x801063a00 --- And thes two are really nasty (ZFS): lock order reversal: 1st 0xffffff00195ed620 zfs (zfs) @ /usr/src/sys/kern/vfs_vnops.c:531 2nd 0xffffff00080ab360 user map (user map) @ /usr/src/sys/vm/vm_map.c:3198 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_slock() at _sx_slock+0x55 vm_map_lookup() at vm_map_lookup+0x47 vm_fault() at vm_fault+0xfe trap_pfault() at trap_pfault+0x128 trap() at trap+0x347 calltrap() at calltrap+0x8 --- trap 0xc, rip = 0xffffffff8081ed4b, rsp = 0xfffffffe97261850, rbp = 0xfffffffe972618d0 --- copyout() at copyout+0x3b dmu_read_uio() at dmu_read_uio+0x98 zfs_freebsd_read() at zfs_freebsd_read+0x56d vn_read() at vn_read+0x267 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x54 syscall() at syscall+0x1bf Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x80647becc, rsp = 0x7ffffffed8c8, rbp = 0x806682f80 --- lock order reversal: 1st 0xffffff0019bc2270 zfs (zfs) @ /usr/src/sys/kern/vfs_subr.c:2071 2nd 0xffffff0019a77448 filedesc structure (filedesc structure) @ /usr/src/sys/kern/vfs_syscalls.c:1720 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_slock() at _sx_slock+0x55 kern_symlinkat() at kern_symlinkat+0x1f8 syscall() at syscall+0x1bf Xfast_syscall() at Xfast_syscall+0xab --- syscall (57, FreeBSD ELF64, symlink), rip = 0x8049cb13c, rsp = 0x7fffffffd6f8, rbp = 0x7fffffffdd10 --- And one with NFS: lock order reversal: 1st 0xffffff00080a6940 user map (user map) @ /usr/src/sys/vm/vm_map.c:3198 2nd 0xffffff00566a9ba8 nfs (nfs) @ /usr/src/sys/kern/vfs_subr.c:2071 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e __lockmgr_args() at __lockmgr_args+0xca8 vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x47 vget() at vget+0x8b vnode_pager_lock() at vnode_pager_lock+0x1d0 vm_fault() at vm_fault+0x1e2 trap_pfault() at trap_pfault+0x128 trap() at trap+0x347 calltrap() at calltrap+0x8 --- trap 0xc, rip = 0xffffffff8081edcd, rsp = 0xfffffffe972b67d0, rbp = 0xfffffffe972b6850 --- copyin() at copyin+0x3d ffs_write() at ffs_write+0x2f8 VOP_WRITE_APV() at VOP_WRITE_APV+0xfe vn_write() at vn_write+0x23f dofilewrite() at dofilewrite+0x85 kern_writev() at kern_writev+0x60 write() at write+0x54 syscall() at syscall+0x1bf Xfast_syscall() at Xfast_syscall+0xab --- syscall (4, FreeBSD ELF64, write), rip = 0x80072beac, rsp = 0x7fffffffe5d8, rbp = 0x800537000 --- Can you beat my 6 LoRs? ;) FreeBSD zelda.local 8.0-CURRENT FreeBSD 8.0-CURRENT #0: Thu Jan 29 22:15:16 CET 2009 root@zelda.local:/usr/obj/usr/src/sys/ZELDA amd64 -- Martin