From owner-freebsd-current@FreeBSD.ORG Sun Feb 8 10:44:52 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA1BB1065674; Sun, 8 Feb 2009 10:44:52 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from kennaway-macbookpro.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7AEA68FC24; Sun, 8 Feb 2009 10:44:50 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <498EB79F.4010905@FreeBSD.org> Date: Sun, 08 Feb 2009 11:44:47 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.19 (Macintosh/20081209) MIME-Version: 1.0 To: Martin References: <1233007263.9302.2.camel@localhost.localdomain> <20090129233220.1ed64e6d@zelda.local> In-Reply-To: <20090129233220.1ed64e6d@zelda.local> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: sean.bruno@dsl-only.net, sbruno@freebsd.org, freebsd-current@freebsd.org Subject: Re: UFS Witness LoR + 5 other LoRs X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Feb 2009 10:44:53 -0000 Martin wrote: > Am Mon, 26 Jan 2009 14:01:03 -0800 > schrieb Sean Bruno : > >> Anyone else seeing this LoR today? >> >> lock order reversal: >> 1st 0xc456b044 user map (user map) >> @ /root/bsd/head/sys/vm/vm_map.c:3198 >> 2nd 0xc4911ad0 ufs (ufs) @ /root/bsd/head/sys/kern/vfs_subr.c:2071 >> KDB: stack backtrace: >> db_trace_self_wrapper(c0bf0913,c417990c,c087a4b5,c0c1458c,0,...) at >> db_trace_self_wrapper+0x26 >> kdb_backtrace(c0c1458c,0,c4521728,c4526730,c4179968,...) at >> kdb_backtrace+0x29 >> _witness_debugger(c0bf3680,c4911ad0,c0be6d88,c4526730,c0bfa429,...) at >> _witness_debugger+0x25 >> witness_checkorder(c4911ad0,1,c0bfa429,817,0,...) at >> witness_checkorder +0x839 >> __lockmgr_args(c4911ad0,200501,c4911aec,0,0,...) at >> __lockmgr_args+0x237 >> ffs_lock(c4179a78,c087a25b,c0c1697a,200501,c4911a78,...) at ffs_lock >> +0x8a VOP_LOCK1_APV(c0cf7ca0,c4179a78,c4567e24,c0d0bdc0,c4911a78,...) >> at VOP_LOCK1_APV+0xb5 >> _vn_lock(c4911a78,200501,c0bfa429,817,4,...) at _vn_lock+0x5e >> vget(c4911a78,200501,c4567d80,4b4,0,...) at vget+0xc9 >> vnode_pager_lock(c187c1f0,0,c0c13f07,127,c4179c18,...) at >> vnode_pager_lock+0x1e0 >> vm_fault(c456b000,80db000,2,8,80db620,...) at vm_fault+0x1df >> trap_pfault(5,0,c0c241d1,2e7,c4565d34,...) at trap_pfault+0x118 >> trap(c4179d38) at trap+0x289 >> calltrap() at calltrap+0x6 >> --- trap 0xc, eip = 0x80480e5, esp = 0xbfbfeef0, ebp = 0xbfbfef10 --- > > Hi, > > I have similar LORs. One of them might be affecting Gnome2. I cannot > even save anything once the LOR happened. nautilus is also hanging > around. I cannot even kill it. This occurs since I migrated > my /usr/home to ZFS (see fourth, fifth LoR). > > Here one like yours: > > lock order reversal: > 1st 0xffffff0002402070 user map (user map) > @ /usr/src/sys/vm/vm_map.c:3198 2nd 0xffffff0002b8a7f8 ufs (ufs) > @ /usr/src/sys/kern/vfs_subr.c:2071 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > _witness_debugger() at _witness_debugger+0x2e > witness_checkorder() at witness_checkorder+0x81e > __lockmgr_args() at __lockmgr_args+0xca8 > ffs_lock() at ffs_lock+0x8c > VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b > _vn_lock() at _vn_lock+0x47 > vget() at vget+0x8b > vnode_pager_lock() at vnode_pager_lock+0x1d0 > vm_fault() at vm_fault+0x1e2 > trap_pfault() at trap_pfault+0x128 > trap() at trap+0x51c > calltrap() at calltrap+0x8 > --- trap 0xc, rip = 0x40014f, rsp = 0x7fffffffee70, rbp = > 0x7fffffffee90 --- > > > Here a second one: > > lock order reversal: > 1st 0xfffffffe803eb610 bufwait (bufwait) > @ /usr/src/sys/kern/vfs_bio.c:2443 2nd 0xffffff00080d2000 dirhash > (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:275 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > _witness_debugger() at _witness_debugger+0x2e > witness_checkorder() at witness_checkorder+0x81e > _sx_xlock() at _sx_xlock+0x55 > ufsdirhash_acquire() at ufsdirhash_acquire+0x33 > ufsdirhash_add() at ufsdirhash_add+0x19 > ufs_direnter() at ufs_direnter+0x88b > ufs_mkdir() at ufs_mkdir+0x633 > VOP_MKDIR_APV() at VOP_MKDIR_APV+0x93 > kern_mkdirat() at kern_mkdirat+0x2b1 > syscall() at syscall+0x1bf > Xfast_syscall() at Xfast_syscall+0xab > --- syscall (136, FreeBSD ELF64, mkdir), rip = 0x800719e9c, rsp = > 0x7fffffffed08, rbp = 0x7fffffffef76 --- > > > Here the third: > > lock order reversal: > 1st 0xffffff0008648248 filedesc structure (filedesc structure) > @ /usr/src/sys/kern/kern_descrip.c:1076 2nd 0xffffff0019110ba8 ufs > (ufs) @ /usr/src/sys/kern/vfs_subr.c:4057 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > _witness_debugger() at _witness_debugger+0x2e > witness_checkorder() at witness_checkorder+0x81e > __lockmgr_args() at __lockmgr_args+0xc2a > ffs_lock() at ffs_lock+0x8c > VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b > _vn_lock() at _vn_lock+0x47 > knlist_remove_kq() at knlist_remove_kq+0x73 > knote_fdclose() at knote_fdclose+0x177 > kern_close() at kern_close+0xe6 > syscall() at syscall+0x1bf > Xfast_syscall() at Xfast_syscall+0xab > --- syscall (6, FreeBSD ELF64, close), rip = 0x800e35e8c, rsp = > 0x7fffffffe678, rbp = 0x801063a00 --- > > > > And thes two are really nasty (ZFS): > > lock order reversal: > 1st 0xffffff00195ed620 zfs (zfs) @ /usr/src/sys/kern/vfs_vnops.c:531 > 2nd 0xffffff00080ab360 user map (user map) > @ /usr/src/sys/vm/vm_map.c:3198 KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > _witness_debugger() at _witness_debugger+0x2e > witness_checkorder() at witness_checkorder+0x81e > _sx_slock() at _sx_slock+0x55 > vm_map_lookup() at vm_map_lookup+0x47 > vm_fault() at vm_fault+0xfe > trap_pfault() at trap_pfault+0x128 > trap() at trap+0x347 > calltrap() at calltrap+0x8 > --- trap 0xc, rip = 0xffffffff8081ed4b, rsp = 0xfffffffe97261850, rbp = > 0xfffffffe972618d0 --- copyout() at copyout+0x3b > dmu_read_uio() at dmu_read_uio+0x98 > zfs_freebsd_read() at zfs_freebsd_read+0x56d > vn_read() at vn_read+0x267 > dofileread() at dofileread+0xa1 > kern_readv() at kern_readv+0x60 > read() at read+0x54 > syscall() at syscall+0x1bf > Xfast_syscall() at Xfast_syscall+0xab > --- syscall (3, FreeBSD ELF64, read), rip = 0x80647becc, rsp = > 0x7ffffffed8c8, rbp = 0x806682f80 --- > > lock order reversal: > 1st 0xffffff0019bc2270 zfs (zfs) @ /usr/src/sys/kern/vfs_subr.c:2071 > 2nd 0xffffff0019a77448 filedesc structure (filedesc structure) > @ /usr/src/sys/kern/vfs_syscalls.c:1720 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > _witness_debugger() at _witness_debugger+0x2e > witness_checkorder() at witness_checkorder+0x81e > _sx_slock() at _sx_slock+0x55 > kern_symlinkat() at kern_symlinkat+0x1f8 > syscall() at syscall+0x1bf > Xfast_syscall() at Xfast_syscall+0xab > --- syscall (57, FreeBSD ELF64, symlink), rip = 0x8049cb13c, rsp = > 0x7fffffffd6f8, rbp = 0x7fffffffdd10 --- > > And one with NFS: > > lock order reversal: > 1st 0xffffff00080a6940 user map (user map) > @ /usr/src/sys/vm/vm_map.c:3198 2nd 0xffffff00566a9ba8 nfs (nfs) > @ /usr/src/sys/kern/vfs_subr.c:2071 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > _witness_debugger() at _witness_debugger+0x2e > witness_checkorder() at witness_checkorder+0x81e > __lockmgr_args() at __lockmgr_args+0xca8 > vop_stdlock() at vop_stdlock+0x39 > VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b > _vn_lock() at _vn_lock+0x47 > vget() at vget+0x8b > vnode_pager_lock() at vnode_pager_lock+0x1d0 > vm_fault() at vm_fault+0x1e2 > trap_pfault() at trap_pfault+0x128 > trap() at trap+0x347 > calltrap() at calltrap+0x8 > --- trap 0xc, rip = 0xffffffff8081edcd, rsp = 0xfffffffe972b67d0, rbp = > 0xfffffffe972b6850 --- copyin() at copyin+0x3d > ffs_write() at ffs_write+0x2f8 > VOP_WRITE_APV() at VOP_WRITE_APV+0xfe > vn_write() at vn_write+0x23f > dofilewrite() at dofilewrite+0x85 > kern_writev() at kern_writev+0x60 > write() at write+0x54 > syscall() at syscall+0x1bf > Xfast_syscall() at Xfast_syscall+0xab > --- syscall (4, FreeBSD ELF64, write), rip = 0x80072beac, rsp = > 0x7fffffffe5d8, rbp = 0x800537000 --- > > > Can you beat my 6 LoRs? ;) Several of these are widely reported, and harmless (not sure about the filedesc ones though). Which one do you think is causing your problem? Kris