From owner-freebsd-current@FreeBSD.ORG Tue Dec 18 22:11:23 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6CF1016A41B for ; Tue, 18 Dec 2007 22:11:23 +0000 (UTC) (envelope-from emaste@freebsd.org) Received: from gw.sandvine.com (gw.sandvine.com [199.243.201.138]) by mx1.freebsd.org (Postfix) with ESMTP id 603F213C43E for ; Tue, 18 Dec 2007 22:11:21 +0000 (UTC) (envelope-from emaste@freebsd.org) Received: from labgw2.phaedrus.sandvine.com ([192.168.3.11]) by gw.sandvine.com with Microsoft SMTPSVC(6.0.3790.3959); Tue, 18 Dec 2007 17:10:18 -0500 Received: by labgw2.phaedrus.sandvine.com (Postfix, from userid 12627) id 6CD08116FF; Tue, 18 Dec 2007 17:10:18 -0500 (EST) Date: Tue, 18 Dec 2007 17:10:18 -0500 From: Ed Maste To: Jeff Roberson Message-ID: <20071218221018.GA71002@sandvine.com> References: <20071218194312.GB40055@sandvine.com> <20071218101806.J899@desktop> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20071218101806.J899@desktop> User-Agent: Mutt/1.4.2.1i X-OriginalArrivalTime: 18 Dec 2007 22:10:18.0638 (UTC) FILETIME=[C64BDAE0:01C841C2] Cc: bzeeb+freebsd+lor@zabbadoz.net, freebsd-current@freebsd.org, Ed Maste Subject: Re: LOR kqueue / sleep mtxpool for LOR list X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2007 22:11:23 -0000 On Tue, Dec 18, 2007 at 10:20:21AM -1000, Jeff Roberson wrote: > On Tue, 18 Dec 2007, Ed Maste wrote: > > >I just saw this LOR during startup of an application while testing > >8-CURRENT on a dev box. I've done no investigation yet. > > > >FreeBSD TPC-D13-08.phaedrus 8.0-CURRENT FreeBSD 8.0-CURRENT #0: Mon Dec 17 > >16:27:54 EST 2007 > >emaste@bsd-build3.phaedrus:/d2/emaste/HEAD/obj/d2/emaste/HEAD/src/sys/GENERIC i386 > > > >lock order reversal: > >1st 0xc42b7580 kqueue (kqueue) @ > >/d2/emaste/HEAD/src/sys/kern/kern_event.c:1397 > >2nd 0xc3ece8e0 sleep mtxpool (sleep mtxpool) @ > >/d2/emaste/HEAD/src/sys/kern/sys_generic.c:1255 > >[...] > > Thanks, this is interesting. The mtxpool should always be the leaf. The > opposite order is the bug. Can you add them to the static order list and > tell me where the reverse order is observed? After adding it to the list this is what I got: Mounting NFS file systems:lock order reversal: 1st 0xc3ece280 sleep mtxpool (sleep mtxpool) @ /d2/emaste/HEAD/src/sys/kern/kern_event.c:993 2nd 0xc4320d80 kqueue (kqueue) @ /d2/emaste/HEAD/src/sys/kern/kern_event.c:1001 KDB: stack backtrace: db_trace_self_wrapper(c0b08831,e683daac,c079cabe,c0b0ada3,c4320d80,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0b0ada3,c4320d80,c0b01d5d,c0b01d5d,c0b01a4d,...) at kdb_backtrace+0x29 witness_checkorder(c4320d80,9,c0b01a4d,3e9,7db,...) at witness_checkorder+0x6de _mtx_lock_flags(c4320d80,0,c0b01a4d,3e9,0,...) at _mtx_lock_flags+0xbc kqueue_acquire(c445eaa0,4,e683db5c,c0b04436,f0,...) at kqueue_acquire+0x72 kern_kevent(c445eaa0,4,1,1,e683dc60,...) at kern_kevent+0x47 kevent(c445eaa0,e683dcfc,18,c0b0bacf,c0bb92e8,...) at kevent+0x19b syscall(e683dd38) at syscall+0x2b3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (363, FreeBSD ELF32, kevent), eip = 0x2813dafb, esp = 0xbfbfd28c, ebp = 0xbfbfd388 --- -Ed