From owner-freebsd-i386@FreeBSD.ORG Thu Oct 28 02:30:34 2004 Return-Path: Delivered-To: freebsd-i386@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2191516A4DD for ; Thu, 28 Oct 2004 02:30:34 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id C701943D72 for ; Thu, 28 Oct 2004 02:30:24 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.11/8.12.11) with ESMTP id i9S2UOMw054396 for ; Thu, 28 Oct 2004 02:30:24 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.11/8.12.11/Submit) id i9S2UOuP054395; Thu, 28 Oct 2004 02:30:24 GMT (envelope-from gnats) Resent-Date: Thu, 28 Oct 2004 02:30:24 GMT Resent-Message-Id: <200410280230.i9S2UOuP054395@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-i386@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Frank Mayhar Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2E28316A4CE for ; Thu, 28 Oct 2004 02:22:17 +0000 (GMT) Received: from tinker.exit.com (tinker.exit.com [206.223.0.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id C07CA43D5E for ; Thu, 28 Oct 2004 02:22:16 +0000 (GMT) (envelope-from frank@lap.exit.com) Received: from lap.exit.com (lap.exit.com [206.223.0.35]) by tinker.exit.com (8.13.1/8.12.9) with ESMTP id i9S2OUoJ059789 for ; Wed, 27 Oct 2004 19:24:30 -0700 (PDT) (envelope-from frank@lap.exit.com) Received: from lap.exit.com (localhost [127.0.0.1]) by lap.exit.com (8.13.1/8.13.1) with ESMTP id i9S2MGio000746 for ; Wed, 27 Oct 2004 19:22:16 -0700 (PDT) (envelope-from frank@lap.exit.com) Received: (from frank@localhost) by lap.exit.com (8.13.1/8.13.1/Submit) id i9S2MFpC000745; Wed, 27 Oct 2004 19:22:15 -0700 (PDT) (envelope-from frank) Message-Id: <200410280222.i9S2MFpC000745@lap.exit.com> Date: Wed, 27 Oct 2004 19:22:15 -0700 (PDT) From: Frank Mayhar To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Subject: i386/73224: Lock order reversal in ntoskrnl_timercall() X-BeenThere: freebsd-i386@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Frank Mayhar List-Id: I386-specific issues for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Oct 2004 02:30:34 -0000 >Number: 73224 >Category: i386 >Synopsis: Lock order reversal in ntoskrnl_timercall() >Confidential: no >Severity: critical >Priority: high >Responsible: freebsd-i386 >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Thu Oct 28 02:30:24 GMT 2004 >Closed-Date: >Last-Modified: >Originator: Frank Mayhar >Release: FreeBSD 5.3-STABLE i386 >Organization: Exit Consulting >Environment: System: FreeBSD lap 5.3-STABLE FreeBSD 5.3-STABLE #6: Wed Oct 27 18:08:57 PDT 2004 frank@lap:/home/obj/usr/src/sys/AUTON i386 This happened on boot of a DIAGNOSTIC kernel. lock order reversal 1st 0xc06c90c0 dont_sleep_in_callout (dont_sleep_in_callout) @ /usr/src/sys/kern/kern_timeout.c:257 2nd 0xc06c75a0 Giant (Giant) @ /usr/src/sys/modules/ndis/../../compat/ndis/subr_ntoskrnl.c:1647 KDB: stack backtrace: kdb_backtrace(0,ffffffff,c06cf7c0,c06d06e8,c069a2dc) at kdb_backtrace+0x29 witness_checkorder(c06c75a0,9,c0d0a5f2,66f) at witness_checkorder+0x544 _mtx_lock_flags(c06c75a0,0,c0d0a5f2,66f,c1fd3360) at _mtx_lock_flags+0x5b ntoskrnl_timercall(c1fd3360,c06c90c0,0,c0665fbd,101) at ntoskrnl_timercall+0x98 softclock(0) at softclock+0x1af ithread_loop(c1d8fc80,d55e8d48,c1d8fc80,c04e6160,0) at ithread_loop+0x124 fork_exit(c04e6160,c1d8fc80,d55e8d48) at fork_exit+0xa4 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xd55e8d7c, ebp = 0 --- KDB: enter: witness_checkorder #1 0xc0517083 in witness_checkorder (lock=0xc06c75a0, flags=0x9, file=0xc0d0a5f2 "/usr/src/sys/modules/ndis/../../compat/ndis/subr_ntoskrnl.c", line=0x66f) at /usr/src/sys/kern/subr_witness.c:952 #2 0xc04f0233 in _mtx_lock_flags (m=0xc06c75a0, opts=0x0, file=0xc0d0a5f2 "/usr/src/sys/modules/ndis/../../compat/ndis/subr_ntoskrnl.c", line=0x66f) at /usr/src/sys/kern/kern_mutex.c:271 #3 0xc0d07410 in ntoskrnl_timercall (arg=0xc1fd3360) at /usr/src/sys/modules/ndis/../../compat/ndis/subr_ntoskrnl.c:1647 #4 0xc0503307 in softclock (dummy=0x0) at /usr/src/sys/kern/kern_timeout.c:259 #5 0xc04e6284 in ithread_loop (arg=0xc1d8fc80) at /usr/src/sys/kern/kern_intr.c:547 #6 0xc04e5694 in fork_exit (callout=0xc04e6160 , arg=0xc1d8fc80, frame=0xd55e8d48) at /usr/src/sys/kern/kern_fork.c:811 #7 0xc0614e3c in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:209 The ntoskrnl_timercall() drops Giant on entry and tries to pick it up again on exit. I suspect that it shouldn't do anything with Giant at all, but I don't know the code well enough to say that for certain. Assuming that my suspicion is correct, though, the patch would be: Index: subr_ntoskrnl.c =================================================================== RCS file: /cvs/repos/src/sys/compat/ndis/subr_ntoskrnl.c,v retrieving revision 1.43.2.1 diff -u -r1.43.2.1 subr_ntoskrnl.c --- subr_ntoskrnl.c 13 Oct 2004 19:23:33 -0000 1.43.2.1 +++ subr_ntoskrnl.c 28 Oct 2004 02:14:39 -0000 @@ -1616,8 +1616,6 @@ ktimer *timer; struct timeval tv; - mtx_unlock(&Giant); - timer = arg; timer->k_header.dh_inserted = FALSE; @@ -1644,8 +1642,6 @@ ntoskrnl_wakeup(&timer->k_header); - mtx_lock(&Giant); - return; } >Description: >How-To-Repeat: >Fix: >Release-Note: >Audit-Trail: >Unformatted: