From owner-freebsd-net@FreeBSD.ORG Sat Dec 5 02:50:05 2009 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1799C106566C for ; Sat, 5 Dec 2009 02:50:05 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id E13E18FC12 for ; Sat, 5 Dec 2009 02:50:04 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id nB52o4jE035346 for ; Sat, 5 Dec 2009 02:50:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id nB52o4P0035345; Sat, 5 Dec 2009 02:50:04 GMT (envelope-from gnats) Date: Sat, 5 Dec 2009 02:50:04 GMT Message-Id: <200912050250.nB52o4P0035345@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Benjamin Kaduk Cc: Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_lock and iwn0 softc lock X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Benjamin Kaduk List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Dec 2009 02:50:05 -0000 The following reply was made to PR kern/140036; it has been noted by GNATS. From: Benjamin Kaduk To: Bernhard Schmidt , sam@freebsd.org Cc: bug-followup@freebsd.org Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_lock and iwn0 softc lock Date: Fri, 4 Dec 2009 21:49:42 -0500 (EST) Okay, I've been getting these lockups fairly frequently -- in fact, there was this one room where I was getting them every five minutes or so. I hypothesized that dissociation/association events might be triggers, so I set dev.iwn.0.debug=-1 and started wandering around this building (which is chock full of APs). I get: panic: lock (sleep mutex) iwn0_com_lock not locked @ /usr/devel/iwn/freebsd/sys/modules/iwn/../../dev/iwn/if_iwn.c:3254 Unfortunately, it appears to have messed up my machine pretty badly, as only numlock blinks the LED, and I don't have a db> prompt ... Trying again without the debugging spew gets a useful debugger, though. "show alllocks" produces empty output (?!). backtrace is: kdb_enter panic witness_unlock _mtx_unlock_flags iwn_raw_xmit ieee80211_send_probereq beacom_miss taskqueue_run taskqueue_thread_loop fork_exit Looking at the coredump, I'm inclined to suspect this block of ieee80211_proto.c (in beacon_miss() ): 1.46 sam 1379: /* XXX locking */ 1380: TAILQ_FOREACH(vap, &ic->ic_vaps, iv_next) { 1.38 sam 1381: /* 1.46 sam 1382: * We only pass events through for sta vap's in RUN state; 1383: * may be too restrictive but for now this saves all the 1384: * handlers duplicating these checks. 1.38 sam 1385: */ 1.46 sam 1386: if (vap->iv_opmode == IEEE80211_M_STA && 1.64 sam 1387: vap->iv_state >= IEEE80211_S_RUN && 1.46 sam 1388: vap->iv_bmiss != NULL) 1389: vap->iv_bmiss(vap); Sam, do you have any thoughts as to why throwing a IEEE80211_LOCK(ic)/IEEE80211_UNLOCK(ic) block around the TAILQ_FOREACH might not be a good idea? I'm currently running with that, which gives me a new LOR (iwn_com_lock @ /usr/src/sys/net80211/ieee80211_scan.c:683 and iwn0 @ /usr/devel/iwn/freebsd/sys/modules/iwn/../../dev/iwn/if_iwn.c:3289) but it hasn't locked up or panic()ed on me, yet. I am happy to pull more information from the coredump if needed. Thanks, Ben Kaduk