From owner-freebsd-net@FreeBSD.ORG Sat Dec 5 03:46:06 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C036B1065670 for ; Sat, 5 Dec 2009 03:46:06 +0000 (UTC) (envelope-from jamesbrandongooch@gmail.com) Received: from mail-ew0-f216.google.com (mail-ew0-f216.google.com [209.85.219.216]) by mx1.freebsd.org (Postfix) with ESMTP id 546978FC12 for ; Sat, 5 Dec 2009 03:46:06 +0000 (UTC) Received: by ewy8 with SMTP id 8so3562675ewy.35 for ; Fri, 04 Dec 2009 19:46:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=44GT0iAEMFQngcSdVLZb9keZTF7ls/D1QfPyLUz5YCs=; b=WoGCQXv7+RDwpbtDe8NKfrje2QZ18H9OLA8xlJM5EnT+VFxR0Wyt8Br51nFK++q2Az HvGM8EDdSUM0OekhBju9a0sAZ5l3qdcyKRcHgrr6x5a/F/4cGRqg6bODNfC6sG70tLhT WhemX7w/NAWZBa6yx7jTZKnVVF39izgrCaDtE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=uoz6kaTWm9sLd4rX1tOLxWGn9NkviKTrbzNMTaCHugQpFbYXR84KWMp+czyqo/v8i0 Dq/YwCWIA+P7Dr8HbUCOP9T+Ux43tBYrBh8GCvZkMZ3An4bHOX9aEMdHImm4y53jORtv ZfADl2ke8HJrtQGgAIJg/CNv2D0M1lF0kyp1I= MIME-Version: 1.0 Received: by 10.213.23.80 with SMTP id q16mr3660732ebb.3.1259984763798; Fri, 04 Dec 2009 19:46:03 -0800 (PST) In-Reply-To: <200912050250.nB52o4P0035345@freefall.freebsd.org> References: <200912050250.nB52o4P0035345@freefall.freebsd.org> Date: Sat, 5 Dec 2009 03:46:03 +0000 Message-ID: <179b97fb0912041946j13760914ycc2c5145d8df9ee@mail.gmail.com> From: Brandon Gooch To: Benjamin Kaduk , freebsd-net@freebsd.org, Bernhard Schmidt Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_lock and iwn0 softc lock X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Dec 2009 03:46:06 -0000 On Sat, Dec 5, 2009 at 2:50 AM, Benjamin Kaduk wrote: > The following reply was made to PR kern/140036; it has been noted by GNAT= S. > > From: Benjamin Kaduk > To: Bernhard Schmidt , sam@freebsd.org > Cc: bug-followup@freebsd.org > Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_l= ock > =A0and iwn0 softc lock > Date: Fri, 4 Dec 2009 21:49:42 -0500 (EST) > > =A0Okay, I've been getting these lockups fairly frequently -- in fact, > =A0there was this one room where I was getting them every five minutes > =A0or so. =A0I hypothesized that dissociation/association events might > =A0be triggers, so I set dev.iwn.0.debug=3D-1 and started wandering aroun= d > =A0this building (which is chock full of APs). > =A0I get: > =A0panic: lock (sleep mutex) iwn0_com_lock not locked @ > =A0/usr/devel/iwn/freebsd/sys/modules/iwn/../../dev/iwn/if_iwn.c:3254 > > =A0Unfortunately, it appears to have messed up my machine pretty badly, > =A0as only numlock blinks the LED, and I don't have a db> prompt ... > > =A0Trying again without the debugging spew gets a useful debugger, though= . > =A0"show alllocks" produces empty output (?!). > =A0backtrace is: > =A0kdb_enter > =A0panic > =A0witness_unlock > =A0_mtx_unlock_flags > =A0iwn_raw_xmit > =A0ieee80211_send_probereq > =A0beacom_miss > =A0taskqueue_run > =A0taskqueue_thread_loop > =A0fork_exit > > =A0Looking at the coredump, I'm inclined to suspect this block of > =A0ieee80211_proto.c (in beacon_miss() ): > =A01.46 =A0 =A0 =A0sam =A0 1379: =A0 =A0 =A0 =A0/* XXX locking */ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A01380: =A0 =A0 =A0 =A0TAILQ_FOREACH(vap= , &ic->ic_vaps, iv_next) { > =A01.38 =A0 =A0 =A0sam =A0 1381: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* > =A01.46 =A0 =A0 =A0sam =A0 1382: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * We onl= y pass events through for sta vap's in RUN state; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A01383: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = * may be too restrictive but for now this saves all the > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A01384: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = * handlers duplicating these checks. > =A01.38 =A0 =A0 =A0sam =A0 1385: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ > =A01.46 =A0 =A0 =A0sam =A0 1386: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (vap->= iv_opmode =3D=3D IEEE80211_M_STA && > =A01.64 =A0 =A0 =A0sam =A0 1387: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0v= ap->iv_state >=3D IEEE80211_S_RUN && > =A01.46 =A0 =A0 =A0sam =A0 1388: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0v= ap->iv_bmiss !=3D NULL) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A01389: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0vap->iv_bmiss(vap); > > > =A0Sam, do you have any thoughts as to why throwing a > =A0IEEE80211_LOCK(ic)/IEEE80211_UNLOCK(ic) block around the TAILQ_FOREACH > =A0might not be a good idea? > =A0I'm currently running with that, which gives me a new LOR > =A0(iwn_com_lock @ /usr/src/sys/net80211/ieee80211_scan.c:683 and > =A0iwn0 @ /usr/devel/iwn/freebsd/sys/modules/iwn/../../dev/iwn/if_iwn.c:3= 289) > =A0but it hasn't locked up or panic()ed on me, yet. > > > =A0I am happy to pull more information from the coredump if needed. > Ben, have you tried Bernhard Schmidt's driver? He's recently updated it with a reordering of locking code: http://svn.techwires.net/viewvc/viewvc.cgi/svnrepos/projects/freebsd/sys/de= v/iwn/if_iwn.c?view=3Dlog I've been using the code from his repository for a while, and it's much more stable than what's in the tree currently. -Brandon