From owner-freebsd-wireless@FreeBSD.ORG Wed Jan 25 14:56:03 2012 Return-Path: Delivered-To: freebsd-wireless@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE95B106566B for ; Wed, 25 Jan 2012 14:56:03 +0000 (UTC) (envelope-from moonlightakkiy@yahoo.ca) Received: from nm21-vm2.bullet.mail.ne1.yahoo.com (nm21-vm2.bullet.mail.ne1.yahoo.com [98.138.91.209]) by mx1.freebsd.org (Postfix) with SMTP id 72EFC8FC12 for ; Wed, 25 Jan 2012 14:56:03 +0000 (UTC) Received: from [98.138.90.50] by nm21.bullet.mail.ne1.yahoo.com with NNFMP; 25 Jan 2012 14:43:33 -0000 Received: from [98.138.226.61] by tm3.bullet.mail.ne1.yahoo.com with NNFMP; 25 Jan 2012 14:43:33 -0000 Received: from [127.0.0.1] by smtp212.mail.ne1.yahoo.com with NNFMP; 25 Jan 2012 14:43:33 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.ca; s=s1024; t=1327502613; bh=yySdmhTmBTzxWOMDUHH37QqkbPPjrSyLca4668ITP8k=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:Received:Received:MIME-Version:Received:Received:Date:Message-ID:Subject:From:To:Content-Type:Content-Transfer-Encoding; b=0nVUDG/j03xx4LePBv2g8vA9UsfhocJ+5BLxxj1QVQfciCcNJv7r8x1Sa0xhOYYCxu8CP/ybNg1BpmZ06pjkbqL41w9MnABW1P1pqLRUdH9j4M57BEVSig5oFE5X0w4lt9/ZsYwRNbPMAUO/u/R/cenwLMcKQLhM3q4LKOCVAjg= X-Yahoo-Newman-Id: 769013.82562.bm@smtp212.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: CCufYGUVM1nO0r_XvI7pZ7dWnCYSV5tBRViHSf01tPwLl3K PM8eZukMIqyeuweBS4GHyMr42mKxEE3Em0nEqqq3344cDOF8VrCpRr0fIw9X XbI..2cqLuUOFaDMYLn2U_lJaOUjjbZZwYnpZ7_psLUSD0esPqU6hInuxR4u UnavG2IMWkxeIcSIsRBkPxEGsXqiyUnj5dMIV3E9xfiUNHZzeCVC0XoTPoII vR1lK9lyAlWux7DcMAU4XBnF5Re1flnPSTR7vUaiywBbdmT9Mt50V0kf1nkN xKdwrCgZ4wHK_zuLK5NnnpbwzYnzvHS3NK2kcC0Zrka.iLiL4MoWmSi1gKKH xYcozeUbMqG2W6oFXgqm8nnggcsV7mpzHgAmyqddw6pq9lDsmeBEzlY3.kZN Yitg_FI7XT0vbUlrZ0MddqUMlkjIUuQ4_n4jImWH7bmjN7SE- X-Yahoo-SMTP: Xr6qjFWswBAEmd20sAvB4Q3keqXvXsIH9TjJ Received: from mail-iy0-f182.google.com (moonlightakkiy@209.85.210.182 with plain) by smtp212.mail.ne1.yahoo.com with SMTP; 25 Jan 2012 06:43:33 -0800 PST Received: by iaeo4 with SMTP id o4so1840680iae.13 for ; Wed, 25 Jan 2012 06:43:33 -0800 (PST) MIME-Version: 1.0 Received: by 10.42.131.136 with SMTP id z8mr15667909ics.5.1327502613199; Wed, 25 Jan 2012 06:43:33 -0800 (PST) Received: by 10.231.37.2 with HTTP; Wed, 25 Jan 2012 06:43:33 -0800 (PST) Date: Wed, 25 Jan 2012 07:43:33 -0700 Message-ID: From: PseudoCylon To: Adrian Chadd , freebsd-wireless@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Subject: Re: net80211 race conditions seen in -HEAD X-BeenThere: freebsd-wireless@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussions of 802.11 stack, tools device driver development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 14:56:03 -0000 > ------------------------------ > > Message: 14 > Date: Sat, 21 Jan 2012 21:40:12 -0800 > From: Adrian Chadd > Subject: net80211 race conditions seen in -HEAD > To: freebsd-wireless@freebsd.org > Message-ID: > =A0 =A0 =A0 =A0 > Content-Type: text/plain; charset=3DISO-8859-1 > > Hi, > > I've noticed some kernel panics in net80211/ath in -HEAD. It in all > instances boils down to a now-invalid ieee80211_node - either it's > partially allocated/copied, or it's been recently freed. > > > > This became increasingly obvious when doing DFS CAC, as the kernel was no= w > changing the channel quite frequently on me whilst simulating/processing > radar events. I've since found I can mostly reproduce it in the lab (when > surrounded by ridiculous levels of RX intereference traffic, triggering a= ll > kinds of events) whilst creating/destroying VAPs. > > Now that I have debugging code in place (which as a side effect makes it > very difficult now to cause a crash, let alone tickle the race condition) > it's glaringly obvious what's going on. > > There's five contexts stuff can occur, at least in the net80211/ath case: > > * the swi (ie ath_intr(), ath_beacon_proc) > * the ath taskqueue; > * the net80211 taskqueue; > * the ioctl() context, coming up from a userland process; > * a callout running in the clock thread. > > Now, callouts should _hopefully_ be grabbing and releasing locks correctl= y. > We've found a few spots where they weren't (leading to quite silly state > races and crashes.) > > I'm going to ignore the obvious possible problems with multiple concurren= t > processes doing ioctl()s. l'm simply going to operate on the principle th= at > the multiple-ioctl() path is fine. > > It seems that -obtaining- references to vap->iv_bss aren't locked. So in > (say) ieee80211_sta_join1() the iv_bss node can be dereferenced and freed= . > If this is going on concurrently with (say) something going on in the > net80211 taskqueue (eg a newstate call) then I _think_ it's possible for > the ath_newstate() code to get a reference to vap->iv_bss simultaneously > with it being freed in ieee80211_sta_join1() (or similar.) So the > ath_newstate() code will be assigned a 'ni' that has just been freed. > > I've seen another crash in the net80211_ht code where it _looks_ like the > bss node wasn't entirely setup - bsschan was 0xffff - so the kernel panic= ed > hard there. > > This likely explains a lot of the "weird stuff" people have been reportin= g. > I also think the bgscan race is related to this - I can't help but wonder > if the bgscan callout/event is also coinciding with wpa_supplicant doing > stuff, and a race condition ends up leaving the vap w/ the sta power save > flag set. > > I don't yet have a solution to all of this - I just wanted to brain dump > what I've seen thus far. > Here is my brain dump. While ago usb wifi drivers had the slimier issue (race in 80211 stack). It's worth checking this rev. http://svnweb.freebsd.org/base?view=3Drevision&revision=3D212127 AK