From owner-freebsd-stable@FreeBSD.ORG Sat Dec 4 13:32:30 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2DE3710656BA for ; Sat, 4 Dec 2010 13:32:30 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 87A388FC16 for ; Sat, 4 Dec 2010 13:32:29 +0000 (UTC) Received: by wyf19 with SMTP id 19so10539901wyf.13 for ; Sat, 04 Dec 2010 05:32:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=v121kr/+trZf29ZZlTk9AB2Ni26KRMyZCD3TWYlfoms=; b=Yd7VWwwY+zgNIPY9+LMLQ+2kCW6DwXQUtwg+lPXDbCzmP5AXAEqipFj/nHlgCemJL2 fWiowQniFcMu/G0o6PZdso0anDivGsMzjsWeF6pYJJRwQq2oxVt1jiuxhfKbCfKaBp/T F3DTxgUZ7lXWi4PB22np63Og9Z18O14sz3HfE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=Q+QlN31fr7PqlYzEZxIw3u35kv2GefXQLKxjWCJkPM7jkL5/tbmwOVK1Ev9krYmV7P 2ckRrrA2gsawcm6FhdeMdjMDyBR/ZeBog/0gvAQ99VUJYFooTHHUjlO1nvYn2Y30MHsq xl991XGuGC1yBvN2w8Dh6nxWnCBHuhALkFYSU= MIME-Version: 1.0 Received: by 10.216.179.81 with SMTP id g59mr487232wem.35.1291469548240; Sat, 04 Dec 2010 05:32:28 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.216.65.210 with HTTP; Sat, 4 Dec 2010 05:32:27 -0800 (PST) In-Reply-To: References: <4B521FC2.4050402@errno.com> <4B535AAE.3060308@errno.com> <4B5BA0C1.8010901@errno.com> Date: Sat, 4 Dec 2010 21:32:27 +0800 X-Google-Sender-Auth: IN8UO4GJWoLt2MxUoh-_hD5QsGM Message-ID: From: Adrian Chadd To: Russell Yount Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: atheros broadcast/multicast corruption with multiple hostap's X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Dec 2010 13:32:30 -0000 Just FYI, (and sorry for resurrecting such an old thread!) The mcast keysearch changes you've suggested have broken CCMP handling on at least the AR9160 in -HEAD. kern/150148 also notes that between 8.0-REL and 8.1-REL CCMP WPA for the poster also broke. The main change here related to crypto/key handling is the mcast key search enable that's been enabled. Reverting those changes so mcast key search is enabled fixes 11n WPA (which requires CCMP.) I bet it'd also fix kern/150148. I'm guessing there's some work needed in the key handling code in if_ath. Russell, are you still interested in this problem? Would you be interested in giving me a hand? Thanks Russell/Sam, Adrian On 25 January 2010 08:16, Russell Yount wrote: > On Sat, Jan 23, 2010 at 8:22 PM, Sam Leffler wrote: > >> Russell Yount wrote: >> > On Sun, Jan 17, 2010 at 1:45 PM, Sam Leffler > > > wrote: >> > >> > =A0 =A0 Russell Yount wrote: >> > >> > >> > >> > =A0 =A0 =A0 =A0 On Sat, Jan 16, 2010 at 3:21 PM, Sam Leffler > > =A0 =A0 =A0 =A0 > =A0> =A0 =A0 =A0 =A0 >> wrote: >> > >> > =A0 =A0 =A0 =A0 =A0 =A0Russell Yount wrote: >> > >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0It seems AP to client broadcasts/multic= asts traffic is >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0broken when using WPA2/802.11i with mul= tiple hostapds in >> 8.0. >> > >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Only the SSID associated with the last = hostapd to be >> > =A0 =A0 =A0 =A0 started has >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0AP to client broadcasts/multicasts bein= g delivered >> correctly. >> > >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0The AP and client are 8.0 freebsd syste= ms althought I see >> > =A0 =A0 =A0 =A0 same >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0problems with windows XP as a client. >> > >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0The AP has 4 hostapds configured to use= TLS with client >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0certificates for >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0authentication. (hostapd recompiled wit= h >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0HOSTAPD_CFLAGS=3D-DEAP_SERVER) >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0The AP and client radio are shown as at= h0: AR5212 mac 5.9 >> > =A0 =A0 =A0 =A0 RF5112 >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0phy 4.3 >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0in dmesg. >> > >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Client authenticate using client certif= icates associate >> > =A0 =A0 =A0 =A0 correctly >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0to all 4 SSIDs. Unicast traffic flows c= orrectly between >> > =A0 =A0 =A0 =A0 clients >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0and AP >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0for all for 4 SSIDs. Client to AP broad= cast/multicast >> > =A0 =A0 =A0 =A0 traffic works >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0on of 4 SSIDs. AP to client broadcast/m= ulticast traffic >> > =A0 =A0 =A0 =A0 only works >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0on 1 of the SSIDs. I have documented th= is using ARP >> > =A0 =A0 =A0 =A0 broadcasts, >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0but normal IP broadcasts also observed = to corrupted. >> > >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0When an ARP request is send through the= AP to an >> > =A0 =A0 =A0 =A0 associated client >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0it seems to be trashed on any of the SS= ID except the one >> > =A0 =A0 =A0 =A0 associated >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0with the last hostapd to be started. He= re is the output of >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0client side >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0tcpdump showing the problems. >> > >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0In the first client side tcpdump with t= he hostapd >> associated >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0with the SSID >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0being associaed with the last hostapd s= tarted and the >> traffic >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0flowing >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0normally. >> > >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0In the second client side tcpdump with = the hostapd >> associated >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0with the SSID >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0being not the last hostapd started the = ARP request is >> resent >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0multiple times >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0and appears corrupted. >> > >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0I would really like to find a fix for t= his. >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Any help would be greatly appreciated. >> > >> > >> > =A0 =A0 =A0 =A0 =A0 =A0This sounds like the crypto encap of the frame = is clobbering >> the >> > =A0 =A0 =A0 =A0 =A0 =A0mbuf contents. =A0You can verify this by settin= g up multiple >> > =A0 =A0 =A0 =A0 vaps w/o >> > =A0 =A0 =A0 =A0 =A0 =A0WPA. =A0If this is the problem look for the mbu= f copy logic for >> > =A0 =A0 =A0 =A0 mcast >> > =A0 =A0 =A0 =A0 =A0 =A0frames and make sure a deep copy is done. >> > >> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Sam >> > >> > =A0 =A0 =A0 =A0 =A0 =A0The four VAPs broadcast traffic works find with= out WPA if I >> > =A0 =A0 =A0 =A0 do not start hostapds on them >> > =A0 =A0 =A0 =A0 =A0I have been trying to discovery why broadcast traff= ic only >> > =A0 =A0 =A0 =A0 works correctly on the VAP associated with the last ho= stapd to >> > =A0 =A0 =A0 =A0 be started. I have move with VAP has the working broad= cast >> > =A0 =A0 =A0 =A0 traffic by restarting the hostapd >> > =A0 =A0 =A0 =A0 associated with it. >> > =A0 =A0 =A0 =A0 =A0It would seem something in the WPA/802.1x layer ini= tialization >> > =A0 =A0 =A0 =A0 remembers which hostapd was started last and that affe= cted the >> > =A0 =A0 =A0 =A0 crypto encap. >> > =A0 =A0 =A0 =A0 =A0I keep looking but do not see any place in the code= that could >> > =A0 =A0 =A0 =A0 account for this. >> > =A0 =A0 =A0 =A0 =A0It seems the corrupt crypto encap also happens on b= roadcast >> > =A0 =A0 =A0 =A0 between stations. >> > =A0 =A0 =A0 =A0 Please correct me if I am wrong: >> > =A0 =A0 =A0 =A0 but when using hostapd normally traffic is bridged wit= hing the >> card. >> > =A0 =A0 =A0 =A0 So if a station sends to the VAP a broadcast it is act= aully >> > =A0 =A0 =A0 =A0 sending a non- broadcast frame to the AP >> > =A0 =A0 =A0 =A0 and the AP sends the frame to all the other stations. >> > >> > >> > =A0 =A0 I told you waht the likely problem is. =A0Look in the net80211= layer >> > =A0 =A0 in the kernel for the problem. >> > >> > =A0 =A0 =A0 =A0 =A0 =A0Sam >> > >> > >> > >> > >> > =A0I tried to find problems in mbuf corruption >> > in ieee80211_output.c by placing >> > >> > =A0 m =3D m_unshare(m,M_NOWAIT); >> > =A0 if (m =3D=3D NULL) { >> > =A0 =A0 =A0 IEEE80211_DPRINTF(vap, IEEE80211_MSG_OUTPUT, >> > =A0 =A0 =A0 "%s: cannot get writable mbuf\n", __func__); >> > =A0 =A0 =A0 return NULL; >> > =A0 } >> > >> > at begining ieee80211_mbuf_adjust() and at >> > beginning of ieee80211_encap() with no change >> > in the broadcast traffic behaviour. >> > >> > I tried then to in ieee80211_crypto.c substituting >> > >> > =A0 flags |=3D IEEE80211_KEY_SWCRYPT; >> > >> > for the encryption capabilities test code >> > >> > =A0 if ((ic->ic_cryptocaps & (1<> > =A0 =A0 =A0 =A0IEEE80211_DPRINTF(vap, IEEE80211_MSG_CRYPTO, >> > =A0 =A0 =A0 =A0"%s: no h/w support for cipher %s, falling back to s/w\= n", >> > =A0 =A0 =A0 =A0 =A0 =A0__func__, cip->ic_name); >> > =A0 =A0 =A0 =A0flags |=3D IEEE80211_KEY_SWCRYPT; >> > =A0 =A0} >> > >> > to force all the encryption to be done in software. >> > >> > This fixed the broadcast traffic problem but without >> > hardware support its very slow and really loads machine. >> > Enabling in the debug code to ath and net80211 >> > >> > and enabled ATH_DEBUG_KEYCACHE in if_ath.c and >> > IEEE80211_MSG_CRYPTO in net80211 code. >> > >> > It seems that all the VAPS sets the broadcast key for >> > mac ff:ff:ff:ff:ff:ff in the ath device so I assume >> > they conflict and the last one setting the key is the >> > working one; that would explain why the last hostapd >> > started is the only one with working broadcast code >> > to clients. >> > >> > In if_ath.c the code >> > >> > =A0if ((k->wk_flags & IEEE80211_KEY_GROUP) && sc->sc_mcastkey) { >> > =A0 =A0/* >> > =A0 =A0 * Group keys on hardware that supports multicast frame >> > =A0 =A0 * key search use a mac that is the sender's address with >> > =A0 =A0 * the high bit set instead of the app-specified address. >> > =A0 =A0 */ >> > =A0 =A0 =A0IEEE80211_ADDR_COPY(gmac, bss->ni_macaddr); >> > =A0 =A0 =A0gmac[0] |=3D 0x80; >> > =A0 =A0 =A0mac =3D gmac; >> > =A0} else >> > =A0 =A0 mac =3D k->wk_macaddr; >> > >> > seems to indicate that for multiple VAPs the ath chips >> > needs to be able to distinguish between broadcast keys >> > by using a permutation of VAPs bssid. >> > >> > But in if_athvar.h the code does not seem complete >> > and sc->sc_mcastkey is also set false. >> > >> > =A0#ifdef notyet >> > =A0#define ath_hal_hasmcastkeysearch(_ah) \ >> > =A0 =A0 =A0 =A0 (ath_hal_getcapability(_ah, HAL_CAP_MCAST_KEYSRCH, 0, = NULL) =3D=3D >> > HAL_OK) >> > =A0#define ath_hal_getmcastkeysearch(_ah) \ >> > =A0 =A0 =A0 =A0 (ath_hal_getcapability(_ah, HAL_CAP_MCAST_KEYSRCH, 1, = NULL) =3D=3D >> > HAL_OK) >> > =A0#else >> > =A0#define ath_hal_getmcastkeysearch(_ah) =A00 >> > =A0#endif >> > >> > I am using cards with an AR5212 which does seem to have >> > multiple bssid support working so I hope they should also >> > support mcastkeysearch capability. Maybe only some >> > firmware revisions have this? >> > >> > Do you know what the status of the HAL code is for >> > supporting this? Why it is commented out in if_athvar.h? >> >> Good work analyzing things; been a long time since I looked at this. =A0= I >> vaguely recall disabling mcastkey searching because of problems with >> WEP. =A0I'm surprised WPA is broken as that was a standard test case. >> >> You can try enabling the notyet code and see if the right thing happens. >> =A0I don't see any indication of the mac rev for your part but I expect = it >> supports this as it was only very early parts that had issues. >> >> If enabling the mcastkey search mechanism doesn't fix this I might've >> broken things with changes to explicitly mark group keys when hostapd >> plumbs their contents. =A0I recall doing this for mwl which doesn't have >> an indexed key table like ath (it uses the mac address of the local bss >> and/or associated station to find the data structure where crypto keys >> are stored). >> >> I haven't looked at this stuff in a long time and can't setup a system >> to test but if you keep pushing on this I'll try to help w/ advise. >> >> =A0 =A0 =A0 =A0Sam >> > > Sam, > I have the ath working with multiple hostap's and multicast key search an= d > also > have the station side working, but I do not like how I got the station si= de > working. > Let me explain what I have done and what I am tryinig to figure out now. > I would like to submit a patch once I get the last issues resolved. > > In if_athvar.h I uncommented your macros > > =A0#define ath_hal_hasmcastkeysearch(_ah) \ > =A0 =A0 =A0 =A0(ath_hal_getcapability(_ah, HAL_CAP_MCAST_KEYSRCH, 0, NULL= ) =3D=3D > HAL_OK) > =A0#define ath_hal_getmcastkeysearch(_ah) \ > =A0 =A0 =A0 =A0(ath_hal_getcapability(_ah, HAL_CAP_MCAST_KEYSRCH, 1, NULL= ) =3D=3D > HAL_OK) > > and added a macro > > #define ath_hal_setmcastkeysearch(_ah, _v) \ > =A0 =A0 =A0 ath_hal_setcapability(_ah, HAL_CAP_MCAST_KEYSRCH, 0, _v, NULL= ) > In if_ath.c I added > > =A0 =A0 =A0/* > =A0 =A0 =A0 =A0* if multicast key search is supported by device enable it > =A0 =A0 =A0 =A0*/ > =A0 =A0 =A0 if (ath_hal_hasmcastkeysearch(sc->sc_ah)) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (!ath_hal_getmcastkeysearch(sc->sc_ah)) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ath_hal_setmcastkeysearch(sc-= >sc_ah,1); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 } > =A0 =A0 =A0 } > > just before > > =A0 =A0 =A0sc->sc_mcastkey =3D ath_hal_getmcastkeysearch(ah); > > in ath_attach(). > > Then in ieee80211_ioctl.c I changed ieee80211_ioctl_setkey() so > =A0not to assign a slot when opering in hostap mode by changing > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * Global slots start off w/o any assigned= key index. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * Force one here for consistency with IEE= E80211_IOC_WEPKEY. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (wk->wk_keyix =3D=3D IEEE80211_KEYIX_NO= NE) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0wk->wk_keyix =3D kid; > to be > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * Global slots start off w/o any assigned= key index. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * Force one here for consistency with IEE= E80211_IOC_WEPKEY. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (vap->iv_opmode !=3D IEEE80211_M_HOSTAP > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0&& wk->wk_keyix =3D=3D IEEE80211_KEYIX= _NONE) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0wk->wk_keyix =3D kid; > > to preserve the station mode operation I had to keep the key index > assignment > when not VAP is not operating in hostapd mode. This seems wrong to me. > > Now here is the question I am trying to understand. Group keys are specia= l > as they are used in only one direction only; sending on hostap and receiv= ing > on a station. > > The multicast key search code when operating in hostap mode permits the > lookup > of the key for encryption by sending VAP bssid on transmission of multica= st > traffic. > > Is there a corresponding station side multicast key search that would let > the lookup > of the encryption key be done on receive by looking up the sending VAP fo= r > received > multicast traffic. If so it seems it could enable multiple stations also = to > work on one > device. > > I noticed in the linux legacy hal ar5212_keycache.c the following > code/comment > > =A0 =A0 =A0 =A0u_int32_t validBit =3D AR_KEYTABLE_VALID; > =A0 =A0 =A0 =A0...... > =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * =A0If upper layers have requested mcast= MACaddr lookup, > then > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * =A0signify this to the hw by setting th= e (poorly named) > validBit > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * =A0to 0. =A0Yes, really 0. The hardware= specs, > pcu_registers.txt, is > =A0 =A0 =A0 =A0 * =A0has incorrectly named ValidBit. It should be called = "Unicast". > =A0 =A0 =A0 =A0 * =A0When the Key Cache entry is to decrypt Unicast frame= s, then this > =A0 =A0 =A0 =A0 * =A0bit should be '1'; for multicast and broadcast frame= s, this bit > is '0'. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (mac[0] & 0x01) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0validBit =3D 0; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0..... > =A0 =A0 =A0 =A0OS_KC_WRITE(ah, AR_KEYTABLE_MAC0(entry), macLo); > =A0 =A0 =A0 =A0OS_KC_WRITE(ah, AR_KEYTABLE_MAC1(entry), macHi | validBit)= ; > > where as in the freebsd hal in ar5212_keycache.c the AR_KEYTABLE_VALID > is always set > > =A0 =A0 =A0 =A0OS_REG_WRITE(ah, AR_KEYTABLE_MAC0(entry), macLo); > =A0 =A0 =A0 =A0OS_REG_WRITE(ah, AR_KEYTABLE_MAC1(entry), macHi | > AR_KEYTABLE_VALID); > by not setting the AR_KEYTABLE_VALID bit would the multicast key search c= ode > work for station mode? I am just guessing here, but seemed like a likely > explaination? > > -Russ > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >