From owner-freebsd-stable@FreeBSD.ORG Wed Dec 8 02:19:51 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C934F106564A; Wed, 8 Dec 2010 02:19:51 +0000 (UTC) (envelope-from russell.yount@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 55A078FC0A; Wed, 8 Dec 2010 02:19:51 +0000 (UTC) Received: by qwj9 with SMTP id 9so650969qwj.13 for ; Tue, 07 Dec 2010 18:19:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=7YvKL2t19qlZZhTODSf6b5u2Y3VcjGC8jbSfAa2C/J8=; b=r4WLjAr3e2A6LBkz4f+jmO+t0Ymwb5/fRsIlfHpwz1bihVaXm0+qv6rADLP62aiak9 ahO9Hc41J+4WEzPmMqBw++IRPd/kudV73Hu/hsdWnJ6AMUXcdhXcRelQAlCRnPxMPdWW qP1FyJfBNAKfMXigMagem9TlRctnpZYbZH7sA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=dCWxI0JbvcfX8NVuz07SvGCFQfS5zTFTvTvLWhZuJCHcr+kaz7O4xPETqe4ABl9EVa NbB5iALObdrijDhDS0Gd+iS8CAfneYpHv+gufMZJqdHvSJUbkDJRGySHdoh9QpNOH90g u2pjrKVCIgHpiud/mcgYWnly4dB/VM9Oua4lU= MIME-Version: 1.0 Received: by 10.229.212.72 with SMTP id gr8mr6316616qcb.177.1291773240209; Tue, 07 Dec 2010 17:54:00 -0800 (PST) Received: by 10.220.13.148 with HTTP; Tue, 7 Dec 2010 17:54:00 -0800 (PST) In-Reply-To: References: <4B521FC2.4050402@errno.com> <4B535AAE.3060308@errno.com> <4B5BA0C1.8010901@errno.com> Date: Tue, 7 Dec 2010 20:54:00 -0500 Message-ID: From: Russell Yount To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@freebsd.org Subject: Re: atheros broadcast/multicast corruption with multiple hostap's X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 02:19:51 -0000 Adrian, Yes, I can help track down this. I have only 11agb radios though: NL-5354MP + Aries2 5004 MP Atheros 4G / CM9 The changes I made definiately work for these chips, I have 4 APs operating with 4 SSID each all configured with WPA2 using x509 certs. Currently, the kernel I am using is 8.1. I seem to recall that the hardware abstration layer for different chipsets treated handling of multicast reception differently. Could you send me pointers to what problems are described? Sorry, I took so long to reply, been rather busy, I do not check this email account as often as my others. -Russ On Sat, Dec 4, 2010 at 8:32 AM, Adrian Chadd wrote: > Just FYI, (and sorry for resurrecting such an old thread!) > > The mcast keysearch changes you've suggested have broken CCMP handling > on at least the AR9160 in -HEAD. > > kern/150148 also notes that between 8.0-REL and 8.1-REL CCMP WPA for > the poster also broke. The main change here related to crypto/key > handling is the mcast key search enable that's been enabled. > > Reverting those changes so mcast key search is enabled fixes 11n WPA > (which requires CCMP.) I bet it'd also fix kern/150148. > > I'm guessing there's some work needed in the key handling code in if_ath. > > Russell, are you still interested in this problem? Would you be > interested in giving me a hand? > > Thanks Russell/Sam, > > > Adrian > > On 25 January 2010 08:16, Russell Yount wrote: > > On Sat, Jan 23, 2010 at 8:22 PM, Sam Leffler wrote: > > > >> Russell Yount wrote: > >> > On Sun, Jan 17, 2010 at 1:45 PM, Sam Leffler >> > > wrote: > >> > > >> > Russell Yount wrote: > >> > > >> > > >> > > >> > On Sat, Jan 16, 2010 at 3:21 PM, Sam Leffler >> > >> > >> wrote: > >> > > >> > Russell Yount wrote: > >> > > >> > It seems AP to client broadcasts/multicasts traffic is > >> > broken when using WPA2/802.11i with multiple hostapds > in > >> 8.0. > >> > > >> > Only the SSID associated with the last hostapd to be > >> > started has > >> > AP to client broadcasts/multicasts being delivered > >> correctly. > >> > > >> > The AP and client are 8.0 freebsd systems althought I > see > >> > same > >> > problems with windows XP as a client. > >> > > >> > The AP has 4 hostapds configured to use TLS with client > >> > certificates for > >> > authentication. (hostapd recompiled with > >> > HOSTAPD_CFLAGS=-DEAP_SERVER) > >> > The AP and client radio are shown as ath0: AR5212 mac > 5.9 > >> > RF5112 > >> > phy 4.3 > >> > in dmesg. > >> > > >> > Client authenticate using client certificates associate > >> > correctly > >> > to all 4 SSIDs. Unicast traffic flows correctly between > >> > clients > >> > and AP > >> > for all for 4 SSIDs. Client to AP broadcast/multicast > >> > traffic works > >> > on of 4 SSIDs. AP to client broadcast/multicast traffic > >> > only works > >> > on 1 of the SSIDs. I have documented this using ARP > >> > broadcasts, > >> > but normal IP broadcasts also observed to corrupted. > >> > > >> > When an ARP request is send through the AP to an > >> > associated client > >> > it seems to be trashed on any of the SSID except the > one > >> > associated > >> > with the last hostapd to be started. Here is the output > of > >> > client side > >> > tcpdump showing the problems. > >> > > >> > In the first client side tcpdump with the hostapd > >> associated > >> > with the SSID > >> > being associaed with the last hostapd started and the > >> traffic > >> > flowing > >> > normally. > >> > > >> > In the second client side tcpdump with the hostapd > >> associated > >> > with the SSID > >> > being not the last hostapd started the ARP request is > >> resent > >> > multiple times > >> > and appears corrupted. > >> > > >> > I would really like to find a fix for this. > >> > Any help would be greatly appreciated. > >> > > >> > > >> > This sounds like the crypto encap of the frame is > clobbering > >> the > >> > mbuf contents. You can verify this by setting up multiple > >> > vaps w/o > >> > WPA. If this is the problem look for the mbuf copy logic > for > >> > mcast > >> > frames and make sure a deep copy is done. > >> > > >> > Sam > >> > > >> > The four VAPs broadcast traffic works find without WPA if I > >> > do not start hostapds on them > >> > I have been trying to discovery why broadcast traffic only > >> > works correctly on the VAP associated with the last hostapd to > >> > be started. I have move with VAP has the working broadcast > >> > traffic by restarting the hostapd > >> > associated with it. > >> > It would seem something in the WPA/802.1x layer > initialization > >> > remembers which hostapd was started last and that affected the > >> > crypto encap. > >> > I keep looking but do not see any place in the code that > could > >> > account for this. > >> > It seems the corrupt crypto encap also happens on broadcast > >> > between stations. > >> > Please correct me if I am wrong: > >> > but when using hostapd normally traffic is bridged withing the > >> card. > >> > So if a station sends to the VAP a broadcast it is actaully > >> > sending a non- broadcast frame to the AP > >> > and the AP sends the frame to all the other stations. > >> > > >> > > >> > I told you waht the likely problem is. Look in the net80211 layer > >> > in the kernel for the problem. > >> > > >> > Sam > >> > > >> > > >> > > >> > > >> > I tried to find problems in mbuf corruption > >> > in ieee80211_output.c by placing > >> > > >> > m = m_unshare(m,M_NOWAIT); > >> > if (m == NULL) { > >> > IEEE80211_DPRINTF(vap, IEEE80211_MSG_OUTPUT, > >> > "%s: cannot get writable mbuf\n", __func__); > >> > return NULL; > >> > } > >> > > >> > at begining ieee80211_mbuf_adjust() and at > >> > beginning of ieee80211_encap() with no change > >> > in the broadcast traffic behaviour. > >> > > >> > I tried then to in ieee80211_crypto.c substituting > >> > > >> > flags |= IEEE80211_KEY_SWCRYPT; > >> > > >> > for the encryption capabilities test code > >> > > >> > if ((ic->ic_cryptocaps & (1< >> > IEEE80211_DPRINTF(vap, IEEE80211_MSG_CRYPTO, > >> > "%s: no h/w support for cipher %s, falling back to s/w\n", > >> > __func__, cip->ic_name); > >> > flags |= IEEE80211_KEY_SWCRYPT; > >> > } > >> > > >> > to force all the encryption to be done in software. > >> > > >> > This fixed the broadcast traffic problem but without > >> > hardware support its very slow and really loads machine. > >> > Enabling in the debug code to ath and net80211 > >> > > >> > and enabled ATH_DEBUG_KEYCACHE in if_ath.c and > >> > IEEE80211_MSG_CRYPTO in net80211 code. > >> > > >> > It seems that all the VAPS sets the broadcast key for > >> > mac ff:ff:ff:ff:ff:ff in the ath device so I assume > >> > they conflict and the last one setting the key is the > >> > working one; that would explain why the last hostapd > >> > started is the only one with working broadcast code > >> > to clients. > >> > > >> > In if_ath.c the code > >> > > >> > if ((k->wk_flags & IEEE80211_KEY_GROUP) && sc->sc_mcastkey) { > >> > /* > >> > * Group keys on hardware that supports multicast frame > >> > * key search use a mac that is the sender's address with > >> > * the high bit set instead of the app-specified address. > >> > */ > >> > IEEE80211_ADDR_COPY(gmac, bss->ni_macaddr); > >> > gmac[0] |= 0x80; > >> > mac = gmac; > >> > } else > >> > mac = k->wk_macaddr; > >> > > >> > seems to indicate that for multiple VAPs the ath chips > >> > needs to be able to distinguish between broadcast keys > >> > by using a permutation of VAPs bssid. > >> > > >> > But in if_athvar.h the code does not seem complete > >> > and sc->sc_mcastkey is also set false. > >> > > >> > #ifdef notyet > >> > #define ath_hal_hasmcastkeysearch(_ah) \ > >> > (ath_hal_getcapability(_ah, HAL_CAP_MCAST_KEYSRCH, 0, NULL) == > >> > HAL_OK) > >> > #define ath_hal_getmcastkeysearch(_ah) \ > >> > (ath_hal_getcapability(_ah, HAL_CAP_MCAST_KEYSRCH, 1, NULL) == > >> > HAL_OK) > >> > #else > >> > #define ath_hal_getmcastkeysearch(_ah) 0 > >> > #endif > >> > > >> > I am using cards with an AR5212 which does seem to have > >> > multiple bssid support working so I hope they should also > >> > support mcastkeysearch capability. Maybe only some > >> > firmware revisions have this? > >> > > >> > Do you know what the status of the HAL code is for > >> > supporting this? Why it is commented out in if_athvar.h? > >> > >> Good work analyzing things; been a long time since I looked at this. I > >> vaguely recall disabling mcastkey searching because of problems with > >> WEP. I'm surprised WPA is broken as that was a standard test case. > >> > >> You can try enabling the notyet code and see if the right thing happens. > >> I don't see any indication of the mac rev for your part but I expect it > >> supports this as it was only very early parts that had issues. > >> > >> If enabling the mcastkey search mechanism doesn't fix this I might've > >> broken things with changes to explicitly mark group keys when hostapd > >> plumbs their contents. I recall doing this for mwl which doesn't have > >> an indexed key table like ath (it uses the mac address of the local bss > >> and/or associated station to find the data structure where crypto keys > >> are stored). > >> > >> I haven't looked at this stuff in a long time and can't setup a system > >> to test but if you keep pushing on this I'll try to help w/ advise. > >> > >> Sam > >> > > > > Sam, > > I have the ath working with multiple hostap's and multicast key search > and > > also > > have the station side working, but I do not like how I got the station > side > > working. > > Let me explain what I have done and what I am tryinig to figure out now. > > I would like to submit a patch once I get the last issues resolved. > > > > In if_athvar.h I uncommented your macros > > > > #define ath_hal_hasmcastkeysearch(_ah) \ > > (ath_hal_getcapability(_ah, HAL_CAP_MCAST_KEYSRCH, 0, NULL) == > > HAL_OK) > > #define ath_hal_getmcastkeysearch(_ah) \ > > (ath_hal_getcapability(_ah, HAL_CAP_MCAST_KEYSRCH, 1, NULL) == > > HAL_OK) > > > > and added a macro > > > > #define ath_hal_setmcastkeysearch(_ah, _v) \ > > ath_hal_setcapability(_ah, HAL_CAP_MCAST_KEYSRCH, 0, _v, NULL) > > In if_ath.c I added > > > > /* > > * if multicast key search is supported by device enable it > > */ > > if (ath_hal_hasmcastkeysearch(sc->sc_ah)) { > > if (!ath_hal_getmcastkeysearch(sc->sc_ah)) { > > ath_hal_setmcastkeysearch(sc->sc_ah,1); > > } > > } > > > > just before > > > > sc->sc_mcastkey = ath_hal_getmcastkeysearch(ah); > > > > in ath_attach(). > > > > Then in ieee80211_ioctl.c I changed ieee80211_ioctl_setkey() so > > not to assign a slot when opering in hostap mode by changing > > > > /* > > * Global slots start off w/o any assigned key index. > > * Force one here for consistency with > IEEE80211_IOC_WEPKEY. > > */ > > if (wk->wk_keyix == IEEE80211_KEYIX_NONE) > > wk->wk_keyix = kid; > > to be > > /* > > * Global slots start off w/o any assigned key index. > > * Force one here for consistency with > IEEE80211_IOC_WEPKEY. > > */ > > if (vap->iv_opmode != IEEE80211_M_HOSTAP > > && wk->wk_keyix == IEEE80211_KEYIX_NONE) > > wk->wk_keyix = kid; > > > > to preserve the station mode operation I had to keep the key index > > assignment > > when not VAP is not operating in hostapd mode. This seems wrong to me. > > > > Now here is the question I am trying to understand. Group keys are > special > > as they are used in only one direction only; sending on hostap and > receiving > > on a station. > > > > The multicast key search code when operating in hostap mode permits the > > lookup > > of the key for encryption by sending VAP bssid on transmission of > multicast > > traffic. > > > > Is there a corresponding station side multicast key search that would let > > the lookup > > of the encryption key be done on receive by looking up the sending VAP > for > > received > > multicast traffic. If so it seems it could enable multiple stations also > to > > work on one > > device. > > > > I noticed in the linux legacy hal ar5212_keycache.c the following > > code/comment > > > > u_int32_t validBit = AR_KEYTABLE_VALID; > > ...... > > /* > > * If upper layers have requested mcast MACaddr lookup, > > then > > * signify this to the hw by setting the (poorly named) > > validBit > > * to 0. Yes, really 0. The hardware specs, > > pcu_registers.txt, is > > * has incorrectly named ValidBit. It should be called "Unicast". > > * When the Key Cache entry is to decrypt Unicast frames, then > this > > * bit should be '1'; for multicast and broadcast frames, this > bit > > is '0'. > > */ > > if (mac[0] & 0x01) { > > validBit = 0; > > } > > ..... > > OS_KC_WRITE(ah, AR_KEYTABLE_MAC0(entry), macLo); > > OS_KC_WRITE(ah, AR_KEYTABLE_MAC1(entry), macHi | validBit); > > > > where as in the freebsd hal in ar5212_keycache.c the AR_KEYTABLE_VALID > > is always set > > > > OS_REG_WRITE(ah, AR_KEYTABLE_MAC0(entry), macLo); > > OS_REG_WRITE(ah, AR_KEYTABLE_MAC1(entry), macHi | > > AR_KEYTABLE_VALID); > > by not setting the AR_KEYTABLE_VALID bit would the multicast key search > code > > work for station mode? I am just guessing here, but seemed like a likely > > explaination? > > > > -Russ > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org > " > > >