From owner-freebsd-stable@FreeBSD.ORG Tue Jan 19 03:29:03 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E9110106566C for ; Tue, 19 Jan 2010 03:29:03 +0000 (UTC) (envelope-from russell.yount@gmail.com) Received: from qw-out-2122.google.com (qw-out-2122.google.com [74.125.92.27]) by mx1.freebsd.org (Postfix) with ESMTP id 9DEFD8FC12 for ; Tue, 19 Jan 2010 03:29:03 +0000 (UTC) Received: by qw-out-2122.google.com with SMTP id 5so795135qwd.7 for ; Mon, 18 Jan 2010 19:29:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=Cus1oYS0qSJ5vKy1D6hzTR+vyZjt6PniOCEKGpJhdHM=; b=W+Xq5QPlyHj30BnbTBpwwRex8quD5Aqe4yxPe2yMijsRHMtUnO+UJ1HyXFGOml62xY dMgGDfZF4OWcKXpN+7Uxo2PQoUt4n+aTCJfSt8tR6wzYI2iEP3dJ5EpwasR4qhE2lGnW sqG2S9P+fJ+qXd0gTOrPpncIbMX1s80Ioroz0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=B12YryYgocv4xwYXbEocqbPMYNenUAV1Xm8ehupA1UlpoWhTDcBp/RNXykTwFCyAuc N1DzMdpLqRTZk6C0NXDXhJ8+AnaM0DcGryYZKCTO8MneF9AHzuiQuX7OGZ/Eskl/puo+ y4voih0gRJT4/zdWlrdWcajwMIDFS8KPE9+OI= MIME-Version: 1.0 Received: by 10.220.126.224 with SMTP id d32mr408936vcs.117.1263871742639; Mon, 18 Jan 2010 19:29:02 -0800 (PST) In-Reply-To: <4B535AAE.3060308@errno.com> References: <4B521FC2.4050402@errno.com> <4B535AAE.3060308@errno.com> Date: Mon, 18 Jan 2010 22:29:02 -0500 Message-ID: From: Russell Yount To: Sam Leffler Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: russell.yount@gmail.com, freebsd-stable@freebsd.org Subject: Re: atheros broadcast/multicast corruption with multiple hostap's X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Jan 2010 03:29:04 -0000 On Sun, Jan 17, 2010 at 1:45 PM, Sam Leffler wrote: > Russell Yount wrote: > > >> >> On Sat, Jan 16, 2010 at 3:21 PM, Sam Leffler > sam@errno.com>> wrote: >> >> Russell Yount wrote: >> >> It seems AP to client broadcasts/multicasts traffic is >> broken when using WPA2/802.11i with multiple hostapds in 8.0. >> >> Only the SSID associated with the last hostapd to be started has >> AP to client broadcasts/multicasts being delivered correctly. >> >> The AP and client are 8.0 freebsd systems althought I see same >> problems with windows XP as a client. >> >> The AP has 4 hostapds configured to use TLS with client >> certificates for >> authentication. (hostapd recompiled with >> HOSTAPD_CFLAGS=-DEAP_SERVER) >> The AP and client radio are shown as ath0: AR5212 mac 5.9 RF5112 >> phy 4.3 >> in dmesg. >> >> Client authenticate using client certificates associate correctly >> to all 4 SSIDs. Unicast traffic flows correctly between clients >> and AP >> for all for 4 SSIDs. Client to AP broadcast/multicast traffic works >> on of 4 SSIDs. AP to client broadcast/multicast traffic only works >> on 1 of the SSIDs. I have documented this using ARP broadcasts, >> but normal IP broadcasts also observed to corrupted. >> >> When an ARP request is send through the AP to an associated client >> it seems to be trashed on any of the SSID except the one associated >> with the last hostapd to be started. Here is the output of >> client side >> tcpdump showing the problems. >> >> In the first client side tcpdump with the hostapd associated >> with the SSID >> being associaed with the last hostapd started and the traffic >> flowing >> normally. >> >> In the second client side tcpdump with the hostapd associated >> with the SSID >> being not the last hostapd started the ARP request is resent >> multiple times >> and appears corrupted. >> >> I would really like to find a fix for this. >> Any help would be greatly appreciated. >> >> >> This sounds like the crypto encap of the frame is clobbering the >> mbuf contents. You can verify this by setting up multiple vaps w/o >> WPA. If this is the problem look for the mbuf copy logic for mcast >> frames and make sure a deep copy is done. >> >> Sam >> >> The four VAPs broadcast traffic works find without WPA if I do not >> start hostapds on them >> I have been trying to discovery why broadcast traffic only works >> correctly on the VAP associated with the last hostapd to be started. I have >> move with VAP has the working broadcast traffic by restarting the hostapd >> associated with it. >> It would seem something in the WPA/802.1x layer initialization remembers >> which hostapd was started last and that affected the crypto encap. >> I keep looking but do not see any place in the code that could account >> for this. >> It seems the corrupt crypto encap also happens on broadcast between >> stations. >> Please correct me if I am wrong: >> but when using hostapd normally traffic is bridged withing the card. >> So if a station sends to the VAP a broadcast it is actaully sending a non- >> broadcast frame to the AP >> and the AP sends the frame to all the other stations. >> > > I told you waht the likely problem is. Look in the net80211 layer in the > kernel for the problem. > > Sam > I tried to find problems in mbuf corruption in ieee80211_output.c by placing m = m_unshare(m,M_NOWAIT); if (m == NULL) { IEEE80211_DPRINTF(vap, IEEE80211_MSG_OUTPUT, "%s: cannot get writable mbuf\n", __func__); return NULL; } at begining ieee80211_mbuf_adjust() and at beginning of ieee80211_encap() with no change in the broadcast traffic behaviour. I tried then to in ieee80211_crypto.c substituting flags |= IEEE80211_KEY_SWCRYPT; for the encryption capabilities test code if ((ic->ic_cryptocaps & (1<ic_name); flags |= IEEE80211_KEY_SWCRYPT; } to force all the encryption to be done in software. This fixed the broadcast traffic problem but without hardware support its very slow and really loads machine. Enabling in the debug code to ath and net80211 and enabled ATH_DEBUG_KEYCACHE in if_ath.c and IEEE80211_MSG_CRYPTO in net80211 code. It seems that all the VAPS sets the broadcast key for mac ff:ff:ff:ff:ff:ff in the ath device so I assume they conflict and the last one setting the key is the working one; that would explain why the last hostapd started is the only one with working broadcast code to clients. In if_ath.c the code if ((k->wk_flags & IEEE80211_KEY_GROUP) && sc->sc_mcastkey) { /* * Group keys on hardware that supports multicast frame * key search use a mac that is the sender's address with * the high bit set instead of the app-specified address. */ IEEE80211_ADDR_COPY(gmac, bss->ni_macaddr); gmac[0] |= 0x80; mac = gmac; } else mac = k->wk_macaddr; seems to indicate that for multiple VAPs the ath chips needs to be able to distinguish between broadcast keys by using a permutation of VAPs bssid. But in if_athvar.h the code does not seem complete and sc->sc_mcastkey is also set false. #ifdef notyet #define ath_hal_hasmcastkeysearch(_ah) \ (ath_hal_getcapability(_ah, HAL_CAP_MCAST_KEYSRCH, 0, NULL) == HAL_OK) #define ath_hal_getmcastkeysearch(_ah) \ (ath_hal_getcapability(_ah, HAL_CAP_MCAST_KEYSRCH, 1, NULL) == HAL_OK) #else #define ath_hal_getmcastkeysearch(_ah) 0 #endif I am using cards with an AR5212 which does seem to have multiple bssid support working so I hope they should also support mcastkeysearch capability. Maybe only some firmware revisions have this? Do you know what the status of the HAL code is for supporting this? Why it is commented out in if_athvar.h? -Russ