From owner-svn-src-user@FreeBSD.ORG Sun Jun 12 09:18:48 2011 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A31AD106566C; Sun, 12 Jun 2011 09:18:48 +0000 (UTC) (envelope-from adrian@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 92AF68FC13; Sun, 12 Jun 2011 09:18:48 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id p5C9Imr1016562; Sun, 12 Jun 2011 09:18:48 GMT (envelope-from adrian@svn.freebsd.org) Received: (from adrian@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id p5C9Immf016557; Sun, 12 Jun 2011 09:18:48 GMT (envelope-from adrian@svn.freebsd.org) Message-Id: <201106120918.p5C9Immf016557@svn.freebsd.org> From: Adrian Chadd Date: Sun, 12 Jun 2011 09:18:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r223004 - user/adrian/if_ath_tx/sys/dev/ath X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jun 2011 09:18:48 -0000 Author: adrian Date: Sun Jun 12 09:18:48 2011 New Revision: 223004 URL: http://svn.freebsd.org/changeset/base/223004 Log: Add code to flush the software queues whenever the interface is reset. * Add functionality to flush all software-queued packets for a TID. * Add a function to drain all software queued packets for the given node * Add a function that drains all software queued packets for the given interface - so, all nodes for all vaps. This is needed because now that packets are on the sw q as well as hw q, an interface reset, channel change, etc can result in software queued packets whose rate control bits now don't match what the channel is setup for. For example, during scan, the interface is in a non-ht40 mode and any ht/40 packets queued to it will end up annoying the baseband. ath_vap_delete() also deleted all entries from the hw and sw txq's; it's possible we could get away with only flushing entries for the given vap's sw tx queues. Maybe I'll do that later. Note: I've discovered no node locking is done, so the rate control code is likely still racy. Blah. Finally, I'm not sure if the ieee80211 common lock should be held when iterating like this over the nodes. It doesn't seem so - it isn't held at the moment. I'm still concerned about subtle race conditions with the TX path and other things such as scan and interface reset. I'll do some further digging later as it seems that breaking out the TX path into said software queue has made some racy stuff a little more .. obvious. Modified: user/adrian/if_ath_tx/sys/dev/ath/if_ath.c user/adrian/if_ath_tx/sys/dev/ath/if_ath_misc.h user/adrian/if_ath_tx/sys/dev/ath/if_ath_tx.c user/adrian/if_ath_tx/sys/dev/ath/if_ath_tx.h Modified: user/adrian/if_ath_tx/sys/dev/ath/if_ath.c ============================================================================== --- user/adrian/if_ath_tx/sys/dev/ath/if_ath.c Sun Jun 12 02:05:59 2011 (r223003) +++ user/adrian/if_ath_tx/sys/dev/ath/if_ath.c Sun Jun 12 09:18:48 2011 (r223004) @@ -175,7 +175,6 @@ static void ath_tx_cleanup(struct ath_so static void ath_tx_proc_q0(void *, int); static void ath_tx_proc_q0123(void *, int); static void ath_tx_proc(void *, int); -static void ath_tx_draintxq(struct ath_softc *, struct ath_txq *); static int ath_chan_set(struct ath_softc *, struct ieee80211_channel *); static void ath_draintxq(struct ath_softc *); static void ath_stoprecv(struct ath_softc *); @@ -202,6 +201,7 @@ static void ath_setcurmode(struct ath_so static void ath_announce(struct ath_softc *); static void ath_dfs_tasklet(void *, int); +static void ath_sc_flushtxq(struct ath_softc *sc); #ifdef IEEE80211_SUPPORT_TDMA static void ath_tdma_settimers(struct ath_softc *sc, u_int32_t nexttbtt, @@ -1130,7 +1130,8 @@ ath_vap_delete(struct ieee80211vap *vap) * the vap state by any frames pending on the tx queues. */ ath_hal_intrset(ah, 0); /* disable interrupts */ - ath_draintxq(sc); /* stop xmit side */ + ath_draintxq(sc); /* stop hw xmit side */ + ath_sc_flushtxq(sc); /* drain sw xmit side */ ath_stoprecv(sc); /* stop recv side */ } @@ -1678,6 +1679,7 @@ ath_stop_locked(struct ifnet *ifp) ath_hal_intrset(ah, 0); } ath_draintxq(sc); + ath_sc_flushtxq(sc); /* drain sw xmit side */ if (!sc->sc_invalid) { ath_stoprecv(sc); ath_hal_phydisable(ah); @@ -1714,6 +1716,7 @@ ath_reset(struct ifnet *ifp) ath_hal_intrset(ah, 0); /* disable interrupts */ ath_draintxq(sc); /* stop xmit side */ + ath_sc_flushtxq(sc); /* drain sw xmit side */ ath_stoprecv(sc); /* stop recv side */ ath_settkipmic(sc); /* configure TKIP MIC handling */ /* NB: indicate channel change so we do a full reset */ @@ -4204,13 +4207,43 @@ ath_tx_proc(void *arg, int npending) ath_start(ifp); } -static void +/* + * This is currently used by ath_tx_draintxq() and + * ath_tx_tid_free_pkts(). + * + * It recycles a single ath_buf. + */ +void +ath_tx_buf_drainone(struct ath_softc *sc, struct ath_buf *bf) +{ + struct ieee80211_node *ni; + + bus_dmamap_unload(sc->sc_dmat, bf->bf_dmamap); + ni = bf->bf_node; + bf->bf_node = NULL; + if (ni != NULL) { + /* + * Do any callback and reclaim the node reference. + */ + if (bf->bf_m->m_flags & M_TXCB) + ieee80211_process_callback(ni, bf->bf_m, -1); + ieee80211_free_node(ni); + } + m_freem(bf->bf_m); + bf->bf_m = NULL; + bf->bf_flags &= ~ATH_BUF_BUSY; + + ATH_TXBUF_LOCK(sc); + STAILQ_INSERT_TAIL(&sc->sc_txbuf, bf, bf_list); + ATH_TXBUF_UNLOCK(sc); +} + +void ath_tx_draintxq(struct ath_softc *sc, struct ath_txq *txq) { #ifdef ATH_DEBUG struct ath_hal *ah = sc->sc_ah; #endif - struct ieee80211_node *ni; struct ath_buf *bf; u_int ix; @@ -4244,24 +4277,7 @@ ath_tx_draintxq(struct ath_softc *sc, st bf->bf_m->m_len, 0, -1); } #endif /* ATH_DEBUG */ - bus_dmamap_unload(sc->sc_dmat, bf->bf_dmamap); - ni = bf->bf_node; - bf->bf_node = NULL; - if (ni != NULL) { - /* - * Do any callback and reclaim the node reference. - */ - if (bf->bf_m->m_flags & M_TXCB) - ieee80211_process_callback(ni, bf->bf_m, -1); - ieee80211_free_node(ni); - } - m_freem(bf->bf_m); - bf->bf_m = NULL; - bf->bf_flags &= ~ATH_BUF_BUSY; - - ATH_TXBUF_LOCK(sc); - STAILQ_INSERT_TAIL(&sc->sc_txbuf, bf, bf_list); - ATH_TXBUF_UNLOCK(sc); + ath_tx_buf_drainone(sc, bf); } } @@ -4434,6 +4450,7 @@ ath_chan_set(struct ath_softc *sc, struc */ ath_hal_intrset(ah, 0); /* disable interrupts */ ath_draintxq(sc); /* clear pending tx frames */ + ath_sc_flushtxq(sc); /* drain sw xmit side */ ath_stoprecv(sc); /* turn off frame recv */ if (!ath_hal_reset(ah, sc->sc_opmode, chan, AH_TRUE, &status)) { if_printf(ifp, "%s: unable to reset " @@ -5731,5 +5748,38 @@ ath_dfs_tasklet(void *p, int npending) } } +/* + * Flush all software queued packets for the given VAP. + * + * The ieee80211 common lock should be held. + */ +static void +ath_vap_flush_node(void *arg, struct ieee80211_node *ni) +{ + struct ath_softc *sc = (struct ath_softc *) arg; + + ath_tx_node_flush(sc, ATH_NODE(ni)); +} + +/* + * Flush all software queued packets for the given sc. + * + * The ieee80211 common lock should be held. + */ +static void +ath_sc_flushtxq(struct ath_softc *sc) +{ + struct ieee80211com *ic = sc->sc_ifp->if_l2com; + + //IEEE80211_LOCK_ASSERT(ic); + /* Debug if we don't own the lock! */ + if (! mtx_owned(IEEE80211_LOCK_OBJ(ic))) { + device_printf(sc->sc_dev, "%s: comlock not owned?\n", + __func__); + } + + ieee80211_iterate_nodes(&ic->ic_sta, ath_vap_flush_node, sc); +} + MODULE_VERSION(if_ath, 1); MODULE_DEPEND(if_ath, wlan, 1, 1, 1); /* 802.11 media layer */ Modified: user/adrian/if_ath_tx/sys/dev/ath/if_ath_misc.h ============================================================================== --- user/adrian/if_ath_tx/sys/dev/ath/if_ath_misc.h Sun Jun 12 02:05:59 2011 (r223003) +++ user/adrian/if_ath_tx/sys/dev/ath/if_ath_misc.h Sun Jun 12 09:18:48 2011 (r223004) @@ -54,5 +54,7 @@ extern struct ath_buf * ath_getbuf(struc extern struct ath_buf * _ath_getbuf_locked(struct ath_softc *sc); extern int ath_reset(struct ifnet *); +extern void ath_tx_draintxq(struct ath_softc *sc, struct ath_txq *txq); +extern void ath_tx_buf_drainone(struct ath_softc *sc, struct ath_buf *bf); #endif Modified: user/adrian/if_ath_tx/sys/dev/ath/if_ath_tx.c ============================================================================== --- user/adrian/if_ath_tx/sys/dev/ath/if_ath_tx.c Sun Jun 12 02:05:59 2011 (r223003) +++ user/adrian/if_ath_tx/sys/dev/ath/if_ath_tx.c Sun Jun 12 09:18:48 2011 (r223004) @@ -1371,21 +1371,45 @@ ath_tx_tid_txq_unmark(struct ath_softc * * * Since net80211 shouldn't free the node until the last packets * have been sent, this function should never have to free any - * packets. + * packets when a node is freed. + * + * It can also be called on an active node during an interface + * reset or state transition. */ static void ath_tx_tid_free_pkts(struct ath_softc *sc, struct ath_node *an, int tid) { struct ath_tid *atid = &an->an_tid[tid]; - struct ieee80211_node *ni = &an->an_node; + struct ath_buf *bf; - /* XXX TODO */ - /* For now, just print a loud warning if it occurs */ - if (! STAILQ_EMPTY(&atid->axq_q)) - device_printf(sc->sc_dev, "%s: AID %d TID %d queue not " - "empty on queue destroy!\n", - __func__, ni->ni_associd, tid); + + /* Walk the queue, free frames */ + ATH_TXQ_LOCK(atid); + for (;;) { + bf = STAILQ_FIRST(&atid->axq_q); + if (bf == NULL) + break; + ATH_TXQ_REMOVE_HEAD(atid, bf_list); + ath_tx_buf_drainone(sc, bf); + } + ATH_TXQ_UNLOCK(atid); +} + +/* + * Flush all software queued packets for the given node. + * + * This protects ath_node behind ATH_LOCK for now. + */ +void +ath_tx_node_flush(struct ath_softc *sc, struct ath_node *an) +{ + int tid; + + ATH_LOCK(sc); + for (tid = 0; tid < IEEE80211_TID_SIZE; tid++) + ath_tx_tid_free_pkts(sc, an, tid); + ATH_UNLOCK(sc); } /* @@ -1399,11 +1423,19 @@ ath_tx_tid_cleanup(struct ath_softc *sc, { int i; struct ath_tid *atid; + struct ieee80211_node *ni = &an->an_node; for (i = 0; i < IEEE80211_TID_SIZE; i++) { atid = &an->an_tid[i]; /* Free packets in sw queue */ + /* For now, just print a loud warning if it occurs */ + ATH_TXQ_LOCK(atid); + if (! STAILQ_EMPTY(&atid->axq_q)) + device_printf(sc->sc_dev, "%s: AID %d TID %d queue not " + "empty on queue destroy!\n", + __func__, ni->ni_associd, i); + ATH_TXQ_UNLOCK(atid); ath_tx_tid_free_pkts(sc, an, i); /* Mark hw-queued packets as having no parent now */ Modified: user/adrian/if_ath_tx/sys/dev/ath/if_ath_tx.h ============================================================================== --- user/adrian/if_ath_tx/sys/dev/ath/if_ath_tx.h Sun Jun 12 02:05:59 2011 (r223003) +++ user/adrian/if_ath_tx/sys/dev/ath/if_ath_tx.h Sun Jun 12 09:18:48 2011 (r223004) @@ -32,6 +32,7 @@ #define __IF_ATH_TX_H__ extern void ath_freetx(struct mbuf *m); +extern void ath_tx_node_flush(struct ath_softc *sc, struct ath_node *an); extern void ath_txfrag_cleanup(struct ath_softc *sc, ath_bufhead *frags, struct ieee80211_node *ni); extern int ath_txfrag_setup(struct ath_softc *sc, ath_bufhead *frags,