From owner-svn-src-head@freebsd.org Sat Nov 21 03:27:33 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AD9A1479A31; Sat, 21 Nov 2020 03:27:33 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CdJks4bXbz3JDd; Sat, 21 Nov 2020 03:27:33 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 903B7215BB; Sat, 21 Nov 2020 03:27:33 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0AL3RXOR084831; Sat, 21 Nov 2020 03:27:33 GMT (envelope-from np@FreeBSD.org) Received: (from np@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0AL3RXRh084830; Sat, 21 Nov 2020 03:27:33 GMT (envelope-from np@FreeBSD.org) Message-Id: <202011210327.0AL3RXRh084830@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: np set sender to np@FreeBSD.org using -f From: Navdeep Parhar Date: Sat, 21 Nov 2020 03:27:33 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r367917 - head/sys/dev/cxgbe X-SVN-Group: head X-SVN-Commit-Author: np X-SVN-Commit-Paths: head/sys/dev/cxgbe X-SVN-Commit-Revision: 367917 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Nov 2020 03:27:33 -0000 Author: np Date: Sat Nov 21 03:27:32 2020 New Revision: 367917 URL: https://svnweb.freebsd.org/changeset/base/367917 Log: cxgbe(4): Catch up with in-flight netmap rx before destroying queues. The netmap application using the driver is responsible for replenishing the receive freelists and they may be totally depleted when the application exits. Packets in flight, if any, might block the pipeline in case there aren't enough buffers left in the freelist. Avoid this by filling up the freelists with a driver allocated buffer. MFC after: 1 week Sponsored by: Chelsio Communications Modified: head/sys/dev/cxgbe/adapter.h head/sys/dev/cxgbe/t4_netmap.c Modified: head/sys/dev/cxgbe/adapter.h ============================================================================== --- head/sys/dev/cxgbe/adapter.h Sat Nov 21 00:53:31 2020 (r367916) +++ head/sys/dev/cxgbe/adapter.h Sat Nov 21 03:27:32 2020 (r367917) @@ -749,6 +749,9 @@ struct sge_nm_rxq { bus_dma_tag_t fl_desc_tag; bus_dmamap_t fl_desc_map; bus_addr_t fl_ba; + + void *bb; /* bit bucket for packets with nowhere to go. */ + uma_zone_t bb_zone; }; #define INVALID_NM_TXQ_CNTXT_ID ((u_int)(-1)) Modified: head/sys/dev/cxgbe/t4_netmap.c ============================================================================== --- head/sys/dev/cxgbe/t4_netmap.c Sat Nov 21 00:53:31 2020 (r367916) +++ head/sys/dev/cxgbe/t4_netmap.c Sat Nov 21 03:27:32 2020 (r367917) @@ -42,6 +42,8 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include +#include #include #include #include @@ -586,6 +588,8 @@ cxgbe_netmap_on(struct adapter *sc, struct vi_info *vi alloc_nm_rxq_hwq(vi, nm_rxq, tnl_cong(vi->pi, nm_cong_drop)); nm_rxq->fl_hwidx = hwidx; + nm_rxq->bb_zone = rxb->zone; + nm_rxq->bb = uma_zalloc(nm_rxq->bb_zone, M_WAITOK); slot = netmap_reset(na, NR_RX, i, 0); MPASS(slot != NULL); /* XXXNM: error check, not assert */ @@ -628,6 +632,31 @@ cxgbe_netmap_on(struct adapter *sc, struct vi_info *vi return (cxgbe_netmap_rss(sc, vi, ifp, na)); } +static void +flush_nm_rxq(struct adapter *sc, struct vi_info *vi, struct sge_nm_rxq *nm_rxq) +{ + int i, n; + u_int fl_pidx, fl_pidx_target, hw_cidx_desc; + const uint64_t ba = pmap_kextract((vm_offset_t)nm_rxq->bb); + + hw_cidx_desc = nm_rxq->fl_cidx / 8; + if (hw_cidx_desc == 0) + fl_pidx_target = nm_rxq->fl_sidx2 - 8; + else + fl_pidx_target = (hw_cidx_desc - 1) * 8; + MPASS((fl_pidx_target & 7) == 0); + + fl_pidx = nm_rxq->fl_pidx; + MPASS((fl_pidx & 7) == 0); + for (n = 0; fl_pidx != fl_pidx_target; n++) { + for (i = 0; i < 8; i++, fl_pidx++) + nm_rxq->fl_desc[fl_pidx] = htobe64(ba | nm_rxq->fl_hwidx); + if (__predict_false(fl_pidx == nm_rxq->fl_sidx2)) + fl_pidx = 0; + } + t4_write_reg(sc, sc->sge_kdoorbell_reg, nm_rxq->fl_db_val | V_PIDX(n)); +} + static int cxgbe_netmap_off(struct adapter *sc, struct vi_info *vi, struct ifnet *ifp, struct netmap_adapter *na) @@ -652,6 +681,23 @@ cxgbe_netmap_off(struct adapter *sc, struct vi_info *v if (rc != 0) return (rc); /* error message logged already. */ + /* + * First pass over the rx queues to make sure they're all caught up. + * + * The freelists could be out of buffers and we may need to arrange + * things so that any packets still in flight (after TP's cong_drop + * logic but not yet DMA'd) have somewhere to go and do not block the + * pipeline. Do this before trying to free any queue. + */ + for_each_nm_rxq(vi, i, nm_rxq) { + nm_state = atomic_load_int(&nm_rxq->nm_state); + kring = na->rx_rings[nm_rxq->nid]; + if (nm_state == NM_OFF || !nm_kring_pending_off(kring)) + continue; + MPASS(nm_rxq->iq_cntxt_id != INVALID_NM_RXQ_CNTXT_ID); + flush_nm_rxq(sc, vi, nm_rxq); + } + for_each_nm_txq(vi, i, nm_txq) { struct sge_qstat *spg = (void *)&nm_txq->desc[nm_txq->sidx]; @@ -688,6 +734,8 @@ cxgbe_netmap_off(struct adapter *sc, struct vi_info *v pause("nmst", 1); free_nm_rxq_hwq(vi, nm_rxq); + uma_zfree(nm_rxq->bb_zone, nm_rxq->bb); + nm_rxq->bb = NULL; /* XXX: netmap, not the driver, should do this. */ kring->rhead = kring->rcur = kring->nr_hwcur = 0;