From owner-svn-src-all@freebsd.org Wed Dec 2 20:54:03 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id ED17F4A6034; Wed, 2 Dec 2020 20:54:03 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CmWRH6PH7z3nFV; Wed, 2 Dec 2020 20:54:03 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CE8A85A48; Wed, 2 Dec 2020 20:54:03 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0B2Ks31A056377; Wed, 2 Dec 2020 20:54:03 GMT (envelope-from np@FreeBSD.org) Received: (from np@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0B2Ks3Uj056375; Wed, 2 Dec 2020 20:54:03 GMT (envelope-from np@FreeBSD.org) Message-Id: <202012022054.0B2Ks3Uj056375@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: np set sender to np@FreeBSD.org using -f From: Navdeep Parhar Date: Wed, 2 Dec 2020 20:54:03 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r368283 - head/sys/dev/cxgbe X-SVN-Group: head X-SVN-Commit-Author: np X-SVN-Commit-Paths: head/sys/dev/cxgbe X-SVN-Commit-Revision: 368283 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Dec 2020 20:54:04 -0000 Author: np Date: Wed Dec 2 20:54:03 2020 New Revision: 368283 URL: https://svnweb.freebsd.org/changeset/base/368283 Log: cxgbe(4): Revert r367917. r367917 fixed the backpressure on the netmap rxq being stopped but that doesn't help if some other netmap rxq is starved (because it is stopping too although the driver doesn't know this yet) and blocks the pipeline. An alternate fix that works in all cases will be checked in instead. Sponsored by: Chelsio Communications Modified: head/sys/dev/cxgbe/adapter.h head/sys/dev/cxgbe/t4_netmap.c Modified: head/sys/dev/cxgbe/adapter.h ============================================================================== --- head/sys/dev/cxgbe/adapter.h Wed Dec 2 19:58:50 2020 (r368282) +++ head/sys/dev/cxgbe/adapter.h Wed Dec 2 20:54:03 2020 (r368283) @@ -749,9 +749,6 @@ struct sge_nm_rxq { bus_dma_tag_t fl_desc_tag; bus_dmamap_t fl_desc_map; bus_addr_t fl_ba; - - void *bb; /* bit bucket for packets with nowhere to go. */ - uma_zone_t bb_zone; }; #define INVALID_NM_TXQ_CNTXT_ID ((u_int)(-1)) Modified: head/sys/dev/cxgbe/t4_netmap.c ============================================================================== --- head/sys/dev/cxgbe/t4_netmap.c Wed Dec 2 19:58:50 2020 (r368282) +++ head/sys/dev/cxgbe/t4_netmap.c Wed Dec 2 20:54:03 2020 (r368283) @@ -42,8 +42,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include -#include #include #include #include @@ -588,8 +586,6 @@ cxgbe_netmap_on(struct adapter *sc, struct vi_info *vi alloc_nm_rxq_hwq(vi, nm_rxq, tnl_cong(vi->pi, nm_cong_drop)); nm_rxq->fl_hwidx = hwidx; - nm_rxq->bb_zone = rxb->zone; - nm_rxq->bb = uma_zalloc(nm_rxq->bb_zone, M_WAITOK); slot = netmap_reset(na, NR_RX, i, 0); MPASS(slot != NULL); /* XXXNM: error check, not assert */ @@ -632,31 +628,6 @@ cxgbe_netmap_on(struct adapter *sc, struct vi_info *vi return (cxgbe_netmap_rss(sc, vi, ifp, na)); } -static void -flush_nm_rxq(struct adapter *sc, struct vi_info *vi, struct sge_nm_rxq *nm_rxq) -{ - int i, n; - u_int fl_pidx, fl_pidx_target, hw_cidx_desc; - const uint64_t ba = pmap_kextract((vm_offset_t)nm_rxq->bb); - - hw_cidx_desc = nm_rxq->fl_cidx / 8; - if (hw_cidx_desc == 0) - fl_pidx_target = nm_rxq->fl_sidx2 - 8; - else - fl_pidx_target = (hw_cidx_desc - 1) * 8; - MPASS((fl_pidx_target & 7) == 0); - - fl_pidx = nm_rxq->fl_pidx; - MPASS((fl_pidx & 7) == 0); - for (n = 0; fl_pidx != fl_pidx_target; n++) { - for (i = 0; i < 8; i++, fl_pidx++) - nm_rxq->fl_desc[fl_pidx] = htobe64(ba | nm_rxq->fl_hwidx); - if (__predict_false(fl_pidx == nm_rxq->fl_sidx2)) - fl_pidx = 0; - } - t4_write_reg(sc, sc->sge_kdoorbell_reg, nm_rxq->fl_db_val | V_PIDX(n)); -} - static int cxgbe_netmap_off(struct adapter *sc, struct vi_info *vi, struct ifnet *ifp, struct netmap_adapter *na) @@ -681,23 +652,6 @@ cxgbe_netmap_off(struct adapter *sc, struct vi_info *v if (rc != 0) return (rc); /* error message logged already. */ - /* - * First pass over the rx queues to make sure they're all caught up. - * - * The freelists could be out of buffers and we may need to arrange - * things so that any packets still in flight (after TP's cong_drop - * logic but not yet DMA'd) have somewhere to go and do not block the - * pipeline. Do this before trying to free any queue. - */ - for_each_nm_rxq(vi, i, nm_rxq) { - nm_state = atomic_load_int(&nm_rxq->nm_state); - kring = na->rx_rings[nm_rxq->nid]; - if (nm_state == NM_OFF || !nm_kring_pending_off(kring)) - continue; - MPASS(nm_rxq->iq_cntxt_id != INVALID_NM_RXQ_CNTXT_ID); - flush_nm_rxq(sc, vi, nm_rxq); - } - for_each_nm_txq(vi, i, nm_txq) { struct sge_qstat *spg = (void *)&nm_txq->desc[nm_txq->sidx]; @@ -734,8 +688,6 @@ cxgbe_netmap_off(struct adapter *sc, struct vi_info *v pause("nmst", 1); free_nm_rxq_hwq(vi, nm_rxq); - uma_zfree(nm_rxq->bb_zone, nm_rxq->bb); - nm_rxq->bb = NULL; /* XXX: netmap, not the driver, should do this. */ kring->rhead = kring->rcur = kring->nr_hwcur = 0;