From owner-svn-src-head@freebsd.org Mon Oct 22 23:36:13 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F254FE8E82; Mon, 22 Oct 2018 23:36:13 +0000 (UTC) (envelope-from freebsd@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 225B579988; Mon, 22 Oct 2018 23:36:12 +0000 (UTC) (envelope-from freebsd@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w9MNa925014744; Mon, 22 Oct 2018 16:36:09 -0700 (PDT) (envelope-from freebsd@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w9MNa9Zm014743; Mon, 22 Oct 2018 16:36:09 -0700 (PDT) (envelope-from freebsd) From: "Rodney W. Grimes" Message-Id: <201810222336.w9MNa9Zm014743@pdx.rh.CN85.dnsmgr.net> Subject: Re: svn commit: r339626 - head/sys/dev/cxgbe In-Reply-To: <201810222306.w9MN6Oiu080382@repo.freebsd.org> To: Navdeep Parhar Date: Mon, 22 Oct 2018 16:36:09 -0700 (PDT) CC: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Reply-To: rgrimes@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2018 23:36:13 -0000 > Author: np > Date: Mon Oct 22 23:06:23 2018 > New Revision: 339626 > URL: https://svnweb.freebsd.org/changeset/base/339626 > > Log: > cxgbe(4): Use automatic cidx updates with ofld and ctrl queues. > > The bits that explicitly request cidx updates do not work reliably with > all possible WRs that can be sent over the queue. The F_FW_WR_EQUIQ > requests that still remain may also have to be replaced with explicit > credit flush WRs in the future. > > MFC after: 2 days Please note in your RFA to re@ your reasons for requesting early merge to a frozen stable branch. I see good reason for this to be approved as it would be good to get this in the next snapshot, but it usually would be rejected if there is not a reason given. (There is a normal 3 day minimum on MFC after to a frozen branch) Thanks, Rod Sponsored by: Chelsio Communications > > Modified: > head/sys/dev/cxgbe/t4_sge.c > > Modified: head/sys/dev/cxgbe/t4_sge.c > ============================================================================== > --- head/sys/dev/cxgbe/t4_sge.c Mon Oct 22 22:24:32 2018 (r339625) > +++ head/sys/dev/cxgbe/t4_sge.c Mon Oct 22 23:06:23 2018 (r339626) > @@ -2089,12 +2089,13 @@ drain_wrq_wr_list(struct adapter *sc, struct sge_wrq * > > if (available < eq->sidx / 4 && > atomic_cmpset_int(&eq->equiq, 0, 1)) { > + /* > + * XXX: This is not 100% reliable with some > + * types of WRs. But this is a very unusual > + * situation for an ofld/ctrl queue anyway. > + */ > dst->equiq_to_len16 |= htobe32(F_FW_WR_EQUIQ | > F_FW_WR_EQUEQ); > - eq->equeqidx = eq->pidx; > - } else if (IDXDIFF(eq->pidx, eq->equeqidx, eq->sidx) >= 32) { > - dst->equiq_to_len16 |= htobe32(F_FW_WR_EQUEQ); > - eq->equeqidx = eq->pidx; > } > > dbdiff += n; > @@ -2644,12 +2645,13 @@ commit_wrq_wr(struct sge_wrq *wrq, void *w, struct wrq > available = IDXDIFF(eq->cidx, eq->pidx, eq->sidx) - 1; > if (available < eq->sidx / 4 && > atomic_cmpset_int(&eq->equiq, 0, 1)) { > + /* > + * XXX: This is not 100% reliable with some > + * types of WRs. But this is a very unusual > + * situation for an ofld/ctrl queue anyway. > + */ > dst->equiq_to_len16 |= htobe32(F_FW_WR_EQUIQ | > F_FW_WR_EQUEQ); > - eq->equeqidx = pidx; > - } else if (IDXDIFF(eq->pidx, eq->equeqidx, eq->sidx) >= 32) { > - dst->equiq_to_len16 |= htobe32(F_FW_WR_EQUEQ); > - eq->equeqidx = pidx; > } > > ring_eq_db(wrq->adapter, eq, ndesc); > @@ -3584,6 +3586,23 @@ free_nm_txq(struct vi_info *vi, struct sge_nm_txq *nm_ > } > #endif > > +/* > + * Returns a reasonable automatic cidx flush threshold for a given queue size. > + */ > +static u_int > +qsize_to_fthresh(int qsize) > +{ > + u_int fthresh; > + > + while (!powerof2(qsize)) > + qsize++; > + fthresh = ilog2(qsize); > + if (fthresh > X_CIDXFLUSHTHRESH_128) > + fthresh = X_CIDXFLUSHTHRESH_128; > + > + return (fthresh); > +} > + > static int > ctrl_eq_alloc(struct adapter *sc, struct sge_eq *eq) > { > @@ -3607,7 +3626,7 @@ ctrl_eq_alloc(struct adapter *sc, struct sge_eq *eq) > c.dcaen_to_eqsize = > htobe32(V_FW_EQ_CTRL_CMD_FBMIN(X_FETCHBURSTMIN_64B) | > V_FW_EQ_CTRL_CMD_FBMAX(X_FETCHBURSTMAX_512B) | > - V_FW_EQ_CTRL_CMD_CIDXFTHRESH(X_CIDXFLUSHTHRESH_32) | > + V_FW_EQ_CTRL_CMD_CIDXFTHRESH(qsize_to_fthresh(qsize)) | > V_FW_EQ_CTRL_CMD_EQSIZE(qsize)); > c.eqaddr = htobe64(eq->ba); > > @@ -3689,12 +3708,13 @@ ofld_eq_alloc(struct adapter *sc, struct vi_info *vi, > c.alloc_to_len16 = htonl(F_FW_EQ_OFLD_CMD_ALLOC | > F_FW_EQ_OFLD_CMD_EQSTART | FW_LEN16(c)); > c.fetchszm_to_iqid = > - htonl(V_FW_EQ_OFLD_CMD_HOSTFCMODE(X_HOSTFCMODE_NONE) | > + htonl(V_FW_EQ_OFLD_CMD_HOSTFCMODE(X_HOSTFCMODE_STATUS_PAGE) | > V_FW_EQ_OFLD_CMD_PCIECHN(eq->tx_chan) | > F_FW_EQ_OFLD_CMD_FETCHRO | V_FW_EQ_OFLD_CMD_IQID(eq->iqid)); > c.dcaen_to_eqsize = > htobe32(V_FW_EQ_OFLD_CMD_FBMIN(X_FETCHBURSTMIN_64B) | > V_FW_EQ_OFLD_CMD_FBMAX(X_FETCHBURSTMAX_512B) | > + V_FW_EQ_OFLD_CMD_CIDXFTHRESH(qsize_to_fthresh(qsize)) | > V_FW_EQ_OFLD_CMD_EQSIZE(qsize)); > c.eqaddr = htobe64(eq->ba); > > @@ -3732,8 +3752,9 @@ alloc_eq(struct adapter *sc, struct vi_info *vi, struc > if (rc) > return (rc); > > - eq->pidx = eq->cidx = 0; > - eq->equeqidx = eq->dbidx = 0; > + eq->pidx = eq->cidx = eq->dbidx = 0; > + /* Note that equeqidx is not used with sge_wrq (OFLD/CTRL) queues. */ > + eq->equeqidx = 0; > eq->doorbells = sc->doorbells; > > switch (eq->flags & EQ_TYPEMASK) { > > -- Rod Grimes rgrimes@freebsd.org