From nobody Wed Feb 25 02:24:05 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4fLJLZ12xfz6TcWC for ; Wed, 25 Feb 2026 02:24:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4fLJLZ0QWhz3fNt for ; Wed, 25 Feb 2026 02:24:06 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1771986246; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=sXIsiDLs9MZebfc0p4toXHHkUtBk9Od6rhpjmhGn9Ig=; b=mjinFCMEw9ev2HXwcZA9FvLN2BJlsYjaQLXOCCkJZP8VmDW7ko1rgfAe8LQ3/XwyGTbhGb 04nfJtE8cKBOASLTbuQll7bbUjKzF0QflvJBZuJdjrUDIXX1Idz8dp73UuLGkkBBq6tBLk 5huMhxs9eghIcIV6knvDEH9SoS2r8fSQQWhJR7u2p8QDp+SlYSrd9nNqIIOlMp/UjOPl9B 3BBxKRku/MAXjPFiQD5Z92QlVVDR4lSFsVLaIqLukmtnPXMNAKbWGTZqfZTskFBlgvT2SN 9ZMTdjewMxDK7Rvgya/xhHC0SZMkSwcGGxQm1ndruVPxsmMqTm7jM5wHSH40Nw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1771986246; a=rsa-sha256; cv=none; b=eqFm6MAjWztiYuXnIBQ3A9jUU4BwYUDQn+p4lL0Ihpreq52zRtd0O51vwBVkccDr38SWS6 9Z91kG3UlrR6Z0BfJpr9XgauY+dZDpzsaNgZzw56n1pkX+RjFI7wtFpRYdD7BcAyC+CX1Z +IW7VuSCD9TepA3CZq5eyURYJT/HGXGnB6n4DR66OaDC/iZS8fpVak0oDcrFm4tnP8Q3vU iWJNU776jnrmrRkdEMQWY0QY2zquqeduwzGe5exBtI4s8V3J8STYlfWH8NGBQcYyDRX2+F 70uMtc2FiJ9GXFjzQK/Ixv9a3g0MX2760BBZ9S9E9Wj5g6IKPEJNq7e+USKqHw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1771986246; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=sXIsiDLs9MZebfc0p4toXHHkUtBk9Od6rhpjmhGn9Ig=; b=VelTqQFaoex4RhNl17SjcJmDMFdgdX2FjX0c+rCwpn9dk14wfJl0TCK83K/yIbAFKUjywP zdPlwCktB1aHUdYA+NPtHlfpY1SvVI4Wd4fzdikDXKnrhL/PVqgKU9ANBLxgr0yiwcvV76 YJge9+j6KmmTcD9MUaWbX91r+FaMuzmoMvPRLl35RsGqfkIdP+pgXVAH5IeZ74Qzy4gL98 iEUircroVcF/t5jCCdGorICth3DQmMR2xygHbd99E9WusIqtpPJMIVN9Oj8u2MxSgf+4mA KxVO8JRT0k9rp9WMc8y2o0FzFUAvDKoJFNT1fjLzFg2xsz3PchegLjlcrTqFFA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4fLJLY6wF9zk1q for ; Wed, 25 Feb 2026 02:24:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 21a33 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Wed, 25 Feb 2026 02:24:05 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: e43730e585fa - main - cxgbe tom: Use the same WRs as iSCSI to send PDUs for NVMe List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: e43730e585fab3385c5b774b83e95e16c984dacc Auto-Submitted: auto-generated Date: Wed, 25 Feb 2026 02:24:05 +0000 Message-Id: <699e5d45.21a33.29061bdf@gitrepo.freebsd.org> The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=e43730e585fab3385c5b774b83e95e16c984dacc commit e43730e585fab3385c5b774b83e95e16c984dacc Author: John Baldwin AuthorDate: 2026-02-25 02:23:39 +0000 Commit: John Baldwin CommitDate: 2026-02-25 02:23:39 +0000 cxgbe tom: Use the same WRs as iSCSI to send PDUs for NVMe Reviewed by: np (earlier version) Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D55470 --- sys/dev/cxgbe/tom/t4_cpl_io.c | 112 +++++------------------------------------- 1 file changed, 13 insertions(+), 99 deletions(-) diff --git a/sys/dev/cxgbe/tom/t4_cpl_io.c b/sys/dev/cxgbe/tom/t4_cpl_io.c index 5c39ae5fa8f3..7e1c497240c2 100644 --- a/sys/dev/cxgbe/tom/t4_cpl_io.c +++ b/sys/dev/cxgbe/tom/t4_cpl_io.c @@ -496,9 +496,6 @@ t4_close_conn(struct adapter *sc, struct toepcb *toep) #define MIN_ISO_TX_CREDITS (howmany(sizeof(struct cpl_tx_data_iso), 16)) #define MIN_TX_CREDITS(iso) \ (MIN_OFLD_TX_CREDITS + ((iso) ? MIN_ISO_TX_CREDITS : 0)) -#define MIN_OFLD_TX_V2_CREDITS (howmany(sizeof(struct fw_ofld_tx_data_v2_wr) + 1, 16)) -#define MIN_TX_V2_CREDITS(iso) \ - (MIN_OFLD_TX_V2_CREDITS + ((iso) ? MIN_ISO_TX_CREDITS : 0)) _Static_assert(MAX_OFLD_TX_CREDITS <= MAX_OFLD_TX_SDESC_CREDITS, "MAX_OFLD_TX_SDESC_CREDITS too small"); @@ -546,46 +543,6 @@ max_dsgl_nsegs(int tx_credits, int iso) return (nseg); } -/* Maximum amount of immediate data we could stuff in a WR */ -static inline int -max_imm_payload_v2(int tx_credits, int iso) -{ - const int iso_cpl_size = iso ? sizeof(struct cpl_tx_data_iso) : 0; - - KASSERT(tx_credits >= 0 && - tx_credits <= MAX_OFLD_TX_CREDITS, - ("%s: %d credits", __func__, tx_credits)); - - if (tx_credits < MIN_TX_V2_CREDITS(iso)) - return (0); - - return (tx_credits * 16 - sizeof(struct fw_ofld_tx_data_v2_wr) - - iso_cpl_size); -} - -/* Maximum number of SGL entries we could stuff in a WR */ -static inline int -max_dsgl_nsegs_v2(int tx_credits, int iso, int imm_payload) -{ - int nseg = 1; /* ulptx_sgl has room for 1, rest ulp_tx_sge_pair */ - int sge_pair_credits = tx_credits - MIN_TX_V2_CREDITS(iso); - - KASSERT(tx_credits >= 0 && - tx_credits <= MAX_OFLD_TX_CREDITS, - ("%s: %d credits", __func__, tx_credits)); - - if (tx_credits < MIN_TX_V2_CREDITS(iso) || - sge_pair_credits <= howmany(imm_payload, 16)) - return (0); - sge_pair_credits -= howmany(imm_payload, 16); - - nseg += 2 * (sge_pair_credits * 16 / 24); - if ((sge_pair_credits * 16) % 24 == 16) - nseg++; - - return (nseg); -} - static inline void write_tx_wr(void *dst, struct toepcb *toep, int fw_wr_opcode, unsigned int immdlen, unsigned int plen, uint8_t credits, int shove, @@ -613,35 +570,6 @@ write_tx_wr(void *dst, struct toepcb *toep, int fw_wr_opcode, } } -static inline void -write_tx_v2_wr(void *dst, struct toepcb *toep, int fw_wr_opcode, - unsigned int immdlen, unsigned int plen, uint8_t credits, int shove, - int ulp_submode) -{ - struct fw_ofld_tx_data_v2_wr *txwr = dst; - uint32_t flags; - - memset(txwr, 0, sizeof(*txwr)); - txwr->op_to_immdlen = htobe32(V_WR_OP(fw_wr_opcode) | - V_FW_WR_IMMDLEN(immdlen)); - txwr->flowid_len16 = htobe32(V_FW_WR_FLOWID(toep->tid) | - V_FW_WR_LEN16(credits)); - txwr->plen = htobe32(plen); - flags = V_TX_ULP_MODE(ULP_MODE_NVMET) | V_TX_ULP_SUBMODE(ulp_submode) | - V_TX_URG(0) | V_TX_SHOVE(shove); - - if (toep->params.tx_align > 0) { - if (plen < 2 * toep->params.emss) - flags |= F_FW_OFLD_TX_DATA_WR_LSODISABLE; - else - flags |= F_FW_OFLD_TX_DATA_WR_ALIGNPLD | - (toep->params.nagle == 0 ? 0 : - F_FW_OFLD_TX_DATA_WR_ALIGNPLDSHOVE); - } - - txwr->lsodisable_to_flags = htobe32(flags); -} - /* * Generate a DSGL from a starting mbuf. The total number of segments and the * maximum segments in any one mbuf are provided. @@ -1300,7 +1228,7 @@ write_nvme_mbuf_wr(struct toepcb *toep, struct mbuf *sndptr) { struct mbuf *m; const struct nvme_tcp_common_pdu_hdr *hdr; - struct fw_v2_nvmet_tx_data_wr *txwr; + struct fw_ofld_tx_data_wr *txwr; struct cpl_tx_data_iso *cpl_iso; void *p; struct wrqe *wr; @@ -1330,29 +1258,16 @@ write_nvme_mbuf_wr(struct toepcb *toep, struct mbuf *sndptr) return (wr); } - /* - * The first mbuf is the PDU header that is always sent as - * immediate data. - */ - imm_data = sndptr->m_len; - iso = mbuf_iscsi_iso(sndptr); - max_imm = max_imm_payload_v2(tx_credits, iso); - - /* - * Not enough credits for the PDU header. - */ - if (imm_data > max_imm) - return (NULL); - - max_nsegs = max_dsgl_nsegs_v2(tx_credits, iso, imm_data); + max_imm = max_imm_payload(tx_credits, iso); + max_nsegs = max_dsgl_nsegs(tx_credits, iso); iso_mss = mbuf_iscsi_iso_mss(sndptr); - plen = imm_data; + plen = 0; nsegs = 0; max_nsegs_1mbuf = 0; /* max # of SGL segments in any one mbuf */ nomap_mbuf_seen = false; - for (m = sndptr->m_next; m != NULL; m = m->m_next) { + for (m = sndptr; m != NULL; m = m->m_next) { int n; if (m->m_flags & M_EXTPG) @@ -1440,13 +1355,13 @@ write_nvme_mbuf_wr(struct toepcb *toep, struct mbuf *sndptr) if (iso) wr_len += sizeof(struct cpl_tx_data_iso); if (plen <= max_imm && !nomap_mbuf_seen) { - /* Immediate data tx for full PDU */ + /* Immediate data tx */ imm_data = plen; wr_len += plen; nsegs = 0; } else { /* DSGL tx for PDU data */ - wr_len += roundup2(imm_data, 16); + imm_data = 0; wr_len += sizeof(struct ulptx_sgl) + ((3 * (nsegs - 1)) / 2 + ((nsegs - 1) & 1)) * 8; } @@ -1460,7 +1375,7 @@ write_nvme_mbuf_wr(struct toepcb *toep, struct mbuf *sndptr) credits = howmany(wr->wr_len, 16); if (iso) { - write_tx_v2_wr(txwr, toep, FW_V2_NVMET_TX_DATA_WR, + write_tx_wr(txwr, toep, FW_ISCSI_TX_DATA_WR, imm_data + sizeof(struct cpl_tx_data_iso), adjusted_plen, credits, shove, ulp_submode | ULP_ISO); cpl_iso = (struct cpl_tx_data_iso *)(txwr + 1); @@ -1470,16 +1385,15 @@ write_nvme_mbuf_wr(struct toepcb *toep, struct mbuf *sndptr) hdr->pdo); p = cpl_iso + 1; } else { - write_tx_v2_wr(txwr, toep, FW_V2_NVMET_TX_DATA_WR, imm_data, + write_tx_wr(txwr, toep, FW_OFLD_TX_DATA_WR, imm_data, adjusted_plen, credits, shove, ulp_submode); p = txwr + 1; } - /* PDU header (and immediate data payload). */ - m_copydata(sndptr, 0, imm_data, p); - if (nsegs != 0) { - p = roundup2((char *)p + imm_data, 16); - write_tx_sgl(p, sndptr->m_next, NULL, nsegs, max_nsegs_1mbuf); + if (imm_data != 0) { + m_copydata(sndptr, 0, plen, p); + } else { + write_tx_sgl(p, sndptr, NULL, nsegs, max_nsegs_1mbuf); if (wr_len & 0xf) { uint64_t *pad = (uint64_t *)((uintptr_t)txwr + wr_len); *pad = 0;