From nobody Thu Jan 29 15:40:01 2026 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4f23HP43cPz6Qrdj for ; Thu, 29 Jan 2026 15:40:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R13" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4f23HP0zfKz3y0r for ; Thu, 29 Jan 2026 15:40:01 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1769701201; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=sFHy8hDg6ZhPkDL0QTMyy+s0YsOHEXewV3qZHTVe0d0=; b=JI+PGeffdOxs5vkoqo/+iFxgwkKrG754BIko16VGAzWOj/7LzZtzJhH0FIiBfdzghKsPxj oCzx2FeRYf4Tazu3uMPBk1qegoB2f15O3uj/WKSMeUBzsoxiEOM/mD35SRa573qrVbb5PQ 64a2xFtRrlWrSzTNmAe4lCGoJry1XvftED/tOE4pZzn54FwKfBP+/6e0yCFlW0LpnomPJx qUCl5HFxQr+4+Si/0miBLqVPauu17i0ggy6tUfO/VYolyJjLdAkxHXcBiQ6LlAC8TLtAZA tYHXAAXhMMWTwF9ERuFRSyj9chToM+s158fSPyV74jRKSdt6u3yv4/VcVVYs2g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1769701201; a=rsa-sha256; cv=none; b=jn3eYC0FrE6ssv5HIYDFwEdiGj5aAQjR1g36lJziwhbV47Pzp//j4rDRbPONRvqw2/WylA heLw7rpwWs5//jmDB9JdLNmISVyJJJHIi4ZnBVwiRCyxEPbsQqeZnnMAr1AmM1cGk5T9Xa q70XoSjf64yEr7FjxKkBwEHbB4We7rVxi/xsQ04HGrl6ekDtlvHwKHc7rMTdInMMgKIVV6 UTmOp23pUIeh7vG1onQMMZ9Ve714TnfbGtxcIWsrAAb2p3vRsAdA1rHBepQwKZyKNeQCkz v3OH5jf02U4N4cd+NJbYPQ4i+y4uqmAGfcwa/eTDc2YrWyERjJ8AHPOjPHbMWw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1769701201; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=sFHy8hDg6ZhPkDL0QTMyy+s0YsOHEXewV3qZHTVe0d0=; b=HjriCmcmcL8Scd6fiw2pecMKKDco9PKTQLDc1MhjAVFUG2yKNchip8xXbp4DhIQIPJuUZf hVFs2PuKkU6YCEigjEnJ7OlgxsabpW0RXFkD40fLkiopS8iorGlz9nx93g39wiFUmm4ZZH GgWhMrymbdv5Y5NOC8CerO/0C/dQLgYi6MhCyqt6r6QASKWHlm0TQy7RDERAl6EXeTRS7b mI9ywWjKhfW+rYN1cBMpFJdjPwXBILPOOtlY5pb7xWUrOcbN/iihR7hqxqjGggXd1a20W+ ZWYxMGCgdB9OBphDIEimw5lS8RjEVIwEfOw0yP1gQ0/LYd3IE4NTsczeGCyBvg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4f23HP0L61zkqP for ; Thu, 29 Jan 2026 15:40:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 3a421 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 29 Jan 2026 15:40:01 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: John Baldwin Subject: git: bde291c654c2 - stable/14 - cxgbe tom: Support sending "raw" WR mbufs on plain TCP and TLS sockets List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-branches@freebsd.org Sender: owner-dev-commits-src-branches@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: bde291c654c21d58534cef5b49965901971708b8 Auto-Submitted: auto-generated Date: Thu, 29 Jan 2026 15:40:01 +0000 Message-Id: <697b7f51.3a421.7b485c1b@gitrepo.freebsd.org> The branch stable/14 has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=bde291c654c21d58534cef5b49965901971708b8 commit bde291c654c21d58534cef5b49965901971708b8 Author: John Baldwin AuthorDate: 2025-09-11 21:10:39 +0000 Commit: John Baldwin CommitDate: 2026-01-29 15:27:34 +0000 cxgbe tom: Support sending "raw" WR mbufs on plain TCP and TLS sockets This is intended to support control work requests that manipulate connection state but do not transmit data. Raw WRs are transmitted immediately if possible, otherwise they are queued to an mbufq in the toe pcb until sufficient credits are available. Raw WRs take precendence over transmitting socket data. Reviewed by: np Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D47761 (cherry picked from commit e2a2a7581c9663a08b8d92a34040d6fcdfc20816) --- sys/dev/cxgbe/tom/t4_cpl_io.c | 85 +++++++++++++++++++++++++++++++++++++++++++ sys/dev/cxgbe/tom/t4_tls.c | 13 +++++++ sys/dev/cxgbe/tom/t4_tom.h | 2 + 3 files changed, 100 insertions(+) diff --git a/sys/dev/cxgbe/tom/t4_cpl_io.c b/sys/dev/cxgbe/tom/t4_cpl_io.c index 9aaa243ba9e1..6c706bd6a454 100644 --- a/sys/dev/cxgbe/tom/t4_cpl_io.c +++ b/sys/dev/cxgbe/tom/t4_cpl_io.c @@ -619,6 +619,48 @@ write_tx_sgl(void *dst, struct mbuf *start, struct mbuf *stop, int nsegs, int n) __func__, nsegs, start, stop)); } +bool +t4_push_raw_wr(struct adapter *sc, struct toepcb *toep, struct mbuf *m) +{ +#ifdef INVARIANTS + struct inpcb *inp = toep->inp; +#endif + struct wrqe *wr; + struct ofld_tx_sdesc *txsd; + u_int credits, plen; + + INP_WLOCK_ASSERT(inp); + MPASS(mbuf_raw_wr(m)); + plen = m->m_pkthdr.len; + credits = howmany(plen, 16); + if (credits > toep->tx_credits) + return (false); + + wr = alloc_wrqe(roundup2(plen, 16), &toep->ofld_txq->wrq); + if (wr == NULL) + return (false); + + m_copydata(m, 0, plen, wrtod(wr)); + m_freem(m); + + toep->tx_credits -= credits; + if (toep->tx_credits < MIN_OFLD_TX_CREDITS) + toep->flags |= TPF_TX_SUSPENDED; + + KASSERT(toep->txsd_avail > 0, ("%s: no txsd", __func__)); + KASSERT(credits <= MAX_OFLD_TX_SDESC_CREDITS, + ("%s: tx_credits %u too large", __func__, credits)); + txsd = &toep->txsd[toep->txsd_pidx]; + txsd->plen = 0; + txsd->tx_credits = credits; + if (__predict_false(++toep->txsd_pidx == toep->txsd_total)) + toep->txsd_pidx = 0; + toep->txsd_avail--; + + t4_wrq_tx(sc, wr); + return (true); +} + /* * Max number of SGL entries an offload tx work request can have. This is 41 * (1 + 40) for a full 512B work request. @@ -651,6 +693,7 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop) struct tcpcb *tp = intotcpcb(inp); struct socket *so = inp->inp_socket; struct sockbuf *sb = &so->so_snd; + struct mbufq *pduq = &toep->ulp_pduq; int tx_credits, shove, compl, sowwakeup; struct ofld_tx_sdesc *txsd; bool nomap_mbuf_seen; @@ -695,6 +738,19 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop) max_imm = max_imm_payload(tx_credits, 0); max_nsegs = max_dsgl_nsegs(tx_credits, 0); + if (__predict_false((sndptr = mbufq_first(pduq)) != NULL)) { + if (!t4_push_raw_wr(sc, toep, sndptr)) { + toep->flags |= TPF_TX_SUSPENDED; + return; + } + + m = mbufq_dequeue(pduq); + MPASS(m == sndptr); + + txsd = &toep->txsd[toep->txsd_pidx]; + continue; + } + SOCKBUF_LOCK(sb); sowwakeup = drop; if (drop) { @@ -1253,6 +1309,35 @@ t4_push_data(struct adapter *sc, struct toepcb *toep, int drop) t4_push_frames(sc, toep, drop); } +void +t4_raw_wr_tx(struct adapter *sc, struct toepcb *toep, struct mbuf *m) +{ +#ifdef INVARIANTS + struct inpcb *inp = toep->inp; +#endif + + INP_WLOCK_ASSERT(inp); + + /* + * If there are other raw WRs enqueued, enqueue to preserve + * FIFO ordering. + */ + if (!mbufq_empty(&toep->ulp_pduq)) { + mbufq_enqueue(&toep->ulp_pduq, m); + return; + } + + /* + * Cannot call t4_push_data here as that will lock so_snd and + * some callers of this run in rx handlers with so_rcv locked. + * Instead, just try to transmit this WR. + */ + if (!t4_push_raw_wr(sc, toep, m)) { + mbufq_enqueue(&toep->ulp_pduq, m); + toep->flags |= TPF_TX_SUSPENDED; + } +} + int t4_tod_output(struct toedev *tod, struct tcpcb *tp) { diff --git a/sys/dev/cxgbe/tom/t4_tls.c b/sys/dev/cxgbe/tom/t4_tls.c index e3937737d7e2..d2a72c9eee2e 100644 --- a/sys/dev/cxgbe/tom/t4_tls.c +++ b/sys/dev/cxgbe/tom/t4_tls.c @@ -496,6 +496,7 @@ t4_push_ktls(struct adapter *sc, struct toepcb *toep, int drop) struct tcpcb *tp = intotcpcb(inp); struct socket *so = inp->inp_socket; struct sockbuf *sb = &so->so_snd; + struct mbufq *pduq = &toep->ulp_pduq; int tls_size, tx_credits, shove, sowwakeup; struct ofld_tx_sdesc *txsd; char *buf; @@ -538,6 +539,18 @@ t4_push_ktls(struct adapter *sc, struct toepcb *toep, int drop) for (;;) { tx_credits = min(toep->tx_credits, MAX_OFLD_TX_CREDITS); + if (__predict_false((m = mbufq_first(pduq)) != NULL)) { + if (!t4_push_raw_wr(sc, toep, m)) { + toep->flags |= TPF_TX_SUSPENDED; + return; + } + + (void)mbufq_dequeue(pduq); + + txsd = &toep->txsd[toep->txsd_pidx]; + continue; + } + SOCKBUF_LOCK(sb); sowwakeup = drop; if (drop) { diff --git a/sys/dev/cxgbe/tom/t4_tom.h b/sys/dev/cxgbe/tom/t4_tom.h index 3b2243aeb69f..3dfa24a33f85 100644 --- a/sys/dev/cxgbe/tom/t4_tom.h +++ b/sys/dev/cxgbe/tom/t4_tom.h @@ -529,6 +529,8 @@ int t4_send_rst(struct toedev *, struct tcpcb *); void t4_set_tcb_field(struct adapter *, struct sge_wrq *, struct toepcb *, uint16_t, uint64_t, uint64_t, int, int); void t4_push_pdus(struct adapter *, struct toepcb *, int); +bool t4_push_raw_wr(struct adapter *, struct toepcb *, struct mbuf *); +void t4_raw_wr_tx(struct adapter *, struct toepcb *, struct mbuf *); /* t4_ddp.c */ int t4_init_ppod_region(struct ppod_region *, struct t4_range *, u_int,