From nobody Fri Oct 3 00:04:14 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cd8663fFYz6BMjJ; Fri, 03 Oct 2025 00:04:14 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4cd86625xnz3s29; Fri, 03 Oct 2025 00:04:14 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1759449854; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=+3ISNAhwu30pT/NPD402MO4DVaAbY9YfAH5hxXxycj4=; b=ipGqiIa7IhbN9Bp+u8cwB9mNa5lJfQq1xST1na2Zfo+xbt5bvAvUdFAGGvzpZCn0SbxpCs bh+lBNtTNwO20L+GPfoSRgpSZ+dJ4XR6nvQ+AwLsO9qdDaltp/Bbhhs5yj0RTxO+MA2MBC 7r+x4Fy+n401+sq3CsuIwai4MJjMesgTYitpfhJrTdLH83Ved3mp7ZBuan2MzKdmd7uIM1 7HKqaktX7FD1c9psxeGAMPEKsackWRHOWgNRFFXXdvoVwOgAKYkG58+IIAEClfgnd8upZ6 axkXMcUKJzq6pzCcA9vrt6xqMQr9nsQZAJB6acJSsqWb0crrjJ/KxYB/DfwVNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1759449854; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=+3ISNAhwu30pT/NPD402MO4DVaAbY9YfAH5hxXxycj4=; b=eUEcwTxW1Pp+Ljn8IbPTSwVjSH0AUriFPU120vs9YlACVZ4pyz0kzYNoY3WGzBagKb07St ATndZbWxIxTeipedBJR+9uHfnc11vEPCGBB9tpqhmgWI7tQjX7Pih31cmeYdh41bZnuW2Q cbsMse2KqOUvxcAEzeep3X1jM/I4NCk0ZCAkmyznyiY44xHPETDYiZyQmUPjlsGRkwSFWt zCmwHiZ7vNO+Wf5L1zK+Zrud7vtEqEWyu3lArd5Lgi8PWnqtASAsxyRA9DCyBv8rZ97kPT CAyguG2WqSZs3siBS2FCrxlx3zppxVlUsGdTBXlQsnhZ0Rm3BCesLx4RANZPFg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1759449854; a=rsa-sha256; cv=none; b=fD6t1OmpiXxLC46jugqs+Zc2kB+yTh0UWLJVAuWoEO/WjDJJnV0KHWvqi9t4cRy8gMpVop 3tgdYmRsVEfGxFeiutzlqxlV2r1X7tFfBrspEWk1/RvL+gtfdOsYaK1dwNv52fX5vXIiNh 5DcsVWm9bjNdFr7hk2H5sN+27fUpiVtVQA9w7U1JbdUJLvutxYnc4Jw+8+URKei9OJaR5i KLFPjJz13xVWqVj7fMECebYoqcmZQUIMwKPKEN9I+iDjqI+Vu7S7dh54l6hbZMoUaDsvxT do8kztATNKcuo7GN5lW76BDCfCDTzCbTtFMyzTh1q6zKQHSp/msqo3Msp4Ff8g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4cd8661hSyzVcq; Fri, 03 Oct 2025 00:04:14 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 59304EtN056522; Fri, 3 Oct 2025 00:04:14 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 59304EK1056519; Fri, 3 Oct 2025 00:04:14 GMT (envelope-from git) Date: Fri, 3 Oct 2025 00:04:14 GMT Message-Id: <202510030004.59304EK1056519@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Navdeep Parhar Subject: git: 340c4db70266 - stable/15 - cxgbe tom: Support sending "raw" WR mbufs on plain TCP and TLS sockets List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: np X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: 340c4db7026661a88dba2b828f1da5d4d1eef3ec Auto-Submitted: auto-generated The branch stable/15 has been updated by np: URL: https://cgit.FreeBSD.org/src/commit/?id=340c4db7026661a88dba2b828f1da5d4d1eef3ec commit 340c4db7026661a88dba2b828f1da5d4d1eef3ec Author: John Baldwin AuthorDate: 2025-09-11 21:10:39 +0000 Commit: Navdeep Parhar CommitDate: 2025-10-02 22:09:03 +0000 cxgbe tom: Support sending "raw" WR mbufs on plain TCP and TLS sockets This is intended to support control work requests that manipulate connection state but do not transmit data. Raw WRs are transmitted immediately if possible, otherwise they are queued to an mbufq in the toe pcb until sufficient credits are available. Raw WRs take precendence over transmitting socket data. Reviewed by: np Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D47761 (cherry picked from commit e2a2a7581c9663a08b8d92a34040d6fcdfc20816) --- sys/dev/cxgbe/tom/t4_cpl_io.c | 85 +++++++++++++++++++++++++++++++++++++++++++ sys/dev/cxgbe/tom/t4_tls.c | 13 +++++++ sys/dev/cxgbe/tom/t4_tom.h | 2 + 3 files changed, 100 insertions(+) diff --git a/sys/dev/cxgbe/tom/t4_cpl_io.c b/sys/dev/cxgbe/tom/t4_cpl_io.c index 9ecb4aeee939..a7b4162db3b5 100644 --- a/sys/dev/cxgbe/tom/t4_cpl_io.c +++ b/sys/dev/cxgbe/tom/t4_cpl_io.c @@ -619,6 +619,48 @@ write_tx_sgl(void *dst, struct mbuf *start, struct mbuf *stop, int nsegs, int n) __func__, nsegs, start, stop)); } +bool +t4_push_raw_wr(struct adapter *sc, struct toepcb *toep, struct mbuf *m) +{ +#ifdef INVARIANTS + struct inpcb *inp = toep->inp; +#endif + struct wrqe *wr; + struct ofld_tx_sdesc *txsd; + u_int credits, plen; + + INP_WLOCK_ASSERT(inp); + MPASS(mbuf_raw_wr(m)); + plen = m->m_pkthdr.len; + credits = howmany(plen, 16); + if (credits > toep->tx_credits) + return (false); + + wr = alloc_wrqe(roundup2(plen, 16), &toep->ofld_txq->wrq); + if (wr == NULL) + return (false); + + m_copydata(m, 0, plen, wrtod(wr)); + m_freem(m); + + toep->tx_credits -= credits; + if (toep->tx_credits < MIN_OFLD_TX_CREDITS) + toep->flags |= TPF_TX_SUSPENDED; + + KASSERT(toep->txsd_avail > 0, ("%s: no txsd", __func__)); + KASSERT(credits <= MAX_OFLD_TX_SDESC_CREDITS, + ("%s: tx_credits %u too large", __func__, credits)); + txsd = &toep->txsd[toep->txsd_pidx]; + txsd->plen = 0; + txsd->tx_credits = credits; + if (__predict_false(++toep->txsd_pidx == toep->txsd_total)) + toep->txsd_pidx = 0; + toep->txsd_avail--; + + t4_wrq_tx(sc, wr); + return (true); +} + /* * Max number of SGL entries an offload tx work request can have. This is 41 * (1 + 40) for a full 512B work request. @@ -651,6 +693,7 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop) struct tcpcb *tp = intotcpcb(inp); struct socket *so = inp->inp_socket; struct sockbuf *sb = &so->so_snd; + struct mbufq *pduq = &toep->ulp_pduq; int tx_credits, shove, compl, sowwakeup; struct ofld_tx_sdesc *txsd; bool nomap_mbuf_seen; @@ -695,6 +738,19 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop) max_imm = max_imm_payload(tx_credits, 0); max_nsegs = max_dsgl_nsegs(tx_credits, 0); + if (__predict_false((sndptr = mbufq_first(pduq)) != NULL)) { + if (!t4_push_raw_wr(sc, toep, sndptr)) { + toep->flags |= TPF_TX_SUSPENDED; + return; + } + + m = mbufq_dequeue(pduq); + MPASS(m == sndptr); + + txsd = &toep->txsd[toep->txsd_pidx]; + continue; + } + SOCKBUF_LOCK(sb); sowwakeup = drop; if (drop) { @@ -1253,6 +1309,35 @@ t4_push_data(struct adapter *sc, struct toepcb *toep, int drop) t4_push_frames(sc, toep, drop); } +void +t4_raw_wr_tx(struct adapter *sc, struct toepcb *toep, struct mbuf *m) +{ +#ifdef INVARIANTS + struct inpcb *inp = toep->inp; +#endif + + INP_WLOCK_ASSERT(inp); + + /* + * If there are other raw WRs enqueued, enqueue to preserve + * FIFO ordering. + */ + if (!mbufq_empty(&toep->ulp_pduq)) { + mbufq_enqueue(&toep->ulp_pduq, m); + return; + } + + /* + * Cannot call t4_push_data here as that will lock so_snd and + * some callers of this run in rx handlers with so_rcv locked. + * Instead, just try to transmit this WR. + */ + if (!t4_push_raw_wr(sc, toep, m)) { + mbufq_enqueue(&toep->ulp_pduq, m); + toep->flags |= TPF_TX_SUSPENDED; + } +} + int t4_tod_output(struct toedev *tod, struct tcpcb *tp) { diff --git a/sys/dev/cxgbe/tom/t4_tls.c b/sys/dev/cxgbe/tom/t4_tls.c index 857832aafa5c..3dd3c28e3a86 100644 --- a/sys/dev/cxgbe/tom/t4_tls.c +++ b/sys/dev/cxgbe/tom/t4_tls.c @@ -496,6 +496,7 @@ t4_push_ktls(struct adapter *sc, struct toepcb *toep, int drop) struct tcpcb *tp = intotcpcb(inp); struct socket *so = inp->inp_socket; struct sockbuf *sb = &so->so_snd; + struct mbufq *pduq = &toep->ulp_pduq; int tls_size, tx_credits, shove, sowwakeup; struct ofld_tx_sdesc *txsd; char *buf; @@ -538,6 +539,18 @@ t4_push_ktls(struct adapter *sc, struct toepcb *toep, int drop) for (;;) { tx_credits = min(toep->tx_credits, MAX_OFLD_TX_CREDITS); + if (__predict_false((m = mbufq_first(pduq)) != NULL)) { + if (!t4_push_raw_wr(sc, toep, m)) { + toep->flags |= TPF_TX_SUSPENDED; + return; + } + + (void)mbufq_dequeue(pduq); + + txsd = &toep->txsd[toep->txsd_pidx]; + continue; + } + SOCKBUF_LOCK(sb); sowwakeup = drop; if (drop) { diff --git a/sys/dev/cxgbe/tom/t4_tom.h b/sys/dev/cxgbe/tom/t4_tom.h index 3b2243aeb69f..3dfa24a33f85 100644 --- a/sys/dev/cxgbe/tom/t4_tom.h +++ b/sys/dev/cxgbe/tom/t4_tom.h @@ -529,6 +529,8 @@ int t4_send_rst(struct toedev *, struct tcpcb *); void t4_set_tcb_field(struct adapter *, struct sge_wrq *, struct toepcb *, uint16_t, uint64_t, uint64_t, int, int); void t4_push_pdus(struct adapter *, struct toepcb *, int); +bool t4_push_raw_wr(struct adapter *, struct toepcb *, struct mbuf *); +void t4_raw_wr_tx(struct adapter *, struct toepcb *, struct mbuf *); /* t4_ddp.c */ int t4_init_ppod_region(struct ppod_region *, struct t4_range *, u_int,