From nobody Thu Sep 11 21:10:51 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cN9Fl5Zxpz67R63; Thu, 11 Sep 2025 21:10:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4cN9Fl4w8dz40Jr; Thu, 11 Sep 2025 21:10:51 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1757625051; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=s39rgA4juf1rrELqZ9LUCxjGtX5mlyY0CyjZfJWh/8A=; b=tF8DqZw9pcFzN8UMeCnkftZQEXnMWHCwyg3C5t35BiBl19+Xgz208571qxchmbHidBzKdl 4YKOOicYMTCPdT35gQudZWsSnRFXeftWrhEPWFOpZ8xva516TMcvhz9N5v2DCbBzqrsamC AD0yl5vc518LfjviLAqWBp8PfUu5dxMAklrE8sf7F1Q0rsHcTH+hEXuqfT8CU3jzJp6xCQ T/ydaZZDiOhC8Go8GYiOdH5aL5elb2ciowxK9JgINeT5Zp5jy4LTrdbXkOjxo19rwgI+JH firNwARaQSgABlvu9/O1j1BCcBaJF0uY+SWSrWPduELIXzrA2Xfb8s9fQoC83Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1757625051; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=s39rgA4juf1rrELqZ9LUCxjGtX5mlyY0CyjZfJWh/8A=; b=XM6YLjmA25p2HlhOyj3ZXDgSp/+MDvwMmyEK+EpBczZIVMAPzxTTf1AT2gJ6VAQa0GT1Jp edVxDdZMxLBJsWDcL666EkxJb2Tp16aFQ8YdUQwIa5HFFwFn/mriZEZojdfx9TKimx7lBY EQ/47n84H/WN9eVZdc4912/5cAQnxo9JunV6g10UJGRynwB8PlEhLHv8pPkTaZGoYvv5ib 84Z2vYz/idWxjaB9b1oejT1tHutLtmqSw+EAyjuq5y48UH/y2qbYg9ptVGcP+SVOlWbn67 K8c9vhhInQBdz+0qKZ9qL9GE9Qs7oZDk4UAJhOzAUhm3hWHJnPAf3JyuqymMkA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1757625051; a=rsa-sha256; cv=none; b=xBkinUty5860dOzMJhBCCp+0YQ4eyCZJWc+d5UYVpxICPvBfWdQJ0pT25AnkvbvNPJfTDz b7JnR23m8pqULDaL7HKX4jMiB34qZnFQuIe5ztyym4Oo6vPK1wajmltx+Z9W/1pu9+bZlQ JsVaN1L2dJ9gK9U6gx7G3j+iQJgM9/Phau9KyPKsC2VI9k1+H9B2N2KzuosFhKHw+cPKI5 cNFpQk+etgwIZidUuJlrBJi/+M9p4hnzHBSx2UEETpNHk0qjCXmEe2QO45RqlrodLRG4TT kf7OOM0xiSXV3CqmYzlqR/o5aEK0rU4P0jMBQqEwFj3iBsO3Yl2w2dwCjniAEQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4cN9Fl4PcQz7nf; Thu, 11 Sep 2025 21:10:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 58BLApPc058480; Thu, 11 Sep 2025 21:10:51 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 58BLApNG058460; Thu, 11 Sep 2025 21:10:51 GMT (envelope-from git) Date: Thu, 11 Sep 2025 21:10:51 GMT Message-Id: <202509112110.58BLApNG058460@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: e2a2a7581c96 - main - cxgbe tom: Support sending "raw" WR mbufs on plain TCP and TLS sockets List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: e2a2a7581c9663a08b8d92a34040d6fcdfc20816 Auto-Submitted: auto-generated The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=e2a2a7581c9663a08b8d92a34040d6fcdfc20816 commit e2a2a7581c9663a08b8d92a34040d6fcdfc20816 Author: John Baldwin AuthorDate: 2025-09-11 21:10:39 +0000 Commit: John Baldwin CommitDate: 2025-09-11 21:10:39 +0000 cxgbe tom: Support sending "raw" WR mbufs on plain TCP and TLS sockets This is intended to support control work requests that manipulate connection state but do not transmit data. Raw WRs are transmitted immediately if possible, otherwise they are queued to an mbufq in the toe pcb until sufficient credits are available. Raw WRs take precendence over transmitting socket data. Reviewed by: np Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D47761 --- sys/dev/cxgbe/tom/t4_cpl_io.c | 85 +++++++++++++++++++++++++++++++++++++++++++ sys/dev/cxgbe/tom/t4_tls.c | 13 +++++++ sys/dev/cxgbe/tom/t4_tom.h | 2 + 3 files changed, 100 insertions(+) diff --git a/sys/dev/cxgbe/tom/t4_cpl_io.c b/sys/dev/cxgbe/tom/t4_cpl_io.c index 9ecb4aeee939..a7b4162db3b5 100644 --- a/sys/dev/cxgbe/tom/t4_cpl_io.c +++ b/sys/dev/cxgbe/tom/t4_cpl_io.c @@ -619,6 +619,48 @@ write_tx_sgl(void *dst, struct mbuf *start, struct mbuf *stop, int nsegs, int n) __func__, nsegs, start, stop)); } +bool +t4_push_raw_wr(struct adapter *sc, struct toepcb *toep, struct mbuf *m) +{ +#ifdef INVARIANTS + struct inpcb *inp = toep->inp; +#endif + struct wrqe *wr; + struct ofld_tx_sdesc *txsd; + u_int credits, plen; + + INP_WLOCK_ASSERT(inp); + MPASS(mbuf_raw_wr(m)); + plen = m->m_pkthdr.len; + credits = howmany(plen, 16); + if (credits > toep->tx_credits) + return (false); + + wr = alloc_wrqe(roundup2(plen, 16), &toep->ofld_txq->wrq); + if (wr == NULL) + return (false); + + m_copydata(m, 0, plen, wrtod(wr)); + m_freem(m); + + toep->tx_credits -= credits; + if (toep->tx_credits < MIN_OFLD_TX_CREDITS) + toep->flags |= TPF_TX_SUSPENDED; + + KASSERT(toep->txsd_avail > 0, ("%s: no txsd", __func__)); + KASSERT(credits <= MAX_OFLD_TX_SDESC_CREDITS, + ("%s: tx_credits %u too large", __func__, credits)); + txsd = &toep->txsd[toep->txsd_pidx]; + txsd->plen = 0; + txsd->tx_credits = credits; + if (__predict_false(++toep->txsd_pidx == toep->txsd_total)) + toep->txsd_pidx = 0; + toep->txsd_avail--; + + t4_wrq_tx(sc, wr); + return (true); +} + /* * Max number of SGL entries an offload tx work request can have. This is 41 * (1 + 40) for a full 512B work request. @@ -651,6 +693,7 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop) struct tcpcb *tp = intotcpcb(inp); struct socket *so = inp->inp_socket; struct sockbuf *sb = &so->so_snd; + struct mbufq *pduq = &toep->ulp_pduq; int tx_credits, shove, compl, sowwakeup; struct ofld_tx_sdesc *txsd; bool nomap_mbuf_seen; @@ -695,6 +738,19 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop) max_imm = max_imm_payload(tx_credits, 0); max_nsegs = max_dsgl_nsegs(tx_credits, 0); + if (__predict_false((sndptr = mbufq_first(pduq)) != NULL)) { + if (!t4_push_raw_wr(sc, toep, sndptr)) { + toep->flags |= TPF_TX_SUSPENDED; + return; + } + + m = mbufq_dequeue(pduq); + MPASS(m == sndptr); + + txsd = &toep->txsd[toep->txsd_pidx]; + continue; + } + SOCKBUF_LOCK(sb); sowwakeup = drop; if (drop) { @@ -1253,6 +1309,35 @@ t4_push_data(struct adapter *sc, struct toepcb *toep, int drop) t4_push_frames(sc, toep, drop); } +void +t4_raw_wr_tx(struct adapter *sc, struct toepcb *toep, struct mbuf *m) +{ +#ifdef INVARIANTS + struct inpcb *inp = toep->inp; +#endif + + INP_WLOCK_ASSERT(inp); + + /* + * If there are other raw WRs enqueued, enqueue to preserve + * FIFO ordering. + */ + if (!mbufq_empty(&toep->ulp_pduq)) { + mbufq_enqueue(&toep->ulp_pduq, m); + return; + } + + /* + * Cannot call t4_push_data here as that will lock so_snd and + * some callers of this run in rx handlers with so_rcv locked. + * Instead, just try to transmit this WR. + */ + if (!t4_push_raw_wr(sc, toep, m)) { + mbufq_enqueue(&toep->ulp_pduq, m); + toep->flags |= TPF_TX_SUSPENDED; + } +} + int t4_tod_output(struct toedev *tod, struct tcpcb *tp) { diff --git a/sys/dev/cxgbe/tom/t4_tls.c b/sys/dev/cxgbe/tom/t4_tls.c index 857832aafa5c..3dd3c28e3a86 100644 --- a/sys/dev/cxgbe/tom/t4_tls.c +++ b/sys/dev/cxgbe/tom/t4_tls.c @@ -496,6 +496,7 @@ t4_push_ktls(struct adapter *sc, struct toepcb *toep, int drop) struct tcpcb *tp = intotcpcb(inp); struct socket *so = inp->inp_socket; struct sockbuf *sb = &so->so_snd; + struct mbufq *pduq = &toep->ulp_pduq; int tls_size, tx_credits, shove, sowwakeup; struct ofld_tx_sdesc *txsd; char *buf; @@ -538,6 +539,18 @@ t4_push_ktls(struct adapter *sc, struct toepcb *toep, int drop) for (;;) { tx_credits = min(toep->tx_credits, MAX_OFLD_TX_CREDITS); + if (__predict_false((m = mbufq_first(pduq)) != NULL)) { + if (!t4_push_raw_wr(sc, toep, m)) { + toep->flags |= TPF_TX_SUSPENDED; + return; + } + + (void)mbufq_dequeue(pduq); + + txsd = &toep->txsd[toep->txsd_pidx]; + continue; + } + SOCKBUF_LOCK(sb); sowwakeup = drop; if (drop) { diff --git a/sys/dev/cxgbe/tom/t4_tom.h b/sys/dev/cxgbe/tom/t4_tom.h index 3b2243aeb69f..3dfa24a33f85 100644 --- a/sys/dev/cxgbe/tom/t4_tom.h +++ b/sys/dev/cxgbe/tom/t4_tom.h @@ -529,6 +529,8 @@ int t4_send_rst(struct toedev *, struct tcpcb *); void t4_set_tcb_field(struct adapter *, struct sge_wrq *, struct toepcb *, uint16_t, uint64_t, uint64_t, int, int); void t4_push_pdus(struct adapter *, struct toepcb *, int); +bool t4_push_raw_wr(struct adapter *, struct toepcb *, struct mbuf *); +void t4_raw_wr_tx(struct adapter *, struct toepcb *, struct mbuf *); /* t4_ddp.c */ int t4_init_ppod_region(struct ppod_region *, struct t4_range *, u_int,