From nobody Fri Oct 3 00:04:16 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cd8685HjWz6BMbB; Fri, 03 Oct 2025 00:04:16 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4cd86843DTz3rff; Fri, 03 Oct 2025 00:04:16 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1759449856; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=scttQgo1F5MRCDjBhqYucmtYItcgF8p/ZDwwz0OmakA=; b=kKiratnAzZciO2cR6pdX8M32nWk9VvWfDao9XDKkRrLrllH/eIfqw4X4miDkw0F1tecC84 fiFlxp2wgzUhwFHNqh2geCvaTJAZzBRXUfZbP1RFqrhEwqDHu94uzwkP39vjj2NlZyVjGE I6e8r/vK2y3cVBA7CVH8khMfLrxhHwHj4R2vGkWZGFWw/8LL/zsYbBME2YQOrDaGhDXOl/ o8egSYTl40ZBkm4mRNwNXcrqS6Jp5NCThLFx5DALg++xmAVNxaHGjlwsV2LLtEymuwyLcD 6wQbSrCGxaBTFqjbOufwz/tNI3eyZA7Vi7Xf3G6wvTpAiQHEZn2nEzA5r8dTwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1759449856; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=scttQgo1F5MRCDjBhqYucmtYItcgF8p/ZDwwz0OmakA=; b=QQyCZyignVJur6s7kIIYzygnPORsP9wfwwx8inU8IF6Ms2qPnkA1stFOpOaTSgV3NVC1q7 zLU2SJcipOz/XXoOZoi7cy8nMRlZ9y+MyWlD9hu4OpS+oujDphen+HsRtWrhMizyy7spnb GTk1vWIu5DYciXY23hE0MA+cxmtIWWww2CStey9kUIcFoIZbBFPxsTcaFe2hXCUqURyhtF rK/RgCE4zZPDWLszit2OJcVtj/h2Eot6mTzufol66hIolfCzJl7aoILcRAJ9ijuqiGKmmG nMjTPoBBDU38lXhcjWFASU5DP0fsrmaUz2qjtMVEds4mGgsVPNRhDGybXYyEXg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1759449856; a=rsa-sha256; cv=none; b=FzXQ5t5hL6Yyvuk0auoI1lerkf8/++xKSJ/NikYGRoJTkVcy0wP9EWYJCHvMLZzOvHewpt Jv6SZmu4NB3iSgfLx0QnpoDcmanufmjG+U38qfoM23Jp0WuwYRc2EeeVJCuX4RfJ2jTrgS hdgMX989ImFqiF/ZHc5ZIoG9mtz/eSGwCOn2cuCEN9JzeeQAs4J+EmBDP6MfpWD2LdUuAz c97asu8bStFJ8paWk8Z4an4gPXIt3kQCthz5hj0zje0spbIQ5/6SvsYEpAEKWujn0QdLNx Ut3dCxYQBJY0oU/Zz2yqNBjm8Xz7J30Of8ntaOcOdvSFnU37KLEa+JP4SkoGbw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4cd8683cShzVjM; Fri, 03 Oct 2025 00:04:16 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 59304GS6056593; Fri, 3 Oct 2025 00:04:16 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 59304GA5056590; Fri, 3 Oct 2025 00:04:16 GMT (envelope-from git) Date: Fri, 3 Oct 2025 00:04:16 GMT Message-Id: <202510030004.59304GA5056590@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Navdeep Parhar Subject: git: f69054d53654 - stable/15 - cxgbe tom: Send auxiliary TLS work requests as raw WR mbufs List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: np X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: f69054d5365447a9e71c75a9a6f5913ad63beade Auto-Submitted: auto-generated The branch stable/15 has been updated by np: URL: https://cgit.FreeBSD.org/src/commit/?id=f69054d5365447a9e71c75a9a6f5913ad63beade commit f69054d5365447a9e71c75a9a6f5913ad63beade Author: John Baldwin AuthorDate: 2025-09-11 21:10:40 +0000 Commit: Navdeep Parhar CommitDate: 2025-10-02 22:09:03 +0000 cxgbe tom: Send auxiliary TLS work requests as raw WR mbufs In particular, the work request to update the TCB state when switching from plain TCP to TLS is now queued on the connection's offload transmit queue rather than over the per-port control queue. This also handles an unlikely edge case that a connection does not have sufficient credits to transmit other work requests synchronously such as programming the key in on-card memory or altering TCB fields. Reviewed by: np Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D47763 (cherry picked from commit d869395ac4bd248da7c5bdc67afb19ca89fbeeee) --- sys/dev/cxgbe/tom/t4_cpl_io.c | 36 +++++++++++++++++----------- sys/dev/cxgbe/tom/t4_tls.c | 56 +++++++++++++++++++++---------------------- sys/dev/cxgbe/tom/t4_tom.h | 2 ++ 3 files changed, 51 insertions(+), 43 deletions(-) diff --git a/sys/dev/cxgbe/tom/t4_cpl_io.c b/sys/dev/cxgbe/tom/t4_cpl_io.c index a7b4162db3b5..be20ea42474e 100644 --- a/sys/dev/cxgbe/tom/t4_cpl_io.c +++ b/sys/dev/cxgbe/tom/t4_cpl_io.c @@ -2039,25 +2039,16 @@ do_fw4_ack(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) } void -t4_set_tcb_field(struct adapter *sc, struct sge_wrq *wrq, struct toepcb *toep, +write_set_tcb_field(struct adapter *sc, void *dst, struct toepcb *toep, uint16_t word, uint64_t mask, uint64_t val, int reply, int cookie) { - struct wrqe *wr; - struct cpl_set_tcb_field *req; - struct ofld_tx_sdesc *txsd; + struct cpl_set_tcb_field *req = dst; MPASS((cookie & ~M_COOKIE) == 0); if (reply) { MPASS(cookie != CPL_COOKIE_RESERVED); } - wr = alloc_wrqe(sizeof(*req), wrq); - if (wr == NULL) { - /* XXX */ - panic("%s: allocation failure.", __func__); - } - req = wrtod(wr); - INIT_TP_WR_MIT_CPL(req, CPL_SET_TCB_FIELD, toep->tid); req->reply_ctrl = htobe16(V_QUEUENO(toep->ofld_rxq->iq.abs_id)); if (reply == 0) @@ -2065,12 +2056,29 @@ t4_set_tcb_field(struct adapter *sc, struct sge_wrq *wrq, struct toepcb *toep, req->word_cookie = htobe16(V_WORD(word) | V_COOKIE(cookie)); req->mask = htobe64(mask); req->val = htobe64(val); +} + +void +t4_set_tcb_field(struct adapter *sc, struct sge_wrq *wrq, struct toepcb *toep, + uint16_t word, uint64_t mask, uint64_t val, int reply, int cookie) +{ + struct wrqe *wr; + struct ofld_tx_sdesc *txsd; + const u_int len = sizeof(struct cpl_set_tcb_field); + + wr = alloc_wrqe(len, wrq); + if (wr == NULL) { + /* XXX */ + panic("%s: allocation failure.", __func__); + } + write_set_tcb_field(sc, wrtod(wr), toep, word, mask, val, reply, + cookie); + if (wrq->eq.type == EQ_OFLD) { txsd = &toep->txsd[toep->txsd_pidx]; - _Static_assert(howmany(sizeof(*req), 16) <= - MAX_OFLD_TX_SDESC_CREDITS, + _Static_assert(howmany(len, 16) <= MAX_OFLD_TX_SDESC_CREDITS, "MAX_OFLD_TX_SDESC_CREDITS too small"); - txsd->tx_credits = howmany(sizeof(*req), 16); + txsd->tx_credits = howmany(len, 16); txsd->plen = 0; KASSERT(toep->tx_credits >= txsd->tx_credits && toep->txsd_avail > 0, diff --git a/sys/dev/cxgbe/tom/t4_tls.c b/sys/dev/cxgbe/tom/t4_tls.c index 3dd3c28e3a86..ad72c6a6b025 100644 --- a/sys/dev/cxgbe/tom/t4_tls.c +++ b/sys/dev/cxgbe/tom/t4_tls.c @@ -61,11 +61,21 @@ static void t4_set_tls_tcb_field(struct toepcb *toep, uint16_t word, uint64_t mask, - uint64_t val) + uint64_t val, int reply, int cookie) { struct adapter *sc = td_adapter(toep->td); + struct mbuf *m; - t4_set_tcb_field(sc, &toep->ofld_txq->wrq, toep, word, mask, val, 0, 0); + m = alloc_raw_wr_mbuf(sizeof(struct cpl_set_tcb_field)); + if (m == NULL) { + /* XXX */ + panic("%s: out of memory", __func__); + } + + write_set_tcb_field(sc, mtod(m, void *), toep, word, mask, val, reply, + cookie); + + t4_raw_wr_tx(sc, toep, m); } /* TLS and DTLS common routines */ @@ -88,10 +98,9 @@ tls_tx_key(struct toepcb *toep) static void t4_set_rx_quiesce(struct toepcb *toep) { - struct adapter *sc = td_adapter(toep->td); - t4_set_tcb_field(sc, &toep->ofld_txq->wrq, toep, W_TCB_T_FLAGS, - V_TF_RX_QUIESCE(1), V_TF_RX_QUIESCE(1), 1, CPL_COOKIE_TOM); + t4_set_tls_tcb_field(toep, W_TCB_T_FLAGS, V_TF_RX_QUIESCE(1), + V_TF_RX_QUIESCE(1), 1, CPL_COOKIE_TOM); } /* Clear TF_RX_QUIESCE to re-enable receive. */ @@ -99,7 +108,7 @@ static void t4_clear_rx_quiesce(struct toepcb *toep) { - t4_set_tls_tcb_field(toep, W_TCB_T_FLAGS, V_TF_RX_QUIESCE(1), 0); + t4_set_tls_tcb_field(toep, W_TCB_T_FLAGS, V_TF_RX_QUIESCE(1), 0, 0, 0); } /* TLS/DTLS content type for CPL SFO */ @@ -145,16 +154,15 @@ get_tp_plen_max(struct ktls_session *tls) return (tls->params.max_frame_len <= 8192 ? plen : FC_TP_PLEN_MAX); } -/* Send request to get the key-id */ +/* Send request to save the key in on-card memory. */ static int tls_program_key_id(struct toepcb *toep, struct ktls_session *tls, int direction) { struct tls_ofld_info *tls_ofld = &toep->tls; struct adapter *sc = td_adapter(toep->td); - struct ofld_tx_sdesc *txsd; int keyid; - struct wrqe *wr; + struct mbuf *m; struct tls_key_req *kwr; struct tls_keyctx *kctx; @@ -173,12 +181,12 @@ tls_program_key_id(struct toepcb *toep, struct ktls_session *tls, return (ENOSPC); } - wr = alloc_wrqe(TLS_KEY_WR_SZ, &toep->ofld_txq->wrq); - if (wr == NULL) { + m = alloc_raw_wr_mbuf(TLS_KEY_WR_SZ); + if (m == NULL) { t4_free_tls_keyid(sc, keyid); return (ENOMEM); } - kwr = wrtod(wr); + kwr = mtod(m, struct tls_key_req *); memset(kwr, 0, TLS_KEY_WR_SZ); t4_write_tlskey_wr(tls, direction, toep->tid, F_FW_WR_COMPL, keyid, @@ -190,17 +198,7 @@ tls_program_key_id(struct toepcb *toep, struct ktls_session *tls, tls_ofld->rx_key_addr = keyid; t4_tls_key_ctx(tls, direction, kctx); - txsd = &toep->txsd[toep->txsd_pidx]; - _Static_assert(DIV_ROUND_UP(TLS_KEY_WR_SZ, 16) <= - MAX_OFLD_TX_SDESC_CREDITS, "MAX_OFLD_TX_SDESC_CREDITS too small"); - txsd->tx_credits = DIV_ROUND_UP(TLS_KEY_WR_SZ, 16); - txsd->plen = 0; - toep->tx_credits -= txsd->tx_credits; - if (__predict_false(++toep->txsd_pidx == toep->txsd_total)) - toep->txsd_pidx = 0; - toep->txsd_avail--; - - t4_wrq_tx(sc, wr); + t4_raw_wr_tx(sc, toep, m); return (0); } @@ -1099,7 +1097,7 @@ out: static void tls_update_tcb(struct adapter *sc, struct toepcb *toep, uint64_t seqno) { - struct wrqe *wr; + struct mbuf *m; struct work_request_hdr *wrh; struct ulp_txpkt *ulpmc; int fields, key_offset, len; @@ -1128,14 +1126,14 @@ tls_update_tcb(struct adapter *sc, struct toepcb *toep, uint64_t seqno) KASSERT(len <= SGE_MAX_WR_LEN, ("%s: WR with %d TCB field updates too large", __func__, fields)); - wr = alloc_wrqe(len, toep->ctrlq); - if (wr == NULL) { + m = alloc_raw_wr_mbuf(len); + if (m == NULL) { /* XXX */ panic("%s: out of memory", __func__); } - wrh = wrtod(wr); - INIT_ULPTX_WRH(wrh, len, 1, 0); /* atomic */ + wrh = mtod(m, struct work_request_hdr *); + INIT_ULPTX_WRH(wrh, len, 1, toep->tid); /* atomic */ ulpmc = (struct ulp_txpkt *)(wrh + 1); /* @@ -1180,7 +1178,7 @@ tls_update_tcb(struct adapter *sc, struct toepcb *toep, uint64_t seqno) ulpmc = mk_set_tcb_field_ulp(sc, ulpmc, toep->tid, W_TCB_T_FLAGS, V_TF_RX_QUIESCE(1), 0); - t4_wrq_tx(sc, wr); + t4_raw_wr_tx(sc, toep, m); } /* diff --git a/sys/dev/cxgbe/tom/t4_tom.h b/sys/dev/cxgbe/tom/t4_tom.h index af6d37931b2e..4fb87d92d91e 100644 --- a/sys/dev/cxgbe/tom/t4_tom.h +++ b/sys/dev/cxgbe/tom/t4_tom.h @@ -531,6 +531,8 @@ void t4_set_tcb_field(struct adapter *, struct sge_wrq *, struct toepcb *, void t4_push_pdus(struct adapter *, struct toepcb *, int); bool t4_push_raw_wr(struct adapter *, struct toepcb *, struct mbuf *); void t4_raw_wr_tx(struct adapter *, struct toepcb *, struct mbuf *); +void write_set_tcb_field(struct adapter *, void *, struct toepcb *, uint16_t, + uint64_t, uint64_t, int, int); /* t4_ddp.c */ int t4_init_ppod_region(struct ppod_region *, struct t4_range *, u_int,