From nobody Fri Oct 3 00:04:50 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cd86q0g41z6BMk6; Fri, 03 Oct 2025 00:04:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4cd86p5WdWz3t0N; Fri, 03 Oct 2025 00:04:50 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1759449890; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=fWkqNx99PGfxZM4RGZ/jxuVR7BtDut9XcXHE2lT0n+E=; b=Ycd/EO2/7kodUNoJnglUOXTEi6g8IHXxmMhbADtyWxJqZpzX5lmJLwRK6m2x1ffaOuCAUU SARh9Fs8GQGoJdtiOERrD9EgtRa+zIP/er+kUtmYEQnWy1UlZVUGF7ddcC6hze0IHY+nh4 7lxjLkfhVgTiyyaCYWQWZjH5dVCK2nqRxZDgpqsVZet3wfhamzwkgnK5B0yho24cWI+adG FoCrPzSLqX7TLNm2KNTLg2vyXAOLhvwySuABxVj2jB+d/dEHcrDMHO1xlxvpH2aEcIx32V wJL4iShp0da2sszZme9KmWxNMGTdCzB/KwcqUOk8p4hKRfe10zkSI+lqSfjW7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1759449890; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=fWkqNx99PGfxZM4RGZ/jxuVR7BtDut9XcXHE2lT0n+E=; b=ClTPHJg6rxkAK5GCNgc7yyBWj1MxaBqwdOl7f6gYx1MFEj7e9pTDL9cs8iiFhmgGwIuc9B Qk6CO7ZvwDqwGaW6RD4KXPh2afqSZtz6EDuOOvRH5aeg2azAKnap0nkFQ260qIs952NQqe byUdTWbaC1NkOZMoRjPsd+PzPjG8dErX40cRRqYbr8cpAN6trAIuHydqRQrWzdzL0DAl8k I1xgWEoHLF6KajzvUi5aR0URta0KLWJ8IlWg7xUUHgr8konZCY3hRVCaE5N2r1j8BQbAR/ q6ptQTuC2yLAWBvckEVmgI6lMruWNnZSKoHThBYPjW1BkxB4z0W3ndGNaVdM1A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1759449890; a=rsa-sha256; cv=none; b=qXcRzK6O0Q7mjFi6d+xbMgIeC+D9bsBA9yJvoBKOcM7PCi6uNL0yIMXUBvu8iZuYpVdUBU HJDjNY/yNjcLeCW8SKovbipcD/UUuxwLEQKCTXKfmp520puJWP00LZvTGOafb1iq0Rp2Hq Xqe5THus+76y5ebmUcrpHHHDMx5NKSr6ZQojsJQEU4AEgtWX8wiQZpMxT6Mr4yQJ+OzdYN 0tyTMAgcA194Jbcq3SJNrOmnQYElM3l03cVLedhQAnsTn+vW98hmzx2RT56vDv0vW+GjvD V2XISsnp9cJ2+qx2Q5nQvSb9UltWRWB/kdqlBj2IW/xDy2sXgk94rEBny3laqA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4cd86p4rW2zTlK; Fri, 03 Oct 2025 00:04:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 59304oNb057660; Fri, 3 Oct 2025 00:04:50 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 59304ojx057657; Fri, 3 Oct 2025 00:04:50 GMT (envelope-from git) Date: Fri, 3 Oct 2025 00:04:50 GMT Message-Id: <202510030004.59304ojx057657@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Navdeep Parhar Subject: git: 1e062655dc66 - stable/15 - cxgbe: Permit multiple handlers for CPL6_FW_PLD List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: np X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: 1e062655dc66bb0d914dc837cdcd08307f9ca2b4 Auto-Submitted: auto-generated The branch stable/15 has been updated by np: URL: https://cgit.FreeBSD.org/src/commit/?id=1e062655dc66bb0d914dc837cdcd08307f9ca2b4 commit 1e062655dc66bb0d914dc837cdcd08307f9ca2b4 Author: John Baldwin AuthorDate: 2025-09-29 15:12:09 +0000 Commit: Navdeep Parhar CommitDate: 2025-10-02 22:09:09 +0000 cxgbe: Permit multiple handlers for CPL6_FW_PLD Currently this CPL is only used to handle replies from lookaside crypto requests submitted by ccr(4). However, in the future this request will be returned for other requests. Use the low bit in the cookie field as a way to identify replies to ccr(4) vs other use cases. This should be safe as 'struct cryptop' pointers should be word-aligned. Sponsored by: Chelsio Communications (cherry picked from commit 96469647bce10132408db9b436dae8941356453b) --- sys/dev/cxgbe/adapter.h | 17 +++++++++++++++++ sys/dev/cxgbe/crypto/t4_crypto.c | 27 +++++++++++++++++++++++---- sys/dev/cxgbe/t4_sge.c | 28 ++++++++++++++++++++++++++-- 3 files changed, 66 insertions(+), 6 deletions(-) diff --git a/sys/dev/cxgbe/adapter.h b/sys/dev/cxgbe/adapter.h index 9064d148cba9..64fa4526d9f4 100644 --- a/sys/dev/cxgbe/adapter.h +++ b/sys/dev/cxgbe/adapter.h @@ -412,6 +412,23 @@ enum { NUM_CPL_COOKIES = 8 /* Limited by M_COOKIE. Do not increase. */ }; +/* + * Crypto replies use the low bit in the 64-bit cookie of CPL_FW6_PLD as a + * CPL cookie to identify the sender/receiver. + */ +enum { + CPL_FW6_COOKIE_CCR = 0, + + NUM_CPL_FW6_COOKIES = 2 /* Low bits of cookie value. */ +}; + +_Static_assert(powerof2(NUM_CPL_FW6_COOKIES), + "NUM_CPL_FW6_COOKIES must be a power of 2"); + +#define CPL_FW6_COOKIE_MASK (NUM_CPL_FW6_COOKIES - 1) + +#define CPL_FW6_PLD_COOKIE(cpl) (be64toh((cpl)->data[1]) & ~CPL_FW6_COOKIE_MASK) + struct sge_iq; struct rss_header; typedef int (*cpl_handler_t)(struct sge_iq *, const struct rss_header *, diff --git a/sys/dev/cxgbe/crypto/t4_crypto.c b/sys/dev/cxgbe/crypto/t4_crypto.c index 24dfdc1f23db..80e31b1159fd 100644 --- a/sys/dev/cxgbe/crypto/t4_crypto.c +++ b/sys/dev/cxgbe/crypto/t4_crypto.c @@ -208,6 +208,7 @@ struct ccr_softc { counter_u64_t stats_pad_error; counter_u64_t stats_sglist_error; counter_u64_t stats_process_error; + counter_u64_t stats_pointer_error; counter_u64_t stats_sw_fallback; struct sysctl_ctx_list ctx; @@ -1884,6 +1885,9 @@ ccr_sysctls(struct ccr_softc *sc) SYSCTL_ADD_COUNTER_U64(ctx, children, OID_AUTO, "process_error", CTLFLAG_RD, &sc->stats_process_error, "Requests failed during queueing"); + SYSCTL_ADD_COUNTER_U64(ctx, children, OID_AUTO, "pointer_error", + CTLFLAG_RD, &sc->stats_pointer_error, + "Requests with a misaligned request pointer"); SYSCTL_ADD_COUNTER_U64(ctx, children, OID_AUTO, "sw_fallback", CTLFLAG_RD, &sc->stats_sw_fallback, "Requests processed by falling back to software"); @@ -1991,6 +1995,7 @@ ccr_attach(device_t dev) sc->stats_pad_error = counter_u64_alloc(M_WAITOK); sc->stats_sglist_error = counter_u64_alloc(M_WAITOK); sc->stats_process_error = counter_u64_alloc(M_WAITOK); + sc->stats_pointer_error = counter_u64_alloc(M_WAITOK); sc->stats_sw_fallback = counter_u64_alloc(M_WAITOK); ccr_sysctls(sc); @@ -2037,6 +2042,7 @@ ccr_detach(device_t dev) counter_u64_free(sc->stats_pad_error); counter_u64_free(sc->stats_sglist_error); counter_u64_free(sc->stats_process_error); + counter_u64_free(sc->stats_pointer_error); counter_u64_free(sc->stats_sw_fallback); for_each_port(sc->adapter, i) { ccr_free_port(sc, i); @@ -2534,6 +2540,16 @@ ccr_process(device_t dev, struct cryptop *crp, int hint) s = crypto_get_driver_session(crp->crp_session); sc = device_get_softc(dev); + /* + * Request pointers with the low bit set in the pointer can't + * be stored as the cookie in the CPL_FW6_PLD reply. + */ + if (((uintptr_t)crp & CPL_FW6_COOKIE_MASK) != 0) { + counter_u64_add(sc->stats_pointer_error, 1); + error = EINVAL; + goto out_unlocked; + } + mtx_lock(&s->lock); error = ccr_populate_sglist(s->sg_input, &crp->crp_buf); if (error == 0 && CRYPTO_HAS_OUTPUT_BUFFER(crp)) @@ -2640,6 +2656,7 @@ ccr_process(device_t dev, struct cryptop *crp, int hint) out: mtx_unlock(&s->lock); +out_unlocked: if (error) { crp->crp_etype = error; crypto_done(crp); @@ -2649,7 +2666,7 @@ out: } static int -do_cpl6_fw_pld(struct sge_iq *iq, const struct rss_header *rss, +fw6_pld_ccr(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) { struct ccr_softc *sc; @@ -2664,7 +2681,7 @@ do_cpl6_fw_pld(struct sge_iq *iq, const struct rss_header *rss, else cpl = (const void *)(rss + 1); - crp = (struct cryptop *)(uintptr_t)be64toh(cpl->data[1]); + crp = (struct cryptop *)(uintptr_t)CPL_FW6_PLD_COOKIE(cpl); s = crypto_get_driver_session(crp->crp_session); status = be64toh(cpl->data[0]); if (CHK_MAC_ERR_BIT(status) || CHK_PAD_ERR_BIT(status)) @@ -2718,10 +2735,12 @@ ccr_modevent(module_t mod, int cmd, void *arg) switch (cmd) { case MOD_LOAD: - t4_register_cpl_handler(CPL_FW6_PLD, do_cpl6_fw_pld); + t4_register_shared_cpl_handler(CPL_FW6_PLD, fw6_pld_ccr, + CPL_FW6_COOKIE_CCR); return (0); case MOD_UNLOAD: - t4_register_cpl_handler(CPL_FW6_PLD, NULL); + t4_register_shared_cpl_handler(CPL_FW6_PLD, NULL, + CPL_FW6_COOKIE_CCR); return (0); default: return (EOPNOTSUPP); diff --git a/sys/dev/cxgbe/t4_sge.c b/sys/dev/cxgbe/t4_sge.c index 155a47552b40..6a32aeae4a82 100644 --- a/sys/dev/cxgbe/t4_sge.c +++ b/sys/dev/cxgbe/t4_sge.c @@ -350,6 +350,7 @@ cpl_handler_t l2t_write_rpl_handlers[NUM_CPL_COOKIES]; cpl_handler_t act_open_rpl_handlers[NUM_CPL_COOKIES]; cpl_handler_t abort_rpl_rss_handlers[NUM_CPL_COOKIES]; cpl_handler_t fw4_ack_handlers[NUM_CPL_COOKIES]; +cpl_handler_t fw6_pld_handlers[NUM_CPL_FW6_COOKIES]; void t4_register_an_handler(an_handler_t h) @@ -479,6 +480,21 @@ fw4_ack_handler(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) return (fw4_ack_handlers[cookie](iq, rss, m)); } +static int +fw6_pld_handler(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) +{ + const struct cpl_fw6_pld *cpl; + uint64_t cookie; + + if (m != NULL) + cpl = mtod(m, const void *); + else + cpl = (const void *)(rss + 1); + cookie = be64toh(cpl->data[1]) & CPL_FW6_COOKIE_MASK; + + return (fw6_pld_handlers[cookie](iq, rss, m)); +} + static void t4_init_shared_cpl_handlers(void) { @@ -488,6 +504,7 @@ t4_init_shared_cpl_handlers(void) t4_register_cpl_handler(CPL_ACT_OPEN_RPL, act_open_rpl_handler); t4_register_cpl_handler(CPL_ABORT_RPL_RSS, abort_rpl_rss_handler); t4_register_cpl_handler(CPL_FW4_ACK, fw4_ack_handler); + t4_register_cpl_handler(CPL_FW6_PLD, fw6_pld_handler); } void @@ -496,8 +513,12 @@ t4_register_shared_cpl_handler(int opcode, cpl_handler_t h, int cookie) uintptr_t *loc; MPASS(opcode < nitems(t4_cpl_handler)); - MPASS(cookie > CPL_COOKIE_RESERVED); - MPASS(cookie < NUM_CPL_COOKIES); + if (opcode == CPL_FW6_PLD) { + MPASS(cookie < NUM_CPL_FW6_COOKIES); + } else { + MPASS(cookie > CPL_COOKIE_RESERVED); + MPASS(cookie < NUM_CPL_COOKIES); + } MPASS(t4_cpl_handler[opcode] != NULL); switch (opcode) { @@ -516,6 +537,9 @@ t4_register_shared_cpl_handler(int opcode, cpl_handler_t h, int cookie) case CPL_FW4_ACK: loc = (uintptr_t *)&fw4_ack_handlers[cookie]; break; + case CPL_FW6_PLD: + loc = (uintptr_t *)&fw6_pld_handlers[cookie]; + break; default: MPASS(0); return;