From nobody Mon Sep 29 15:22:39 2025 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cb4gh0jNGz68NZ6; Mon, 29 Sep 2025 15:22:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4cb4gg3YWPz46cg; Mon, 29 Sep 2025 15:22:39 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1759159359; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zo80kyc5cxuTUyw5hLZbGR+zv/ax6HRxC9jD9W2zW24=; b=LGNzLcDBcHps3NGALQSxxYcIYqWoM2JneRw7tmmm9/se5PkCtETDTwWE0goap7+nD1XfOS Io5cid7IBOE7XIW4U8YjosHkIH/TpCuyRi08yCTlrQfly5dMZ3ZlBADNl4udyZWXLiry3n xoG1HW81Hv9BNaTaRTHrz5XkJcM1a/3kNzcq8FRsbVEcAcVY5N625wnk4MmFXMuTQZeRuv roKQ9tA/BczCnkJb8pr+zD0N1cgB13+vsxt1L1Q5A/btJtiWqCdvZgcpCElZyhlek9aZBk +Ymk6zMLsJMycu3EGPzLbT6PG9SnfVnlVnzTtXiknBmUqKNQ4Ymj4BpJVh2f5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1759159359; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zo80kyc5cxuTUyw5hLZbGR+zv/ax6HRxC9jD9W2zW24=; b=Kd+1u+SOCyMAoCfYEg9UvvUlt9C9mxJE3AV7mpfw/NRe4V9JORH905ElcpHeMS1poFZq3C Pyc86/Y1waqpWp7HhaRryczsexG8zlOh8sqOcBJvATbmNW5ObKs14lJeJtbKNCT9jmpe4v HmBXa0b2+dREz8aMAqeF63X/lV5tUDhXt+g9Z2jdfcnP3RN9aOQBuUnvWkU3Qqr2Vv2A63 EpYQzFLwOD6C3bbYvFfJNzsKjJsWym9z41+Q8sRgkev5DOWQhJzPrYkHAiaqJSftKQdQjA ukPPhJBvzCSo6EtIhQEo1Vr1O899W4Ct+nn1KyGImHSarxmMd1a57seCZgF2hg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1759159359; a=rsa-sha256; cv=none; b=GeXci7RQaBGeTFTNpF5gMITw4QMvYzc3viq+hlaD6oroudDpL0hb8FRr/nNywUjTTrE2Jj 2qn5dGdCDqQaTRajaClguGH2Kngi3qdSOKiZIEcMfRlenpjorCmBcG/x83gh3isz6IAgIo 55lgUsJ6AwuWPfWEYuBm8hGMhZxolcppwZFEFtXN4FcALJHs2W7kcEYJVREcCJKampkmCK zjkg4VgviwYvgiS0HgsAFYLxqJUNdwUi5HNtXz1uDESU3zuVDq86JNlg9pCDcAFrLR86wY 7nrx0vpVGHarINO5j1uDN8r/Te1PBXiIlXJnRIeLzSlSqpp7ucgWWSKBLEq34w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4cb4gg31cGz191q; Mon, 29 Sep 2025 15:22:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 58TFMdc8012156; Mon, 29 Sep 2025 15:22:39 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 58TFMdvE012153; Mon, 29 Sep 2025 15:22:39 GMT (envelope-from git) Date: Mon, 29 Sep 2025 15:22:39 GMT Message-Id: <202509291522.58TFMdvE012153@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Navdeep Parhar Subject: git: 96469647bce1 - main - cxgbe: Permit multiple handlers for CPL6_FW_PLD List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: np X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 96469647bce10132408db9b436dae8941356453b Auto-Submitted: auto-generated The branch main has been updated by np: URL: https://cgit.FreeBSD.org/src/commit/?id=96469647bce10132408db9b436dae8941356453b commit 96469647bce10132408db9b436dae8941356453b Author: John Baldwin AuthorDate: 2025-09-29 15:12:09 +0000 Commit: Navdeep Parhar CommitDate: 2025-09-29 15:19:13 +0000 cxgbe: Permit multiple handlers for CPL6_FW_PLD Currently this CPL is only used to handle replies from lookaside crypto requests submitted by ccr(4). However, in the future this request will be returned for other requests. Use the low bit in the cookie field as a way to identify replies to ccr(4) vs other use cases. This should be safe as 'struct cryptop' pointers should be word-aligned. MFC after: 3 days Sponsored by: Chelsio Communications --- sys/dev/cxgbe/adapter.h | 17 +++++++++++++++++ sys/dev/cxgbe/crypto/t4_crypto.c | 27 +++++++++++++++++++++++---- sys/dev/cxgbe/t4_sge.c | 28 ++++++++++++++++++++++++++-- 3 files changed, 66 insertions(+), 6 deletions(-) diff --git a/sys/dev/cxgbe/adapter.h b/sys/dev/cxgbe/adapter.h index 9064d148cba9..64fa4526d9f4 100644 --- a/sys/dev/cxgbe/adapter.h +++ b/sys/dev/cxgbe/adapter.h @@ -412,6 +412,23 @@ enum { NUM_CPL_COOKIES = 8 /* Limited by M_COOKIE. Do not increase. */ }; +/* + * Crypto replies use the low bit in the 64-bit cookie of CPL_FW6_PLD as a + * CPL cookie to identify the sender/receiver. + */ +enum { + CPL_FW6_COOKIE_CCR = 0, + + NUM_CPL_FW6_COOKIES = 2 /* Low bits of cookie value. */ +}; + +_Static_assert(powerof2(NUM_CPL_FW6_COOKIES), + "NUM_CPL_FW6_COOKIES must be a power of 2"); + +#define CPL_FW6_COOKIE_MASK (NUM_CPL_FW6_COOKIES - 1) + +#define CPL_FW6_PLD_COOKIE(cpl) (be64toh((cpl)->data[1]) & ~CPL_FW6_COOKIE_MASK) + struct sge_iq; struct rss_header; typedef int (*cpl_handler_t)(struct sge_iq *, const struct rss_header *, diff --git a/sys/dev/cxgbe/crypto/t4_crypto.c b/sys/dev/cxgbe/crypto/t4_crypto.c index 24dfdc1f23db..80e31b1159fd 100644 --- a/sys/dev/cxgbe/crypto/t4_crypto.c +++ b/sys/dev/cxgbe/crypto/t4_crypto.c @@ -208,6 +208,7 @@ struct ccr_softc { counter_u64_t stats_pad_error; counter_u64_t stats_sglist_error; counter_u64_t stats_process_error; + counter_u64_t stats_pointer_error; counter_u64_t stats_sw_fallback; struct sysctl_ctx_list ctx; @@ -1884,6 +1885,9 @@ ccr_sysctls(struct ccr_softc *sc) SYSCTL_ADD_COUNTER_U64(ctx, children, OID_AUTO, "process_error", CTLFLAG_RD, &sc->stats_process_error, "Requests failed during queueing"); + SYSCTL_ADD_COUNTER_U64(ctx, children, OID_AUTO, "pointer_error", + CTLFLAG_RD, &sc->stats_pointer_error, + "Requests with a misaligned request pointer"); SYSCTL_ADD_COUNTER_U64(ctx, children, OID_AUTO, "sw_fallback", CTLFLAG_RD, &sc->stats_sw_fallback, "Requests processed by falling back to software"); @@ -1991,6 +1995,7 @@ ccr_attach(device_t dev) sc->stats_pad_error = counter_u64_alloc(M_WAITOK); sc->stats_sglist_error = counter_u64_alloc(M_WAITOK); sc->stats_process_error = counter_u64_alloc(M_WAITOK); + sc->stats_pointer_error = counter_u64_alloc(M_WAITOK); sc->stats_sw_fallback = counter_u64_alloc(M_WAITOK); ccr_sysctls(sc); @@ -2037,6 +2042,7 @@ ccr_detach(device_t dev) counter_u64_free(sc->stats_pad_error); counter_u64_free(sc->stats_sglist_error); counter_u64_free(sc->stats_process_error); + counter_u64_free(sc->stats_pointer_error); counter_u64_free(sc->stats_sw_fallback); for_each_port(sc->adapter, i) { ccr_free_port(sc, i); @@ -2534,6 +2540,16 @@ ccr_process(device_t dev, struct cryptop *crp, int hint) s = crypto_get_driver_session(crp->crp_session); sc = device_get_softc(dev); + /* + * Request pointers with the low bit set in the pointer can't + * be stored as the cookie in the CPL_FW6_PLD reply. + */ + if (((uintptr_t)crp & CPL_FW6_COOKIE_MASK) != 0) { + counter_u64_add(sc->stats_pointer_error, 1); + error = EINVAL; + goto out_unlocked; + } + mtx_lock(&s->lock); error = ccr_populate_sglist(s->sg_input, &crp->crp_buf); if (error == 0 && CRYPTO_HAS_OUTPUT_BUFFER(crp)) @@ -2640,6 +2656,7 @@ ccr_process(device_t dev, struct cryptop *crp, int hint) out: mtx_unlock(&s->lock); +out_unlocked: if (error) { crp->crp_etype = error; crypto_done(crp); @@ -2649,7 +2666,7 @@ out: } static int -do_cpl6_fw_pld(struct sge_iq *iq, const struct rss_header *rss, +fw6_pld_ccr(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) { struct ccr_softc *sc; @@ -2664,7 +2681,7 @@ do_cpl6_fw_pld(struct sge_iq *iq, const struct rss_header *rss, else cpl = (const void *)(rss + 1); - crp = (struct cryptop *)(uintptr_t)be64toh(cpl->data[1]); + crp = (struct cryptop *)(uintptr_t)CPL_FW6_PLD_COOKIE(cpl); s = crypto_get_driver_session(crp->crp_session); status = be64toh(cpl->data[0]); if (CHK_MAC_ERR_BIT(status) || CHK_PAD_ERR_BIT(status)) @@ -2718,10 +2735,12 @@ ccr_modevent(module_t mod, int cmd, void *arg) switch (cmd) { case MOD_LOAD: - t4_register_cpl_handler(CPL_FW6_PLD, do_cpl6_fw_pld); + t4_register_shared_cpl_handler(CPL_FW6_PLD, fw6_pld_ccr, + CPL_FW6_COOKIE_CCR); return (0); case MOD_UNLOAD: - t4_register_cpl_handler(CPL_FW6_PLD, NULL); + t4_register_shared_cpl_handler(CPL_FW6_PLD, NULL, + CPL_FW6_COOKIE_CCR); return (0); default: return (EOPNOTSUPP); diff --git a/sys/dev/cxgbe/t4_sge.c b/sys/dev/cxgbe/t4_sge.c index 155a47552b40..6a32aeae4a82 100644 --- a/sys/dev/cxgbe/t4_sge.c +++ b/sys/dev/cxgbe/t4_sge.c @@ -350,6 +350,7 @@ cpl_handler_t l2t_write_rpl_handlers[NUM_CPL_COOKIES]; cpl_handler_t act_open_rpl_handlers[NUM_CPL_COOKIES]; cpl_handler_t abort_rpl_rss_handlers[NUM_CPL_COOKIES]; cpl_handler_t fw4_ack_handlers[NUM_CPL_COOKIES]; +cpl_handler_t fw6_pld_handlers[NUM_CPL_FW6_COOKIES]; void t4_register_an_handler(an_handler_t h) @@ -479,6 +480,21 @@ fw4_ack_handler(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) return (fw4_ack_handlers[cookie](iq, rss, m)); } +static int +fw6_pld_handler(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) +{ + const struct cpl_fw6_pld *cpl; + uint64_t cookie; + + if (m != NULL) + cpl = mtod(m, const void *); + else + cpl = (const void *)(rss + 1); + cookie = be64toh(cpl->data[1]) & CPL_FW6_COOKIE_MASK; + + return (fw6_pld_handlers[cookie](iq, rss, m)); +} + static void t4_init_shared_cpl_handlers(void) { @@ -488,6 +504,7 @@ t4_init_shared_cpl_handlers(void) t4_register_cpl_handler(CPL_ACT_OPEN_RPL, act_open_rpl_handler); t4_register_cpl_handler(CPL_ABORT_RPL_RSS, abort_rpl_rss_handler); t4_register_cpl_handler(CPL_FW4_ACK, fw4_ack_handler); + t4_register_cpl_handler(CPL_FW6_PLD, fw6_pld_handler); } void @@ -496,8 +513,12 @@ t4_register_shared_cpl_handler(int opcode, cpl_handler_t h, int cookie) uintptr_t *loc; MPASS(opcode < nitems(t4_cpl_handler)); - MPASS(cookie > CPL_COOKIE_RESERVED); - MPASS(cookie < NUM_CPL_COOKIES); + if (opcode == CPL_FW6_PLD) { + MPASS(cookie < NUM_CPL_FW6_COOKIES); + } else { + MPASS(cookie > CPL_COOKIE_RESERVED); + MPASS(cookie < NUM_CPL_COOKIES); + } MPASS(t4_cpl_handler[opcode] != NULL); switch (opcode) { @@ -516,6 +537,9 @@ t4_register_shared_cpl_handler(int opcode, cpl_handler_t h, int cookie) case CPL_FW4_ACK: loc = (uintptr_t *)&fw4_ack_handlers[cookie]; break; + case CPL_FW6_PLD: + loc = (uintptr_t *)&fw6_pld_handlers[cookie]; + break; default: MPASS(0); return;