From nobody Tue Apr 28 06:45:03 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4g4WC515bKz6bR2d for ; Tue, 28 Apr 2026 06:45:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R13" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4g4WC35wF5z3Z9M for ; Tue, 28 Apr 2026 06:45:03 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1777358703; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=TfFWKTl0Oi6zbsV9D0Ko97afxhrStHBwDviccmldZMM=; b=SBsAjcCzJGYen2skxkkAtUkFRGF9Ew23rKHem9RidBm7dNXdQWK+5HE9pFZuAUF6SYi2X8 AwcQbVYrBS23CcmutvQR5MEohQigqjIr3BxdIf6AHO4W7wwsiVYrf+F77yyAgg0W7RuHzR W+MyrQq4nZi4PbHhxbc/ECahPFINAR1ARo0f2YmiwTvX1mhWts3Ba+wAauyj5qJ+xrMdnK IWMUXP/QOMNqOpme75WS/uibGiF5zldothdp3PUxwPBgFbKeKZY6ZK3YOMcB1nc6Tk38SY 7wjiJFPR7C/vxjM/BX9GXQxJ+emP05YXmYR5jiXDn6lyOWXryea3Tq8OmCJaYQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1777358703; a=rsa-sha256; cv=none; b=e+S8bBv7AvOn0Yw6Lgjri78bj/tKttb2eyp24abesZHKKMrh7fzzaH5dgcnpk/0Xr+6sh7 ApPaGLzFtVhTVxQwE406eAwhWAscFbNmZVGYkgKjQgikHNJvK9y2ifIjhGpVir+aXlFwDQ ngjsCnVTFk3T4BvmfheYxhjCG43l19dLfQkm5IdDujEaB776NJHRylDgUFZ7g5T395SDkV ZsiLcz8iY8ed8wpOsP9qSotzl4w803pgZwoODZY3jY1yO35obPzUvZkvD3YXaEzyImcl5A fnAoCt6ouj7L8AWGcw/uEOdUbfmZrkEM3zwPndFZHZEP1/zTWgIiYYk7KvQolQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1777358703; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=TfFWKTl0Oi6zbsV9D0Ko97afxhrStHBwDviccmldZMM=; b=lAfYNamKgMM2l1dlq9xzmJdyAaNE/jXVjW7PBB1Js4/Mv5KlFUM1njSpWGwlqBvCufr87f CFkuZvnR6qrCxRVVgqAkHNDcXrIe5eMaRlrVB4Z044xHMLR1CSadLocJMCkYLQujB1Tb7H dpTamyZBqlLJW8Qok9NFAslMpf4OEBQcABJl+5qaYczGhxBfUWpl5octb4qDiXFPOF+7kL zFzTyYDRI8jxHqMJoGvZ76Ii9qLLBnGHLUhfg88r5FCK55J+o0QHslxiq9yyibP23T0wke OBl0rqsmBJWwkgyqFiQoW5WloKIUZpjm1yW8XHbPwZf6NgB2AdsNClEDe8Ka8g== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4g4WC350BYzjsb for ; Tue, 28 Apr 2026 06:45:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 275f8 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Tue, 28 Apr 2026 06:45:03 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Cc: Chandrakanth Patil From: Sumit Saxena Subject: git: c972c5acbac4 - main - bnxt_en: VF ring reservation, HWRM registration, and PF-only operation guards List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: ssaxena X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: c972c5acbac472a5dc797856f39f478862b6c6ea Auto-Submitted: auto-generated Date: Tue, 28 Apr 2026 06:45:03 +0000 Message-Id: <69f0576f.275f8.102c997@gitrepo.freebsd.org> The branch main has been updated by ssaxena: URL: https://cgit.FreeBSD.org/src/commit/?id=c972c5acbac472a5dc797856f39f478862b6c6ea commit c972c5acbac472a5dc797856f39f478862b6c6ea Author: Chandrakanth Patil AuthorDate: 2026-04-02 23:04:45 +0000 Commit: Sumit Saxena CommitDate: 2026-04-28 06:35:09 +0000 bnxt_en: VF ring reservation, HWRM registration, and PF-only operation guards VFs require separate HWRM commands for ring reservation and async completion ring setup, so a common PF/VF dispatcher is introduced and the async CR path is extended to handle both. The PF must populate the VF request forwarding bitmap during driver registration so the firmware correctly forwards VF-originated HWRM commands. VF reservation strategy and min-guaranteed capability flags are now parsed for correct resource partitioning, and PF-only operations (DCB, NVM, package version sysctl) are guarded against VF invocation. The short command buffer allocation is also reordered before the function reset to ensure extended HWRM messages are available when needed, a prerequisite uncovered during VF bring-up. MFC after: 1 month Reviewed by: ssaxena Differential Revision: https://reviews.freebsd.org/D56232 --- sys/dev/bnxt/bnxt_en/bnxt.h | 1 + sys/dev/bnxt/bnxt_en/bnxt_dcb.c | 3 ++ sys/dev/bnxt/bnxt_en/bnxt_hwrm.c | 105 ++++++++++++++++++++++++++++++++----- sys/dev/bnxt/bnxt_en/bnxt_hwrm.h | 1 + sys/dev/bnxt/bnxt_en/bnxt_sysctl.c | 9 ++-- sys/dev/bnxt/bnxt_en/if_bnxt.c | 23 ++++++-- 6 files changed, 121 insertions(+), 21 deletions(-) diff --git a/sys/dev/bnxt/bnxt_en/bnxt.h b/sys/dev/bnxt/bnxt_en/bnxt.h index 4172e5e86b4f..6d00cd4d601d 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt.h +++ b/sys/dev/bnxt/bnxt_en/bnxt.h @@ -1398,5 +1398,6 @@ int bnxt_dcb_ieee_delapp(struct bnxt_softc *softc, struct bnxt_dcb_app *app); int bnxt_dcb_ieee_listapp(struct bnxt_softc *softc, struct bnxt_dcb_app *app, size_t nitems, int *num_inputs); void bnxt_set_flags_by_devid(struct bnxt_softc *softc); +int bnxt_hwrm_reserve_rings(struct bnxt_softc *softc); #endif /* _BNXT_H */ diff --git a/sys/dev/bnxt/bnxt_en/bnxt_dcb.c b/sys/dev/bnxt/bnxt_en/bnxt_dcb.c index e0643f200021..d169be6e5c7b 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_dcb.c +++ b/sys/dev/bnxt/bnxt_en/bnxt_dcb.c @@ -326,6 +326,9 @@ bnxt_hwrm_get_dcbx_app(struct bnxt_softc *softc, struct bnxt_dcb_app *app, if (softc->hwrm_spec_code < 0x10601) return 0; + if (BNXT_VF(softc)) + return 0; + bnxt_hwrm_cmd_hdr_init(softc, &get, HWRM_FW_GET_STRUCTURED_DATA); n = BNXT_IEEE_8021QAZ_MAX_TCS; diff --git a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c index 944824a1c182..b89a44e4677b 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c +++ b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c @@ -79,6 +79,14 @@ long bnxt_rx_pkts_pri_arr_base_off[] = {BNXT_RX_STATS_PRI_ENTRIES(rx_packets)}; long bnxt_tx_bytes_pri_arr_base_off[] = {BNXT_TX_STATS_PRI_ENTRIES(tx_bytes)}; long bnxt_tx_pkts_pri_arr_base_off[] = {BNXT_TX_STATS_PRI_ENTRIES(tx_packets)}; +uint16_t bnxt_vf_req_snif[] = { + HWRM_FUNC_CFG, + HWRM_FUNC_VF_CFG, + HWRM_PORT_PHY_QCFG, + HWRM_CFA_L2_FILTER_ALLOC, + HWRM_OEM_CMD, +}; + static int bnxt_hwrm_err_map(uint16_t err) { @@ -757,9 +765,21 @@ int bnxt_hwrm_func_resc_qcaps(struct bnxt_softc *softc, bool all) hw_resc->max_stat_ctxs = le16toh(resp->max_stat_ctx); if (BNXT_CHIP_P5_PLUS(softc)) { - hw_resc->max_nqs = le16toh(resp->max_msix); + hw_resc->max_nqs = hw_resc->max_irqs = le16toh(resp->max_msix); hw_resc->max_hw_ring_grps = hw_resc->max_rx_rings; } + + if (BNXT_PF(softc)) { + struct bnxt_pf_info *pf = &softc->pf; + + pf->vf_resv_strategy = le16toh(resp->vf_reservation_strategy); + if (pf->vf_resv_strategy > BNXT_VF_RESV_STRATEGY_MINIMAL_STATIC) + pf->vf_resv_strategy = BNXT_VF_RESV_STRATEGY_MAXIMAL; + + if (resp->flags & + htole16(FUNC_RESOURCE_QCAPS_RESP_FLAGS_MIN_GUARANTEED)) + softc->fw_cap |= BNXT_FW_CAP_VF_RES_MIN_GUARANTEED; + } hwrm_func_resc_qcaps_exit: BNXT_HWRM_UNLOCK(softc); @@ -1054,6 +1074,26 @@ int bnxt_hwrm_func_drv_rgtr(struct bnxt_softc *bp, unsigned long *bmap, int bmap req.os_type = htole16(HWRM_FUNC_DRV_RGTR_INPUT_OS_TYPE_FREEBSD); if (BNXT_PF(bp)) { + u32 data[8]; + int i; + + memset(data, 0, sizeof(data)); + for (i = 0; i < ARRAY_SIZE(bnxt_vf_req_snif); i++) { + u16 cmd = bnxt_vf_req_snif[i]; + unsigned int bit, idx; + + if ((bp->fw_cap & BNXT_FW_CAP_LINK_ADMIN) && + (cmd == HWRM_PORT_PHY_QCFG)) + continue; + + idx = cmd / 32; + bit = cmd % 32; + data[idx] |= 1 << bit; + } + + for (i = 0; i < 8; i++) + req.vf_req_fwd[i] = cpu_to_le32(data[i]); + req.enables |= htole32(HWRM_FUNC_DRV_RGTR_INPUT_ENABLES_VF_REQ_FWD); } @@ -2299,25 +2339,63 @@ bnxt_hwrm_reserve_pf_rings(struct bnxt_softc *softc) return hwrm_send_message(softc, &req, sizeof(req)); } +#define BNXT_VF_MAX_L2_CTX 4 +int bnxt_hwrm_reserve_vf_rings(struct bnxt_softc *softc) +{ + struct hwrm_func_vf_cfg_input req = {0}; + + bnxt_hwrm_cmd_hdr_init(softc, &req, HWRM_FUNC_VF_CFG); + + req.enables |= htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_NUM_RSSCOS_CTXS); + req.enables |= htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_NUM_CMPL_RINGS); + req.enables |= htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_NUM_TX_RINGS); + req.enables |= htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_NUM_RX_RINGS); + req.enables |= htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_NUM_VNICS); + req.enables |= htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_NUM_STAT_CTXS); + req.enables |= htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_NUM_L2_CTXS); + req.num_rsscos_ctxs = htole16(0x8); + req.num_cmpl_rings = htole16(BNXT_MAX_NUM_QUEUES * 2); + req.num_tx_rings = htole16(BNXT_MAX_NUM_QUEUES); + req.num_rx_rings = htole16(BNXT_MAX_NUM_QUEUES); + req.num_vnics = htole16(BNXT_MAX_NUM_QUEUES); + req.num_stat_ctxs = htole16(BNXT_MAX_NUM_QUEUES * 2); + req.num_l2_ctxs = htole16(BNXT_VF_MAX_L2_CTX); + + return hwrm_send_message(softc, &req, sizeof(req)); +} + int bnxt_cfg_async_cr(struct bnxt_softc *softc) { int rc = 0; - struct hwrm_func_cfg_input req = {0}; - if (!BNXT_PF(softc)) - return 0; + if (BNXT_PF(softc)) { + struct hwrm_func_cfg_input req = {0}; - bnxt_hwrm_cmd_hdr_init(softc, &req, HWRM_FUNC_CFG); + bnxt_hwrm_cmd_hdr_init(softc, &req, HWRM_FUNC_CFG); - req.fid = htole16(0xffff); - req.enables = htole32(HWRM_FUNC_CFG_INPUT_ENABLES_ASYNC_EVENT_CR); - if (BNXT_CHIP_P5_PLUS(softc)) - req.async_event_cr = htole16(softc->nq_rings[0].ring.phys_id); - else - req.async_event_cr = htole16(softc->def_cp_ring.ring.phys_id); + req.fid = htole16(0xffff); + req.enables = htole32(HWRM_FUNC_CFG_INPUT_ENABLES_ASYNC_EVENT_CR); + if (BNXT_CHIP_P5_PLUS(softc)) + req.async_event_cr = htole16(softc->nq_rings[0].ring.phys_id); + else + req.async_event_cr = htole16(softc->def_cp_ring.ring.phys_id); - rc = hwrm_send_message(softc, &req, sizeof(req)); + rc = hwrm_send_message(softc, &req, sizeof(req)); + } else { + /* VF needs to configure async event completion ring using HWRM_FUNC_VF_CFG */ + struct hwrm_func_vf_cfg_input req = {0}; + + bnxt_hwrm_cmd_hdr_init(softc, &req, HWRM_FUNC_VF_CFG); + + req.enables = htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_ASYNC_EVENT_CR); + if (BNXT_CHIP_P5_PLUS(softc)) + req.async_event_cr = htole16(softc->nq_rings[0].ring.phys_id); + else + req.async_event_cr = htole16(softc->def_cp_ring.ring.phys_id); + + rc = hwrm_send_message(softc, &req, sizeof(req)); + } return rc; } @@ -2391,6 +2469,9 @@ bnxt_hwrm_nvm_find_dir_entry(struct bnxt_softc *softc, uint16_t type, (void *)softc->hwrm_cmd_resp.idi_vaddr; int rc = 0; uint32_t old_timeo; + + if (BNXT_VF(softc)) + return 0; MPASS(ordinal); diff --git a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.h b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.h index 126cad977c82..b10981d9d53f 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.h +++ b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.h @@ -141,4 +141,5 @@ int bnxt_hwrm_passthrough(struct bnxt_softc *softc, void *req, uint32_t req_len, int _hwrm_send_message(struct bnxt_softc *, void *, uint32_t); int hwrm_send_message(struct bnxt_softc *, void *, uint32_t); void bnxt_hwrm_cmd_hdr_init(struct bnxt_softc *, void *, uint16_t); +int bnxt_hwrm_reserve_vf_rings (struct bnxt_softc *softc); #endif diff --git a/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c b/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c index 51438e657546..741cba4588aa 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c +++ b/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c @@ -1219,10 +1219,11 @@ bnxt_create_ver_sysctls(struct bnxt_softc *softc) "chip_type", CTLFLAG_RD, vi->chip_type > MAX_CHIP_TYPE ? bnxt_chip_type[MAX_CHIP_TYPE] : bnxt_chip_type[vi->chip_type], 0, "RoCE firmware name"); - SYSCTL_ADD_PROC(&vi->ver_ctx, SYSCTL_CHILDREN(oid), OID_AUTO, - "package_ver", CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, - softc, 0, bnxt_package_ver_sysctl, "A", - "currently installed package version"); + if (!BNXT_VF(softc)) + SYSCTL_ADD_PROC(&vi->ver_ctx, SYSCTL_CHILDREN(oid), OID_AUTO, + "package_ver", CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, + softc, 0, bnxt_package_ver_sysctl, "A", + "currently installed package version"); SYSCTL_ADD_PROC(&vi->ver_ctx, SYSCTL_CHILDREN(oid), OID_AUTO, "hwrm_min_ver", CTLTYPE_STRING | CTLFLAG_RWTUN | CTLFLAG_MPSAFE, softc, 0, bnxt_hwrm_min_ver_sysctl, "A", diff --git a/sys/dev/bnxt/bnxt_en/if_bnxt.c b/sys/dev/bnxt/bnxt_en/if_bnxt.c index b48c28f002d8..80ca53db4782 100644 --- a/sys/dev/bnxt/bnxt_en/if_bnxt.c +++ b/sys/dev/bnxt/bnxt_en/if_bnxt.c @@ -2197,7 +2197,8 @@ static int bnxt_open(struct bnxt_softc *bp) } if (BNXT_CHIP_P5_PLUS(bp)) - bnxt_hwrm_reserve_pf_rings(bp); + bnxt_hwrm_reserve_rings(bp); + /* Get the current configuration of this function */ rc = bnxt_hwrm_func_qcfg(bp); if (rc) { @@ -2435,6 +2436,16 @@ static void bnxt_sp_task(struct work_struct *work) clear_bit(BNXT_STATE_IN_SP_TASK, &bp->state); } +int +bnxt_hwrm_reserve_rings(struct bnxt_softc *softc) +{ + if (BNXT_PF(softc)) + return bnxt_hwrm_reserve_pf_rings(softc); + + else + return bnxt_hwrm_reserve_vf_rings(softc); +} + /* Device setup and teardown */ static int bnxt_attach_pre(if_ctx_t ctx) @@ -2510,15 +2521,15 @@ bnxt_attach_pre(if_ctx_t ctx) goto ver_fail; } - /* Now perform a function reset */ - rc = bnxt_hwrm_func_reset(softc); - if ((softc->flags & BNXT_FLAG_SHORT_CMD) || softc->hwrm_max_ext_req_len > BNXT_HWRM_MAX_REQ_LEN) { rc = bnxt_alloc_hwrm_short_cmd_req(softc); if (rc) goto hwrm_short_cmd_alloc_fail; } + + /* Now perform a function reset */ + rc = bnxt_hwrm_func_reset(softc); if ((softc->ver_info->chip_num == BCM57508) || (softc->ver_info->chip_num == BCM57504) || @@ -2670,8 +2681,10 @@ bnxt_attach_pre(if_ctx_t ctx) /* Get the queue config */ bnxt_get_wol_settings(softc); + if (BNXT_CHIP_P5_PLUS(softc)) - bnxt_hwrm_reserve_pf_rings(softc); + bnxt_hwrm_reserve_rings(softc); + rc = bnxt_hwrm_func_qcfg(softc); if (rc) { device_printf(softc->dev, "attach: hwrm func qcfg failed\n");