From nobody Tue Apr 28 06:45:03 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4g4WC50z2cz6bR8F for ; Tue, 28 Apr 2026 06:45:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R13" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4g4WC40QLWz3ZSM for ; Tue, 28 Apr 2026 06:45:04 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1777358704; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=TfFWKTl0Oi6zbsV9D0Ko97afxhrStHBwDviccmldZMM=; b=gqxoMK0MQGKdKIBC+kZ9xVO275UmZepdcxeQsnIz38HWiZHW4MTXWPiXCIxx8pHIjAm8Jk V3jBpsTgYA6sgiFQ7LCAZi2EDzArQQPgR49ckRdoDUIx7Yhy6CYHaS5HogD45CMES13yxm PcLBRl2M/o59pa38AE8gyd66YvYAbbYGpo8mfl8Q5kCqcCfgf7AQ2EaIK14nAgZmn2mmmw kGn65mCvXFf/jGOauV2Z6bSHUBZZeNoKJDRZupG/8QOmZpWD6wpvzOuw4p8BhQ73xnt81q 3r20WGK6MKCoHxUFzfODUgto7a1ekHsvOiSdbR6SI0LIEfXsu7TM5jKXsLVO+Q== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1777358704; a=rsa-sha256; cv=none; b=E7ng7V4KmCPbLb+8NH1wADgQ7eXmUR6tp/sLMWVUXVw6U02gOMLszEPh4wnCBWN6p5jrRr BzzM0kNzJMMjXISzVBirqIR0xSA/im0pm810ztIUwkNTGSyD6KLTY/ha3JYqBByZVNDq6d xFjR2wslNc17hxBmcwh8jfa0EVulFKNeHF0IjOSIowDtEUmcJ8QIo1//Wl7pCMRuAAav3w YTBvVZ5MRyjBmEZ/+N5Y1MVewBHe40WwFH8pPgnpqOs8+f0Tg37Y67K4AXg8TOjw1IPwSY 1Y4sKCTMUtjv4RZ9mYN35FkxidY4XbvsgIYWZLH9jvJ3bNTIGRJY04OtFdDdlQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1777358704; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=TfFWKTl0Oi6zbsV9D0Ko97afxhrStHBwDviccmldZMM=; b=LSN/9MiM5uBVfOFsAIziefah/famNCrDqFeyifQTcaTpf9ckglplKzDkwqtUCCNM/Mfclz L4E+Z62KPZrd7FAfWPxj8ctYwbQ8dJLCn1aXx5mLL+QV7/lYsJcZRGbotEdobO++1v+gU6 pv3cBty9u1EO0y2rIciD573jIxGD+JZ5HUYIcr2F2iVMszZBGd86u/YhePUdsHKANK5GWc 3QZL8h42jQDlK8/R0LUBgzR39xBD51DqNDHxISO+EPbW06g536EMe/GmcUmyVvzApwRzsp 4sWpwh3nf8xGCQ5PGBeLCAnZNfdZiM2A4b7lfBh4sZ5K56b14JtIwd3SxjL5Cw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4g4WC350CGzhqm for ; Tue, 28 Apr 2026 06:45:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 275f8 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Tue, 28 Apr 2026 06:45:03 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Cc: Chandrakanth Patil From: Sumit Saxena Subject: git: c972c5acbac4 - main - bnxt_en: VF ring reservation, HWRM registration, and PF-only operation guards List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: ssaxena X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: c972c5acbac472a5dc797856f39f478862b6c6ea Auto-Submitted: auto-generated Date: Tue, 28 Apr 2026 06:45:03 +0000 Message-Id: <69f0576f.275f8.102c997@gitrepo.freebsd.org> The branch main has been updated by ssaxena: URL: https://cgit.FreeBSD.org/src/commit/?id=c972c5acbac472a5dc797856f39f478862b6c6ea commit c972c5acbac472a5dc797856f39f478862b6c6ea Author: Chandrakanth Patil AuthorDate: 2026-04-02 23:04:45 +0000 Commit: Sumit Saxena CommitDate: 2026-04-28 06:35:09 +0000 bnxt_en: VF ring reservation, HWRM registration, and PF-only operation guards VFs require separate HWRM commands for ring reservation and async completion ring setup, so a common PF/VF dispatcher is introduced and the async CR path is extended to handle both. The PF must populate the VF request forwarding bitmap during driver registration so the firmware correctly forwards VF-originated HWRM commands. VF reservation strategy and min-guaranteed capability flags are now parsed for correct resource partitioning, and PF-only operations (DCB, NVM, package version sysctl) are guarded against VF invocation. The short command buffer allocation is also reordered before the function reset to ensure extended HWRM messages are available when needed, a prerequisite uncovered during VF bring-up. MFC after: 1 month Reviewed by: ssaxena Differential Revision: https://reviews.freebsd.org/D56232 --- sys/dev/bnxt/bnxt_en/bnxt.h | 1 + sys/dev/bnxt/bnxt_en/bnxt_dcb.c | 3 ++ sys/dev/bnxt/bnxt_en/bnxt_hwrm.c | 105 ++++++++++++++++++++++++++++++++----- sys/dev/bnxt/bnxt_en/bnxt_hwrm.h | 1 + sys/dev/bnxt/bnxt_en/bnxt_sysctl.c | 9 ++-- sys/dev/bnxt/bnxt_en/if_bnxt.c | 23 ++++++-- 6 files changed, 121 insertions(+), 21 deletions(-) diff --git a/sys/dev/bnxt/bnxt_en/bnxt.h b/sys/dev/bnxt/bnxt_en/bnxt.h index 4172e5e86b4f..6d00cd4d601d 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt.h +++ b/sys/dev/bnxt/bnxt_en/bnxt.h @@ -1398,5 +1398,6 @@ int bnxt_dcb_ieee_delapp(struct bnxt_softc *softc, struct bnxt_dcb_app *app); int bnxt_dcb_ieee_listapp(struct bnxt_softc *softc, struct bnxt_dcb_app *app, size_t nitems, int *num_inputs); void bnxt_set_flags_by_devid(struct bnxt_softc *softc); +int bnxt_hwrm_reserve_rings(struct bnxt_softc *softc); #endif /* _BNXT_H */ diff --git a/sys/dev/bnxt/bnxt_en/bnxt_dcb.c b/sys/dev/bnxt/bnxt_en/bnxt_dcb.c index e0643f200021..d169be6e5c7b 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_dcb.c +++ b/sys/dev/bnxt/bnxt_en/bnxt_dcb.c @@ -326,6 +326,9 @@ bnxt_hwrm_get_dcbx_app(struct bnxt_softc *softc, struct bnxt_dcb_app *app, if (softc->hwrm_spec_code < 0x10601) return 0; + if (BNXT_VF(softc)) + return 0; + bnxt_hwrm_cmd_hdr_init(softc, &get, HWRM_FW_GET_STRUCTURED_DATA); n = BNXT_IEEE_8021QAZ_MAX_TCS; diff --git a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c index 944824a1c182..b89a44e4677b 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c +++ b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.c @@ -79,6 +79,14 @@ long bnxt_rx_pkts_pri_arr_base_off[] = {BNXT_RX_STATS_PRI_ENTRIES(rx_packets)}; long bnxt_tx_bytes_pri_arr_base_off[] = {BNXT_TX_STATS_PRI_ENTRIES(tx_bytes)}; long bnxt_tx_pkts_pri_arr_base_off[] = {BNXT_TX_STATS_PRI_ENTRIES(tx_packets)}; +uint16_t bnxt_vf_req_snif[] = { + HWRM_FUNC_CFG, + HWRM_FUNC_VF_CFG, + HWRM_PORT_PHY_QCFG, + HWRM_CFA_L2_FILTER_ALLOC, + HWRM_OEM_CMD, +}; + static int bnxt_hwrm_err_map(uint16_t err) { @@ -757,9 +765,21 @@ int bnxt_hwrm_func_resc_qcaps(struct bnxt_softc *softc, bool all) hw_resc->max_stat_ctxs = le16toh(resp->max_stat_ctx); if (BNXT_CHIP_P5_PLUS(softc)) { - hw_resc->max_nqs = le16toh(resp->max_msix); + hw_resc->max_nqs = hw_resc->max_irqs = le16toh(resp->max_msix); hw_resc->max_hw_ring_grps = hw_resc->max_rx_rings; } + + if (BNXT_PF(softc)) { + struct bnxt_pf_info *pf = &softc->pf; + + pf->vf_resv_strategy = le16toh(resp->vf_reservation_strategy); + if (pf->vf_resv_strategy > BNXT_VF_RESV_STRATEGY_MINIMAL_STATIC) + pf->vf_resv_strategy = BNXT_VF_RESV_STRATEGY_MAXIMAL; + + if (resp->flags & + htole16(FUNC_RESOURCE_QCAPS_RESP_FLAGS_MIN_GUARANTEED)) + softc->fw_cap |= BNXT_FW_CAP_VF_RES_MIN_GUARANTEED; + } hwrm_func_resc_qcaps_exit: BNXT_HWRM_UNLOCK(softc); @@ -1054,6 +1074,26 @@ int bnxt_hwrm_func_drv_rgtr(struct bnxt_softc *bp, unsigned long *bmap, int bmap req.os_type = htole16(HWRM_FUNC_DRV_RGTR_INPUT_OS_TYPE_FREEBSD); if (BNXT_PF(bp)) { + u32 data[8]; + int i; + + memset(data, 0, sizeof(data)); + for (i = 0; i < ARRAY_SIZE(bnxt_vf_req_snif); i++) { + u16 cmd = bnxt_vf_req_snif[i]; + unsigned int bit, idx; + + if ((bp->fw_cap & BNXT_FW_CAP_LINK_ADMIN) && + (cmd == HWRM_PORT_PHY_QCFG)) + continue; + + idx = cmd / 32; + bit = cmd % 32; + data[idx] |= 1 << bit; + } + + for (i = 0; i < 8; i++) + req.vf_req_fwd[i] = cpu_to_le32(data[i]); + req.enables |= htole32(HWRM_FUNC_DRV_RGTR_INPUT_ENABLES_VF_REQ_FWD); } @@ -2299,25 +2339,63 @@ bnxt_hwrm_reserve_pf_rings(struct bnxt_softc *softc) return hwrm_send_message(softc, &req, sizeof(req)); } +#define BNXT_VF_MAX_L2_CTX 4 +int bnxt_hwrm_reserve_vf_rings(struct bnxt_softc *softc) +{ + struct hwrm_func_vf_cfg_input req = {0}; + + bnxt_hwrm_cmd_hdr_init(softc, &req, HWRM_FUNC_VF_CFG); + + req.enables |= htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_NUM_RSSCOS_CTXS); + req.enables |= htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_NUM_CMPL_RINGS); + req.enables |= htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_NUM_TX_RINGS); + req.enables |= htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_NUM_RX_RINGS); + req.enables |= htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_NUM_VNICS); + req.enables |= htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_NUM_STAT_CTXS); + req.enables |= htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_NUM_L2_CTXS); + req.num_rsscos_ctxs = htole16(0x8); + req.num_cmpl_rings = htole16(BNXT_MAX_NUM_QUEUES * 2); + req.num_tx_rings = htole16(BNXT_MAX_NUM_QUEUES); + req.num_rx_rings = htole16(BNXT_MAX_NUM_QUEUES); + req.num_vnics = htole16(BNXT_MAX_NUM_QUEUES); + req.num_stat_ctxs = htole16(BNXT_MAX_NUM_QUEUES * 2); + req.num_l2_ctxs = htole16(BNXT_VF_MAX_L2_CTX); + + return hwrm_send_message(softc, &req, sizeof(req)); +} + int bnxt_cfg_async_cr(struct bnxt_softc *softc) { int rc = 0; - struct hwrm_func_cfg_input req = {0}; - if (!BNXT_PF(softc)) - return 0; + if (BNXT_PF(softc)) { + struct hwrm_func_cfg_input req = {0}; - bnxt_hwrm_cmd_hdr_init(softc, &req, HWRM_FUNC_CFG); + bnxt_hwrm_cmd_hdr_init(softc, &req, HWRM_FUNC_CFG); - req.fid = htole16(0xffff); - req.enables = htole32(HWRM_FUNC_CFG_INPUT_ENABLES_ASYNC_EVENT_CR); - if (BNXT_CHIP_P5_PLUS(softc)) - req.async_event_cr = htole16(softc->nq_rings[0].ring.phys_id); - else - req.async_event_cr = htole16(softc->def_cp_ring.ring.phys_id); + req.fid = htole16(0xffff); + req.enables = htole32(HWRM_FUNC_CFG_INPUT_ENABLES_ASYNC_EVENT_CR); + if (BNXT_CHIP_P5_PLUS(softc)) + req.async_event_cr = htole16(softc->nq_rings[0].ring.phys_id); + else + req.async_event_cr = htole16(softc->def_cp_ring.ring.phys_id); - rc = hwrm_send_message(softc, &req, sizeof(req)); + rc = hwrm_send_message(softc, &req, sizeof(req)); + } else { + /* VF needs to configure async event completion ring using HWRM_FUNC_VF_CFG */ + struct hwrm_func_vf_cfg_input req = {0}; + + bnxt_hwrm_cmd_hdr_init(softc, &req, HWRM_FUNC_VF_CFG); + + req.enables = htole32(HWRM_FUNC_VF_CFG_INPUT_ENABLES_ASYNC_EVENT_CR); + if (BNXT_CHIP_P5_PLUS(softc)) + req.async_event_cr = htole16(softc->nq_rings[0].ring.phys_id); + else + req.async_event_cr = htole16(softc->def_cp_ring.ring.phys_id); + + rc = hwrm_send_message(softc, &req, sizeof(req)); + } return rc; } @@ -2391,6 +2469,9 @@ bnxt_hwrm_nvm_find_dir_entry(struct bnxt_softc *softc, uint16_t type, (void *)softc->hwrm_cmd_resp.idi_vaddr; int rc = 0; uint32_t old_timeo; + + if (BNXT_VF(softc)) + return 0; MPASS(ordinal); diff --git a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.h b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.h index 126cad977c82..b10981d9d53f 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_hwrm.h +++ b/sys/dev/bnxt/bnxt_en/bnxt_hwrm.h @@ -141,4 +141,5 @@ int bnxt_hwrm_passthrough(struct bnxt_softc *softc, void *req, uint32_t req_len, int _hwrm_send_message(struct bnxt_softc *, void *, uint32_t); int hwrm_send_message(struct bnxt_softc *, void *, uint32_t); void bnxt_hwrm_cmd_hdr_init(struct bnxt_softc *, void *, uint16_t); +int bnxt_hwrm_reserve_vf_rings (struct bnxt_softc *softc); #endif diff --git a/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c b/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c index 51438e657546..741cba4588aa 100644 --- a/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c +++ b/sys/dev/bnxt/bnxt_en/bnxt_sysctl.c @@ -1219,10 +1219,11 @@ bnxt_create_ver_sysctls(struct bnxt_softc *softc) "chip_type", CTLFLAG_RD, vi->chip_type > MAX_CHIP_TYPE ? bnxt_chip_type[MAX_CHIP_TYPE] : bnxt_chip_type[vi->chip_type], 0, "RoCE firmware name"); - SYSCTL_ADD_PROC(&vi->ver_ctx, SYSCTL_CHILDREN(oid), OID_AUTO, - "package_ver", CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, - softc, 0, bnxt_package_ver_sysctl, "A", - "currently installed package version"); + if (!BNXT_VF(softc)) + SYSCTL_ADD_PROC(&vi->ver_ctx, SYSCTL_CHILDREN(oid), OID_AUTO, + "package_ver", CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, + softc, 0, bnxt_package_ver_sysctl, "A", + "currently installed package version"); SYSCTL_ADD_PROC(&vi->ver_ctx, SYSCTL_CHILDREN(oid), OID_AUTO, "hwrm_min_ver", CTLTYPE_STRING | CTLFLAG_RWTUN | CTLFLAG_MPSAFE, softc, 0, bnxt_hwrm_min_ver_sysctl, "A", diff --git a/sys/dev/bnxt/bnxt_en/if_bnxt.c b/sys/dev/bnxt/bnxt_en/if_bnxt.c index b48c28f002d8..80ca53db4782 100644 --- a/sys/dev/bnxt/bnxt_en/if_bnxt.c +++ b/sys/dev/bnxt/bnxt_en/if_bnxt.c @@ -2197,7 +2197,8 @@ static int bnxt_open(struct bnxt_softc *bp) } if (BNXT_CHIP_P5_PLUS(bp)) - bnxt_hwrm_reserve_pf_rings(bp); + bnxt_hwrm_reserve_rings(bp); + /* Get the current configuration of this function */ rc = bnxt_hwrm_func_qcfg(bp); if (rc) { @@ -2435,6 +2436,16 @@ static void bnxt_sp_task(struct work_struct *work) clear_bit(BNXT_STATE_IN_SP_TASK, &bp->state); } +int +bnxt_hwrm_reserve_rings(struct bnxt_softc *softc) +{ + if (BNXT_PF(softc)) + return bnxt_hwrm_reserve_pf_rings(softc); + + else + return bnxt_hwrm_reserve_vf_rings(softc); +} + /* Device setup and teardown */ static int bnxt_attach_pre(if_ctx_t ctx) @@ -2510,15 +2521,15 @@ bnxt_attach_pre(if_ctx_t ctx) goto ver_fail; } - /* Now perform a function reset */ - rc = bnxt_hwrm_func_reset(softc); - if ((softc->flags & BNXT_FLAG_SHORT_CMD) || softc->hwrm_max_ext_req_len > BNXT_HWRM_MAX_REQ_LEN) { rc = bnxt_alloc_hwrm_short_cmd_req(softc); if (rc) goto hwrm_short_cmd_alloc_fail; } + + /* Now perform a function reset */ + rc = bnxt_hwrm_func_reset(softc); if ((softc->ver_info->chip_num == BCM57508) || (softc->ver_info->chip_num == BCM57504) || @@ -2670,8 +2681,10 @@ bnxt_attach_pre(if_ctx_t ctx) /* Get the queue config */ bnxt_get_wol_settings(softc); + if (BNXT_CHIP_P5_PLUS(softc)) - bnxt_hwrm_reserve_pf_rings(softc); + bnxt_hwrm_reserve_rings(softc); + rc = bnxt_hwrm_func_qcfg(softc); if (rc) { device_printf(softc->dev, "attach: hwrm func qcfg failed\n");