Date: Fri, 07 Aug 2026 12:45:49 +0000 From: Kevin Bowling <kbowling@FreeBSD.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: git: dda6a00a5202 - main - ixgbe: recover from X550 malicious-driver events Message-ID: <6a75d37d.3aae5.54e1f796@gitrepo.freebsd.org>
index | next in thread | raw e-mail
The branch main has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=dda6a00a5202154b4f83925b8e08bdb03228d4ce commit dda6a00a5202154b4f83925b8e08bdb03228d4ce Author: Kevin Bowling <kbowling@FreeBSD.org> AuthorDate: 2026-07-31 13:16:36 +0000 Commit: Kevin Bowling <kbowling@FreeBSD.org> CommitDate: 2026-08-07 12:45:43 +0000 ixgbe: recover from X550 malicious-driver events The shared X550 code provides malicious-driver detection, event decoding, and per-pool recovery operations, but the PF never enables or services them. A malformed VF descriptor can therefore go undetected and avoid the per-pool recovery path supplied by the MAC. Configure IOV state while VF DMA remains disabled, then enable MDD and activate the VFs only after PF queue initialization is complete. On an MDD event, withdraw mailbox CTS and gate the VF pool through PFVFTE and PFVFRE. Retain the per-queue WQBR blocks until the VF enters a new reset epoch; PFVFTE can still permit descriptor fetches into the internal queue, so releasing WQBR early would allow a hostile VF to retrigger MDD before it resets. Send the non-CTS reset notification after servicing the VF mailbox. Let a posted VF request win mailbox arbitration, defer notification if the pass produced a response, and retry failed notifications from the periodic admin pass. Poll WQBR so recovery does not depend on another mailbox interrupt edge, while suppressing already-fenced pools. Latch a PF reset request until the next hardware initialization. The X550 datasheet defines every bit of WQBR_RX and WQBR_TX as a queue bit, so an all-ones value is valid. Reject it only when IXGBE_STATUS, which has reserved-zero bits, also reads as all ones and confirms dead MMIO. Temporarily disable MDD around live multiqueue SRRCTL drop-mode updates, which hardware otherwise reports as queue-context changes. Serialize that window with the iflib context lock and resample pending work after MDD is restored. Apply the per-pool recovery model used by igb(4) in a2ed165f0049 to the existing DPDK-derived X550 hooks. The same register interface is documented for X552 and X553, so cover the entire X550 family. Document that VF traffic remains disabled until the reset handshake completes. MFC after: 2 weeks Relnotes: yes --- share/man/man4/ix.4 | 9 ++- sys/dev/ixgbe/if_ix.c | 50 ++++++++++-- sys/dev/ixgbe/if_sriov.c | 190 ++++++++++++++++++++++++++++++++++++++++---- sys/dev/ixgbe/ixgbe.h | 3 + sys/dev/ixgbe/ixgbe_mbx.c | 21 ++++- sys/dev/ixgbe/ixgbe_sriov.h | 22 +++-- sys/dev/ixgbe/ixgbe_x550.c | 22 +++-- 7 files changed, 279 insertions(+), 38 deletions(-) diff --git a/share/man/man4/ix.4 b/share/man/man4/ix.4 index 62fe4e1f9287..8f2bb107add5 100644 --- a/share/man/man4/ix.4 +++ b/share/man/man4/ix.4 @@ -29,7 +29,7 @@ .\" .\" * Other names and brands may be claimed as the property of others. .\" -.Dd August 6, 2026 +.Dd August 7, 2026 .Dt IX 4 .Os .Sh NAME @@ -101,6 +101,13 @@ configuration. The hardware MAC anti-spoofing control can validate only the VF's primary source address, so the driver does not silently weaken an enabled policy to install secondary addresses. +.Pp +Ethertype anti-spoofing and malicious-driver detection and recovery are +available only on X550-family devices. +The driver cannot enable those hardware protections on earlier devices such +as 82599. +After a malicious-driver event, the PF keeps the offending VF's transmit and +receive paths disabled until the VF completes a new reset handshake. .Sh HARDWARE The .Nm diff --git a/sys/dev/ixgbe/if_ix.c b/sys/dev/ixgbe/if_ix.c index d18f60984b8f..4606e735b821 100644 --- a/sys/dev/ixgbe/if_ix.c +++ b/sys/dev/ixgbe/if_ix.c @@ -4080,6 +4080,11 @@ ixgbe_if_init(if_ctx_t ctx) /* Setup DMA Coalescing */ ixgbe_config_dmac(sc); + if (sc->feat_en & IXGBE_FEATURE_SRIOV) { + ixgbe_enable_mdd(hw); + ixgbe_activate_vfs(sc); + } + /* And now turn on interrupts */ ixgbe_if_enable_intr(ctx); @@ -4724,6 +4729,8 @@ ixgbe_if_stop(if_ctx_t ctx) INIT_DEBUGOUT("ixgbe_if_stop: begin\n"); + if (sc->feat_en & IXGBE_FEATURE_SRIOV) + ixgbe_disable_mdd(hw); ixgbe_reset_hw(hw); hw->adapter_stopped = false; ixgbe_stop_adapter(hw); @@ -5156,6 +5163,7 @@ static int ixgbe_sysctl_flowcntl(SYSCTL_HANDLER_ARGS) { struct ixgbe_softc *sc; + struct sx *ctx_lock; int error, fc; sc = (struct ixgbe_softc *)arg1; @@ -5165,11 +5173,15 @@ ixgbe_sysctl_flowcntl(SYSCTL_HANDLER_ARGS) if ((error) || (req->newptr == NULL)) return (error); - /* Don't bother if it's not changed */ + /* Serialize the live register update with the administrative task. */ + ctx_lock = iflib_ctx_lock_get(sc->ctx); + sx_xlock(ctx_lock); if (fc == sc->hw.fc.current_mode) - return (0); - - return ixgbe_set_flowcntl(sc, fc); + error = 0; + else + error = ixgbe_set_flowcntl(sc, fc); + sx_xunlock(ctx_lock); + return (error); } /* ixgbe_sysctl_flowcntl */ /************************************************************************ @@ -5184,21 +5196,43 @@ ixgbe_sysctl_flowcntl(SYSCTL_HANDLER_ARGS) static int ixgbe_set_flowcntl(struct ixgbe_softc *sc, int fc) { + bool enable_drop, mdd_active; + switch (fc) { case ixgbe_fc_rx_pause: case ixgbe_fc_tx_pause: case ixgbe_fc_full: - if (sc->num_rx_queues > 1) - ixgbe_disable_rx_drop(sc); + enable_drop = false; break; case ixgbe_fc_none: - if (sc->num_rx_queues > 1) - ixgbe_enable_rx_drop(sc); + enable_drop = true; break; default: return (EINVAL); } + /* Updating SRRCTL on a live queue is itself an MDD violation. */ + mdd_active = sc->num_rx_queues > 1 && + (sc->feat_en & IXGBE_FEATURE_SRIOV) != 0 && + (if_getdrvflags(iflib_get_ifp(sc->ctx)) & IFF_DRV_RUNNING) != 0; + if (mdd_active) + ixgbe_disable_mdd(&sc->hw); + if (sc->num_rx_queues > 1) { + if (enable_drop) + ixgbe_enable_rx_drop(sc); + else + ixgbe_disable_rx_drop(sc); + } + if (mdd_active) { + ixgbe_enable_mdd(&sc->hw); + /* Service an event whose interrupt edge was lost while masked. */ + if (ixgbe_mbx_pending(sc)) { + atomic_set_32(&sc->task_requests, + IXGBE_REQUEST_TASK_MBX); + iflib_admin_intr_deferred(sc->ctx); + } + } + sc->hw.fc.requested_mode = fc; /* Don't autoneg if forcing a value */ diff --git a/sys/dev/ixgbe/if_sriov.c b/sys/dev/ixgbe/if_sriov.c index 848a53df290c..98c31b3d7a17 100644 --- a/sys/dev/ixgbe/if_sriov.c +++ b/sys/dev/ixgbe/if_sriov.c @@ -43,6 +43,8 @@ MALLOC_DEFINE(M_IXGBE_SRIOV, "ix_sriov", "ix SR-IOV allocations"); #define IXGBE_VF_MBX_CLEANUP_GRACE (2 * SBT_1S) +static const struct timeval ixgbe_mdd_log_interval = { 2, 0 }; + /************************************************************************ * ixgbe_define_iov_schemas ************************************************************************/ @@ -264,7 +266,9 @@ ixgbe_ping_all_vfs(struct ixgbe_softc *sc) for (int i = 0; i < sc->num_vfs; i++) { vf = &sc->vfs[i]; - if (vf->flags & IXGBE_VF_ACTIVE) + if ((vf->flags & + (IXGBE_VF_ACTIVE | IXGBE_VF_TRAFFIC_DISABLED)) == + IXGBE_VF_ACTIVE) ixgbe_send_vf_msg(&sc->hw, vf, IXGBE_PF_CONTROL_MSG); } } /* ixgbe_ping_all_vfs */ @@ -556,6 +560,11 @@ ixgbe_process_vf_reset(struct ixgbe_softc *sc, struct ixgbe_vf *vf) IXGBE_PVFTDWBALn(queue_count, vf->pool, i), 0); } + /* VFLR also leaves malicious-driver queue blocks asserted. */ + ixgbe_restore_mdd_vf(hw, vf->pool); + + vf->flags &= ~(IXGBE_VF_INIT_DONE | IXGBE_VF_MDD_BLOCKED | + IXGBE_VF_MDD_NOTIFY_PENDING); vf->api_ver = IXGBE_API_VER_UNKNOWN; } /* ixgbe_process_vf_reset */ @@ -570,7 +579,10 @@ ixgbe_vf_enable_transmit(struct ixgbe_softc *sc, struct ixgbe_vf *vf) vf_index = IXGBE_VF_INDEX(vf->pool); vfte = IXGBE_READ_REG(hw, IXGBE_VFTE(vf_index)); - vfte |= IXGBE_VF_BIT(vf->pool); + if (vf->flags & IXGBE_VF_TRAFFIC_DISABLED) + vfte &= ~IXGBE_VF_BIT(vf->pool); + else + vfte |= IXGBE_VF_BIT(vf->pool); IXGBE_WRITE_REG(hw, IXGBE_VFTE(vf_index), vfte); } /* ixgbe_vf_enable_transmit */ @@ -585,7 +597,8 @@ ixgbe_vf_enable_receive(struct ixgbe_softc *sc, struct ixgbe_vf *vf) vf_index = IXGBE_VF_INDEX(vf->pool); vfre = IXGBE_READ_REG(hw, IXGBE_VFRE(vf_index)); - if (ixgbe_vf_frame_size_compatible(sc, vf)) + if (!(vf->flags & IXGBE_VF_TRAFFIC_DISABLED) && + ixgbe_vf_frame_size_compatible(sc, vf)) vfre |= IXGBE_VF_BIT(vf->pool); else vfre &= ~IXGBE_VF_BIT(vf->pool); @@ -614,7 +627,7 @@ ixgbe_vf_reset_msg(struct ixgbe_softc *sc, struct ixgbe_vf *vf, uint32_t *msg) ixgbe_vf_enable_transmit(sc, vf); ixgbe_vf_enable_receive(sc, vf); - vf->flags |= IXGBE_VF_CTS; + vf->flags |= IXGBE_VF_CTS | IXGBE_VF_INIT_DONE; resp[0] = IXGBE_VF_RESET | ack; bcopy(vf->ether_addr, &resp[1], ETHER_ADDR_LEN); @@ -1038,6 +1051,104 @@ ixgbe_cleanup_vf_mbx(struct ixgbe_softc *sc, struct ixgbe_vf *vf) vf->mbx_cleanup_deadline = now + IXGBE_VF_MBX_CLEANUP_GRACE; } +static void +ixgbe_notify_vf_mdd_reset(struct ixgbe_softc *sc, struct ixgbe_vf *vf) +{ + u32 msg; + int error; + + msg = IXGBE_PF_CONTROL_MSG | IXGBE_VT_MSGTYPE_FAILURE; + error = ixgbe_write_mbx(&sc->hw, &msg, 1, vf->pool); + if (error == IXGBE_SUCCESS) { + vf->flags &= ~IXGBE_VF_MDD_NOTIFY_PENDING; + return; + } + + /* The periodic admin pass retries after any mailbox collision. */ + if (ratecheck(&vf->last_mdd_log, &ixgbe_mdd_log_interval)) + device_printf(sc->dev, + "could not notify VF %u of malicious-driver reset: %d; " + "will retry\n", vf->pool, error); +} + +static void +ixgbe_handle_mdd(struct ixgbe_softc *sc) +{ + struct ixgbe_hw *hw; + struct ixgbe_vf *vf; + bool pf_reset; + u32 rx_cause, tx_cause; + u32 vf_bitmap[2] = {}; + int pool; + + hw = &sc->hw; + ixgbe_mdd_event(hw, vf_bitmap); + if (vf_bitmap[0] != 0 || vf_bitmap[1] != 0) { + tx_cause = IXGBE_READ_REG(hw, IXGBE_LVMMC_TX); + rx_cause = IXGBE_READ_REG(hw, IXGBE_LVMMC_RX); + } else { + tx_cause = 0; + rx_cause = 0; + } + pf_reset = false; + for (pool = 0; pool < 64; pool++) { + if ((vf_bitmap[pool / 32] & (1U << (pool % 32))) == 0) + continue; + + if (pool == sc->pool) { + if (sc->iov_pf_mdd_reset_pending) + continue; + sc->iov_pf_mdd_reset_pending = true; + pf_reset = true; + if (ratecheck(&sc->iov_last_mdd_log, + &ixgbe_mdd_log_interval)) + device_printf(sc->dev, + "malicious-driver event on PF pool " + "(last tx cause %#x, last rx cause %#x); " + "resetting PF\n", + tx_cause, rx_cause); + continue; + } + if (pool >= sc->num_vfs) { + ixgbe_restore_mdd_vf(hw, pool); + continue; + } + vf = &sc->vfs[pool]; + if (!(vf->flags & IXGBE_VF_ACTIVE)) { + ixgbe_restore_mdd_vf(hw, pool); + continue; + } + + if ((vf->flags & IXGBE_VF_MDD_BLOCKED) != 0) + continue; + if (ratecheck(&vf->last_mdd_log, &ixgbe_mdd_log_interval)) + device_printf(sc->dev, + "malicious-driver event from VF %u " + "(last tx cause %#x, last rx cause %#x); " + "requesting VF reset\n", + vf->pool, tx_cause, rx_cause); + + /* + * Keep both the pool gate and the per-queue WQBR block asserted. + * PFVFTE stops packet-data fetches but can still allow descriptor + * fetches into the internal queue, so releasing WQBR here would + * leave a hostile VF able to retrigger MDD before it resets. + * ixgbe_process_vf_reset() releases WQBR in the new reset epoch. + */ + vf->flags |= IXGBE_VF_MDD_BLOCKED; + vf->flags &= ~IXGBE_VF_CTS; + ixgbe_vf_enable_transmit(sc, vf); + ixgbe_vf_enable_receive(sc, vf); + IXGBE_WRITE_FLUSH(hw); + vf->flags |= IXGBE_VF_MDD_NOTIFY_PENDING; + } + + if (pf_reset) { + iflib_request_reset(sc->ctx); + iflib_admin_intr_deferred(sc->ctx); + } +} + /* Tasklet for handling VF -> PF mailbox messages */ void ixgbe_handle_mbx(void *context) @@ -1046,14 +1157,16 @@ ixgbe_handle_mbx(void *context) struct ixgbe_softc *sc = iflib_get_softc(ctx); struct ixgbe_hw *hw; struct ixgbe_vf *vf; - bool cleanup_pending, reset_pending, reset_seen; + bool cleanup_pending, mbx_activity, reset_pending, reset_seen; int i; hw = &sc->hw; cleanup_pending = false; + ixgbe_handle_mdd(sc); for (i = 0; i < sc->num_vfs; i++) { vf = &sc->vfs[i]; + mbx_activity = false; if (vf->flags & IXGBE_VF_ACTIVE) { reset_seen = hw->mbx.ops[vf->pool].check_for_rst(hw, @@ -1069,6 +1182,7 @@ ixgbe_handle_mbx(void *context) if (hw->mbx.ops[vf->pool].check_for_msg(hw, vf->pool) == 0) { + mbx_activity = true; if (ixgbe_process_vf_msg(ctx, vf, reset_pending)) vf->flags &= ~IXGBE_VF_MBX_CLEANUP; } @@ -1077,8 +1191,15 @@ ixgbe_handle_mbx(void *context) ixgbe_cleanup_vf_mbx(sc, vf); if (hw->mbx.ops[vf->pool].check_for_ack(hw, - vf->pool) == 0) + vf->pool) == 0) { + mbx_activity = true; ixgbe_process_vf_ack(sc, vf); + } + + /* Do not overwrite a response from this mailbox pass. */ + if (!mbx_activity && + (vf->flags & IXGBE_VF_MDD_NOTIFY_PENDING) != 0) + ixgbe_notify_vf_mdd_reset(sc, vf); if (vf->flags & IXGBE_VF_MBX_CLEANUP) cleanup_pending = true; @@ -1088,16 +1209,17 @@ ixgbe_handle_mbx(void *context) } /* ixgbe_handle_mbx */ /* - * VFREQ, VFACK, and a bare VFLR can remain latched without a usable shared - * mailbox interrupt across a PF stop/restart. Sample their aggregate - * registers from the periodic admin pass so work does not depend on another - * edge. + * VFREQ, VFACK, a bare VFLR, and WQBR can remain latched without a usable + * shared mailbox interrupt across a PF stop/restart or an MDD mask interval. + * Sample their aggregate registers from the periodic admin pass so work does + * not depend on another edge. Pending asynchronous MDD notifications use the + * same pass to retry after the VF wins a mailbox collision. */ bool ixgbe_mbx_pending(struct ixgbe_softc *sc) { struct ixgbe_hw *hw; - uint32_t active_mbx[4], active_rst[2], events; + uint32_t active_mbx[4], active_rst[2], events, vf_mdd[2]; int i, index; if (sc->num_vfs == 0) @@ -1105,9 +1227,12 @@ ixgbe_mbx_pending(struct ixgbe_softc *sc) bzero(active_mbx, sizeof(active_mbx)); bzero(active_rst, sizeof(active_rst)); + bzero(vf_mdd, sizeof(vf_mdd)); for (i = 0; i < sc->num_vfs; i++) { if ((sc->vfs[i].flags & IXGBE_VF_ACTIVE) == 0) continue; + if ((sc->vfs[i].flags & IXGBE_VF_MDD_NOTIFY_PENDING) != 0) + return (true); index = IXGBE_PFMBICR_INDEX(sc->vfs[i].pool); active_mbx[index] |= IXGBE_PFMBICR_VFREQ_VF1 << @@ -1146,6 +1271,16 @@ ixgbe_mbx_pending(struct ixgbe_softc *sc) if ((events & active_rst[index]) != 0) return (true); } + ixgbe_mdd_event(hw, vf_mdd); + /* A fenced VF retains WQBR until reset; do not reschedule for it. */ + for (i = 0; i < sc->num_vfs; i++) { + if ((sc->vfs[i].flags & IXGBE_VF_MDD_BLOCKED) != 0) + vf_mdd[i / 32] &= ~(1U << (i % 32)); + } + if (sc->iov_pf_mdd_reset_pending) + vf_mdd[sc->pool / 32] &= ~(1U << (sc->pool % 32)); + if (vf_mdd[0] != 0 || vf_mdd[1] != 0) + return (true); return (false); } @@ -1237,6 +1372,7 @@ ixgbe_if_iov_init(if_ctx_t ctx, u16 num_vfs, const nvlist_t *config) sc->num_vfs = num_vfs; sc->iov_mbx_cleanup_pending = false; + sc->iov_pf_mdd_reset_pending = false; ixgbe_init_mbx_params_pf(&sc->hw); sc->feat_en |= IXGBE_FEATURE_SRIOV; @@ -1253,6 +1389,7 @@ err_init_iov: sc->pool = 0; sc->iov_mode = IXGBE_NO_VM; sc->iov_mbx_cleanup_pending = false; + sc->iov_pf_mdd_reset_pending = false; return (retval); } /* ixgbe_if_iov_init */ @@ -1268,6 +1405,7 @@ ixgbe_if_iov_uninit(if_ctx_t ctx) sc = iflib_get_softc(ctx); hw = &sc->hw; + ixgbe_disable_mdd(hw); /* Enable rx/tx for the PF and disable it for all VFs. */ pf_reg = IXGBE_VF_INDEX(sc->pool); @@ -1329,6 +1467,7 @@ ixgbe_if_iov_uninit(if_ctx_t ctx) sc->iov_vfta_valid = false; sc->iov_vlan_promisc = false; sc->iov_mbx_cleanup_pending = false; + sc->iov_pf_mdd_reset_pending = false; (void)ixgbe_clear_vfta(hw); ixgbe_setup_vlan_hw_support(ctx); } /* ixgbe_if_iov_uninit */ @@ -1341,6 +1480,8 @@ ixgbe_init_vf(struct ixgbe_softc *sc, struct ixgbe_vf *vf) s32 error; hw = &sc->hw; + vf->flags &= ~(IXGBE_VF_INIT_DONE | IXGBE_VF_MDD_BLOCKED | + IXGBE_VF_MDD_NOTIFY_PENDING); if (!(vf->flags & IXGBE_VF_ACTIVE)) return (IXGBE_SUCCESS); @@ -1365,12 +1506,22 @@ ixgbe_init_vf(struct ixgbe_softc *sc, struct ixgbe_vf *vf) } ixgbe_vf_set_anti_spoof(sc, vf); + vf->flags |= IXGBE_VF_INIT_DONE; + return (IXGBE_SUCCESS); +} /* ixgbe_init_vf */ + +static void +ixgbe_activate_vf(struct ixgbe_softc *sc, struct ixgbe_vf *vf) +{ + if ((vf->flags & (IXGBE_VF_ACTIVE | IXGBE_VF_INIT_DONE)) != + (IXGBE_VF_ACTIVE | IXGBE_VF_INIT_DONE) || + (vf->flags & IXGBE_VF_TRAFFIC_DISABLED)) + return; + ixgbe_vf_enable_transmit(sc, vf); ixgbe_vf_enable_receive(sc, vf); - ixgbe_send_vf_msg(&sc->hw, vf, IXGBE_PF_CONTROL_MSG); - return (IXGBE_SUCCESS); -} /* ixgbe_init_vf */ +} /* ixgbe_activate_vf */ void ixgbe_initialize_iov(struct ixgbe_softc *sc) @@ -1381,6 +1532,7 @@ ixgbe_initialize_iov(struct ixgbe_softc *sc) if (sc->iov_mode == IXGBE_NO_VM) return; + sc->iov_pf_mdd_reset_pending = false; /* RMW appropriate registers based on IOV mode */ /* Read... */ @@ -1434,6 +1586,15 @@ ixgbe_initialize_iov(struct ixgbe_softc *sc) } } /* ixgbe_initialize_iov */ +void +ixgbe_activate_vfs(struct ixgbe_softc *sc) +{ + int i; + + for (i = 0; i < sc->num_vfs; i++) + ixgbe_activate_vf(sc, &sc->vfs[i]); +} /* ixgbe_activate_vfs */ + /* Check the max frame setting of all active VF's */ void @@ -1518,6 +1679,7 @@ ixgbe_if_iov_vf_add(if_ctx_t ctx, u16 vfnum, const nvlist_t *config) ixgbe_vf_clear_vlans(sc, vf, true); return (ENOSPC); } + ixgbe_activate_vf(sc, vf); return (0); } /* ixgbe_if_iov_vf_add */ diff --git a/sys/dev/ixgbe/ixgbe.h b/sys/dev/ixgbe/ixgbe.h index 20f8fce8f04b..9cefae3c5b06 100644 --- a/sys/dev/ixgbe/ixgbe.h +++ b/sys/dev/ixgbe/ixgbe.h @@ -357,6 +357,7 @@ struct ixgbe_vf { u_int rar_index; u_int maximum_frame_size; uint32_t flags; + struct timeval last_mdd_log; uint8_t ether_addr[ETHER_ADDR_LEN]; uint16_t mc_hash[IXGBE_MAX_VF_MC]; uint32_t vlans[IXGBE_VFTA_SIZE]; @@ -463,6 +464,8 @@ struct ixgbe_softc { bool iov_vfta_valid; bool iov_vlan_promisc; bool iov_mbx_cleanup_pending; + bool iov_pf_mdd_reset_pending; + struct timeval iov_last_mdd_log; /* Bypass */ struct ixgbe_bp_data bypass; diff --git a/sys/dev/ixgbe/ixgbe_mbx.c b/sys/dev/ixgbe/ixgbe_mbx.c index eefe4543f3fe..82cbe0a18eb3 100644 --- a/sys/dev/ixgbe/ixgbe_mbx.c +++ b/sys/dev/ixgbe/ixgbe_mbx.c @@ -964,8 +964,17 @@ static s32 ixgbe_write_mbx_pf_legacy(struct ixgbe_hw *hw, u32 *msg, u16 size, if (ret_val) return ret_val; + /* + * A VF request wins over an asynchronous PF message. Recheck VFREQ + * after acquiring PFU so the request and its payload remain intact for + * the normal mailbox path. + */ + if (ixgbe_check_for_msg_pf(hw, vf_id) == IXGBE_SUCCESS) { + ixgbe_release_mbx_lock_pf(hw, vf_id); + return IXGBE_ERR_MBX; + } + /* flush msg and acks as we are overwriting the message buffer */ - ixgbe_check_for_msg_pf(hw, vf_id); ixgbe_clear_msg_pf(hw, vf_id); ixgbe_check_for_ack_pf(hw, vf_id); ixgbe_clear_ack_pf(hw, vf_id); @@ -1006,6 +1015,16 @@ static s32 ixgbe_write_mbx_pf(struct ixgbe_hw *hw, u32 *msg, u16 size, if (ret_val) goto out; + /* + * A VF request wins over an asynchronous PF message. PFU now keeps + * PFMBMEM stable, so recheck VFREQ under ownership and leave the request + * intact for the normal mailbox path to consume. + */ + if (ixgbe_check_for_msg_pf(hw, vf_id) == IXGBE_SUCCESS) { + ret_val = IXGBE_ERR_MBX; + goto out; + } + /* flush msg and acks as we are overwriting the message buffer */ ixgbe_clear_msg_pf(hw, vf_id); ixgbe_clear_ack_pf(hw, vf_id); diff --git a/sys/dev/ixgbe/ixgbe_sriov.h b/sys/dev/ixgbe/ixgbe_sriov.h index 1ae54863befc..c78c51f02fc6 100644 --- a/sys/dev/ixgbe/ixgbe_sriov.h +++ b/sys/dev/ixgbe/ixgbe_sriov.h @@ -43,15 +43,19 @@ #include <net/iflib.h> #include "ixgbe_mbx.h" -#define IXGBE_VF_CTS (1 << 0) /* VF is clear to send. */ -#define IXGBE_VF_CAP_MAC (1 << 1) /* VF is permitted to change MAC. */ -#define IXGBE_VF_CAP_VLAN (1 << 2) /* VF is permitted to join vlans. */ -#define IXGBE_VF_ACTIVE (1 << 3) /* VF is active. */ -#define IXGBE_VF_ANTI_SPOOF (1 << 4) /* Enforce source identity. */ -#define IXGBE_VF_ALLOW_PROMISC (1 << 5) /* VF may request promiscuity. */ -#define IXGBE_VF_MBX_CLEANUP (1 << 6) /* Reset mailbox cleanup pending. */ +#define IXGBE_VF_CTS (1U << 0) /* VF is clear to send. */ +#define IXGBE_VF_CAP_MAC (1U << 1) /* VF is permitted to change MAC. */ +#define IXGBE_VF_CAP_VLAN (1U << 2) /* VF is permitted to join vlans. */ +#define IXGBE_VF_ACTIVE (1U << 3) /* VF is active. */ +#define IXGBE_VF_ANTI_SPOOF (1U << 4) /* Enforce source identity. */ +#define IXGBE_VF_ALLOW_PROMISC (1U << 5) /* VF may request promiscuity. */ +#define IXGBE_VF_MBX_CLEANUP (1U << 6) /* Reset cleanup pending. */ +#define IXGBE_VF_INIT_DONE (1U << 7) /* Hardware state is ready. */ +#define IXGBE_VF_MDD_BLOCKED (1U << 11) /* VF is gated after MDD. */ +#define IXGBE_VF_MDD_NOTIFY_PENDING (1U << 12) /* Retry reset notice. */ +#define IXGBE_VF_TRAFFIC_DISABLED IXGBE_VF_MDD_BLOCKED #define IXGBE_VF_INDEX(vmdq) ((vmdq) / 32) -#define IXGBE_VF_BIT(vmdq) (1 << ((vmdq) % 32)) +#define IXGBE_VF_BIT(vmdq) (1U << ((vmdq) % 32)) #define IXGBE_VT_MSG_MASK 0xFFFF @@ -78,6 +82,7 @@ int ixgbe_if_iov_init(if_ctx_t, u16, const nvlist_t *); int ixgbe_iov_validate(struct ixgbe_softc *, u16); void ixgbe_if_iov_uninit(if_ctx_t); void ixgbe_initialize_iov(struct ixgbe_softc *); +void ixgbe_activate_vfs(struct ixgbe_softc *); void ixgbe_recalculate_max_frame(struct ixgbe_softc *); void ixgbe_ping_all_vfs(struct ixgbe_softc *); u_int ixgbe_iov_rebuild_mta(struct ixgbe_softc *); @@ -95,6 +100,7 @@ u32 ixgbe_get_mrqc(int); #define ixgbe_init_iov(_a,_b,_c) #define ixgbe_uninit_iov(_a) #define ixgbe_initialize_iov(_a) +#define ixgbe_activate_vfs(_a) #define ixgbe_recalculate_max_frame(_a) #define ixgbe_ping_all_vfs(_a) #define ixgbe_define_iov_schemas(_a,_b) diff --git a/sys/dev/ixgbe/ixgbe_x550.c b/sys/dev/ixgbe/ixgbe_x550.c index f6ce0d10b9e2..2891bc258db8 100644 --- a/sys/dev/ixgbe/ixgbe_x550.c +++ b/sys/dev/ixgbe/ixgbe_x550.c @@ -1375,7 +1375,7 @@ void ixgbe_restore_mdd_vf_X550(struct ixgbe_hw *hw, u32 vf) **/ void ixgbe_mdd_event_X550(struct ixgbe_hw *hw, u32 *vf_bitmap) { - u32 wqbr; + u32 rx_wqbr, tx_wqbr, wqbr; u32 i, j, reg, q, shift, vf, idx; DEBUGFUNC("ixgbe_mdd_event_X550"); @@ -1397,8 +1397,18 @@ void ixgbe_mdd_event_X550(struct ixgbe_hw *hw, u32 *vf_bitmap) /* Read WQBR_TX and WQBR_RX and check for malicious queues */ for (i = 0; i < 4; i++) { - wqbr = IXGBE_READ_REG(hw, IXGBE_WQBR_TX(i)); - wqbr |= IXGBE_READ_REG(hw, IXGBE_WQBR_RX(i)); + tx_wqbr = IXGBE_READ_REG(hw, IXGBE_WQBR_TX(i)); + rx_wqbr = IXGBE_READ_REG(hw, IXGBE_WQBR_RX(i)); + /* + * Every WQBR bit represents a queue, so all ones is a valid + * sample. Use a register with reserved-zero bits to distinguish + * that value from failed MMIO on a removed device. + */ + if (__predict_false((tx_wqbr == UINT32_MAX || + rx_wqbr == UINT32_MAX) && + IXGBE_READ_REG(hw, IXGBE_STATUS) == UINT32_MAX)) + return; + wqbr = tx_wqbr | rx_wqbr; if (!wqbr) continue; @@ -1406,7 +1416,7 @@ void ixgbe_mdd_event_X550(struct ixgbe_hw *hw, u32 *vf_bitmap) /* Get malicious queue */ for (j = 0; j < 32 && wqbr; j++) { - if (!(wqbr & (1 << j))) + if (!(wqbr & (1U << j))) continue; /* Get queue from bitmask */ @@ -1417,8 +1427,8 @@ void ixgbe_mdd_event_X550(struct ixgbe_hw *hw, u32 *vf_bitmap) /* Set vf bit in vf_bitmap */ idx = vf / 32; - vf_bitmap[idx] |= (1 << (vf % 32)); - wqbr &= ~(1 << j); + vf_bitmap[idx] |= (1U << (vf % 32)); + wqbr &= ~(1U << j); } } }home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6a75d37d.3aae5.54e1f796>