From nobody Thu Aug 6 11:44:07 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hG56527c5z6mvF9 for ; Thu, 06 Aug 2026 11:44:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hG5651Lfdz3qtT for ; Thu, 06 Aug 2026 11:44:13 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786016653; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=j4S7NEskLdHwvQK+gPfLuaoVijhjy59YasXVAjrIR7c=; b=v3SnPpKNNAkV3at9tWl84JAVOufEaX3KjWPPAx58QzwPUKFevNFywk7eAJThexebXlXoSn +UKDXDpCuNUQkifP/FnjqW4xbntuBgrlUs4YoIgHiUfQj6MwBeJbjUdxuRynIOgcwpZliS kBLYxAXzHx1RgMrOKoxrgYE1EMCHDCEt1QFP9QQDNqSPatCj31yr4QmsUyoejs/N9znSeg mH14Qg8/Cr1OedCNr/gn4OMV8DDJQZGmbp4oBoBAyv17sTxIvtKXShsuxKZHFkTXlZrP9A GOasDG894WIv2RPW/dPr1zoEczBmhkN1/Mw8y3rqRHcbaQmmFw8xDJMOmPeJCg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1786016653; a=rsa-sha256; cv=none; b=DHWjb7oOZJxBtnod1WKdhTBIPA1L4zT+66Ls5a75Y21zGRRoJygC8yst9zHPZNwD5npuWz aWjYc+F8tjIrjFxGM5yhP3jan2fhi8f8yxnNTrvcjrdlwGY3oLgFEZVcxcYCrod7c5Sfpe vG6yzLicp14Tt9R2FTxBsfS5HLIZFf1u1Pr2sWYq2vO9Hu3OFJPOxXNTFTDdLM6YA47Wkh uLdC3OI7ruDSab/nRUJzHoao4DgLrdAsgm5Vb0gSGc3eBtwhOKKSSMbkt5xcpUmag4SFQr gBTwlRM/RHN9wMSwCLfScO8KA9bMnb2vqyz96u/fVP+67rrb6FCaVvF+f25Ghw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786016653; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=j4S7NEskLdHwvQK+gPfLuaoVijhjy59YasXVAjrIR7c=; b=iY/6BX1cw9PmC4nPdir7AzeBMH6x2MVU1/NV4sbhk2Isyc9Y79PGZcvshBKmeOjE41qrmU 5CDeBnRhfJrOsevkd3SeCsDxNH9isLK4FVX4eWkbPXcnOnDRyIfCOzvnf2YOlyP2Kd7EU9 UL4W6rOaMEZby6srKIFRnTu/Gg/PmfRomdffTgOhfiyWVMkIbBauMcfGc7BT4/vAtgdrVv IKoa1q2lxFS0x/OJhQTkWr2PQV8nv83KRVFQygISuPREb23VA5/BAYrXh3++Dr050W18Fj ITqi2aYDX0xBClC813VAuwi+tUTg9VVhd1KiT6gbCYmyH8LyJdjl6h6ubW+eZw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hG56505n5z409 for ; Thu, 06 Aug 2026 11:44:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 32308 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 06 Aug 2026 11:44:07 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Kevin Bowling Subject: git: 1a180b4c86fc - main - ixgbe: rebuild the shared multicast table List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 1a180b4c86fc7534596cfbe451e25e403db81666 Auto-Submitted: auto-generated Date: Thu, 06 Aug 2026 11:44:07 +0000 Message-Id: <6a747387.32308.644718c8@gitrepo.freebsd.org> The branch main has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=1a180b4c86fc7534596cfbe451e25e403db81666 commit 1a180b4c86fc7534596cfbe451e25e403db81666 Author: Kevin Bowling AuthorDate: 2026-07-31 11:53:17 +0000 Commit: Kevin Bowling CommitDate: 2026-08-06 11:42:18 +0000 ixgbe: rebuild the shared multicast table The MTA is shared by the PF and all VFs. The VF mailbox handler only ORs new bits, so hashes survive list removal and VF reset. Conversely, PF multicast updates replace the whole table with PF-only state and discard live VF filters. Rebuild the table from the PF list and every active VF whenever either changes. Clear VF multicast state during reset and PF reinitialization, and remove all VF hashes on SR-IOV teardown. Keep the software shadow and multicast control state synchronized, and avoid writes to unchanged MTA registers. Adapt the aggregate desired-state rebuild introduced for igb(4) in a2ed165f0049 and its write-elision scheme from 350211ab1782 to ixgbe's shared MTA. MFC after: 1 week --- sys/dev/ixgbe/if_ix.c | 85 ++++++++++++++++++++++++++++++++++++++++----- sys/dev/ixgbe/if_sriov.c | 39 +++++++++++---------- sys/dev/ixgbe/ixgbe.h | 1 + sys/dev/ixgbe/ixgbe_sriov.h | 1 + 4 files changed, 99 insertions(+), 27 deletions(-) diff --git a/sys/dev/ixgbe/if_ix.c b/sys/dev/ixgbe/if_ix.c index b7ca21d4a313..74444c439891 100644 --- a/sys/dev/ixgbe/if_ix.c +++ b/sys/dev/ixgbe/if_ix.c @@ -3838,6 +3838,7 @@ ixgbe_if_init(if_ctx_t ctx) hw->addr_ctrl.rar_used_count = 1; ixgbe_init_hw(hw); + sc->iov_mta_valid = false; ixgbe_initialize_iov(sc); @@ -4190,6 +4191,67 @@ ixgbe_mc_filter_apply(void *arg, struct sockaddr_dl *sdl, u_int idx) return (1); } /* ixgbe_mc_filter_apply */ +#ifdef PCI_IOV +/* + * The MTA is shared by the PF and every VF. Rebuild it from all owners + * because an individual bit cannot be cleared safely when hashes collide. + */ +u_int +ixgbe_iov_rebuild_mta(struct ixgbe_softc *sc) +{ + struct ixgbe_hw *hw; + struct ixgbe_mc_addr *mta; + struct ixgbe_vf *vf; + u32 old_mta[IXGBE_MAX_MTA]; + u32 hash; + u_int i, mcnt; + int vf_index; + + hw = &sc->hw; + mta = sc->mta; + bzero(mta, sizeof(*mta) * MAX_NUM_MULTICAST_ADDRESSES); + mcnt = if_foreach_llmaddr(iflib_get_ifp(sc->ctx), + ixgbe_mc_filter_apply, sc); + + bcopy(hw->mac.mta_shadow, old_mta, sizeof(old_mta)); + bzero(hw->mac.mta_shadow, sizeof(hw->mac.mta_shadow)); + hw->addr_ctrl.num_mc_addrs = mcnt; + hw->addr_ctrl.mta_in_use = 0; + + for (i = 0; i < mcnt; i++) + ixgbe_set_mta(hw, mta[i].addr); + + for (vf_index = 0; vf_index < sc->num_vfs; vf_index++) { + vf = &sc->vfs[vf_index]; + if (!(vf->flags & IXGBE_VF_ACTIVE)) + continue; + + for (i = 0; i < vf->num_mc_hashes; i++) { + hash = vf->mc_hash[i] & 0xfff; + hw->mac.mta_shadow[(hash >> 5) & + (hw->mac.mcft_size - 1)] |= + 1U << (hash & 0x1f); + hw->addr_ctrl.mta_in_use++; + } + } + + for (i = 0; i < hw->mac.mcft_size; i++) { + if (sc->iov_mta_valid && + old_mta[i] == hw->mac.mta_shadow[i]) + continue; + IXGBE_WRITE_REG_ARRAY(hw, IXGBE_MTA(0), i, + hw->mac.mta_shadow[i]); + } + sc->iov_mta_valid = true; + + IXGBE_WRITE_REG(hw, IXGBE_MCSTCTRL, + (hw->addr_ctrl.mta_in_use != 0 ? IXGBE_MCSTCTRL_MFE : 0) | + hw->mac.mc_filter_type); + + return (mcnt); +} +#endif + static void ixgbe_if_multi_set(if_ctx_t ctx) { @@ -4202,16 +4264,23 @@ ixgbe_if_multi_set(if_ctx_t ctx) IOCTL_DEBUGOUT("ixgbe_if_multi_set: begin"); - mta = sc->mta; - bzero(mta, sizeof(*mta) * MAX_NUM_MULTICAST_ADDRESSES); +#ifdef PCI_IOV + if (sc->feat_en & IXGBE_FEATURE_SRIOV) { + mcnt = ixgbe_iov_rebuild_mta(sc); + } else +#endif + { + mta = sc->mta; + bzero(mta, sizeof(*mta) * MAX_NUM_MULTICAST_ADDRESSES); - mcnt = if_foreach_llmaddr(iflib_get_ifp(ctx), ixgbe_mc_filter_apply, - sc); + mcnt = if_foreach_llmaddr(iflib_get_ifp(ctx), + ixgbe_mc_filter_apply, sc); - if (mcnt < MAX_NUM_MULTICAST_ADDRESSES) { - update_ptr = (u8 *)mta; - ixgbe_update_mc_addr_list(&sc->hw, update_ptr, mcnt, - ixgbe_mc_array_itr, true); + if (mcnt < MAX_NUM_MULTICAST_ADDRESSES) { + update_ptr = (u8 *)mta; + ixgbe_update_mc_addr_list(&sc->hw, update_ptr, mcnt, + ixgbe_mc_array_itr, true); + } } fctrl = IXGBE_READ_REG(&sc->hw, IXGBE_FCTRL); diff --git a/sys/dev/ixgbe/if_sriov.c b/sys/dev/ixgbe/if_sriov.c index b9e5205cf1b9..dd67680d9c7d 100644 --- a/sys/dev/ixgbe/if_sriov.c +++ b/sys/dev/ixgbe/if_sriov.c @@ -303,9 +303,15 @@ ixgbe_vf_frame_size_compatible(struct ixgbe_softc *sc, struct ixgbe_vf *vf) static void ixgbe_process_vf_reset(struct ixgbe_softc *sc, struct ixgbe_vf *vf) { + bool rebuild_mta; + ixgbe_vf_set_default_vlan(sc, vf, vf->default_vlan); - // XXX clear multicast addresses + rebuild_mta = vf->num_mc_hashes != 0; + vf->num_mc_hashes = 0; + bzero(vf->mc_hash, sizeof(vf->mc_hash)); + if (rebuild_mta) + ixgbe_iov_rebuild_mta(sc); ixgbe_clear_rar(&sc->hw, vf->rar_index); ixgbe_toggle_txdctl(&sc->hw, vf->pool); @@ -431,30 +437,24 @@ ixgbe_vf_set_mc_addr(struct ixgbe_softc *sc, struct ixgbe_vf *vf, u32 *msg) { u16 *list = (u16*)&msg[1]; int entries; - u32 vmolr, vec_bit, vec_reg, mta_reg; + u32 vmolr; entries = (msg[0] & IXGBE_VT_MSGINFO_MASK) >> IXGBE_VT_MSGINFO_SHIFT; entries = min(entries, IXGBE_MAX_VF_MC); vmolr = IXGBE_READ_REG(&sc->hw, IXGBE_VMOLR(vf->pool)); + vmolr &= ~IXGBE_VMOLR_ROMPE; + IXGBE_WRITE_REG(&sc->hw, IXGBE_VMOLR(vf->pool), vmolr); + bzero(vf->mc_hash, sizeof(vf->mc_hash)); + bcopy(list, vf->mc_hash, entries * sizeof(*list)); vf->num_mc_hashes = entries; + ixgbe_iov_rebuild_mta(sc); - /* Set the appropriate MTA bit */ - for (int i = 0; i < entries; i++) { - vf->mc_hash[i] = list[i]; - vec_reg = (vf->mc_hash[i] >> 5) & 0x7F; - vec_bit = vf->mc_hash[i] & 0x1F; - mta_reg = IXGBE_READ_REG(&sc->hw, IXGBE_MTA(vec_reg)); - mta_reg |= (1 << vec_bit); - IXGBE_WRITE_REG(&sc->hw, IXGBE_MTA(vec_reg), mta_reg); - } - - if (entries == 0) - vmolr &= ~IXGBE_VMOLR_ROMPE; - else + if (entries != 0) { vmolr |= IXGBE_VMOLR_ROMPE; - IXGBE_WRITE_REG(&sc->hw, IXGBE_VMOLR(vf->pool), vmolr); + IXGBE_WRITE_REG(&sc->hw, IXGBE_VMOLR(vf->pool), vmolr); + } ixgbe_send_vf_success(sc, vf, msg[0]); } /* ixgbe_vf_set_mc_addr */ @@ -759,9 +759,10 @@ ixgbe_if_iov_uninit(if_ctx_t ctx) IXGBE_WRITE_REG(hw, IXGBE_VT_CTL, 0); + sc->num_vfs = 0; + ixgbe_iov_rebuild_mta(sc); free(sc->vfs, M_IXGBE_SRIOV); sc->vfs = NULL; - sc->num_vfs = 0; sc->feat_en &= ~IXGBE_FEATURE_SRIOV; } /* ixgbe_if_iov_uninit */ @@ -782,8 +783,8 @@ ixgbe_init_vf(struct ixgbe_softc *sc, struct ixgbe_vf *vf) IXGBE_WRITE_REG(hw, IXGBE_PFMBIMR(vf_index), pfmbimr); ixgbe_vf_set_default_vlan(sc, vf, vf->vlan_tag); - - // XXX multicast addresses + vf->num_mc_hashes = 0; + bzero(vf->mc_hash, sizeof(vf->mc_hash)); if (ixgbe_validate_mac_addr(vf->ether_addr) == 0) { ixgbe_set_rar(&sc->hw, vf->rar_index, diff --git a/sys/dev/ixgbe/ixgbe.h b/sys/dev/ixgbe/ixgbe.h index 9120ca5a37ff..ae246433348e 100644 --- a/sys/dev/ixgbe/ixgbe.h +++ b/sys/dev/ixgbe/ixgbe.h @@ -441,6 +441,7 @@ struct ixgbe_softc { int num_vfs; int pool; struct ixgbe_vf *vfs; + bool iov_mta_valid; /* Bypass */ struct ixgbe_bp_data bypass; diff --git a/sys/dev/ixgbe/ixgbe_sriov.h b/sys/dev/ixgbe/ixgbe_sriov.h index 3c456ee819f2..c3a988bda886 100644 --- a/sys/dev/ixgbe/ixgbe_sriov.h +++ b/sys/dev/ixgbe/ixgbe_sriov.h @@ -72,6 +72,7 @@ void ixgbe_if_iov_uninit(if_ctx_t); void ixgbe_initialize_iov(struct ixgbe_softc *); void ixgbe_recalculate_max_frame(struct ixgbe_softc *); void ixgbe_ping_all_vfs(struct ixgbe_softc *); +u_int ixgbe_iov_rebuild_mta(struct ixgbe_softc *); int ixgbe_pci_iov_detach(device_t); void ixgbe_define_iov_schemas(device_t, int *); void ixgbe_align_all_queue_indices(struct ixgbe_softc *);