From nobody Thu Jul 30 05:24:44 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4h9d1T29jfz6mhs1 for ; Thu, 30 Jul 2026 05:24:45 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4h9d1T13ZQz3byZ for ; Thu, 30 Jul 2026 05:24:45 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1785389085; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=JjccX5npAtwPZdmJzr2Kby6Of2Ogh3+mhTz2OLU6xd0=; b=v89QInzNqj6bFdmn2A1X2zb1ul/QxDQ2Rr7Cace9FZUTxz+FNULfI0JL4KDhKMGpNb9d50 spywlARBdSCC9oSyP7NSN/C9n7tFkBu+cIgwYhF+Zyd7WSvRWZZUX/7JI7y7Spm1fFHx2W NXaqQJx2MH0xmNLFUxLWcG4bM6Jm06uPE2gwDxNoSOhBg3KhB0X9lX/L+5An6c7EFMYNIQ M/aguVJqujuAiiWwWwccgZ/H6mNobyj7CGCFN1Fm8ldodemXPHAZqHD36CnopUh1YKu1vZ 98hbLrNdMK9l5frNEf0JiG8Dj2KiHohyqKaL7qHPT2jtbwKVC7juWXTP54i/Gg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1785389085; a=rsa-sha256; cv=none; b=W5ksmhsBzVTzFcSXid5pXO5Aj7V3La2Nlf09+9c22y55/ZuLvQREWhe7Vp+a135l9xKl8v 7DEqI7JY+lPbVgRo1iIHDUg5Ai2zOMzgMJ3Ql/E0VxmAj2Xws+D1dTH/zoquFB29oJBO7i QEQlCURsnUm0KGHddFagkdFwmVx9njosJF3T8x9t2qR1I1MAQq6fJRuLGji1wi/Ymr92y1 6UJP3s71NS5ZwDmdN0fuLE63URnDFuhRwClR4W3Mb+ReMR+JPZ/qCPOZcRiZ9R1vrBrFG/ RAXY63uYl6hur9OmepLHuru1/XiGhM2iHgP0NvzSVfMt0OY3qLcqJOMDiJKtHw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1785389085; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=JjccX5npAtwPZdmJzr2Kby6Of2Ogh3+mhTz2OLU6xd0=; b=gKtedQOdDMLaYHaNznM/4UFWPSGoB0V7tj+4fcz3BqsjgLc371pS1QEoyxqW/B+70VKYJc 5uqAdrlR3J8tTW2MQcH7oc4HYSdsY4v1TypxFU2cwcS4+0yv+VOPvvHcZ61jyUxnFUkn4m oYl3ijtNSUqOlCXZgyBeuW5/YLcWzUXUectC+iUBWRrzKw6r8PGQd5GGYsyeAtHA33AsRk sIOU0B97u3dpXZWoeG5xtsADcrHtpUaVSAPxieoLg+J7Y78DXSExEv5oHgpSD8uvtU6zia 4QODdKEvPJU+EhjHjMPrfyGORvZxdNYs1oMKEKr4RfFNliootBCh2kiFUH2lTg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4h9d1S6mX2z10SH for ; Thu, 30 Jul 2026 05:24:44 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 34a9f by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 30 Jul 2026 05:24:44 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Kevin Bowling Subject: git: 5f02c8439786 - main - igb(4)/igbv(4): Document SR-IOV PF and VF functionality List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 5f02c8439786728392f250ff6fa772ce1787b74c Auto-Submitted: auto-generated Date: Thu, 30 Jul 2026 05:24:44 +0000 Message-Id: <6a6ae01c.34a9f.67e5eefb@gitrepo.freebsd.org> The branch main has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=5f02c8439786728392f250ff6fa772ce1787b74c commit 5f02c8439786728392f250ff6fa772ce1787b74c Author: Kevin Bowling AuthorDate: 2026-07-29 23:14:47 +0000 Commit: Kevin Bowling CommitDate: 2026-07-30 05:18:25 +0000 igb(4)/igbv(4): Document SR-IOV PF and VF functionality Document supported controllers, PF and VF naming, PCI_IOV and IOMMU requirements, queue and lifecycle constraints, iovctl schema, filtering and anti-spoof policy, mailbox and MDD recovery, shared hardware limits, rate control, and statistics cadence. Relnotes: yes Sponsored by: BBOX.io --- share/man/man4/Makefile | 2 + share/man/man4/em.4 | 173 ++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 171 insertions(+), 4 deletions(-) diff --git a/share/man/man4/Makefile b/share/man/man4/Makefile index cd7a16c58923..9ced5eef0af4 100644 --- a/share/man/man4/Makefile +++ b/share/man/man4/Makefile @@ -723,6 +723,8 @@ MLINKS+=edsc.4 if_edsc.4 MLINKS+=em.4 if_em.4 \ em.4 igb.4 \ em.4 if_igb.4 \ + em.4 igbv.4 \ + em.4 if_igbv.4 \ em.4 lem.4 \ em.4 if_lem.4 MLINKS+=enc.4 if_enc.4 diff --git a/share/man/man4/em.4 b/share/man/man4/em.4 index 4835d41320b2..328e1e77b18b 100644 --- a/share/man/man4/em.4 +++ b/share/man/man4/em.4 @@ -32,13 +32,14 @@ .\" .\" * Other names and brands may be claimed as the property of others. .\" -.Dd March 20, 2024 +.Dd July 29, 2026 .Dt EM 4 .Os .Sh NAME .Nm em , .Nm lem , -.Nm igb +.Nm igb , +.Nm igbv .Nd "Intel(R) PRO/1000 Gigabit Ethernet adapter driver" .Sh SYNOPSIS To compile this driver into the kernel, @@ -49,6 +50,11 @@ kernel configuration file: .Cd "device em" .Ed .Pp +PF SR-IOV support additionally requires: +.Bd -ragged -offset indent +.Cd "options PCI_IOV" +.Ed +.Pp Alternatively, to load the driver as a module at boot time, place the following line in .Xr loader.conf 5 : @@ -78,9 +84,12 @@ The .Nm driver provides support for PCI Express Gigabit Ethernet adapters based on the Intel 82575, 82576, 82580, i210, i211, and i35x. -These appear as +Physical functions appear as .Nm igb interfaces to maintain compatibility with existing infrastructure. +Virtual functions provided by 82576 and I350 physical functions appear as +.Nm igbv +interfaces. .Pp The driver supports Transmit/Receive checksum offload and Jumbo Frames on all but 82542-based adapters. @@ -161,6 +170,159 @@ is not specified, the driver defaults to .Pp For more information on configuring this device, see .Xr ifconfig 8 . +.Sh SR-IOV +When the kernel is built with +.Dv PCI_IOV , +the +.Nm igb +PF driver supports SR-IOV on 82576 and I350 controllers. +Up to seven VFs can be created per PF. +PF SR-IOV is not featured on 82575, 82580, I210, I211, or I354 controllers. +.Pp +Passing a VF through to a guest requires an IOMMU to isolate its DMA. +The packet filters described below do not constrain descriptor DMA; +.Xr bhyve 8 +refuses to assign a passthrough device when it cannot create an IOMMU domain. +.Pp +The supported SR-IOV configuration uses one PF transmit and receive queue and +requires MSI-X. +I350 requires this layout; 82576 hardware can combine VMDq with per-pool RSS, +but that mode is not implemented by this driver. +The PF must be administratively up and running when VFs are created and must +remain up for VF mailbox and datapath service. +Creating or destroying VFs temporarily stops and restarts a running PF. +Set the following per-device +.Xr iflib 4 +loader tunables before the PF attaches: +.Bd -literal -offset indent +dev.igb.N.iflib.override_ntxqs=1 +dev.igb.N.iflib.override_nrxqs=1 +.Ed +.Pp +The following VF parameters are accepted in +.Xr iovctl.conf 5 : +.Bl -tag -width "mac-anti-spoof" +.It Va mac-addr +Assign a unicast MAC address to the VF. +If omitted, the PF uses the stable address generator described by +.Xr ether_gen_addr 9 +with a distinct name for each VF. +.It Va mac-anti-spoof +Enable source MAC anti-spoofing. +The default is true. +The check binds transmissions to MAC addresses currently assigned to that VF. +It prevents impersonating the PF or a sibling VF, but a VF permitted to change +its MAC can select an address belonging to an off-host station. +.It Va allow-set-mac +Permit the VF to replace its primary MAC address, whether assigned by +.Va mac-addr +or generated by the PF, and to install up to three auxiliary unicast +addresses. +Each auxiliary address expands both the VF's receive filters and the source +addresses permitted by transmit anti-spoofing. +The default is false. +.It Va allow-promisc +Permit explicit VF mailbox requests for unicast or multicast promiscuous mode +within its VLAN membership. +The default is false. +VF promiscuous-mode changes are applied asynchronously. +If the PF refuses a request, the VF logs the failure after +.Xr ifconfig 8 +has returned. +.It Va vlan +Assign the VF an access VLAN from 1 through 4095. +The PF inserts this tag on VF transmissions, admits only this VLAN to the VF, +and strips the tag on reception so the VF operates on untagged traffic. +VF mailbox requests to change VLAN membership are rejected while an access +VLAN is assigned. +The default value, +.Dq trunk , +instead permits the VF to manage its VLAN memberships through the mailbox. +VLAN 0 cannot be assigned as an access VLAN. +.El +.Pp +I350 hides an assigned access VLAN in hardware. +On 82576, transparent delivery depends on the VF driver ignoring stripped +VLAN metadata for VLANs it did not request; the in-tree +.Nm igbv +VF driver does so. +DPDK guests and +.Fx +guests with an older +.Nm igb +driver do not apply this workaround and can expose the assigned VLAN tag +instead. +.Pp +The VF mailbox carries at most 30 multicast hashes. +If the VF reports more, the PF automatically enables reception of all +multicast within that VF's VLAN membership. +This overflow fallback is independent of +.Va allow-promisc . +The multicast hash table is shared by the port, so hash collisions can also +deliver matching multicast traffic to another multicast-enabled pool in the +same VLAN. +.Pp +A VF with +.Va allow-set-mac +enabled can install at most three auxiliary unicast receive addresses. +These filters come from a controller-wide pool; exhaustion makes later +requests fail without replacing existing filters. +.Pp +The hardware provides 32 VLAN-to-pool filter entries for VLANs assigned to +VFs, including PF-assigned access VLANs, and rejects a request that would +exceed 32 distinct VF VLANs. +A trunk VF's implicit VID-0 membership consumes one shared entry, leaving +31 entries for nonzero VF VLANs while any trunk VF is active. +.Pp +Each VF can burst 64 VLAN-add requests and then receives eight additional +additions per second. +This permits two complete 31-VLAN replays while bounding repeated global +filter-table rebuilds. +VLAN removals and requests that do not change membership are not limited. +A PF-wide reinitialization replenishes the burst; VF-initiated resets do not. +The +.Nm igbv +VF driver retries failed VLAN additions at the PF's sustained rate for up to +eight seconds. +This heals transient rate-limit failures without polling indefinitely when +the shared VLAN table is full. +.Pp +On 82576, a PF-only VLAN uses the default PF pool without consuming one of +these entries; an entry shared by the PF and VFs includes all member pools. +I350 can additionally place the PF in VLAN-promiscuous mode when its PF +memberships do not fit. +Because the entries are shared, VF allocations can force this PF fallback. +DMA coalescing is disabled while I350 SR-IOV is active. +.Pp +The PF enables malicious-driver detection. +Hardware stops a VF transmit queue that submits a malformed descriptor, such as +a legacy descriptor in a virtualized mode, or a descriptor requesting VLAN +insertion while an access VLAN is assigned. +The PF logs +.Ql malicious-driver event , +clears the offending VF's queue enables and PF-owned filter state, and sends a +control message directing the guest to complete a new reset handshake. +Recovery is unconditional; there is no tunable to disable it. +A VF that submits further malformed descriptors is stopped and recovered again. +.Pp +Transmissions failing the anti-spoof checks are dropped by hardware without +stopping the queue or resetting the VF. +The PF logs +.Ql spoof event detected +for an offending VF at most once every two seconds. +.Pp +If a VF retains mailbox ownership while the PF is reading a request, the PF +retries with a bounded backoff, then logs +.Ql mailbox remained busy +and revokes that VF's clear-to-send state. +Requests from that VF are ignored until a hardware reset begins a new mailbox +epoch. +A guest that resets through the VF control register or through a function level +reset recovers automatically; a guest that only sends the reset mailbox message +does not. +.Pp +PF and VF statistics are sampled once per interface watchdog interval, so +exported counters can trail the hardware by up to 500 milliseconds. .Sh HARDWARE The .Nm @@ -331,7 +493,10 @@ issue to .Xr ng_ether 4 , .Xr polling 4 , .Xr vlan 4 , -.Xr ifconfig 8 +.Xr iovctl.conf 5 , +.Xr ifconfig 8 , +.Xr iovctl 8 , +.Xr ether_gen_addr 9 .Sh HISTORY The .Nm