Date: Thu, 30 Jul 2026 05:24:44 +0000 From: Kevin Bowling <kbowling@FreeBSD.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: git: 5f02c8439786 - main - igb(4)/igbv(4): Document SR-IOV PF and VF functionality Message-ID: <6a6ae01c.34a9f.67e5eefb@gitrepo.freebsd.org>
index | next in thread | raw e-mail
The branch main has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=5f02c8439786728392f250ff6fa772ce1787b74c commit 5f02c8439786728392f250ff6fa772ce1787b74c Author: Kevin Bowling <kbowling@FreeBSD.org> AuthorDate: 2026-07-29 23:14:47 +0000 Commit: Kevin Bowling <kbowling@FreeBSD.org> CommitDate: 2026-07-30 05:18:25 +0000 igb(4)/igbv(4): Document SR-IOV PF and VF functionality Document supported controllers, PF and VF naming, PCI_IOV and IOMMU requirements, queue and lifecycle constraints, iovctl schema, filtering and anti-spoof policy, mailbox and MDD recovery, shared hardware limits, rate control, and statistics cadence. Relnotes: yes Sponsored by: BBOX.io --- share/man/man4/Makefile | 2 + share/man/man4/em.4 | 173 ++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 171 insertions(+), 4 deletions(-) diff --git a/share/man/man4/Makefile b/share/man/man4/Makefile index cd7a16c58923..9ced5eef0af4 100644 --- a/share/man/man4/Makefile +++ b/share/man/man4/Makefile @@ -723,6 +723,8 @@ MLINKS+=edsc.4 if_edsc.4 MLINKS+=em.4 if_em.4 \ em.4 igb.4 \ em.4 if_igb.4 \ + em.4 igbv.4 \ + em.4 if_igbv.4 \ em.4 lem.4 \ em.4 if_lem.4 MLINKS+=enc.4 if_enc.4 diff --git a/share/man/man4/em.4 b/share/man/man4/em.4 index 4835d41320b2..328e1e77b18b 100644 --- a/share/man/man4/em.4 +++ b/share/man/man4/em.4 @@ -32,13 +32,14 @@ .\" .\" * Other names and brands may be claimed as the property of others. .\" -.Dd March 20, 2024 +.Dd July 29, 2026 .Dt EM 4 .Os .Sh NAME .Nm em , .Nm lem , -.Nm igb +.Nm igb , +.Nm igbv .Nd "Intel(R) PRO/1000 Gigabit Ethernet adapter driver" .Sh SYNOPSIS To compile this driver into the kernel, @@ -49,6 +50,11 @@ kernel configuration file: .Cd "device em" .Ed .Pp +PF SR-IOV support additionally requires: +.Bd -ragged -offset indent +.Cd "options PCI_IOV" +.Ed +.Pp Alternatively, to load the driver as a module at boot time, place the following line in .Xr loader.conf 5 : @@ -78,9 +84,12 @@ The .Nm driver provides support for PCI Express Gigabit Ethernet adapters based on the Intel 82575, 82576, 82580, i210, i211, and i35x. -These appear as +Physical functions appear as .Nm igb interfaces to maintain compatibility with existing infrastructure. +Virtual functions provided by 82576 and I350 physical functions appear as +.Nm igbv +interfaces. .Pp The driver supports Transmit/Receive checksum offload and Jumbo Frames on all but 82542-based adapters. @@ -161,6 +170,159 @@ is not specified, the driver defaults to .Pp For more information on configuring this device, see .Xr ifconfig 8 . +.Sh SR-IOV +When the kernel is built with +.Dv PCI_IOV , +the +.Nm igb +PF driver supports SR-IOV on 82576 and I350 controllers. +Up to seven VFs can be created per PF. +PF SR-IOV is not featured on 82575, 82580, I210, I211, or I354 controllers. +.Pp +Passing a VF through to a guest requires an IOMMU to isolate its DMA. +The packet filters described below do not constrain descriptor DMA; +.Xr bhyve 8 +refuses to assign a passthrough device when it cannot create an IOMMU domain. +.Pp +The supported SR-IOV configuration uses one PF transmit and receive queue and +requires MSI-X. +I350 requires this layout; 82576 hardware can combine VMDq with per-pool RSS, +but that mode is not implemented by this driver. +The PF must be administratively up and running when VFs are created and must +remain up for VF mailbox and datapath service. +Creating or destroying VFs temporarily stops and restarts a running PF. +Set the following per-device +.Xr iflib 4 +loader tunables before the PF attaches: +.Bd -literal -offset indent +dev.igb.N.iflib.override_ntxqs=1 +dev.igb.N.iflib.override_nrxqs=1 +.Ed +.Pp +The following VF parameters are accepted in +.Xr iovctl.conf 5 : +.Bl -tag -width "mac-anti-spoof" +.It Va mac-addr +Assign a unicast MAC address to the VF. +If omitted, the PF uses the stable address generator described by +.Xr ether_gen_addr 9 +with a distinct name for each VF. +.It Va mac-anti-spoof +Enable source MAC anti-spoofing. +The default is true. +The check binds transmissions to MAC addresses currently assigned to that VF. +It prevents impersonating the PF or a sibling VF, but a VF permitted to change +its MAC can select an address belonging to an off-host station. +.It Va allow-set-mac +Permit the VF to replace its primary MAC address, whether assigned by +.Va mac-addr +or generated by the PF, and to install up to three auxiliary unicast +addresses. +Each auxiliary address expands both the VF's receive filters and the source +addresses permitted by transmit anti-spoofing. +The default is false. +.It Va allow-promisc +Permit explicit VF mailbox requests for unicast or multicast promiscuous mode +within its VLAN membership. +The default is false. +VF promiscuous-mode changes are applied asynchronously. +If the PF refuses a request, the VF logs the failure after +.Xr ifconfig 8 +has returned. +.It Va vlan +Assign the VF an access VLAN from 1 through 4095. +The PF inserts this tag on VF transmissions, admits only this VLAN to the VF, +and strips the tag on reception so the VF operates on untagged traffic. +VF mailbox requests to change VLAN membership are rejected while an access +VLAN is assigned. +The default value, +.Dq trunk , +instead permits the VF to manage its VLAN memberships through the mailbox. +VLAN 0 cannot be assigned as an access VLAN. +.El +.Pp +I350 hides an assigned access VLAN in hardware. +On 82576, transparent delivery depends on the VF driver ignoring stripped +VLAN metadata for VLANs it did not request; the in-tree +.Nm igbv +VF driver does so. +DPDK guests and +.Fx +guests with an older +.Nm igb +driver do not apply this workaround and can expose the assigned VLAN tag +instead. +.Pp +The VF mailbox carries at most 30 multicast hashes. +If the VF reports more, the PF automatically enables reception of all +multicast within that VF's VLAN membership. +This overflow fallback is independent of +.Va allow-promisc . +The multicast hash table is shared by the port, so hash collisions can also +deliver matching multicast traffic to another multicast-enabled pool in the +same VLAN. +.Pp +A VF with +.Va allow-set-mac +enabled can install at most three auxiliary unicast receive addresses. +These filters come from a controller-wide pool; exhaustion makes later +requests fail without replacing existing filters. +.Pp +The hardware provides 32 VLAN-to-pool filter entries for VLANs assigned to +VFs, including PF-assigned access VLANs, and rejects a request that would +exceed 32 distinct VF VLANs. +A trunk VF's implicit VID-0 membership consumes one shared entry, leaving +31 entries for nonzero VF VLANs while any trunk VF is active. +.Pp +Each VF can burst 64 VLAN-add requests and then receives eight additional +additions per second. +This permits two complete 31-VLAN replays while bounding repeated global +filter-table rebuilds. +VLAN removals and requests that do not change membership are not limited. +A PF-wide reinitialization replenishes the burst; VF-initiated resets do not. +The +.Nm igbv +VF driver retries failed VLAN additions at the PF's sustained rate for up to +eight seconds. +This heals transient rate-limit failures without polling indefinitely when +the shared VLAN table is full. +.Pp +On 82576, a PF-only VLAN uses the default PF pool without consuming one of +these entries; an entry shared by the PF and VFs includes all member pools. +I350 can additionally place the PF in VLAN-promiscuous mode when its PF +memberships do not fit. +Because the entries are shared, VF allocations can force this PF fallback. +DMA coalescing is disabled while I350 SR-IOV is active. +.Pp +The PF enables malicious-driver detection. +Hardware stops a VF transmit queue that submits a malformed descriptor, such as +a legacy descriptor in a virtualized mode, or a descriptor requesting VLAN +insertion while an access VLAN is assigned. +The PF logs +.Ql malicious-driver event , +clears the offending VF's queue enables and PF-owned filter state, and sends a +control message directing the guest to complete a new reset handshake. +Recovery is unconditional; there is no tunable to disable it. +A VF that submits further malformed descriptors is stopped and recovered again. +.Pp +Transmissions failing the anti-spoof checks are dropped by hardware without +stopping the queue or resetting the VF. +The PF logs +.Ql spoof event detected +for an offending VF at most once every two seconds. +.Pp +If a VF retains mailbox ownership while the PF is reading a request, the PF +retries with a bounded backoff, then logs +.Ql mailbox remained busy +and revokes that VF's clear-to-send state. +Requests from that VF are ignored until a hardware reset begins a new mailbox +epoch. +A guest that resets through the VF control register or through a function level +reset recovers automatically; a guest that only sends the reset mailbox message +does not. +.Pp +PF and VF statistics are sampled once per interface watchdog interval, so +exported counters can trail the hardware by up to 500 milliseconds. .Sh HARDWARE The .Nm @@ -331,7 +493,10 @@ issue to .Xr ng_ether 4 , .Xr polling 4 , .Xr vlan 4 , -.Xr ifconfig 8 +.Xr iovctl.conf 5 , +.Xr ifconfig 8 , +.Xr iovctl 8 , +.Xr ether_gen_addr 9 .Sh HISTORY The .Nmhome | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6a6ae01c.34a9f.67e5eefb>