From nobody Tue May 21 23:25:13 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VkVsP2rhpz5L48T; Tue, 21 May 2024 23:25:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VkVsP2M73z4d6j; Tue, 21 May 2024 23:25:13 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1716333913; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=KyOABYotkSVoeFaiMvz2Di2/5Aq+aGI0Ylh6v8d76dE=; b=pHriW3KyaAh6qNZi4TfOtqIcjx1zstR4JDwhxsiMQrqGpHFMdMiRMcnQBWwyAfonur2/NZ Wx2n8NrOO3PtUMrAHS2txwCkepZXagaA4hQGoPrXRpD4vhRA5NpQfpGCg0AMbsNyQ4w3oX JCvR4X3PAFggVq+GuDcOHpx1ePwR5mjRhqG+T8eH9JR9L8XiONGe4/i4X1RyI2cmVDnVA2 ZosmPFl686HPsjVHdKB+MQp+zv5cBlXIgY7ah8yEEnd+iXsL35AcZt42vkZenqDMxuJ9k5 zc9mX5Rs4tEF5V/1x+Diosk4K1MsfCS6NwufjDFRN5NSc0zowvjnpmaHgRzOgg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1716333913; a=rsa-sha256; cv=none; b=L0eQ9TAr9N4F9kHwiW4eSlEvyPCdKZ7M/OwMD+C5VsePGmZsnneVY1mOkwVTVlMjyvNl9R 9o602huL1ceESC1pxFHRZxPBLwppsS0IKEDmdZWYtuUZhCE2TD3nUtwx7iwdmPs/+ZbmT0 /NdegJaBvyexoHb2xlxBj7xhwEWVoQtXImqk10h2cfkTR6YfTYIwepu/JH6LAQOSymt8IY P2qAPDfvYrOc7nV8lrOfVABevqNVsXwarOiUbSdKNhJvgLF9qAYxR4Om0hPceNtFNiGuCS 46oAJ9Eg3zEzvXkQMw/PR4KphHOkRcHGdKyP/d0MQObm/7Ux9qv0IezPkfAX/A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1716333913; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=KyOABYotkSVoeFaiMvz2Di2/5Aq+aGI0Ylh6v8d76dE=; b=NK/u9ftnN0XiBPOxoLtKI97g0WpzbXMjh2fJbTE0wPrs2uqE2AXbp5iBiyycR1uIIfKigM lti3mc7zqjI7U5Hn9GDkk/+TlaWEGwxTS8DrIYOO4gwiU3VbEKHryFYuQ+tL02lQGwIZtx b0u57kBU3+gS9nl6jbrtIUKcXYxX+DmeQOifyCda52o9cXQvvQPBZf58LmgOrbO6Sb9dR1 /S2eBdglOERkkfKogyHKxaEhsSBv7YbkoTqLPiJojWaPmBi5KG87YRnLdOv2ue8eZlOy4q vzZB4XYEs/P9TxnY83JTrA7tqWJXvpFNwEa9Vbt1ZvzmacUpnpGQ2qYJb+/Kvg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4VkVsP1yN9zhP3; Tue, 21 May 2024 23:25:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 44LNPDCQ080936; Tue, 21 May 2024 23:25:13 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 44LNPDND080933; Tue, 21 May 2024 23:25:13 GMT (envelope-from git) Date: Tue, 21 May 2024 23:25:13 GMT Message-Id: <202405212325.44LNPDND080933@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Dag-Erling =?utf-8?Q?Sm=C3=B8rgrav?= Subject: git: ba222f6fb4b2 - main - iavf(4): Improve man page List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: des X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: ba222f6fb4b226ab24beaa13a8591dcb624ae70f Auto-Submitted: auto-generated The branch main has been updated by des: URL: https://cgit.FreeBSD.org/src/commit/?id=ba222f6fb4b226ab24beaa13a8591dcb624ae70f commit ba222f6fb4b226ab24beaa13a8591dcb624ae70f Author: Eric Joyner AuthorDate: 2024-05-21 23:24:31 +0000 Commit: Dag-Erling Smørgrav CommitDate: 2024-05-21 23:24:31 +0000 iavf(4): Improve man page MFC after: 3 days Reviewed by: erj Differential Revision: https://reviews.freebsd.org/D43093 --- share/man/man4/iavf.4 | 348 ++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 282 insertions(+), 66 deletions(-) diff --git a/share/man/man4/iavf.4 b/share/man/man4/iavf.4 index 2e265a5a2fbb..d55e084e82cc 100644 --- a/share/man/man4/iavf.4 +++ b/share/man/man4/iavf.4 @@ -1,3 +1,6 @@ +.\"- +.\" SPDX-License-Identifier: BSD-3-Clause +.\" .\" Copyright (c) 2013-2018, Intel Corporation .\" All rights reserved. .\" @@ -29,12 +32,12 @@ .\" .\" * Other names and brands may be claimed as the property of others. .\" -.Dd January 30, 2019 +.Dd May 21, 2024 .Dt IAVF 4 .Os .Sh NAME .Nm iavf -.Nd "Intel Adaptive Virtual Function driver" +.Nd "Intel Ethernet Adaptive Virtual Function Driver" .Sh SYNOPSIS To compile this driver into the kernel, place the following lines in your kernel configuration file: @@ -51,87 +54,300 @@ if_iavf_load="YES" .Sh DESCRIPTION The .Nm -driver provides support for the PCI Virtual Functions from the 700 Series of -ethernet devices and newer product families. -The driver supports Jumbo Frames, TX/RX checksum offload, -TCP segmentation offload (TSO), Large Receive Offload (LRO), VLAN -tag insertion/extraction, VLAN checksum offload, VLAN TSO, and -Receive Side Steering (RSS), all for both IPv4 and IPv6. -For further hardware information and questions related to hardware -requirements, see -.Pa http://support.intel.com/ . -.Pp -Support for Jumbo Frames is provided via the interface MTU setting. -Selecting an MTU larger than 1500 bytes with the +driver provides support for any PCI Virtual Function created from certain +Intel Ethernet devices. +This driver is compatible with virtual functions bound to devices based on the +following: +.Pp +.Bl -bullet -compact +.It +Intel\(rg Ethernet Controller E810\-C +.It +Intel\(rg Ethernet Controller E810\-XXV +.It +Intel\(rg Ethernet Connection E822\-C +.It +Intel\(rg Ethernet Connection E822\-L +.It +Intel\(rg Ethernet Connection E823\-C +.It +Intel\(rg Ethernet Connection E823\-L +.It +Intel\(rg Ethernet Controller I710 +.It +Intel\(rg Ethernet Controller X710 +.It +Intel\(rg Ethernet Controller XL710 +.It +Intel\(rg Ethernet Network Connection X722 +.It +Intel\(rg Ethernet Controller XXV710 +.It +Intel\(rg Ethernet Controller V710 +.El +.Pp +The associated Physical Function (PF) drivers for this VF driver are: +.Pp +.Bl -bullet -compact +.It +.Xr ice 4 +.It +.Xr ixl 4 +.El +.Pp +For questions related to hardware requirements, refer to the documentation +supplied with your Intel Ethernet Adapter. +All hardware requirements listed apply to use with +.Fx . +.Ss The VF Driver +The VF driver is normally used in a virtualized environment where a host driver +manages SR\-IOV, and provides a VF device to the guest. +.Pp +In the +.Fx +guest, the iavf driver would be loaded and will function using +the VF device assigned to it. +.Pp +The VF driver provides most of the same functionality as the core driver, but +is actually a subordinate to the host. +Access to many controls is accomplished by a request to the host via what is +called the "Admin queue." +These are startup and initialization events, however; once in operation, the +device is self\-contained and should achieve near native performance. +.Pp +Some notable limitations of the VF environment: +.Bl -bullet +.It +The PF can configure the VF to allow promiscuous mode, using a configuration +parameter in +.Xr iovctl.conf 5 ; +otherwise, promiscuous mode will not work +.It +Media info is not available from the PF, so the active media will always be +displayed as auto in .Xr ifconfig 8 -utility configures the adapter to receive and transmit Jumbo Frames. -The maximum MTU size for Jumbo Frames is 9706. +.El +.Ss Adaptive Virtual Function +Adaptive Virtual Function (AVF) allows the virtual function driver, or VF, to +adapt to changing feature sets of the physical function driver (PF) with which +it is associated. +This allows system administrators to update a PF without having to update all +the VFs associated with it. +All AVFs have a single common device ID and branding string. .Pp -Offloads are also controlled via the interface, for instance, -checksumming for both IPv4 and IPv6 can be set and unset, TSO4 -and/or TSO6, and finally LRO can be set and unset. +AVFs have a minimum set of features known as "base mode," but may provide +additional features depending on what features are available in the PF with +which the AVF is associated. +The following are base mode features: +.Bl -bullet -compact +.It +4 Queue Pairs (QP) and associated Configuration Status Registers (CSRs) +for Tx/Rx +.It +iavf descriptors and ring format +.It +Descriptor write\-back completion +.It +1 control queue, with iavf descriptors, CSRs and ring format +.It +5 MSI\-X interrupt vectors and corresponding iavf CSRs +.It +1 Interrupt Throttle Rate (ITR) index +.It +1 Virtual Station Interface (VSI) per VF +.It +1 Traffic Class (TC), TC0 +.It +Receive Side Scaling (RSS) with 64 entry indirection table and key, +configured through the PF +.It +1 unicast MAC address reserved per VF +.It +8 MAC address filters for each VF on an Intel\(rg Ethernet 800 Series device +.It +16 MAC address filters for each VF on an Intel\(rg Ethernet 700 Series device +.It +Stateless offloads \- non\-tunneled checksums +.It +AVF device ID +.It +HW mailbox is used for VF to PF communications +.El +.Sh CONFIGURATION AND TUNING +.Ss Important System Configuration Changes +It is important to note that 100G operation can generate high +numbers of interrupts, often incorrectly being interpreted as +a storm condition in the kernel. +It is suggested that this be resolved by setting +.Va hw.intr_storm_threshold +to 0. .Pp -For more information on configuring this device, see -.Xr ifconfig 8 . +The default is 1000. .Pp -.Em NOTE : -This +Best throughput results are seen with a large MTU; use 9706 if possible. +The default number of descriptors per ring is 1024. +Increasing this may improve performance, depending on your use case. +.Ss Configuring for no iflib +.Xr iflib 4 +is a common framework for network interface drivers for +.Fx +that uses a shared set of sysctl names. +.Pp +The default .Nm -driver is only for Virtual Functions. -For 700 series Physical Functions, use the -.Xr ixl 4 -driver. -.Sh LOADER TUNABLES -Tunables can be set at the -.Xr loader 8 -prompt before booting the kernel or stored in -.Xr loader.conf 5 . -.Bl -tag -width indent -.It Va hw.iavf.rx_itr -The RX interrupt rate value, set to 62 (124 usec) by default. -.It Va hw.iavf.tx_itr -The TX interrupt rate value, set to 122 (244 usec) by default. -.It Va hw.iavf.enable_head_writeback -When the driver is finding the last TX descriptor processed by the hardware, -use a value written to memory by the hardware instead of scanning the -descriptor ring for completed descriptors. -Disabled by default; this mimics the "legacy" TX behavior found in -.Xr ixgbe 4 . -to ensure compatibility with future, non-700 series VF devices. +driver depends on it, but it can be compiled without it. +.Ss Jumbo Frames +Jumbo Frames support is enabled by changing the Maximum Transmission Unit (MTU) +to a value larger than the default value of 1500. +.Pp +Use the +.Xr ifconfig 8 +command to increase the MTU size. +.Pp +To confirm the MTU used between two specific devices, use +.Xr route 8 : +.Bd -literal -offset indent +route get +.Ed +.Pp +NOTE: +.Bl -bullet +.It +The maximum MTU setting for jumbo frames is 9706. +This corresponds to the maximum jumbo frame size of 9728 bytes. +.It +This driver will attempt to use multiple page-sized buffers to receive +each jumbo packet. +This should help to avoid buffer starvation issues when allocating receive +packets. +.It +Packet loss may have a greater impact on throughput when you use jumbo +frames. +If you observe a drop in performance after enabling jumbo frames, enabling +flow control may mitigate the issue. .El -.Sh SUPPORT -For general information and support, -go to the Intel support website at: -.Pa http://support.intel.com/ . +.Ss Checksum Offload +Checksum offloading supports both TCP and UDP packets and is supported for both +transmit and receive. +.Pp +TSO (TCP Segmentation Offload) supports both IPv4 and IPv6. +Both of these features are enabled and disabled via +.Xr ifconfig 8 . .Pp -If an issue is identified with this driver with a supported adapter, -email all the specific information related to the issue to -.Mt freebsd@intel.com . +NOTE: +.Bl -bullet -compact +.It +TSO requires Tx checksum; if Tx checksum is disabled then TSO will also +be disabled. +.El +.Ss LRO +LRO (Large Receive Offload) may provide Rx performance improvement. +However, it is incompatible with packet\-forwarding workloads. +You should carefully evaluate the environment and enable LRO when possible. +.Ss Rx and Tx Descriptor Rings +Allows you to set the Rx and Tx descriptor rings independently. +Set them via these +.Xr iflib 4 +sysctls: +.Bl -tag -width indent +.It dev.iavf.#.iflib.override_nrxds +.It dev.iavf.#.iflib.override_ntxds +.El +.Ss Link\-Level Flow Control (LFC) +The VF driver does not have access to flow control settings. +It must be managed from the host side. .Sh SEE ALSO .Xr arp 4 , +.Xr ice 4 , +.Xr iflib 4 , .Xr ixl 4 , .Xr netintro 4 , .Xr vlan 4 , -.Xr ifconfig 8 , -.Xr iflib 9 +.Xr ifconfig 8 +.Pp +See the +.Dq Intel\(rg Ethernet Adapters and Devices User Guide +for additional information on features. +It is available on the Intel website at either of the following: +.Bl -bullet +.It +.Lk https://cdrdv2.intel.com/v1/dl/getContent/705831 +.It +.Lk https://www.intel.com/content/www/us/en/download/19373/adapter\-user\-guide\-for\-intel\-ethernet\-adapters.html +.El +.Pp +For information on how to identify your adapter, and for the latest Intel +network drivers, refer to the Intel Support website: +.Aq Lk http://www.intel.com/support +.Sh CAVEATS +.Ss Driver Buffer Overflow Fix +The fix to resolve CVE\-2016\-8105, referenced in Intel SA\-00069 +.Aq Lk https://www.intel.com/content/www/us/en/security\-center/advisory/intel\-sa\-00069.html , +is included in this and future versions of the driver. +.Ss Network Memory Buffer Allocation +.Fx +may have a low number of network memory buffers (mbufs) by default. +If your mbuf value is too low, it may cause the driver to fail to initialize +and/or cause the system to become unresponsive. +You can check to see if the system is mbuf\-starved by running +.Li "netstat -m" . +Increase the number of mbufs by editing the lines below in +.Xr sysctl.conf 5 : +.Bd -literal -offset indent +kern.ipc.nmbclusters +kern.ipc.nmbjumbop +kern.ipc.nmbjumbo9 +kern.ipc.nmbjumbo16 +kern.ipc.nmbufs +.Ed +.Pp +The amount of memory that you allocate is system specific, and may require +some trial and error. +Also, increasing the following in +.Xr sysctl.conf 5 +could help increase +network performance: +.Bd -literal -offset indent +kern.ipc.maxsockbuf +net.inet.tcp.sendspace +net.inet.tcp.recvspace +net.inet.udp.maxdgram +net.inet.udp.recvspace +.Ed +.Ss UDP Stress Test Dropped Packet Issue +Under small packet UDP stress with the +.Nm +driver, the system may drop UDP packets due to socket buffers being full. +Setting the PF driver's Flow Control variables to the minimum may resolve the +issue. +.Ss Disable LRO when routing/bridging +LRO must be turned off when forwarding traffic. +.Sh SUPPORT +For general information, go to the Intel support website at +.Aq Lk http://www.intel.com/support/ . +.Pp +If an issue is identified with the released source code on a supported kernel +with a supported adapter, email the specific information related to the issue +to +.Aq Mt freebsd@intel.com . +.Sh LEGAL +Intel\(rg is a trademark or registered trademark of Intel Corporation +or its subsidiaries in the United States and / or other countries. +.Pp +Other names and brands may be claimed as the property of others. .Sh HISTORY The .Nm device driver first appeared in -.Fx 10.1 . -under the name "ixlv" +.Fx 10.1 +under the name +.Nm ixlv . It was converted to use -.Xr iflib 9 -and changed to its current name in -.Fx 12 . +.Xr iflib 4 +and renamed in +.Fx 12.4 . .Sh AUTHORS -.An -nosplit The .Nm -driver was written by -.An Jack Vogel Aq Mt jfv@freebsd.org -and -.An Eric Joyner Aq Mt erj@freebsd.org . -.Sh CAVEATS -This driver is supposed to function on VFs spawned from future network devices by Intel, -but at the time of this writing, has only been tested on the 700 series VFs. +driver was written by the +.An Intel Corporation Aq Mt freebsd@intel.com