From nobody Thu May 23 02:44:11 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VlCDX0ymtz5LZ69; Thu, 23 May 2024 02:44:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VlCDX0JBzz44Gs; Thu, 23 May 2024 02:44:12 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1716432252; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=je6mt2LTvqaWS4bpcPtnaWGZoOUF42D3IPP+G3Yz6LU=; b=BjrpL82qG2bfzkS5OsCq68eOkzHNEnDNfl20WHHSre82VXOJHcxhPi+ReA1gwUE9VnYvlk c4gBDtqpQZUj8A2dMT3WSaY5nqFGFCa0y6uVkHQaIOyIrZxE3e1K/MOgbJtEzwo1XXD0Cv zoVPyQbJPaMURqa2HiR1/xUIc3vV4loDtDa7Idf9x8Xvzaf3CvyBbitLFTJfLCsWUnmz2w hbzuoT6O/EU4F1jwFeKJ1yMbecPGikXJlYdSRDPjVby06yQ9NevUHEkAvJ1NVfCCch4aTG Zm/LYV0ZMuk316Oa5t1beOMfkrpVH0hGkZs66Xl6yzNfFXbtXnEjJJkJbn/AoA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1716432252; a=rsa-sha256; cv=none; b=E9UrDsh+wwjgCt3XEHUUPZcsbOZ6kFggA3+1jMpmnqbHvv6HYIH8pdcRpL4y1KxW2O1ILo 39aSUKYVnmkf3BKsNkdrRbdY2lSvEFzRCjTCsMrPpRosjxp/TZKiGsWUaeI9iaenBmEdua r2tZSTixfti2GY9OALvwnIBLTGC4m0LIxmaKXX0yMGZYxmGHRnE5mjAvUqemRVe5gVNM6j 3O4xxhWpSJYueXzMz+oP4srpBUj2eqXkHJIOeq81ADtemK8VmQr6i14Jf+IZT4/03XlCdF hMVBCamMdOwvq6tbJFRWX5RkTLI3E5dsELj7HGpgb0mrhNMs+iyO16SAhlPT1g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1716432252; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=je6mt2LTvqaWS4bpcPtnaWGZoOUF42D3IPP+G3Yz6LU=; b=qHJgJ2DXHnjxZXAbVOo5y64SpKNvFK/iV8n/3C4fP2brG0QUQ9b5RoeXRD5/yf0bRNC05Q /fAyOC29YVIUxcnrwHB8h8ukPIYlECuuyeuz8KTAOcjZ6AydKHGQIlP2SRUDxDW/iIv61S NRxGdFMVCvq8uC+rmiFDIMPnYo4Ro8gL/kAqUm2LVq2PKBl1adZs+33hIpXMAN658SQm9L lCa3KZ3xBsNJ5IdrhYryF1y9jhvy/mywb9/hdLxIMGCifNfPo6lpXwbyIhV3Dr6yIjIxaE w79TxGxn/pCf9XlY44E1Pzyt5505btp+KFBoMhja1d+Oktg2IFk0cCgv+AgOxQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4VlCDW7235zXby; Thu, 23 May 2024 02:44:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 44N2iBcW056772; Thu, 23 May 2024 02:44:11 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 44N2iB5m056769; Thu, 23 May 2024 02:44:11 GMT (envelope-from git) Date: Thu, 23 May 2024 02:44:11 GMT Message-Id: <202405230244.44N2iB5m056769@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Dag-Erling =?utf-8?Q?Sm=C3=B8rgrav?= Subject: git: d823e80ef5a7 - stable/13 - iavf(4): Improve man page List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: des X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: d823e80ef5a709ee8c1f9e138eedcda6c1cbd1b5 Auto-Submitted: auto-generated The branch stable/13 has been updated by des: URL: https://cgit.FreeBSD.org/src/commit/?id=d823e80ef5a709ee8c1f9e138eedcda6c1cbd1b5 commit d823e80ef5a709ee8c1f9e138eedcda6c1cbd1b5 Author: Eric Joyner AuthorDate: 2024-05-21 23:24:31 +0000 Commit: Dag-Erling Smørgrav CommitDate: 2024-05-23 02:43:49 +0000 iavf(4): Improve man page MFC after: 3 days Reviewed by: erj Differential Revision: https://reviews.freebsd.org/D43093 (cherry picked from commit ba222f6fb4b226ab24beaa13a8591dcb624ae70f) --- share/man/man4/iavf.4 | 348 ++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 282 insertions(+), 66 deletions(-) diff --git a/share/man/man4/iavf.4 b/share/man/man4/iavf.4 index 2e265a5a2fbb..d55e084e82cc 100644 --- a/share/man/man4/iavf.4 +++ b/share/man/man4/iavf.4 @@ -1,3 +1,6 @@ +.\"- +.\" SPDX-License-Identifier: BSD-3-Clause +.\" .\" Copyright (c) 2013-2018, Intel Corporation .\" All rights reserved. .\" @@ -29,12 +32,12 @@ .\" .\" * Other names and brands may be claimed as the property of others. .\" -.Dd January 30, 2019 +.Dd May 21, 2024 .Dt IAVF 4 .Os .Sh NAME .Nm iavf -.Nd "Intel Adaptive Virtual Function driver" +.Nd "Intel Ethernet Adaptive Virtual Function Driver" .Sh SYNOPSIS To compile this driver into the kernel, place the following lines in your kernel configuration file: @@ -51,87 +54,300 @@ if_iavf_load="YES" .Sh DESCRIPTION The .Nm -driver provides support for the PCI Virtual Functions from the 700 Series of -ethernet devices and newer product families. -The driver supports Jumbo Frames, TX/RX checksum offload, -TCP segmentation offload (TSO), Large Receive Offload (LRO), VLAN -tag insertion/extraction, VLAN checksum offload, VLAN TSO, and -Receive Side Steering (RSS), all for both IPv4 and IPv6. -For further hardware information and questions related to hardware -requirements, see -.Pa http://support.intel.com/ . -.Pp -Support for Jumbo Frames is provided via the interface MTU setting. -Selecting an MTU larger than 1500 bytes with the +driver provides support for any PCI Virtual Function created from certain +Intel Ethernet devices. +This driver is compatible with virtual functions bound to devices based on the +following: +.Pp +.Bl -bullet -compact +.It +Intel\(rg Ethernet Controller E810\-C +.It +Intel\(rg Ethernet Controller E810\-XXV +.It +Intel\(rg Ethernet Connection E822\-C +.It +Intel\(rg Ethernet Connection E822\-L +.It +Intel\(rg Ethernet Connection E823\-C +.It +Intel\(rg Ethernet Connection E823\-L +.It +Intel\(rg Ethernet Controller I710 +.It +Intel\(rg Ethernet Controller X710 +.It +Intel\(rg Ethernet Controller XL710 +.It +Intel\(rg Ethernet Network Connection X722 +.It +Intel\(rg Ethernet Controller XXV710 +.It +Intel\(rg Ethernet Controller V710 +.El +.Pp +The associated Physical Function (PF) drivers for this VF driver are: +.Pp +.Bl -bullet -compact +.It +.Xr ice 4 +.It +.Xr ixl 4 +.El +.Pp +For questions related to hardware requirements, refer to the documentation +supplied with your Intel Ethernet Adapter. +All hardware requirements listed apply to use with +.Fx . +.Ss The VF Driver +The VF driver is normally used in a virtualized environment where a host driver +manages SR\-IOV, and provides a VF device to the guest. +.Pp +In the +.Fx +guest, the iavf driver would be loaded and will function using +the VF device assigned to it. +.Pp +The VF driver provides most of the same functionality as the core driver, but +is actually a subordinate to the host. +Access to many controls is accomplished by a request to the host via what is +called the "Admin queue." +These are startup and initialization events, however; once in operation, the +device is self\-contained and should achieve near native performance. +.Pp +Some notable limitations of the VF environment: +.Bl -bullet +.It +The PF can configure the VF to allow promiscuous mode, using a configuration +parameter in +.Xr iovctl.conf 5 ; +otherwise, promiscuous mode will not work +.It +Media info is not available from the PF, so the active media will always be +displayed as auto in .Xr ifconfig 8 -utility configures the adapter to receive and transmit Jumbo Frames. -The maximum MTU size for Jumbo Frames is 9706. +.El +.Ss Adaptive Virtual Function +Adaptive Virtual Function (AVF) allows the virtual function driver, or VF, to +adapt to changing feature sets of the physical function driver (PF) with which +it is associated. +This allows system administrators to update a PF without having to update all +the VFs associated with it. +All AVFs have a single common device ID and branding string. .Pp -Offloads are also controlled via the interface, for instance, -checksumming for both IPv4 and IPv6 can be set and unset, TSO4 -and/or TSO6, and finally LRO can be set and unset. +AVFs have a minimum set of features known as "base mode," but may provide +additional features depending on what features are available in the PF with +which the AVF is associated. +The following are base mode features: +.Bl -bullet -compact +.It +4 Queue Pairs (QP) and associated Configuration Status Registers (CSRs) +for Tx/Rx +.It +iavf descriptors and ring format +.It +Descriptor write\-back completion +.It +1 control queue, with iavf descriptors, CSRs and ring format +.It +5 MSI\-X interrupt vectors and corresponding iavf CSRs +.It +1 Interrupt Throttle Rate (ITR) index +.It +1 Virtual Station Interface (VSI) per VF +.It +1 Traffic Class (TC), TC0 +.It +Receive Side Scaling (RSS) with 64 entry indirection table and key, +configured through the PF +.It +1 unicast MAC address reserved per VF +.It +8 MAC address filters for each VF on an Intel\(rg Ethernet 800 Series device +.It +16 MAC address filters for each VF on an Intel\(rg Ethernet 700 Series device +.It +Stateless offloads \- non\-tunneled checksums +.It +AVF device ID +.It +HW mailbox is used for VF to PF communications +.El +.Sh CONFIGURATION AND TUNING +.Ss Important System Configuration Changes +It is important to note that 100G operation can generate high +numbers of interrupts, often incorrectly being interpreted as +a storm condition in the kernel. +It is suggested that this be resolved by setting +.Va hw.intr_storm_threshold +to 0. .Pp -For more information on configuring this device, see -.Xr ifconfig 8 . +The default is 1000. .Pp -.Em NOTE : -This +Best throughput results are seen with a large MTU; use 9706 if possible. +The default number of descriptors per ring is 1024. +Increasing this may improve performance, depending on your use case. +.Ss Configuring for no iflib +.Xr iflib 4 +is a common framework for network interface drivers for +.Fx +that uses a shared set of sysctl names. +.Pp +The default .Nm -driver is only for Virtual Functions. -For 700 series Physical Functions, use the -.Xr ixl 4 -driver. -.Sh LOADER TUNABLES -Tunables can be set at the -.Xr loader 8 -prompt before booting the kernel or stored in -.Xr loader.conf 5 . -.Bl -tag -width indent -.It Va hw.iavf.rx_itr -The RX interrupt rate value, set to 62 (124 usec) by default. -.It Va hw.iavf.tx_itr -The TX interrupt rate value, set to 122 (244 usec) by default. -.It Va hw.iavf.enable_head_writeback -When the driver is finding the last TX descriptor processed by the hardware, -use a value written to memory by the hardware instead of scanning the -descriptor ring for completed descriptors. -Disabled by default; this mimics the "legacy" TX behavior found in -.Xr ixgbe 4 . -to ensure compatibility with future, non-700 series VF devices. +driver depends on it, but it can be compiled without it. +.Ss Jumbo Frames +Jumbo Frames support is enabled by changing the Maximum Transmission Unit (MTU) +to a value larger than the default value of 1500. +.Pp +Use the +.Xr ifconfig 8 +command to increase the MTU size. +.Pp +To confirm the MTU used between two specific devices, use +.Xr route 8 : +.Bd -literal -offset indent +route get +.Ed +.Pp +NOTE: +.Bl -bullet +.It +The maximum MTU setting for jumbo frames is 9706. +This corresponds to the maximum jumbo frame size of 9728 bytes. +.It +This driver will attempt to use multiple page-sized buffers to receive +each jumbo packet. +This should help to avoid buffer starvation issues when allocating receive +packets. +.It +Packet loss may have a greater impact on throughput when you use jumbo +frames. +If you observe a drop in performance after enabling jumbo frames, enabling +flow control may mitigate the issue. .El -.Sh SUPPORT -For general information and support, -go to the Intel support website at: -.Pa http://support.intel.com/ . +.Ss Checksum Offload +Checksum offloading supports both TCP and UDP packets and is supported for both +transmit and receive. +.Pp +TSO (TCP Segmentation Offload) supports both IPv4 and IPv6. +Both of these features are enabled and disabled via +.Xr ifconfig 8 . .Pp -If an issue is identified with this driver with a supported adapter, -email all the specific information related to the issue to -.Mt freebsd@intel.com . +NOTE: +.Bl -bullet -compact +.It +TSO requires Tx checksum; if Tx checksum is disabled then TSO will also +be disabled. +.El +.Ss LRO +LRO (Large Receive Offload) may provide Rx performance improvement. +However, it is incompatible with packet\-forwarding workloads. +You should carefully evaluate the environment and enable LRO when possible. +.Ss Rx and Tx Descriptor Rings +Allows you to set the Rx and Tx descriptor rings independently. +Set them via these +.Xr iflib 4 +sysctls: +.Bl -tag -width indent +.It dev.iavf.#.iflib.override_nrxds +.It dev.iavf.#.iflib.override_ntxds +.El +.Ss Link\-Level Flow Control (LFC) +The VF driver does not have access to flow control settings. +It must be managed from the host side. .Sh SEE ALSO .Xr arp 4 , +.Xr ice 4 , +.Xr iflib 4 , .Xr ixl 4 , .Xr netintro 4 , .Xr vlan 4 , -.Xr ifconfig 8 , -.Xr iflib 9 +.Xr ifconfig 8 +.Pp +See the +.Dq Intel\(rg Ethernet Adapters and Devices User Guide +for additional information on features. +It is available on the Intel website at either of the following: +.Bl -bullet +.It +.Lk https://cdrdv2.intel.com/v1/dl/getContent/705831 +.It +.Lk https://www.intel.com/content/www/us/en/download/19373/adapter\-user\-guide\-for\-intel\-ethernet\-adapters.html +.El +.Pp +For information on how to identify your adapter, and for the latest Intel +network drivers, refer to the Intel Support website: +.Aq Lk http://www.intel.com/support +.Sh CAVEATS +.Ss Driver Buffer Overflow Fix +The fix to resolve CVE\-2016\-8105, referenced in Intel SA\-00069 +.Aq Lk https://www.intel.com/content/www/us/en/security\-center/advisory/intel\-sa\-00069.html , +is included in this and future versions of the driver. +.Ss Network Memory Buffer Allocation +.Fx +may have a low number of network memory buffers (mbufs) by default. +If your mbuf value is too low, it may cause the driver to fail to initialize +and/or cause the system to become unresponsive. +You can check to see if the system is mbuf\-starved by running +.Li "netstat -m" . +Increase the number of mbufs by editing the lines below in +.Xr sysctl.conf 5 : +.Bd -literal -offset indent +kern.ipc.nmbclusters +kern.ipc.nmbjumbop +kern.ipc.nmbjumbo9 +kern.ipc.nmbjumbo16 +kern.ipc.nmbufs +.Ed +.Pp +The amount of memory that you allocate is system specific, and may require +some trial and error. +Also, increasing the following in +.Xr sysctl.conf 5 +could help increase +network performance: +.Bd -literal -offset indent +kern.ipc.maxsockbuf +net.inet.tcp.sendspace +net.inet.tcp.recvspace +net.inet.udp.maxdgram +net.inet.udp.recvspace +.Ed +.Ss UDP Stress Test Dropped Packet Issue +Under small packet UDP stress with the +.Nm +driver, the system may drop UDP packets due to socket buffers being full. +Setting the PF driver's Flow Control variables to the minimum may resolve the +issue. +.Ss Disable LRO when routing/bridging +LRO must be turned off when forwarding traffic. +.Sh SUPPORT +For general information, go to the Intel support website at +.Aq Lk http://www.intel.com/support/ . +.Pp +If an issue is identified with the released source code on a supported kernel +with a supported adapter, email the specific information related to the issue +to +.Aq Mt freebsd@intel.com . +.Sh LEGAL +Intel\(rg is a trademark or registered trademark of Intel Corporation +or its subsidiaries in the United States and / or other countries. +.Pp +Other names and brands may be claimed as the property of others. .Sh HISTORY The .Nm device driver first appeared in -.Fx 10.1 . -under the name "ixlv" +.Fx 10.1 +under the name +.Nm ixlv . It was converted to use -.Xr iflib 9 -and changed to its current name in -.Fx 12 . +.Xr iflib 4 +and renamed in +.Fx 12.4 . .Sh AUTHORS -.An -nosplit The .Nm -driver was written by -.An Jack Vogel Aq Mt jfv@freebsd.org -and -.An Eric Joyner Aq Mt erj@freebsd.org . -.Sh CAVEATS -This driver is supposed to function on VFs spawned from future network devices by Intel, -but at the time of this writing, has only been tested on the 700 series VFs. +driver was written by the +.An Intel Corporation Aq Mt freebsd@intel.com