Date: Mon, 8 Mar 2010 19:41:22 GMT From: Tom Hicks <thicks@averesystems.com> To: freebsd-gnats-submit@FreeBSD.org Subject: kern/144561: ixgbe driver errors Message-ID: <201003081941.o28JfM2p010984@www.freebsd.org> Resent-Message-ID: <201003081950.o28Jo1rn024446@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 144561
>Category: kern
>Synopsis: ixgbe driver errors
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: change-request
>Submitter-Id: current-users
>Arrival-Date: Mon Mar 08 19:50:01 UTC 2010
>Closed-Date:
>Last-Modified:
>Originator: Tom Hicks
>Release: 7.1-RELEASE-p3
>Organization:
Avere Systems
>Environment:
FreeBSD <obscured> 7.1-RELEASE-p3 FreeBSD 7.1-RELEASE-p3 #35: Mon Mar 8 10:32:29 EST 2010 <obscured>@<obscured>/amd64/compile/GENERIC amd64
>Description:
I have encountered the following problems in the HEAD version of the driver (but compiled to run on FreeBSD 7.1). All of the problems identified should exist in any version of the kernel though.
1. memory corruption of mbuf headers.
2. TCP segmentation offload not usable with checksum offload enabled
3. Add-in cards without SFP+ modules plugged in will fail ixgbe_attach (regression from 7.1 version of the driver)
4. Interfaces will be reset when an AF_INET address is added (via SIOCSIFADDR) (regression from 7.1 version of the driver)
5. Requested alignment (1 byte alignment) of the receive/transmit descriptor rings does not match the hardware alignment. This is probably not a real problem today, as the returned memory always appears to be 4K aligned anyhow, but could become a problem on new architectures.
6. Errored frames (crc errors, length errors) cause kernel panics on next use of the receive ring entry. This problem exists in all versions of the ixgbe driver (including the 7.1 version).
>How-To-Repeat:
For 1, try to use the driver with either an 82598 or 82599 based card. The kernel will panic quickly
For 2, try to enable TSO, send large segments by enqueuing 4K+ data in repeated send calls using a TCP socket, and monitor the tso counts in the driver.
For 3, don't plug in an SFP+ module. A corresponding interface (ie. ix0) will only show up when an SFP+ module is inserted.
For 4:
ifconfig ix0 (verify that status is active)
ifconfig ix0 192.168.1.2 netmask 255.255.255.0 (add address)
ifconfig ix0 (status will show no carrier for 5-10 seconds, then active again)
For 5: not really repeatable, this was found while looking for the memory corruption in 1 as part of a code review.
For 6, cause an errored packet to be injected (via a raw socket), or put in a fail point that causes a packet properly received to be rejected in ixgbe_rxeof.
>Fix:
Patch attached with submission follows:
Index: sys/dev/ixgbe/ixgbe.c
===================================================================
--- sys/dev/ixgbe/ixgbe.c (revision 204874)
+++ sys/dev/ixgbe/ixgbe.c (working copy)
@@ -576,7 +576,7 @@
} else if (error == IXGBE_ERR_SFP_NOT_SUPPORTED)
device_printf(dev,"Unsupported SFP+ Module\n");
- if (error) {
+ if (error && error!=IXGBE_ERR_SFP_NOT_PRESENT) {
error = EIO;
device_printf(dev,"Hardware Initialization Failure\n");
goto err_late;
@@ -944,6 +944,44 @@
switch (command) {
+ case SIOCSIFADDR:
+ IOCTL_DEBUGOUT("ioctl: SIOCTSIFADDR (Set Interface ADDR)");
+ {
+ struct ifaddr *ifa = (struct ifaddr *)data;
+ int avoid_reset = 0;
+
+#if defined(INET) || __FreeBSD_version < 800000
+ if (ifa->ifa_addr->sa_family == AF_INET) {
+ avoid_reset=1;
+ }
+#endif /* defined(INET) */
+
+#if defined(INET6) || __FreeBSD_version < 800000
+ if (ifa->ifa_addr->sa_family == AF_INET6) {
+ avoid_reset=1;
+ }
+#endif /* defined(INET6) */
+
+ if (avoid_reset) {
+ /*
+ * Since resetting hardware takes a very long time
+ * and results in link renegotiation, only
+ * initialize the hardware only when it is absolutely
+ * required.
+ */
+ ifp->if_flags |= IFF_UP;
+ if (!(ifp->if_drv_flags & IFF_DRV_RUNNING)) {
+ IXGBE_CORE_LOCK(adapter);
+ ixgbe_init_locked(adapter);
+ IXGBE_CORE_LOCK(adapter);
+ }
+ arp_ifinit(ifp, ifa);
+ } else {
+ error = ether_ioctl(ifp, command, data);
+ }
+ }
+ break;
+
case SIOCSIFMTU:
IOCTL_DEBUGOUT("ioctl: SIOCSIFMTU (Set Interface MTU)");
if (ifr->ifr_mtu > IXGBE_MAX_FRAME_SIZE - ETHER_HDR_LEN) {
@@ -1122,7 +1160,7 @@
if (ifp->if_capenable & IFCAP_TSO4)
ifp->if_hwassist |= CSUM_TSO;
if (ifp->if_capenable & IFCAP_TXCSUM)
- ifp->if_hwassist = (CSUM_TCP | CSUM_UDP);
+ ifp->if_hwassist |= (CSUM_TCP | CSUM_UDP);
/* Set MTU size */
if (ifp->if_mtu > ETHERMTU) {
@@ -2614,7 +2652,7 @@
int r;
r = bus_dma_tag_create(NULL, /* parent */
- 1, 0, /* alignment, bounds */
+ 16, 0, /* alignment, bounds */
BUS_SPACE_MAXADDR, /* lowaddr */
BUS_SPACE_MAXADDR, /* highaddr */
NULL, NULL, /* filter, filterarg */
@@ -4270,22 +4308,12 @@
#endif
}
} else {
- ifp->if_ierrors++;
- /* Reuse loaded DMA map and just update mbuf chain */
- mh->m_len = MHLEN;
- mh->m_flags |= M_PKTHDR;
- mh->m_next = NULL;
- mp->m_len = mp->m_pkthdr.len = adapter->rx_mbuf_sz;
- mp->m_data = mp->m_ext.ext_buf;
- if (mp->m_next) { /* Free chain */
- sendmp = mp->m_next;
- m_free(sendmp);
+ ++ifp->if_ierrors;
+ ++processed;
+ if (sendmp) {
+ m_freem(sendmp);
+ sendmp = NULL;
}
- mp->m_next = NULL;
- if (adapter->max_frame_size <=
- (MCLBYTES - ETHER_ALIGN))
- m_adj(mp, ETHER_ALIGN);
- sendmp = NULL;
}
bus_dmamap_sync(rxr->rxdma.dma_tag, rxr->rxdma.dma_map,
BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE);
Index: sys/dev/ixgbe/ixgbe.h
===================================================================
--- sys/dev/ixgbe/ixgbe.h (revision 204874)
+++ sys/dev/ixgbe/ixgbe.h (working copy)
@@ -176,7 +176,7 @@
#define MSIX_82599_BAR 4
#define IXGBE_TSO_SIZE 65535
#define IXGBE_TX_BUFFER_SIZE ((u32) 1514)
-#define IXGBE_RX_HDR 256
+#define IXGBE_RX_HDR 128
#define IXGBE_VFTA_SIZE 128
#define IXGBE_BR_SIZE 4096
#define CSUM_OFFLOAD 7 /* Bits in csum flags */
Index: sys/dev/ixgbe/ixgbe_82598.c
===================================================================
--- sys/dev/ixgbe/ixgbe_82598.c (revision 204874)
+++ sys/dev/ixgbe/ixgbe_82598.c (working copy)
@@ -255,10 +255,6 @@
ret_val = ixgbe_get_sfp_init_sequence_offsets(hw,
&list_offset,
&data_offset);
- if (ret_val != IXGBE_SUCCESS) {
- ret_val = IXGBE_ERR_SFP_NOT_SUPPORTED;
- goto out;
- }
break;
default:
break;
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201003081941.o28JfM2p010984>