Date: Thu, 27 Feb 2014 19:02:01 -0700 From: Kevin Bowling <kevin.bowling@kev009.com> To: freebsd-net@freebsd.org Subject: Re: FreeBSD 10 network flapping, ix driver unreliable? Message-ID: <leoqmd$e7i$2@ger.gmane.org> In-Reply-To: <ldohqb$s2c$1@ger.gmane.org> References: <ldohqb$s2c$1@ger.gmane.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2/15/2014 1:14 PM, Kevin Bowling wrote: > Hi, > > I have FreeBSD 10.0-RELEASE installed on two Dell C6100 nodes. Each > node has an Intel X520-DA2 dual port 10gig card. One of the ports on > each go to a switch using direct attach coaxial cables. The other port > is directly connected between the two nodes (think crossover in twisted > pair terminology) again using direct attach coaxial cables. > > On both machines, and on both ports (including the "crossover"), the > links flap several times per day. > > I've pasted the output of lspci -vv and dmesg here: > https://gist.github.com/kev009/9024442 > > There's nothing outstanding about the setup otherwise. I suspected some > interaction with the switch initially but the "crossover" has eliminated > that suspicion. > > It seems the ix driver is not very reliable under common conditions, > i.e. https://forums.freebsd.org/viewtopic.php?f=7&t=44570 and a search > of this list. Any recommendations or tests? > > Regards, > Kevin Bowling > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > After descending a rather dark rabbit hole, I'm pleased to have found a simple solution! On some of these cards, there are known firmware problems. The driver sometimes tries to compensate, but these code paths probably receive far less testing and look pretty fragile. It seems some version of NIC firmware are particularly flaky with DA cables. Some spam in the Linux dmesg led me to this, which does not appear to be in the FreeBSD ixgbe driver, although I did not stay in Linux long enough to see if it fully fixed the problem: http://markmail.org/message/ivsjxoyfbvzv7mvo Instead, I found a way to update the microcode. My card and server are Dell and I was able to use this live cd to do the firmware upgrade: http://linux.dell.com/files/openmanage-contributions/om-firmware-live/ which applies a NIC firmware package like http://www.dell.com/support/drivers/us/en/19/driverdetails?driverid=HKK1W Both of my systems appear to be stable; I'll comment if there are issues over the next few days. Intel is less than forthcoming about these microcode updates; I'm not sure if the preboot code (https://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&ProdId=3591&DwnldID=19186&ProductFamily=Network+Connectivity&ProductLine=Intel%C2%AE+Server+Adapters&ProductProduct=Intel%C2%AE+Ethernet+Converged+Network+Adapter+X520+Series&lang=eng) contains the NIC microcode or if you must obtain it from a vendor. Regards, Kevin Bowling
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?leoqmd$e7i$2>