Date: Sat, 08 May 2010 14:02:04 -0400 From: joe <joe@hostedcontent.com> To: Jack Vogel <jfvogel@gmail.com> Cc: Ian FREISLICH <ianf@clue.co.za>, Fabien Thomas <fabien.thomas@netasq.com>, freebsd-current@freebsd.org Subject: Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board. Message-ID: <4BE5A71C.8080609@hostedcontent.com> In-Reply-To: <AANLkTincmLc5qGkfnQsGWO8OGjaotR_wquRxIiBBLQz9@mail.gmail.com> References: <4BE565E5.9030505@hostedcontent.com> <4BE529FF.5000008@hostedcontent.com> <C15B47BC-AF5F-4AF4-899C-98155EC5A4D4@netasq.com> <E1OAhhO-00021o-DH@clue.co.za> <E1OAlmh-0002Dj-Mm@clue.co.za> <4BE59434.9070308@hostedcontent.com> <AANLkTikLmEwr15mPz9Hz1kB97dXUNhkOS8R47d68rm1z@mail.gmail.com> <4BE599B0.60203@hostedcontent.com> <AANLkTikVu2mfnLXzYEms4RO0toTvBctARq1lQ2ZmnVOT@mail.gmail.com> <4BE59DBD.4000105@hostedcontent.com> <AANLkTincmLc5qGkfnQsGWO8OGjaotR_wquRxIiBBLQz9@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 05/08/2010 02:21 PM, Jack Vogel wrote: > The cable, its a simple thing but make SURE you try that, a slightly > damaged one can do weird things and its quick to check, don't overlook > it. > > Jack > > > On Sat, May 8, 2010 at 10:22 AM, joe <joe@hostedcontent.com > <mailto:joe@hostedcontent.com>> wrote: > > On 05/08/2010 01:53 PM, Jack Vogel wrote: > > I still am not clear on this system, how many ports are on it, > and its > an 82576? > Sounds to me like you've proven its not on the box if you can do > fine > when its > on its own. So change ports in the switch, as I said, change > cables, must be > something in that environment. > > Jack > > > On Sat, May 8, 2010 at 10:04 AM, joe <joe@hostedcontent.com > <mailto:joe@hostedcontent.com> > <mailto:joe@hostedcontent.com <mailto:joe@hostedcontent.com>>> > wrote: > > On 05/08/2010 01:31 PM, Jack Vogel wrote: > > Looks like something to do with system C, you might > isolate it, > and try > a back > to back connection with its NICs, change cables, look at > BIOS > settings, > change > the slot the nic is in... All just off the top of my head. > > Jack > > > On Sat, May 8, 2010 at 9:41 AM, joe > <joe@hostedcontent.com <mailto:joe@hostedcontent.com> > <mailto:joe@hostedcontent.com <mailto:joe@hostedcontent.com>> > <mailto:joe@hostedcontent.com <mailto:joe@hostedcontent.com> > <mailto:joe@hostedcontent.com <mailto:joe@hostedcontent.com>>>> > > wrote: > > On 05/08/2010 11:17 AM, Ian FREISLICH wrote: > > joe wrote: > > On 05/08/2010 06:55 AM, Ian FREISLICH wrote: > > joe wrote: > > I have just tried your > suggeston and > it has > no effect for me ;( > > > Do you have another brand of NIC that > you can > try? At > least that > will isolate whether it's igb(4) or > something else. > > > I will grab a new nic today and try...my > options are > limited > though. > Here are the nics i can get my hands on > > TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter > (supported > by fbsd?) > > > Based on the RTL8168B chip. Should be supported > by the > re(4) > driver. > > Intel (EXPI9301CT) Gigabit CT Desktop > Adapter (yet > another > intel nic) > > > i82574L chip. Should be supported by the em(4) > driver. > I have had > good performance in the past with this driver > and less than > satisfactory performance with the igb(4) driver. > > That may not be your problem though. Before you > go out > and buy, > have a look at the amount of interrupt time your > slow > machine spends > in 'top' or 'systat -vm'. systat will also show the > interrupt rate > for each driver, perhaps it's not doing > interrupt moderation > properly. > This will manifest as more than about a 1000 per > second. > There are > loader tunables for the driver to increase the > number of > transfer > descriptors and to tune interrupt moderation. > > You could try running trafshow (port) on the > interface while > performing the transfer. Perhaps promiscuous > mode will > turn off > some hardware feature that will improve things. > It may > however > break hardware vlanning as it does on my 82575GB > 4 port > igb card. > > Ian > > -- > Ian Freislich > > > I bought those two cards anyways, im in a rush to > figure out > this > problem. That being said i am still encountering the > exact same > problem regardless on which network card i am > running. I am at a > complete loss. I am about to try a raid card to see > if the > problem > might lay within the onboard sata ports. I did pull the > server and > brought it home so that i can test more things quicker. > > I am going to try using a raid card instead of the > onboard sata > ports and see if i still encounter the same problem. > I would > love > any suggestions you may have on where to go from here to > figure out > where the problem might be. > > joe > > _______________________________________________ > freebsd-current@freebsd.org <mailto:freebsd-current@freebsd.org> > <mailto:freebsd-current@freebsd.org > <mailto:freebsd-current@freebsd.org>> > <mailto:freebsd-current@freebsd.org > <mailto:freebsd-current@freebsd.org> > <mailto:freebsd-current@freebsd.org > <mailto:freebsd-current@freebsd.org>>> > > mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org > <mailto:freebsd-current-unsubscribe@freebsd.org> > <mailto:freebsd-current-unsubscribe@freebsd.org > <mailto:freebsd-current-unsubscribe@freebsd.org>> > <mailto:freebsd-current-unsubscribe@freebsd.org > <mailto:freebsd-current-unsubscribe@freebsd.org> > <mailto:freebsd-current-unsubscribe@freebsd.org > <mailto:freebsd-current-unsubscribe@freebsd.org>>>" > > > > I think it might have something to so with the nics / > switch, and > their features. I brought the box home, plugged into my gb > switch, > and i am able to FTP data to the server at around 35MB/sec. > > I dont know what would cause this other than some sort of > issue with > the the 3 different types of nics and the switch i am using. > > Any suggestions? > > > > There are two embedded intel 82576 nics on this motherboard. I do > believe i have proven it is not the box itself as it is capable of > high incoming throughput. I have other servers on the switch which > do 55MB/sec without issues. I believe it is a combination of this > server and/or the nics i have and the switch i am using. It's the > only logical explanation if i get the desired throughput on my home > switch but not on the switch that is collocated. I will try updating > the firmware of the switch tonight as well as bringing the switch i > use at home with me. > > Thanks for the suggestion Jack! Replacing the cables and trying different ports on the switch was the first thing i did when first encountered this problem. One thing i did notice was that the switch has jumbo frames disabled while the cards had it enabled. I tried disabling it on the cards but it didnt seem to have an impact. I am hoping a firmware upgrade to the switch will resolve the problem and if not then i'll just replace the switch at the DC with the switch i have at home... Thank you (and everyone else) for all their help with this matter! Regards, Joe
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4BE5A71C.8080609>