From owner-freebsd-current@FreeBSD.ORG Sun May 9 17:14:36 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BFE47106566C for ; Sun, 9 May 2010 17:14:36 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-ww0-f54.google.com (mail-ww0-f54.google.com [74.125.82.54]) by mx1.freebsd.org (Postfix) with ESMTP id 28EB08FC12 for ; Sun, 9 May 2010 17:14:35 +0000 (UTC) Received: by wwd20 with SMTP id 20so642820wwd.13 for ; Sun, 09 May 2010 10:14:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=lXkL1qyj7A9oFe0rsYDZ8OQ3WAOLznn/ICzLvQVVq4M=; b=QF8hi3dEXJjC1VKNuAMk7Dxza8OO2DQQa4LV9pcmFpIhMQHU5zsF9n4+kVC68LNFXS 8kgmhhTJUtjZz4JH/846tlyQ6r3+VYL/hKopE6ZTTHg+nou72QM3ALXZ/aEungmN5yT8 Fn0tA2y+s/OquWYf3K8D5k+0miGMNw835Ikok= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=vnHel0k4Sd6j1+OpQ/YosVayvc3QKFO66m8zPDCVJeT+J048+Zywh+wu+6ppQ4HYQu gE4/WR5do+IfmJCAdjEK6TG1VGf/h263XvHtSfY/SLp5LgJHmNnf63DyRx+TLY0oHZDB YsO29BHh07T6krlg7CBWUehGhUazMBa9wYQMk= MIME-Version: 1.0 Received: by 10.216.161.14 with SMTP id v14mr1650557wek.179.1273425275228; Sun, 09 May 2010 10:14:35 -0700 (PDT) Received: by 10.216.29.129 with HTTP; Sun, 9 May 2010 10:14:35 -0700 (PDT) In-Reply-To: <4BE6864B.5060906@hostedcontent.com> References: <4BE565E5.9030505@hostedcontent.com> <4BE59434.9070308@hostedcontent.com> <4BE599B0.60203@hostedcontent.com> <4BE59DBD.4000105@hostedcontent.com> <4BE6864B.5060906@hostedcontent.com> Date: Sun, 9 May 2010 10:14:35 -0700 Message-ID: From: Jack Vogel To: joe Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Ian FREISLICH , Fabien Thomas , freebsd-current@freebsd.org Subject: Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board. X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 May 2010 17:14:36 -0000 On Sun, May 9, 2010 at 2:54 AM, joe wrote: > On 05/08/2010 02:21 PM, Jack Vogel wrote: > >> The cable, its a simple thing but make SURE you try that, a slightly >> damaged one can do weird things and its quick to check, don't overlook >> it. >> >> Jack >> >> >> On Sat, May 8, 2010 at 10:22 AM, joe > > wrote: >> >> On 05/08/2010 01:53 PM, Jack Vogel wrote: >> >> I still am not clear on this system, how many ports are on it, >> and its >> an 82576? >> Sounds to me like you've proven its not on the box if you can do >> fine >> when its >> on its own. So change ports in the switch, as I said, change >> cables, must be >> something in that environment. >> >> Jack >> >> >> On Sat, May 8, 2010 at 10:04 AM, joe > >> >> >> wrote: >> >> On 05/08/2010 01:31 PM, Jack Vogel wrote: >> >> Looks like something to do with system C, you might >> isolate it, >> and try >> a back >> to back connection with its NICs, change cables, look at >> BIOS >> settings, >> change >> the slot the nic is in... All just off the top of my head. >> >> Jack >> >> >> On Sat, May 8, 2010 at 9:41 AM, joe >> >> > >> >> >>> >> >> wrote: >> >> On 05/08/2010 11:17 AM, Ian FREISLICH wrote: >> >> joe wrote: >> >> On 05/08/2010 06:55 AM, Ian FREISLICH wrote: >> >> joe wrote: >> >> I have just tried your >> suggeston and >> it has >> no effect for me ;( >> >> >> Do you have another brand of NIC that >> you can >> try? At >> least that >> will isolate whether it's igb(4) or >> something else. >> >> >> I will grab a new nic today and try...my >> options are >> limited >> though. >> Here are the nics i can get my hands on >> >> TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter >> (supported >> by fbsd?) >> >> >> Based on the RTL8168B chip. Should be supported >> by the >> re(4) >> driver. >> >> Intel (EXPI9301CT) Gigabit CT Desktop >> Adapter (yet >> another >> intel nic) >> >> >> i82574L chip. Should be supported by the em(4) >> driver. >> I have had >> good performance in the past with this driver >> and less than >> satisfactory performance with the igb(4) driver. >> >> That may not be your problem though. Before you >> go out >> and buy, >> have a look at the amount of interrupt time your >> slow >> machine spends >> in 'top' or 'systat -vm'. systat will also show >> the >> interrupt rate >> for each driver, perhaps it's not doing >> interrupt moderation >> properly. >> This will manifest as more than about a 1000 per >> second. >> There are >> loader tunables for the driver to increase the >> number of >> transfer >> descriptors and to tune interrupt moderation. >> >> You could try running trafshow (port) on the >> interface while >> performing the transfer. Perhaps promiscuous >> mode will >> turn off >> some hardware feature that will improve things. >> It may >> however >> break hardware vlanning as it does on my 82575GB >> 4 port >> igb card. >> >> Ian >> >> -- >> Ian Freislich >> >> >> I bought those two cards anyways, im in a rush to >> figure out >> this >> problem. That being said i am still encountering the >> exact same >> problem regardless on which network card i am >> running. I am at a >> complete loss. I am about to try a raid card to see >> if the >> problem >> might lay within the onboard sata ports. I did pull the >> server and >> brought it home so that i can test more things quicker. >> >> I am going to try using a raid card instead of the >> onboard sata >> ports and see if i still encounter the same problem. >> I would >> love >> any suggestions you may have on where to go from here >> to >> figure out >> where the problem might be. >> >> joe >> >> _______________________________________________ >> freebsd-current@freebsd.org >> > > >> > >> > >> >> >> mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to >> "freebsd-current-unsubscribe@freebsd.org >> >> > > >> > >> > >>" >> >> >> >> I think it might have something to so with the nics / >> switch, and >> their features. I brought the box home, plugged into my gb >> switch, >> and i am able to FTP data to the server at around 35MB/sec. >> >> I dont know what would cause this other than some sort of >> issue with >> the the 3 different types of nics and the switch i am using. >> >> Any suggestions? >> >> >> >> There are two embedded intel 82576 nics on this motherboard. I do >> believe i have proven it is not the box itself as it is capable of >> high incoming throughput. I have other servers on the switch which >> do 55MB/sec without issues. I believe it is a combination of this >> server and/or the nics i have and the switch i am using. It's the >> only logical explanation if i get the desired throughput on my home >> switch but not on the switch that is collocated. I will try updating >> the firmware of the switch tonight as well as bringing the switch i >> use at home with me. >> >> >> > Here is a follow-up just incase anyone ever encounters this problem again. > I updated the firmware on the switch, made sure jumbo packets were enabled, > and the switch was restarted. I'm now seeing the throughput i had expected > on this box! > > 897755008 bytes received in 00:11 (72.74 MB/s) > > Once again, thank you all for the help! > > Joe > Glad the issue is resolved Joe. Jack