From owner-freebsd-performance@FreeBSD.ORG Wed Apr 21 19:52:37 2010 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 69B87106566C for ; Wed, 21 Apr 2010 19:52:37 +0000 (UTC) (envelope-from ssanders@softhammer.net) Received: from smtp-hq1.opnet.com (smtp-hq1.opnet.com [192.104.65.248]) by mx1.freebsd.org (Postfix) with ESMTP id 2BF608FC19 for ; Wed, 21 Apr 2010 19:52:36 +0000 (UTC) Received: from [172.16.12.251] (wtn12251.opnet.com [172.16.12.251]) by smtp.opnet.com (Postfix) with ESMTPSA id C1AC117E80B6; Wed, 21 Apr 2010 15:52:35 -0400 (EDT) Message-ID: <4BCF5783.9050007@softhammer.net> Date: Wed, 21 Apr 2010 15:52:35 -0400 From: Stephen Sanders User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.9) Gecko/20100317 Lightning/1.0b1 Thunderbird/3.0.4 MIME-Version: 1.0 To: Jack Vogel References: <4BCF0C9A.10005@softhammer.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Brandon Gooch , freebsd-performance@freebsd.org Subject: Re: FreeBSD 8.0 ixgbe Poor Performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Apr 2010 19:52:37 -0000 I'd be most pleased to get near 9k. I'm running FreeBSD 8.0 amd64 on both of the the test hosts. I've reset the configurations to system default as I was getting no where with sysctl and loader.conf settings. The motherboards have been configured to do MSI interrupts. The S5000PAL has a MSI to old style interrupt BIOS setting that confuses the driver interrupt setup. The 10Gbps cards should be plugged into the 8x PCI-E slots on both hosts. I'm double checking that claim right now and will get back later. Thanks On 4/21/2010 2:13 PM, Jack Vogel wrote: > When you get into the 10G world your performance will only be as good > as your weakest link, what I mean is if you connect to something that has > less than stellar bus and/or memory performance it is going to throttle > everything. > > Running back to back with two good systems you should be able to get > near line rate (9K range). Things that can effect that: 64 bit kernel, > TSO, LRO, how many queues come to mind. The default driver config > should get you there, so tell me more about your hardware/os config?? > > Jack > > > > On Wed, Apr 21, 2010 at 8:04 AM, Brandon Gooch > wrote: > > >> On Wed, Apr 21, 2010 at 9:32 AM, Stephen Sanders >> wrote: >> >>> I am running speed tests on a pair of systems equipped with Intel 10Gbps >>> cards and am getting poor performance. >>> >>> iperf and tcpdump testing indicates that the card is running at roughly >>> 2.5Gbps max transmit/receive. >>> >>> My attempts at turning fiddling with netisr, polling, and varying the >>> buffer sizes has been fruitless. I'm sure there is something that I'm >>> missing so I'm hoping for suggestions. >>> >>> There are two systems that are connected head to head via cross over >>> cable. The two systems have the same hardware configuration. The >>> hardware is as follows: >>> >>> 2 Intel E5430 (Quad core) @ 2.66 Ghz >>> Intel S5000PAL Motherboard >>> 16GB Memory >>> >>> My iperf command line for the client is: >>> >>> iperf -t 10 -c 169.0.0.1 -w 2.5M -l 2.5M >>> >>> My TCP dump test command lines are: >>> >>> tcpdump -i ix0 -w/dev/null >>> tcpreplay -i ix0 -t -l 0 -K ./test.pcap >>> >> If you're running 8.0-RELEASE, you might try updating to 8-STABLE. >> Jack Vogel recently committed updated Intel NIC driver code: >> >> http://svn.freebsd.org/viewvc/base/stable/8/sys/dev/ixgbe/ >> >> -Brandon >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to " >> freebsd-performance-unsubscribe@freebsd.org" >> >> > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > >