Date: Tue, 23 Oct 2012 17:01:20 -0700 From: Garrett Cooper <yanegomi@gmail.com> To: Jack Vogel <jfvogel@gmail.com> Cc: Jack F Vogel <jfv@freebsd.org>, freebsd-net@freebsd.org Subject: Re: ixgbe TSO performance degrades by ~30% between 7.4 and 8.2/9.0/9.1 Message-ID: <CAGH67wRgp4g8yPFGyd8e-6WTG1qOKKXBS2atn6Uxinnj=RUBOA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
On Tue, Oct 23, 2012 at 4:58 PM, Jack Vogel <jfvogel@gmail.com> wrote:
> It you mean the ixgbe driver please call it that, the IS an ixgb driver
> which is for
> VERY old PCI cards, from the context I assume you mean the newer hardware :)
Yeah... I meant ixgbe. Subject line fixed :).
Thanks!
-Garrett
> On Tue, Oct 23, 2012 at 3:57 PM, Garrett Cooper <yanegomi@gmail.com> wrote:
>>
>> Hi,
>>
>> Doing some poking around at the ixgb driver with a card I have at
>> $work using netperf and two machines hooked up over crossover, I
>> discovered that while ixgb's throughput performance was fantastic on
>> 7.3/7.4, thoughput performance of the card is degraded on 8.2/9.0/9.1
>> by ~30% (9400Mbps on 7.4 -> 6294Mbps on 9.0 for example). LRO
>> performance on the other hand is fantastic and doesn't degrade with
>> the card across FreeBSD versions. Performance remains constant with
>> ixgb across 8.2/9.0/9.1. I didn't observe the CPU usage.
>>
>> More details:
>>
>> The machines are hooked up in the following configuration:
>>
>> -----------------------
>> --------------------
>> | Machine 1 | cxgb | <- 10Gbit fibre -> | ix1 | Machine 2 |
>> -----------------------
>> ---------------------
>>
>> Machine Configuration:
>>
>> The card in Machine 2 is an 82599EB card according to pciconf -lv.
>>
>> /boot/loader.conf tunables (most of these are set according to 9.x
>> defaults in order to establish a sane baseline):
>>
>> kern.ipc.nmbjumbo9=262144
>> kern.ipc.nmbjumbo16=262144
>> kern.ipc.nmbclusters=262144
>> kern.ipc.nmbjumbop=262144
>> kern.ipc.maxsockbuf=2097152
>>
>> /etc/sysctl.conf tunables:
>>
>> net.inet.tcp.recvspace=65536
>> net.inet.tcp.recvspace_inc=16384
>> net.inet.tcp.recvspace_max=2097152
>> net.inet.tcp.sendspace=32768
>> net.inet.tcp.sendbuf_max=2097152
>> net.inet.tcp.sendbuf_inc=8192
>>
>> Kernel Config:
>>
>> Machine 1 is running a custom version of FreeBSD. The version has been
>> constant over the course of my testing. Can give vague details on the
>> config, but can't give some specific details.
>> Machine 2 is running 7.4/8.2/9.0/9.1 with a GENERIC kernel.
>>
>> Networking configuration:
>>
>> - Machine 1 has an IPv4 address of 10.10.10.1; IPv6 is not configured.
>> The interface mtu is 1500.
>> - Machine 2 has an IPv4 address of 10.10.10.2; IPv6 is not configured.
>> The interface mtu is 1500.
>>
>> Netperf configuration:
>>
>> - netserver is run on both machines; I don't add any additional
>> arguments to the netserver invocation so it just goes off and forks.
>> - netperf is run like: netperf -cCjt TCP_STREAM -H <IP-ADDRESS>
>>
>> I was wondering if this was a known issue and/or others had seen
>> similar problems with this card. I haven't gone into profiling the
>> kernel yet with DTrace, but if no one gets back to me before sometime
>> later on this week/next week that will be my next course of action for
>> tracking down the source of the performance problem.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAGH67wRgp4g8yPFGyd8e-6WTG1qOKKXBS2atn6Uxinnj=RUBOA>