Date: Tue, 23 Oct 2012 15:57:01 -0700 From: Garrett Cooper <yanegomi@gmail.com> To: freebsd-net@freebsd.org Cc: Jack F Vogel <jfv@freebsd.org> Subject: ixgb TSO performance degrades by ~30% between 7.4 and 8.2/9.0/9.1 Message-ID: <CAGH67wTjVw8O7i9OCvHVJjVjCAXsNHNhs0LobDi4G9iukgf%2BeQ@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hi, Doing some poking around at the ixgb driver with a card I have at $work using netperf and two machines hooked up over crossover, I discovered that while ixgb's throughput performance was fantastic on 7.3/7.4, thoughput performance of the card is degraded on 8.2/9.0/9.1 by ~30% (9400Mbps on 7.4 -> 6294Mbps on 9.0 for example). LRO performance on the other hand is fantastic and doesn't degrade with the card across FreeBSD versions. Performance remains constant with ixgb across 8.2/9.0/9.1. I didn't observe the CPU usage. More details: The machines are hooked up in the following configuration: ----------------------- -------------------- | Machine 1 | cxgb | <- 10Gbit fibre -> | ix1 | Machine 2 | ----------------------- --------------------- Machine Configuration: The card in Machine 2 is an 82599EB card according to pciconf -lv. /boot/loader.conf tunables (most of these are set according to 9.x defaults in order to establish a sane baseline): kern.ipc.nmbjumbo9=262144 kern.ipc.nmbjumbo16=262144 kern.ipc.nmbclusters=262144 kern.ipc.nmbjumbop=262144 kern.ipc.maxsockbuf=2097152 /etc/sysctl.conf tunables: net.inet.tcp.recvspace=65536 net.inet.tcp.recvspace_inc=16384 net.inet.tcp.recvspace_max=2097152 net.inet.tcp.sendspace=32768 net.inet.tcp.sendbuf_max=2097152 net.inet.tcp.sendbuf_inc=8192 Kernel Config: Machine 1 is running a custom version of FreeBSD. The version has been constant over the course of my testing. Can give vague details on the config, but can't give some specific details. Machine 2 is running 7.4/8.2/9.0/9.1 with a GENERIC kernel. Networking configuration: - Machine 1 has an IPv4 address of 10.10.10.1; IPv6 is not configured. The interface mtu is 1500. - Machine 2 has an IPv4 address of 10.10.10.2; IPv6 is not configured. The interface mtu is 1500. Netperf configuration: - netserver is run on both machines; I don't add any additional arguments to the netserver invocation so it just goes off and forks. - netperf is run like: netperf -cCjt TCP_STREAM -H <IP-ADDRESS> I was wondering if this was a known issue and/or others had seen similar problems with this card. I haven't gone into profiling the kernel yet with DTrace, but if no one gets back to me before sometime later on this week/next week that will be my next course of action for tracking down the source of the performance problem. Thanks! -Garrett
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAGH67wTjVw8O7i9OCvHVJjVjCAXsNHNhs0LobDi4G9iukgf%2BeQ>