Date: Fri, 13 Jan 2012 12:49:26 -0500 From: Iordan Iordanov <iordan@cdf.toronto.edu> To: freebsd-bugs@freebsd.org Cc: CDF Admin <admin@cdf.toronto.edu> Subject: Intel x520-t2: interrupt storm detected + throttling Message-ID: <4F106EA6.3000003@cdf.toronto.edu>
next in thread | raw e-mail | index | archive | help
Hello, I decided to send a separate message about another problem that we witnessed while testing 10GbaseT connectivity between a pair of Intel x520-t2 interconnected with cat6 cable. While we were testing we got a multitude of kernel messages of this form: interrupt storm detected on "irq271:"; throttling interrupt source interrupt storm detected on "irq271:"; throttling interrupt source interrupt storm detected on "irq271:"; throttling interrupt source An easy way for us to reproduce the problem was this. Start an nc process listening on one machine: fw4# nc -l 8080 > /dev/null Then, start piping from /dev/zero into nc from the other machine: fw3# dd if=/dev/zero bs=4096k | nc 192.168.1.200 8080 Almost immediately, on the server-side, we start getting the messages. The client-side is silent. When I reversed the client/server roles, the scenario was repeated the other way around to prove to myself it's the server that reports the issue. The transfer speed for a single process over TCP appears to be about 240 megabytes/s or about 2 gigabits/s. However, with multiple nc processes we are able to push this to about 4 gigabits/s (similar to our iperf tests). A curious, and probably easy to explain fact is that when doing transfers over UDP (the -u flag to nc), we do not get the "interrupt storm" messages, and indeed, the interrupt rate in "top" is 1/2 that of when we do the TCP test under the same conditions. More information about our hardware is: Motherboard: Supermicro H8DGU-F CPU: AMD 6128 8-core Memory: 16GB DDR3 NICs: Intel x520-t2 10GbaseT adapters An ifconfig ix1 output for one of the machines: fw4# ifconfig ix1 ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO> ether 00:1b:21:d6:19:51 inet 192.168.1.200 netmask 0xffffff00 broadcast 192.168.1.255 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect (autoselect <full-duplex>) status: active Many thanks for any help/action on this! Sincerely, Iordan Iordanov CDF System Administrator University of Toronto
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4F106EA6.3000003>