Date: Mon, 30 Sep 2013 12:10:27 +0300 From: Andriy Kopystyansky <anri@polynet.lviv.ua> To: freebsd-net@freebsd.org Subject: Intel 82580 lagg(4) problem Message-ID: <20130930121027.Horde.taTEbLTlKvRSSUADQ3_q1_A@webmail.polynet.lviv.ua>
next in thread | raw e-mail | index | archive | help
Greetings! We've got a strange problem with lagg(4) interfaces built on Intel's 82580 chipset igb (dual port 1Gb card). We cannot send over lagg interface more than ~1Gb/s, yet receiving ~2GB over it. Looks like something indicating the problem can be seen here: anri@host:[8:13]~#ifstat -i lagg0 -i igb1 -i igb3 1 lagg0 igb1 igb3 KB/s in KB/s out KB/s in KB/s out KB/s in KB/s out 9116.50 26515.16 4147.70 28871.43 5004.86 23683.31 8423.08 26544.62 3853.22 28980.60 4594.21 23520.97 8796.48 26395.28 4248.46 28344.00 4567.42 23978.17 Note (IN) traffic - everything's ok, about 50% on each igb interfaces and total summary on lagg0. But (OUT) traffic looks weird - there is more traffic on the single igb1 than on lagg0! Tried with default driver came with the system, also the new one igb-2.3.10 from Intel site - no luck. Testing lagg interface built on other cards (em, bce) on the same machine shows expected normal behavior for both directions. --------------------------------------- anri@host:[9:02]~#uname -a FreeBSD host 9.1-RELEASE-p4 FreeBSD 9.1-RELEASE-p4 #5: Fri Sep 20 14:52:18 EEST 2013 anri@host:/usr/obj/usr/src/sys/NODEONE amd64 -------------------------------------- Interfaces: lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO> ether 90:e2:ba:36:95:1d inet 172.16.0.1 netmask 0xffffffc0 broadcast 172.16.0.63 media: Ethernet autoselect status: active laggproto lacp lagghash l2,l3,l4 laggport: igb3 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> laggport: igb1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO> ether 90:e2:ba:36:95:1d media: Ethernet autoselect (1000baseT <full-duplex>) status: active igb3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO> ether 90:e2:ba:36:95:1d media: Ethernet autoselect (1000baseT <full-duplex>) status: active --------------------------------------- Hardware: IBM 3550 with 2x Intel(R) Xeon(R) CPU E5430 @ 2.66GHz (amd64) pciconf -lv: igb1@pci0:19:0:1: class=0x020000 card=0x12b28086 chip=0x15168086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82580 Gigabit Network Connection' class = network subclass = ethernet --------------------------------------- /boot/loader.conf: hw.igb.rxd=4096 hw.igb.txd=4096 hw.igb.max_interrupt_rate=32000 hw.igb.num_queues=2 net.isr.maxthreads=8 net.isr.dispatch=direct net.isr.bindthreads=1 net.link.ifqmaxlen=10240 /etc/sysctl.conf: (relevant part) dev.igb.1.rx_processing_limit=-1 dev.igb.3.rx_processing_limit=-1 net.inet.ip.intr_queue_maxlen=512 net.route.netisr_maxqlen=4096 net.inet.ip.redirect=0 net.inet.ip.fastforwarding=1 kern.maxfiles=204800 kern.maxfilesperproc=200000 kern.ipc.nmbclusters=204800 kern.ipc.maxsockets=204800 Any tips, advices? thanks in advance, with best regards, Andriy Kopystyansky
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20130930121027.Horde.taTEbLTlKvRSSUADQ3_q1_A>