Date: Mon, 30 Sep 2013 02:51:34 -0700 From: Adrian Chadd <adrian@freebsd.org> To: Andriy Kopystyansky <anri@polynet.lviv.ua> Cc: FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: Intel 82580 lagg(4) problem Message-ID: <CAJ-Vmo=00SzCNtcKAxnQpWXdDi4CGo5=_u_Wpejt8SA0Dh=M%2Bw@mail.gmail.com> In-Reply-To: <20130930121027.Horde.taTEbLTlKvRSSUADQ3_q1_A@webmail.polynet.lviv.ua> References: <20130930121027.Horde.taTEbLTlKvRSSUADQ3_q1_A@webmail.polynet.lviv.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, There's a lagg/lacp option to not hash based on the incoming assigned flowid on the incoming mbuf when transmitting but instead calculate its own hash for outbound transmission. It's net.link.lagg.0.use_flowid . Set it to 0 and retry. Thanks, -adrian On 30 September 2013 02:10, Andriy Kopystyansky <anri@polynet.lviv.ua>wrote: > Greetings! > We've got a strange problem with lagg(4) interfaces built on Intel's 82580 > chipset igb (dual port 1Gb card). > We cannot send over lagg interface more than ~1Gb/s, yet receiving ~2GB > over it. > Looks like something indicating the problem can be seen here: > > anri@host:[8:13]~#ifstat -i lagg0 -i igb1 -i igb3 1 > lagg0 igb1 igb3 > KB/s in KB/s out KB/s in KB/s out KB/s in KB/s out > 9116.50 26515.16 4147.70 28871.43 5004.86 23683.31 > 8423.08 26544.62 3853.22 28980.60 4594.21 23520.97 > 8796.48 26395.28 4248.46 28344.00 4567.42 23978.17 > > Note (IN) traffic - everything's ok, about 50% on each igb interfaces and > total summary on lagg0. > But (OUT) traffic looks weird - there is more traffic on the single igb1 > than on lagg0! > Tried with default driver came with the system, also the new one > igb-2.3.10 from Intel site - no luck. > > Testing lagg interface built on other cards (em, bce) on the same machine > shows expected normal behavior for both directions. > ------------------------------**--------- > anri@host:[9:02]~#uname -a > FreeBSD host 9.1-RELEASE-p4 FreeBSD 9.1-RELEASE-p4 #5: Fri Sep 20 14:52:18 > EEST 2013 anri@host:/usr/obj/usr/src/**sys/NODEONE amd64 > ------------------------------**-------- > Interfaces: > lagg0: flags=8843<UP,BROADCAST,**RUNNING,SIMPLEX,MULTICAST> metric 0 mtu > 1500 > options=401bb<RXCSUM,TXCSUM,**VLAN_MTU,VLAN_HWTAGGING,JUMBO_** > MTU,VLAN_HWCSUM,TSO4,VLAN_**HWTSO> > ether 90:e2:ba:36:95:1d > inet 172.16.0.1 netmask 0xffffffc0 broadcast 172.16.0.63 > media: Ethernet autoselect > status: active > laggproto lacp lagghash l2,l3,l4 > laggport: igb3 flags=1c<ACTIVE,COLLECTING,**DISTRIBUTING> > laggport: igb1 flags=1c<ACTIVE,COLLECTING,**DISTRIBUTING> > > igb1: flags=8843<UP,BROADCAST,**RUNNING,SIMPLEX,MULTICAST> metric 0 mtu > 1500 > options=401bb<RXCSUM,TXCSUM,**VLAN_MTU,VLAN_HWTAGGING,JUMBO_** > MTU,VLAN_HWCSUM,TSO4,VLAN_**HWTSO> > ether 90:e2:ba:36:95:1d > media: Ethernet autoselect (1000baseT <full-duplex>) > status: active > > igb3: flags=8843<UP,BROADCAST,**RUNNING,SIMPLEX,MULTICAST> metric 0 mtu > 1500 > options=401bb<RXCSUM,TXCSUM,**VLAN_MTU,VLAN_HWTAGGING,JUMBO_** > MTU,VLAN_HWCSUM,TSO4,VLAN_**HWTSO> > ether 90:e2:ba:36:95:1d > media: Ethernet autoselect (1000baseT <full-duplex>) > status: active > > ------------------------------**--------- > Hardware: IBM 3550 with 2x Intel(R) Xeon(R) CPU E5430 @ 2.66GHz (amd64) > pciconf -lv: > igb1@pci0:19:0:1: class=0x020000 card=0x12b28086 chip=0x15168086 > rev=0x01 hdr=0x00 > vendor = 'Intel Corporation' > device = '82580 Gigabit Network Connection' > class = network > subclass = ethernet > > ------------------------------**--------- > /boot/loader.conf: > hw.igb.rxd=4096 > hw.igb.txd=4096 > hw.igb.max_interrupt_rate=**32000 > hw.igb.num_queues=2 > net.isr.maxthreads=8 > net.isr.dispatch=direct > net.isr.bindthreads=1 > net.link.ifqmaxlen=10240 > > /etc/sysctl.conf: (relevant part) > > dev.igb.1.rx_processing_limit=**-1 > dev.igb.3.rx_processing_limit=**-1 > net.inet.ip.intr_queue_maxlen=**512 > net.route.netisr_maxqlen=4096 > net.inet.ip.redirect=0 > net.inet.ip.fastforwarding=1 > kern.maxfiles=204800 > kern.maxfilesperproc=200000 > kern.ipc.nmbclusters=204800 > kern.ipc.maxsockets=204800 > > Any tips, advices? > > thanks in advance, with best regards, > Andriy Kopystyansky > > > ______________________________**_________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/**mailman/listinfo/freebsd-net<http://lists.freebsd.org/mailman/listinfo/freebsd-net>; > To unsubscribe, send any mail to "freebsd-net-unsubscribe@**freebsd.org<freebsd-net-unsubscribe@freebsd.org> > " >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmo=00SzCNtcKAxnQpWXdDi4CGo5=_u_Wpejt8SA0Dh=M%2Bw>