Date: Tue, 10 Jul 2012 10:35:08 +0200 From: Damien Fleuriot <ml@my.gd> To: freebsd-net@freebsd.org Subject: Re: lacp lagg port flags do not show correctly resulting in poor traffic distribution/performance Message-ID: <4FFBE93C.3080808@my.gd> In-Reply-To: <20120710071011.GB91639@DataIX.net> References: <5E4F49720D0BAD499EE1F01232234BA877435B2E28@AVEXMB1.qlogic.org> <20120710071011.GB91639@DataIX.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 7/10/12 9:10 AM, Jason Hellenthal wrote: > > > On Mon, Jul 09, 2012 at 05:38:24PM -0700, Adarsh Joshi wrote: >> Hi, >> >> I am trying to configure lacp lagg interfaces with 2 systems connected back to back as follows: >> >> Ifconfig lagg0 create >> Ifconfig lagg0 laggproto lacp laggport ql0 laggport ql1 192.168.100.1 netmask 255.255.255.0 >> >> Sometimes, the lag interface comes up correctly but sometimes the laggport flags do not show properly. Instead of 1c<ACTIVE,COLLECTING,DISTRIBUTING>, it shows values of 18. I have seen similar issues reported on various forums with no solution. >> Looking at the lagg driver code and reading the standard, I thought the laggport flags ( defined in if_lagg.h) are based on the LACP_STATE_BITS in file ieee8023ad_lacp.h. But the following ifconfig -v output does not make any sense to me. >> >> My concern is that when all the interfaces show flags as 1c, the traffic is distributed across both the interfaces uniformly and I get aggregated throughput. If not, the traffic flows only on 1 interface. >> >> Is this a bug? How do I solve this? Or am I doing something wrong? >> >> I am using Free-BSD 9.0 release. >> >> System 1: >> # ifconfig -v lagg0 >> lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 >> options=13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4> >> ether 00:0e:1e:08:05:20 >> inet 192.168.100.1 netmask 0xffffff00 broadcast 192.168.100.255 >> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> >> media: Ethernet autoselect >> status: active >> groups: lagg >> laggproto lacp >> lag id: [(8000,00-0E-1E-08-05-20,0213,0000,0000), >> (8000,00-0E-1E-04-2C-F0,0213,0000,0000)] >> laggport: ql1 flags=18<COLLECTING,DISTRIBUTING> state=7D >> [(8000,00-0E-1E-08-05-20,0213,8000,000F), >> (FFFF,00-00-00-00-00-00,0000,FFFF,0000)] >> laggport: ql0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D >> [(8000,00-0E-1E-08-05-20,0213,8000,000E), >> (8000,00-0E-1E-04-2C-F0,0213,8000,000E)] >> >> System 2: >> >> # ifconfig -v lagg0 >> lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 >> options=13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4> >> ether 00:0e:1e:04:2c:f0 >> inet 192.168.100.2 netmask 0xffffff00 broadcast 192.168.100.255 >> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> >> media: Ethernet autoselect >> status: active >> groups: lagg >> laggproto lacp >> lag id: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000), >> (FFFF,00-00-00-00-00-00,0000,0000,0000)] >> laggport: ql1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=7D >> [(8000,00-0E-1E-04-2C-F0,0213,8000,000F), >> (FFFF,00-00-00-00-00-00,0000,FFFF,0000)] >> laggport: ql0 flags=18<COLLECTING,DISTRIBUTING> state=3D >> [(8000,00-0E-1E-04-2C-F0,0213,8000,000E), >> (8000,00-0E-1E-08-05-20,0213,8000,000E)] >> >> > > Just for reference ... (stable/8 @ r238264) > > lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > options=80048<VLAN_MTU,POLLING,LINKSTATE> > ether 00:0c:41:21:1d:b5 > inet 192.168.XX.X netmask 0xffffff00 broadcast 192.168.XX.XXX > media: Ethernet autoselect > status: active > groups: lagg > laggproto lacp lagghash l2,l3,l4 > lag id: [(8000,00-0C-41-21-1D-B5,00E6,0000,0000), > (FFFF,00-00-00-00-00-00,0000,0000,0000)] > laggport: dc1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=7D > [(8000,00-0C-41-21-1D-B5,00E6,8000,0002), > (FFFF,00-00-00-00-00-00,0000,FFFF,0000)] > laggport: dc0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=7D > [(8000,00-0C-41-21-1D-B5,00E6,8000,0001), > (FFFF,00-00-00-00-00-00,0000,FFFF,0000)] > > > They have had flags = 1c for quite some time and state = 7D > > And just to show the variation ... > > > dc0: > yesterday 8.53 MiB / 2.61 MiB / 11.14 MiB > today 693 KiB / 156 KiB / 849 KiB > > dc1: > yesterday 19.00 MiB / 1.79 MiB / 20.78 MiB > today 496 KiB / 103 KiB / 599 KiB > > lagg0: > yesterday 27.53 MiB / 3.71 MiB / 31.24 MiB > today 1.16 MiB / 172 KiB / 1.33 MiB > > > I believe (know) there has been some changes in the LAgg code in > stable/9 and stable/8 recently so you may want to check into that. > > Given this is LAgg and LACP you will see some variation regardless but I > recall a point that it seemed like one interface was being favored over > the other quite repeatedly or obsessively that had me second guessing > whether it was doing the right thing. > > LACP in Cisco is quite different than how we treat it here in FreeBSD as > it tends to use the interfaces quite evenly all the time so that also > has me second guessing whether the right thing is happening here. ( in > PAgP and LACP modes ). > Note that you can configure the way the switch load balances traffic thus: switch(config)#port-channel load-balance ? dst-ip Dst IP Addr dst-mac Dst Mac Addr src-dst-ip Src XOR Dst IP Addr src-dst-mac Src XOR Dst Mac Addr src-ip Src IP Addr src-mac Src Mac Addr You may also run simulations on the switch to see what interface from a port-channel would be used: switch#test etherchannel load-balance interface po48 ip 10.1.2.3 88.190.45.1 Would select Te1/1/2 of Po48 switch#test etherchannel load-balance interface po48 ip 10.1.2.3 88.190.45.9 Would select Te2/1/2 of Po48 switch#test etherchannel load-balance interface po48 ip 10.1.2.3 88.190.45.10 Would select Te2/1/2 of Po48 Hope this helps.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4FFBE93C.3080808>