Date: Tue, 10 Jul 2012 10:53:00 -0700 From: Adarsh Joshi <adarsh.joshi@qlogic.com> To: Andrew Boyer <aboyer@averesystems.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: RE: lacp lagg port flags do not show correctly resulting in poor traffic distribution/performance Message-ID: <5E4F49720D0BAD499EE1F01232234BA877435B2F66@AVEXMB1.qlogic.org> In-Reply-To: <5E4F49720D0BAD499EE1F01232234BA877435B2F2E@AVEXMB1.qlogic.org> References: <5E4F49720D0BAD499EE1F01232234BA877435B2E28@AVEXMB1.qlogic.org> <4C5FC147-2F49-4AE0-ADF3-C5381DE6580F@averesystems.com> <5E4F49720D0BAD499EE1F01232234BA877435B2F2E@AVEXMB1.qlogic.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Andrew,
Here are the logs with LACP_DEBUG defined in ieee802.3ad_lacp.c,
after typing
Ifconfig lagg0 create
ifconfig lagg0 laggproto lacp laggport ql0 laggport ql1 192.168.100.1 netma=
sk 255.255.255.0
I compiled it as a standalone driver by the way.
System 1:
# ifconfig -v lagg0
lagg0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 15=
00
options=3D13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4>
ether 00:0e:1e:08:05:20
inet 192.168.100.1 netmask 0xffffff00 broadcast 192.168.100.255
nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: active
groups: lagg
laggproto lacp
lag id: [(8000,00-0E-1E-08-05-20,01D3,0000,0000),
(8000,00-0E-1E-04-2C-F0,0213,0000,0000)]
laggport: ql1 flags=3D18<COLLECTING,DISTRIBUTING> state=3D7D
[(8000,00-0E-1E-08-05-20,01D3,8000,000D),
(FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
laggport: ql0 flags=3D1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D3D
[(8000,00-0E-1E-08-05-20,01D3,8000,000C),
(8000,00-0E-1E-04-2C-F0,0213,8000,000E)]
System 2:
# ifconfig -v lagg0
lagg0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 15=
00
options=3D13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4>
ether 00:0e:1e:04:2c:f0
inet 192.168.100.2 netmask 0xffffff00 broadcast 192.168.100.255
nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: active
groups: lagg
laggproto lacp
lag id: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),
(FFFF,00-00-00-00-00-00,0000,0000,0000)]
laggport: ql1 flags=3D1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D7D
[(8000,00-0E-1E-04-2C-F0,0213,8000,000F),
(FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
laggport: ql0 flags=3D18<COLLECTING,DISTRIBUTING> state=3D3D
[(8000,00-0E-1E-04-2C-F0,0213,8000,000E),
(8000,00-0E-1E-08-05-20,01D3,8000,000C)]
System 1 logs :
Jul 10 10:38:49 bsd-14 kernel: lacp_attach[738] : lacp attached
Jul 10 10:38:49 bsd-14 kernel: lacp_attach[740] : lacp_defined
Jul 10 10:38:49 bsd-14 kernel: lagg0: link state changed to UP
Jul 10 10:38:49 bsd-14 kernel: ql0: media changed 0x0 -> 0x100033, ether =
=3D 1, fdx =3D 1, link =3D 1
Jul 10 10:38:49 bsd-14 kernel: ql0: -> UNSELECTED
Jul 10 10:38:49 bsd-14 kernel: ql1: media changed 0x0 -> 0x100033, ether =
=3D 1, fdx =3D 1, link =3D 1
Jul 10 10:38:49 bsd-14 kernel: ql1: -> UNSELECTED
Jul 10 10:38:49 bsd-14 kernel: lacp_select_tx_port: no active aggregator
Jul 10 10:38:50 bsd-14 kernel: ql1: port lagid=3D[(8000,00-0E-1E-08-05-20,0=
1D3,8000,000D),(0000,00-00-00-00-00-00,0000,0000,0000)]
Jul 10 10:38:50 bsd-14 kernel: ql1: aggregator created
Jul 10 10:38:50 bsd-14 kernel: ql1: aggregator lagid=3D[(8000,00-0E-1E-08-0=
5-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)]
Jul 10 10:38:50 bsd-14 kernel: ql1: mux_state 0 -> 1
Jul 10 10:38:50 bsd-14 kernel: ql0: port lagid=3D[(8000,00-0E-1E-08-05-20,0=
1D3,8000,000C),(0000,00-00-00-00-00-00,0000,0000,0000)]
Jul 10 10:38:50 bsd-14 kernel: ql0: aggregator created
Jul 10 10:38:50 bsd-14 kernel: ql0: aggregator lagid=3D[(8000,00-0E-1E-08-0=
5-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)]
Jul 10 10:38:50 bsd-14 kernel: ql0: mux_state 0 -> 1
Jul 10 10:38:51 bsd-14 kernel: ql1: lacpdu transmit
Jul 10 10:38:51 bsd-14 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,00=
0D)
Jul 10 10:38:51 bsd-14 kernel: actor.state=3D85<ACTIVITY,AGGREGATION,EXPIRE=
D>
Jul 10 10:38:51 bsd-14 kernel: partner=3D(0000,00-00-00-00-00-00,0000,0000,=
0000)
Jul 10 10:38:51 bsd-14 kernel: partner.state=3D2<TIMEOUT>
Jul 10 10:38:51 bsd-14 kernel: maxdelay=3D0
Jul 10 10:38:51 bsd-14 kernel: ql0: lacpdu transmit
Jul 10 10:38:51 bsd-14 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,00=
0C)
Jul 10 10:38:51 bsd-14 kernel: actor.state=3D85<ACTIVITY,AGGREGATION,EXPIRE=
D>
Jul 10 10:38:51 bsd-14 kernel: partner=3D(0000,00-00-00-00-00-00,0000,0000,=
0000)
Jul 10 10:38:51 bsd-14 kernel: partner.state=3D2<TIMEOUT>
Jul 10 10:38:51 bsd-14 kernel: maxdelay=3D0
Jul 10 10:38:51 bsd-14 kernel: ql0: lacpdu receive
Jul 10 10:38:51 bsd-14 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,00=
0E)
Jul 10 10:38:51 bsd-14 kernel: actor.state=3D5<ACTIVITY,AGGREGATION>
Jul 10 10:38:51 bsd-14 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,=
000C)
Jul 10 10:38:51 bsd-14 kernel: partner.state=3D85<ACTIVITY,AGGREGATION,EXPI=
RED>
Jul 10 10:38:51 bsd-14 kernel: maxdelay=3D0
Jul 10 10:38:51 bsd-14 kernel: ql0: old pstate 2<TIMEOUT>
Jul 10 10:38:51 bsd-14 kernel: ql0: new pstate 5<ACTIVITY,AGGREGATION>
Jul 10 10:38:51 bsd-14 kernel: ql0: partner timeout changed
Jul 10 10:38:51 bsd-14 kernel: ql0: lacpdu receive
Jul 10 10:38:51 bsd-14 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,00=
0E)
Jul 10 10:38:51 bsd-14 kernel: actor.state=3D5<ACTIVITY,AGGREGATION>
Jul 10 10:38:51 bsd-14 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,=
000C)
Jul 10 10:38:51 bsd-14 kernel: partner.state=3D85<ACTIVITY,AGGREGATION,EXPI=
RED>
Jul 10 10:38:51 bsd-14 kernel: maxdelay=3D0
Jul 10 10:38:52 bsd-14 kernel: ql1: lacp_sm_rx_timer: EXPIRED -> DEFAULTED
Jul 10 10:38:52 bsd-14 kernel: ql1: partner timeout changed
Jul 10 10:38:52 bsd-14 kernel: ql1: lacp_sm_mux_timer: aggregator [(8000,00=
-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], p=
ending 1 -> 0
Jul 10 10:38:52 bsd-14 kernel: ql1: collecting disabled
Jul 10 10:38:52 bsd-14 kernel: lacp_aggregator_delref: lagid=3D[(8000,00-0E=
-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], refc=
nt 1 -> 0
Jul 10 10:38:52 bsd-14 kernel: ql1: mux_state 1 -> 0
Jul 10 10:38:52 bsd-14 kernel: ql1: lacpdu transmit
Jul 10 10:38:52 bsd-14 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,00=
0D)
Jul 10 10:38:52 bsd-14 kernel: actor.state=3D45<ACTIVITY,AGGREGATION,DEFAUL=
TED>
Jul 10 10:38:52 bsd-14 kernel: partner=3D(FFFF,00-00-00-00-00-00,0000,FFFF,=
0000)
Jul 10 10:38:52 bsd-14 kernel: partner.state=3D3c<AGGREGATION,SYNC,COLLECTI=
NG,DISTRIBUTING>
Jul 10 10:38:52 bsd-14 kernel: maxdelay=3D0
Jul 10 10:38:52 bsd-14 kernel: ql0: lacp_sm_mux_timer: aggregator [(8000,00=
-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], p=
ending 1 -> 0
Jul 10 10:38:52 bsd-14 kernel: ql0: collecting disabled
Jul 10 10:38:52 bsd-14 kernel: lacp_aggregator_delref: lagid=3D[(8000,00-0E=
-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], refc=
nt 1 -> 0
Jul 10 10:38:52 bsd-14 kernel: ql0: mux_state 1 -> 0
Jul 10 10:38:52 bsd-14 kernel: ql0: lacpdu transmit
Jul 10 10:38:52 bsd-14 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,00=
0C)
Jul 10 10:38:52 bsd-14 kernel: actor.state=3D5<ACTIVITY,AGGREGATION>
Jul 10 10:38:52 bsd-14 kernel: partner=3D(8000,00-0E-1E-04-2C-F0,0213,8000,=
000E)
Jul 10 10:38:52 bsd-14 kernel: partner.state=3D5<ACTIVITY,AGGREGATION>
Jul 10 10:38:52 bsd-14 kernel: maxdelay=3D0
Jul 10 10:38:53 bsd-14 kernel: ql1: port lagid=3D[(8000,00-0E-1E-08-05-20,0=
1D3,8000,000D),(FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
Jul 10 10:38:53 bsd-14 kernel: ql1: aggregator created
Jul 10 10:38:53 bsd-14 kernel: ql1: aggregator lagid=3D[(8000,00-0E-1E-08-0=
5-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)]
Jul 10 10:38:53 bsd-14 kernel: ql1: mux_state 0 -> 1
Jul 10 10:38:53 bsd-14 kernel: ql0: port lagid=3D[(8000,00-0E-1E-08-05-20,0=
1D3,8000,000C),(8000,00-0E-1E-04-2C-F0,0213,8000,000E)]
Jul 10 10:38:53 bsd-14 kernel: ql0: aggregator created
Jul 10 10:38:53 bsd-14 kernel: ql0: aggregator lagid=3D[(8000,00-0E-1E-08-0=
5-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0213,0000,0000)]
Jul 10 10:38:53 bsd-14 kernel: ql0: mux_state 0 -> 1
Jul 10 10:38:54 bsd-14 kernel: ql0: lacpdu receive
Jul 10 10:38:54 bsd-14 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,00=
0E)
Jul 10 10:38:54 bsd-14 kernel: actor.state=3Dd<ACTIVITY,AGGREGATION,SYNC>
Jul 10 10:38:54 bsd-14 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,=
000C)
Jul 10 10:38:54 bsd-14 kernel: partner.state=3D5<ACTIVITY,AGGREGATION>
Jul 10 10:38:54 bsd-14 kernel: maxdelay=3D0
Jul 10 10:38:54 bsd-14 kernel: ql0: old pstate 5<ACTIVITY,AGGREGATION>
Jul 10 10:38:54 bsd-14 kernel: ql0: new pstate d<ACTIVITY,AGGREGATION,SYNC>
Jul 10 10:38:55 bsd-14 kernel: ql1: lacp_sm_mux_timer: aggregator [(8000,00=
-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)], p=
ending 1 -> 0
Jul 10 10:38:55 bsd-14 kernel: ql1: collecting disabled
Jul 10 10:38:55 bsd-14 kernel: ql1: mux_state 1 -> 2
Jul 10 10:38:55 bsd-14 kernel: ql1: collecting enabled
Jul 10 10:38:55 bsd-14 kernel: ql1: mux_state 2 -> 3
Jul 10 10:38:55 bsd-14 kernel: ql1: enable distributing on aggregator [(800=
0,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)=
], nports 0 -> 1
Jul 10 10:38:55 bsd-14 kernel: lacp_select_active_aggregator:
Jul 10 10:38:55 bsd-14 kernel: [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FF=
FF,00-00-00-00-00-00,0000,0000,0000)], speed=3D10000000000, nports=3D1
Jul 10 10:38:55 bsd-14 kernel: active aggregator changed
Jul 10 10:38:55 bsd-14 kernel: old (none)
Jul 10 10:38:55 bsd-14 kernel: new [(8000,00-0E-1E-08-05-20,01D3,0000,0000)=
,(FFFF,00-00-00-00-00-00,0000,0000,0000)]
Jul 10 10:38:55 bsd-14 kernel: Set table 1 with 1 ports
Jul 10 10:38:55 bsd-14 kernel: lacp_suppress_distributing
Jul 10 10:38:55 bsd-14 kernel: ql1: marker transmit, port=3D13, sys=3D00:0e=
:1e:08:05:20, id=3D1
Jul 10 10:38:55 bsd-14 kernel: ql0: marker transmit, port=3D12, sys=3D00:0e=
:1e:08:05:20, id=3D1
Jul 10 10:38:55 bsd-14 kernel: ql1: mux_state 3 -> 4
Jul 10 10:38:55 bsd-14 kernel: ql1: lacpdu transmit
Jul 10 10:38:55 bsd-14 kernel: ql0: marker response, port=3D12, sys=3D00:0e=
:1e:08:05:20, id=3D1
Jul 10 10:38:55 bsd-14 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,00=
0D)
Jul 10 10:38:55 bsd-14 kernel: actor.state=3D7d<ACTIVITY,AGGREGATION,SYNC,C=
OLLECTING,DISTRIBUTING,DEFAULTED>
Jul 10 10:38:55 bsd-14 kernel: partner=3D(FFFF,00-00-00-00-00-00,0000,FFFF,=
0000)
Jul 10 10:38:55 bsd-14 kernel: partner.state=3D3c<AGGREGATION,SYNC,COLLECTI=
NG,DISTRIBUTING>
Jul 10 10:38:55 bsd-14 kernel: maxdelay=3D0
Jul 10 10:38:55 bsd-14 kernel: ql0: lacp_sm_mux_timer: aggregator [(8000,00=
-0E-1E-08-05-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0213,0000,0000)], p=
ending 1 -> 0
Jul 10 10:38:55 bsd-14 kernel: ql0: collecting disabled
Jul 10 10:38:55 bsd-14 kernel: ql0: mux_state 1 -> 2
Jul 10 10:38:55 bsd-14 kernel: ql0: collecting enabled
Jul 10 10:38:55 bsd-14 kernel: ql0: mux_state 2 -> 3
Jul 10 10:38:55 bsd-14 kernel: ql0: lacpdu transmit
Jul 10 10:38:55 bsd-14 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,00=
0C)
Jul 10 10:38:55 bsd-14 kernel: actor.state=3D1d<ACTIVITY,AGGREGATION,SYNC,C=
OLLECTING>
Jul 10 10:38:55 bsd-14 kernel: partner=3D(8000,00-0E-1E-04-2C-F0,0213,8000,=
000E)
Jul 10 10:38:55 bsd-14 kernel: partner.state=3Dd<ACTIVITY,AGGREGATION,SYNC>
Jul 10 10:38:55 bsd-14 kernel: maxdelay=3D0
Jul 10 10:38:55 bsd-14 kernel: ql0: lacpdu receive
Jul 10 10:38:55 bsd-14 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,00=
0E)
Jul 10 10:38:55 bsd-14 kernel: actor.state=3D3d<ACTIVITY,AGGREGATION,SYNC,C=
OLLECTING,DISTRIBUTING>
Jul 10 10:38:55 bsd-14 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,=
000C)
Jul 10 10:38:55 bsd-14 kernel: partner.state=3D1d<ACTIVITY,AGGREGATION,SYNC=
,COLLECTING>
Jul 10 10:38:55 bsd-14 kernel: maxdelay=3D0
Jul 10 10:38:55 bsd-14 kernel: ql0: old pstate d<ACTIVITY,AGGREGATION,SYNC>
Jul 10 10:38:55 bsd-14 kernel: ql0: new pstate 3d<ACTIVITY,AGGREGATION,SYNC=
,COLLECTING,DISTRIBUTING>
Jul 10 10:38:56 bsd-14 kernel: ql0: enable distributing on aggregator [(800=
0,00-0E-1E-08-05-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0213,0000,0000)=
], nports 0 -> 1
Jul 10 10:38:56 bsd-14 kernel: lacp_select_active_aggregator:
Jul 10 10:38:56 bsd-14 kernel: [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FF=
FF,00-00-00-00-00-00,0000,0000,0000)], speed=3D10000000000, nports=3D1
Jul 10 10:38:56 bsd-14 kernel: [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(80=
00,00-0E-1E-04-2C-F0,0213,0000,0000)], speed=3D10000000000, nports=3D1
Jul 10 10:38:56 bsd-14 kernel: active aggregator changed
Jul 10 10:38:56 bsd-14 kernel: old [(8000,00-0E-1E-08-05-20,01D3,0000,0000)=
,(FFFF,00-00-00-00-00-00,0000,0000,0000)]
Jul 10 10:38:56 bsd-14 kernel: new [(8000,00-0E-1E-08-05-20,01D3,0000,0000)=
,(8000,00-0E-1E-04-2C-F0,0213,0000,0000)]
Jul 10 10:38:56 bsd-14 kernel: Set table 0 with 1 ports
Jul 10 10:38:56 bsd-14 kernel: lacp_suppress_distributing
Jul 10 10:38:56 bsd-14 kernel: ql1: marker transmit, port=3D13, sys=3D00:0e=
:1e:08:05:20, id=3D2
Jul 10 10:38:56 bsd-14 kernel: ql0: marker transmit, port=3D12, sys=3D00:0e=
:1e:08:05:20, id=3D2
Jul 10 10:38:56 bsd-14 kernel: ql0: mux_state 3 -> 4
Jul 10 10:38:56 bsd-14 kernel: ql0: marker response, port=3D12, sys=3D00:0e=
:1e:08:05:20, id=3D2
Jul 10 10:38:59 bsd-14 kernel: lacp_transit_expire
^C
System 2 logs :
Jul 10 02:38:24 bsd-15 kernel: lacp_attach[738] : lacp attached
Jul 10 02:38:24 bsd-15 kernel: lacp_attach[740] : lacp_defined
Jul 10 02:38:24 bsd-15 kernel: lagg0: link state changed to UP
Jul 10 02:38:24 bsd-15 kernel: ql0: media changed 0x0 -> 0x100033, ether =
=3D 1, fdx =3D 1, link =3D 1
Jul 10 02:38:24 bsd-15 kernel: ql0: -> UNSELECTED
Jul 10 02:38:24 bsd-15 kernel: ql1: media changed 0x0 -> 0x100033, ether =
=3D 1, fdx =3D 1, link =3D 1
Jul 10 02:38:24 bsd-15 kernel: ql1: -> UNSELECTED
Jul 10 02:38:24 bsd-15 kernel: lacp_select_tx_port: no active aggregator
Jul 10 02:38:25 bsd-15 kernel: ql1: port lagid=3D[(8000,00-0E-1E-04-2C-F0,0=
213,8000,000F),(0000,00-00-00-00-00-00,0000,0000,0000)]
Jul 10 02:38:25 bsd-15 kernel: ql1: aggregator created
Jul 10 02:38:25 bsd-15 kernel: ql1: aggregator lagid=3D[(8000,00-0E-1E-04-2=
C-F0,0213,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)]
Jul 10 02:38:25 bsd-15 kernel: ql1: mux_state 0 -> 1
Jul 10 02:38:25 bsd-15 kernel: ql0: port lagid=3D[(8000,00-0E-1E-04-2C-F0,0=
213,8000,000E),(0000,00-00-00-00-00-00,0000,0000,0000)]
Jul 10 02:38:25 bsd-15 kernel: ql0: aggregator created
Jul 10 02:38:25 bsd-15 kernel: ql0: aggregator lagid=3D[(8000,00-0E-1E-04-2=
C-F0,0213,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)]
Jul 10 02:38:25 bsd-15 kernel: ql0: mux_state 0 -> 1
Jul 10 02:38:26 bsd-15 kernel: ql0: lacpdu receive
Jul 10 02:38:26 bsd-15 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,00=
0C)
Jul 10 02:38:26 bsd-15 kernel: actor.state=3D85<ACTIVITY,AGGREGATION,EXPIRE=
D>
Jul 10 02:38:26 bsd-15 kernel: partner=3D(0000,00-00-00-00-00-00,0000,0000,=
0000)
Jul 10 02:38:26 bsd-15 kernel: partner.state=3D2<TIMEOUT>
Jul 10 02:38:26 bsd-15 kernel: maxdelay=3D0
Jul 10 02:38:26 bsd-15 kernel: ql0: lacp_sm_rx_update_ntt: assert ntt
Jul 10 02:38:26 bsd-15 kernel: ql0: old pstate 2<TIMEOUT>
Jul 10 02:38:26 bsd-15 kernel: ql0: new pstate 85<ACTIVITY,AGGREGATION,EXPI=
RED>
Jul 10 02:38:26 bsd-15 kernel: ql0: partner timeout changed
Jul 10 02:38:26 bsd-15 kernel: ql0: lacpdu transmit
Jul 10 02:38:26 bsd-15 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,00=
0E)
Jul 10 02:38:26 bsd-15 kernel: actor.state=3D5<ACTIVITY,AGGREGATION>
Jul 10 02:38:26 bsd-15 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,=
000C)
Jul 10 02:38:26 bsd-15 kernel: partner.state=3D85<ACTIVITY,AGGREGATION,EXPI=
RED>
Jul 10 02:38:26 bsd-15 kernel: maxdelay=3D0
Jul 10 02:38:26 bsd-15 kernel: ql1: lacpdu transmit
Jul 10 02:38:26 bsd-15 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,00=
0F)
Jul 10 02:38:26 bsd-15 kernel: actor.state=3D85<ACTIVITY,AGGREGATION,EXPIRE=
D>
Jul 10 02:38:26 bsd-15 kernel: partner=3D(0000,00-00-00-00-00-00,0000,0000,=
0000)
Jul 10 02:38:26 bsd-15 kernel: partner.state=3D2<TIMEOUT>
Jul 10 02:38:26 bsd-15 kernel: maxdelay=3D0
Jul 10 02:38:26 bsd-15 kernel: ql0: collecting disabled
Jul 10 02:38:26 bsd-15 kernel: lacp_aggregator_delref: lagid=3D[(8000,00-0E=
-1E-04-2C-F0,0213, 0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], ref=
cnt 1 -> 0
Jul 10 02:38:26 bsd-15 kernel: ql0: mux_state 1 -> 0
Jul 10 02:38:26 bsd-15 kernel: ql0: lacpdu transmit
Jul 10 02:38:26 bsd-15 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,00=
0E)
Jul 10 02:38:26 bsd-15 kernel: actor.state=3D5<ACTIVITY,AGGREGATION>
Jul 10 02:38:26 bsd-15 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,=
000C)
Jul 10 02:38:26 bsd-15 kernel: partner.state=3D85<ACTIVITY,AGGREGATION,EXPI=
RED>
Jul 10 02:38:26 bsd-15 kernel: maxdelay=3D0
Jul 10 02:38:27 bsd-15 kernel: ql0: lacpdu receive
Jul 10 02:38:27 bsd-15 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,00=
0C)
Jul 10 02:38:27 bsd-15 kernel: actor.state=3D5<ACTIVITY,AGGREGATION>
Jul 10 02:38:27 bsd-15 kernel: partner=3D(8000,00-0E-1E-04-2C-F0,0213,8000,=
000E)
Jul 10 02:38:27 bsd-15 kernel: partner.state=3D5<ACTIVITY,AGGREGATION>
Jul 10 02:38:27 bsd-15 kernel: maxdelay=3D0
Jul 10 02:38:27 bsd-15 kernel: ql0: old pstate 85<ACTIVITY,AGGREGATION,EXPI=
RED>
Jul 10 02:38:27 bsd-15 kernel: ql0: new pstate 5<ACTIVITY,AGGREGATION>
Jul 10 02:38:27 bsd-15 kernel: ql1: lacp_sm_rx_timer: EXPIRED -> DEFAULTED
Jul 10 02:38:27 bsd-15 kernel: ql1: partner timeout changed
Jul 10 02:38:27 bsd-15 kernel: ql1: lacp_sm_mux_timer: aggregator [(8000,00=
-0E-1E-04-2C-F0, 0213,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], =
pending 1 -> 0
Jul 10 02:38:27 bsd-15 kernel: ql1: collecting disabled
Jul 10 02:38:27 bsd-15 kernel: lacp_aggregator_delref: lagid=3D[(8000,00-0E=
-1E-04-2C-F0,0213, 0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], ref=
cnt 1 -> 0
Jul 10 02:38:27 bsd-15 kernel: ql1: mux_state 1 -> 0
Jul 10 02:38:27 bsd-15 kernel: ql1: lacpdu transmit
Jul 10 02:38:27 bsd-15 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,00=
0F)
Jul 10 02:38:27 bsd-15 kernel: actor.state=3D45<ACTIVITY,AGGREGATION,DEFAUL=
TED>
Jul 10 02:38:27 bsd-15 kernel: partner=3D(FFFF,00-00-00-00-00-00,0000,FFFF,=
0000)
Jul 10 02:38:27 bsd-15 kernel: partner.state=3D3c<AGGREGATION,SYNC,COLLECTI=
NG,DISTRIBUTING>
Jul 10 02:38:27 bsd-15 kernel: maxdelay=3D0
Jul 10 02:38:27 bsd-15 kernel: ql0: port lagid=3D[(8000,00-0E-1E-04-2C-F0,0=
213,8000,000E),(8000,00-0E-1E-08-05-20,01D3,8000,000C)]
Jul 10 02:38:27 bsd-15 kernel: ql0: aggregator created
Jul 10 02:38:27 bsd-15 kernel: ql0: aggregator lagid=3D[(8000,00-0E-1E-04-2=
C-F0,0213,0000,0000),(8000,00-0E-1E-08-05-20,01D3,0000,0000)]
Jul 10 02:38:27 bsd-15 kernel: ql0: mux_state 0 -> 1
Jul 10 02:38:28 bsd-15 kernel: ql1: port lagid=3D[(8000,00-0E-1E-04-2C-F0,0=
213,8000,000F),(FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
Jul 10 02:38:28 bsd-15 kernel: ql1: aggregator created
Jul 10 02:38:28 bsd-15 kernel: ql1: aggregator lagid=3D[(8000,00-0E-1E-04-2=
C-F0,0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)]
Jul 10 02:38:28 bsd-15 kernel: ql1: mux_state 0 -> 1
Jul 10 02:38:29 bsd-15 kernel: ql0: lacp_sm_mux_timer: aggregator [(8000,00=
-0E-1E-04-2C-F0, 0213,0000,0000),(8000,00-0E-1E-08-05-20,01D3,0000,0000)], =
pending 1 -> 0
Jul 10 02:38:29 bsd-15 kernel: ql0: collecting disabled
Jul 10 02:38:29 bsd-15 kernel: ql0: mux_state 1 -> 2
Jul 10 02:38:29 bsd-15 kernel: ql0: lacpdu transmit
Jul 10 02:38:29 bsd-15 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,00=
0E)
Jul 10 02:38:29 bsd-15 kernel: actor.state=3Dd<ACTIVITY,AGGREGATION,SYNC>
Jul 10 02:38:29 bsd-15 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,=
000C)
Jul 10 02:38:29 bsd-15 kernel: partner.state=3D5<ACTIVITY,AGGREGATION>
Jul 10 02:38:29 bsd-15 kernel: maxdelay=3D0
Jul 10 02:38:30 bsd-15 kernel: ql0: lacpdu receive
Jul 10 02:38:30 bsd-15 kernel: actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,00=
0C)
Jul 10 02:38:30 bsd-15 kernel: actor.state=3D1d<ACTIVITY,AGGREGATION,SYNC,C=
OLLECTING>
Jul 10 02:38:30 bsd-15 kernel: partner=3D(8000,00-0E-1E-04-2C-F0,0213,8000,=
000E)
Jul 10 02:38:30 bsd-15 kernel: partner.state=3Dd<ACTIVITY,AGGREGATION,SYNC>
Jul 10 02:38:30 bsd-15 kernel: maxdelay=3D0
Jul 10 02:38:30 bsd-15 kernel: ql0: old pstate 5<ACTIVITY,AGGREGATION>
Jul 10 02:38:30 bsd-15 kernel: ql0: new pstate 1d<ACTIVITY,AGGREGATION,SYNC=
,COLLECTING>
Jul 10 02:38:30 bsd-15 kernel: ql1: lacp_sm_mux_timer: aggregator [(8000,00=
-0E-1E-04-2C-F0, 0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)], =
pending 1 -> 0
Jul 10 02:38:30 bsd-15 kernel: ql1: collecting disabled
Jul 10 02:38:30 bsd-15 kernel: ql1: mux_state 1 -> 2
Jul 10 02:38:30 bsd-15 kernel: ql1: collecting enabled
Jul 10 02:38:30 bsd-15 kernel: ql1: mux_state 2 -> 3
Jul 10 02:38:30 bsd-15 kernel: ql1: enable distributing on aggregator [(800=
0,00-0E-1E-04-2C-F0,0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)=
], nports 0 -> 1
Jul 10 02:38:30 bsd-15 kernel: lacp_select_active_aggregator:
Jul 10 02:38:30 bsd-15 kernel: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(FF=
FF,00-00-00-00-00-00,0000,0000,0000)], speed=3D10000000000, nports=3D1
Jul 10 02:38:30 bsd-15 kernel: active aggregator changed
Jul 10 02:38:30 bsd-15 kernel: old (none)
Jul 10 02:38:30 bsd-15 kernel: new [(8000,00-0E-1E-04-2C-F0,0213,0000,0000)=
,(FFFF,00-00-00 00-00-00,0000,0000,0000)]
Jul 10 02:38:30 bsd-15 kernel: Set table 1 with 1 ports
Jul 10 02:38:30 bsd-15 kernel: lacp_suppress_distributing
Jul 10 02:38:30 bsd-15 kernel: ql1: marker transmit, port=3D15, sys=3D00:0e=
:1e:04:2c:f0, id=3D1
Jul 10 02:38:30 bsd-15 kernel: ql0: marker transmit, port=3D14, sys=3D00:0e=
:1e:04:2c:f0, id=3D1
Jul 10 02:38:30 bsd-15 kernel: ql1: mux_state 3 -> 4
Jul 10 02:38:30 bsd-15 kernel: ql1: lacpdu transmit
Jul 10 02:38:30 bsd-15 kernel: ql0: marker response, port=3D14, sys=3D00:0e=
:1e:04:2c:f0, id=3D1
Jul 10 02:38:30 bsd-15 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,00=
0F)
Jul 10 02:38:30 bsd-15 kernel: actor.state=3D7d<ACTIVITY,AGGREGATION,SYNC,C=
OLLECTING,DISTRIBUTING,DEFAULTED>
Jul 10 02:38:30 bsd-15 kernel: partner=3D(FFFF,00-00-00-00-00-00,0000,FFFF,=
0000)
Jul 10 02:38:30 bsd-15 kernel: partner.state=3D3c<AGGREGATION,SYNC,COLLECTI=
NG,DISTRIBUTING>
Jul 10 02:38:30 bsd-15 kernel: maxdelay=3D0
Jul 10 02:38:30 bsd-15 kernel: ql0: collecting enabled
Jul 10 02:38:30 bsd-15 kernel: ql0: mux_state 2 -> 3
Jul 10 02:38:30 bsd-15 kernel: ql0: enable distributing on aggregator [(800=
0,00-0E-1E-04-2C-F0,0213,0000,0000),(8000,00-0E-1E-08-05-20,01D3,0000,0000)=
], nports 0 -> 1
Jul 10 02:38:30 bsd-15 kernel: lacp_select_active_aggregator:
Jul 10 02:38:30 bsd-15 kernel: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(80=
00,00-0E-1E-08-05-20,01D3,0000,0000)], speed=3D10000000000, nports=3D1
Jul 10 02:38:30 bsd-15 kernel: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(FF=
FF,00-00-00-00-00-00,0000,0000,0000)], speed=3D10000000000, nports=3D1
Jul 10 02:38:30 bsd-15 kernel: active aggregator not changed
Jul 10 02:38:30 bsd-15 kernel: new [(8000,00-0E-1E-04-2C-F0,0213,0000,0000)=
,(FFFF,00-00-00-00-00-00,0000,0000,0000)]
Jul 10 02:38:30 bsd-15 kernel: ql0: mux_state 3 -> 4
Jul 10 02:38:30 bsd-15 kernel: ql0: lacpdu transmit
Jul 10 02:38:30 bsd-15 kernel: actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,00=
0E)
Jul 10 02:38:30 bsd-15 kernel: actor.state=3D3d<ACTIVITY,AGGREGATION,SYNC,C=
OLLECTING,DISTRIBUTING>
Jul 10 02:38:30 bsd-15 kernel: partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,=
000C)
Jul 10 02:38:30 bsd-15 kernel: partner.state=3D1d<ACTIVITY,AGGREGATION,SYNC=
,COLLECTING>
Jul 10 02:38:30 bsd-15 kernel: maxdelay=3D0
Jul 10 02:38:33 bsd-15 kernel: lacp_transit_expire
^C
Let me know if you need more info.
Thanks
Adarsh
-----Original Message-----
From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] =
On Behalf Of Adarsh Joshi
Sent: Tuesday, July 10, 2012 10:08 AM
To: Andrew Boyer
Cc: freebsd-net@freebsd.org
Subject: RE: lacp lagg port flags do not show correctly resulting in poor t=
raffic distribution/performance
Andrew,
Thanks for the reply.
The reason for my suspicion on the portflags is thus (extracted from the if=
config output in my previous mail):
System 1:
Laggport: ql1 flags =3D 18 state =3D 7D
Laggport: ql0 flags =3D 1c state =3D 3D
System 2:
Laggport: ql1 flags =3D 1c state =3D 7D
Laggport: ql0 flags =3D 18 state =3D 3D
I should have explained my setup to you before. Here it is.
Both the ql0 interfaces of the 2 systems are connected using a single cable=
and ql1 interfaces of the 2 systems are connected using a single cable.
System 1 System 2
ql0 <=3D=3D=3D=3D=3D=3D=3D> ql0
ql1 <=3D=3D=3D=3D=3D=3D=3D> ql1
With this setup, I don't think it is possible for ports ql0 to talk to thei=
r partners (each other) and ql1 ports not getting a response from their par=
tner and still get the lagg configuration I have posted.
I thought the portflags are dependent on the LACP state. But I see differen=
t flags for the same LACP state (For the state 7D, ql1 on system 1 shows fl=
ags =3D 18 and ql1 on system 2 shows flags =3D 1c).
Or is my understanding totally wrong?
I will send the LACP_DEBUG logs within the hour.
Thanks
Adarsh
From: Andrew Boyer [mailto:aboyer@averesystems.com]
Sent: Tuesday, July 10, 2012 5:57 AM
To: Adarsh Joshi
Cc: freebsd-net@freebsd.org
Subject: Re: lacp lagg port flags do not show correctly resulting in poor t=
raffic distribution/performance
On Jul 9, 2012, at 8:38 PM, Adarsh Joshi wrote:
Hi,
I am trying to configure lacp lagg interfaces with 2 systems connected back=
to back as follows:
Ifconfig lagg0 create
Ifconfig lagg0 laggproto lacp laggport ql0 laggport ql1 192.168.100.1 netma=
sk 255.255.255.0
Sometimes, the lag interface comes up correctly but sometimes the laggport =
flags do not show properly. Instead of 1c<ACTIVE,COLLECTING,DISTRIBUTING>, =
it shows values of 18. I have seen similar issues reported on various forum=
s with no solution.
Looking at the lagg driver code and reading the standard, I thought the lag=
gport flags ( defined in if_lagg.h) are based on the LACP_STATE_BITS in fil=
e ieee8023ad_lacp.h. But the following ifconfig -v output does not make any=
sense to me.
My concern is that when all the interfaces show flags as 1c, the traffic is=
distributed across both the interfaces uniformly and I get aggregated thro=
ughput. If not, the traffic flows only on 1 interface.
Is this a bug? How do I solve this? Or am I doing something wrong?
I am using Free-BSD 9.0 release.
System 1:
# ifconfig -v lagg0
lag id: [(8000,00-0E-1E-08-05-20,0213,0000,0000),
(8000,00-0E-1E-04-2C-F0,0213,0000,0000)]
laggport: ql1 flags=3D18<COLLECTING,DISTRIBUTING> state=3D7D
[(8000,00-0E-1E-08-05-20,0213,8000,000F),
(FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
laggport: ql0 flags=3D1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D3D
[(8000,00-0E-1E-08-05-20,0213,8000,000E),
(8000,00-0E-1E-04-2C-F0,0213,8000,000E)]
System 2:
# ifconfig -v lagg0
lag id: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),
(FFFF,00-00-00-00-00-00,0000,0000,0000)]
laggport: ql1 flags=3D1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D7D
[(8000,00-0E-1E-04-2C-F0,0213,8000,000F),
(FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
laggport: ql0 flags=3D18<COLLECTING,DISTRIBUTING> state=3D3D
[(8000,00-0E-1E-04-2C-F0,0213,8000,000E),
(8000,00-0E-1E-08-05-20,0213,8000,000E)]
thanks
Adarsh
I don't think you have a port flags problem per se; the flags are correctly=
displaying the state of the lagg. Your problem is that your systems aren'=
t negotiating the correct lagg configuration. Each tuple after the laggpor=
t represents the [(actor state),(partner state)]. Ports ql0 have been able=
to talk to their partners (each other). Neither ql1 port has seen a respo=
nse from a partner, though.
You could try restarting the state machine on one box with 'ifconfig lagg0 =
laggproto lacp'. To see the negotiation you'll need to rebuild your kernel=
with '#define LACP_DEBUG 1' added to the top of sys/net/ieee802.3ad_lacp.c=
. Or upgrade to a newer stable snapshot that has the net.lacp_debug sysctl=
and turn it on.
Or just turn off LACP. What does it get you in this configuration?
Hope this helps,
Andrew
--------------------------------------------------
Andrew Boyer aboyer@averesystems.com<mailto:aboyer@averesystems.com>
________________________________
This message and any attached documents contain information from QLogic Cor=
poration or its wholly-owned subsidiaries that may be confidential. If you =
are not the intended recipient, you may not read, copy, distribute, or use =
this information. If you have received this transmission in error, please n=
otify the sender immediately by reply e-mail and then delete this message.
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
This message and any attached documents contain information from QLogic Cor=
poration or its wholly-owned subsidiaries that may be confidential. If you =
are not the intended recipient, you may not read, copy, distribute, or use =
this information. If you have received this transmission in error, please n=
otify the sender immediately by reply e-mail and then delete this message.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5E4F49720D0BAD499EE1F01232234BA877435B2F66>