Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 27 Jul 2012 11:11:41 -0400
From:      Andrew Boyer <aboyer@averesystems.com>
To:        Adarsh Joshi <adarsh.joshi@qlogic.com>
Cc:        "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>
Subject:   Re: lacp lagg port flags do not show correctly resulting in poor traffic distribution/performance
Message-ID:  <9BC66C40-6F4B-48A0-AEC0-DCAD0C6863EF@averesystems.com>
In-Reply-To: <5E4F49720D0BAD499EE1F01232234BA877435B2F66@AVEXMB1.qlogic.org>
References:  <5E4F49720D0BAD499EE1F01232234BA877435B2E28@AVEXMB1.qlogic.org> <4C5FC147-2F49-4AE0-ADF3-C5381DE6580F@averesystems.com> <5E4F49720D0BAD499EE1F01232234BA877435B2F2E@AVEXMB1.qlogic.org> <5E4F49720D0BAD499EE1F01232234BA877435B2F66@AVEXMB1.qlogic.org>

next in thread | previous in thread | raw e-mail | index | archive | help
Adarsh,
Sorry for the delay.

I'm not an LACP protocol expert, but looking at your logs I don't see =
ql1 on either node receiving a lacpdu response.  Are you certain that =
link is working?

-Andrew

On Jul 10, 2012, at 1:53 PM, Adarsh Joshi wrote:

> Andrew,
>=20
> Here are the logs with LACP_DEBUG defined in ieee802.3ad_lacp.c,
>=20
> after typing
>=20
> Ifconfig lagg0 create
> ifconfig lagg0 laggproto lacp laggport ql0 laggport ql1 192.168.100.1 =
netmask 255.255.255.0
>=20
> I compiled it as a standalone driver by the way.
>=20
> System 1:
>=20
> # ifconfig -v lagg0
> lagg0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 =
mtu 1500
>        =
options=3D13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4>
>        ether 00:0e:1e:08:05:20
>        inet 192.168.100.1 netmask 0xffffff00 broadcast 192.168.100.255
>        nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>        media: Ethernet autoselect
>        status: active
>        groups: lagg
>        laggproto lacp
>        lag id: [(8000,00-0E-1E-08-05-20,01D3,0000,0000),
>                 (8000,00-0E-1E-04-2C-F0,0213,0000,0000)]
>        laggport: ql1 flags=3D18<COLLECTING,DISTRIBUTING> state=3D7D
>                [(8000,00-0E-1E-08-05-20,01D3,8000,000D),
>                 (FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
>        laggport: ql0 flags=3D1c<ACTIVE,COLLECTING,DISTRIBUTING> =
state=3D3D
>                [(8000,00-0E-1E-08-05-20,01D3,8000,000C),
>                 (8000,00-0E-1E-04-2C-F0,0213,8000,000E)]
>=20
>=20
> System 2:
>=20
> # ifconfig -v lagg0
> lagg0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 =
mtu 1500
>        =
options=3D13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4>
>        ether 00:0e:1e:04:2c:f0
>        inet 192.168.100.2 netmask 0xffffff00 broadcast 192.168.100.255
>        nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>        media: Ethernet autoselect
>        status: active
>        groups: lagg
>        laggproto lacp
>        lag id: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),
>                 (FFFF,00-00-00-00-00-00,0000,0000,0000)]
>        laggport: ql1 flags=3D1c<ACTIVE,COLLECTING,DISTRIBUTING> =
state=3D7D
>                [(8000,00-0E-1E-04-2C-F0,0213,8000,000F),
>                 (FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
>        laggport: ql0 flags=3D18<COLLECTING,DISTRIBUTING> state=3D3D
>                [(8000,00-0E-1E-04-2C-F0,0213,8000,000E),
>                 (8000,00-0E-1E-08-05-20,01D3,8000,000C)]
>=20
>=20
> System 1 logs :
>=20
> Jul 10 10:38:49 bsd-14 kernel: lacp_attach[738] : lacp attached
> Jul 10 10:38:49 bsd-14 kernel: lacp_attach[740] : lacp_defined
> Jul 10 10:38:49 bsd-14 kernel: lagg0: link state changed to UP
> Jul 10 10:38:49 bsd-14 kernel: ql0: media changed 0x0 -> 0x100033, =
ether =3D 1, fdx =3D 1, link =3D 1
> Jul 10 10:38:49 bsd-14 kernel: ql0: -> UNSELECTED
> Jul 10 10:38:49 bsd-14 kernel: ql1: media changed 0x0 -> 0x100033, =
ether =3D 1, fdx =3D 1, link =3D 1
> Jul 10 10:38:49 bsd-14 kernel: ql1: -> UNSELECTED
> Jul 10 10:38:49 bsd-14 kernel: lacp_select_tx_port: no active =
aggregator
> Jul 10 10:38:50 bsd-14 kernel: ql1: port =
lagid=3D[(8000,00-0E-1E-08-05-20,01D3,8000,000D),(0000,00-00-00-00-00-00,0=
000,0000,0000)]
> Jul 10 10:38:50 bsd-14 kernel: ql1: aggregator created
> Jul 10 10:38:50 bsd-14 kernel: ql1: aggregator =
lagid=3D[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0=
000,0000,0000)]
> Jul 10 10:38:50 bsd-14 kernel: ql1: mux_state 0 -> 1
> Jul 10 10:38:50 bsd-14 kernel: ql0: port =
lagid=3D[(8000,00-0E-1E-08-05-20,01D3,8000,000C),(0000,00-00-00-00-00-00,0=
000,0000,0000)]
> Jul 10 10:38:50 bsd-14 kernel: ql0: aggregator created
> Jul 10 10:38:50 bsd-14 kernel: ql0: aggregator =
lagid=3D[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0=
000,0000,0000)]
> Jul 10 10:38:50 bsd-14 kernel: ql0: mux_state 0 -> 1
> Jul 10 10:38:51 bsd-14 kernel: ql1: lacpdu transmit
> Jul 10 10:38:51 bsd-14 kernel: =
actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,000D)
> Jul 10 10:38:51 bsd-14 kernel: =
actor.state=3D85<ACTIVITY,AGGREGATION,EXPIRED>
> Jul 10 10:38:51 bsd-14 kernel: =
partner=3D(0000,00-00-00-00-00-00,0000,0000,0000)
> Jul 10 10:38:51 bsd-14 kernel: partner.state=3D2<TIMEOUT>
> Jul 10 10:38:51 bsd-14 kernel: maxdelay=3D0
> Jul 10 10:38:51 bsd-14 kernel: ql0: lacpdu transmit
> Jul 10 10:38:51 bsd-14 kernel: =
actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,000C)
> Jul 10 10:38:51 bsd-14 kernel: =
actor.state=3D85<ACTIVITY,AGGREGATION,EXPIRED>
> Jul 10 10:38:51 bsd-14 kernel: =
partner=3D(0000,00-00-00-00-00-00,0000,0000,0000)
> Jul 10 10:38:51 bsd-14 kernel: partner.state=3D2<TIMEOUT>
> Jul 10 10:38:51 bsd-14 kernel: maxdelay=3D0
> Jul 10 10:38:51 bsd-14 kernel: ql0: lacpdu receive
> Jul 10 10:38:51 bsd-14 kernel: =
actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,000E)
> Jul 10 10:38:51 bsd-14 kernel: actor.state=3D5<ACTIVITY,AGGREGATION>
> Jul 10 10:38:51 bsd-14 kernel: =
partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,000C)
> Jul 10 10:38:51 bsd-14 kernel: =
partner.state=3D85<ACTIVITY,AGGREGATION,EXPIRED>
> Jul 10 10:38:51 bsd-14 kernel: maxdelay=3D0
> Jul 10 10:38:51 bsd-14 kernel: ql0: old pstate 2<TIMEOUT>
> Jul 10 10:38:51 bsd-14 kernel: ql0: new pstate 5<ACTIVITY,AGGREGATION>
> Jul 10 10:38:51 bsd-14 kernel: ql0: partner timeout changed
> Jul 10 10:38:51 bsd-14 kernel: ql0: lacpdu receive
> Jul 10 10:38:51 bsd-14 kernel: =
actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,000E)
> Jul 10 10:38:51 bsd-14 kernel: actor.state=3D5<ACTIVITY,AGGREGATION>
> Jul 10 10:38:51 bsd-14 kernel: =
partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,000C)
> Jul 10 10:38:51 bsd-14 kernel: =
partner.state=3D85<ACTIVITY,AGGREGATION,EXPIRED>
> Jul 10 10:38:51 bsd-14 kernel: maxdelay=3D0
> Jul 10 10:38:52 bsd-14 kernel: ql1: lacp_sm_rx_timer: EXPIRED -> =
DEFAULTED
> Jul 10 10:38:52 bsd-14 kernel: ql1: partner timeout changed
> Jul 10 10:38:52 bsd-14 kernel: ql1: lacp_sm_mux_timer: aggregator =
[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000=
,0000)], pending 1 -> 0
> Jul 10 10:38:52 bsd-14 kernel: ql1: collecting disabled
> Jul 10 10:38:52 bsd-14 kernel: lacp_aggregator_delref: =
lagid=3D[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0=
000,0000,0000)], refcnt 1 -> 0
> Jul 10 10:38:52 bsd-14 kernel: ql1: mux_state 1 -> 0
> Jul 10 10:38:52 bsd-14 kernel: ql1: lacpdu transmit
> Jul 10 10:38:52 bsd-14 kernel: =
actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,000D)
> Jul 10 10:38:52 bsd-14 kernel: =
actor.state=3D45<ACTIVITY,AGGREGATION,DEFAULTED>
> Jul 10 10:38:52 bsd-14 kernel: =
partner=3D(FFFF,00-00-00-00-00-00,0000,FFFF,0000)
> Jul 10 10:38:52 bsd-14 kernel: =
partner.state=3D3c<AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
> Jul 10 10:38:52 bsd-14 kernel: maxdelay=3D0
> Jul 10 10:38:52 bsd-14 kernel: ql0: lacp_sm_mux_timer: aggregator =
[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000=
,0000)], pending 1 -> 0
> Jul 10 10:38:52 bsd-14 kernel: ql0: collecting disabled
> Jul 10 10:38:52 bsd-14 kernel: lacp_aggregator_delref: =
lagid=3D[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0=
000,0000,0000)], refcnt 1 -> 0
> Jul 10 10:38:52 bsd-14 kernel: ql0: mux_state 1 -> 0
> Jul 10 10:38:52 bsd-14 kernel: ql0: lacpdu transmit
> Jul 10 10:38:52 bsd-14 kernel: =
actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,000C)
> Jul 10 10:38:52 bsd-14 kernel: actor.state=3D5<ACTIVITY,AGGREGATION>
> Jul 10 10:38:52 bsd-14 kernel: =
partner=3D(8000,00-0E-1E-04-2C-F0,0213,8000,000E)
> Jul 10 10:38:52 bsd-14 kernel: partner.state=3D5<ACTIVITY,AGGREGATION>
> Jul 10 10:38:52 bsd-14 kernel: maxdelay=3D0
> Jul 10 10:38:53 bsd-14 kernel: ql1: port =
lagid=3D[(8000,00-0E-1E-08-05-20,01D3,8000,000D),(FFFF,00-00-00-00-00-00,0=
000,FFFF,0000)]
> Jul 10 10:38:53 bsd-14 kernel: ql1: aggregator created
> Jul 10 10:38:53 bsd-14 kernel: ql1: aggregator =
lagid=3D[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0=
000,0000,0000)]
> Jul 10 10:38:53 bsd-14 kernel: ql1: mux_state 0 -> 1
> Jul 10 10:38:53 bsd-14 kernel: ql0: port =
lagid=3D[(8000,00-0E-1E-08-05-20,01D3,8000,000C),(8000,00-0E-1E-04-2C-F0,0=
213,8000,000E)]
> Jul 10 10:38:53 bsd-14 kernel: ql0: aggregator created
> Jul 10 10:38:53 bsd-14 kernel: ql0: aggregator =
lagid=3D[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0=
213,0000,0000)]
> Jul 10 10:38:53 bsd-14 kernel: ql0: mux_state 0 -> 1
> Jul 10 10:38:54 bsd-14 kernel: ql0: lacpdu receive
> Jul 10 10:38:54 bsd-14 kernel: =
actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,000E)
> Jul 10 10:38:54 bsd-14 kernel: =
actor.state=3Dd<ACTIVITY,AGGREGATION,SYNC>
> Jul 10 10:38:54 bsd-14 kernel: =
partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,000C)
> Jul 10 10:38:54 bsd-14 kernel: partner.state=3D5<ACTIVITY,AGGREGATION>
> Jul 10 10:38:54 bsd-14 kernel: maxdelay=3D0
> Jul 10 10:38:54 bsd-14 kernel: ql0: old pstate 5<ACTIVITY,AGGREGATION>
> Jul 10 10:38:54 bsd-14 kernel: ql0: new pstate =
d<ACTIVITY,AGGREGATION,SYNC>
> Jul 10 10:38:55 bsd-14 kernel: ql1: lacp_sm_mux_timer: aggregator =
[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000=
,0000)], pending 1 -> 0
> Jul 10 10:38:55 bsd-14 kernel: ql1: collecting disabled
> Jul 10 10:38:55 bsd-14 kernel: ql1: mux_state 1 -> 2
> Jul 10 10:38:55 bsd-14 kernel: ql1: collecting enabled
> Jul 10 10:38:55 bsd-14 kernel: ql1: mux_state 2 -> 3
> Jul 10 10:38:55 bsd-14 kernel: ql1: enable distributing on aggregator =
[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000=
,0000)], nports 0 -> 1
> Jul 10 10:38:55 bsd-14 kernel: lacp_select_active_aggregator:
> Jul 10 10:38:55 bsd-14 kernel: =
[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000=
,0000)], speed=3D10000000000, nports=3D1
> Jul 10 10:38:55 bsd-14 kernel: active aggregator changed
> Jul 10 10:38:55 bsd-14 kernel: old (none)
> Jul 10 10:38:55 bsd-14 kernel: new =
[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000=
,0000)]
> Jul 10 10:38:55 bsd-14 kernel: Set table 1 with 1 ports
> Jul 10 10:38:55 bsd-14 kernel: lacp_suppress_distributing
> Jul 10 10:38:55 bsd-14 kernel: ql1: marker transmit, port=3D13, =
sys=3D00:0e:1e:08:05:20, id=3D1
> Jul 10 10:38:55 bsd-14 kernel: ql0: marker transmit, port=3D12, =
sys=3D00:0e:1e:08:05:20, id=3D1
> Jul 10 10:38:55 bsd-14 kernel: ql1: mux_state 3 -> 4
> Jul 10 10:38:55 bsd-14 kernel: ql1: lacpdu transmit
> Jul 10 10:38:55 bsd-14 kernel: ql0: marker response, port=3D12, =
sys=3D00:0e:1e:08:05:20, id=3D1
> Jul 10 10:38:55 bsd-14 kernel: =
actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,000D)
> Jul 10 10:38:55 bsd-14 kernel: =
actor.state=3D7d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING,DEFAULT=
ED>
> Jul 10 10:38:55 bsd-14 kernel: =
partner=3D(FFFF,00-00-00-00-00-00,0000,FFFF,0000)
> Jul 10 10:38:55 bsd-14 kernel: =
partner.state=3D3c<AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
> Jul 10 10:38:55 bsd-14 kernel: maxdelay=3D0
> Jul 10 10:38:55 bsd-14 kernel: ql0: lacp_sm_mux_timer: aggregator =
[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0213,0000=
,0000)], pending 1 -> 0
> Jul 10 10:38:55 bsd-14 kernel: ql0: collecting disabled
> Jul 10 10:38:55 bsd-14 kernel: ql0: mux_state 1 -> 2
> Jul 10 10:38:55 bsd-14 kernel: ql0: collecting enabled
> Jul 10 10:38:55 bsd-14 kernel: ql0: mux_state 2 -> 3
> Jul 10 10:38:55 bsd-14 kernel: ql0: lacpdu transmit
> Jul 10 10:38:55 bsd-14 kernel: =
actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,000C)
> Jul 10 10:38:55 bsd-14 kernel: =
actor.state=3D1d<ACTIVITY,AGGREGATION,SYNC,COLLECTING>
> Jul 10 10:38:55 bsd-14 kernel: =
partner=3D(8000,00-0E-1E-04-2C-F0,0213,8000,000E)
> Jul 10 10:38:55 bsd-14 kernel: =
partner.state=3Dd<ACTIVITY,AGGREGATION,SYNC>
> Jul 10 10:38:55 bsd-14 kernel: maxdelay=3D0
> Jul 10 10:38:55 bsd-14 kernel: ql0: lacpdu receive
> Jul 10 10:38:55 bsd-14 kernel: =
actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,000E)
> Jul 10 10:38:55 bsd-14 kernel: =
actor.state=3D3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
> Jul 10 10:38:55 bsd-14 kernel: =
partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,000C)
> Jul 10 10:38:55 bsd-14 kernel: =
partner.state=3D1d<ACTIVITY,AGGREGATION,SYNC,COLLECTING>
> Jul 10 10:38:55 bsd-14 kernel: maxdelay=3D0
> Jul 10 10:38:55 bsd-14 kernel: ql0: old pstate =
d<ACTIVITY,AGGREGATION,SYNC>
> Jul 10 10:38:55 bsd-14 kernel: ql0: new pstate =
3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
> Jul 10 10:38:56 bsd-14 kernel: ql0: enable distributing on aggregator =
[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0213,0000=
,0000)], nports 0 -> 1
> Jul 10 10:38:56 bsd-14 kernel: lacp_select_active_aggregator:
> Jul 10 10:38:56 bsd-14 kernel: =
[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000=
,0000)], speed=3D10000000000, nports=3D1
> Jul 10 10:38:56 bsd-14 kernel: =
[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0213,0000=
,0000)], speed=3D10000000000, nports=3D1
> Jul 10 10:38:56 bsd-14 kernel: active aggregator changed
> Jul 10 10:38:56 bsd-14 kernel: old =
[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000=
,0000)]
> Jul 10 10:38:56 bsd-14 kernel: new =
[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0213,0000=
,0000)]
> Jul 10 10:38:56 bsd-14 kernel: Set table 0 with 1 ports
> Jul 10 10:38:56 bsd-14 kernel: lacp_suppress_distributing
> Jul 10 10:38:56 bsd-14 kernel: ql1: marker transmit, port=3D13, =
sys=3D00:0e:1e:08:05:20, id=3D2
> Jul 10 10:38:56 bsd-14 kernel: ql0: marker transmit, port=3D12, =
sys=3D00:0e:1e:08:05:20, id=3D2
> Jul 10 10:38:56 bsd-14 kernel: ql0: mux_state 3 -> 4
> Jul 10 10:38:56 bsd-14 kernel: ql0: marker response, port=3D12, =
sys=3D00:0e:1e:08:05:20, id=3D2
> Jul 10 10:38:59 bsd-14 kernel: lacp_transit_expire
> ^C
>=20
>=20
>=20
>=20
> System 2 logs :
>=20
> Jul 10 02:38:24 bsd-15 kernel: lacp_attach[738] : lacp attached
> Jul 10 02:38:24 bsd-15 kernel: lacp_attach[740] : lacp_defined
> Jul 10 02:38:24 bsd-15 kernel: lagg0: link state changed to UP
> Jul 10 02:38:24 bsd-15 kernel: ql0: media changed 0x0 -> 0x100033, =
ether =3D 1, fdx =3D 1, link =3D 1
> Jul 10 02:38:24 bsd-15 kernel: ql0: -> UNSELECTED
> Jul 10 02:38:24 bsd-15 kernel: ql1: media changed 0x0 -> 0x100033, =
ether =3D 1, fdx =3D 1, link =3D 1
> Jul 10 02:38:24 bsd-15 kernel: ql1: -> UNSELECTED
> Jul 10 02:38:24 bsd-15 kernel: lacp_select_tx_port: no active =
aggregator
> Jul 10 02:38:25 bsd-15 kernel: ql1: port =
lagid=3D[(8000,00-0E-1E-04-2C-F0,0213,8000,000F),(0000,00-00-00-00-00-00,0=
000,0000,0000)]
> Jul 10 02:38:25 bsd-15 kernel: ql1: aggregator created
> Jul 10 02:38:25 bsd-15 kernel: ql1: aggregator =
lagid=3D[(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(0000,00-00-00-00-00-00,0=
000,0000,0000)]
> Jul 10 02:38:25 bsd-15 kernel: ql1: mux_state 0 -> 1
> Jul 10 02:38:25 bsd-15 kernel: ql0: port =
lagid=3D[(8000,00-0E-1E-04-2C-F0,0213,8000,000E),(0000,00-00-00-00-00-00,0=
000,0000,0000)]
> Jul 10 02:38:25 bsd-15 kernel: ql0: aggregator created
> Jul 10 02:38:25 bsd-15 kernel: ql0: aggregator =
lagid=3D[(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(0000,00-00-00-00-00-00,0=
000,0000,0000)]
> Jul 10 02:38:25 bsd-15 kernel: ql0: mux_state 0 -> 1
> Jul 10 02:38:26 bsd-15 kernel: ql0: lacpdu receive
> Jul 10 02:38:26 bsd-15 kernel: =
actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,000C)
> Jul 10 02:38:26 bsd-15 kernel: =
actor.state=3D85<ACTIVITY,AGGREGATION,EXPIRED>
> Jul 10 02:38:26 bsd-15 kernel: =
partner=3D(0000,00-00-00-00-00-00,0000,0000,0000)
> Jul 10 02:38:26 bsd-15 kernel: partner.state=3D2<TIMEOUT>
> Jul 10 02:38:26 bsd-15 kernel: maxdelay=3D0
> Jul 10 02:38:26 bsd-15 kernel: ql0: lacp_sm_rx_update_ntt: assert ntt
> Jul 10 02:38:26 bsd-15 kernel: ql0: old pstate 2<TIMEOUT>
> Jul 10 02:38:26 bsd-15 kernel: ql0: new pstate =
85<ACTIVITY,AGGREGATION,EXPIRED>
> Jul 10 02:38:26 bsd-15 kernel: ql0: partner timeout changed
> Jul 10 02:38:26 bsd-15 kernel: ql0: lacpdu transmit
> Jul 10 02:38:26 bsd-15 kernel: =
actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,000E)
> Jul 10 02:38:26 bsd-15 kernel: actor.state=3D5<ACTIVITY,AGGREGATION>
> Jul 10 02:38:26 bsd-15 kernel: =
partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,000C)
> Jul 10 02:38:26 bsd-15 kernel: =
partner.state=3D85<ACTIVITY,AGGREGATION,EXPIRED>
> Jul 10 02:38:26 bsd-15 kernel: maxdelay=3D0
> Jul 10 02:38:26 bsd-15 kernel: ql1: lacpdu transmit
> Jul 10 02:38:26 bsd-15 kernel: =
actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,000F)
> Jul 10 02:38:26 bsd-15 kernel: =
actor.state=3D85<ACTIVITY,AGGREGATION,EXPIRED>
> Jul 10 02:38:26 bsd-15 kernel: =
partner=3D(0000,00-00-00-00-00-00,0000,0000,0000)
> Jul 10 02:38:26 bsd-15 kernel: partner.state=3D2<TIMEOUT>
> Jul 10 02:38:26 bsd-15 kernel: maxdelay=3D0
> Jul 10 02:38:26 bsd-15 kernel: ql0: collecting disabled
> Jul 10 02:38:26 bsd-15 kernel: lacp_aggregator_delref: =
lagid=3D[(8000,00-0E-1E-04-2C-F0,0213, =
0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], refcnt 1 -> 0
>=20
> Jul 10 02:38:26 bsd-15 kernel: ql0: mux_state 1 -> 0
> Jul 10 02:38:26 bsd-15 kernel: ql0: lacpdu transmit
> Jul 10 02:38:26 bsd-15 kernel: =
actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,000E)
> Jul 10 02:38:26 bsd-15 kernel: actor.state=3D5<ACTIVITY,AGGREGATION>
> Jul 10 02:38:26 bsd-15 kernel: =
partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,000C)
> Jul 10 02:38:26 bsd-15 kernel: =
partner.state=3D85<ACTIVITY,AGGREGATION,EXPIRED>
> Jul 10 02:38:26 bsd-15 kernel: maxdelay=3D0
> Jul 10 02:38:27 bsd-15 kernel: ql0: lacpdu receive
> Jul 10 02:38:27 bsd-15 kernel: =
actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,000C)
> Jul 10 02:38:27 bsd-15 kernel: actor.state=3D5<ACTIVITY,AGGREGATION>
> Jul 10 02:38:27 bsd-15 kernel: =
partner=3D(8000,00-0E-1E-04-2C-F0,0213,8000,000E)
> Jul 10 02:38:27 bsd-15 kernel: partner.state=3D5<ACTIVITY,AGGREGATION>
> Jul 10 02:38:27 bsd-15 kernel: maxdelay=3D0
> Jul 10 02:38:27 bsd-15 kernel: ql0: old pstate =
85<ACTIVITY,AGGREGATION,EXPIRED>
> Jul 10 02:38:27 bsd-15 kernel: ql0: new pstate 5<ACTIVITY,AGGREGATION>
> Jul 10 02:38:27 bsd-15 kernel: ql1: lacp_sm_rx_timer: EXPIRED -> =
DEFAULTED
> Jul 10 02:38:27 bsd-15 kernel: ql1: partner timeout changed
> Jul 10 02:38:27 bsd-15 kernel: ql1: lacp_sm_mux_timer: aggregator =
[(8000,00-0E-1E-04-2C-F0, =
0213,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], pending 1 -> 0
> Jul 10 02:38:27 bsd-15 kernel: ql1: collecting disabled
> Jul 10 02:38:27 bsd-15 kernel: lacp_aggregator_delref: =
lagid=3D[(8000,00-0E-1E-04-2C-F0,0213, =
0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], refcnt 1 -> 0
>=20
> Jul 10 02:38:27 bsd-15 kernel: ql1: mux_state 1 -> 0
> Jul 10 02:38:27 bsd-15 kernel: ql1: lacpdu transmit
> Jul 10 02:38:27 bsd-15 kernel: =
actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,000F)
> Jul 10 02:38:27 bsd-15 kernel: =
actor.state=3D45<ACTIVITY,AGGREGATION,DEFAULTED>
> Jul 10 02:38:27 bsd-15 kernel: =
partner=3D(FFFF,00-00-00-00-00-00,0000,FFFF,0000)
> Jul 10 02:38:27 bsd-15 kernel: =
partner.state=3D3c<AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
> Jul 10 02:38:27 bsd-15 kernel: maxdelay=3D0
> Jul 10 02:38:27 bsd-15 kernel: ql0: port =
lagid=3D[(8000,00-0E-1E-04-2C-F0,0213,8000,000E),(8000,00-0E-1E-08-05-20,0=
1D3,8000,000C)]
> Jul 10 02:38:27 bsd-15 kernel: ql0: aggregator created
> Jul 10 02:38:27 bsd-15 kernel: ql0: aggregator =
lagid=3D[(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(8000,00-0E-1E-08-05-20,0=
1D3,0000,0000)]
> Jul 10 02:38:27 bsd-15 kernel: ql0: mux_state 0 -> 1
> Jul 10 02:38:28 bsd-15 kernel: ql1: port =
lagid=3D[(8000,00-0E-1E-04-2C-F0,0213,8000,000F),(FFFF,00-00-00-00-00-00,0=
000,FFFF,0000)]
> Jul 10 02:38:28 bsd-15 kernel: ql1: aggregator created
> Jul 10 02:38:28 bsd-15 kernel: ql1: aggregator =
lagid=3D[(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(FFFF,00-00-00-00-00-00,0=
000,0000,0000)]
> Jul 10 02:38:28 bsd-15 kernel: ql1: mux_state 0 -> 1
> Jul 10 02:38:29 bsd-15 kernel: ql0: lacp_sm_mux_timer: aggregator =
[(8000,00-0E-1E-04-2C-F0, =
0213,0000,0000),(8000,00-0E-1E-08-05-20,01D3,0000,0000)], pending 1 -> 0
> Jul 10 02:38:29 bsd-15 kernel: ql0: collecting disabled
> Jul 10 02:38:29 bsd-15 kernel: ql0: mux_state 1 -> 2
> Jul 10 02:38:29 bsd-15 kernel: ql0: lacpdu transmit
> Jul 10 02:38:29 bsd-15 kernel: =
actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,000E)
> Jul 10 02:38:29 bsd-15 kernel: =
actor.state=3Dd<ACTIVITY,AGGREGATION,SYNC>
> Jul 10 02:38:29 bsd-15 kernel: =
partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,000C)
> Jul 10 02:38:29 bsd-15 kernel: partner.state=3D5<ACTIVITY,AGGREGATION>
> Jul 10 02:38:29 bsd-15 kernel: maxdelay=3D0
> Jul 10 02:38:30 bsd-15 kernel: ql0: lacpdu receive
> Jul 10 02:38:30 bsd-15 kernel: =
actor=3D(8000,00-0E-1E-08-05-20,01D3,8000,000C)
> Jul 10 02:38:30 bsd-15 kernel: =
actor.state=3D1d<ACTIVITY,AGGREGATION,SYNC,COLLECTING>
> Jul 10 02:38:30 bsd-15 kernel: =
partner=3D(8000,00-0E-1E-04-2C-F0,0213,8000,000E)
> Jul 10 02:38:30 bsd-15 kernel: =
partner.state=3Dd<ACTIVITY,AGGREGATION,SYNC>
> Jul 10 02:38:30 bsd-15 kernel: maxdelay=3D0
> Jul 10 02:38:30 bsd-15 kernel: ql0: old pstate 5<ACTIVITY,AGGREGATION>
> Jul 10 02:38:30 bsd-15 kernel: ql0: new pstate =
1d<ACTIVITY,AGGREGATION,SYNC,COLLECTING>
> Jul 10 02:38:30 bsd-15 kernel: ql1: lacp_sm_mux_timer: aggregator =
[(8000,00-0E-1E-04-2C-F0, =
0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)], pending 1 -> 0
> Jul 10 02:38:30 bsd-15 kernel: ql1: collecting disabled
> Jul 10 02:38:30 bsd-15 kernel: ql1: mux_state 1 -> 2
> Jul 10 02:38:30 bsd-15 kernel: ql1: collecting enabled
> Jul 10 02:38:30 bsd-15 kernel: ql1: mux_state 2 -> 3
> Jul 10 02:38:30 bsd-15 kernel: ql1: enable distributing on aggregator =
[(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000=
,0000)], nports 0 -> 1
> Jul 10 02:38:30 bsd-15 kernel: lacp_select_active_aggregator:
> Jul 10 02:38:30 bsd-15 kernel: =
[(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000=
,0000)], speed=3D10000000000, nports=3D1
> Jul 10 02:38:30 bsd-15 kernel: active aggregator changed
> Jul 10 02:38:30 bsd-15 kernel: old (none)
> Jul 10 02:38:30 bsd-15 kernel: new =
[(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(FFFF,00-00-00 =
00-00-00,0000,0000,0000)]
> Jul 10 02:38:30 bsd-15 kernel: Set table 1 with 1 ports
> Jul 10 02:38:30 bsd-15 kernel: lacp_suppress_distributing
> Jul 10 02:38:30 bsd-15 kernel: ql1: marker transmit, port=3D15, =
sys=3D00:0e:1e:04:2c:f0, id=3D1
> Jul 10 02:38:30 bsd-15 kernel: ql0: marker transmit, port=3D14, =
sys=3D00:0e:1e:04:2c:f0, id=3D1
> Jul 10 02:38:30 bsd-15 kernel: ql1: mux_state 3 -> 4
> Jul 10 02:38:30 bsd-15 kernel: ql1: lacpdu transmit
> Jul 10 02:38:30 bsd-15 kernel: ql0: marker response, port=3D14, =
sys=3D00:0e:1e:04:2c:f0, id=3D1
> Jul 10 02:38:30 bsd-15 kernel: =
actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,000F)
> Jul 10 02:38:30 bsd-15 kernel: =
actor.state=3D7d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING,DEFAULT=
ED>
> Jul 10 02:38:30 bsd-15 kernel: =
partner=3D(FFFF,00-00-00-00-00-00,0000,FFFF,0000)
> Jul 10 02:38:30 bsd-15 kernel: =
partner.state=3D3c<AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
> Jul 10 02:38:30 bsd-15 kernel: maxdelay=3D0
> Jul 10 02:38:30 bsd-15 kernel: ql0: collecting enabled
> Jul 10 02:38:30 bsd-15 kernel: ql0: mux_state 2 -> 3
> Jul 10 02:38:30 bsd-15 kernel: ql0: enable distributing on aggregator =
[(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(8000,00-0E-1E-08-05-20,01D3,0000=
,0000)], nports 0 -> 1
> Jul 10 02:38:30 bsd-15 kernel: lacp_select_active_aggregator:
> Jul 10 02:38:30 bsd-15 kernel: =
[(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(8000,00-0E-1E-08-05-20,01D3,0000=
,0000)], speed=3D10000000000, nports=3D1
> Jul 10 02:38:30 bsd-15 kernel: =
[(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000=
,0000)], speed=3D10000000000, nports=3D1
> Jul 10 02:38:30 bsd-15 kernel: active aggregator not changed
> Jul 10 02:38:30 bsd-15 kernel: new =
[(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000=
,0000)]
> Jul 10 02:38:30 bsd-15 kernel: ql0: mux_state 3 -> 4
> Jul 10 02:38:30 bsd-15 kernel: ql0: lacpdu transmit
> Jul 10 02:38:30 bsd-15 kernel: =
actor=3D(8000,00-0E-1E-04-2C-F0,0213,8000,000E)
> Jul 10 02:38:30 bsd-15 kernel: =
actor.state=3D3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
> Jul 10 02:38:30 bsd-15 kernel: =
partner=3D(8000,00-0E-1E-08-05-20,01D3,8000,000C)
> Jul 10 02:38:30 bsd-15 kernel: =
partner.state=3D1d<ACTIVITY,AGGREGATION,SYNC,COLLECTING>
> Jul 10 02:38:30 bsd-15 kernel: maxdelay=3D0
> Jul 10 02:38:33 bsd-15 kernel: lacp_transit_expire
> ^C
>=20
> Let me know if you need more info.
>=20
> Thanks
> Adarsh
>=20
>=20
> -----Original Message-----
> From: owner-freebsd-net@freebsd.org =
[mailto:owner-freebsd-net@freebsd.org] On Behalf Of Adarsh Joshi
> Sent: Tuesday, July 10, 2012 10:08 AM
> To: Andrew Boyer
> Cc: freebsd-net@freebsd.org
> Subject: RE: lacp lagg port flags do not show correctly resulting in =
poor traffic distribution/performance
>=20
> Andrew,
>=20
> Thanks for the reply.
>=20
> The reason for my suspicion on the portflags is thus (extracted from =
the ifconfig output in my previous mail):
>=20
> System 1:
> Laggport: ql1 flags =3D 18 state =3D 7D
> Laggport: ql0 flags =3D 1c state =3D 3D
>=20
> System 2:
> Laggport: ql1 flags =3D 1c state =3D 7D
> Laggport: ql0 flags =3D 18 state =3D 3D
>=20
> I should have explained my setup to you before. Here it is.
> Both the ql0 interfaces of the 2 systems are connected using a single =
cable and ql1 interfaces of the 2 systems are connected using a single =
cable.
>=20
>               System 1                             System 2
>                              ql0 <=3D=3D=3D=3D=3D=3D=3D> ql0
>                              ql1 <=3D=3D=3D=3D=3D=3D=3D> ql1
>=20
> With this setup, I don't think it is possible for ports ql0 to talk to =
their partners (each other) and ql1 ports not getting a response from =
their partner and still get the lagg configuration I have posted.
>=20
> I thought the portflags are dependent on the LACP state. But I see =
different flags for the same LACP state (For the state 7D, ql1 on system =
1 shows flags =3D 18 and ql1 on system 2 shows flags =3D 1c).
>=20
> Or is my understanding totally wrong?
>=20
> I will send the LACP_DEBUG logs within the hour.
>=20
> Thanks
> Adarsh
>=20
> From: Andrew Boyer [mailto:aboyer@averesystems.com]
> Sent: Tuesday, July 10, 2012 5:57 AM
> To: Adarsh Joshi
> Cc: freebsd-net@freebsd.org
> Subject: Re: lacp lagg port flags do not show correctly resulting in =
poor traffic distribution/performance
>=20
>=20
> On Jul 9, 2012, at 8:38 PM, Adarsh Joshi wrote:
>=20
>=20
> Hi,
>=20
> I am trying to configure lacp lagg interfaces with 2 systems connected =
back to back as follows:
>=20
> Ifconfig lagg0 create
> Ifconfig lagg0 laggproto lacp laggport ql0 laggport ql1 192.168.100.1 =
netmask 255.255.255.0
>=20
> Sometimes, the lag interface comes up correctly but sometimes the =
laggport flags do not show properly. Instead of =
1c<ACTIVE,COLLECTING,DISTRIBUTING>, it shows values of 18. I have seen =
similar issues reported on various forums with no solution.
> Looking at the lagg driver code and reading the standard, I thought =
the laggport flags ( defined in if_lagg.h) are based on the =
LACP_STATE_BITS in file ieee8023ad_lacp.h. But the following ifconfig -v =
output does not make any sense to me.
>=20
> My concern is that when all the interfaces show flags as 1c, the =
traffic is distributed across both the interfaces uniformly and I get =
aggregated throughput. If not, the traffic flows only on 1 interface.
>=20
> Is this a bug? How do I solve this? Or am I doing something wrong?
>=20
> I am using Free-BSD 9.0 release.
>=20
> System 1:
> # ifconfig -v lagg0
>       lag id: [(8000,00-0E-1E-08-05-20,0213,0000,0000),
>                (8000,00-0E-1E-04-2C-F0,0213,0000,0000)]
>       laggport: ql1 flags=3D18<COLLECTING,DISTRIBUTING> state=3D7D
>               [(8000,00-0E-1E-08-05-20,0213,8000,000F),
>                (FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
>       laggport: ql0 flags=3D1c<ACTIVE,COLLECTING,DISTRIBUTING> =
state=3D3D
>               [(8000,00-0E-1E-08-05-20,0213,8000,000E),
>                (8000,00-0E-1E-04-2C-F0,0213,8000,000E)]
>=20
> System 2:
>=20
> # ifconfig -v lagg0
>       lag id: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),
>                (FFFF,00-00-00-00-00-00,0000,0000,0000)]
>       laggport: ql1 flags=3D1c<ACTIVE,COLLECTING,DISTRIBUTING> =
state=3D7D
>              [(8000,00-0E-1E-04-2C-F0,0213,8000,000F),
>                (FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
>       laggport: ql0 flags=3D18<COLLECTING,DISTRIBUTING> state=3D3D
>               [(8000,00-0E-1E-04-2C-F0,0213,8000,000E),
>                (8000,00-0E-1E-08-05-20,0213,8000,000E)]
>=20
>=20
> thanks
> Adarsh
>=20
> I don't think you have a port flags problem per se; the flags are =
correctly displaying the state of the lagg.  Your problem is that your =
systems aren't negotiating the correct lagg configuration.  Each tuple =
after the laggport represents the [(actor state),(partner state)].  =
Ports ql0 have been able to talk to their partners (each other).  =
Neither ql1 port has seen a response from a partner, though.
>=20
> You could try restarting the state machine on one box with 'ifconfig =
lagg0 laggproto lacp'.  To see the negotiation you'll need to rebuild =
your kernel with '#define LACP_DEBUG 1' added to the top of =
sys/net/ieee802.3ad_lacp.c.  Or upgrade to a newer stable snapshot that =
has the net.lacp_debug sysctl and turn it on.
>=20
> Or just turn off LACP.  What does it get you in this configuration?
>=20
> Hope this helps,
>  Andrew
>=20
> --------------------------------------------------
> Andrew Boyer       =
aboyer@averesystems.com<mailto:aboyer@averesystems.com>
>=20
>=20
>=20
>=20
>=20
> ________________________________
> This message and any attached documents contain information from =
QLogic Corporation or its wholly-owned subsidiaries that may be =
confidential. If you are not the intended recipient, you may not read, =
copy, distribute, or use this information. If you have received this =
transmission in error, please notify the sender immediately by reply =
e-mail and then delete this message.
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>=20
>=20
> This message and any attached documents contain information from =
QLogic Corporation or its wholly-owned subsidiaries that may be =
confidential. If you are not the intended recipient, you may not read, =
copy, distribute, or use this information. If you have received this =
transmission in error, please notify the sender immediately by reply =
e-mail and then delete this message.
>=20

--------------------------------------------------
Andrew Boyer	aboyer@averesystems.com







Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9BC66C40-6F4B-48A0-AEC0-DCAD0C6863EF>