Date: Thu, 5 Jul 2007 06:19:03 +1200 From: Andrew Thompson <thompsa@FreeBSD.org> To: Pete French <petefrench@ticketswitch.com> Cc: stable@freebsd.org Subject: Re: should if_lagg balance outbound traffic on an lacp connection ? Message-ID: <20070704181903.GA26719@heff.fud.org.nz> In-Reply-To: <E1I665O-0005gC-3r@dilbert.ticketswitch.com> References: <E1I665O-0005gC-3r@dilbert.ticketswitch.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Jul 04, 2007 at 03:44:06PM +0100, Pete French wrote:
> Having recently discovered if_lagg in stable I have spent the last couple
> of days experimentsin with this and using it to aggregate connections between
> a pair of servers and a pair of Cisco switches. It all appears to be
> functioning, but looking at the stats I see this:
>
> Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll
> bce0 1500 <Link#1> 00:19:bb:33:00:de 7545268 0 9303330 0 0
> bce1 1500 <Link#2> 00:19:bb:33:00:de 7099168 0 126 0 0
> lo0 16384 <Link#3> 15377 0 15376 0 0
> lo0 16384 fe80:3::1 fe80:3::1 0 - 0 - -
> lo0 16384 localhost ::1 60 - 60 - -
> lo0 16384 your-net localhost 14222 - 14221 - -
> lagg0 1500 <Link#4> 00:19:bb:33:00:de 7545187 0 9303208 58 0
> lagg0 1500 10.17.16/20 turpentine 7539218 - 9305974 - -
>
> Note that whilst I am getting evenly split traffic comming in from the switch,
> outgoing traffic is all heading out on bce0. The ifconfig output looks
> like this:
>
> options=3b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
> ether 00:19:bb:33:00:de
> media: Ethernet autoselect (1000baseTX <full-duplex>)
> status: active
> lagg: laggdev lagg0
> bce1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> options=3b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
> ether 00:19:bb:33:00:de
> media: Ethernet autoselect (1000baseTX <full-duplex>)
> status: active
> lagg: laggdev lagg0
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
> inet6 ::1 prefixlen 128
> inet 127.0.0.1 netmask 0xff000000
> lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> inet 10.17.19.0 netmask 0xfffff000 broadcast 10.17.31.255
> ether 00:19:bb:33:00:de
> media: Ethernet autoselect
> status: active
> lagg: laggproto lacp
> laggport bce1 =18<COLLECTING,DISTRIBUTING>
> laggport bce0 =1c<ACTIVE,COLLECTING,DISTRIBUTING>
>
bce1 is a concern here as it is not in the ACTIVE state. On your switch
have a look at the lacp stats, here is an example from mine with a 4
port aggregation.
c2950#sh lacp neighbor
Flags: S - Device is requesting Slow LACPDUs
F - Device is requesting Fast LACPDUs
A - Device is in Active mode P - Device is in Passive mode
Channel group 1 neighbors
Partner's information:
LACP port Oper Port Port
Port Flags Priority Dev ID Age Key Number State
Fa0/11 SA 32768 0005.5d71.8db8 4s 0xE6 0x1 0x3D
Fa0/12 SA 32768 0005.5d71.8db8 5s 0xE6 0x2 0x3D
Fa0/13 SA 32768 0005.5d71.8db8 4s 0xE6 0x3 0x3D
Fa0/14 SA 32768 0005.5d71.8db8 3s 0xE6 0x4 0x3D
c2950#sh lacp internal
Flags: S - Device is requesting Slow LACPDUs
F - Device is requesting Fast LACPDUs
A - Device is in Active mode P - Device is in Passive mode
Channel group 1
LACP port Admin Oper Port Port
Port Flags State Priority Key Key Number State
Fa0/11 SA bndl 32768 0x1 0x1 0xB 0x3D
Fa0/12 SA bndl 32768 0x1 0x1 0xC 0x3D
Fa0/13 SA bndl 32768 0x1 0x1 0xD 0x3D
Fa0/14 SA bndl 32768 0x1 0x1 0xE 0x3D
As you can see all ports are in 'bndl' state which means they have been negoiated.
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 00:05:5d:71:8d:b8
media: Ethernet autoselect
status: active
lagg: laggproto lacp
laggport ste3 =1c<ACTIVE,COLLECTING,DISTRIBUTING>
laggport ste2 =1c<ACTIVE,COLLECTING,DISTRIBUTING>
laggport ste1 =1c<ACTIVE,COLLECTING,DISTRIBUTING>
laggport ste0 =1c<ACTIVE,COLLECTING,DISTRIBUTING>
And all ports here are ACTIVE. I have been meaning to add IOCTLS to
display more lacp stats to help debug this sort of thing.
Andrew
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070704181903.GA26719>