From owner-freebsd-stable@FreeBSD.ORG Wed Jul 4 18:31:31 2007 Return-Path: X-Original-To: stable@freebsd.org Delivered-To: freebsd-stable@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 85FE716A400 for ; Wed, 4 Jul 2007 18:31:31 +0000 (UTC) (envelope-from thompsa@FreeBSD.org) Received: from heff.fud.org.nz (203-109-251-39.static.bliink.ihug.co.nz [203.109.251.39]) by mx1.freebsd.org (Postfix) with ESMTP id AC48713C468 for ; Wed, 4 Jul 2007 18:31:30 +0000 (UTC) (envelope-from thompsa@FreeBSD.org) Received: by heff.fud.org.nz (Postfix, from userid 1001) id E6A531CC58; Thu, 5 Jul 2007 06:19:03 +1200 (NZST) Date: Thu, 5 Jul 2007 06:19:03 +1200 From: Andrew Thompson To: Pete French Message-ID: <20070704181903.GA26719@heff.fud.org.nz> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.13 (2006-08-11) Cc: stable@freebsd.org Subject: Re: should if_lagg balance outbound traffic on an lacp connection ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2007 18:31:31 -0000 On Wed, Jul 04, 2007 at 03:44:06PM +0100, Pete French wrote: > Having recently discovered if_lagg in stable I have spent the last couple > of days experimentsin with this and using it to aggregate connections between > a pair of servers and a pair of Cisco switches. It all appears to be > functioning, but looking at the stats I see this: > > Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll > bce0 1500 00:19:bb:33:00:de 7545268 0 9303330 0 0 > bce1 1500 00:19:bb:33:00:de 7099168 0 126 0 0 > lo0 16384 15377 0 15376 0 0 > lo0 16384 fe80:3::1 fe80:3::1 0 - 0 - - > lo0 16384 localhost ::1 60 - 60 - - > lo0 16384 your-net localhost 14222 - 14221 - - > lagg0 1500 00:19:bb:33:00:de 7545187 0 9303208 58 0 > lagg0 1500 10.17.16/20 turpentine 7539218 - 9305974 - - > > Note that whilst I am getting evenly split traffic comming in from the switch, > outgoing traffic is all heading out on bce0. The ifconfig output looks > like this: > > options=3b > ether 00:19:bb:33:00:de > media: Ethernet autoselect (1000baseTX ) > status: active > lagg: laggdev lagg0 > bce1: flags=8843 mtu 1500 > options=3b > ether 00:19:bb:33:00:de > media: Ethernet autoselect (1000baseTX ) > status: active > lagg: laggdev lagg0 > lo0: flags=8049 mtu 16384 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 > inet6 ::1 prefixlen 128 > inet 127.0.0.1 netmask 0xff000000 > lagg0: flags=8843 mtu 1500 > inet 10.17.19.0 netmask 0xfffff000 broadcast 10.17.31.255 > ether 00:19:bb:33:00:de > media: Ethernet autoselect > status: active > lagg: laggproto lacp > laggport bce1 =18 > laggport bce0 =1c > bce1 is a concern here as it is not in the ACTIVE state. On your switch have a look at the lacp stats, here is an example from mine with a 4 port aggregation. c2950#sh lacp neighbor Flags: S - Device is requesting Slow LACPDUs F - Device is requesting Fast LACPDUs A - Device is in Active mode P - Device is in Passive mode Channel group 1 neighbors Partner's information: LACP port Oper Port Port Port Flags Priority Dev ID Age Key Number State Fa0/11 SA 32768 0005.5d71.8db8 4s 0xE6 0x1 0x3D Fa0/12 SA 32768 0005.5d71.8db8 5s 0xE6 0x2 0x3D Fa0/13 SA 32768 0005.5d71.8db8 4s 0xE6 0x3 0x3D Fa0/14 SA 32768 0005.5d71.8db8 3s 0xE6 0x4 0x3D c2950#sh lacp internal Flags: S - Device is requesting Slow LACPDUs F - Device is requesting Fast LACPDUs A - Device is in Active mode P - Device is in Passive mode Channel group 1 LACP port Admin Oper Port Port Port Flags State Priority Key Key Number State Fa0/11 SA bndl 32768 0x1 0x1 0xB 0x3D Fa0/12 SA bndl 32768 0x1 0x1 0xC 0x3D Fa0/13 SA bndl 32768 0x1 0x1 0xD 0x3D Fa0/14 SA bndl 32768 0x1 0x1 0xE 0x3D As you can see all ports are in 'bndl' state which means they have been negoiated. lagg0: flags=8843 metric 0 mtu 1500 ether 00:05:5d:71:8d:b8 media: Ethernet autoselect status: active lagg: laggproto lacp laggport ste3 =1c laggport ste2 =1c laggport ste1 =1c laggport ste0 =1c And all ports here are ACTIVE. I have been meaning to add IOCTLS to display more lacp stats to help debug this sort of thing. Andrew