From owner-freebsd-net@FreeBSD.ORG Wed Jan 27 13:43:57 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F204D106568D for ; Wed, 27 Jan 2010 13:43:56 +0000 (UTC) (envelope-from shteryana@gmail.com) Received: from mail-ew0-f218.google.com (mail-ew0-f218.google.com [209.85.219.218]) by mx1.freebsd.org (Postfix) with ESMTP id 8078D8FC1E for ; Wed, 27 Jan 2010 13:43:56 +0000 (UTC) Received: by ewy10 with SMTP id 10so1265853ewy.3 for ; Wed, 27 Jan 2010 05:43:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:reply-to:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=TJOQnRwvHegq9xq8r0NU/MM8L2hyfqPKxvNPjqWtkS0=; b=q++zo8AikMlKSthohT6niGDp4kElw37cGQ1hm7qLv9XMUYhEloEwOU5sav/Rl8+Bbi +BKZJ4KmYuv3QEibopBBvpNuu3ecUyx0y9Q+hTzP/+lgNBinSMWfGo+dl34yt8gEPD2U Js5etvxOVFJDAVKIn8EE6E76mrtqN82TLRTxU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:reply-to:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=YhGkWT5+Gv4SkVvcKR1bhJX9Zr6Hex0IbmXIfHEy2Zm6+HHKOws1a9z4rvRCnHzZG0 vhpZMEv5Ut9IM37ebPSlGKMwiabvZ1GPB0VA/HowJTJxIsvRnrEBCHkx2vEN0P5oPF5K l9gn8rpaLJQuMW4P/mQPkjoNoFNV/VCequ5A4= MIME-Version: 1.0 Sender: shteryana@gmail.com Received: by 10.213.48.10 with SMTP id p10mr1322831ebf.18.1264599835000; Wed, 27 Jan 2010 05:43:55 -0800 (PST) In-Reply-To: <8E2B6E11A79B4958B94B44A41C7D4298@multiplay.co.uk> References: <8E2B6E11A79B4958B94B44A41C7D4298@multiplay.co.uk> Date: Wed, 27 Jan 2010 15:43:54 +0200 X-Google-Sender-Auth: a69e11204942f103 Message-ID: <61b573981001270543k6514a5g8d4f679777b69b49@mail.gmail.com> From: Shteryana Shopova To: Steven Hartland Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: Re: Problems getting lagg to balance using lacp X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: syrinx@FreeBSD.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jan 2010 13:43:57 -0000 On Wed, Jan 27, 2010 at 2:40 PM, Steven Hartland wrote: > I've just setup lagg on one of our servers and run some > basic tests and although everything seems to be reporting > correctly no balancing appears to be happening. > > The following is the setup: > > FreeBSD side: > ifconfig_em0=3D"up" > ifconfig_em1=3D"up" > cloned_interfaces=3D"lagg0" > ifconfig_lagg0=3D"laggproto lacp laggport em0 laggport em1" > ipv4_addrs_lagg0=3D"10.10.1.66/24" > > Cisco side: > interface Port-channel2 > description SERVER:: test - lagg0 > switchport > switchport access vlan 96 > switchport mode access > no ip address > end > > interface GigabitEthernet3/35 > description SERVER:: test - em0 > switchport > switchport access vlan 96 > switchport mode access > no ip address > channel-protocol lacp > channel-group 2 mode active > end > > interface GigabitEthernet3/43 > description SERVER:: test - em1 > switchport > switchport access vlan 96 > switchport mode access > no ip address > channel-protocol lacp > channel-group 2 mode active > end > > Config:- > em0: flags=3D8843 metric 0 mtu 15= 00 > =C2=A0 =C2=A0 =C2=A0 options=3D19b > =C2=A0 =C2=A0 =C2=A0 ether 00:30:48:33:ec:44 > =C2=A0 =C2=A0 =C2=A0 media: Ethernet autoselect (1000baseT ) > =C2=A0 =C2=A0 =C2=A0 status: active > em1: flags=3D8843 metric 0 mtu 15= 00 > =C2=A0 =C2=A0 =C2=A0 options=3D19b > =C2=A0 =C2=A0 =C2=A0 ether 00:30:48:33:ec:44 > =C2=A0 =C2=A0 =C2=A0 media: Ethernet autoselect (1000baseT ) > =C2=A0 =C2=A0 =C2=A0 status: active > lo0: flags=3D8049 metric 0 mtu 16384 > =C2=A0 =C2=A0 =C2=A0 options=3D3 > =C2=A0 =C2=A0 =C2=A0 inet 127.0.0.1 netmask 0xff000000 lagg0: > flags=3D8843 metric 0 mtu 1500 > =C2=A0 =C2=A0 =C2=A0 options=3D19b > =C2=A0 =C2=A0 =C2=A0 ether 00:30:48:33:ec:44 > =C2=A0 =C2=A0 =C2=A0 inet 10.10.1.66 netmask 0xffffff00 broadcast 10.10.1= .255 > =C2=A0 =C2=A0 =C2=A0 inet 10.10.1.83 netmask 0xffffffff broadcast 10.10.1= .83 > =C2=A0 =C2=A0 =C2=A0 media: Ethernet autoselect > =C2=A0 =C2=A0 =C2=A0 status: active > =C2=A0 =C2=A0 =C2=A0 laggproto lacp > =C2=A0 =C2=A0 =C2=A0 laggport: em1 flags=3D1c > =C2=A0 =C2=A0 =C2=A0 laggport: em0 flags=3D1c > > show lacp neighbor =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Flags: =C2=A0S - Device= is > requesting Slow LACPDUs =C2=A0 =C2=A0 =C2=A0 F - Device is requesting Fas= t LACPDUs > =C2=A0 =C2=A0 =C2=A0 A - Device is in Active mode =C2=A0 =C2=A0 =C2=A0 P = - Device is in Passive mode > Channel group 2 neighbors > > Partner's information: > > =C2=A0 =C2=A0 =C2=A0 =C2=A0 Partner Partner =C2=A0 LACP Partner =C2=A0Par= tner =C2=A0 Partner =C2=A0Partner > Partner > Port =C2=A0 =C2=A0 =C2=A0Flags =C2=A0 State =C2=A0 =C2=A0 Port Priority A= dmin Key Oper Key Port Number > Port State > Gi3/35 =C2=A0 =C2=A0SA =C2=A0 =C2=A0 =C2=A0bndl =C2=A0 =C2=A0 =C2=A032768= =C2=A0 =C2=A0 =C2=A0 =C2=A0 0x0 =C2=A0 =C2=A0 =C2=A0 0x90 =C2=A0 =C2=A0 0x= 1 > 0x3D =C2=A0Gi3/43 =C2=A0 =C2=A0SA =C2=A0 =C2=A0 =C2=A0bndl =C2=A0 =C2=A0 = =C2=A032768 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0x0 =C2=A0 =C2=A0 =C2=A0 0x90 =C2= =A0 =C2=A0 0x2 > =C2=A0 0x3D > > Stats: > > =C2=A0 =C2=A0 Interface =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Traffic =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Peak =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0Total > =C2=A0 =C2=A0 =C2=A0 =C2=A0 lagg0 =C2=A0in =C2=A0 =C2=A0 =C2=A00.351 KB/s= =C2=A0 =C2=A0 =C2=A0 =C2=A0117.349 MB/s =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A02.695 GB > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0out =C2=A0 =C2=A0 = 0.345 KB/s =C2=A0 =C2=A0 =C2=A0 =C2=A0114.183 MB/s =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A08.103 GB > > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 em1 =C2=A0in =C2=A0 =C2=A0 =C2=A00.233= KB/s =C2=A0 =C2=A0 =C2=A0 =C2=A0117.349 MB/s =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A02.695 GB > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0out =C2=A0 =C2=A0 = 0.000 KB/s =C2=A0 =C2=A0 =C2=A0 =C2=A0132.535 MB/s =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A08.890 GB > > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 em0 =C2=A0in =C2=A0 =C2=A0 =C2=A00.116= KB/s =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A02.701 KB/s =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0331.405 KB > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0out =C2=A0 =C2=A0 = 0.345 KB/s =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A09.093 KB/s =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0631.883 KB > > > Name =C2=A0 =C2=A0Mtu Network =C2=A0 =C2=A0 =C2=A0 Address =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Ipkts Ierrs =C2=A0 =C2=A0Opkts Oerrs > =C2=A0Coll > em0 =C2=A0 =C2=A01500 =C2=A0 =C2=A0 =C2=A000:30:48:33:ec:44 =C2= =A0 =C2=A0 5381 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 5188 =C2=A0 =C2=A0 0 > =C2=A0 0 > em1 =C2=A0 =C2=A01500 =C2=A0 =C2=A0 =C2=A000:30:48:33:ec:44 =C2= =A05987608 =C2=A0 =C2=A0 0 =C2=A04015584 =C2=A0 =C2=A0 0 > =C2=A0 0 > lo0 =C2=A0 16384 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0= 0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 0 > =C2=A0 0 > lo0 =C2=A0 16384 your-net =C2=A0 =C2=A0 =C2=A0localhost =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 - =C2=A0 =C2=A0 =C2= =A0 =C2=A00 =C2=A0 =C2=A0 - > =C2=A0 - > lagg0 =C2=A01500 =C2=A0 =C2=A0 =C2=A000:30:48:33:ec:44 =C2=A0599= 2849 =C2=A0 =C2=A0 0 =C2=A04016823 =C2=A0 =C2=A0 8 > =C2=A0 0 > lagg0 =C2=A01500 10.10.1.0 =C2=A0 test =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= 5985503 =C2=A0 =C2=A0 - =C2=A06475013 =C2=A0 =C2=A0 - =C2=A0 =C2=A0 - > lagg0 =C2=A01500 10.10.1.83/ jail1 =C2=A0 =C2=A0 =C2=A0 19 =C2=A0 =C2=A0 = - =C2=A0 =C2=A0 =C2=A0 =C2=A00 =C2=A0 =C2=A0 - =C2=A0 =C2=A0 - > > > Anyone got any ideas, as far as I can see lacp is negotiating correctly j= ust > no balancing is happening :( > > =C2=A0 Regards > =C2=A0 Steve > What types of traffic streams are you testing this with? if_lagg will use the SRC/DST MACs and IP addresses for IP traffic to decide which member port of the lagg to sent the traffic out to, balancing of incoming traffic should be done by the Cisco on the same principle. Try a different laggproto for the if_lagg interface e.g. roundrobin with channel-group mode on set on the switchports. cheers, Shteryana