From owner-freebsd-net@freebsd.org Wed Mar 14 01:09:53 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F367AF32F32 for ; Wed, 14 Mar 2018 01:09:52 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6815C8005A for ; Wed, 14 Mar 2018 01:09:51 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id w2E19c4G010050 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 14 Mar 2018 02:09:39 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: justlikeef@gmail.com Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id w2E19XRV046384 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 14 Mar 2018 08:09:33 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Cisco shows LACP not enabled on 11.1U2, BCM 57810S-t, 2960X IOS 15.2 To: Rob Hutton , freebsd-net@freebsd.org References: From: Eugene Grosbein Message-ID: <5AA87648.5060604@grosbein.net> Date: Wed, 14 Mar 2018 08:09:28 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=2.2 required=5.0 tests=BAYES_00, LOCAL_FROM, RDNS_NONE autolearn=no autolearn_force=no version=3.4.1 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS * 2.6 LOCAL_FROM From my domains X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on hz.grosbein.net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Mar 2018 01:09:53 -0000 14.03.2018 1:28, Rob Hutton wrote: > Trying to get to a LACP Trunk config working on FreeNAS, but I'm not sure > if this is an upstream issue. First step is just LACP bundle in edge mode > (no vlans) and bundle will not form. The FreeNAS side shows it as being > active. The switch is reporting that LACP is not enabled on the individual > ports. We have tried multiple versions of IOS from 15.0 to 15.2(6)E1. I > have about 10 windows servers with the same NICs on the same switch that > are negotiating LACP successfully as well as tagging vlans, so the hardware > pieces seem to be OK. I have tried configuring from both the GUI and the > CL with the same result. > > Cisco 2960X > > switch config: > interface Port-channel10 > switchport access vlan 1900 > switchport mode access > ! > interface GigabitEthernet1/0/27 > switchport access vlan 1900 > switchport mode access > channel-group 10 mode active > ! > interface GigabitEthernet1/0/28 > switchport access vlan 1900 > switchport mode access > channel-group 10 mode active > > logs from switch: > Mar 13 17:38:09.626: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/27, > changed state to up > Mar 13 17:38:09.665: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/28, > changed state to up > Mar 13 17:38:15.753: %EC-5-L3DONTBNDL2: Gi1/0/28 suspended: LACP currently > not enabled on the remote port. > Mar 13 17:38:16.005: %EC-5-L3DONTBNDL2: Gi1/0/27 suspended: LACP currently > not enabled on the remote port. > > Switch#sh etherc summ > Flags: D - down P - bundled in port-channel > I - stand-alone s - suspended > H - Hot-standby (LACP only) > R - Layer3 S - Layer2 > U - in use N - not in use, no aggregation > f - failed to allocate aggregator > > M - not in use, minimum links not met > m - not in use, port not aggregated due to minimum links not met > u - unsuitable for bundling > w - waiting to be aggregated > d - default port > > A - formed by Auto LAG > > > Number of channel-groups in use: 1 > Number of aggregators: 1 > > Group Port-channel Protocol Ports > ------+-------------+-----------+----------------------------------------------- > 10 Po10(SD) LACP Gi1/0/27(w) Gi1/0/28(w) > > Switch#sh int gi1/0/27 > GigabitEthernet1/0/27 is up, line protocol is down (notconnect) > > Switch#sh int gi1/0/28 > GigabitEthernet1/0/28 is up, line protocol is down (notconnect) > > > FreeNAS > ifconfig > > bxe0: flags=8843 metric 0 mtu 1500 > options=527bb M,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO> > ether 00:0e:1e:86:7c:70 > hwaddr 00:0e:1e:86:7c:70 > nd6 options=9 > media: Ethernet autoselect (10Gbase-T ) > status: active > > bxe1: flags=8843 metric 0 mtu 1500 > options=527bb M,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO> > ether 00:0e:1e:86:7c:70 > hwaddr 00:0e:1e:86:7c:72 > nd6 options=9 > media: Ethernet autoselect (10Gbase-T ) > status: active > > lagg0: flags=8843 metric 0 mtu 1500 > options=527bb M,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO> > ether 00:0e:1e:86:7c:70 > nd6 options=9 > media: Ethernet autoselect > status: active > groups: lagg > laggproto lacp lagghash l2,l3,l4 > laggport: bxe0 flags=0<> > laggport: bxe1 flags=0<> > > Last edited: 12 minutes ago See also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=213606 In short: bxe(4) driver fails to setup hardware so it would receive incoming multicast LACP frames unless switched to promisc. mode. For now, replace the NIC with one from distinct manufacturer (Intel etc.)