From owner-freebsd-net@FreeBSD.ORG Thu Aug 1 18:15:58 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 18D25443 for ; Thu, 1 Aug 2013 18:15:58 +0000 (UTC) (envelope-from joemoog@ebureau.com) Received: from internet06.ebureau.com (internet06.ebureau.com [65.127.24.25]) by mx1.freebsd.org (Postfix) with ESMTP id E9777227F for ; Thu, 1 Aug 2013 18:15:57 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by internet06.ebureau.com (Postfix) with ESMTP id DF50937375F6 for ; Thu, 1 Aug 2013 13:07:03 -0500 (CDT) X-Virus-Scanned: amavisd-new at ebureau.com Received: from internet06.ebureau.com ([127.0.0.1]) by localhost (internet06.ebureau.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id paFpH8g7XYXr for ; Thu, 1 Aug 2013 13:07:02 -0500 (CDT) Received: from nail.office.ebureau.com (nail.office.ebureau.com [10.10.20.23]) by internet06.ebureau.com (Postfix) with ESMTPSA id DE71D37375DA for ; Thu, 1 Aug 2013 13:07:02 -0500 (CDT) From: Joe Moog Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Intel 4-port ethernet adaptor link aggregation issue Message-Id: Date: Thu, 1 Aug 2013 13:07:02 -0500 To: freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1786.1\)) X-Mailer: Apple Mail (2.1786.1) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Aug 2013 18:15:58 -0000 We have an iXsystems 1U server (E5) with an Intel 4-port ethernet NIC = installed, model I350-T4 (manufactured May of 2013). We're trying to = bind the 4 ports on this NIC together into a single lagg port, connected = LACP to a distribution switch (Cisco 4900-series). We are able to = successfully bind the 2 on-board ethernet ports to a single lagg, = however the NIC is not so cooperative. At first we thought we had a bad = NIC, but a replacement has not fixed the issue. We are thinking there = may be a driver limitation with these Intel ethernet NICs when = attempting to bind more than 2 ports to a lagg.=20 FreeBSD version: FreeBSD 9.1-PRERELEASE #0 r244125: Wed Dec 12 11:47:47 CST 2012 rc.conf: # LINK AGGREGATION ifconfig_igb2=3D"UP" ifconfig_igb3=3D"UP" ifconfig_igb4=3D"UP" ifconfig_igb5=3D"UP" cloned_interfaces=3D"lagg0" ifconfig_lagg0=3D"laggproto lacp laggport igb2 laggport igb3 laggport = igb4 laggport igb5" ifconfig_lagg0=3D"inet 192.168.1.14 netmask 255.255.255.0" We've confirmed that the lagg module is loaded (clearly, as the pair of = on-board ethernet ports can be bound successfully). Binding various = combinations of ports on the NIC yields odd results, as sometimes the = first one in the list does not negotiate properly, sometimes the last = one in the list fails negotiation. Adding interfaces to lagg = individually versus all at the same time does not seem to make any = difference. At one point we even tried to assign unique and separate IP = addresses to the ethernet ports individually, and only a couple of the = ports would actually come active and respond to any sort of network = activity. Due to this issue with the number of "usable" ports even = beyond the link aggregation failure, this is sort of what leads us to = believe there may be an issue with the drivers for this card. We've searched the 'net/lists fairly extensively, and have seen very few = instances where people have tried to bind more than 2 ports to a lagg = with FreeBSD. Again, 2 ports is no problem, so long as we use the = on-board ports; it's the introduction of the Intel NIC and 2 more ports = that has us stuck.=20 Has anybody had any success with such a setup?=20 Joe