From owner-freebsd-net@FreeBSD.ORG Thu Aug 1 22:15:10 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 774A8D79 for ; Thu, 1 Aug 2013 22:15:10 +0000 (UTC) (envelope-from joemoog@ebureau.com) Received: from internet06.ebureau.com (internet06.ebureau.com [65.127.24.25]) by mx1.freebsd.org (Postfix) with ESMTP id 4B93B2DFF for ; Thu, 1 Aug 2013 22:15:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by internet06.ebureau.com (Postfix) with ESMTP id 2A49937439E6; Thu, 1 Aug 2013 17:15:09 -0500 (CDT) X-Virus-Scanned: amavisd-new at ebureau.com Received: from internet06.ebureau.com ([127.0.0.1]) by localhost (internet06.ebureau.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lZjPsz50ufhK; Thu, 1 Aug 2013 17:14:58 -0500 (CDT) Received: from nail.office.ebureau.com (nail.office.ebureau.com [10.10.20.23]) by internet06.ebureau.com (Postfix) with ESMTPSA id BB9F637439C6; Thu, 1 Aug 2013 17:14:57 -0500 (CDT) Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1786.1\)) Subject: Re: Intel 4-port ethernet adaptor link aggregation issue From: Joe Moog In-Reply-To: Date: Thu, 1 Aug 2013 17:14:56 -0500 Message-Id: <2A0C085A-1AAF-42D7-867B-6CDD1143B4AC@ebureau.com> References: To: Ryan Stone X-Mailer: Apple Mail (2.1786.1) Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Aug 2013 22:15:10 -0000 On Aug 1, 2013, at 4:27 PM, Joe Moog wrote: > On Aug 1, 2013, at 3:55 PM, Ryan Stone wrote: >=20 >> Have you tried using only two ports, but both from the NIC? My = suspicion would be that the problem is in the lagg's handling of more = than 2 ports rather than the driver, especially given that it is the igb = driver in all cases. >=20 > Ryan: >=20 > We have done this successfully with two ports on the NIC, on another = hardware-identical host. That said, it is entirely possible that this is = a shortcoming of lagg.=20 >=20 > Can you think of any sort of workaround? Our desired implementation = really requires the inclusion of all 4 ports in the lagg. Failing this = we're looking at the likelihood of 10G ethernet, but with that comes = significant overhead, both cost and administration (before anybody tries = to force the cost debate, remember that there are 10G router modules and = 10G-capable distribution switches involved, never mind the cabling and = SFPs -- it's not just a $600 10G card for the host). I'd like to defer = that requirement as long as possible. 4 aggregated gig ports would serve = us perfectly well for the near-term. >=20 > Thanks >=20 > Joe UPDATE: After additional testing, I'm beginning to suspect the igb = driver. With our setup, ifconfig identifies all the ethernet ports as = igb(0-5). I configured igb0 with a single static IP address (say, = 192.168.1.10), and was able to connect to the host administratively. = While connected, I enabled another port as a second standalone port, = again with a unique address (say, 192.168.1.20), and was able to access = the host via that interface as well. The problem arises when we attempt = to similarly add a third interface to the mix -- and it doesn't seem to = matter what interface(s) we use, or in what order we activate them. = Always on the third interface, that third interface fails to respond = despite showing "active" both in ifconfig and on the switch. If there is anything else I could try that would be useful to help = identify where the issue may reside, please let me know. Thanks Joe