From owner-freebsd-net@FreeBSD.ORG Thu Aug 1 22:49:24 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 30F784C3 for ; Thu, 1 Aug 2013 22:49:24 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-ve0-x232.google.com (mail-ve0-x232.google.com [IPv6:2607:f8b0:400c:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E21092F4E for ; Thu, 1 Aug 2013 22:49:23 +0000 (UTC) Received: by mail-ve0-f178.google.com with SMTP id ox1so3015601veb.23 for ; Thu, 01 Aug 2013 15:49:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ClCadpAHw3ThriVoeukvZ/lwIQ25bzPzBcjFZwWmyLc=; b=mkddPSdC5TuEe4FiiB42JADrzTwR4QH8l1MF9zubX8/6rGbxlCi+be1HpRCTr1S+mE moFFs6IfrBAQcyDWuHQIUbWgC3IoHwEawcSaGYIRutrmGKaGMoPix03q5NivX8E6m/Kx lKK82ejJmn87pCIQN8dvnAMbAor1cdHdTfWget5sW8Lb93HJK6vV9+K+lGrJpx8vRIGK aPzHjE5BrRf5uHmcOuuNomcRpqg7EurXXCif1LF4qwHWlXfg8YwMXuZsNiAm/lhmuoml Ew3+Dhfjj3rMw7Y4SkEemgEqBhYL6hTRwdIcxgwisp3Zi5MIdN6dmxBoj2J0g68Rp8VX 9Tdg== MIME-Version: 1.0 X-Received: by 10.220.43.71 with SMTP id v7mr1144187vce.63.1375397363067; Thu, 01 Aug 2013 15:49:23 -0700 (PDT) Received: by 10.220.159.141 with HTTP; Thu, 1 Aug 2013 15:49:23 -0700 (PDT) In-Reply-To: <2A0C085A-1AAF-42D7-867B-6CDD1143B4AC@ebureau.com> References: <2A0C085A-1AAF-42D7-867B-6CDD1143B4AC@ebureau.com> Date: Thu, 1 Aug 2013 15:49:23 -0700 Message-ID: Subject: Re: Intel 4-port ethernet adaptor link aggregation issue From: Jack Vogel To: Joe Moog Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net , Ryan Stone X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Aug 2013 22:49:24 -0000 On Thu, Aug 1, 2013 at 3:14 PM, Joe Moog wrote: > On Aug 1, 2013, at 4:27 PM, Joe Moog wrote: > > > On Aug 1, 2013, at 3:55 PM, Ryan Stone wrote: > > > >> Have you tried using only two ports, but both from the NIC? My > suspicion would be that the problem is in the lagg's handling of more than > 2 ports rather than the driver, especially given that it is the igb driver > in all cases. > > > > Ryan: > > > > We have done this successfully with two ports on the NIC, on another > hardware-identical host. That said, it is entirely possible that this is a > shortcoming of lagg. > > > > Can you think of any sort of workaround? Our desired implementation > really requires the inclusion of all 4 ports in the lagg. Failing this > we're looking at the likelihood of 10G ethernet, but with that comes > significant overhead, both cost and administration (before anybody tries to > force the cost debate, remember that there are 10G router modules and > 10G-capable distribution switches involved, never mind the cabling and SFPs > -- it's not just a $600 10G card for the host). I'd like to defer that > requirement as long as possible. 4 aggregated gig ports would serve us > perfectly well for the near-term. > > > > Thanks > > > > Joe > > UPDATE: After additional testing, I'm beginning to suspect the igb driver. > With our setup, ifconfig identifies all the ethernet ports as igb(0-5). I > configured igb0 with a single static IP address (say, 192.168.1.10), and > was able to connect to the host administratively. While connected, I > enabled another port as a second standalone port, again with a unique > address (say, 192.168.1.20), and was able to access the host via that > interface as well. The problem arises when we attempt to similarly add a > third interface to the mix -- and it doesn't seem to matter what > interface(s) we use, or in what order we activate them. Always on the third > interface, that third interface fails to respond despite showing "active" > both in ifconfig and on the switch. > > If there is anything else I could try that would be useful to help > identify where the issue may reside, please let me know. > > Well, you're using a PRERELEASE of 9.1, leads me to wonder how old the igb driver is also. First step would be to try all this on more recent bits... I'd go for HEAD or at least 9.2 BETA as a start. We don't use or test lagg within Intel, but we've tested the quad port adapter and not seen an issue with a third port not working. Good luck, Jack