From owner-freebsd-net@FreeBSD.ORG Thu Aug 1 20:55:08 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 32871D80 for ; Thu, 1 Aug 2013 20:55:08 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-ob0-x235.google.com (mail-ob0-x235.google.com [IPv6:2607:f8b0:4003:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F1A172A02 for ; Thu, 1 Aug 2013 20:55:07 +0000 (UTC) Received: by mail-ob0-f181.google.com with SMTP id dn14so4694164obc.26 for ; Thu, 01 Aug 2013 13:55:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=FUoQgcoETc9pgWBvvqivyitUWn8TvENzyxoISgEjzGc=; b=sk/sAkCDSuhobR/TbKsn6JU1COvKurjbDs47Bt3Fev+IDYFOFA+6TAIspi7GjW3tRO YNHdWBcFVvzl3wssHTTmgfmWNuiqRnCOhdkbnRHg4CM91UizGPyCUv4wqsZu5YHGPZlF CqJWZfiGKCnwaH1wpSXx2DMkt7gQJK9YYwp+bQctFRwG8V5WwCRvZNU8z8G13LGTaR/8 JzCKhTcFNj7s0saXby0WdV4J6KN2bVlZs/nfzP1HnpguRyCOIR+U9lSXIrPeycnBZ8V3 Hx+sl8W3JqpOAZx86XktcbkYC4BzV4BWnhNfUif8BKPWw4MC3mqgpd5UF2YLWdMkvUVe qLYw== MIME-Version: 1.0 X-Received: by 10.182.165.198 with SMTP id za6mr2604357obb.88.1375390507295; Thu, 01 Aug 2013 13:55:07 -0700 (PDT) Received: by 10.76.118.11 with HTTP; Thu, 1 Aug 2013 13:55:07 -0700 (PDT) In-Reply-To: References: Date: Thu, 1 Aug 2013 16:55:07 -0400 Message-ID: Subject: Re: Intel 4-port ethernet adaptor link aggregation issue From: Ryan Stone To: Joe Moog Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Aug 2013 20:55:08 -0000 On Thu, Aug 1, 2013 at 2:07 PM, Joe Moog wrote: > We have an iXsystems 1U server (E5) with an Intel 4-port ethernet NIC > installed, model I350-T4 (manufactured May of 2013). We're trying to bind > the 4 ports on this NIC together into a single lagg port, connected LACP to > a distribution switch (Cisco 4900-series). We are able to successfully bind > the 2 on-board ethernet ports to a single lagg, however the NIC is not so > cooperative. At first we thought we had a bad NIC, but a replacement has > not fixed the issue. We are thinking there may be a driver limitation with > these Intel ethernet NICs when attempting to bind more than 2 ports to a > lagg. > > FreeBSD version: > FreeBSD 9.1-PRERELEASE #0 r244125: Wed Dec 12 11:47:47 CST 2012 > > rc.conf: > # LINK AGGREGATION > ifconfig_igb2="UP" > ifconfig_igb3="UP" > ifconfig_igb4="UP" > ifconfig_igb5="UP" > cloned_interfaces="lagg0" > ifconfig_lagg0="laggproto lacp laggport igb2 laggport igb3 laggport igb4 > laggport igb5" > ifconfig_lagg0="inet 192.168.1.14 netmask 255.255.255.0" > > We've confirmed that the lagg module is loaded (clearly, as the pair of > on-board ethernet ports can be bound successfully). Binding various > combinations of ports on the NIC yields odd results, as sometimes the first > one in the list does not negotiate properly, sometimes the last one in the > list fails negotiation. Adding interfaces to lagg individually versus all > at the same time does not seem to make any difference. At one point we even > tried to assign unique and separate IP addresses to the ethernet ports > individually, and only a couple of the ports would actually come active and > respond to any sort of network activity. Due to this issue with the number > of "usable" ports even beyond the link aggregation failure, this is sort of > what leads us to believe there may be an issue with the drivers for this > card. > > We've searched the 'net/lists fairly extensively, and have seen very few > instances where people have tried to bind more than 2 ports to a lagg with > FreeBSD. Again, 2 ports is no problem, so long as we use the on-board > ports; it's the introduction of the Intel NIC and 2 more ports that has us > stuck. > > Has anybody had any success with such a setup? > > Joe > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > Have you tried using only two ports, but both from the NIC? My suspicion would be that the problem is in the lagg's handling of more than 2 ports rather than the driver, especially given that it is the igb driver in all cases.