From owner-freebsd-net@FreeBSD.ORG Thu Aug 1 22:36:14 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 12807212 for ; Thu, 1 Aug 2013 22:36:14 +0000 (UTC) (envelope-from sean_bruno@yahoo.com) Received: from nm26.bullet.mail.ne1.yahoo.com (nm26.bullet.mail.ne1.yahoo.com [98.138.90.89]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B3AE42EDA for ; Thu, 1 Aug 2013 22:36:13 +0000 (UTC) Received: from [98.138.90.51] by nm26.bullet.mail.ne1.yahoo.com with NNFMP; 01 Aug 2013 22:36:06 -0000 Received: from [98.138.226.63] by tm4.bullet.mail.ne1.yahoo.com with NNFMP; 01 Aug 2013 22:36:05 -0000 Received: from [127.0.0.1] by smtp214.mail.ne1.yahoo.com with NNFMP; 01 Aug 2013 22:36:05 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1375396565; bh=2594Yt1jva7C6Jhxv9At4bRBjGqy8VaoieiccQ79/zo=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Subject:From:Reply-To:To:Cc:In-Reply-To:References:Content-Type:Date:Message-ID:Mime-Version:X-Mailer; b=3hnTqyr5Id94AeecnV0AtgQoj4+d9u3FXQlKxADPus4dkNC4z9QQ8z30pDQm0ldXnt9h9aRnm9P1IepeYRhJNArMrTbUm7nvjVJ0qwe01wkOuHgsZBn2hj8RV4pT4NlkZyhtzI71DyubXTPChGZilm3ILcUW5aUgvXb206NOM6k= X-Yahoo-Newman-Id: 911450.20909.bm@smtp214.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: iwXsPXsVM1kNhIQFGGlI1ryHQy9IVlJLBd5OAEGDzitIj_s 77S2bSuCSRzZN85ntuNmt5snh9xvaO2QzlyaRFNveZXVFa76Db06Uhtwvh2S l9mgnwQ9zmHP.0stRRdM26NpsPKsdlR0xG7V4aodfgjIDBJSxKqA2iipdKcb FpBVds_1mg_ouVEbFbf_AI4ynEGfQUYIN8lY6ffLrrNVw0V.LLTiHT1QXa67 mFu4ubsej7BPE9snZGsBEDIz8.skf9qorxxWQtbEDYCVNc4eCMEnjZz.1aYM ZgL2YyX0LdJ8kXdoe.5048RVGCv6hgyICbNEvID06t6jerjnSSP6Dxu71K8x 5gbFcL3Jnsk6UUFb8RKetpAb3yxipiKVP4Oc0ErGmRpnDTDwgthrBrjqalve 1XpihA9GNXdq4atL3cKI23OraYBiwnIw8f7H7kgfSzQzb9JP97bwzeXdL3gm ._X_ybd0kQ8xdyAx9zTtYadJvP8LwBY_epefv6gKrqdENNTgRijxzV1vBOQ8 BK3do_RX7DQWxFKuQJmYlXpZK5CHAfN2YWd80bqD1Pklj0fE6vEBO3vvNqO0 SfoxdOTA- X-Yahoo-SMTP: u5BKR6OswBC_iZJVfGRoMkTIpc8pEA4- X-Rocket-Received: from [10.73.160.242] (sean_bruno@209.131.62.116 with ) by smtp214.mail.ne1.yahoo.com with SMTP; 01 Aug 2013 15:36:05 -0700 PDT Subject: Re: Intel 4-port ethernet adaptor link aggregation issue From: Sean Bruno To: Joe Moog In-Reply-To: <2A0C085A-1AAF-42D7-867B-6CDD1143B4AC@ebureau.com> References: <2A0C085A-1AAF-42D7-867B-6CDD1143B4AC@ebureau.com> Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-UqWbkiq00zC5guDUdWJy" Date: Thu, 01 Aug 2013 15:36:04 -0700 Message-ID: <1375396564.1481.37.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Cc: freebsd-net , Ryan Stone X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Aug 2013 22:36:14 -0000 --=-UqWbkiq00zC5guDUdWJy Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable On Thu, 2013-08-01 at 17:14 -0500, Joe Moog wrote: > On Aug 1, 2013, at 4:27 PM, Joe Moog wrote: >=20 > > On Aug 1, 2013, at 3:55 PM, Ryan Stone wrote: > >=20 > >> Have you tried using only two ports, but both from the NIC? My suspic= ion would be that the problem is in the lagg's handling of more than 2 port= s rather than the driver, especially given that it is the igb driver in all= cases. > >=20 > > Ryan: > >=20 > > We have done this successfully with two ports on the NIC, on another ha= rdware-identical host. That said, it is entirely possible that this is a sh= ortcoming of lagg.=20 > >=20 > > Can you think of any sort of workaround? Our desired implementation rea= lly requires the inclusion of all 4 ports in the lagg. Failing this we're l= ooking at the likelihood of 10G ethernet, but with that comes significant o= verhead, both cost and administration (before anybody tries to force the co= st debate, remember that there are 10G router modules and 10G-capable distr= ibution switches involved, never mind the cabling and SFPs -- it's not just= a $600 10G card for the host). I'd like to defer that requirement as long = as possible. 4 aggregated gig ports would serve us perfectly well for the n= ear-term. > >=20 > > Thanks > >=20 > > Joe >=20 > UPDATE: After additional testing, I'm beginning to suspect the igb driver= . With our setup, ifconfig identifies all the ethernet ports as igb(0-5). I= configured igb0 with a single static IP address (say, 192.168.1.10), and w= as able to connect to the host administratively. While connected, I enabled= another port as a second standalone port, again with a unique address (say= , 192.168.1.20), and was able to access the host via that interface as well= . The problem arises when we attempt to similarly add a third interface to = the mix -- and it doesn't seem to matter what interface(s) we use, or in wh= at order we activate them. Always on the third interface, that third interf= ace fails to respond despite showing "active" both in ifconfig and on the s= witch. >=20 > If there is anything else I could try that would be useful to help identi= fy where the issue may reside, please let me know. >=20 > Thanks >=20 > Joe Your test seems to indicate that the *first* port on the quad-port card is causing you issues as the on-board interfaces igb0/1 are working fine. Can you bring up *any* ports on the quad-port card? Are you sure that device enumeration is correct in the host o/s and that port 1 on the aud-port card is really igb2, port 2 is igb3, etc ? Sean --=-UqWbkiq00zC5guDUdWJy Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (FreeBSD) iQEcBAABAgAGBQJR+uLUAAoJEBkJRdwI6BaHwVQIAIKIYlO2JZrSVfbidu4vXxIh TyjC8vLzLh28ExLnxqmoADi+dE4TNQULAHkQ1nWjrkMS8SmiafQLXiEk5UoBWE21 yzGDcr33HSZcAve1Mp+c7N3bhtzKFYcWeKQDzJKnnT4rsYIffxykanrn0QwrHk12 6MXZHXElYLA9OywDV9R04o63k7gn/wElRmEuzrBANoYmgV1TluOdG1U5rNnGC7dH kB0iOWnxPDybA0v8/76N+4wChjUY5hEaGOxEg0L3O+xfrbFYg69ZC+t8lt92N3uv gKls5iCYOA7yJqJw625kWt619wexc+H5156Qpp056+CkYMdwWzBht95GizAVpKM= =eXTi -----END PGP SIGNATURE----- --=-UqWbkiq00zC5guDUdWJy--