From owner-freebsd-net@FreeBSD.ORG Wed Feb 5 20:53:21 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B0C45C92 for ; Wed, 5 Feb 2014 20:53:21 +0000 (UTC) Received: from nm30-vm1.bullet.mail.ne1.yahoo.com (nm30-vm1.bullet.mail.ne1.yahoo.com [98.138.90.46]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 58BC4185A for ; Wed, 5 Feb 2014 20:53:16 +0000 (UTC) Received: from [98.138.100.112] by nm30.bullet.mail.ne1.yahoo.com with NNFMP; 05 Feb 2014 20:53:08 -0000 Received: from [98.138.84.47] by tm103.bullet.mail.ne1.yahoo.com with NNFMP; 05 Feb 2014 20:53:08 -0000 Received: from [127.0.0.1] by smtp115.mail.ne1.yahoo.com with NNFMP; 05 Feb 2014 20:53:08 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1391633588; bh=bmoWQdjlHSr2IEMME9/ZD89dx1b0WPNlSvBEoxP9+qw=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id:References:To:X-Mailer; b=DdF6vt+Lyf4siZMJ3xZz/B9CM1vWYfsKSBRTN9ddrvuO04TTb6uXUaKI0P8ci7dyzGC3WFvpCB+R4yim1ontIUuPSqNvteVpTZExJ3ml/RQEawZTiknL6j4mivu5MP2hiN7+smCQjyWabewWKwTl2gQv2VDKQZWB80RI59+ntRI= X-Yahoo-Newman-Id: 935762.26179.bm@smtp115.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: xlEth8EVM1mrFyrmVUhnVk9yRuNqEr.IH301kGwYFDMRK39 bwlczijf9w79kxPxr4214SLYowmUXRcbaPb5.ywQwpMmT9Wc.TtWDBnQ.DID r6cXJkmac0r32Out6cOO0fDvHC6XmbDv0xdM2Vn91S2ShgKc5LFUtwEZfPQR jwieyLsyMbKTYCjNxyy2CkDggMwrmt2HCPRoGtdWeuop8gt.pXj9ygzTY82X dxCce8UARn74Sk2xBBYdJ69uB1WCPxKri7OCwT9HgSw4t9iA8S3Ww.AQ_fxc 1QaM8iLQaZ84gQQiFXZC6xedsE5phNdwXPJE1anfGyfFvqEfOMzN0vuwWKBv xuT4VJatrb0xWv9e1HnJbjHi3XPqh8od5jnHEbKbgfO18x.L2KILHTGGErzy UIM5WMscyDgslI9R3wSXf697HZwyX36.yz1nhkStJV_uFadb4LmI33Oz6RQb lB5LwxfdGwI_u.84OkzysZTsTGmXaZ5gdbT.DdwSYoqph1zrF.ewY2vKXElj 61Y0Cy24kjXqrNTqNcqSDL3qsoTvwUFsSNLGOqPYHJPrP.oDi71V6BRma_aX tWfgFtyNIRtUIor9mvxVTMSm_2xND5JEuYkU41MKtVvlHh9NBbXJEsrlhcFG u_FgnvEYpuu5Uc7Qho3XjekCLPO34HsZw4uUhIkDjjIzHaIhvsA-- X-Yahoo-SMTP: clhABp.swBB7fs.LwIJpv3jkWgo2NU8- X-Rocket-Received: from [172.20.10.4] (scott4long@70.208.21.38 with plain [98.139.211.125]) by smtp115.mail.ne1.yahoo.com with SMTP; 05 Feb 2014 12:53:08 -0800 PST Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: kern/185967: Link Aggregation LAGG: LACP not working in 10.0 From: Scott Long In-Reply-To: <6AEEC659-3788-4D2D-92A9-A1F6DD59A661@ebureau.com> Date: Wed, 5 Feb 2014 13:53:05 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <922B15DE-C888-453F-AFA1-048BFA279DF1@yahoo.com> References: <6AEEC659-3788-4D2D-92A9-A1F6DD59A661@ebureau.com> To: Joe Moog X-Mailer: Apple Mail (2.1827) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Feb 2014 20:53:21 -0000 On Feb 5, 2014, at 1:14 PM, Joe Moog wrote: >> Date: Mon, 03 Feb 2014 09:40:30 +0100 >> From: Ben >> To: freebsd-net@freebsd.org >> Subject: Re: kern/185967: Link Aggregation LAGG: LACP not working in >> 10.0 >> Message-ID: <52EF55FE.8030901@niessen.ch> >> Content-Type: text/plain; charset=3Dwindows-1252; format=3Dflowed >>=20 >> Hi Scott, >>=20 >> I had tried to set it in /etc/sysctl.conf but seems it didnt work. = But=20 >> will I try again and report back. >>=20 >> The settings of the switch have not been changed and are set to LACP. = It=20 >> worked before so I guess the switch should not be the problem. Maybe=20= >> some incompatibility between FreeBSD + igb-driver + switch (Juniper=20= >> EX3300-48T). >>=20 >> I will update you after setting the sysctl setting. It seems to be=20 >> "dynamic", I guess 0 reflects the index of LACP lagg devices. Can I=20= >> switch off the strict mode globally in /etc/sysctl.conf? >>=20 >> Thanks for your help. >>=20 >> Regards >> Ben >>=20 >> On 03.02.2014 09:31, Scott Long wrote: >>> Hi, >>>=20 >>> You?re probably running into the consequences of r253687. Check to = see the value of ?sysctl net.link.lagg.0.lacp.lacp_strict_mode?. If = it?s ?1? then set it to 0. My original intention was for this to = default to 0, but apparently that didn?t happen. However, the fact that = strict mode doesn?t seem to work at all for you might hint that your = switch either isn?t configured correctly for LACP, or doesn?t actually = support LACP at all. You might want to investigate that. >>>=20 >>> Scott >>>=20 >>> On Feb 3, 2014, at 1:17 AM, Ben wrote: >>>=20 >>>> Hi, >>>>=20 >>>> I upgraded from FreeBSD 9.2-RELEASE to 10.0-RELEASE. FreeBSD 9.2 = was configured to use LACP with two igb devices. >>>>=20 >>>> Now it stopped working after the upgrade. >>>>=20 >>>> This is a screenshot of ifconfig -a after the upgrade to FreeBSD = 10.0-RELEASE: http://tinypic.com/view.php?pic=3D28jvgpw&s=3D5#.Uu9PXT1dVPM= >>>>=20 >>>> A PR is currently open: = http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dkern/185967 >>>>=20 >>>> It is set to low, but I would like somebody to have a look into it = as it obviously has a great influence on our infrastructure. The only = way to "solve" it is currently switching back to FreeBSD 9.2. >>>>=20 >>>> The suggested fix "use failover" seems not to work. >>>>=20 >>>> Thank you for your help. >>>>=20 >>>> Best regards >>>> Ben >>>>=20 >=20 > Our experience appears to differ. We have 4-pot LAGG configured on an = Intel ethernet NIC (igb drivers), connected via LACP to 4 ports on a = Cisco Cat4948, host initially configured with FreeBSD 9.2-RELEASE and = upgraded to 10.0-RELEASE. Following the upgrade, everything works as = expected without making any additional adjustments. (We did initially = have to increase the mbuf_cluster allowance to get 4-port LAGG working = with 9.2, but that may be immaterial to this conversation.) >=20 > As an outsider looking in, the issue seems to crop up in cases where = switch configurations have not been set specifically to force (active) = LACP, or it's something related to use with mixed ethernet drivers = (e.g., bge mixed with igb, as in the case of the linked PR), or possibly = with different switch manufacturer's handling of FreeBSD's LACP = negotiation (in both this case and the PR, Juniper). Whether or not this = needs to be addressed from within FreeBSD itself I will leave to the = experts. As a follow-up, Ben=92s problem was that his switch was set for passive = mode and thus not sending out heartbeats. The FreeBSD LACP driver = accidentally switched its default from permissive to strict mode, and in = doing so required the reception of heartbeats in order to operate. = Compounding the problem was that the sysctl to change the behavior was = completely untested and useless because it set the state of the ports = too late in the initialization process. These problems will be = addressed for the 10.1 release. However, once Ben set his switch to = active LACP mode, everything worked. Scott