From owner-freebsd-net@FreeBSD.ORG Mon Feb 3 10:01:52 2014 Return-Path: <owner-freebsd-net@FreeBSD.ORG> Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F774668 for <freebsd-net@freebsd.org>; Mon, 3 Feb 2014 10:01:52 +0000 (UTC) Received: from nm7-vm6.bullet.mail.ne1.yahoo.com (nm7-vm6.bullet.mail.ne1.yahoo.com [98.138.91.100]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3066A14AF for <freebsd-net@freebsd.org>; Mon, 3 Feb 2014 10:01:51 +0000 (UTC) Received: from [98.138.101.132] by nm7.bullet.mail.ne1.yahoo.com with NNFMP; 03 Feb 2014 09:58:29 -0000 Received: from [98.138.226.58] by tm20.bullet.mail.ne1.yahoo.com with NNFMP; 03 Feb 2014 09:58:29 -0000 Received: from [127.0.0.1] by smtp209.mail.ne1.yahoo.com with NNFMP; 03 Feb 2014 09:58:29 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1391421509; bh=E93OgI/imELzsRZY6yqpolQ18HmQi1S7FrrA96/8UoQ=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id:References:To:X-Mailer; b=MIJCUIFNfhyhrgN/BCcpGrIc/SUU/2h/rsLDNTKlbIYl2aO6etKg/QYoaepXNvWsdR5OajrT9usJ8xFj0QYTcynPys5/T11RMELW41Vkdvdg7Er4FST4OvXnjAYLxSAq7silbQf2EL/HHeFj+f5GZIcofmokym4KD42iKFNRFyo= X-Yahoo-Newman-Id: 531205.5257.bm@smtp209.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: FtRZJe0VM1lWRH.V3d.Eey9Jgs0CxPrXHKHpoBMyHahOL5g IjO8rwjHkTPTxKqice0lx_SjBTw9qLVTLS5BXlvNc9j2iKYJtdaH602iZ.yS WJaeKSbeZhlgiJwsKnQW5vFKYx3fLa0rbxusbY0vImaJ6MH4BLkqquoWggO3 .qnbXEkouiPk.bjxIGNsmSw84eJ.ZqX1Djmj0weOzsc1CFIu4XShyVCZVTQP 3p35ONMarT.luUL7iPtHIoJf1_QFuVL1VJcIGq21PvP9U1zKevRJDFba8Eru Gxa8B6CHkIOdT3OooqRHyaeoEs15biGfbFHVVN3PdXCYW_TZ_2e0PnlkJ_Rn IzM0pIPvZBWsopO4O5Ksb2c91niLgKV3iuChONCQYSFnw4HvLJQz6yBJmjIR pc6OF4z.Nnx2vmcjYScGtSUKIE.u0EkU3pGBpRpCGgShWosPGVsCSoL6mZU6 6aQYqDKp4sVeuopgGvkN2I9juw7FBYBsqM2wEsMO174ewx4r.nxw_2xb1xnk CYOl3S9_yizYyRbHZpouKS894DEqNYrnRZ9jvD8V3Zn_TMdnRiauuYd3NhPX aPbBWLwlRxV8LhDvylgX5o5dCeYmE6UUQ92QGfrjvSMR9kgkJ5a1HoKD59rX m2xua_bp0r_z.WZc9XDop1QCUIYyD18OqS_u0kKJKPAJAOAnVmyI- X-Yahoo-SMTP: clhABp.swBB7fs.LwIJpv3jkWgo2NU8- X-Rocket-Received: from [10.64.24.117] (scott4long@69.53.236.251 with plain [98.139.211.125]) by smtp209.mail.ne1.yahoo.com with SMTP; 03 Feb 2014 09:58:29 +0000 UTC Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: kern/185967: Link Aggregation LAGG: LACP not working in 10.0 From: Scott Long <scott4long@yahoo.com> In-Reply-To: <52EF6690.3010509@niessen.ch> Date: Mon, 3 Feb 2014 02:58:26 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <202BD17C-E68A-4B27-B7EF-E5D84AA89176@yahoo.com> References: <52EF50A7.1050205@niessen.ch> <1C608452-6F29-486D-BC0F-CCC7853665C7@yahoo.com> <52EF55FE.8030901@niessen.ch> <1798FE17-5718-4125-8B00-1B00DC44B828@yahoo.com> <52EF5D1E.2000306@niessen.ch> <B2A60D0D-26AA-4EB4-B5E6-D44164C9AE83@yahoo.com> <52EF6194.5060305@niessen.ch> <8585EA2E-116E-45A6-877D-DC8D4460C965@yahoo.com> <52EF6690.3010509@niessen.ch> To: Ben <mailinglists@niessen.ch> X-Mailer: Apple Mail (2.1827) Cc: FreeBSD Net <freebsd-net@freebsd.org> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/> List-Post: <mailto:freebsd-net@freebsd.org> List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=subscribe> X-List-Received-Date: Mon, 03 Feb 2014 10:01:52 -0000 Hi, If you can, please test the patch I sent and let me know the results. = I=92ll check it into FreeBSD 11 and 10 if it works for you. Thanks, Scott On Feb 3, 2014, at 2:51 AM, Ben <mailinglists@niessen.ch> wrote: > Thank you for your detailed explanation. >=20 > If I understand correctly the switch is probably not set up correctly, = right? >=20 > I will try to have it configured correctly first. >=20 > Thanks a lot for your help! >=20 > Regards > Ben >=20 > On 03.02.2014 10:45, Scott Long wrote: >> Ok, please try the patch I emailed earlier. Since you=92re not = seeing any receive messages, it means that your switch isn=92t = generating any LACP heartbeats. The difference between FreeBSD 9.x and = 10 is that in 9.x, it ran in =93optimistic=94 mode, meaning that it = didn=92t rely on getting receive messages from the switch, and only took = a channel down if the link state went down. In strict mode, it looks = for the receive messages and only transitions to a full operational = state if it gets them. So while I know it=92s easy to point at the = problem being FreeBSD 10, seeing as FreeBSD 9 worked for you, please = check to make sure that your switch is set up correctly. >>=20 >> I authored the original change that went into FreeBSD 10, and I tried = to make it so that strict_mode=3D0 would keep everything working as it = did in 9. I guess that since you=92re getting no receive messages from = the switch at all that we need to disable strict mode on setup, not = afterwards. Apply the patch and everything should work as it did in = FreeBSD 9. >>=20 >> Scott >>=20 >> On Feb 3, 2014, at 2:29 AM, Ben <mailinglists@niessen.ch> wrote: >>=20 >>> Yes, via sysctl and /etc/sysctl.conf >>>=20 >>> I waited now roughly 20 minutes without touching it but no = difference. >>>=20 >>> No, I only see these transmit messages, no receive. >>>=20 >>> Thanks >>> Ben >>>=20 >>> On 03.02.2014 10:25, Scott Long wrote: >>>> Did you set it to 0 via the sysctl? You might need to wait for = several minutes if you set it after setting up the links. >>>>=20 >>>> Also, the message that you=92re seeing is from your machine = transmitting PDU packets. Are you seeing any "lacpdu receive=94 = messages on the console? >>>>=20 >>>> Thanks, >>>> Scott >>>>=20 >>>> On Feb 3, 2014, at 2:10 AM, Ben <mailinglists@niessen.ch> wrote: >>>>=20 >>>>> Hi, >>>>>=20 >>>>> I set strict mode to 0 but no use. I do receive PDU messages. >>>>>=20 >>>>> igb0: lacpdu transmit >>>>> actor=3D(...) >>>>> actor.state=3D4d<ACTIVITY,AGGREGATION,SYNC,DEFAULTED> >>>>> partner=3D(...) >>>>> partner.state=3D0 >>>>> maxdelay=3D0 >>>>>=20 >>>>> Thanks >>>>> Ben >>>>>=20 >>>>> On 03.02.2014 10:03, Scott Long wrote: >>>>>> Hi, >>>>>>=20 >>>>>> Unfortunately, you can=92t control the strict mode globally. My = apologies for this mess, I=92ll make sure that it=92s fixed for FreeBSD = 10.1. If the sysctl doesn=92t help then maybe consider compiling a = custom kernel with it defaulted to 0. You=92ll need to open = /sys/net/ieee802ad_lacp.c and look for the function lacp_attach(). = You=92ll see the strict_mode assign underneath that. I=92ll also send = you a patch in a few minutes. Until then, try enabling = net.link.lagg.lacp.debug=3D1 and see if you=92re receiving heartbeat = PDU=92s from your switch. >>>>>>=20 >>>>>> Scott >>>>>>=20 >>>>>> On Feb 3, 2014, at 1:40 AM, Ben <mailinglists@niessen.ch> wrote: >>>>>>=20 >>>>>>> Hi Scott, >>>>>>>=20 >>>>>>> I had tried to set it in /etc/sysctl.conf but seems it didnt = work. But will I try again and report back. >>>>>>>=20 >>>>>>> The settings of the switch have not been changed and are set to = LACP. It worked before so I guess the switch should not be the problem. = Maybe some incompatibility between FreeBSD + igb-driver + switch = (Juniper EX3300-48T). >>>>>>>=20 >>>>>>> I will update you after setting the sysctl setting. It seems to = be "dynamic", I guess 0 reflects the index of LACP lagg devices. Can I = switch off the strict mode globally in /etc/sysctl.conf? >>>>>>>=20 >>>>>>> Thanks for your help. >>>>>>>=20 >>>>>>> Regards >>>>>>> Ben >>>>>>>=20 >>>>>>> On 03.02.2014 09:31, Scott Long wrote: >>>>>>>> Hi, >>>>>>>>=20 >>>>>>>> You=92re probably running into the consequences of r253687. = Check to see the value of =91sysctl = net.link.lagg.0.lacp.lacp_strict_mode=92. If it=92s =911=92 then set it = to 0. My original intention was for this to default to 0, but = apparently that didn=92t happen. However, the fact that strict mode = doesn=92t seem to work at all for you might hint that your switch either = isn=92t configured correctly for LACP, or doesn=92t actually support = LACP at all. You might want to investigate that. >>>>>>>>=20 >>>>>>>> Scott >>>>>>>>=20 >>>>>>>> On Feb 3, 2014, at 1:17 AM, Ben <mailinglists@niessen.ch> = wrote: >>>>>>>>=20 >>>>>>>>> Hi, >>>>>>>>>=20 >>>>>>>>> I upgraded from FreeBSD 9.2-RELEASE to 10.0-RELEASE. FreeBSD = 9.2 was configured to use LACP with two igb devices. >>>>>>>>>=20 >>>>>>>>> Now it stopped working after the upgrade. >>>>>>>>>=20 >>>>>>>>> This is a screenshot of ifconfig -a after the upgrade to = FreeBSD 10..0-RELEASE: = http://tinypic.com/view.php?pic=3D28jvgpw&s=3D5#.Uu9PXT1dVPM >>>>>>>>>=20 >>>>>>>>> A PR is currently open: = http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dkern/185967 >>>>>>>>>=20 >>>>>>>>> It is set to low, but I would like somebody to have a look = into it as it obviously has a great influence on our infrastructure. The = only way to "solve" it is currently switching back to FreeBSD 9.2. >>>>>>>>>=20 >>>>>>>>> The suggested fix "use failover" seems not to work. >>>>>>>>>=20 >>>>>>>>> Thank you for your help. >>>>>>>>>=20 >>>>>>>>> Best regards >>>>>>>>> Ben >>>>>>>>> _______________________________________________ >>>>>>>>> freebsd-net@freebsd.org mailing list >>>>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>>>>>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>>>>>> _______________________________________________ >>>>>>>> freebsd-net@freebsd.org mailing list >>>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>>>>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>> _______________________________________________ >>>>>>> freebsd-net@freebsd.org mailing list >>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>>>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>>>> _______________________________________________ >>>>>> freebsd-net@freebsd.org mailing list >>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>> _______________________________________________ >>>>> freebsd-net@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>>=20 >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>=20 >> !DSPAM:1,52ef6540888822133843295! >>=20 >>=20 >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"