From owner-freebsd-net@FreeBSD.ORG Mon Feb 3 09:45:24 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 69690BB6 for ; Mon, 3 Feb 2014 09:45:24 +0000 (UTC) Received: from nm13-vm5.bullet.mail.ne1.yahoo.com (nm13-vm5.bullet.mail.ne1.yahoo.com [98.138.91.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 296F912BC for ; Mon, 3 Feb 2014 09:45:23 +0000 (UTC) Received: from [98.138.100.114] by nm13.bullet.mail.ne1.yahoo.com with NNFMP; 03 Feb 2014 09:45:17 -0000 Received: from [98.138.226.128] by tm105.bullet.mail.ne1.yahoo.com with NNFMP; 03 Feb 2014 09:45:17 -0000 Received: from [127.0.0.1] by smtp215.mail.ne1.yahoo.com with NNFMP; 03 Feb 2014 09:45:17 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1391420717; bh=DnnYJ9F06W3nRbQyNUapAvSg3jVEzLjXVD0yJO9xZuI=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id:References:To:X-Mailer; b=tWq+ydi7Hzl9rzWuwE7UBQ3T1nRkC5zHfcRWTP7ZmuJ57d3ii6/cAnIrrDS+NJ7sZoeaNWMzmHZn2hx8lQ1adxnoGJFleVh15Juwei+5PDkmctSOKJ8RPqeHD8OCxbXbdhWZvvJIs9NlSWHYbntdAOAWCbA1PvmL5AmaEYASrB0= X-Yahoo-Newman-Id: 110000.62549.bm@smtp215.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: nFu00PUVM1nF5.hwjQZPljWuc7DMLzpZqv_s_K7SbUX3DBL lZ0H5lrPB4zTPquy7H.iMccOUWpACcSifWBcGihSxPv22yG4r2rHNQUXmG1p rpbqf831QJlmNWWj5n2WRq64AiQl.58Myv80zMO0Uvez3ReTNaHd4qQwi3EE WCnHR6oA1yh_3yoINFDIgh79kPZOmvFY4_N9uuoFSn3.k0HithoueUZsY8Tt 7UVxE7c2saci1U6HREBcz39gz8rhHU.2I7pbBEx.HTi2L.BGnKdquQWjJvQL UKfaBWDNytAt4KxJjDsrriXWOezCOM910W5v4iTfN77IEYvolgWOkSRybDMU 2J39YqxI5M3eMdUun1BrFk013RGY53KZJQ5T0LA9xFuMcdomxxbqEsYieIW6 V_rtkJaxubb.Vfnf5iO3BGXS6fg9M0wdJIhVKoIbYZiaglg0UIp0j8iUp0qR lw_oJ99doDAx2RgU8AsUdchugSDbVjJNgFBLY_OarupGOs6ybaNrrtdV02M4 AIoRNAGbnbGfn.97qe9Yxg.Qw7UIoa6EOyLCkiv.lNQozyIIQUIDg6NejUjz hdfZAF4s3ujIR5rzMa4xqVpkET23D27mnYj8dAtQgjVRZLfXysF_5SpUdVWR z6DzEcDP_PdmhIZG4x23quKL6Axv9rFdsL7_Sx3.yJnnSLIegcLvuKMuQ X-Yahoo-SMTP: clhABp.swBB7fs.LwIJpv3jkWgo2NU8- X-Rocket-Received: from [10.64.24.117] (scott4long@69.53.236.251 with plain [98.139.211.125]) by smtp215.mail.ne1.yahoo.com with SMTP; 03 Feb 2014 01:45:17 -0800 PST Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: kern/185967: Link Aggregation LAGG: LACP not working in 10.0 From: Scott Long In-Reply-To: <52EF6194.5060305@niessen.ch> Date: Mon, 3 Feb 2014 02:45:13 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <8585EA2E-116E-45A6-877D-DC8D4460C965@yahoo.com> References: <52EF50A7.1050205@niessen.ch> <1C608452-6F29-486D-BC0F-CCC7853665C7@yahoo.com> <52EF55FE.8030901@niessen.ch> <1798FE17-5718-4125-8B00-1B00DC44B828@yahoo.com> <52EF5D1E.2000306@niessen.ch> <52EF6194.5060305@niessen.ch> To: Ben X-Mailer: Apple Mail (2.1827) Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Feb 2014 09:45:24 -0000 Ok, please try the patch I emailed earlier. Since you=92re not seeing = any receive messages, it means that your switch isn=92t generating any = LACP heartbeats. The difference between FreeBSD 9.x and 10 is that in = 9.x, it ran in =93optimistic=94 mode, meaning that it didn=92t rely on = getting receive messages from the switch, and only took a channel down = if the link state went down. In strict mode, it looks for the receive = messages and only transitions to a full operational state if it gets = them. So while I know it=92s easy to point at the problem being FreeBSD = 10, seeing as FreeBSD 9 worked for you, please check to make sure that = your switch is set up correctly. I authored the original change that went into FreeBSD 10, and I tried to = make it so that strict_mode=3D0 would keep everything working as it did = in 9. I guess that since you=92re getting no receive messages from the = switch at all that we need to disable strict mode on setup, not = afterwards. Apply the patch and everything should work as it did in = FreeBSD 9. Scott On Feb 3, 2014, at 2:29 AM, Ben wrote: > Yes, via sysctl and /etc/sysctl.conf >=20 > I waited now roughly 20 minutes without touching it but no difference. >=20 > No, I only see these transmit messages, no receive. >=20 > Thanks > Ben >=20 > On 03.02.2014 10:25, Scott Long wrote: >> Did you set it to 0 via the sysctl? You might need to wait for = several minutes if you set it after setting up the links. >>=20 >> Also, the message that you=92re seeing is from your machine = transmitting PDU packets. Are you seeing any "lacpdu receive=94 = messages on the console? >>=20 >> Thanks, >> Scott >>=20 >> On Feb 3, 2014, at 2:10 AM, Ben wrote: >>=20 >>> Hi, >>>=20 >>> I set strict mode to 0 but no use. I do receive PDU messages. >>>=20 >>> igb0: lacpdu transmit >>> actor=3D(...) >>> actor.state=3D4d >>> partner=3D(...) >>> partner.state=3D0 >>> maxdelay=3D0 >>>=20 >>> Thanks >>> Ben >>>=20 >>> On 03.02.2014 10:03, Scott Long wrote: >>>> Hi, >>>>=20 >>>> Unfortunately, you can=92t control the strict mode globally. My = apologies for this mess, I=92ll make sure that it=92s fixed for FreeBSD = 10.1. If the sysctl doesn=92t help then maybe consider compiling a = custom kernel with it defaulted to 0. You=92ll need to open = /sys/net/ieee802ad_lacp.c and look for the function lacp_attach(). = You=92ll see the strict_mode assign underneath that. I=92ll also send = you a patch in a few minutes. Until then, try enabling = net.link.lagg.lacp.debug=3D1 and see if you=92re receiving heartbeat = PDU=92s from your switch. >>>>=20 >>>> Scott >>>>=20 >>>> On Feb 3, 2014, at 1:40 AM, Ben wrote: >>>>=20 >>>>> Hi Scott, >>>>>=20 >>>>> I had tried to set it in /etc/sysctl.conf but seems it didnt work. = But will I try again and report back. >>>>>=20 >>>>> The settings of the switch have not been changed and are set to = LACP. It worked before so I guess the switch should not be the problem. = Maybe some incompatibility between FreeBSD + igb-driver + switch = (Juniper EX3300-48T). >>>>>=20 >>>>> I will update you after setting the sysctl setting. It seems to be = "dynamic", I guess 0 reflects the index of LACP lagg devices. Can I = switch off the strict mode globally in /etc/sysctl.conf? >>>>>=20 >>>>> Thanks for your help. >>>>>=20 >>>>> Regards >>>>> Ben >>>>>=20 >>>>> On 03.02.2014 09:31, Scott Long wrote: >>>>>> Hi, >>>>>>=20 >>>>>> You=92re probably running into the consequences of r253687. = Check to see the value of =91sysctl = net.link.lagg.0.lacp.lacp_strict_mode=92. If it=92s =911=92 then set it = to 0. My original intention was for this to default to 0, but = apparently that didn=92t happen. However, the fact that strict mode = doesn=92t seem to work at all for you might hint that your switch either = isn=92t configured correctly for LACP, or doesn=92t actually support = LACP at all. You might want to investigate that. >>>>>>=20 >>>>>> Scott >>>>>>=20 >>>>>> On Feb 3, 2014, at 1:17 AM, Ben wrote: >>>>>>=20 >>>>>>> Hi, >>>>>>>=20 >>>>>>> I upgraded from FreeBSD 9.2-RELEASE to 10.0-RELEASE. FreeBSD 9.2 = was configured to use LACP with two igb devices. >>>>>>>=20 >>>>>>> Now it stopped working after the upgrade. >>>>>>>=20 >>>>>>> This is a screenshot of ifconfig -a after the upgrade to FreeBSD = 10.0-RELEASE: http://tinypic.com/view.php?pic=3D28jvgpw&s=3D5#.Uu9PXT1dVPM= >>>>>>>=20 >>>>>>> A PR is currently open: = http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dkern/185967 >>>>>>>=20 >>>>>>> It is set to low, but I would like somebody to have a look into = it as it obviously has a great influence on our infrastructure. The only = way to "solve" it is currently switching back to FreeBSD 9.2. >>>>>>>=20 >>>>>>> The suggested fix "use failover" seems not to work. >>>>>>>=20 >>>>>>> Thank you for your help. >>>>>>>=20 >>>>>>> Best regards >>>>>>> Ben >>>>>>> _______________________________________________ >>>>>>> freebsd-net@freebsd.org mailing list >>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>>>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>>>> _______________________________________________ >>>>>> freebsd-net@freebsd.org mailing list >>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>> _______________________________________________ >>>>> freebsd-net@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" >>=20 >> !DSPAM:1,52ef6078888821231914487!