From owner-freebsd-hackers@freebsd.org Sun Jul 12 10:22:25 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D1FA835EE22 for ; Sun, 12 Jul 2020 10:22:25 +0000 (UTC) (envelope-from kmsujit@gmail.com) Received: from mail-il1-x136.google.com (mail-il1-x136.google.com [IPv6:2607:f8b0:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B4NBS5f4Vz4NKm for ; Sun, 12 Jul 2020 10:22:24 +0000 (UTC) (envelope-from kmsujit@gmail.com) Received: by mail-il1-x136.google.com with SMTP id h16so8745327ilj.11 for ; Sun, 12 Jul 2020 03:22:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=udt2co6tmE6FYSHCixrqVImzqrVJfzRHgZwIF3KLmm0=; b=Ka1OtuCESbD0GPtlIgnL4ktxh0zBV1KtcQsgTxDVhWKF4T+AEc2EvkWn0jFUJk5XSk wdOR5ixRSexhpqQb7afLsWyl4sCtlsgDwL8YgFwowVC/Q5CVLn4RybtJOQ/tAoPq4Y8G yhslcUCKEunNfXmZz+thg9Y9KJZB7scKwxfzErQ1UvK2G/UqSjtmUZYYUlfAXxYSOakT U9k4p60hEdHmqBWco7SYPaws5yxPT2rxtrY2pnqTa2UF/I/WApNicVIVQDJEagB2lxEh NtGMOK9Ra+nIZCFwkU0txf0nzjLmUz8ypfCG7Rf8UW5nKcFOz2QnGccXKMqpvdmDQVdN G+TQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=udt2co6tmE6FYSHCixrqVImzqrVJfzRHgZwIF3KLmm0=; b=QUEbTNP5UxAcpZNI/+6FyBn9FpjjgkLJA0/fXBKC/qv5NU9AWqEhLGn8epBTMQOgUW yCNxFMKNve2/DBswnYXvn5LTXGjl7gO9gu0mm3zXEBizWDzSnqQ/aZ0MRS7wcBNetglP FQnjKjij3wo7T3dX7WfqgZuFzvBQwuuBVciBG/qtRPNixPwiLaNbLQWOExtRMC1un+AT 2v3/Ryx0MwRBkng5crlE0InhCReX51YzaCU++7K/IXKASJvteG2Zo+FLe1Rs8uUe4Pma KDdcfgOGozsoBnQOsgsAKaYYX61aLtxROMxOtH60H7oFLshI6pHAHMeKkDXhebOz4s1f XRlg== X-Gm-Message-State: AOAM530D+3oLi2kBAs0fGTx7brH5HTXnWvCnW1u96FLtCku+Z3s5W2GF BeIt1tFRjMYhLWJj0JySb7+2aocax63O/b0veDdeRU0t X-Google-Smtp-Source: ABdhPJxb/lQHtxDNpIZEGFvrS/avU8Hy++8lapXK7PEaugzrRC53+wBqk1VN+R+BJUlYgmserJPM+j6fJAuapHjEB5s= X-Received: by 2002:a92:dc0f:: with SMTP id t15mr53891350iln.218.1594549343065; Sun, 12 Jul 2020 03:22:23 -0700 (PDT) MIME-Version: 1.0 References: <906D4A4A-95E1-4AB3-9D64-44DBC3987E51@longcount.org> In-Reply-To: <906D4A4A-95E1-4AB3-9D64-44DBC3987E51@longcount.org> From: Sujit K M Date: Sun, 12 Jul 2020 15:52:11 +0530 Message-ID: Subject: Re: [talk] LACP debug question To: Mark Saad Cc: Hackers freeBSD , NYCBUG Talk X-Rspamd-Queue-Id: 4B4NBS5f4Vz4NKm X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=Ka1OtuCE; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of kmsujit@gmail.com designates 2607:f8b0:4864:20::136 as permitted sender) smtp.mailfrom=kmsujit@gmail.com X-Spamd-Result: default: False [-3.02 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.001]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; NEURAL_HAM_LONG(-1.01)[-1.012]; URI_COUNT_ODD(1.00)[1]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::136:from]; NEURAL_HAM_SHORT(-1.00)[-1.002]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jul 2020 10:22:25 -0000 On Sat, Jul 11, 2020, 10:03 PM Mark Saad wrote: > All > This is a repost from net . I am looking for someone who can help me > better understand this LACP disconnect that I am seeing on 12-stable . Th= e > server is a router with Solarflare nics attached to a Pair of arista 7050= =E2=80=99s > . > > Can you help me understand what I am looking at here. I enabled the lacp > debug until I finally saw the issue I noted before. Due to some log > rotation part of the message is clipped. > Here is a part the full thing is on patebin https://pastebin.com/BGtbxcBf > > > 30 2020-07-09T15:47:04.145885+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: > lacp_sm_rx_timer: CURRENT -> EXPIRED > 31 2020-07-09T15:47:04.145895+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: > Interface stopped DISTRIBUTING, possible flapping > 32 2020-07-09T15:47:04.145895+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: > collecting enabled > https://en.m.wikipedia.org/wiki/Link_aggregation#:~:text=3DWithin%20the%20I= EEE%20specification%2C%20the,form%20a%20single%20logical%20channel . The expired and stopped distribution might beat the purpose of LACP. > > 33 2020-07-09T15:47:04.145896+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: > disable distributing on aggregator > [(8000,00-0F-53-69-7C-20,00F2,0000,0000),(8000,46-4C-A8-68-13-47,006A,000= 0,0000)], > nports 2 -> 1 > 34 2020-07-09T15:47:04.145911+00:00 ch1-c104-sdn02-mgmt kernel: sfxge1: > lacp_select_tx_port: waiting transit > 35 2020-07-09T15:47:04.145912+00:00 ch1-c104-sdn02-mgmt kernel: marker > transmit, port=3D6, sys=3D00:0f:53:69:7c:20, id=3D487 > 36 2020-07-09T15:47:04.145912+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: > sfxge1: lacp_select_tx_port: waiting transit > 37 2020-07-09T15:47:04.145913+00:00 ch1-c104-sdn02-mgmt kernel: marker > transmit, port=3D5, sys=3D00:0f:53:69:7c:20, id=3D487 > 38 2020-07-09T15:47:04.145914+00:00 ch1-c104-sdn02-mgmt kernel: marker > response, port=3D6, sys=3D00:0f:53:69:7c:20, id=3D487 > 39 2020-07-09T15:47:04.145914+00:00 ch1-c104-sdn02-mgmt kernel: > [(8000,00-0F-53-69-7C-20,00F2,0000,0000),(8000,46-4C-A8-68-13-47,006A,000= 0,0000)], > speed=3D10000000000, nports=3D1 > 40 2020-07-09T15:47:04.145922+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: > lacp_select_tx_port: waiting transit > 41 2020-07-09T15:47:04.145923+00:00 ch1-c104-sdn02-mgmt kernel: > lacp_select_tx_port: waiting transit > 42 2020-07-09T15:47:04.145923+00:00 ch1-c104-sdn02-mgmt kernel: active > aggregator not changed > 43 2020-07-09T15:47:04.145924+00:00 ch1-c104-sdn02-mgmt kernel: > lacp_select_tx_port: waiting transit > 44 2020-07-09T15:47:04.145925+00:00 ch1-c104-sdn02-mgmt kernel: marker > response, port=3D5, sys=3D00:0f:53:69:7c:20, id=3D487 > 45 2020-07-09T15:47:04.145925+00:00 ch1-c104-sdn02-mgmt kernel: > lacp_select_tx_port: waiting transit > 46 2020-07-09T15:47:04.145926+00:00 ch1-c104-sdn02-mgmt kernel: new > [(8000,00-0F-53-69-7C-20,00F2,0000,0000),(8000,46-4C-A8-68-13-47,006A,000= 0,0000)] > 47 2020-07-09T15:47:04.145926+00:00 ch1-c104-sdn02-mgmt kernel: Set table > 1 with 1 ports > 48 2020-07-09T15:47:04.145927+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: > mux_state 4 -> 3 > 49 2020-07-09T15:47:04.145928+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: > collecting disabled > 50 2020-07-09T15:47:04.145930+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: > mux_state 3 -> 2 > 51 2020-07-09T15:47:04.145931+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: > lacpdu transmit > 52 2020-07-09T15:47:04.145931+00:00 ch1-c104-sdn02-mgmt kernel: > actor=3D(8000,00-0F-53-69-7C-20,00F2,8000,0005) > 53 2020-07-09T15:47:04.145932+00:00 ch1-c104-sdn02-mgmt kernel: > actor.state=3D8d > 54 2020-07-09T15:47:04.145932+00:00 ch1-c104-sdn02-mgmt kernel: > partner=3D(8000,46-4C-A8-68-13-47,006A,8000,0006) > 55 2020-07-09T15:47:04.145933+00:00 ch1-c104-sdn02-mgmt kernel: > partner.state=3D37 > 56 2020-07-09T15:47:04.145935+00:00 ch1-c104-sdn02-mgmt kernel: maxdelay= =3D0 > 57 2020-07-09T15:47:04.145937+00:00 ch1-c104-sdn02-mgmt kernel: queue > flush complete > --- > Mark Saad | nonesuch@longcount.org > _______________________________________________ > talk mailing list > talk@lists.nycbug.org > http://lists.nycbug.org:8080/mailman/listinfo/talk >