From owner-freebsd-hackers@freebsd.org Sat Jul 11 16:32:35 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EECBF368461 for ; Sat, 11 Jul 2020 16:32:35 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-qk1-x731.google.com (mail-qk1-x731.google.com [IPv6:2607:f8b0:4864:20::731]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B3wS31LMbz49M1 for ; Sat, 11 Jul 2020 16:32:34 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: by mail-qk1-x731.google.com with SMTP id l6so8337572qkc.6 for ; Sat, 11 Jul 2020 09:32:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=longcount-org.20150623.gappssmtp.com; s=20150623; h=content-transfer-encoding:from:mime-version:date:subject:message-id :cc:to; bh=QgxGD1vEpUsCkrxjNtzS1RQYLauyMghommnXXTLnHts=; b=2LKmjwaD4cB9f+ZABoCaiYGYFgnCw9FhJXcDsuJm4tJp1nDlhe7PWvgqr05OSSuRkl RSrSFa0A+r/WAbb1iPYBjdp4bBdF0BGCR3qBUrJz+HVcfwD3SVGomAuPPyj3YNdPGC1H Ap5mnM0bxwfUpGSlk8qbYqaOxLJQ6xKFpxc3AQMOiSP0QbYDK9qf/VVuaLaduCH1d01b GVx3KpQ9tT7ecxcEFlZOz2yckAkORPOpEDtqFhl33ed8M7++3NY6R2ubA3tk8qIgUU9D pcfJ/KSTeWzi1q+ovQ6WMzhxCrPedLheSuyt0INhZR4zoLsW9+PeC2xR6gg6lDGz1Xa+ V1DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version:date :subject:message-id:cc:to; bh=QgxGD1vEpUsCkrxjNtzS1RQYLauyMghommnXXTLnHts=; b=gsSZvfIVF57MD4TaI8pzMw1EEybGPiyIYXaM9QzUGEHg5xP64i2XC+CqMR+ht6gf+Y OPTqmNVodkaM8tU/c1wxAiOf0LHdDKh7SKW4AP+fnv7WU7ibntDdOat7nqOszCmCfydW zgGWTKXgTJWx/lIkdsEg/DnrIDQ6S2O7k9coPXCmogq82bLW5rE+ePw1ig1XQDYpQ9PT DswxD7q6vDTaCHo3BEXYE9KAUcHPgNyza21Fcr0APuTkrk873guzdyvxKdmPh7Z1C5zD eD0KRqGEGYpPoI4Ttrkw6O5NyAT8o8KiIJbAZ6vOTpD3RM0NRTfV4L1wV60RfSOiSsEB K+Eg== X-Gm-Message-State: AOAM533rBXFR0E6yaE7yhfxNGnGY0x//QDa8xrPbd3aeqYvzZtzlcKI+ k2R14R4Dsf3Wwb1/LxFqS0KJryeT89f23A== X-Google-Smtp-Source: ABdhPJzQKlhi8WG37osElUwxKo85nh+hTdZP5WwaCG7O/SLWbSrrGPUzHmeSgEfGZpq+7f+hGMp4hA== X-Received: by 2002:a05:620a:958:: with SMTP id w24mr75426951qkw.20.1594485153856; Sat, 11 Jul 2020 09:32:33 -0700 (PDT) Received: from [192.168.1.31] (ool-457b63be.dyn.optonline.net. [69.123.99.190]) by smtp.gmail.com with ESMTPSA id y67sm11896584qka.101.2020.07.11.09.32.33 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 11 Jul 2020 09:32:33 -0700 (PDT) From: Mark Saad Mime-Version: 1.0 (1.0) Date: Sat, 11 Jul 2020 12:32:31 -0400 Subject: LACP debug question Message-Id: <906D4A4A-95E1-4AB3-9D64-44DBC3987E51@longcount.org> Cc: NYCBUG Talk To: Hackers freeBSD X-Mailer: iPhone Mail (17F80) X-Rspamd-Queue-Id: 4B3wS31LMbz49M1 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=longcount-org.20150623.gappssmtp.com header.s=20150623 header.b=2LKmjwaD; dmarc=none; spf=none (mx1.freebsd.org: domain of nonesuch@longcount.org has no SPF policy when checking 2607:f8b0:4864:20::731) smtp.mailfrom=nonesuch@longcount.org X-Spamd-Result: default: False [-1.68 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[longcount-org.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-0.98)[-0.977]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-0.97)[-0.970]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[longcount.org]; NEURAL_SPAM_SHORT(0.07)[0.068]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[longcount-org.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::731:from]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[69.123.99.190:received] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Jul 2020 16:32:36 -0000 All This is a repost from net . I am looking for someone who can help me bette= r understand this LACP disconnect that I am seeing on 12-stable . The server= is a router with Solarflare nics attached to a Pair of arista 7050=E2=80=99= s .=20 Can you help me understand what I am looking at here. I enabled the lacp deb= ug until I finally saw the issue I noted before. Due to some log rotation pa= rt of the message is clipped. Here is a part the full thing is on patebin https://pastebin.com/BGtbxcBf 30 2020-07-09T15:47:04.145885+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: lacp= _sm_rx_timer: CURRENT -> EXPIRED 31 2020-07-09T15:47:04.145895+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: Inte= rface stopped DISTRIBUTING, possible flapping 32 2020-07-09T15:47:04.145895+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: coll= ecting enabled 33 2020-07-09T15:47:04.145896+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: disa= ble distributing on aggregator [(8000,00-0F-53-69-7C-20,00F2,0000,0000),(800= 0,46-4C-A8-68-13-47,006A,0000,0000)], nports 2 -> 1 34 2020-07-09T15:47:04.145911+00:00 ch1-c104-sdn02-mgmt kernel: sfxge1: lacp= _select_tx_port: waiting transit 35 2020-07-09T15:47:04.145912+00:00 ch1-c104-sdn02-mgmt kernel: marker trans= mit, port=3D6, sys=3D00:0f:53:69:7c:20, id=3D487 36 2020-07-09T15:47:04.145912+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: sfxg= e1: lacp_select_tx_port: waiting transit 37 2020-07-09T15:47:04.145913+00:00 ch1-c104-sdn02-mgmt kernel: marker trans= mit, port=3D5, sys=3D00:0f:53:69:7c:20, id=3D487 38 2020-07-09T15:47:04.145914+00:00 ch1-c104-sdn02-mgmt kernel: marker respo= nse, port=3D6, sys=3D00:0f:53:69:7c:20, id=3D487 39 2020-07-09T15:47:04.145914+00:00 ch1-c104-sdn02-mgmt kernel: [(8000,00-0F-= 53-69-7C-20,00F2,0000,0000),(8000,46-4C-A8-68-13-47,006A,0000,0000)], speed=3D= 10000000000, nports=3D1 40 2020-07-09T15:47:04.145922+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: lacp= _select_tx_port: waiting transit 41 2020-07-09T15:47:04.145923+00:00 ch1-c104-sdn02-mgmt kernel: lacp_select_= tx_port: waiting transit 42 2020-07-09T15:47:04.145923+00:00 ch1-c104-sdn02-mgmt kernel: active aggre= gator not changed 43 2020-07-09T15:47:04.145924+00:00 ch1-c104-sdn02-mgmt kernel: lacp_select_= tx_port: waiting transit 44 2020-07-09T15:47:04.145925+00:00 ch1-c104-sdn02-mgmt kernel: marker respo= nse, port=3D5, sys=3D00:0f:53:69:7c:20, id=3D487 45 2020-07-09T15:47:04.145925+00:00 ch1-c104-sdn02-mgmt kernel: lacp_select_= tx_port: waiting transit 46 2020-07-09T15:47:04.145926+00:00 ch1-c104-sdn02-mgmt kernel: new [(8000,0= 0-0F-53-69-7C-20,00F2,0000,0000),(8000,46-4C-A8-68-13-47,006A,0000,0000)] 47 2020-07-09T15:47:04.145926+00:00 ch1-c104-sdn02-mgmt kernel: Set table 1 w= ith 1 ports 48 2020-07-09T15:47:04.145927+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: mux_= state 4 -> 3 49 2020-07-09T15:47:04.145928+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: coll= ecting disabled 50 2020-07-09T15:47:04.145930+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: mux_= state 3 -> 2 51 2020-07-09T15:47:04.145931+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: lacp= du transmit 52 2020-07-09T15:47:04.145931+00:00 ch1-c104-sdn02-mgmt kernel: actor=3D(800= 0,00-0F-53-69-7C-20,00F2,8000,0005) 53 2020-07-09T15:47:04.145932+00:00 ch1-c104-sdn02-mgmt kernel: actor.state=3D= 8d 54 2020-07-09T15:47:04.145932+00:00 ch1-c104-sdn02-mgmt kernel: partner=3D(8= 000,46-4C-A8-68-13-47,006A,8000,0006) 55 2020-07-09T15:47:04.145933+00:00 ch1-c104-sdn02-mgmt kernel: partner.stat= e=3D37 56 2020-07-09T15:47:04.145935+00:00 ch1-c104-sdn02-mgmt kernel: maxdelay=3D0= 57 2020-07-09T15:47:04.145937+00:00 ch1-c104-sdn02-mgmt kernel: queue flush c= omplete --- Mark Saad | nonesuch@longcount.org=