From owner-freebsd-hackers@freebsd.org Sun Jul 12 15:58:27 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B8E14366BD4 for ; Sun, 12 Jul 2020 15:58:27 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-qt1-x831.google.com (mail-qt1-x831.google.com [IPv6:2607:f8b0:4864:20::831]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B4WfB4Fd4z3Rc0 for ; Sun, 12 Jul 2020 15:58:26 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: by mail-qt1-x831.google.com with SMTP id x62so8222364qtd.3 for ; Sun, 12 Jul 2020 08:58:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=longcount-org.20150623.gappssmtp.com; s=20150623; h=content-transfer-encoding:from:mime-version:subject:date:message-id :references:cc:in-reply-to:to; bh=MfuI6TRieYmMZsyYMPDh6t9jZ9QRD20xPkmY79C1FTI=; b=eNsk5otijU2bY8ucw6JcDEdXZUqrXevHmQ26iSnpIazXejvdqHhnglng9rTOmgphZY ixAvQQGsxdy5Nti+Dpp/dCR8kr9r/3pI1etklOasaejTJVY6yqZQAPmv2sf7z0gUbBLe Hys9FFvvVP3zkD0fWvICVr2HYHVNegKG6WP6M3bNrQ4Gs9CL9m8DT4wmE3ZoGmfyozNl GbXpRTqJxTfa3vMggKpaZTcEOq2wyjetf9iYiFAGCIIPsIT2BwTWeUI34EqiMg4fDxVc kzdbKNj+alqa8qN+swQkO6wUjxvXSaBsUYIcBr2qdwguJRdJ5ij8FyKNx/KnPSpxePub 8hWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:date:message-id:references:cc:in-reply-to:to; bh=MfuI6TRieYmMZsyYMPDh6t9jZ9QRD20xPkmY79C1FTI=; b=JG+xoulY0YltyrzJfOZQFJwFGRttKcEdq/APT+rBfj5pHI6mOZjB+aQ5yjRIpJ0WVm cMKtRSa0St1ePEThIaqQ/BvxgYJZyU4mn3ya8LtINmthq7/PiJV6YS+Owoyc0dA/gXTD RuULVRlT4+Nl73rzk61YoOm6g5lDJtskOtjTW8QREeBLgwle6FH1dnb9xlZdqNQ1ASJS 7pjnDAufg16jEl9FXRsJRbUKuRVMif65gK38x643lpSsOQCTmGFN7UzN+/6wCvex9MMg aNFgywp5HUqL20UGA6n9kSVW6C3GmQnOTIjXVc99OcYAqD4l1CWMHgfXVqjo/A7j5Aip lp7Q== X-Gm-Message-State: AOAM533Iv8YsL2W98mt/1PXO5JNGM3qHCyfOZktkIv+m6yJYp9/2CHWx cN6BDXcnUUtRvNO908x+DPLoVw== X-Google-Smtp-Source: ABdhPJzNcFoFeWNWpPNAktmlHUBrYYu1vHmW7gPXFg7eKPjsnQ64r6XLsp3jFy8O/k5cLDRbSpXUKg== X-Received: by 2002:ac8:c7:: with SMTP id d7mr52197304qtg.235.1594569505384; Sun, 12 Jul 2020 08:58:25 -0700 (PDT) Received: from [192.168.1.31] (ool-457b63be.dyn.optonline.net. [69.123.99.190]) by smtp.gmail.com with ESMTPSA id o15sm15536918qko.67.2020.07.12.08.58.23 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 12 Jul 2020 08:58:24 -0700 (PDT) From: Mark Saad Mime-Version: 1.0 (1.0) Subject: Re: [talk] LACP debug question Date: Sun, 12 Jul 2020 11:58:23 -0400 Message-Id: References: Cc: Hackers freeBSD , NYCBUG Talk In-Reply-To: To: Sujit K M X-Mailer: iPhone Mail (17F80) X-Rspamd-Queue-Id: 4B4WfB4Fd4z3Rc0 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=longcount-org.20150623.gappssmtp.com header.s=20150623 header.b=eNsk5oti; dmarc=none; spf=none (mx1.freebsd.org: domain of nonesuch@longcount.org has no SPF policy when checking 2607:f8b0:4864:20::831) smtp.mailfrom=nonesuch@longcount.org X-Spamd-Result: default: False [-1.58 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; URI_COUNT_ODD(1.00)[7]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[longcount-org.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.03)[-1.032]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[69.123.99.190:received]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.81)[-0.811]; R_DKIM_ALLOW(-0.20)[longcount-org.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-0.94)[-0.939]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[longcount.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::831:from]; R_SPF_NA(0.00)[no SPF record]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jul 2020 15:58:27 -0000 Sujit I know that this is LACP debug . I am trying to understand the specific co= nditions that this LACP disconnect sequence is occurring. It=E2=80=99s a od= d thing, it=E2=80=99s happening during times of relatively low was tcp traff= ic . The router and is not highly loaded and I can=E2=80=99t find a reason w= hy this is occurring .=20 --- Mark Saad | nonesuch@longcount.org On Jul 12, 2020, at 6:22 AM, Sujit K M wrote: >=20 > =EF=BB=BF >=20 >=20 >> On Sat, Jul 11, 2020, 10:03 PM Mark Saad wrote: >> All >> This is a repost from net . I am looking for someone who can help me be= tter understand this LACP disconnect that I am seeing on 12-stable . The ser= ver is a router with Solarflare nics attached to a Pair of arista 7050=E2=80= =99s .=20 >>=20 >> Can you help me understand what I am looking at here. I enabled the lacp d= ebug until I finally saw the issue I noted before. Due to some log rotation p= art of the message is clipped. >> Here is a part the full thing is on patebin https://pastebin.com/BGtbxcBf= >>=20 >>=20 >> 30 2020-07-09T15:47:04.145885+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: l= acp_sm_rx_timer: CURRENT -> EXPIRED >> 31 2020-07-09T15:47:04.145895+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: I= nterface stopped DISTRIBUTING, possible flapping >> 32 2020-07-09T15:47:04.145895+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: c= ollecting enabled >=20 >=20 > https://en.m.wikipedia.org/wiki/Link_aggregation#:~:text=3DWithin%20the%20= IEEE%20specification%2C%20the,form%20a%20single%20logical%20channel. >=20 > The expired and stopped distribution might beat the purpose of LACP. >>=20 >> 33 2020-07-09T15:47:04.145896+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: d= isable distributing on aggregator [(8000,00-0F-53-69-7C-20,00F2,0000,0000),(= 8000,46-4C-A8-68-13-47,006A,0000,0000)], nports 2 -> 1 >> 34 2020-07-09T15:47:04.145911+00:00 ch1-c104-sdn02-mgmt kernel: sfxge1: l= acp_select_tx_port: waiting transit >> 35 2020-07-09T15:47:04.145912+00:00 ch1-c104-sdn02-mgmt kernel: marker tr= ansmit, port=3D6, sys=3D00:0f:53:69:7c:20, id=3D487 >> 36 2020-07-09T15:47:04.145912+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: s= fxge1: lacp_select_tx_port: waiting transit >> 37 2020-07-09T15:47:04.145913+00:00 ch1-c104-sdn02-mgmt kernel: marker tr= ansmit, port=3D5, sys=3D00:0f:53:69:7c:20, id=3D487 >> 38 2020-07-09T15:47:04.145914+00:00 ch1-c104-sdn02-mgmt kernel: marker re= sponse, port=3D6, sys=3D00:0f:53:69:7c:20, id=3D487 >> 39 2020-07-09T15:47:04.145914+00:00 ch1-c104-sdn02-mgmt kernel: [(8000,00= -0F-53-69-7C-20,00F2,0000,0000),(8000,46-4C-A8-68-13-47,006A,0000,0000)], sp= eed=3D10000000000, nports=3D1 >> 40 2020-07-09T15:47:04.145922+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: l= acp_select_tx_port: waiting transit >> 41 2020-07-09T15:47:04.145923+00:00 ch1-c104-sdn02-mgmt kernel: lacp_sele= ct_tx_port: waiting transit >> 42 2020-07-09T15:47:04.145923+00:00 ch1-c104-sdn02-mgmt kernel: active ag= gregator not changed >> 43 2020-07-09T15:47:04.145924+00:00 ch1-c104-sdn02-mgmt kernel: lacp_sele= ct_tx_port: waiting transit >> 44 2020-07-09T15:47:04.145925+00:00 ch1-c104-sdn02-mgmt kernel: marker re= sponse, port=3D5, sys=3D00:0f:53:69:7c:20, id=3D487 >> 45 2020-07-09T15:47:04.145925+00:00 ch1-c104-sdn02-mgmt kernel: lacp_sele= ct_tx_port: waiting transit >> 46 2020-07-09T15:47:04.145926+00:00 ch1-c104-sdn02-mgmt kernel: new [(800= 0,00-0F-53-69-7C-20,00F2,0000,0000),(8000,46-4C-A8-68-13-47,006A,0000,0000)]= >> 47 2020-07-09T15:47:04.145926+00:00 ch1-c104-sdn02-mgmt kernel: Set table= 1 with 1 ports >> 48 2020-07-09T15:47:04.145927+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: m= ux_state 4 -> 3 >> 49 2020-07-09T15:47:04.145928+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: c= ollecting disabled >> 50 2020-07-09T15:47:04.145930+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: m= ux_state 3 -> 2 >> 51 2020-07-09T15:47:04.145931+00:00 ch1-c104-sdn02-mgmt kernel: sfxge0: l= acpdu transmit >> 52 2020-07-09T15:47:04.145931+00:00 ch1-c104-sdn02-mgmt kernel: actor=3D(= 8000,00-0F-53-69-7C-20,00F2,8000,0005) >> 53 2020-07-09T15:47:04.145932+00:00 ch1-c104-sdn02-mgmt kernel: actor.sta= te=3D8d >> 54 2020-07-09T15:47:04.145932+00:00 ch1-c104-sdn02-mgmt kernel: partner=3D= (8000,46-4C-A8-68-13-47,006A,8000,0006) >> 55 2020-07-09T15:47:04.145933+00:00 ch1-c104-sdn02-mgmt kernel: partner.s= tate=3D37 >> 56 2020-07-09T15:47:04.145935+00:00 ch1-c104-sdn02-mgmt kernel: maxdelay=3D= 0 >> 57 2020-07-09T15:47:04.145937+00:00 ch1-c104-sdn02-mgmt kernel: queue flu= sh complete >> --- >> Mark Saad | nonesuch@longcount.org >> _______________________________________________ >> talk mailing list >> talk@lists.nycbug.org >> http://lists.nycbug.org:8080/mailman/listinfo/talk