From owner-freebsd-net@freebsd.org Tue Feb 9 11:07:32 2021 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5C67053A62E for ; Tue, 9 Feb 2021 11:07:32 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward105o.mail.yandex.net (forward105o.mail.yandex.net [IPv6:2a02:6b8:0:1a2d::608]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DZg8g0jnGz4rWF for ; Tue, 9 Feb 2021 11:07:30 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from myt2-264a32256793.qloud-c.yandex.net (myt2-264a32256793.qloud-c.yandex.net [IPv6:2a02:6b8:c12:1a07:0:640:264a:3225]) by forward105o.mail.yandex.net (Yandex) with ESMTP id 4D1A74203B88; Tue, 9 Feb 2021 14:07:22 +0300 (MSK) Received: from myt2-accb38a5c431.qloud-c.yandex.net (myt2-accb38a5c431.qloud-c.yandex.net [2a02:6b8:c00:2e9b:0:640:accb:38a5]) by myt2-264a32256793.qloud-c.yandex.net (mxback/Yandex) with ESMTP id L5kQuN1zzY-7MHGtApU; Tue, 09 Feb 2021 14:07:22 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1612868842; bh=85XjtzN74OQdZH5WrOTgBQcYPjijkrmS2oh8hF0f3rU=; h=In-Reply-To:Subject:From:Date:References:To:Message-ID; b=IUiyDU6qW+zSUgsR2H+zQ/la3mEBjRwwnFNxHswdRi8duMgUjeqWu8XApGmp6wZCR Jb8Y4cS7DIQakaR6cTDUryAruihcznj35GNoaXaoH0AnZYHmZjpy2BKPuc2hPhVpbK YSxj9bk/4XH67FngaOgiguU5xtHzG61M0fWwRTWQ= Received: by myt2-accb38a5c431.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id eaoFdbuTN1-7Ln8Oo6L; Tue, 09 Feb 2021 14:07:21 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) To: Johannes Lundberg , freebsd-net@freebsd.org References: <84ad7d46-d103-1bf2-29df-2085d4949d30@gmail.com> From: "Andrey V. Elsukov" Subject: BFD failures with bird on FreeBSD (was: LACP BPDU packets priority?) Message-ID: Date: Tue, 9 Feb 2021 14:05:29 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: <84ad7d46-d103-1bf2-29df-2085d4949d30@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4DZg8g0jnGz4rWF X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yandex.ru header.s=mail header.b=IUiyDU6q; dmarc=pass (policy=none) header.from=yandex.ru; spf=pass (mx1.freebsd.org: domain of bu7cher@yandex.ru designates 2a02:6b8:0:1a2d::608 as permitted sender) smtp.mailfrom=bu7cher@yandex.ru X-Spamd-Result: default: False [-3.96 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yandex.ru]; R_SPF_ALLOW(-0.20)[+ip6:2a02:6b8:0:1000::/52]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[yandex.ru:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yandex.ru,none]; NEURAL_HAM_SHORT(-0.96)[-0.958]; FREEMAIL_TO(0.00)[gmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a02:6b8:0:1a2d::608:from]; FREEMAIL_ENVFROM(0.00)[yandex.ru]; ASN(0.00)[asn:13238, ipnet:2a02:6b8::/32, country:RU]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yandex.ru:dkim]; MIME_TRACE(0.00)[0:+]; SUBJECT_HAS_QUESTION(0.00)[]; R_DKIM_ALLOW(-0.20)[yandex.ru:s=mail]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[2a02:6b8:0:1a2d::608:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MAILMAN_DEST(0.00)[freebsd-net] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Feb 2021 11:07:32 -0000 On 04.02.2021 23:56, Johannes Lundberg wrote: > Hi > > We're experiencing unstable lacp lagg and not seeing BPDU packets coming > to the switch we when expect them to. Can anyone answer what is the > priority of those packets? Could it be that they are not being sent from > the FreeBSD host because they are stuck in outgoing queue? > > Please cc me since I'm not subscribed. Hi, recently we faced with somehow similar problem, but with BGP and BFD packets. In our case BFD packets from neighbor do not reach bird daemon in expected time window, and this leads to BFD failure. We did some research and found, that BFD packets were received by the adapter, but then they seems stalled in the adapter queues for at least 2-3 seconds, then all packets passed to the bird daemon in several microseconds, but failure has already happened. First of we configured port mirroring on the switch to capture all packets on the another host. In the mirrored traffic we can see that all packets were send and received in correct time windows. We use Mellanox network adapters, they have ability to attach hardware timestamps to mbufs, so we can know the time when packets were received. I made the patch to libpcap and bpf(4) to be able use these timestamps in tcpdump. https://people.freebsd.org/~ae/bpf_adapter.diff Then we have started two instances of tcpdump, one with host's timestamps (-j host), second with adapter's timestamps (-j adapter). When BFD failure happened, we saw equal ordering in both packet captures. But timestamps are different - packets that were arrived too late have correct receiving timestamp, so if they would not stalled in the queues, then BFD failer would not happened. I know that this story will not help you, but it might be useful for the mail archives :) -- WBR, Andrey V. Elsukov