From nobody Thu Mar 30 02:07:09 2023 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pn6Hq2HMCz42Dn0 for ; Thu, 30 Mar 2023 02:07:19 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pn6Hq02S6z3CBw; Thu, 30 Mar 2023 02:07:19 +0000 (UTC) (envelope-from zlei@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1680142039; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=fSTm75o2Z4dSjRG8KlReSI3dbkP5ddCsQQYVzph563M=; b=NqqopsGgTLeIFAw7yZ7tXElEkj+Av2YVb1TiNylMELU2ZtbG7k1s3FLxoB0Z6WhqnS05gZ Ss5kPiKjZlnuDysTv23gScBC7mWae0G20vpQfls3Z1a7+zVgfuhnJgNpycTJDgEIC4Dutf bTFFpUXc3xmkKrU/Z1c8nzO6HzPAUcBrJP/jAi7nVmq6/mHxc8NOzUgfsxkMdDOoFOwypv XymYreMXhMtJJ4B9LNi0xZQJd4Qga623yATu2BCId9qJ9RDa2sGm/c49LATTPP0S8Pd3iI jG7MHnfoB9AMF81ikAjhXvkTgrSSECTAW/OxT3MWmfqZiMVUCLcr6FsVqqswiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1680142039; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=fSTm75o2Z4dSjRG8KlReSI3dbkP5ddCsQQYVzph563M=; b=ecCS6vqWmti8cjP0GhyhLMWKxlGWcub9tb/cSf6hOmt72Ilr3i6ixa7Gs7QQQ5CAmhSHq1 BiKExKCKnfk23t9YNa6COIyUJ4nn2qTTxdYjYf3JWX/fyrjh6Xp0Bhf2IMVxRAKSiyqjJ6 TQrLfVCrQJzLY42ImJs7U8ExDhjieor+gpNQhPkkxvzo4kCZ/HJ+oQDNW3ouaGaexq2+f+ EE8fEafAakKHsZbHe8qPJzj3bFdgcJd0EjggV4txn+20m5T4oVSmZTTF2fV4P/Eq+AwN8Q PXI50xMbXFNb/+SjnToHmC8NlC4YNHn68avo2UiHiB48m9AaMkI3mn65KqI89A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1680142039; a=rsa-sha256; cv=none; b=YDHHRcpqVPtOVMggebR9bZd8E5hYpvU6kZRhXMOxPoxLlN4/lBKrmELBeQdwk4x4q6W5GJ 6vXx7dWRfbJmNNTpcaC9TtVlrvnTP2vS8wnCAMI7gqG/h+vrL/Vc028yJoaRRwX5D42N+5 3lmpImCWIalQ7MsSgjy325cUJbHIi9VBmLrnV1Kl+pmu2o4ZLU5q9CVxq2Z1F1oCj5ZABX hiI9zsep7q6jAEFqDV8/Azn4yAc54vk+XtJoZ5F2lvcQg9T0UFebCrrO/8o7uY6BsxT0KI 1kPHTqFlsG7HFEqm9pwiN+jmcwFES2HPxt3j3dkGsLbzTrN8omLTPquv14k6FA== Received: from smtpclient.apple (ns1.oxydns.net [45.32.91.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: zlei/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Pn6Hp0mp9zMXX; Thu, 30 Mar 2023 02:07:17 +0000 (UTC) (envelope-from zlei@FreeBSD.org) From: Zhenlei Huang Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_751519AE-ED8B-42B6-9743-15760EE6EFFE" List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: Help testing patch that may help diagnosing the PR 240106 Date: Thu, 30 Mar 2023 10:07:09 +0800 In-Reply-To: <16b52f9e-d146-737e-1fe5-01ad62492c7f@lab.kyngin.net> Cc: freebsd-net@freebsd.org To: overwatch References: <4431A103-64AA-4EDF-9830-ED320161B75C@FreeBSD.org> <4416034C-237B-4D8F-B04E-A22F2D56BF9C@FreeBSD.org> <16b52f9e-d146-737e-1fe5-01ad62492c7f@lab.kyngin.net> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_751519AE-ED8B-42B6-9743-15760EE6EFFE Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Mar 30, 2023, at 2:07 AM, overwatch = wrote: >=20 > Zhenlei, I did get your message here, and I believe I understand the = situation, though I don't know if it represents exactly what I was = seeing with our setup. >=20 > I'll try to install your patch and set the switch back to how it was, = if only to see if it shows up any errors, just as soon as I get a free = minute. >=20 > Thank you for the patch! >=20 > ps - Which OS version for this patch? 13.1 release? The patch is against current, and I've tested on stable/13. It can be applied smoothly on releng/13.2, releng/13.1 and stable/12 = branches. You can do it on whichever branch convenient for you ;) >=20 > -kvs >=20 >=20 > On 3/29/23 00:05, Zhenlei Huang wrote: >>=20 >>> On Mar 29, 2023, at 1:03 PM, Zhenlei Huang wrote: >>>=20 >>> Hi, >>>=20 >>> I write here so that the original PR 240106 is not polluted. >>>=20 >>> Can you please test the attached patch with bridge / lagg setup? >>>=20 >>> For long: >>>=20 >>> In https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D240106#c28 = you encountered >>> problem and I said: >>>=20 >>>> The IF_BRIDGE(4) seems to hide some thing to protect itself get = confused. >>> Actually IF_BRIDGE(4) has a learning mode. You can `man ifconfig` = and refer the >>> `Bridge Interface Parameters` section. >>>=20 >>> By default the learning mode of all bridge members is on, and the = bridge will >>> insert or update an entry to its (internal) forwarding table. When = unicast packets >>> come to the bridge member, the bridge will check if it is for = itself, if not then >>> the packets will be forwarded to one bridge member if a forwarding = entry is found. >>> While the magic is, if the bridge member to be forwarded is the = receiving one, then >>> the packets are silently discarded. >>>=20 >>> That's perfect fine, but will be hard to diagnose if user has wrong = network setup, >>> bridge loops e.g., or some other ones set duplicated ether address = for their nic, >>> or some bad guys / virus / trojans send spoofed packets on the wire. = Those are common >>> and I think it will be good if IF_BRIDGE(4) can emit logs so that = the symptoms will >>> be obvious and it will be easy to diagnose. >>>=20 >>>> If you can confirm, then please config you switch properly. The two = ports cc0 and cc1 connected should be in same link aggregation group. >>> If two ports (on physical switch), say 1 and 2, are not in same link = aggregation group, >>> then packets (typically broadcast ones) received on 1 will be = forwarded to 2, and >>> the lagg interface will be bounce-backed (from port 2) the packets = it send (to port 1). >>> If the lagg happenly be the member of IF_BRIDGE(4), then the bridge = will update >>> its forwarding entry as it learn mac address from lagg interface. >>>=20 >>> Here is a simple diagram, the arrow shows the flow of ARP request = from epair0a. >>>=20 >>> 11:22:33:44:55:66 [1] -> cc0 -> port 1 -> >>> epair0a -> epair0b -> bridge0 -> lagg0 = physical-switch <-> host0 >>> <- <- cc1 <- port 2 <- >>> [2] >>>=20 >>> On [1] bridge0 will learn MAC 11:22:33:44:55:66 on port member = epair0b and add entry, >>> after [2] it will learn same MAC on port member lagg0 and update the = entry. Then >>> subsequent ARP reply (to 11:22:33:44:55:66, epair0a i.e.) sent from = host0 reach bridge0 >>> via lagg0. >>>=20 >>> Apparently bridge0 will dropped the ARP reply as it believes = 11:22:33:44:55:66 (epair0a) is >>> within segment of lagg0. >>>=20 >>>> I'll see if I can teach IF_BRIDGE(4) to emit warnings in case it = get ARP request packet sent from it self. >>> Attached patch will enable IF_BRIDGE(4) to emit logs about MAC = address port flapping. >>> Various hardware vendors have similar facilities. >>>=20 >>>=20 >>> Best regards, >>> Zhenlei >>>=20 >> Sorry forgot the patch. >>=20 >>=20 >>=20 >>=20 >>=20 --Apple-Mail=_751519AE-ED8B-42B6-9743-15760EE6EFFE Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii

On Mar 30, 2023, at 2:07 AM, overwatch <overwatch@lab.kyngin.net> wrote:

Zhenlei, I did get your message here, and I believe I = understand the situation, though I don't know if it represents exactly = what I was seeing with our setup.

I'll try = to install your patch and set the switch back to how it was, if only to = see if it shows up any errors, just as soon as I get a free minute.

Thank you for the patch!

ps - Which OS version for this patch?  13.1 release?

The = patch is against current, and I've tested on stable/13.
It can = be applied smoothly on releng/13.2, releng/13.1 and stable/12 = branches.

You = can do it on whichever branch convenient for you ;)


-kvs


On 3/29/23 00:05, Zhenlei Huang wrote:

On Mar 29, 2023, at 1:03 PM, Zhenlei Huang = <zlei@FreeBSD.org> wrote:

Hi,

I write here so that the = original PR 240106 is not polluted.

Can you = please test the attached patch with bridge / lagg setup?
For long:

In https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D240106#c28<= /a> you encountered
problem and I said:

The IF_BRIDGE(4) seems = to hide some thing to protect itself get confused.
Actually IF_BRIDGE(4) has a learning mode. You = can `man ifconfig` and refer the
`Bridge Interface = Parameters` section.

By default the = learning mode of all bridge members is on, and the bridge will
insert or update an entry to its (internal) forwarding table. = When unicast packets
come to the bridge member, the bridge = will check if it is for itself, if not then
the packets = will be forwarded to one bridge member if a forwarding entry is = found.
While the magic is, if the bridge member to be = forwarded is the receiving one, then
the packets are = silently discarded.

That's perfect fine, = but will be hard to diagnose if user has wrong network setup,
bridge loops e.g., or some other ones set duplicated ether = address for their nic,
or some bad guys / virus / trojans = send spoofed packets on the wire. Those are common
and I = think it will be good if IF_BRIDGE(4) can emit logs so that the symptoms = will
be obvious and it will be easy to diagnose.

If you = can confirm, then please config you switch properly. The two ports cc0 = and cc1 connected should be in same link aggregation group.
If two ports (on physical switch), say 1 and 2, = are not in same link aggregation group,
then packets = (typically broadcast ones) received on 1 will be forwarded to 2, and
the lagg interface will be bounce-backed (from port 2) the = packets it send (to port 1).
If the lagg happenly be the = member of IF_BRIDGE(4), then the bridge will update
its = forwarding entry as it learn mac address from lagg interface.

Here is a simple diagram, the arrow shows the = flow of ARP request from epair0a.

11:22:33:44:55:66 =         [1] =             &n= bsp;    -> cc0 ->  port 1 ->
      epair0a -> epair0b = -> bridge0 -> lagg0 =             &n= bsp;          physical-s= witch <-> host0
=             &n= bsp;           &nbs= p;           <- =        <- cc1 <-  port 2 = <-
=             &n= bsp;           &nbs= p;           [2]
On [1] bridge0 will learn MAC = 11:22:33:44:55:66 on port member epair0b and add entry,
after [2] it will learn same MAC on port member lagg0 and = update the entry. Then
subsequent ARP reply (to = 11:22:33:44:55:66, epair0a i.e.) sent from host0 reach bridge0
via lagg0.

Apparently bridge0 = will dropped the ARP reply as it believes 11:22:33:44:55:66 (epair0a) = is
within segment of lagg0.

I'll see if I can teach = IF_BRIDGE(4) to emit warnings in case it get ARP request packet sent = from it self.
Attached patch will enable = IF_BRIDGE(4) to emit logs about MAC address port flapping.
Various hardware vendors have similar facilities.


Best regards,
Zhenlei

Sorry = forgot the patch.







= --Apple-Mail=_751519AE-ED8B-42B6-9743-15760EE6EFFE--