Date: Thu, 30 Mar 2023 10:07:09 +0800 From: Zhenlei Huang <zlei@FreeBSD.org> To: overwatch <overwatch@lab.kyngin.net> Cc: freebsd-net@freebsd.org Subject: Re: Help testing patch that may help diagnosing the PR 240106 Message-ID: <F487DF63-B7D3-41DE-9C46-9F60D4D55FE9@FreeBSD.org> In-Reply-To: <16b52f9e-d146-737e-1fe5-01ad62492c7f@lab.kyngin.net> References: <4431A103-64AA-4EDF-9830-ED320161B75C@FreeBSD.org> <4416034C-237B-4D8F-B04E-A22F2D56BF9C@FreeBSD.org> <16b52f9e-d146-737e-1fe5-01ad62492c7f@lab.kyngin.net>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] > On Mar 30, 2023, at 2:07 AM, overwatch <overwatch@lab.kyngin.net> wrote: > > Zhenlei, I did get your message here, and I believe I understand the situation, though I don't know if it represents exactly what I was seeing with our setup. > > I'll try to install your patch and set the switch back to how it was, if only to see if it shows up any errors, just as soon as I get a free minute. > > Thank you for the patch! > > ps - Which OS version for this patch? 13.1 release? The patch is against current, and I've tested on stable/13. It can be applied smoothly on releng/13.2, releng/13.1 and stable/12 branches. You can do it on whichever branch convenient for you ;) > > -kvs > > > On 3/29/23 00:05, Zhenlei Huang wrote: >> >>> On Mar 29, 2023, at 1:03 PM, Zhenlei Huang <zlei@FreeBSD.org> wrote: >>> >>> Hi, >>> >>> I write here so that the original PR 240106 is not polluted. >>> >>> Can you please test the attached patch with bridge / lagg setup? >>> >>> For long: >>> >>> In https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106#c28 you encountered >>> problem and I said: >>> >>>> The IF_BRIDGE(4) seems to hide some thing to protect itself get confused. >>> Actually IF_BRIDGE(4) has a learning mode. You can `man ifconfig` and refer the >>> `Bridge Interface Parameters` section. >>> >>> By default the learning mode of all bridge members is on, and the bridge will >>> insert or update an entry to its (internal) forwarding table. When unicast packets >>> come to the bridge member, the bridge will check if it is for itself, if not then >>> the packets will be forwarded to one bridge member if a forwarding entry is found. >>> While the magic is, if the bridge member to be forwarded is the receiving one, then >>> the packets are silently discarded. >>> >>> That's perfect fine, but will be hard to diagnose if user has wrong network setup, >>> bridge loops e.g., or some other ones set duplicated ether address for their nic, >>> or some bad guys / virus / trojans send spoofed packets on the wire. Those are common >>> and I think it will be good if IF_BRIDGE(4) can emit logs so that the symptoms will >>> be obvious and it will be easy to diagnose. >>> >>>> If you can confirm, then please config you switch properly. The two ports cc0 and cc1 connected should be in same link aggregation group. >>> If two ports (on physical switch), say 1 and 2, are not in same link aggregation group, >>> then packets (typically broadcast ones) received on 1 will be forwarded to 2, and >>> the lagg interface will be bounce-backed (from port 2) the packets it send (to port 1). >>> If the lagg happenly be the member of IF_BRIDGE(4), then the bridge will update >>> its forwarding entry as it learn mac address from lagg interface. >>> >>> Here is a simple diagram, the arrow shows the flow of ARP request from epair0a. >>> >>> 11:22:33:44:55:66 [1] -> cc0 -> port 1 -> >>> epair0a -> epair0b -> bridge0 -> lagg0 physical-switch <-> host0 >>> <- <- cc1 <- port 2 <- >>> [2] >>> >>> On [1] bridge0 will learn MAC 11:22:33:44:55:66 on port member epair0b and add entry, >>> after [2] it will learn same MAC on port member lagg0 and update the entry. Then >>> subsequent ARP reply (to 11:22:33:44:55:66, epair0a i.e.) sent from host0 reach bridge0 >>> via lagg0. >>> >>> Apparently bridge0 will dropped the ARP reply as it believes 11:22:33:44:55:66 (epair0a) is >>> within segment of lagg0. >>> >>>> I'll see if I can teach IF_BRIDGE(4) to emit warnings in case it get ARP request packet sent from it self. >>> Attached patch will enable IF_BRIDGE(4) to emit logs about MAC address port flapping. >>> Various hardware vendors have similar facilities. >>> >>> >>> Best regards, >>> Zhenlei >>> >> Sorry forgot the patch. >> >> >> >> >> [-- Attachment #2 --] <html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Mar 30, 2023, at 2:07 AM, overwatch <<a href="mailto:overwatch@lab.kyngin.net" class="">overwatch@lab.kyngin.net</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">Zhenlei, I did get your message here, and I believe I understand the situation, though I don't know if it represents exactly what I was seeing with our setup.<br class=""><br class="">I'll try to install your patch and set the switch back to how it was, if only to see if it shows up any errors, just as soon as I get a free minute.<br class=""><br class="">Thank you for the patch!<br class=""><br class="">ps - Which OS version for this patch? 13.1 release?<br class=""></div></div></blockquote><div><br class=""></div><div>The patch is against current, and I've tested on stable/13.</div><div>It can be applied smoothly on releng/13.2, <span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">releng/13.1 and stable/12 branches.</span></div><div><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""><br class=""></span></div><div><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">You can do it on whichever branch </span><font color="#000000" class=""><span style="caret-color: rgb(0, 0, 0);" class="">convenient for you ;)</span></font></div><br class=""><blockquote type="cite" class=""><div class=""><div class=""><br class="">-kvs<br class=""><br class=""><br class="">On 3/29/23 00:05, Zhenlei Huang wrote:<br class=""><blockquote type="cite" class=""><br class=""><blockquote type="cite" class="">On Mar 29, 2023, at 1:03 PM, Zhenlei Huang <<a href="mailto:zlei@FreeBSD.org" class="">zlei@FreeBSD.org</a>> wrote:<br class=""><br class="">Hi,<br class=""><br class="">I write here so that the original PR 240106 is not polluted.<br class=""><br class="">Can you please test the attached patch with bridge / lagg setup?<br class=""><br class="">For long:<br class=""><br class="">In <a href="https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106#c28" class="">https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106#c28</a>; you encountered<br class="">problem and I said:<br class=""><br class=""><blockquote type="cite" class="">The IF_BRIDGE(4) seems to hide some thing to protect itself get confused.<br class=""></blockquote>Actually IF_BRIDGE(4) has a learning mode. You can `man ifconfig` and refer the<br class="">`Bridge Interface Parameters` section.<br class=""><br class="">By default the learning mode of all bridge members is on, and the bridge will<br class="">insert or update an entry to its (internal) forwarding table. When unicast packets<br class="">come to the bridge member, the bridge will check if it is for itself, if not then<br class="">the packets will be forwarded to one bridge member if a forwarding entry is found.<br class="">While the magic is, if the bridge member to be forwarded is the receiving one, then<br class="">the packets are silently discarded.<br class=""><br class="">That's perfect fine, but will be hard to diagnose if user has wrong network setup,<br class="">bridge loops e.g., or some other ones set duplicated ether address for their nic,<br class="">or some bad guys / virus / trojans send spoofed packets on the wire. Those are common<br class="">and I think it will be good if IF_BRIDGE(4) can emit logs so that the symptoms will<br class="">be obvious and it will be easy to diagnose.<br class=""><br class=""><blockquote type="cite" class="">If you can confirm, then please config you switch properly. The two ports cc0 and cc1 connected should be in same link aggregation group.<br class=""></blockquote>If two ports (on physical switch), say 1 and 2, are not in same link aggregation group,<br class="">then packets (typically broadcast ones) received on 1 will be forwarded to 2, and<br class="">the lagg interface will be bounce-backed (from port 2) the packets it send (to port 1).<br class="">If the lagg happenly be the member of IF_BRIDGE(4), then the bridge will update<br class="">its forwarding entry as it learn mac address from lagg interface.<br class=""><br class="">Here is a simple diagram, the arrow shows the flow of ARP request from epair0a.<br class=""><br class="">11:22:33:44:55:66 [1] -> cc0 -> port 1 -><br class=""> epair0a -> epair0b -> bridge0 -> lagg0 physical-switch <-> host0<br class=""> <- <- cc1 <- port 2 <-<br class=""> [2]<br class=""><br class="">On [1] bridge0 will learn MAC 11:22:33:44:55:66 on port member epair0b and add entry,<br class="">after [2] it will learn same MAC on port member lagg0 and update the entry. Then<br class="">subsequent ARP reply (to 11:22:33:44:55:66, epair0a i.e.) sent from host0 reach bridge0<br class="">via lagg0.<br class=""><br class="">Apparently bridge0 will dropped the ARP reply as it believes 11:22:33:44:55:66 (epair0a) is<br class="">within segment of lagg0.<br class=""><br class=""><blockquote type="cite" class="">I'll see if I can teach IF_BRIDGE(4) to emit warnings in case it get ARP request packet sent from it self.<br class=""></blockquote>Attached patch will enable IF_BRIDGE(4) to emit logs about MAC address port flapping.<br class="">Various hardware vendors have similar facilities.<br class=""><br class=""><br class="">Best regards,<br class="">Zhenlei<br class=""><br class=""></blockquote>Sorry forgot the patch.<br class=""><br class=""><br class=""><br class=""><br class=""><br class=""></blockquote></div></div></blockquote></div><div class=""><div><br class=""></div> </div> <br class=""></body></html>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?F487DF63-B7D3-41DE-9C46-9F60D4D55FE9>