Date: Wed, 10 Jun 2020 12:46:43 +0300 From: Daniel Kalchev <daniel@digsys.bg> To: "Eugene M. Zheganin" <emz@norma.perm.ru> Cc: freebsd-stable@freebsd.org Subject: Re: CARP under Hyper-V: weird things happen Message-ID: <717DD022-DB2E-4DAD-8504-09B67CE344A4@digsys.bg> In-Reply-To: <ed9180b4-fe2c-d516-af9d-cade780885f7@norma.perm.ru> References: <ed9180b4-fe2c-d516-af9d-cade780885f7@norma.perm.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Eugene, Might it be the Hyper-V doesn=E2=80=99t properly implement multicast? Or = there is perhaps some setting in there to let it work. =46rom memory = CARP is not trivial on vmware as well, unless you make special settings. = Some ideas here: = https://docs.netgate.com/pfsense/en/latest/highavailability/troubleshootin= g-high-availability-clusters.html#hypervisor-users-especially-vmware-esx-e= sxi = <https://docs.netgate.com/pfsense/en/latest/highavailability/troubleshooti= ng-high-availability-clusters.html#hypervisor-users-especially-vmware-esx-= esxi> Daniel > On 31 May 2020, at 19:07, Eugene M. Zheganin <emz@norma.perm.ru> = wrote: >=20 > Hello, >=20 > I'm Running 12.0-REL in a VM under W2016S with CARP enabled and paired = to a baremetal FreeBSD server. >=20 > All of a sudden I realized that thjis machine is unable to become a = CARP MASTER - because it sees it's own ACRP announces, but instead of = seeing them from a CARP synthetic MAC address only, it sees additional = extra packets with several MACs derived from the original one (I'm well = awared about the -MacAddressSpoof on SetVmNetworkAdapterVlan switch, and = it's running with this thingg on, but still). These packets always = almost (but not 100%) accompany each valid CARP advertisement. >=20 > Say, we have a CARP-enabled interface: >=20 > vlan2: flags=3D8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> = metric 0 mtu 1500 > description: AS WAN > options=3D80000<LINKSTATE> > ether 00:15:5d:0a:79:12 > inet 91.206.242.9/28 broadcast 91.206.242.15 > inet 91.206.242.12/28 broadcast 91.206.242.15 vhid 3 > groups: vlan > carp: BACKUP vhid 3 advbase 1 advskew 250 > vlan: 2 vlanpcp: 0 parent interface: hn1 > media: Ethernet autoselect (10Gbase-T <full-duplex>) > status: active > nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> >=20 > Notice the MAC and now look at this: >=20 > =3D=3D=3DCut=3D=3D=3D >=20 > [root@gw1:~]# tcpdump -T carp -nepi vlan2 carp > tcpdump: verbose output suppressed, use -v or -vv for full protocol = decode > listening on vlan2, link-type EN10MB (Ethernet), capture size 262144 = bytes > 20:45:54.152619 00:00:5e:00:01:03 > 01:00:5e:00:00:12, ethertype IPv4 = (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: = vhid=3D3 advbase=3D1 advskew=3D100 authlen=3D7 = counter=3D13769798250643227035 >=20 > ^^^ this is the ordinary and valid CARP advertisement, notice the = synthetic MAC which is requiring setting mac address spoofing. >=20 > 20:45:54.152880 9c:8e:99:0f:79:42 > 01:00:5e:00:00:12, ethertype IPv4 = (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: = vhid=3D3 advbase=3D1 advskew=3D100 authlen=3D7 = counter=3D13769798250643227035 >=20 > ^^^ this is some insanity happening >=20 > 20:45:54.153234 9c:8e:99:0f:79:42 > 01:00:5e:00:00:12, ethertype IPv4 = (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: = vhid=3D3 advbase=3D1 advskew=3D100 authlen=3D7 = counter=3D13769798250643227035 >=20 > ^^^ and again >=20 > 20:45:54.153401 9c:8e:99:0f:79:42 > 01:00:5e:00:00:12, ethertype IPv4 = (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: = vhid=3D3 advbase=3D1 advskew=3D100 authlen=3D7 = counter=3D13769798250643227035 >=20 > ^^^ and again >=20 > 20:45:57.562470 00:00:5e:00:01:03 > 01:00:5e:00:00:12, ethertype IPv4 = (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: = vhid=3D3 advbase=3D1 advskew=3D100 authlen=3D7 = counter=3D13769798250643227036 >=20 > ^^^ valid CARP advertisement, next one-second advbase cycle >=20 > 20:45:57.562874 9c:8e:99:0f:79:3c > 01:00:5e:00:00:12, ethertype IPv4 = (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: = vhid=3D3 advbase=3D1 advskew=3D100 authlen=3D7 = counter=3D13769798250643227036 >=20 > ^^^ more insane stuff, notice the NEW (sic !) MAC-address >=20 > 20:45:57.562955 9c:8e:99:0f:79:3c > 01:00:5e:00:00:12, ethertype IPv4 = (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: = vhid=3D3 advbase=3D1 advskew=3D100 authlen=3D7 = counter=3D13769798250643227036 > 20:45:57.562989 9c:8e:99:0f:79:3c > 01:00:5e:00:00:12, ethertype IPv4 = (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: = vhid=3D3 advbase=3D1 advskew=3D100 authlen=3D7 = counter=3D13769798250643227036 > ^C > 8 packets captured > 3195 packets received by filter >=20 > =3D=3D=3DCut=3D=3D=3D >=20 >=20 > Does anyone has, by any chance, some idea about what's happening ? As = soon as I stop CARP stack on this VM these "mad" MACs aren't received = anymore, so I'm pretty confident these are somehow procuced on the = Hyper-V side. >=20 > Another weird this is that vlan1 is refusing to work (seems like = packets are never received on the VM side) unless its configured on = another adapter in the -Untagged (once again powershell term for = SetVmNetworkAdapterVlan). >=20 >=20 > Thanks. >=20 > Eugene. >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?717DD022-DB2E-4DAD-8504-09B67CE344A4>