Date: Tue, 05 Dec 2023 14:21:21 +0000 From: Benoit Chesneau <benoitc@enki-multimedia.eu> To: Kristof Provost <kp@FreeBSD.org> Cc: freebsd-net@freebsd.org Subject: Re: vxlan with IPv6 underlay ? Message-ID: <Q4lNxBr17sBg-pHCFTyIugCDvSpd0oXXH6UBv5yEkJtq2lJKIXG_K4MSDarhwkFt7PAFvaLTLdm-rj6MR-ix7B8BfG-7aINXUg5h2a5VkPg=@enki-multimedia.eu> In-Reply-To: <793EA8DF-BD26-414A-A12D-866356D339DB@FreeBSD.org> References: <bmkPdpqYkXzCqs5CYfBIZO6sXwmR1hU38pCr5cRlBeTxM4VIbzTj_A0YU6yFP2CbVxXHoOLeqHpdBY1reGJS_VvNBesk3J1HBWblpjijl2o=@enki-multimedia.eu> <793EA8DF-BD26-414A-A12D-866356D339DB@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Changing the mtu wasn't enough. To make it works i had to disable vxlan off=
load features:
```
ifconfig mce0 -vxlanhwcsum -vxlanhwtso
```
Then suddenly all started to work. The card is the following:
```
mlx5_core1@pci0:17:0:1:=09class=3D0x020000 rev=3D0x00 hdr=3D0x00 vendor=3D0=
x15b3 device=3D0x1015 subvendor=3D0x1590 subdevice=3D0x00d4
vendor =3D 'Mellanox Technologies'
device =3D 'MT27710 Family [ConnectX-4 Lx]'
class =3D network
subclass =3D ethernet
PCI-e errors =3D Correctable Error Detected
Unsupported Request Detected
Corrected =3D Advisory Non-Fatal Error
```
Which is odd. Wonder why it's needed. Thoughts?
Beno=C3=AEt=20
On Tuesday, December 5th, 2023 at 13:11, Kristof Provost <kp@FreeBSD.org> w=
rote:
> On 4 Dec 2023, at 11:25, Benoit Chesneau wrote:
>=20
> > Hi,
> >=20
> > Is IPv6 underlay fully supported with FreebBSD ? I have created the a t=
unnel and associated an Ipv6 address to each side. I'm able to ping between=
each devicesl. But when I want to curl from the remote side it timeout. Lo=
cally on the remote side it is OK. Is this expected ? Should I rather creat=
e a bridge with vxlan as a member and bind nginx to it ?
> >=20
> > ```
> > $ ifconfig vxlan0 create vxlanid 108 vxlanlocal AAAA:BBBB:110b:102::100=
vxlanremote AAAA:BBBB:110b:102::12
> > $ ifconfig vxlan0
> > vxlan0: flags=3D1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP=
> metric 0 mtu 1430
> > options=3D80020<JUMBO_MTU,LINKSTATE>
> > ether 58:9c:fc:10:ff:eb
> > groups: vxlan
> > vxlan vni 108 local [AAAA:BBBB:102::100]:4789 remote [AAAA:BBBB:110b:10=
2::12]:4789
> > media: Ethernet autoselect (autoselect <full-duplex>)
> > status: active nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> > $ ifconfig vxlan0 inet6 AAAA:BBBB:110b:300::1/64
> > ````
> >=20
> > Ping from remote is ok:
> > `ifconfig vxlan0 vxlan0: flags=3D1008843<UP,BROADCAST,RUNNING,SIMPLEX,M=
ULTICAST,LOWER_UP> metric 0 mtu 1430 options=3D680323<RXCSUM,TXCSUM,JUMBO_M=
TU,TSO4,TSO6,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> ether 58:9c:fc:10:df:1f ine=
t6 fe80::5a9c:fcff:fe10:df1f%vxlan0 prefixlen 64 scopeid 0xf inet6 AAAA:BBB=
B:110b:300::2 prefixlen 64 groups: vxlan vxlan vni 108 local [AAAA:BBBB110b=
:102::12]:4789 remote [AAAA:BBBB:110b:102::100]:4789 media: Ethernet autose=
lect (autoselect <full-duplex>) status: active nd6 options=3D21<PERFORMNUD,=
AUTO_LINKLOCAL> $ ping6 AAAA:BBBB:110b:300::1 PING6(56=3D40+8+8 bytes) AAAA=
:BBBB:110b:300::2 --> AAAA:BBBB::110b:300::1 16 bytes from 2a0e:e701:110b:3=
00::1, icmp_seq=3D0 hlim=3D64 time=3D0.071 ms 16 bytes from 2a0e:e701:110b:=
300::1, icmp_seq=3D1 hlim=3D64 time=3D0.078 ms 16 bytes from 2a0e:e701:110b=
:300::1, icmp_seq=3D2 hlim=3D64 time=3D0.076 ms 16 bytes from 2a0e:e701:110=
b:300::1, icmp_seq=3D3 hlim=3D64 time=3D0.104 ms 16 bytes from 2a0e:e701:11=
0b:300::1, icmp_seq=3D4 hlim=3D64 time=3D0.077 ms^C`
> >=20
> > But when I run `curl -6 -v 'http://[AAAA:BBBB:110b:300::1]'` it timeout=
.
>=20
> That smells like an MTU issue. Make sure you=E2=80=99re not blocking ICMP=
v6, and check with larger ping messages.
>=20
> Best regards,
> Kristof
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Q4lNxBr17sBg-pHCFTyIugCDvSpd0oXXH6UBv5yEkJtq2lJKIXG_K4MSDarhwkFt7PAFvaLTLdm-rj6MR-ix7B8BfG-7aINXUg5h2a5VkPg=>