Date: Wed, 13 Mar 2024 21:17:32 +0800 From: Zhenlei Huang <zlei@FreeBSD.org> To: Benoit Chesneau <benoitc@enki-multimedia.eu> Cc: "freebsd-net@FreeBSD.org" <freebsd-net@freebsd.org>, Alexander Chernikov <melifaro@freebsd.org> Subject: Re: ipv4 route with ipv6 local link nexthop ? Message-ID: <BE4D70C1-885D-4C6B-AE8C-AC62BD450F8A@FreeBSD.org> In-Reply-To: <764E12BF-5D31-4905-98AE-6D745BFD1DC2@FreeBSD.org> References: <Kj1GA9KKe3wVoeEJr0g-4BAiin5cEjGj4ICYDH9LHBT_vpmctoN-2HNS7AjTwNimOVkn5nBoJw50fbLgNdCleBD0p_TZImOet1gFGIfr4-E=@enki-multimedia.eu> <367504DC-48DA-4DFD-9DB6-CC571F0D26B8@FreeBSD.org> <764E12BF-5D31-4905-98AE-6D745BFD1DC2@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_25F69FF8-482D-44F0-9161-0D018DB2AD75 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Mar 13, 2024, at 9:09 PM, Zhenlei Huang <zlei@FreeBSD.org> wrote: >=20 >=20 >=20 >> On Mar 13, 2024, at 12:19 PM, Zhenlei Huang <zlei@FreeBSD.org = <mailto:zlei@FreeBSD.org>> wrote: >>=20 >>=20 >>=20 >>> On Mar 13, 2024, at 4:36 AM, Benoit Chesneau = <benoitc@enki-multimedia.eu <mailto:benoitc@enki-multimedia.eu>> wrote: >>>=20 >>> On latest freebsd 14.0 release , I am trying to join an ipv4 address = over an IPV6 local link using the following command : >>>=20 >>> `route add -net 10.200.1.1/32 -inet6 = fe80::9439:36ff:fef0:7cbd%vlan200` >>>=20 >>> But I get the following error: >>>=20 >>> ``` >>> route add -net 10.200.1.1/32 -inet6 = fe80::9439:36ff:fef0:7cbd%vlan200 >>> add net 10.200.1.1: gateway fe80::9439:36ff:fef0:7cbd%vlan200 fib 0: = Invalid argument >>=20 >> I guess that is regression by transition of route(8) to NETLINK. >>=20 >> Can you please build sbin/route from source without NETLINK and give = it another try ? >>=20 >> ``` >> # cd /usr/src/sbin/route >> # setenv MK_NETLINK_SUPPORT no >> # make >> ``` >=20 > Hi Benoit, >=20 > I managed to repeat this bug. >=20 > As a workaround, add an IPv4 address to the outgoing interface = (vlan200 in your case), > ``` > # ifconfig vlan200 inet x.x.x.x/y > ``` >=20 > then retry adding the route > ``` > # route add -net 10.200.1.1/32 -inet6 = fe80::9439:36ff:fef0:7cbd%vlan200 > ``` >=20 Hi Alexander, I think the logic of adding route from NETLINK routine is not complete. After bumping netlink debug level, ``` # sysctl net.netlink.debug.nl_route_debug_level=3D9 ``` I got these from kernel message: ``` # dmesg | tail ... [nl_route] finalize_nhop: Unable to determine ifa, skipping [nl_route] rtnl_handle_newroute: Error creating nexthop ... ``` =46rom = https://cgit.freebsd.org/src/tree/sys/netlink/route/rt.c?h=3Dreleng/14.0#n= 753 : ``` /* Both nh_ifp and gateway are set */ if (nh->nh_ifa =3D=3D NULL) { const struct sockaddr *gw_sa =3D &nh->gw_sa; if (gw_sa->sa_family !=3D dst->sa_family) { /* * Use dst as the target for determining the = default * preferred ifa IF * 1) the gateway is link-level (e.g. direct = route) * 2) the gateway family is different (e.g. IPv4 = over IPv6). */ gw_sa =3D dst; } struct ifaddr *ifa =3D ifaof_ifpforaddr(gw_sa, = nh->nh_ifp); if (ifa =3D=3D NULL) { NL_LOG(LOG_DEBUG, "Unable to determine ifa, = skipping"); *perror =3D EINVAL; return (NULL); } nhop_set_src(nh, ifa); } ``` If the outgoing interface `nh->nh_ifp` does not have any IPv4 addresses, = we should try ifaddrs on other interfaces, and have 0.0.0.0 as the last sort ( in case the net stack is IPv6 only, = i.e., no IPv4 addresses available ) >>=20 >>> ``` >>>=20 >>> ping6 to `fe80::9439:36ff:fef0:7cbd%vlan200` works. What I am doing = wrong? Configuration of the interface is the following: >>>=20 >>> ``` >>> vlan200: = flags=3D1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric = 0 mtu 1500 >>> = options=3D4600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6,MEXT= PG> >>> ether fe:d9:dd:2d:35:c5 >>> inet6 fe80::fcd9:ddff:fe2d:35c5%vlan200 prefixlen 64 scopeid = 0x5 >>> groups: vlan >>> vlan: 200 vlanproto: 802.1q vlanpcp: 0 parent interface: = iavf0 >>> media: Ethernet autoselect (10Gbase-SR <full-duplex>) >>> status: active >>> nd6 options=3D23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL >>> ``` >>>=20 >>>=20 >>>=20 >>> Beno=C3=AEt=20 >>=20 >> Best regards, >> Zhenlei --Apple-Mail=_25F69FF8-482D-44F0-9161-0D018DB2AD75 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br = class=3D""><div><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D"">On Mar 13, 2024, at 9:09 PM, Zhenlei Huang <<a = href=3D"mailto:zlei@FreeBSD.org" class=3D"">zlei@FreeBSD.org</a>> = wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><meta = charset=3D"UTF-8" class=3D""><br class=3D"Apple-interchange-newline"><br = class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 13px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><blockquote type=3D"cite" class=3D"" style=3D"font-family: = Helvetica; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; orphans: auto; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; = -webkit-text-stroke-width: 0px; text-decoration: none;"><div class=3D"">On= Mar 13, 2024, at 12:19 PM, Zhenlei Huang <<a = href=3D"mailto:zlei@FreeBSD.org" class=3D"">zlei@FreeBSD.org</a>> = wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><div = class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 13px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><br class=3D"Apple-interchange-newline"><br class=3D""><blockquote = type=3D"cite" class=3D""><div class=3D"">On Mar 13, 2024, at 4:36 AM, = Benoit Chesneau <<a href=3D"mailto:benoitc@enki-multimedia.eu" = class=3D"">benoitc@enki-multimedia.eu</a>> wrote:</div><br = class=3D"Apple-interchange-newline"><div class=3D""><div class=3D"" = style=3D"font-family: Arial, sans-serif; font-size: 14px;">On latest = freebsd 14.0 release , I am trying to join an ipv4 address over an IPV6 = local link using the following command :</div><div class=3D"" = style=3D"font-family: Arial, sans-serif; font-size: 14px;"><br = class=3D""></div><div class=3D"" style=3D"font-family: Arial, = sans-serif; font-size: 14px;"><span class=3D"">`route add -net = 10.200.1.1/32 -inet6 fe80::9439:36ff:fef0:7cbd%vlan200`</span><br = class=3D""></div><div class=3D"" style=3D"font-family: Arial, = sans-serif; font-size: 14px;"><span class=3D""><br = class=3D""></span></div><div class=3D"" style=3D"font-family: Arial, = sans-serif; font-size: 14px;"><span class=3D"">But I get the = following error:</span></div><div class=3D"" style=3D"font-family: = Arial, sans-serif; font-size: 14px;"><span class=3D""><br = class=3D""></span></div><div class=3D"" style=3D"font-family: Arial, = sans-serif; font-size: 14px;">```</div><div class=3D"" = style=3D"font-family: Arial, sans-serif; font-size: 14px;"><span = class=3D"">route add -net 10.200.1.1/32 -inet6 = fe80::9439:36ff:fef0:7cbd%vlan200</span><br class=3D""><span = class=3D"">add net 10.200.1.1: gateway fe80::9439:36ff:fef0:7cbd%vlan200 = fib 0: Invalid argument</span><br class=3D""></div></div></blockquote><div= class=3D""><br class=3D""></div><div class=3D"">I guess that is = regression by transition of route(8) to = NETLINK.</div></div></div></blockquote><blockquote type=3D"cite" = class=3D"" style=3D"font-family: Helvetica; font-size: 13px; font-style: = normal; font-variant-caps: normal; font-weight: 400; letter-spacing: = normal; orphans: auto; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; widows: auto; word-spacing: = 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; = text-decoration: none;"><div class=3D""><div class=3D"" = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><div = class=3D""><br class=3D""></div><div class=3D"">Can you please build = sbin/route from source without NETLINK and give it another try = ?</div><div class=3D""><br class=3D""></div><div class=3D"">```</div><div = class=3D""># cd /usr/src/sbin/route</div><div class=3D""># = setenv MK_NETLINK_SUPPORT no</div><div class=3D""># make</div><div = class=3D"">```</div></div></div></blockquote><div style=3D"caret-color: = rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: = normal; font-variant-caps: normal; font-weight: 400; letter-spacing: = normal; text-align: start; text-indent: 0px; text-transform: none; = white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><br class=3D""></div><div = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;" class=3D"">Hi = Benoit,</div><div style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;" class=3D""><br class=3D""></div><div style=3D"caret-color: rgb(0, = 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D"">I managed to repeat this = bug.</div><div style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;" class=3D""><br class=3D""></div><div style=3D"caret-color: rgb(0, = 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><div class=3D"">As a workaround, add = an IPv4 address to the outgoing interface (vlan200 in your = case),</div><div class=3D"">```</div><div class=3D""># ifconfig vlan200 = inet x.x.x.x/y</div><div class=3D"">```</div><div class=3D""><br = class=3D""></div><div class=3D"">then retry adding the route</div><div = class=3D"">```</div><div class=3D""># route add -net 10.200.1.1/32 = -inet6 fe80::9439:36ff:fef0:7cbd%vlan200</div><div = class=3D"">```</div><div class=3D""><br = class=3D""></div></div></div></blockquote><div><br = class=3D""></div><div>Hi Alexander,</div><div><br class=3D""></div><div>I = think the logic of adding route from NETLINK routine is not = complete.</div><div></div><div><br class=3D""></div><div><div>After = bumping netlink debug level,</div><div>```</div><div># sysctl = net.netlink.debug.nl_route_debug_level=3D9</div><div>```</div><div><br = class=3D""></div><div>I got these from kernel = message:</div><div>```</div><div># dmesg | = tail</div><div>...</div><div>[nl_route] finalize_nhop: Unable to = determine ifa, skipping</div><div>[nl_route] rtnl_handle_newroute: Error = creating nexthop</div><div>...</div><div>```</div><div><br = class=3D""></div><div>=46rom <a = href=3D"https://cgit.freebsd.org/src/tree/sys/netlink/route/rt.c?h=3Drelen= g/14.0#n753" = class=3D"">https://cgit.freebsd.org/src/tree/sys/netlink/route/rt.c?h=3Dre= leng/14.0#n753</a> :</div><div><br = class=3D""></div><div>```</div><div><span class=3D"Apple-tab-span" = style=3D"white-space:pre"> </span>/* Both nh_ifp and gateway are = set */</div><div><span class=3D"Apple-tab-span" style=3D"white-space:pre">= </span>if (nh->nh_ifa =3D=3D NULL) {</div><div><span = class=3D"Apple-tab-span" style=3D"white-space:pre"> = </span>const struct sockaddr *gw_sa =3D &nh->gw_sa;</div><div><br = class=3D""></div><div><span class=3D"Apple-tab-span" = style=3D"white-space:pre"> </span>if (gw_sa->sa_family = !=3D dst->sa_family) {</div><div><span class=3D"Apple-tab-span" = style=3D"white-space:pre"> = </span>/*</div><div><span class=3D"Apple-tab-span" = style=3D"white-space:pre"> </span> * Use dst as the = target for determining the default</div><div><span = class=3D"Apple-tab-span" style=3D"white-space:pre"> = </span> * preferred ifa IF</div><div><span class=3D"Apple-tab-span" = style=3D"white-space:pre"> </span> * 1) the gateway = is link-level (e.g. direct route)</div><div><span class=3D"Apple-tab-span"= style=3D"white-space:pre"> </span> * 2) the gateway = family is different (e.g. IPv4 over IPv6).</div><div><span = class=3D"Apple-tab-span" style=3D"white-space:pre"> = </span> */</div><div><span class=3D"Apple-tab-span" = style=3D"white-space:pre"> </span>gw_sa =3D = dst;</div><div><span class=3D"Apple-tab-span" style=3D"white-space:pre"> = </span>}</div><div><br class=3D""></div><div><span = class=3D"Apple-tab-span" style=3D"white-space:pre"> = </span>struct ifaddr *ifa =3D ifaof_ifpforaddr(gw_sa, = nh->nh_ifp);</div><div><span class=3D"Apple-tab-span" = style=3D"white-space:pre"> </span>if (ifa =3D=3D NULL) = {</div><div><span class=3D"Apple-tab-span" style=3D"white-space:pre"> = </span>NL_LOG(LOG_DEBUG, "Unable to determine ifa, = skipping");</div><div><span class=3D"Apple-tab-span" = style=3D"white-space:pre"> </span>*perror =3D = EINVAL;</div><div><span class=3D"Apple-tab-span" = style=3D"white-space:pre"> </span>return = (NULL);</div><div><span class=3D"Apple-tab-span" = style=3D"white-space:pre"> </span>}</div><div><span = class=3D"Apple-tab-span" style=3D"white-space:pre"> = </span>nhop_set_src(nh, ifa);</div><div><span class=3D"Apple-tab-span" = style=3D"white-space:pre"> </span>}</div><div>```</div><div>If the = outgoing interface `<span style=3D"caret-color: rgb(0, 0, 0); color: = rgb(0, 0, 0);" class=3D"">nh->nh_ifp` does not have any IPv4 = addresses, </span>we should try ifaddrs on other = interfaces,</div><div>and have 0.0.0.0 as the last sort ( in case the = net stack is IPv6 only, i.e., no IPv4 addresses available = )</div><div><br class=3D""></div></div><br class=3D""><blockquote = type=3D"cite" class=3D""><div class=3D""><blockquote type=3D"cite" = class=3D"" style=3D"font-family: Helvetica; font-size: 13px; font-style: = normal; font-variant-caps: normal; font-weight: 400; letter-spacing: = normal; orphans: auto; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; widows: auto; word-spacing: = 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; = text-decoration: none;"><div class=3D""><div class=3D"" = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><br = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><div = class=3D"" style=3D"font-family: Arial, sans-serif; font-size: = 14px;">```</div><div class=3D"" style=3D"font-family: Arial, sans-serif; = font-size: 14px;"><br class=3D""></div><div class=3D"" = style=3D"font-family: Arial, sans-serif; font-size: 14px;">ping6 to = `<span class=3D"" style=3D"text-decoration: none; background-color: = rgb(255, 255, 255); display: inline = !important;">fe80::9439:36ff:fef0:7cbd%vlan200` works. What I am doing = wrong? Configuration of the interface is the following:</span></div><div = class=3D"" style=3D"font-family: Arial, sans-serif; font-size: = 14px;"><span class=3D"" style=3D"text-decoration: none; = background-color: rgb(255, 255, 255); display: inline !important;"><br = class=3D""></span></div><div class=3D"" style=3D"font-family: Arial, = sans-serif; font-size: 14px;">```</div><div class=3D"" = style=3D"font-family: Arial, sans-serif; font-size: 14px;"><span = class=3D"">vlan200: = flags=3D1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> = metric 0 mtu 1500</span><div class=3D""><span class=3D""> = = options=3D4600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6,M= EXTPG></span></div><div class=3D""><span class=3D""> = ether fe:d9:dd:2d:35:c5</span></div><div class=3D""><span = class=3D""> inet6 = fe80::fcd9:ddff:fe2d:35c5%vlan200 prefixlen 64 scopeid = 0x5</span></div><div class=3D""><span class=3D""> = groups: vlan</span></div><div class=3D""><span class=3D""> = vlan: 200 vlanproto: 802.1q vlanpcp: 0 parent = interface: iavf0</span></div><div class=3D""><span class=3D""> = media: Ethernet autoselect (10Gbase-SR = <full-duplex>)</span></div><div class=3D""><span class=3D""> = status: active</span></div><span class=3D""> = nd6 = options=3D23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL</span><br = class=3D""></div><div class=3D"" style=3D"font-family: Arial, = sans-serif; font-size: 14px;">```</div><div class=3D"" = style=3D"font-family: Arial, sans-serif; font-size: 14px;"><span = class=3D""><br class=3D""></span></div><div class=3D"" = style=3D"font-family: Arial, sans-serif; font-size: 14px;"><span = class=3D""><br class=3D""></span></div><div class=3D"" = style=3D"font-family: Arial, sans-serif; font-size: 14px;"><br = class=3D""></div><div class=3D"protonmail_signature_block" = style=3D"font-family: Arial, sans-serif; font-size: 14px;"><div = class=3D"protonmail_signature_block-user"><div class=3D"" = style=3D"font-style: normal; font-weight: normal; letter-spacing: = normal; text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; text-decoration: none; font-family: Helvetica; = font-size: = 12px;">Beno=C3=AEt </div></div></div></div></blockquote></div><br = class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 13px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><div class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><div class=3D"">Best regards,</div><div = class=3D"">Zhenlei</div></div></div></blockquote></div></blockquote></div>= <br class=3D""><div class=3D""> <div><br class=3D""></div> </div> <br class=3D""></body></html>= --Apple-Mail=_25F69FF8-482D-44F0-9161-0D018DB2AD75--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BE4D70C1-885D-4C6B-AE8C-AC62BD450F8A>