Date: Wed, 13 Mar 2024 21:17:32 +0800 From: Zhenlei Huang <zlei@FreeBSD.org> To: Benoit Chesneau <benoitc@enki-multimedia.eu> Cc: "freebsd-net@FreeBSD.org" <freebsd-net@freebsd.org>, Alexander Chernikov <melifaro@freebsd.org> Subject: Re: ipv4 route with ipv6 local link nexthop ? Message-ID: <BE4D70C1-885D-4C6B-AE8C-AC62BD450F8A@FreeBSD.org> In-Reply-To: <764E12BF-5D31-4905-98AE-6D745BFD1DC2@FreeBSD.org> References: <Kj1GA9KKe3wVoeEJr0g-4BAiin5cEjGj4ICYDH9LHBT_vpmctoN-2HNS7AjTwNimOVkn5nBoJw50fbLgNdCleBD0p_TZImOet1gFGIfr4-E=@enki-multimedia.eu> <367504DC-48DA-4DFD-9DB6-CC571F0D26B8@FreeBSD.org> <764E12BF-5D31-4905-98AE-6D745BFD1DC2@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] > On Mar 13, 2024, at 9:09 PM, Zhenlei Huang <zlei@FreeBSD.org> wrote: > > > >> On Mar 13, 2024, at 12:19 PM, Zhenlei Huang <zlei@FreeBSD.org <mailto:zlei@FreeBSD.org>> wrote: >> >> >> >>> On Mar 13, 2024, at 4:36 AM, Benoit Chesneau <benoitc@enki-multimedia.eu <mailto:benoitc@enki-multimedia.eu>> wrote: >>> >>> On latest freebsd 14.0 release , I am trying to join an ipv4 address over an IPV6 local link using the following command : >>> >>> `route add -net 10.200.1.1/32 -inet6 fe80::9439:36ff:fef0:7cbd%vlan200` >>> >>> But I get the following error: >>> >>> ``` >>> route add -net 10.200.1.1/32 -inet6 fe80::9439:36ff:fef0:7cbd%vlan200 >>> add net 10.200.1.1: gateway fe80::9439:36ff:fef0:7cbd%vlan200 fib 0: Invalid argument >> >> I guess that is regression by transition of route(8) to NETLINK. >> >> Can you please build sbin/route from source without NETLINK and give it another try ? >> >> ``` >> # cd /usr/src/sbin/route >> # setenv MK_NETLINK_SUPPORT no >> # make >> ``` > > Hi Benoit, > > I managed to repeat this bug. > > As a workaround, add an IPv4 address to the outgoing interface (vlan200 in your case), > ``` > # ifconfig vlan200 inet x.x.x.x/y > ``` > > then retry adding the route > ``` > # route add -net 10.200.1.1/32 -inet6 fe80::9439:36ff:fef0:7cbd%vlan200 > ``` > Hi Alexander, I think the logic of adding route from NETLINK routine is not complete. After bumping netlink debug level, ``` # sysctl net.netlink.debug.nl_route_debug_level=9 ``` I got these from kernel message: ``` # dmesg | tail ... [nl_route] finalize_nhop: Unable to determine ifa, skipping [nl_route] rtnl_handle_newroute: Error creating nexthop ... ``` From https://cgit.freebsd.org/src/tree/sys/netlink/route/rt.c?h=releng/14.0#n753 : ``` /* Both nh_ifp and gateway are set */ if (nh->nh_ifa == NULL) { const struct sockaddr *gw_sa = &nh->gw_sa; if (gw_sa->sa_family != dst->sa_family) { /* * Use dst as the target for determining the default * preferred ifa IF * 1) the gateway is link-level (e.g. direct route) * 2) the gateway family is different (e.g. IPv4 over IPv6). */ gw_sa = dst; } struct ifaddr *ifa = ifaof_ifpforaddr(gw_sa, nh->nh_ifp); if (ifa == NULL) { NL_LOG(LOG_DEBUG, "Unable to determine ifa, skipping"); *perror = EINVAL; return (NULL); } nhop_set_src(nh, ifa); } ``` If the outgoing interface `nh->nh_ifp` does not have any IPv4 addresses, we should try ifaddrs on other interfaces, and have 0.0.0.0 as the last sort ( in case the net stack is IPv6 only, i.e., no IPv4 addresses available ) >> >>> ``` >>> >>> ping6 to `fe80::9439:36ff:fef0:7cbd%vlan200` works. What I am doing wrong? Configuration of the interface is the following: >>> >>> ``` >>> vlan200: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500 >>> options=4600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6,MEXTPG> >>> ether fe:d9:dd:2d:35:c5 >>> inet6 fe80::fcd9:ddff:fe2d:35c5%vlan200 prefixlen 64 scopeid 0x5 >>> groups: vlan >>> vlan: 200 vlanproto: 802.1q vlanpcp: 0 parent interface: iavf0 >>> media: Ethernet autoselect (10Gbase-SR <full-duplex>) >>> status: active >>> nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL >>> ``` >>> >>> >>> >>> Benoît >> >> Best regards, >> Zhenlei [-- Attachment #2 --] <html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Mar 13, 2024, at 9:09 PM, Zhenlei Huang <<a href="mailto:zlei@FreeBSD.org" class="">zlei@FreeBSD.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta charset="UTF-8" class=""><br class="Apple-interchange-newline"><br class="" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><blockquote type="cite" class="" style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;"><div class="">On Mar 13, 2024, at 12:19 PM, Zhenlei Huang <<a href="mailto:zlei@FreeBSD.org" class="">zlei@FreeBSD.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br class="Apple-interchange-newline"><br class=""><blockquote type="cite" class=""><div class="">On Mar 13, 2024, at 4:36 AM, Benoit Chesneau <<a href="mailto:benoitc@enki-multimedia.eu" class="">benoitc@enki-multimedia.eu</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="" style="font-family: Arial, sans-serif; font-size: 14px;">On latest freebsd 14.0 release , I am trying to join an ipv4 address over an IPV6 local link using the following command :</div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;"><br class=""></div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;"><span class="">`route add -net 10.200.1.1/32 -inet6 fe80::9439:36ff:fef0:7cbd%vlan200`</span><br class=""></div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;"><span class=""><br class=""></span></div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;"><span class="">But I get the following error:</span></div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;"><span class=""><br class=""></span></div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;">```</div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;"><span class="">route add -net 10.200.1.1/32 -inet6 fe80::9439:36ff:fef0:7cbd%vlan200</span><br class=""><span class="">add net 10.200.1.1: gateway fe80::9439:36ff:fef0:7cbd%vlan200 fib 0: Invalid argument</span><br class=""></div></div></blockquote><div class=""><br class=""></div><div class="">I guess that is regression by transition of route(8) to NETLINK.</div></div></div></blockquote><blockquote type="cite" class="" style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;"><div class=""><div class="" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div class=""><br class=""></div><div class="">Can you please build sbin/route from source without NETLINK and give it another try ?</div><div class=""><br class=""></div><div class="">```</div><div class=""># cd /usr/src/sbin/route</div><div class=""># setenv MK_NETLINK_SUPPORT no</div><div class=""># make</div><div class="">```</div></div></div></blockquote><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br class=""></div><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">Hi Benoit,</div><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br class=""></div><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">I managed to repeat this bug.</div><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br class=""></div><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><div class="">As a workaround, add an IPv4 address to the outgoing interface (vlan200 in your case),</div><div class="">```</div><div class=""># ifconfig vlan200 inet x.x.x.x/y</div><div class="">```</div><div class=""><br class=""></div><div class="">then retry adding the route</div><div class="">```</div><div class=""># route add -net 10.200.1.1/32 -inet6 fe80::9439:36ff:fef0:7cbd%vlan200</div><div class="">```</div><div class=""><br class=""></div></div></div></blockquote><div><br class=""></div><div>Hi Alexander,</div><div><br class=""></div><div>I think the logic of adding route from NETLINK routine is not complete.</div><div></div><div><br class=""></div><div><div>After bumping netlink debug level,</div><div>```</div><div># sysctl net.netlink.debug.nl_route_debug_level=9</div><div>```</div><div><br class=""></div><div>I got these from kernel message:</div><div>```</div><div># dmesg | tail</div><div>...</div><div>[nl_route] finalize_nhop: Unable to determine ifa, skipping</div><div>[nl_route] rtnl_handle_newroute: Error creating nexthop</div><div>...</div><div>```</div><div><br class=""></div><div>From <a href="https://cgit.freebsd.org/src/tree/sys/netlink/route/rt.c?h=releng/14.0#n753" class="">https://cgit.freebsd.org/src/tree/sys/netlink/route/rt.c?h=releng/14.0#n753</a>; :</div><div><br class=""></div><div>```</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>/* Both nh_ifp and gateway are set */</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>if (nh->nh_ifa == NULL) {</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>const struct sockaddr *gw_sa = &nh->gw_sa;</div><div><br class=""></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>if (gw_sa->sa_family != dst->sa_family) {</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>/*</div><div><span class="Apple-tab-span" style="white-space:pre"> </span> * Use dst as the target for determining the default</div><div><span class="Apple-tab-span" style="white-space:pre"> </span> * preferred ifa IF</div><div><span class="Apple-tab-span" style="white-space:pre"> </span> * 1) the gateway is link-level (e.g. direct route)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span> * 2) the gateway family is different (e.g. IPv4 over IPv6).</div><div><span class="Apple-tab-span" style="white-space:pre"> </span> */</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>gw_sa = dst;</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div><br class=""></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>struct ifaddr *ifa = ifaof_ifpforaddr(gw_sa, nh->nh_ifp);</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>if (ifa == NULL) {</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>NL_LOG(LOG_DEBUG, "Unable to determine ifa, skipping");</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>*perror = EINVAL;</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>return (NULL);</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>nhop_set_src(nh, ifa);</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div>```</div><div>If the outgoing interface `<span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">nh->nh_ifp` does not have any IPv4 addresses, </span>we should try ifaddrs on other interfaces,</div><div>and have 0.0.0.0 as the last sort ( in case the net stack is IPv6 only, i.e., no IPv4 addresses available )</div><div><br class=""></div></div><br class=""><blockquote type="cite" class=""><div class=""><blockquote type="cite" class="" style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;"><div class=""><div class="" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br class=""><blockquote type="cite" class=""><div class=""><div class="" style="font-family: Arial, sans-serif; font-size: 14px;">```</div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;"><br class=""></div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;">ping6 to `<span class="" style="text-decoration: none; background-color: rgb(255, 255, 255); display: inline !important;">fe80::9439:36ff:fef0:7cbd%vlan200` works. What I am doing wrong? Configuration of the interface is the following:</span></div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;"><span class="" style="text-decoration: none; background-color: rgb(255, 255, 255); display: inline !important;"><br class=""></span></div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;">```</div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;"><span class="">vlan200: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500</span><div class=""><span class=""> options=4600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6,MEXTPG></span></div><div class=""><span class=""> ether fe:d9:dd:2d:35:c5</span></div><div class=""><span class=""> inet6 fe80::fcd9:ddff:fe2d:35c5%vlan200 prefixlen 64 scopeid 0x5</span></div><div class=""><span class=""> groups: vlan</span></div><div class=""><span class=""> vlan: 200 vlanproto: 802.1q vlanpcp: 0 parent interface: iavf0</span></div><div class=""><span class=""> media: Ethernet autoselect (10Gbase-SR <full-duplex>)</span></div><div class=""><span class=""> status: active</span></div><span class=""> nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL</span><br class=""></div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;">```</div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;"><span class=""><br class=""></span></div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;"><span class=""><br class=""></span></div><div class="" style="font-family: Arial, sans-serif; font-size: 14px;"><br class=""></div><div class="protonmail_signature_block" style="font-family: Arial, sans-serif; font-size: 14px;"><div class="protonmail_signature_block-user"><div class="" style="font-style: normal; font-weight: normal; letter-spacing: normal; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Helvetica; font-size: 12px;">Benoît </div></div></div></div></blockquote></div><br class="" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div class="" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div class="">Best regards,</div><div class="">Zhenlei</div></div></div></blockquote></div></blockquote></div><br class=""><div class=""> <div><br class=""></div> </div> <br class=""></body></html>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BE4D70C1-885D-4C6B-AE8C-AC62BD450F8A>