From nobody Wed Aug 6 08:55:53 2025 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4bxkfT37trz63R0r for ; Wed, 06 Aug 2025 08:56:01 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4bxkfT2cbpz3Drg; Wed, 06 Aug 2025 08:56:01 +0000 (UTC) (envelope-from zlei@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1754470561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8byaZeivphReWawJo7eB/9S1x6LiTfQQt0QESKf0vZE=; b=GyrM7Vrk/78EDMCazqJZSannXiJhIe3BzZCSl5miiL162x8vvysLPR7068/V6lD7MRYBGk uat/qVS7cM8yq/d2Gj5EZsZwU7ix57WlJbw7mHnM1FYvsXNyDwDG6OgsvLCsEYyQ8kNxwR tgH6XfPxNRXRBR/No7kgQTAEX/JaL3TyOtsJ4WHzxVeS5T/c5oWY7oOES3L7vaaRhN/Og8 0DznqLd+vastWiQh7a9g+rOJP8lMi+cdnvhXqqmCmdsaWNCBvUTJrx+JezPNWRf5C9r7Dt sevYGHnRQL79P7sc+e6QVFuqrpBdTuuC09d6BYX9PddtRYBnnx/SKyzStA7aNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1754470561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8byaZeivphReWawJo7eB/9S1x6LiTfQQt0QESKf0vZE=; b=SjN3xWYLAh8bHz2aXSVtUuujJCXt8blpwkaaosb8hIcdKFrBmlDFRlDHJTuGPRZjR/nrjv AQTVbzEgyxwBx6QbGMMAmLj15fKl4M9HIzFEsTQ2hfZnSn2E67c3P7k+GWlSUbh9I+sM+8 ISupLO34s1qY0h0dXsv/BrnYmVXATKu5gAL0l3tEFfpoZq+5c4MHEkBV2Bgd1Q9pVIAciN 6chz5nm081K4/avkEv925kNUr3HFDjX2ATvqSjDdrdh22i/N3n5PvI8QRlkKW4En3OwihK I3xAOgFpv7nq0dTRMNmJtr8q1Tq0BJTlKJkc9egk6UyQNZGE8Ls/WGnAZs2zZg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1754470561; a=rsa-sha256; cv=none; b=ZVm8qd2xioU78r9l4U7nZd9XP0spzHyRIoBpDSOB0rAz27vg9H/hRWuH/MJBZJXl065fMN 9RuejZ+EtmOUwyCDJq/xhqLHQDDEnvCr/D4bjlns3SykY9QbOB56vWhLAZyTiu6CW9VWZR IPLwRTmnuQGOqczs+goDRIQwSAYDUAQ3C4Cd/svEuZnxF/SauT7IN+t/qEKi+o5GHFROFb iOheEV8+JpIyU5lF5mT3LpKFJ73frk4jmr8QUetW+HoQ9VDQ7BtVzoOA1q1hYBBqfwlrrK Be3JWqa+0TQkpfiMstLE7q6sfVX1MrWMknNgGMEy0NKR/Ao1cd+EdLgnnn9ALg== Received: from smtpclient.apple (ns1.oxydns.net [45.32.91.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: zlei/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4bxkfR6DBYz82Q; Wed, 06 Aug 2025 08:55:59 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Content-Type: text/plain; charset=us-ascii List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.10\)) Subject: Re: Source address selection failure when using RFC5549-style (v4-via-v6) routes From: Zhenlei Huang In-Reply-To: Date: Wed, 6 Aug 2025 16:55:53 +0800 Cc: FreeBSD Net , Lexi Winter Content-Transfer-Encoding: quoted-printable Message-Id: <552E3E74-7CD3-42C8-8BAC-7E3330B30DEF@FreeBSD.org> References: To: Felix X-Mailer: Apple Mail (2.3696.120.41.1.10) > On Aug 5, 2025, at 10:09 PM, Felix wrote: >=20 > I've been experimenting a bit with routing IPv4 via IPv6 next-hops, = something that FreeBSD has supported since 13.1 > [https://reviews.freebsd.org/D30398]. >=20 > My understanding (note: I'm not a professional network engineer) is = that this is normally used with routing protocols like BGP (RFC5549, = often called "extended nexthop") or Babel (RFC9229), where it's = extremely convenient - the links between your routers only need = link-local v6 addresses to forward both v4 and v6 traffic, reducing = configuration and saving IP addresses. Of course, each router should = have /one/ routable v4 and v6 address so that it can return ICMP = messages and so on, which one would typically assign on the loopback = interface. Yes. >=20 > For experimentation, you can create these routes directly, without = running one of these protocols, like so: >=20 > # route add -net 203.0.113.2/32 -inet6 fe80::2%bge2 > add net 203.0.113.2: gateway fe80::2%bge2 >=20 > # netstat -r4n > Routing tables >=20 > Internet: > Destination Gateway Flags Netif Expire > ... > 203.0.113.2 fe80::2%bge2 UGHS bge2 > ... >=20 > What I've discovered: trying to /originate/ a connection from a = machine with such a route doesn't seem to work properly. Any attempts to = open a TCP connection (e.g. ssh) will fail with EHOSTUNREACH: >=20 > # truss ssh 203.0.113.2 > ... > connect(3,{ AF_INET 203.0.113.2:22 },16) ERR#65 'No route to = host' > ... > ssh: connect to host 203.0.113.2 port 22: No route to host > ... > process exit, rval =3D 255 >=20 > When choosing a source address for an outgoing connection, FreeBSD = does a route lookup (which succeeds in this case), and as a comment in = sys/netinet/in_pcb.c helpfully tells us: >=20 > /* > * If the outgoing interface on the route found is not > * a loopback interface, use the address from that interface. > */ >=20 > Because the outgoing interface doesn't /have/ a v4 address, a = consistency check at the end of the function returns EHOSTUNREACH. This is a known limitation of current IPv4 over IPv6 nexthop. I have = ever asked on #net mailing list about the IPv4 source selection but no = good luck. So currently it is mandatory to have an IPv4 address assigned = to the outgoing interface, if originate a connection from the host. Well = certainly that restrictioni is not needed for forward packets. You can apply Lexi's workaround to overcome the issue. >=20 > Linux also supports these v4-via-v6 routes, so I can compare how it = behaves. Firstly, it will use the address specified in the `src` = attribute of the route if there is one (FreeBSD, AFAIK, has no such = notion). Otherwise, it follows some process to pick one of the machine's = addresses to use as a source address, and will (if necessary, like in = this scenario) pick an address from a different interface. >=20 > I'm not sure what the right thing for FreeBSD to do in this = circumstance is. What do you think? I think we should borrow some rules from IPv6 source address selection. = I have not tried Linux, but I think use a global unicast address from = loopback interface is acceptable, especially when the host is a router. >=20 > I also haven't tested whether this same issue affects the generation = of ICMP responses (e.g. TTL expired, packet too big). If it does, that = seems like much more of a concern for using FreeBSD on real routers. True. For IPv4 when generating the ICMP responses, IPv4 source address = selection is also involved. >=20 > Thanks, > - Felix >=20 Best regards, Zhenlei