From nobody Tue Aug 5 14:09:46 2025 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4bxFhF2HPVz63HDD for ; Tue, 05 Aug 2025 14:10:53 +0000 (UTC) (envelope-from felix@ffetc.net) Received: from lipwig.ffetc.net (lipwig.ffetc.net [157.230.104.207]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4bxFhD3J9Kz3HJZ for ; Tue, 05 Aug 2025 14:10:52 +0000 (UTC) (envelope-from felix@ffetc.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ffetc.net header.s=default header.b=CP00eO97; spf=pass (mx1.freebsd.org: domain of felix@ffetc.net designates 157.230.104.207 as permitted sender) smtp.mailfrom=felix@ffetc.net; dmarc=pass (policy=reject) header.from=ffetc.net Received: from [172.16.1.40] (119-15-97-122.static.vdsl.infinite.net.au [119.15.97.122] (may be forged)) (authenticated bits=0) by lipwig.ffetc.net (8.18.1/8.17.2) with ESMTPSA id 575E9l6B822977 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT) for ; Tue, 5 Aug 2025 14:09:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffetc.net; s=default; t=1754402991; bh=lV1ZZA6HE+x4cY9h5jgyQVDaBQ9jrJLSyz7Uq9FHrjo=; h=Date:To:From:Subject:From; b=CP00eO97EDD8UTyAyY0nxC4a000GC6LWR7q399HP8Fh0M1NvOzmzfA+CdZLGLogyT JVmICwUHy4dudyUNZtFNu9ZWjSnR+zUIXSbjZF6zSa3PovOSqnC5nfoPy9PjTMsqQV L/T04uX9BmY9wT82tbmiVSi7/0bZ7Ozb0AdLktrA= Message-ID: Date: Wed, 6 Aug 2025 00:09:46 +1000 List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-AU To: freebsd-net@freebsd.org From: Felix Subject: Source address selection failure when using RFC5549-style (v4-via-v6) routes Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-2.90 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[ffetc.net,reject]; R_SPF_ALLOW(-0.20)[+mx:c]; ONCE_RECEIVED(0.20)[]; R_DKIM_ALLOW(-0.20)[ffetc.net:s=default]; NEURAL_HAM_SHORT(-0.10)[-0.100]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; ASN(0.00)[asn:14061, ipnet:157.230.96.0/20, country:US]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_ONE(0.00)[1]; RCPT_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; FROM_HAS_DN(0.00)[]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[ffetc.net:+] X-Rspamd-Queue-Id: 4bxFhD3J9Kz3HJZ X-Spamd-Bar: -- I've been experimenting a bit with routing IPv4 via IPv6 next-hops, something that FreeBSD has supported since 13.1 [https://reviews.freebsd.org/D30398]. My understanding (note: I'm not a professional network engineer) is that this is normally used with routing protocols like BGP (RFC5549, often called "extended nexthop") or Babel (RFC9229), where it's extremely convenient - the links between your routers only need link-local v6 addresses to forward both v4 and v6 traffic, reducing configuration and saving IP addresses. Of course, each router should have /one/ routable v4 and v6 address so that it can return ICMP messages and so on, which one would typically assign on the loopback interface. For experimentation, you can create these routes directly, without running one of these protocols, like so: # route add -net 203.0.113.2/32 -inet6 fe80::2%bge2 add net 203.0.113.2: gateway fe80::2%bge2 # netstat -r4n Routing tables Internet: Destination Gateway Flags Netif Expire ... 203.0.113.2 fe80::2%bge2 UGHS bge2 ... What I've discovered: trying to /originate/ a connection from a machine with such a route doesn't seem to work properly. Any attempts to open a TCP connection (e.g. ssh) will fail with EHOSTUNREACH: # truss ssh 203.0.113.2 ... connect(3,{ AF_INET 203.0.113.2:22 },16) ERR#65 'No route to host' ... ssh: connect to host 203.0.113.2 port 22: No route to host ... process exit, rval = 255 When choosing a source address for an outgoing connection, FreeBSD does a route lookup (which succeeds in this case), and as a comment in sys/netinet/in_pcb.c helpfully tells us: /* * If the outgoing interface on the route found is not * a loopback interface, use the address from that interface. */ Because the outgoing interface doesn't /have/ a v4 address, a consistency check at the end of the function returns EHOSTUNREACH. Linux also supports these v4-via-v6 routes, so I can compare how it behaves. Firstly, it will use the address specified in the `src` attribute of the route if there is one (FreeBSD, AFAIK, has no such notion). Otherwise, it follows some process to pick one of the machine's addresses to use as a source address, and will (if necessary, like in this scenario) pick an address from a different interface. I'm not sure what the right thing for FreeBSD to do in this circumstance is. What do you think? I also haven't tested whether this same issue affects the generation of ICMP responses (e.g. TTL expired, packet too big). If it does, that seems like much more of a concern for using FreeBSD on real routers. Thanks, - Felix