From nobody Wed Oct 20 18:22:37 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id BC3AB18027EF for ; Wed, 20 Oct 2021 18:27:26 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HZJxT6L7Zz3wDJ for ; Wed, 20 Oct 2021 18:27:25 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.org [185.220.148.12]) by uucp.dinoex.org (8.17.1/8.17.1) with ESMTPS id 19KIR4HF051193 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Wed, 20 Oct 2021 20:27:04 +0200 (CEST) (envelope-from pmc@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: X-Authentication-Warning: uucp.dinoex.sub.de: Host uucp.dinoex.org [185.220.148.12] claimed to be uucp.dinoex.sub.de Received: (from uucp@localhost) by uucp.dinoex.sub.de (8.17.1/8.17.1/Submit) with UUCP id 19KIR4Nh051192 for freebsd-stable@freebsd.org; Wed, 20 Oct 2021 20:27:04 +0200 (CEST) (envelope-from pmc@citylink.dinoex.sub.org) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.16.1/8.16.1) with ESMTP id 19KINe13081588 for ; Wed, 20 Oct 2021 20:23:40 +0200 (CEST) (envelope-from peter@gate.intra.daemon.contact) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by gate.intra.daemon.contact (8.16.1/8.16.1) with ESMTPS id 19KIMbUD081444 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Wed, 20 Oct 2021 20:22:37 +0200 (CEST) (envelope-from peter@gate.intra.daemon.contact) Received: (from peter@localhost) by gate.intra.daemon.contact (8.16.1/8.16.1/Submit) id 19KIMbOm081443 for freebsd-stable@freebsd.org; Wed, 20 Oct 2021 20:22:37 +0200 (CEST) (envelope-from peter) Date: Wed, 20 Oct 2021 20:22:37 +0200 From: Peter To: freebsd-stable@freebsd.org Subject: IPv6 inconsistent local routing Message-ID: List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 185.220.148.12; Sender-helo: uucp.dinoex.sub.de;) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.4 (uucp.dinoex.org [185.220.148.12]); Wed, 20 Oct 2021 20:27:07 +0200 (CEST) X-Rspamd-Queue-Id: 4HZJxT6L7Zz3wDJ X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of pmc@citylink.dinoex.sub.org has no SPF policy when checking 2a0b:f840::12) smtp.mailfrom=pmc@citylink.dinoex.sub.org X-Spamd-Result: default: False [-2.10 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_NA(0.00)[sub.org]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:205376, ipnet:2a0b:f840::/32, country:DE]; RCVD_TLS_LAST(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Hello all, with IPv4 the handling of local traffic was simple: all local traffic would go in and out of the 'lo0' interface, no matter which ip-address was used or on which interface that address would be placed. This gets also obvious from the routing table: root@xxxx:~ # ifconfig vtnet0: flags=8863 metric 0 mtu 1500 inet 192.168.97.15 netmask 0xffffffe0 broadcast 192.168.97.31 inet6 fd00::1 prefixlen 64 root@xxxx:~ # netstat -rn Routing tables Internet: Destination Gateway Flags Netif Expire 127.0.0.1 link#2 UH lo0 192.168.97.0/27 link#1 U vtnet0 192.168.97.15 link#1 UHS lo0 <<<<<< While the network /27 appears with 'vtnet0', the local address itself is at 'lo0'. Practically it looks like this: root@xxxx:~ # ping 192.168.97.15 ipfw: 1 Accept ICMP:8.0 127.0.0.1 192.168.97.15 out via lo0 ipfw: 1 Accept ICMP:8.0 127.0.0.1 192.168.97.15 in via lo0 ipfw: 1 Accept ICMP:0.0 192.168.97.15 127.0.0.1 out via lo0 ipfw: 1 Accept ICMP:0.0 192.168.97.15 127.0.0.1 in via lo0 root@xxxx:~ # telnet 192.168.97.15 7777 ipfw: 1 Accept TCP 192.168.97.15:52401 192.168.97.15:7777 out via lo0 ipfw: 1 Accept TCP 192.168.97.15:52401 192.168.97.15:7777 in via lo0 ipfw: 1 Accept TCP 192.168.97.15:7777 192.168.97.15:52401 out via lo0 ipfw: 1 Accept TCP 192.168.97.15:7777 192.168.97.15:52401 in via lo0 All traffic goes thru 'lo0' (and there we could filter it if e.g. we want to filter between non-vimage jails). But now look at IPv6: root@xxxx:~ # ping fd00::1 ipfw: 1 Accept ICMPv6:128.0 [fd00::1] [fd00::1] out via lo0 ipfw: 1 Accept ICMPv6:128.0 [fd00::1] [fd00::1] in via lo0 ipfw: 1 Accept ICMPv6:129.0 [fd00::1] [fd00::1] out via lo0 ipfw: 1 Accept ICMPv6:129.0 [fd00::1] [fd00::1] in via lo0 root@xxxx:~ # telnet fd00::1 7777 ipfw: 1 Accept TCP [fd00::1]:53821 [fd00::1]:7777 out via lo0 ipfw: 1 Accept TCP [fd00::1]:53821 [fd00::1]:7777 in via vtnet0 <<<<< ipfw: 1 Accept TCP [fd00::1]:7777 [fd00::1]:53821 out via lo0 ipfw: 1 Accept TCP [fd00::1]:7777 [fd00::1]:53821 in via lo0 Upsala! What's the 'vtnet0' doing there?? But that's not yet the full story. This is 13-stable or RELEASE-13.0. And now RELEASE 12.2 (or 12.3-PRERELEASE): root@yyyy:~ # ping6 fd00::1 ipfw: 1 Accept ICMPv6:128.0 [fd00::1] [fd00::1] out via lo0 ipfw: 1 Accept ICMPv6:128.0 [fd00::1] [fd00::1] in via vtnet0 ipfw: 1 Accept ICMPv6:129.0 [fd00::1] [fd00::1] out via lo0 ipfw: 1 Accept ICMPv6:129.0 [fd00::1] [fd00::1] in via vtnet0 root@yyyy:~ # telnet fd00::1 7777 ipfw: 1 Accept TCP [fd00::1]:60375 [fd00::1]:7777 out via lo0 ipfw: 1 Accept TCP [fd00::1]:60375 [fd00::1]:7777 in via vtnet0 ipfw: 1 Accept TCP [fd00::1]:7777 [fd00::1]:60375 out via lo0 ipfw: 1 Accept TCP [fd00::1]:7777 [fd00::1]:60375 in via vtnet0 Urgs. Lets conclude: the behaviour is * inkonsistent between IPv4 and IPv6 * inkonsistent between incoming and outgoing * inkonsistent between originate and answer flow * inkonsistent between protocols (ICMP vs. TCP) * inkonsistent between releases And now I have a problem. I'm trying to enhance my graphical frontend to also work with IPv6: https://forums.freebsd.org/threads/nice-or-ugly.81298/post-522355 But how should I code that, when the behaviour is different with every usecase? And whom might I ask how it would look if it finally settles? (The ipfw mailinglist seems mostly empty.) You might say, it does not usually make sense to route traffic local-to- local over an IPadress of an outbound interface. But actually this is common practice in e.g. failover scenarios, where you don't know in the app configs if your address is currently local or not - you just know it carries the service. It was mentioned that one should usually ignore local traffic in the firewall rules. But that is not the point: as these packets are labelled with the external interface, they will go thru the ruleset for the external interface anyway. And there they will be dropped, because nobody there knows about them. This has also an effect on the tcp dyn keepalive feature. (net.inet.ip.fw.dyn_keepalive) These keepalives usually go thru lo0 incoming. But, with IPv6 on Rel. 12 they go thru the external interface, incoming. Then on Rel. 13 they go thru lo0 again. (And I didn't try Rel. 14 ) Cheerio, PMc