From nobody Wed Oct 20 22:06:00 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6242A1817654 for ; Wed, 20 Oct 2021 22:06:14 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:13b:39f::9f:25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HZPny1Hbsz3w8g for ; Wed, 20 Oct 2021 22:06:14 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id A53178D4A210; Wed, 20 Oct 2021 22:06:05 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 7E298E708BC; Wed, 20 Oct 2021 22:06:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id Oh7JTXwo4k3Q; Wed, 20 Oct 2021 22:06:01 +0000 (UTC) Received: from [169.254.86.56] (unknown [IPv6:fde9:577b:c1a9:4902:4805:d989:df5d:8806]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 63B03E708BA; Wed, 20 Oct 2021 22:06:01 +0000 (UTC) From: "Bjoern A. Zeeb" To: Peter Cc: freebsd-stable@freebsd.org Subject: Re: IPv6 inconsistent local routing Date: Wed, 20 Oct 2021 22:06:00 +0000 X-Mailer: MailMate (2.0BETAr6151) Message-ID: In-Reply-To: References: List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4HZPny1Hbsz3w8g X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On 20 Oct 2021, at 18:22, Peter wrote: > Hello all, > > with IPv4 the handling of local traffic was simple: all local traffic > would go in and out of the 'lo0' interface, no matter which ip-address > was used or on which interface that address would be placed. This gets > also obvious from the routing table: > > > root@xxxx:~ # ifconfig > vtnet0: flags=8863 metric 0 > mtu 1500 > inet 192.168.97.15 netmask 0xffffffe0 broadcast 192.168.97.31 > inet6 fd00::1 prefixlen 64 > > root@xxxx:~ # netstat -rn > Routing tables > > Internet: > Destination Gateway Flags Netif Expire > 127.0.0.1 link#2 UH lo0 > 192.168.97.0/27 link#1 U vtnet0 > 192.168.97.15 link#1 UHS lo0 <<<<<< > > While the network /27 appears with 'vtnet0', the local > address itself is at 'lo0'. > > Practically it looks like this: > > > root@xxxx:~ # ping 192.168.97.15 > ipfw: 1 Accept ICMP:8.0 127.0.0.1 192.168.97.15 out via lo0 > ipfw: 1 Accept ICMP:8.0 127.0.0.1 192.168.97.15 in via lo0 > ipfw: 1 Accept ICMP:0.0 192.168.97.15 127.0.0.1 out via lo0 > ipfw: 1 Accept ICMP:0.0 192.168.97.15 127.0.0.1 in via lo0 > > root@xxxx:~ # telnet 192.168.97.15 7777 > ipfw: 1 Accept TCP 192.168.97.15:52401 192.168.97.15:7777 out via lo0 > ipfw: 1 Accept TCP 192.168.97.15:52401 192.168.97.15:7777 in via lo0 > ipfw: 1 Accept TCP 192.168.97.15:7777 192.168.97.15:52401 out via lo0 > ipfw: 1 Accept TCP 192.168.97.15:7777 192.168.97.15:52401 in via lo0 > > All traffic goes thru 'lo0' (and there we could filter it if e.g. we > want to filter between non-vimage jails). > > But now look at IPv6: > > > root@xxxx:~ # ping fd00::1 > ipfw: 1 Accept ICMPv6:128.0 [fd00::1] [fd00::1] out via lo0 > ipfw: 1 Accept ICMPv6:128.0 [fd00::1] [fd00::1] in via lo0 > ipfw: 1 Accept ICMPv6:129.0 [fd00::1] [fd00::1] out via lo0 > ipfw: 1 Accept ICMPv6:129.0 [fd00::1] [fd00::1] in via lo0 > > root@xxxx:~ # telnet fd00::1 7777 > ipfw: 1 Accept TCP [fd00::1]:53821 [fd00::1]:7777 out via lo0 > ipfw: 1 Accept TCP [fd00::1]:53821 [fd00::1]:7777 in via vtnet0 <<<<< > ipfw: 1 Accept TCP [fd00::1]:7777 [fd00::1]:53821 out via lo0 > ipfw: 1 Accept TCP [fd00::1]:7777 [fd00::1]:53821 in via lo0 > > > Upsala! What's the 'vtnet0' doing there?? > > But that's not yet the full story. This is 13-stable or RELEASE-13.0. > And now RELEASE 12.2 (or 12.3-PRERELEASE): > > > root@yyyy:~ # ping6 fd00::1 > ipfw: 1 Accept ICMPv6:128.0 [fd00::1] [fd00::1] out via lo0 > ipfw: 1 Accept ICMPv6:128.0 [fd00::1] [fd00::1] in via vtnet0 > ipfw: 1 Accept ICMPv6:129.0 [fd00::1] [fd00::1] out via lo0 > ipfw: 1 Accept ICMPv6:129.0 [fd00::1] [fd00::1] in via vtnet0 > > root@yyyy:~ # telnet fd00::1 7777 > ipfw: 1 Accept TCP [fd00::1]:60375 [fd00::1]:7777 out via lo0 > ipfw: 1 Accept TCP [fd00::1]:60375 [fd00::1]:7777 in via vtnet0 > ipfw: 1 Accept TCP [fd00::1]:7777 [fd00::1]:60375 out via lo0 > ipfw: 1 Accept TCP [fd00::1]:7777 [fd00::1]:60375 in via vtnet0 > > > Urgs. > > Lets conclude: the behaviour is > * inkonsistent between IPv4 and IPv6 > * inkonsistent between incoming and outgoing > * inkonsistent between originate and answer flow > * inkonsistent between protocols (ICMP vs. TCP) > * inkonsistent between releases > > > And now I have a problem. I'm trying to enhance my graphical frontend > to also work with IPv6: > https://forums.freebsd.org/threads/nice-or-ugly.81298/post-522355 > > But how should I code that, when the behaviour is different with every > usecase? > And whom might I ask how it would look if it finally settles? (The > ipfw mailinglist seems mostly empty.) > > > > You might say, it does not usually make sense to route traffic > local-to- > local over an IPadress of an outbound interface. But actually this > is common practice in e.g. failover scenarios, where you don't know > in the app configs if your address is currently local or not - you > just > know it carries the service. There used to be: $ sysctl -d net.link.ether.inet.useloopback net.inet6.icmp6.nd6_useloopback net.link.ether.inet.useloopback: Use the loopback interface for local traffic net.inet6.icmp6.nd6_useloopback: which defined this bahviour. These were removed in/after the FreeBSD 10 times I think? I’d start looking from there. > It was mentioned that one should usually ignore local traffic in the > firewall rules. But that is not the point: as these packets are > labelled > with the external interface, they will go thru the ruleset for the > external interface anyway. And there they will be dropped, because > nobody there knows about them. > > > This has also an effect on the tcp dyn keepalive feature. > (net.inet.ip.fw.dyn_keepalive) These keepalives usually go > thru lo0 incoming. But, with IPv6 on Rel. 12 they go thru the external > interface, incoming. Then on Rel. 13 they go thru lo0 again. > (And I didn't try Rel. 14 ) > > > Cheerio, > PMc