From owner-freebsd-net@freebsd.org Mon Jul 6 18:19:44 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id ACEBD36D8B9 for ; Mon, 6 Jul 2020 18:19:44 +0000 (UTC) (envelope-from zeising+freebsd@daemonic.se) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4B0v403P7Lz4Mv1 for ; Mon, 6 Jul 2020 18:19:44 +0000 (UTC) (envelope-from zeising+freebsd@daemonic.se) Received: by mailman.nyi.freebsd.org (Postfix) id 7426636D657; Mon, 6 Jul 2020 18:19:44 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 73EA936D8B8 for ; Mon, 6 Jul 2020 18:19:44 +0000 (UTC) (envelope-from zeising+freebsd@daemonic.se) Received: from mail.daemonic.se (mail.daemonic.se [IPv6:2607:f740:d:20::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4B0v3z2JQLz4MqF for ; Mon, 6 Jul 2020 18:19:42 +0000 (UTC) (envelope-from zeising+freebsd@daemonic.se) Received: from cid.daemonic.se (localhost [IPv6:::1]) by mail.daemonic.se (Postfix) with ESMTP id 4B0v3q05Dbz3mWM for ; Mon, 6 Jul 2020 18:19:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=daemonic.se; h= content-transfer-encoding:content-language:content-type :content-type:mime-version:user-agent:date:date:message-id :subject:subject:from:from:received:received; s=20151023; t= 1594059574; bh=oiXeaPjmPBVYHo+K3Dr1m3BJlt+/0lVqGLWhUPcjEcs=; b=n hd8QWN7rNq29b2XmxOLktU5720poZXZNseRE0JpZLVXL4eqvBMMXnr+ySGxbdLt7 6gAl6tu4Miy1cKyH0X7fxLiVp5FOQMT6yqaIgAynN2DVxDXyJz4fEKijGHbEpdrF zixhDj8zExsC46imtmAtk04A1PCx0zHtJFCcdOK4HM= X-Virus-Scanned: amavisd-new at daemonic.se Received: from mail.daemonic.se ([127.0.0.1]) (using TLS with cipher ECDHE-RSA-AES128-GCM-SHA256) by cid.daemonic.se (mailscanner.daemonic.se [127.0.0.1]) (amavisd-new, port 10587) with ESMTPS id iwMNXEC_Sn1Z for ; Mon, 6 Jul 2020 18:19:34 +0000 (UTC) Received: from garnet.daemonic.se (unknown [IPv6:2001:470:dca9:201:e911:95b5:705c:34b7]) by mail.daemonic.se (Postfix) with ESMTPSA id 4B0v3p1Mpvz3mMv for ; Mon, 6 Jul 2020 18:19:34 +0000 (UTC) To: net@FreeBSD.org From: Niclas Zeising Subject: ndp and routers with link-local addresses Message-ID: Date: Mon, 6 Jul 2020 20:19:32 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4B0v3z2JQLz4MqF X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=daemonic.se header.s=20151023 header.b=n hd8QWN; dmarc=pass (policy=none) header.from=daemonic.se; spf=pass (mx1.freebsd.org: domain of zeising@daemonic.se designates 2607:f740:d:20::25 as permitted sender) smtp.mailfrom=zeising@daemonic.se X-Spamd-Result: default: False [-3.73 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[daemonic.se:s=20151023]; NEURAL_HAM_MEDIUM(-1.01)[-1.014]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[net@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.02)[-1.021]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[daemonic.se:+]; DMARC_POLICY_ALLOW(-0.50)[daemonic.se,none]; NEURAL_HAM_SHORT(-0.70)[-0.696]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36236, ipnet:2607:f740:d::/48, country:US]; TAGGED_FROM(0.00)[freebsd]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jul 2020 18:19:44 -0000 Hi! It should be possible to have routing interfaces on client facing networks with only link-local addresses if you use SLAAC (or specify the default route manually). SLAAC by default uses the link-local address in advertisements, and clients on the network uses the router link-local address as default gateway. However, if the interface on the router facing the client network only has a link-local (and no global unicast) address, NDP neighbor discovery breaks. I have a couple of issues: First off, NDP from the router to the client breaks completely, unless the client starts the conversation. Then the router can use existing NDP table entries, as long as they are valid. Secondly, source address selection on the router side for NDP is strange. Instead of using the source address on the interface directly connected to the client, the router uses a global unicast address, if any is assigned to any interface. This confuses the client. All this is tested on FreeBSD 12.1 (both router and client). I also swapped out the router for a Linux install, using Debian 10.4.0. All the cases below works out of the box on Linux. quick setup using bhyve: router: two interfaces, connect one interface to a bridge on the host client: one interface, connected to the same bridge as the router on the router (vtnet1 is connected to the bridge above) # ifconfig vtnet1 inet6 fe80::1/64 # route -6 add -net 2001:6b8::/64 -iface vtnet1 # sysctl net.inet6.icmp6.nd6_onlink_ns_rfc4861=1 # sysctl net.inet6.ip6.forwarding=1 net.inet6.icmp6.nd6_onlink_ns_rfc4861=1 is needed, otherwise NDP won't work at all. you can then add any address to vtnet1 to test the source address selection, as long as it's not in 2001:6b8::/64 (for instance, use 2001:6b8:1::1/64) on the client (vtnet0 is connected to the bridge above) # ifconfig vtnet0 inet6 2001:6b8::2/64 # route -6 add default fe80::1%vtnet0 dump the traffic on the bridge between the machines On the router, try to ping the client (2001:6b8::2) and see what happens. This does not work. On the client, ping any ipv6 address not in 2001:6b8::/64, and watch NDP. (we don't care about the icmp echo requests, the ping is just to do the NDP stuff). NDP works, using link-local address on the router. Now on the router, ping the client again. This works if the entry is still in the NDP table on the router, which was added in the previous step. clear ndp: ndp -c try again on the router, now it's failing again. Add a global unicast address to the router> # ifconfig vtnet0 inet6 2001:6b8:1::1/64 Ping the client from the router, still does not work ping anything from the client and see which source the NDP packets are using. Linux uses the link-local address of vtnet1 on the router, FreeBSD uses the global unicast address of vtnet0 for packets from the router. This should add things to the NDP table on both router and client. Ping the client on the router again, now it works, for a while, until the router tries to refresh its NDP table, then it stops working. setting net.inet6.icmp6.nd6_onlink_ns_rfc4861=1 on the client and it keeps working. More details: With this setup, NDP works in one direction. I can, from the client, get NDP working and get the ethernet address of the interface with fe80::1 on the router (by trying to ping6 something, for instance). When this is one, the router also have the client in it's ndp neighbor table. Howerver, the opposite does not work. I can't from the router get the information of the client. When I try, from the router, to ping6 the client, all I get is ping6: sendmsg: No buffer space available and I don't see any NDP packets going out on the line. If the entry for the client is already in the router NDP list, then it works. To me, it looks like the router gets confused and for some reason won't do any neighbor discovery on its own, but it works if it is done already. I also found that the source address selection for NDP is a bit strange. It looks like FreeBSD prefers to use a global unicast address as source for ndp neighbor solicitations. If I add an global unicast address to any other interface on the router, that is used as the source, instead of the link-local address on the interface connected to the client. This sometimes confuses the FreeBSD client, at least unless net.inet6.icmp6.nd6_onlink_ns_rfc4861 is set to 1. With Linux this works. I swapped out the router for a Linux one, with the same setup things work out of the box. Linux is also using the closest link-local address as a source for NDP NS. I don't know if the FreeBSD behavior is expected behavior, or if there are bugs, but it is causing interoperability problems. All this are with FreeBSD 12.1. For the Linux router I used Debian 10.4.0. Regards -- Niclas