From nobody Wed Apr 24 02:12:15 2024 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VPMv76WSNz5JXKF for ; Wed, 24 Apr 2024 02:12:19 +0000 (UTC) (envelope-from gshapiro@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VPMv74sh6z49rD for ; Wed, 24 Apr 2024 02:12:19 +0000 (UTC) (envelope-from gshapiro@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713924739; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=ktFLd2624I7QLofJ3VEu/avLu9DzTmrguC5V/6H/VZ0=; b=Y7hTwm1DAJxeUnVZv33xvxv8SJ4OAXQ/YtyaPmchzE1C8fK6MHd1NarO8i+82v7GnmrR5z PeQdSCigcVFciN3XdnBSz0et4oEI5SlLBKCw6dUaQPKznwlgX3+3REs5f9Rai4ZhjlgoMg P0Anlqt7ds8WBvgwi1X19W3aS6SYgXFw94GCYZRDYe77OQ6uW1gD7ZWyAZ5F7HFPjuCpsb LBkGrU+BeJaJ80ljeAPE06H5Md6+doiL4J0piyo2dLZs0JxFxWkM72VREGiLO+olKZXHQY 25P04l1Ga2LuFdRcplgtUJsUiJ93QScB5fwupQ31v3hwT2pFNod6GLtV6BBW2g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1713924739; a=rsa-sha256; cv=none; b=abCLBoYTQAbg0kla+k/ryAQ7YMEdCZoqQqD+MILryMHqAENcjdcKPoDUdk1vdxu8ze64PF 67o49AXJ2SbS2qui93CeLRBos0UShfR9mMO9bvOanDM+ZPsdPNLdeOfZ/GKRzne6mH7uIe nXjtrNW5vxwtrvDm+W/jSoe2JpcWobx8f5NwRk3LDOkCOSY2kY9BVYWnJlaMTAnj438ibk 3jglNfGyaHOJ4q5vPqqnepsDPslSC8/UTUAbu806yNN9NAiqxkrfBLX8m9UtWkBbc5lwAt lFJvrgNxFZQ1n/cp8xe66RfZ+S8osktx/fgHu8dAnCaT5nslz4cCShUekfnVkg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713924739; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=ktFLd2624I7QLofJ3VEu/avLu9DzTmrguC5V/6H/VZ0=; b=OyH/LnjJb3xc8L2ZVps2WwDyMH5dbhszj+yPNgjDAZMDOhrc8GzMhi77QrtBwPptBXsRxN RZy/ArcyD4nDm06iEHS9bGvrypI4UUnuUGIqQymn+10nzOXclKERJTFogFTUC9FJvqm+7a rLQZWLK4DXlrC3tRVvycIMd681e2Nn0BZVBZ5LdrH6FGvJ0+BRMhhGxEa2VQ9prjWQvb4H ZvQ6XOUriymGe0bAj6HY6oj7pdkTDtB0v6r8wJLNzoeu2uORuwMRcZ7Hp+NfPNly1KIIPj RpHe91/RuQ+8ZtAdnCHfV3ibDnQHZ25XMW+roM+ZMlWCRJj3o0D3GlfXRzSlXg== Received: from thornystick.local (thornystick.gshapiro.net [IPv6:2a0a:280:2357:5506::2ee5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: gshapiro) by smtp.freebsd.org (Postfix) with ESMTPSA id 4VPMv64k2Jz15vl for ; Wed, 24 Apr 2024 02:12:18 +0000 (UTC) (envelope-from gshapiro@freebsd.org) Date: Tue, 23 Apr 2024 19:12:15 -0700 From: Gregory Shapiro To: freebsd-net@freebsd.org Subject: Source IPv4 address selection vs BGP IX connection Message-ID: List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Short version: Using FreeBSD as a BGP router has network issues caused by suboptimal default IPv4 source address selection when connected to Internet Exchanges (which are required to use IPs that aren't routable on the Internet). I was hoping to find more elegant workarounds or encourage FreeBSD to add source IPv4 selection akin to the existing IPv6 source address selection (no_prefer_iface and prefer_source). Long version: Unless I'm mistaken, today, there is no way to set the default IPv4 source address for connections like there is with IPv6 (using no_prefer_iface and prefer_source). It appears the default source IP is chosen based on IP address of the outbound interface for the packet. This presents a problem on FreeBSD systems acting as BGP routers that have connections to Internet exchanges (IX). One of the rules of IX IP addresses is that they are must not be routable on the Internet. As a simple example, a system with two Ethernet interfaces, one to the transit provider and one to an IX would look like this: vtnet0: flags=1008843 metric 0 mtu 1500 description: Uplink inet 193.148.250.141 netmask 0xffffff00 broadcast 193.148.250.255 vtnet1: flags=1008843 metric 0 mtu 1500 description: IX inet 185.1.147.211 netmask 0xffffff00 broadcast 185.1.147.255 Then if /etc/resolv.conf contains 8.8.8.8 and BGP selects a route for 8.8.8.0/24 over the IX, you end up with: # route -n get 8.8.8.8 route to: 8.8.8.8 destination: 8.8.8.0 mask: 255.255.255.0 gateway: 185.1.147.22 fib: 0 interface: vtnet1 flags: recvpipe sendpipe ssthresh rtt,msec mtu weight expire 0 0 0 0 1500 1 0 And DNS on the system doesn't work as all DNS requests go out with a source address of 185.1.147.211 (the IX endpoint) which isn't exported as an Internet route. While I can set a static route for 8.8.8.8 for this particular case, it would be messy to have to set up static routes for every possible local connection (other DNS servers, outbound SMTP for periodic/cron mail, etc.). I assume that there is a group of BGP enthusiasts using FreeBSD lurking on freebsd-net. What have you done to solve this problem? I'd also love to hear other tips for running BGP on FreeBSD.