From owner-freebsd-net@freebsd.org Tue Jan 24 01:31:29 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 94FCDCBDE4E for ; Tue, 24 Jan 2017 01:31:29 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org (unknown [IPv6:2602:304:b010:ef20::f2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gw.catspoiler.org", Issuer "gw.catspoiler.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7545662D for ; Tue, 24 Jan 2017 01:31:29 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.15.2/8.15.2) with ESMTP id v0O1VMcu005208 for ; Mon, 23 Jan 2017 17:31:26 -0800 (PST) (envelope-from truckman@FreeBSD.org) Message-Id: <201701240131.v0O1VMcu005208@gw.catspoiler.org> Date: Mon, 23 Jan 2017 17:31:22 -0800 (PST) From: Don Lewis Subject: inheriting fib from an interface To: freebsd-net@FreeBSD.org MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2017 01:31:29 -0000 Let's say that I have an application running on a server that is connected to the Internet via two different ISPs and is using IP addresses (ISP A:10.0.0.10 and ISP B:192.168.1.10) delegated by those two ISPs on it's two interfaces. Responses to requests sent to 10.0.0.10 should be sent via ISP A, and responses to requests sent to 192.168.1.10 should be ISB B. There are a couple of different ways that I can think of to do this: 1) Put the server behind another FreeBSD box that uses policy-based routing to forward the outbound packets to the desired ISP. My understanding is that this only works for packet forwarding and not for locally generated packets. 2) Set net.fibs=2, set separate default routes for the two fibs, modify the application to create and bind sockets to both IP addresses, and call setsockopt(..., SO_SETFIB, ...) on each. This is a bit of a headache because it requires maintaining source code changes for the application. Also the SO_SETFIB settings in the application need to be kept synchronized to the system configuration, which looks like it could be error-prone. Running two instances of the application under setfib might be undesirable. FreeBSD can also associate a fib with an interface. From the brief reading that I've done, it looks like this is only used to tag incoming packets with the fib of the interface that they are received on and thus influence the routing decisions made when forwarding them. It seems like it would be useful for a socket to inherit the fib of the matching interface when bind() is called on it. Since connect() may also do a bind, perhaps the fib should be inherited then as well. Also when a TCP socket listening on INADDR_ANY receives a connection request and returns a new socket via accept(), perhaps that socket should have its fib set as well. Thoughts?