From owner-freebsd-jail@freebsd.org Fri Jun 2 16:17:40 2017 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CCD28BFAB1C for ; Fri, 2 Jun 2017 16:17:40 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from gritton.org (gritton.org [199.192.165.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gritton.org", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id AEC15689DE for ; Fri, 2 Jun 2017 16:17:40 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from gritton.org (gritton.org [199.192.165.131]) by gritton.org (8.15.2/8.15.2) with ESMTP id v52G9OJ0085619; Fri, 2 Jun 2017 10:09:24 -0600 (MDT) (envelope-from jamie@freebsd.org) Received: (from www@localhost) by gritton.org (8.15.2/8.15.2/Submit) id v52G9ODp085617; Fri, 2 Jun 2017 10:09:24 -0600 (MDT) (envelope-from jamie@freebsd.org) X-Authentication-Warning: gritton.org: www set sender to jamie@freebsd.org using -f To: freebsd-jail@freebsd.org Subject: Re: setfib, jails and loopback interfaces X-PHP-Originating-Script: 0:rcube.php MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Fri, 02 Jun 2017 10:09:24 -0600 From: James Gritton Cc: =?UTF-8?Q?Marko_Cupa=C4=87?= In-Reply-To: <20170531103349.244f0fbf@efreet-freebsd.kappastar.com> References: <20170531103349.244f0fbf@efreet-freebsd.kappastar.com> Message-ID: <2483b0d2a12f49924bf0e66bf7c48549@freebsd.org> X-Sender: jamie@freebsd.org User-Agent: Roundcube Webmail/1.2.3 X-Greylist: inspected by milter-greylist-4.6.2 (gritton.org [199.192.165.131]); Fri, 02 Jun 2017 10:09:25 -0600 (MDT) for IP:'199.192.165.131' DOMAIN:'gritton.org' HELO:'gritton.org' FROM:'jamie@freebsd.org' RCPT:'' X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (gritton.org [199.192.165.131]); Fri, 02 Jun 2017 10:09:25 -0600 (MDT) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jun 2017 16:17:40 -0000 On 2017-05-31 02:33, Marko Cupać wrote: > Hi, > > I'm not subscribed to the list, could you please keep me in CC? > > I'm using ezjail as instructed in Handbook, assigning jails > lo1|127.0.0.X,bce0|10.66.66.X addresses, in order to keep jails' > loopback traffic off host's, and in order to be able to keep internal > services on lo1 (such as redis, mongodb, mysql etc.), and external on > bce0 (such as apache, unifi5 etc.). > > Recently I got a server with multiple NICs, and I'd like to serve both > LAN and DMZ services from it. I found some information on how to > accomplish that with setfib: > > # cat /boot/loader.conf > net.fibs=4 > net.add_addr_allfibs=0 > > # cat /etc/rc.conf > ... > cloned_interfaces="lo1" > static_routes="nix nixd" > route_nix="-net 10.66.66.0/24 -interface bce0 -fib 1" > route_nixd="default 10.66.66.254 -fib 1" > ... > > In this setup, services bound to bce0 interface work fine, but they > can't contact internal services on lo1. I guess it has something to do > with jail routing, but can't figure out what. > > Thank you in advance for any hints. I haven't done the lo1 trick before, but I have had jails with addresses on a different FIB. Note that the jail also has an FIB. You probably at least want to set the jail's fib to 1 (exec.fib in jail.conf, I suppose jail_*_fib or whatever in the old rc-based system ezjail still uses). The part I'm not sure about is you probably also want to have lo1's entries in the fib=1 routing table. I don't know the interaction between cloned_interfaces and fib though - that might take some exploring in rc, or a word or two from someone who knows that side of things more than I do. - Jamie