From owner-freebsd-jail@freebsd.org Wed Mar 13 23:13:09 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 251D2154435F for ; Wed, 13 Mar 2019 23:13:09 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from msa1.earth.yoonka.com (yoonka.com [88.98.225.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "msa1.earth.yoonka.com", Issuer "msa1.earth.yoonka.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9F63B74782 for ; Wed, 13 Mar 2019 23:13:07 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from crayon2.yoonka.com (crayon2.yoonka.com [10.70.7.20]) (authenticated bits=0) by msa1.earth.yoonka.com (8.15.2/8.15.2) with ESMTPSA id x2DND0Hr034348 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Wed, 13 Mar 2019 23:13:00 GMT (envelope-from list1@gjunka.com) Subject: Re: exec.fib and a jail in two subnets To: James Gritton Cc: freebsd-jail@freebsd.org References: <6a245a1f51270c71d1da07c55ef51113@gritton.org> From: Grzegorz Junka Message-ID: <8fc3d6df-9f8f-f07c-4e13-1ced74f5c0ad@gjunka.com> Date: Wed, 13 Mar 2019 23:13:00 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.3.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-GB-large X-Rspamd-Queue-Id: 9F63B74782 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of list1@gjunka.com designates 88.98.225.149 as permitted sender) smtp.mailfrom=list1@gjunka.com X-Spamd-Result: default: False [-6.80 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:88.98.225.149]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[gjunka.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[gjunka.com]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.85)[-0.855,0]; IP_SCORE(-3.64)[ip: (-9.53), ipnet: 88.98.192.0/18(-4.77), asn: 56478(-3.81), country: GB(-0.09)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:56478, ipnet:88.98.192.0/18, country:GB]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2019 23:13:09 -0000 >> Many thanks for your response. The second example works with 10.0.0.1 >> but not with 172.16.0.1, otherwise there would be no post. Following on >> your response, lets assume that a process (e.g. nginx) listens on both >> IPs, 10.0.0.2,172.16.0.2. Is it possible to configure fibs or default >> routes or whatever so that when a packet arrives from 10.0.0.1 it is >> send back to 10.0.0.1 and if it arrives from 172.16.0.1 it is send back >> to 172.16.0.1 (thus using default routes from either fib0 or fib1 >> depending if the packet came from a router in one of those network)? If >> not, would it be possible to do this with some iptables/pf rules (which >> I understand in FreeBSD 12 should work in a jail with VNET)? > My understanding (which I admit is imperfect) is that it's not > possible with default routes alone. At the application level, it > would be possible if nginx was either fib-aware, or if it explicitly > bound the source address of its replies - but neither of those are > things typically done at the application level. > > It is possible however at the firewall level; At least I know it's > possible for ipfw (the small corner of the firewall world that I > use). A quick check of ipf and ipfilter man pages didn't show "fib" > anywhere, but don't take my word on those. It also may require a > VNET jail; I've never run a system with your exact setup so I'm > unsure whether the binding to the first (non-vnet) jail address > happens before or after the ipfilter rules. > > - Jamie I am just playing with this now and what I see is that a jail can't be in two fibs at the same time. It looks like the host is able to select the default route depending on the subnet in which is the IP I want to reach, but in the jail, telneting or otherwise trying to reach any IP that isn't in the same subnet as the fib specified in exec.fib, is not working. For example, in jail this works: telnet 172.16.0.1 80 but this doesn't telnet 10.0.0.1 80 On the host both works. And both, the host and the jail have an IP and an alias in both subnets.