From owner-freebsd-jail@FreeBSD.ORG Wed Apr 2 13:39:47 2008 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3AAD71065673 for ; Wed, 2 Apr 2008 13:39:47 +0000 (UTC) (envelope-from josh@endries.org) Received: from www.endries.org (www.endries.org [216.230.164.2]) by mx1.freebsd.org (Postfix) with ESMTP id 2D5EF8FC2C for ; Wed, 2 Apr 2008 13:39:46 +0000 (UTC) (envelope-from josh@endries.org) Received: from localhost (localhost.endries.org [127.0.0.1]) by www.endries.org (Postfix) with ESMTP id 153ECA664AF for ; Wed, 2 Apr 2008 09:23:38 -0400 (EDT) X-Virus-Scanned: amavisd-new at endries.org Received: from www.endries.org ([127.0.0.1]) by localhost (www.endries.org [127.0.0.1]) (amavisd-new, port 10025) with LMTP id L1lApij5MdXp for ; Wed, 2 Apr 2008 09:23:31 -0400 (EDT) Received: from [192.168.0.128] (rrcs-72-43-92-186.nys.biz.rr.com [72.43.92.186]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by www.endries.org (Postfix) with ESMTP id 20318A66422 for ; Wed, 2 Apr 2008 09:23:28 -0400 (EDT) Message-ID: <47F388CB.2000205@endries.org> Date: Wed, 02 Apr 2008 09:23:23 -0400 From: Josh Endries User-Agent: Thunderbird 2.0.0.12 (Windows/20080213) MIME-Version: 1.0 To: freebsd-jail@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Question about pf, NAT and routing. (attempt #2) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2008 13:39:47 -0000 For some reason my other message never made it through, so here we are again! A while ago I needed to give a jail access to two networks/IPs (one external for a web site and one internal for a DB connection). I ended up using a localhost IP (127.0.0.2) for the jail and nat/binat in pf to control where the traffic went, depending on destination. I'm trying to set up a jail now that's similar. My host has multiple interfaces on multiple networks, and the jail is on lo0, and I would like to NAT traffic to internal networks from one IP out one interface, and everything else out another IP through the external interface. I found an email on here from jpaetzel (o/) explaining how to use route-to, and that works; it fixed default route problem (thanks!). Unfortunately that only seems to work if the jail is using an IP on one of the interfaces in question. I suppose there is some sort of problem between the NATing and routing. Here is my crazy config: ra# ifconfig bce0: flags=8843 metric 0 mtu 1500 options=1bb ether 00:1e:0b:ed:f9:ec media: Ethernet autoselect (1000baseTX ) status: active lagg: laggdev lagg0 bce1: flags=8843 metric 0 mtu 1500 options=1bb ether 00:1e:0b:ed:f9:ec media: Ethernet autoselect (none) status: no carrier lagg: laggdev lagg0 lo0: flags=8049 metric 0 mtu 16384 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet6 ::1 prefixlen 128 inet 127.0.0.1 netmask 0xff000000 inet 127.0.0.5 netmask 0xffffffff inet 127.0.0.4 netmask 0xffffffff lagg0: flags=8843 metric 0 mtu 1500 options=1bb ether 00:1e:0b:ed:f9:ec media: Ethernet autoselect status: active laggproto lacp laggport: bce1 flags=20 laggport: bce0 flags=1c vlan2: flags=8843 metric 0 mtu 1500 options=3 ether 00:1e:0b:ed:f9:ec inet 10.1.0.2 netmask 0xffffff00 broadcast 10.1.0.255 media: Ethernet autoselect status: active vlan: 2 parent interface: lagg0 vlan1: flags=8843 metric 0 mtu 1500 options=3 ether 00:1e:0b:ed:f9:ec inet 192.168.0.91 netmask 0xffffff00 broadcast 192.168.0.255 media: Ethernet autoselect status: active vlan: 1 parent interface: lagg0 ra# pfctl -s nat No ALTQ support in kernel ALTQ related functions disabled binat on vlan2 inet from 127.0.0.5 to any -> 10.1.0.2 binat on vlan1 inet from 127.0.0.4 to any -> 192.168.0.91 ra# pfctl -s rules No ALTQ support in kernel ALTQ related functions disabled scrub in all fragment reassemble block drop all pass out route-to (vlan2 10.1.0.1) inet from 10.1.0.2 to ! 10.1.0.0/24 flags S/SA keep state pass out route-to (vlan1 192.168.0.1) inet from 192.168.0.91 to ! 192.168.0.0/24 flags S/SA keep state ra# jls JID IP Address Hostname Path 22 127.0.0.5 dns /jails/dns/root 21 127.0.0.4 mysql /jails/mysql/root The problem is the same. The jail that has the default route for it's NAT (dns) works fine, but the other jail (mysql) doesn't. I get the error "Can't assign requested address". The packets from the mysql jail don't show up anywhere in tcpdump, packets from the dns jail shows up as normal. :D Josh