Date: Mon, 2 Jul 2007 12:43:38 -0500 From: Matt Simerson <matt@tnpi.net> To: freebsd-security@freebsd.org Cc: olli@lurza.secnetix.de Subject: Jails and loopback interfaces Message-ID: <DFDD9B84-5DD1-4076-AB3E-869B19185CFB@tnpi.net>
next in thread | raw e-mail | index | archive | help
I've got a server running FreeBSD 6.2 and PF. The server has a couple
dozen jails on it. Previously, I had a few "private" services such as
MySQL running on loopback IPs (127.0.0.2+) and the rest of the jails
running on the public IPs.
I have to renumber my machine with a new block of public IPs so I
thought I'd be clever and move all the jails onto loopback IPs. Then
I could use PF to redirect the new IPs and the old ones to the
corresponding jails.
The relevant parts of my PF config look like this:
> ext_if="em0"
> loop_if="lo0"
>
> ext_addr="72.29.111.130"
> loop_addr="127.0.0.101"
>
> ext_net ="72.29.111.128/27"
> ext_net2="208.75.180.64/27"
> loop_net="127.0.0.0/8"
>
> lo_simerson = "127.0.0.4"
> lo_toaster = "127.0.0.6"
> lo_tnpi = "127.0.0.13"
>
> simerson = "72.29.111.131"
> toaster = "72.29.111.133"
> tnpi = "72.29.111.140"
>
> # nat the jails running on loopback to their public ip
> nat on $ext_if from $lo_simerson to any -> $simerson
> nat on $ext_if from $lo_toaster to any -> $toaster
> nat on $ext_if from $lo_tnpi to any -> $tnpi
>
> # specific jail rules
> rdr on $ext_if from any to $simerson -> $lo_simerson
> rdr on $ext_if from any to $toaster -> $lo_toaster
> rdr on $ext_if from any to $tnpi -> $lo_tnpi
This works just great. Traffic from the outside world gets redirected
to the correct jail via a rdr rule. The corresponding nat rule alters
the source IP from each jail to its corresponding public IP.
The problem I have with this arrangement is when a jail attempts to
connect to the public IP of another jails, the connection fails. So,
a client running in one jail can't send email to my mail server
running in another jail. This problem is described in detail here:
http://www.openbsd.org/faq/pf/rdr.html
And I have attempted to configure the RDR and NAT combination
solution as shown. But after trying a few dozen iterations, I can't
get it to work. Based on my understand of the PF manual,
conceptually, the problematic connection looks like this:
simerson (127.0.0.4) -> toaster public IP (72.29.111.133)
rewritten to toaster private IP (127.0.0.6)
toaster (127.0.0.6) -> simerson (127.0.0.4)
So the simerson jail gets returned a packet from the toasters
loopback IP, which it ignores because it is not the IP it sent the
connection attempt to.
But no matter what I try, I can't make it work. PF is not denying any
connections. Without resorting to split-horizon DNS, has anyone
gotten this to work reliably on FreeBSD? I'd love to see a working
configuration.
Matt
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?DFDD9B84-5DD1-4076-AB3E-869B19185CFB>