Date: Wed, 1 Mar 2017 11:06:54 -0700 From: Alan Somers <asomers@freebsd.org> To: Julian Elischer <julian@freebsd.org> Cc: Jeff Kletsky <freebsd@wagsky.com>, FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: VNET / netgraph jails -- Locking down? Message-ID: <CAOtMX2inZRL7xRb3MOYqe%2Bc=A_AZajVYOCmWUeDtGwTODZt6Vg@mail.gmail.com> In-Reply-To: <4dada28e-2130-70f3-e4b1-9d20f4185437@freebsd.org> References: <a73d08e4-7d9c-b059-6ae9-d92e8278d03d@wagsky.com> <4dada28e-2130-70f3-e4b1-9d20f4185437@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
I do something similar, but I rely entirely on vnet and PF instead of
netgraph. My host has two ethernet ports, so I use one for the host
and one for all of the jails. That makes the pf setup easier. I use
iocage to configure an ordinary vnet jail, bridged to the host's
second ethernet port. Then I configure pf.conf like this. This
ensures that the only traffic allowed into the jail is that destined
for its intended service, and the only traffic allowed out are
responses to inbound traffic. The jail is able to change its own IP
address, but if it does then PF will block all of its traffic.
www_services = "{ http, https, 8080 }"
host_iface = "em0"
dmz_iface = "em1"
www_jail_iface = "vnet0:1"
www_ip = "192.168.0.40"
set state-policy if-bound
scrub in
block in all
block out all
pass in on $host_iface
pass out on $host_iface
set skip on lo0
# Allow all traffic to the DMZ. Filtering happens on individual vnet
# interfaces
pass in on $dmz_iface
pass out on $dmz_iface
# Put the www jail in a DMZ. Don't allow outgoing traffic from it except for
# the webserver
pass out on $www_jail_iface proto tcp to $www_ip port $www_services keep state
-Alan
On Wed, Mar 1, 2017 at 10:36 AM, Julian Elischer <julian@freebsd.org> wrote:
> many good questions but looking at what you are doing, maybe we should be
> asking you the questions.
> Certainly firewalling on the outside of the jail makes sense. I've not used
> ng_ipfw but it would make sense to do a quick santity check for every packet
> leaving each jail.
>
>
>
> On 14/2/17 9:47 am, Jeff Kletsky wrote:
>>
>> For several years I've been using netgraph to provide connectivity for
>> "service hosts" in jails on a "jail server"
>>
>> Since I'm finally getting the jail server off FreeBSD 9 and solidly onto
>> 11, I've got the chance to rewrite the scripting of how I'm handling jail
>> connectivity and am hoping that I can lock things down a bit better than
>> what I have presently.
>>
>>
>> The approach I use looks similar to that now in the jail examples.
>> Basically
>>
>> /---> ng_eiface_jail1
>> real_interface = ng_ether <---> ng_bridge <---> ng_eiface_jail2
>> \---> ng_eiface_jail3
>>
>> While this works well, it concerns me that the real interface has to be in
>> promiscuous mode (and have autosrc off).
>>
>> If one of the service jails is "taken over" then there isn't a way that I
>> know of to lock out changing the IP address of the interface it has, or
>> potentially gaining access to another VLAN through creation of a cloned
>> interface, especially if the bridge is off the parent interface, not off a
>> VLAN interface.
>>
>>
>> How do people manage this in practice when the jail has the risk of
>> compromise?
>>
>>
>> I prefer approaches where the jail's notion of it's own IP address is the
>> same as that of other hosts connecting to it, at least within my own little
>> private-address-space world.
>>
>>
>> One thing that I've been considering is:
>> * Configure the jail's IP on the real interface (or appropriate VLAN
>> interface) as an alias
>> * Send packets through ng_ipfw to an ng_eiface that the jail gets, using
>> ipfw and a lookup table
>> * Tag the packets on return with ng_tag with a unique identifier for that
>> jail's interface so ipfw can tell the only acceptable source IP
>> * Deny any so-tagged packets that don't have the proper source address
>>
>> (jail ID by itself is not enough for the outbound packets, as some of the
>> jails are dual homed.)
>>
>>
>> Has anyone tried this kind of method? Any other/better suggestions?
>>
>>
>> Would ng_ip_input be the appropriate way to "send" the packets coming from
>> the jail?
>>
>>
>> Thanks!
>>
>>
>> Jeff
>>
>>
>>
>>
>> _______________________________________________
>> freebsd-net@freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>>
>
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2inZRL7xRb3MOYqe%2Bc=A_AZajVYOCmWUeDtGwTODZt6Vg>