From owner-freebsd-net@freebsd.org Tue Feb 14 01:52:46 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6DF64CDDBFF for ; Tue, 14 Feb 2017 01:52:46 +0000 (UTC) (envelope-from jmk@wagsky.com) Received: from bmx.allycomm.com (bmx.allycomm.com [198.199.108.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6222F151D for ; Tue, 14 Feb 2017 01:52:45 +0000 (UTC) (envelope-from jmk@wagsky.com) Received: from jkletsky1-mbp15.guidewire.com (inet.guidewire.com [199.91.42.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by bmx.allycomm.com (Postfix) with ESMTPSA id F3EE3D78B0; Mon, 13 Feb 2017 17:47:07 -0800 (PST) To: freebsd-net@freebsd.org From: Jeff Kletsky Subject: VNET / netgraph jails -- Locking down? Message-ID: Date: Mon, 13 Feb 2017 17:47:07 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Feb 2017 01:52:46 -0000 For several years I've been using netgraph to provide connectivity for "service hosts" in jails on a "jail server" Since I'm finally getting the jail server off FreeBSD 9 and solidly onto 11, I've got the chance to rewrite the scripting of how I'm handling jail connectivity and am hoping that I can lock things down a bit better than what I have presently. The approach I use looks similar to that now in the jail examples. Basically /---> ng_eiface_jail1 real_interface = ng_ether <---> ng_bridge <---> ng_eiface_jail2 \---> ng_eiface_jail3 While this works well, it concerns me that the real interface has to be in promiscuous mode (and have autosrc off). If one of the service jails is "taken over" then there isn't a way that I know of to lock out changing the IP address of the interface it has, or potentially gaining access to another VLAN through creation of a cloned interface, especially if the bridge is off the parent interface, not off a VLAN interface. How do people manage this in practice when the jail has the risk of compromise? I prefer approaches where the jail's notion of it's own IP address is the same as that of other hosts connecting to it, at least within my own little private-address-space world. One thing that I've been considering is: * Configure the jail's IP on the real interface (or appropriate VLAN interface) as an alias * Send packets through ng_ipfw to an ng_eiface that the jail gets, using ipfw and a lookup table * Tag the packets on return with ng_tag with a unique identifier for that jail's interface so ipfw can tell the only acceptable source IP * Deny any so-tagged packets that don't have the proper source address (jail ID by itself is not enough for the outbound packets, as some of the jails are dual homed.) Has anyone tried this kind of method? Any other/better suggestions? Would ng_ip_input be the appropriate way to "send" the packets coming from the jail? Thanks! Jeff