Date: Tue, 12 Aug 2008 10:53:14 -0700 From: Jeff Kletsky <jeff+freebsd@wagsky.com> To: freebsd-questions@freebsd.org Subject: Jails - Pseudo Interfaces/ Virtual Networks - Best Practices? Message-ID: <48A1CE0A.1030208@wagsky.com>
next in thread | raw e-mail | index | archive | help
I'm in the process of planning a transition from VMWare (on CentOS host) hosting service-specific FreeBSD virtual machines to the seemingly more efficient, stable, and lower maintenance cost approach, now that I've figured out at least some of the "magic" of creating jails under FreeBSD. I do have a couple questions as to what the "recommended" approaches to the IP interface for the jails. The questions generally relate to being able to consistently identify traffic from/to the jails, inside and outside the jail host, with tools such as ipfw and tcpdump. One set of concerns that I haven't looked into is discussed in the thread <http://www.freebsd.org/cgi/getmsg.cgi?fetch=1418948+0+/usr/local/www/db/text/2008/freebsd-questions/20080720.freebsd-questions>; where it appears that "To sum up, if jail host running at 10.0.0.1 connects to a jail running at 10.0.0.2, the traffic will occur over lo0, and BOTH endpoints of that connection will use the jail (10.0.0.2) address. To my mind, that can be problematic." With VMWare, I was used to being able to create a pseudo-interface that could be "wired" to a virtual "switch" (hub, from what I can tell) and then that switch could be bridged (or NAT-ed) to a physical interface. This could lead to topologies that "looked like" physical machines and traffic could be controlled between them relatively unambiguously. Some places where the FreeBSD jail "bind to physical interface" model seems to fail for me are described below. I'd appreciate insights into a good strategy to deal with these and related situations: 1) Firewall/bastion host, DNS intended for that host only Here DNS needs to be run for the host itself only, to be able to start up NTP and allow ssh connections for management over the "protected" interface. In this situation, the DNS is *only* supplied to the physical host itself. Binding the jail to 127.0.0.1 doesn't seem right to me. Binding it to the "outside" (public) interface is equally bothersome. 2) Bastion host, SMTP relay Here the firewall/bastion host is: * Accepting connections on its public interface for SMTP connections and relaying to an internal SMTP server * Accepting connections on its public interface for "submission" connections and relaying internally and externally for authenticated users * Accepting connections on an internal IP for SMTP connections (with different Postfix "rules") and relaying both internally and externally * Connecting to an internal DNS server, either in another jail on the same machine, or in an jail on another machine In this case (and related ones) the firewall rules would be simplified if the packets from the jail were actually "in recv" from an identifiable interface so that the packets "looked like" those from a physical host on the DMZ network, allowing the "basic" access control and filtering to be handled by the "general DMZ" sections of the rules. (I'm aware that there is a "jail prisionID" selector in ipfw, but (a) it seems to only handle TCP and UDP, and (b) I'd likely have to repeat topology-related rules for each jail.) Is there a "standard" way to create jails on a virtual interface or, perhaps better yet, "internal" network or switch, that would allow them to be handled en masse by topology-driven rules? I'm hardly a netgraph expert and didn't find any references in the documentation to this kind of use of the facility. Is there a way to hang a "virtual DMZ" off a pseudo-interface somewhere "inside" the host in addition to the "physical DMZ" attached to a external interface - routing tables cover virtual <=> physical communications - packets "in recv" psuedo-interface from the virtual DMZ and ipfw can handle control of communication between two DMZ (and other) subnets This could either be replicated once for each service jail, or a topology that had a "virtual smart switch" connecting multiple pseudo-interfaces that could be used by multiple jails, that then communicated with the rest of the world through a single pseudo-interface. Thanks for taking the time to consider this. Jeff
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?48A1CE0A.1030208>