Date: Tue, 22 Dec 2015 22:42:33 -0600 From: "Matthew D. Fuller" <fullermd@over-yonder.net> To: Garrett Wollman <wollman@bimajority.org> Cc: freebsd-net@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Have I got this VIMAGE setup correct? Message-ID: <20151223044233.GM33115@over-yonder.net> In-Reply-To: <22137.33475.645324.203196@hergotha.csail.mit.edu> References: <22137.33475.645324.203196@hergotha.csail.mit.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Dec 22, 2015 at 12:05:07PM -0500 I heard the voice of Garrett Wollman, and lo! it spake thus: > > The consensus when I asked seemed to be that VIMAGE+jail was the > right combination to give every container its own private loopback > interface, so I tried to build that. I noticed a few things: I've got a server running a dozen or so VIMAGE jails, so I can at least chime in a little... > 1) The kernel prints out a warning message at boot time that VIMAGE > is "highly experimental". Should I be concerned about running this > in production? It hasn't blown up anything for me yet. > 2) Stopping jails with virtual network stacks generates warnings from > UMA about memory being leaked. I'm given to understand that's Known, and presumably Not Quite Trivial To Fix. Since I'm not starting/stopping jails repeatedly as a normal runtime thing, I'm ignoring it. If you were spinning jails up and down dynamically dozens of times a day, I'd want to look more closely at just what is leaking and why... > 3) It wasn't clear (or documented anywhere that I could see) how to > get the host network set up properly. Obviously I'm not going to > have a vlan for every single jail, so it seemed like what most > people were doing was "bridge" along with a bunch of "epair" > interfaces. I ended up with the following: Is what I'm doing, though I'm creating the epair's and adding them to the bridges in the setup script rather than rc.conf (exec.prestart in jail.conf), because that makes it a more manageable IME, and since I'm already doing a bunch of setup in the script anyway... > In each of the jails I have to manually configure a MAC address > using /etc/start_if.epairNb to ensure that it's globally unique, but > then everything seems to work. I hardcode (well, dynamically generated hardcoded) MAC addresses on the epair's in the setup script, since <https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=184149>; bit me hard when I was first setting it up. -- Matthew Fuller (MF4839) | fullermd@over-yonder.net Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20151223044233.GM33115>