Date: Thu, 7 May 2009 09:13:36 -0400 From: Bill Moran <wmoran@potentialtech.com> To: glarkin@FreeBSD.org Cc: Warren Guy <warren.guy@calorieking.com>, freebsd-questions@freebsd.org Subject: Re: Developing and maintaining a rapidly deployable image of an installed system Message-ID: <20090507091336.5cdc3101.wmoran@potentialtech.com> In-Reply-To: <4A02D7B4.5070705@FreeBSD.org> References: <4A025F42.50308@calorieking.com> <20090507080157.93c8e5ee.wmoran@potentialtech.com> <4A02D7B4.5070705@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
In response to Greg Larkin <glarkin@FreeBSD.org>: > Bill Moran wrote: > [...] > > > > The host system is a very basic install -- mostly just give it an IP and > > add users for the administrators. All the ports and details of their > > configs are in the individual jails. It's much more efficient than using > > something like VMware, which has horrific performance penalties. > > > > Hi Bill, > > We've been using VMware with FreeBSD guests for a few years now without > any performance problems. In fact, jails even work well within a FreeBSD > VM. What kinds of problems have you run into? Really? Do you have any machines with more than 30 virtual systems on them? Even our beefiest hardware can't handle more than 10 VMWare machines. We use both jails and VMWare. Each has it's pros and cons. The big pro of VMWare is that you can simulate an entire piece of hardware, which is necessary for much of our lab and testing work. The big con of VMWare is performance and overhead. Another big pro of VMWare is that we can have FreeBSD, and Linux, and MS operating systems all running on the same hypervisor, which jails can't do. The big pro of jails is that we can put a crapload of jails on each physical server. Six or 8 is typical, but we have systems with more than 30 humming along happily. Each jail uses far less disk space than a virtual machine, and uses far less CPU. Also, the jail mechanism puts fewer layers between the OS and the hardware, which means things like network and disk performance suffer very little. Take two equivalent machines and put a FreeBSD jail on one and FreeBSD in a virtual machine on the other and benchmark the network and disk performance on each. You'll find that VMWare loses big time. If that's not enough to convince you, then increase the number of systems on each machine to about 10 and rerun the tests while the systems are under load -- VMWare doesn't scale up nearly as well as jails do. If you're doing purely CPU usage, then the two options appear to be roughly equivalent, although I've never done an actual test. Restarting a jail takes seconds, restarting a VM takes minutes. We've frequently had to go through our dev servers and shut down VMWare virtual machines to free up resources when a few of the VMs were seeing heavy usage. We've had the same problem with jails far less frequently. Like I said earlier, VMWare has its advantages, but performance is not one of them. And yes, we have VMWare VMs with FreeBSD jails inside them. It's a bizarre combination, but it works quite well ... right up until someone wants to load up one of the PostgreSQL servers and the disk issue brings the system to its knees ... that's actually a pretty good benchmark to illustrate the problem: pgbench will show how much VMWare hurts disk performance right out of the gate. I guess the overall issues are more with scalability than performance, but the two issues are linked in such a way that I frequently don't separate them. VMWare starts with a minor performance hit compared to jails, that performance hit increases significantly as you add VMs, whereas jails scale up very well. -- Bill Moran http://www.potentialtech.com http://people.collaborativefusion.com/~wmoran/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20090507091336.5cdc3101.wmoran>