Date: Sat, 17 Feb 2007 02:43:20 +0100 From: Ivan Voras <ivoras@fer.hr> To: freebsd-performance@freebsd.org Subject: Virtual performance Message-ID: <er5mk5$uul$1@sea.gmane.org>
next in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig3C9F71416DC227FBE6A1EBCA
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
I haven't been using virtual machines for production much, but this is
likely to change in the near future. After running some benchmarks, it
looks like there's something very bad with performance under VMWare.
I've tried two things: the big "VMWare Infrastructure" product, version
3.0.1 and the small, free (as in beer) VMWare Infrastructure, but the
findings are the same. As an illustration, consider this header from "top=
":
last pid: 29892; load averages: 1.20, 1.16, 1.09 up 0+00:45:22
01:03:37
38 processes: 3 running, 35 sleeping
CPU states: 3.8% user, 0.0% nice, 96.2% system, 0.0% interrupt, 0.0%
idle
Mem: 29M Active, 251M Inact, 117M Wired, 2080K Cache, 112M Buf, 3363M Fre=
e
Swap: 5120M Total, 5120M Free
Note the "system" time - while this statistic might look like a getpid()
loop, it is, in fact, a sample from the middle of a `make buildkernel`
on 6.2-release. In fact, on a non-virtual machine, a getpid() loop comes
out rougly 60% in sys time + 40% in user time. I don't know why would
compilation, a memory and CPU intensive process, require so much sys
time, and the same is noticeable when running practically any program.
All this time is not spent in a kernel thread - top showing system
threads shows them all as quiescent.
Running unixbench, I see that performance is mainly lost on benchmarks
that do context switching, while numeric performance stays approximately
the same. Most performance loss is on "context1" and "pipe" benchmarks.
Here's a typical result of running unixbench for pipe and context1
benchmark on the above machine:
INDEX VALUES
TEST BASELINE RESULT INDE=
X
Pipe Throughput 12440.0 48528.7 39.=
0
Pipe-based Context Switching 4000.0 9593.8 24.=
0
These should (the INDEX field), for this machine, be somewhere between
350 - 500. Other tests that frequently make kernel calls (like: execl,
file system, syscall overhead) are also very much affected.
Some things I've tried (without luck):
- several different machines and CPUs (Intel, AMD), all on i386 arch
- tried both RELENG_6 and HEAD (WITNESS, INVARIANTS disabled) **
- disabling and enabling SMP **
- disabling and enabling PREEMPTION, ADAPTIVE_MUTEXES and ADAPTIVE_GIANT
- disabling and enabling kern.sched.ipiwakeup.enabled
- decreasing kern.hz
- changing timecounter to TSC
[**] : There's a special case here: I've just finished compiling a UP
kernel for HEAD, and when running unixbench, the "Pipe throughput"
benchmark of unixbench gets an order of magnitude better (to ~~480). I'm
too tired now to dig further on it.
The peculiar thing is that such a slowdown is not present in Linux and
Windows - there it it's something like 10% slowdown at most (compared to
the same machine in non-virtual mode), while doing the same comparison
on FreeBSD it comes on the order of 4x - 5x slower.
I think this finding should be easily reproducable and verifyable -
anyone with a Windows or Linux machine can install the free VMWare
Server and load up FreeBSD as guest.
I don't know whose fault this is, VMWares or FreeBSD's, but
virtualization is popular, and since FreeBSD is very much lagging behind
for server-side virtualization (Xen, VMWare, etc. - jails and vimage
don't count since they're FreeBSD-specific), it would be a shame if it
also got marked as bad at client-side.
Any ideas on what to try next to improve the performance?
--------------enig3C9F71416DC227FBE6A1EBCA
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.4 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFF1l3BldnAQVacBcgRAo5uAKDesq+ncHjcQBhyOaD3TSGuDi6dJACg9N2C
ZIrZCSDHNZKlBGPHRNHqz8E=
=Wr60
-----END PGP SIGNATURE-----
--------------enig3C9F71416DC227FBE6A1EBCA--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?er5mk5$uul$1>