From owner-freebsd-performance@FreeBSD.ORG Sat Feb 17 01:43:48 2007 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 383DF16A401 for ; Sat, 17 Feb 2007 01:43:48 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id C349713C48D for ; Sat, 17 Feb 2007 01:43:47 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1HIEc0-000855-Lv for freebsd-performance@freebsd.org; Sat, 17 Feb 2007 02:43:40 +0100 Received: from 89-172-38-76.adsl.net.t-com.hr ([89.172.38.76]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 17 Feb 2007 02:43:40 +0100 Received: from ivoras by 89-172-38-76.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 17 Feb 2007 02:43:40 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: Ivan Voras Date: Sat, 17 Feb 2007 02:43:20 +0100 Lines: 100 Message-ID: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig3C9F71416DC227FBE6A1EBCA" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-38-76.adsl.net.t-com.hr User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) X-Enigmail-Version: 0.94.1.2 Sender: news Subject: Virtual performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Feb 2007 01:43:48 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig3C9F71416DC227FBE6A1EBCA Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I haven't been using virtual machines for production much, but this is likely to change in the near future. After running some benchmarks, it looks like there's something very bad with performance under VMWare. I've tried two things: the big "VMWare Infrastructure" product, version 3.0.1 and the small, free (as in beer) VMWare Infrastructure, but the findings are the same. As an illustration, consider this header from "top= ": last pid: 29892; load averages: 1.20, 1.16, 1.09 up 0+00:45:22 01:03:37 38 processes: 3 running, 35 sleeping CPU states: 3.8% user, 0.0% nice, 96.2% system, 0.0% interrupt, 0.0% idle Mem: 29M Active, 251M Inact, 117M Wired, 2080K Cache, 112M Buf, 3363M Fre= e Swap: 5120M Total, 5120M Free Note the "system" time - while this statistic might look like a getpid() loop, it is, in fact, a sample from the middle of a `make buildkernel` on 6.2-release. In fact, on a non-virtual machine, a getpid() loop comes out rougly 60% in sys time + 40% in user time. I don't know why would compilation, a memory and CPU intensive process, require so much sys time, and the same is noticeable when running practically any program. All this time is not spent in a kernel thread - top showing system threads shows them all as quiescent. Running unixbench, I see that performance is mainly lost on benchmarks that do context switching, while numeric performance stays approximately the same. Most performance loss is on "context1" and "pipe" benchmarks. Here's a typical result of running unixbench for pipe and context1 benchmark on the above machine: INDEX VALUES TEST BASELINE RESULT INDE= X Pipe Throughput 12440.0 48528.7 39.= 0 Pipe-based Context Switching 4000.0 9593.8 24.= 0 These should (the INDEX field), for this machine, be somewhere between 350 - 500. Other tests that frequently make kernel calls (like: execl, file system, syscall overhead) are also very much affected. Some things I've tried (without luck): - several different machines and CPUs (Intel, AMD), all on i386 arch - tried both RELENG_6 and HEAD (WITNESS, INVARIANTS disabled) ** - disabling and enabling SMP ** - disabling and enabling PREEMPTION, ADAPTIVE_MUTEXES and ADAPTIVE_GIANT - disabling and enabling kern.sched.ipiwakeup.enabled - decreasing kern.hz - changing timecounter to TSC [**] : There's a special case here: I've just finished compiling a UP kernel for HEAD, and when running unixbench, the "Pipe throughput" benchmark of unixbench gets an order of magnitude better (to ~~480). I'm too tired now to dig further on it. The peculiar thing is that such a slowdown is not present in Linux and Windows - there it it's something like 10% slowdown at most (compared to the same machine in non-virtual mode), while doing the same comparison on FreeBSD it comes on the order of 4x - 5x slower. I think this finding should be easily reproducable and verifyable - anyone with a Windows or Linux machine can install the free VMWare Server and load up FreeBSD as guest. I don't know whose fault this is, VMWares or FreeBSD's, but virtualization is popular, and since FreeBSD is very much lagging behind for server-side virtualization (Xen, VMWare, etc. - jails and vimage don't count since they're FreeBSD-specific), it would be a shame if it also got marked as bad at client-side. Any ideas on what to try next to improve the performance? --------------enig3C9F71416DC227FBE6A1EBCA Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF1l3BldnAQVacBcgRAo5uAKDesq+ncHjcQBhyOaD3TSGuDi6dJACg9N2C ZIrZCSDHNZKlBGPHRNHqz8E= =Wr60 -----END PGP SIGNATURE----- --------------enig3C9F71416DC227FBE6A1EBCA--