Date: Mon, 26 Jan 2004 12:12:58 -0500 (EST) From: Robert Watson <rwatson@freebsd.org> To: Poul-Henning Kamp <phk@phk.freebsd.dk> Cc: current@freebsd.org Subject: Re: Hints for precision benchmarking... Message-ID: <Pine.NEB.3.96L.1040126120627.11399N-100000@fledge.watson.org> In-Reply-To: <67153.1075136001@critter.freebsd.dk>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 26 Jan 2004, Poul-Henning Kamp wrote:
> * Run in single user mode. Cron(8) and and other daemons
> only add noise.
Of particular interest here is sshd -- either disable its SSHv1 key
regeneration, or kill the parent sshd (or don't run sshd) during the
tests.
A few obvious ones just worth remembering:
(1) Don't use bgfsck unless you want to benchmark with bgfsck. Also,
disable background_fsck in rc.conf unless you plan not to start the
benchmark for at least 60+fudge seconds after the boot, as rc wakes up
and checks to see if anything needs fscking if it is enabled.
Likewise, make sure you don't have snapshots lying around unless you
mean to teset with snapshots.
(2) If you must leave the system connected to a public network, watch out
for spikes of broadcast traffic. Even though you don't notice it, it
will eat your CPU. Multicast has similar caveats.
(3) If you see unexpectedly bad performance, check for things like
high interrupt volume from an unexpected source. I've seen several
reports of ACPI "misbehaving" and generating excess interrupts under
some of the ACPI code drops. If you're seeing odd results, take a few
snapshots of vmstat -i and look for anything unusual.
(4) Make sure to be careful about optimization parameters for kernel and
userspace, likewise debugging. It's easy to let something slip
through and realize later you weren't comparing the same thing.
(5) And finally, do not ever benchmark with WITNESS and INVARIANTS unless
you are really just interested in benchmarking those features. :-)
I've seen 400%+ drops in performance from running with WITNESS.
Likewise, userspace malloc paramters default differently in -CURRENT
from the way they'd ship in production.
Also, it would be very helpful if, while benchmarking, people would do the
same benchmark with SCHED_ULE and SCHED_4BSD, and see what impact it has.
Scheduler changes are very influential to performance, and we want to make
sure we catch edge cases and glitches that will now be being seen with
more broad testing.
Robert N M Watson FreeBSD Core Team, TrustedBSD Projects
robert@fledge.watson.org Senior Research Scientist, McAfee Research
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.NEB.3.96L.1040126120627.11399N-100000>