From owner-freebsd-current@FreeBSD.ORG Mon Jan 26 09:15:15 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 16F4B16A4CE for ; Mon, 26 Jan 2004 09:15:15 -0800 (PST) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id D108C43D3F for ; Mon, 26 Jan 2004 09:15:11 -0800 (PST) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.10/8.12.10) with ESMTP id i0QHCwUd013556; Mon, 26 Jan 2004 12:12:58 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i0QHCwt8013553; Mon, 26 Jan 2004 12:12:58 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Mon, 26 Jan 2004 12:12:58 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Poul-Henning Kamp In-Reply-To: <67153.1075136001@critter.freebsd.dk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: current@freebsd.org Subject: Re: Hints for precision benchmarking... X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jan 2004 17:15:15 -0000 On Mon, 26 Jan 2004, Poul-Henning Kamp wrote: > * Run in single user mode. Cron(8) and and other daemons > only add noise. Of particular interest here is sshd -- either disable its SSHv1 key regeneration, or kill the parent sshd (or don't run sshd) during the tests. A few obvious ones just worth remembering: (1) Don't use bgfsck unless you want to benchmark with bgfsck. Also, disable background_fsck in rc.conf unless you plan not to start the benchmark for at least 60+fudge seconds after the boot, as rc wakes up and checks to see if anything needs fscking if it is enabled. Likewise, make sure you don't have snapshots lying around unless you mean to teset with snapshots. (2) If you must leave the system connected to a public network, watch out for spikes of broadcast traffic. Even though you don't notice it, it will eat your CPU. Multicast has similar caveats. (3) If you see unexpectedly bad performance, check for things like high interrupt volume from an unexpected source. I've seen several reports of ACPI "misbehaving" and generating excess interrupts under some of the ACPI code drops. If you're seeing odd results, take a few snapshots of vmstat -i and look for anything unusual. (4) Make sure to be careful about optimization parameters for kernel and userspace, likewise debugging. It's easy to let something slip through and realize later you weren't comparing the same thing. (5) And finally, do not ever benchmark with WITNESS and INVARIANTS unless you are really just interested in benchmarking those features. :-) I've seen 400%+ drops in performance from running with WITNESS. Likewise, userspace malloc paramters default differently in -CURRENT from the way they'd ship in production. Also, it would be very helpful if, while benchmarking, people would do the same benchmark with SCHED_ULE and SCHED_4BSD, and see what impact it has. Scheduler changes are very influential to performance, and we want to make sure we catch edge cases and glitches that will now be being seen with more broad testing. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research