From owner-freebsd-current@FreeBSD.ORG Wed Apr 7 18:19:02 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0228316A4CE for ; Wed, 7 Apr 2004 18:19:02 -0700 (PDT) Received: from ozlabs.org (ozlabs.org [203.10.76.45]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7729443D1D for ; Wed, 7 Apr 2004 18:19:01 -0700 (PDT) (envelope-from grog@lemis.com) Received: from blackwater.lemis.com (blackwater.lemis.com [192.109.197.80]) by ozlabs.org (Postfix) with ESMTP id 89EF22BD90 for ; Thu, 8 Apr 2004 11:18:59 +1000 (EST) Received: by blackwater.lemis.com (Postfix, from userid 1004) id 9B8955121E; Thu, 8 Apr 2004 10:48:57 +0930 (CST) Date: Thu, 8 Apr 2004 10:48:57 +0930 From: Greg 'groggy' Lehey To: Ivan Voras Message-ID: <20040408011857.GR23860@wantadilla.lemis.com> References: <40745C07.6030501@fer.hr> <877jwre672.fsf@strauser.com> <40748CD0.5020204@fer.hr> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="yyDlEBSQJ3H1meLT" Content-Disposition: inline In-Reply-To: <40748CD0.5020204@fer.hr> User-Agent: Mutt/1.4.1i Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 cc: current@freebsd.org Subject: Re: Benchmarking X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2004 01:19:02 -0000 --yyDlEBSQJ3H1meLT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thursday, 8 April 2004 at 1:20:48 +0200, Ivan Voras wrote: > Kirk Strauser wrote: > >> Prepare to be flamed (not by me, but in general). The tests are littered >> with comments like: >> >> Missing data in the above table signifies operations that were too fast >> to measure correctly by the bonnie++ program. > >> People will wonder why you chose not to increase the number of files being >> tested so as to get meaningful results. Viewers are left looking at a >> mostly-empty table where the BSDs dominate the results by a large margin, >> but are told: > > Id really didn't occurr to me until it was already to late (too many > systems were benchmarked already). This sounds like another way of saying "I published before I had finished my strategy". You should consider retesting. > >> Linux clearly wins the IO throughput test, having a score upto about >> 130% better than nearest BSD, either by having a better SCSI driver, or >> because the system itself is just faster. >> >> Since you didn't *publish* any results were Linux won, I'll have to assume >> that your private data demonstrated a different conclusion. > > And what do you call the results of bonnie++ sequential IO output test > (in the table)? :) This is what I was reffering to in the cited > text. Agreed, it looks as if Linux is much better there. But the per character tests should have made you wonder what's going on here. Linux can't possibly be 100 times as fast as FreeBSD, and NetBSD can't be 4 times as fast again. I'd assume that you've got some effect related to how the character I/O is compiled. I saw similar (but opposite :-) effects with bonnie a couple of years ago (see http://www.lemis.com/grog/diary-nov2000.html#30). As I commented at the time, something else must be causing the problem. This is one of the reasons why I don't like bonnie(++) as a benchmark. >> Basically, you ran some tests on divergent systems and got some >> results, but that's about the only conclusion I was able to get >> from it. > > You're perfectly entitled to :) You have the beginnings of something interesting here, but I have the feeling you're not done yet. It would be interesting to examine the obvious differences and investigate what's really causing them. Greg -- Note: I discard all HTML mail unseen. Finger grog@FreeBSD.org for PGP public key. See complete headers for address and phone numbers. --yyDlEBSQJ3H1meLT Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.0 (FreeBSD) iD8DBQFAdKiBIubykFB6QiMRAviTAKCJM0JMdv0V8KH/CgK0msBlEATY2wCfTaN5 wE2nwpj3CQHaa1EDDQ8b1J0= =0eYt -----END PGP SIGNATURE----- --yyDlEBSQJ3H1meLT--