Date: Thu, 8 Apr 2004 10:48:57 +0930 From: Greg 'groggy' Lehey <grog@FreeBSD.org> To: Ivan Voras <ivoras@fer.hr> Cc: current@freebsd.org Subject: Re: Benchmarking Message-ID: <20040408011857.GR23860@wantadilla.lemis.com> In-Reply-To: <40748CD0.5020204@fer.hr> References: <40745C07.6030501@fer.hr> <877jwre672.fsf@strauser.com> <40748CD0.5020204@fer.hr>
next in thread | previous in thread | raw e-mail | index | archive | help
--yyDlEBSQJ3H1meLT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thursday, 8 April 2004 at 1:20:48 +0200, Ivan Voras wrote: > Kirk Strauser wrote: > >> Prepare to be flamed (not by me, but in general). The tests are littered >> with comments like: >> >> Missing data in the above table signifies operations that were too fast >> to measure correctly by the bonnie++ program. > >> People will wonder why you chose not to increase the number of files being >> tested so as to get meaningful results. Viewers are left looking at a >> mostly-empty table where the BSDs dominate the results by a large margin, >> but are told: > > Id really didn't occurr to me until it was already to late (too many > systems were benchmarked already). This sounds like another way of saying "I published before I had finished my strategy". You should consider retesting. > >> Linux clearly wins the IO throughput test, having a score upto about >> 130% better than nearest BSD, either by having a better SCSI driver, or >> because the system itself is just faster. >> >> Since you didn't *publish* any results were Linux won, I'll have to assume >> that your private data demonstrated a different conclusion. > > And what do you call the results of bonnie++ sequential IO output test > (in the table)? :) This is what I was reffering to in the cited > text. Agreed, it looks as if Linux is much better there. But the per character tests should have made you wonder what's going on here. Linux can't possibly be 100 times as fast as FreeBSD, and NetBSD can't be 4 times as fast again. I'd assume that you've got some effect related to how the character I/O is compiled. I saw similar (but opposite :-) effects with bonnie a couple of years ago (see http://www.lemis.com/grog/diary-nov2000.html#30). As I commented at the time, something else must be causing the problem. This is one of the reasons why I don't like bonnie(++) as a benchmark. >> Basically, you ran some tests on divergent systems and got some >> results, but that's about the only conclusion I was able to get >> from it. > > You're perfectly entitled to :) You have the beginnings of something interesting here, but I have the feeling you're not done yet. It would be interesting to examine the obvious differences and investigate what's really causing them. Greg -- Note: I discard all HTML mail unseen. Finger grog@FreeBSD.org for PGP public key. See complete headers for address and phone numbers. --yyDlEBSQJ3H1meLT Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.0 (FreeBSD) iD8DBQFAdKiBIubykFB6QiMRAviTAKCJM0JMdv0V8KH/CgK0msBlEATY2wCfTaN5 wE2nwpj3CQHaa1EDDQ8b1J0= =0eYt -----END PGP SIGNATURE----- --yyDlEBSQJ3H1meLT--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040408011857.GR23860>