Date: Mon, 2 Dec 1996 22:35:00 -0600 (CST) From: Joe Greco <jgreco@brasil.moneng.mei.com> To: proff@suburbia.net (Julian Assange) Cc: hackers@freebsd.org Subject: Re: TCP/IP bandwidth bragging Message-ID: <199612030435.WAA25907@brasil.moneng.mei.com> In-Reply-To: <199612030420.PAA15467@suburbia.net> from "Julian Assange" at Dec 3, 96 03:20:06 pm
next in thread | previous in thread | raw e-mail | index | archive | help
> > Look, folks, benchmarks are benchmarks. They are not > > real world performance indicators. They are simply > > relative artificial performance evaluators, and as > > such can be influenced by a wide variety of factors, > > including OS tweaks. I never make the mistake of > > taking a benchmark's results as an absolute comparison > > of apples and oranges. > > Bench marks are not totally useless. Quite often it is hard to find > bottle necks without them. "The system feels slower" isn't going > to do you much good as a diagnostic tool with a monolithic kernel. > This does not imply that they find all bottle-necks. Hi Julian, Well, of course! That goes without saying. Benchmarks exist because they are useful in many cases. In fact, benchmarks are particularly significant when used on the same OS and hardware platform. Benchmarks are less significant when comparing different OS's on the same platform, or the same OS on different platforms, without some careful analysis and interpretation of the results. (I think John Dyson has repeatedly talked about this). I do not think benchmarks are useless. They certainly do an excellent job of testing performance under artificial circumstances, and for many purposes, this is useful information. However, when comparing across operating systems, I would tend to agree that John Dyson is correct when stating that a direct comparison may not be particularly meaningful. I will confess that I typically use a very unscientific set of "benchmarks" to evaluate a new platform and to give me a rough idea how it measures up to what I already know. This is sufficient for me because often I am only interested in order-of-magnitude comparisons. I am aware of this, and I interpret the results accordingly. ... JG
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199612030435.WAA25907>