Date: Sun, 14 Apr 1996 17:24:23 +0000 From: se@x14.mi.uni-koeln.de (Stefan Esser) To: Gerard Roudier <groudier@iplus.fr> Cc: hackers@freebsd.org, linux-kernel@vger.rutgers.edu Subject: Re: Unices are created equal, but ... Message-ID: <199604141524.RAA03026@x14.mi.uni-koeln.de> In-Reply-To: Gerard Roudier <groudier@iplus.fr> "Unices are created equal, but ..." (Apr 13, 22:44)
next in thread | previous in thread | raw e-mail | index | archive | help
On Apr 13, 22:44, Gerard Roudier wrote:
} Subject: Unices are created equal, but ...
}
} Hi all,
Hi Gerard!
I've always appreciated your (well founded!) comments
and criticism on the BSD NCR driver which that you
ported to Linux, but the more I'm surprised to see
such a biased and unfair benchmark posted by you :(
Since I think your conclusions are utterly wrong, I
have appended my results obtained on a 16MB RAM ASUS
SP3G (i.e. a significantly slower system than your
Pentium).
The (old) BYTE benchmark is not a suitable performance
indicator at all! I've made it a port under BSD, since
it can show misconfiguration and problem areas, and I'd
like to know, whether you at least used that version
(available as a "port" or "package" for easy installation).
Some of the tests are bogus (e.g. DC which requires just
7 milliseconds (!!!) of user time on my 486 system ...).
Nearly all its execution time is runtime binding, and a
statically linked version runs seems to perform nearly
an order of magnitude better.
If you used the pre-compiled binaries from the Silkroad
web server, then the comparison is unfair for another
reason: Those binaries were compiled using a very complex
set of options, they are old "a.out" binaries, and some
of the programs executed in the test loops are statically
linked for faster loading (no bind phase at startup).
I've asked people who sent me Linux byte bench results
to try another time with binaries compiled on their
systems, and this is what I got back:
ASUS SP3G, 32MB of RAM, Linux 1.3.59:
Pre-compiled binaries:
Arithmetic Test (type = double) 2541.7 6094.5 2.4
Dhrystone 2 without register variables 22366.3 58554.6 2.6
Execl Throughput Test 16.5 412.6 25.0
File Copy (30 seconds) 179.0 3714.0 20.7
Pipe-based Context Switching Test 1318.5 19639.5 14.9
Shell scripts (8 concurrent) 4.0 14.0 3.5
=========
SUM of 6 items 69.2
AVERAGE 11.5
GCC-2.7.2 compiled binaries (options: -O2 -m486):
Arithmetic Test (type = double) 2541.7 7599.1 3.0
Dhrystone 2 without register variables 22366.3 85827.8 3.8
Execl Throughput Test 16.5 61.2 3.7
File Copy (30 seconds) 179.0 3767.0 21.0
Pipe-based Context Switching Test 1318.5 18626.1 14.1
Shell scripts (8 concurrent) 4.0 14.0 3.5
=========
SUM of 6 items 49.2
AVERAGE 8.2
See the difference ?
This is no Linux-baching, mind you !
It just shows, that the byte benchmark results can
depend to a very high extend on details other than
the operating system used.
I've abridged your report to just those lines that
contain the summary data.
} I get the following results:
}
} P90/Plato/24MB/NCR53C810/IBMS12.
}
} BYTE UNIX Benchmarks (Version 3.11)
} System -- Unix A gerard 1.3.87 #31 Sat Apr 13 18:34:46 GMT 1996 i586
} INDEX VALUES
} TEST BASELINE RESULT INDEX
}
} Arithmetic Test (type = double) 2541.7 15946.0 6.3
} Dhrystone 2 without register variables 22366.3 121950.7 5.5
} Execl Throughput Test 16.5 267.6 16.2
} File Copy (30 seconds) 179.0 3441.0 19.2
} Pipe-based Context Switching Test 1318.5 22788.7 17.3
} Shell scripts (8 concurrent) 4.0 36.0 9.0
} =========
} SUM of 6 items 73.5
} AVERAGE 12.2
}
} BYTE UNIX Benchmarks (Version 3.11)
} System -- Unix B gerard 2.0.5-RELEASE XXXXXXXXXXXX: Fri Oct 20 00:30:52 1995 gerard:/usr/src/sys/compile/GERARD i386
} INDEX VALUES
} TEST BASELINE RESULT INDEX
}
} Arithmetic Test (type = double) 2541.7 17775.0 7.0
} Dhrystone 2 without register variables 22366.3 130585.3 5.8
} Execl Throughput Test 16.5 68.1 4.1
} File Copy (30 seconds) 179.0 3431.0 19.2
} Pipe-based Context Switching Test 1318.5 6304.2 4.8
} Shell scripts (8 concurrent) 4.0 15.0 3.8
} =========
} SUM of 6 items 44.7
} AVERAGE 7.4
And now my ASUS SP3G results (AMD 5x86, 16MB of RAM, NCR SCSI):
BYTE UNIX Benchmarks (Version 3.11)
System -- x14
INDEX VALUES
TEST BASELINE RESULT INDEX
Arithmetic Test (type = double) 2541.7 10932.5 4.3
Dhrystone 2 without register variables 22366.3 109051.0 4.9
Execl Throughput Test 16.5 359.8 21.8
File Copy (30 seconds) 179.0 4840.0 27.0
Pipe-based Context Switching Test 1318.5 13687.1 10.4
Shell scripts (8 concurrent) 4.0 14.0 3.5
=========
SUM of 6 items 71.9
AVERAGE 12.0
You don't have to compare them to your P90 values under
Linux. Just compare with the FreeBSD-2.0.5 results and
see how much difference one year of development effort
can make ... :)
} Even if this benchmark is a little questionnable, I invite people who say or
} write that Unix B is FASTER than Unix A to stop, or to say or write
} the OPPOSITE.
Sorry, no, not at all.
Please post significant benchmark results next time.
Your report didn't even include the minimum information
required for a submission to the Silkroad.Com byte-bench
Web pages ...
Regards, STefan
--
Stefan Esser, Zentrum fuer Paralleles Rechnen Tel: +49 221 4706021
Universitaet zu Koeln, Weyertal 80, 50931 Koeln FAX: +49 221 4705160
==============================================================================
http://www.zpr.uni-koeln.de/~se <se@ZPR.Uni-Koeln.DE>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199604141524.RAA03026>