Date: Tue, 22 Aug 1995 22:24:50 -0700 From: John Dyson <dyson> To: hackers@freebsd.org Subject: Here are my benchmark results, so far Message-ID: <199508230524.WAA10428@freefall.FreeBSD.org>
next in thread | raw e-mail | index | archive | help
I hesitate to post this to the mailing list, but I don't have time
to refine these numbers. Permission is granted to use this information
in any way that you see fit. Note that I don't warrantee this information
at all. However, I have tried to be as fair and complete as I could be.
Caveat Emptor.... Here goes!!!!
Note that unless otherwise stated, all of these runs were done on the
FreeBSD stable tree as of about 20 Aug 95. The Linux runs were done with
the Slackware 2.3 binaries and the Linux 1.3.20 kernel. It appears that
the performance of 1.3.20 is roughly the same as the 1.2.8 kernel. The
Multi-block I/O mode was not enabled on the FreeBSD-stable kernel, and the
kernel built was the standard GENERIC. The Linux kernel was stripped
down to that which would run on my machine (no extra drivers, but the
hard-disk driver was the IDE one.)
These results are not cooked in any way and quite raw. The runs on
the FreeBSD-V2.1 stable kernel were done with a kernel as directly
checked out of CVS and the kernel was not hacked on. Any modifications
to the tests have been outlined below, and every attempt has been made
to make sure that the results are presented fairly. In some cases the
results are in error, and instead of trying to present "corrected" info,
the actual results with annotation are given. Some results with more
current or differently tuned FreeBSD kernels have been provided for
comparison purposes.
Note that Linux was compiled with:
"-O2 -fomit-frame-pointer -fno-strength-reduce -m486"
And unless otherwise noted FreeBSD was compiled with:
"-O"
MACHINE:
486/66 DX2 Micronics ISA MB 20MB -- stock, no mods.
IDE drives: WD540H (128K cache version), IDE interface is
a cheap paddleboard.
---------------------------------------------------------
The following benchmarks measure the page fault performance
of a UNIX-like OS. The performance measured includes the fault
code itself and the paging-in code.
Checking fault performance on FreeBSD (gaussian)
Memory is now initialized, starting test
100 mmaps, 512 pages, 2560 pages touched per mmap
REAL TIME:
paging in time: 1695.8500 ms
average time: 49.4660 us
average time exc. paging in time: 16.5090 us
SYSTEM & USER TIME:
system time: 581.3000 ms
user time: 175.4230 ms
Checking fault performance on Linux (gaussian)
Memory is now initialized, starting test
100 mmaps, 512 pages, 2560 pages touched per mmap
REAL TIME:
paging in time: 8192.3850 ms
average time: 381.9265 us
average time exc. paging in time: 224.1606 us
SYSTEM & USER TIME:
system time: 9610.0000 ms
user time: 250.0000 ms
Checking fault performance on FreeBSD (backwards)
Memory is now initialized, starting test
100 mmaps, 512 pages, 2560 pages touched per mmap
REAL TIME:
paging in time: 2381.1970 ms
average time: 60.2987 us
average time exc. paging in time: 13.9302 us
SYSTEM & USER TIME:
system time: 614.2340 ms
user time: 210.0920 ms
Checking fault performance on Linux (gaussian)
Memory is now initialized, starting test
100 mmaps, 512 pages, 2560 pages touched per mmap
REAL TIME:
paging in time: 8192.3850 ms
average time: 381.9265 us
average time exc. paging in time: 224.1606 us
SYSTEM & USER TIME:
system time: 9610.0000 ms
user time: 250.0000 ms
Note that Linux has much slower paging and also
much slower page faulting than FreeBSD.
---------------------------------------------------------------
The following is the well-known IOZONE benchmark, with fsync
enabled:
FREEBSD V2.1-stable (no MULTI-BLOCK, normal config and build):
IOZONE: Performance Test of Sequential File I/O -- V2.01 (10/21/94)
By Bill Norcott
Operating System: POSIX 1003.1-1990 -- using fsync()
IOZONE: auto-test mode
MB reclen bytes/sec written bytes/sec read
1 512 1220161 6100805
1 1024 1508064 8388608
1 2048 1560671 11184810
1 4096 1290555 12201611
1 8192 1597830 13421772
2 512 1348921 5965232
2 1024 1626881 8659208
2 2048 1698958 10324440
2 4096 1731841 12201611
2 8192 1698958 13421772
4 512 1332185 5478274
4 1024 1516584 7354396
4 2048 1593088 8801162
4 4096 1579032 9942053
4 8192 1612224 10956549
8 512 1319093 5506368
8 1024 1569797 7405116
8 2048 1619520 8947848
8 4096 1612224 9942053
8 8192 1593088 10956549
16 512 1372194 1626881
16 1024 1544952 1607397
16 2048 1654455 1770390
16 4096 1668596 1837026
16 8192 1653182 1824540
Completed series of tests
FreeBSD V2.2-experimental compiled with
"-O2 -fomit-frame-pointer -fno-strength-reduce -m486, and MULTI-BLOCK"
(This is similar to the way Linux 1.3.20 is compiled)
IOZONE: Performance Test of Sequential File I/O -- V2.01 (10/21/94)
By Bill Norcott
Operating System: POSIX 1003.1-1990 -- using fsync()
IOZONE: auto-test mode
MB reclen bytes/sec written bytes/sec read
1 512 1458888 5592405
1 1024 1720740 6391320
1 2048 1890390 12201611
1 4096 2033601 14913080
1 8192 2033601 16777216
2 512 1533916 5835553
2 1024 1864135 8947848
2 2048 2130440 12201611
2 4096 2218474 14913080
2 8192 2255760 17895697
4 512 1529546 5263440
4 1024 1877171 7780737
4 2048 2122019 10129639
4 4096 2265278 12201611
4 8192 2462710 13421772
8 512 1529546 5289368
8 1024 1731841 7780737
8 2048 1992099 10129639
8 4096 2060924 11930464
8 8192 2134675 13256071
16 512 1495462 1870630
16 1024 1846503 2078880
16 2048 2062904 2184622
16 4096 2191309 2246321
16 8192 2138927 2282129
Completed series of tests
Linux 1.3.20:
IOZONE: Performance Test of Sequential File I/O -- V2.01 (10/21/94)
By Bill Norcott
Operating System: POSIX 1003.1-1990 -- using fsync()
IOZONE: auto-test mode
MB reclen bytes/sec written bytes/sec read
1 512 1565038 4559026
1 1024 1777247 6990506
1 2048 1691251 7489828
1 4096 1718977 8065969
1 8192 1718977 8065969
2 512 1436405 4462025
2 1024 1407484 6765006
2 2048 1487341 7231558
2 4096 1446311 7767229
2 8192 1487341 8065969
4 512 1536375 4369066
4 1024 1651300 6657625
4 2048 1600879 7358428
4 4096 1594792 7626007
4 8192 1644825 7767229
8 512 1613193 4279902
8 1024 1607013 6657625
8 2048 1553445 7294441
8 4096 1594792 7626007
8 8192 1701543 7839820
16 512 1585748 922839
16 1024 1556328 1179003
16 2048 1614746 1290555
16 4096 1662756 1305619
16 8192 1638400 1246449
Completed series of tests
Linux 1.3.20 with MULTI-BLOCK 16 enabled:
IOZONE: Performance Test of Sequential File I/O -- V2.01 (10/21/94)
By Bill Norcott
Operating System: POSIX 1003.1-1990 -- using fsync()
IOZONE: auto-test mode
MB reclen bytes/sec written bytes/sec read
1 512 1691251 4559026
1 1024 1807889 6990506
1 2048 1872457 7489828
1 4096 1839607 7489828
1 8192 1807889 8065969
2 512 1664406 4369066
2 1024 1855886 6553600
2 2048 1855886 7489828
2 4096 1664406 7767229
2 8192 1889326 8065969
4 512 1607013 4324024
4 1024 1565038 6765006
4 2048 1588751 7358428
4 4096 1607013 7767229
4 8192 1542023 7108989
8 512 1628855 4324024
8 1024 1740375 6657625
8 2048 1784810 7358428
8 4096 1804001 7626007
8 8192 1788615 7839820
16 512 1747626 954335
16 1024 1760463 985156
16 2048 1798201 1040770
16 4096 1777247 1213103
16 8192 1798201 1493964
Completed series of tests
NOTE:
Linux appears to have less overhead on write system calls, but
significantly higher overhead in reading buffer cache data. Note also, that
when Linux actually has to read from the disk drive, the performance suffers
significantly. Note also, that MULTI-BLOCK I/O doesn't appear to help
the performance of the Linux disk I/O quite as much as it helps FreeBSD.
------------------------------------------------------------
Below are the Bonnie results:
FreeBSD V2.1 - stable
File './Bonnie.4152', size: 20971520
Writing with putc()...done
Rewriting...done
Writing intelligently...done
Reading with getc()...done
Reading intelligently...done
Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
20 949 63.0 1711 16.6 819 14.5 946 61.5 1833 15.4 111.8 10.5
FreeBSD V2.2 - experimental running on J. Dyson's machine, with MULTI-BLOCK
enabled "-O2 -fno-strength-reduce":
File './Bonnie.417', size: 20971520
Writing with putc()...done
Rewriting...done
Writing intelligently...done
Reading with getc()...done
Reading intelligently...done
Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
20 1035 71.3 2221 18.9 1050 14.7 1023 63.8 2233 16.9 59.9 6.9
Linux 1.3.20:
File './Bonnie.7210', size: 20971520
Writing with putc()...done
Rewriting...done
Writing intelligently...done
Reading with getc()...done
Reading intelligently...done
Seeker 3...Seeker 2...Seeker 1...start 'em...done...done...done...
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
20 1055 89.5 2066 64.8 695 43.6 626 65.1 1326 68.7 235.1 47.1
NOTE: The very high random seeks and the sequential output perf are probably
due to the asynchronous writes that Linux uses. Note also that Linux appears
to use much more CPU for I/O operations. (It is probable, however, that the
FreeBSD version of bonnie does not properly measure the overhead of IDE PIO
operations.) If MULTI-BLOCK I/O is enabled, FreeBSD can easily surpass
the write performance of the Linux results above, but
the MULTI-BLOCK I/O mode has not been enabled for this Linux test.
------------------------------------------------------------------
FreeBSD Linux
Null syscall: 20 8 usecs
^^ Really a write system call to /dev/null,
not really null.
Getpid syscall: 7.99 5.57 usecs
Pipe latency: 284 79 usecs
UDP latency: 613 643 usecs
TCP latency: 689 1353 usecs
RPC/udp latency: 1117 1088 usecs
RPC/tcp latency: 1181 1967 usecs
TCP/IP connection: 1108 1550 usecs
Socket using localhost: 5.11 2.08 MB/sec
(A minor mod needed to be made to the socket using localhost benchmark
for FreeBSD to run the test correctly.)
USING STATIC PROGRAMS:
Process fork+exit: 3249 4380 usecs
Process fork+execve: 6838 9365 usecs
Process fork+/bin/sh -c: 27156 44115 usecs
USING STATIC PROGRAMS, FreeBSD compiled with -fomit-frame-pointer:
Process fork+exit: 2821 usecs
Process fork+execve: 5917 usecs
Process fork+/bin/sh -c: 25382 usecs
LINKED WITH SUNOS STYLE SHARED LIBS:
FreeBSD Linux
Process fork+exit: 6162 usecs -----
Process fork+execve: 27716 usecs -----
Process fork+/bin/sh -c: 49842 usecs -----
FreeBSD Compiled with -fomit-frame-pointer
Process fork+exit: 5716 usecs -----
Process fork+execve: 26356 usecs -----
Process fork+/bin/sh -c: 48124 usecs -----
LINKED WITH OLD-STYLE JUMP-TABLE SHARED LIBS
FreeBSD Linux
Process fork+exit: ------ 6629 usecs
Process fork+execve: ------ 19877 usecs
Process fork+/bin/sh -c: ------ 58735 usecs
Pipe bandwidth: 6.82 12.33 MB/sec
^^^^ Pipe buffer size being increased in V2.2,
this result will be close to 8-9MB.
File bandwidth: 1730 1734 KB/sec
Pagefaults on /tmp/XXX: 67 228 usecs
It appears that compiling FreeBSD with -fomit-frame-pointer doesn't help
much, but people in very time critical applications might want to compile
the kernel with this flag. However, by supplying this flag, kernel tracebacks
are frustrating.
---------------------------------------------------
mmap times in microseconds:
FreeBSD Linux
0.01 132 78
0.25 536 78
0.50 749 77
1.00 1092 77
2.00 1845 78
^^^^ Freebsd doesn't preload segments > 2MB (most shared libs
are smaller than that.)
3.00 117 75
4.00 114 77
5.00 125 79
6.00 147 79
7.00 117 80
8.00 133 79
Mmap times are not that critical, and the future pagefault times are spead
up signficantly by the methods used in FreeBSD.
-------------------------------------------------------------
File read bandwidth
FreeBSD Linux
0.02 2.35 2.25
0.03 5.09 4.58
0.06 9.93 9.35
0.12 12.37 10.18
0.19 7.96 10.12
^^^^^ GLITCH
0.25 13.33 9.80
0.38 12.68 8.95
0.50 10.27 8.61
0.75 10.60 8.29
1.00 13.91 8.15
1.50 13.85 8.06
2.00 12.59 8.03
2.50 13.04 8.03
3.00 10.02 7.96
3.50 10.36 7.94
4.00 11.16 7.94
5.00 11.18 7.91
6.00 10.38 7.95
7.00 11.34 7.95
8.00 11.25 7.94
Mmap read bandwidth:
FreeBSD Linux
0.02 26.66 7.88
0.03 27.70 8.44
0.06 26.65 7.91
0.12 27.04 7.58
0.19 25.34 7.38
0.25 24.64 7.07
0.38 21.48 6.64
0.50 21.42 6.48
0.75 21.66 6.35
1.00 22.74 6.31
1.50 22.61 6.63
2.00 22.23 6.60
2.50 15.43 6.82
3.00 15.43 6.69
3.50 15.61 6.59
4.00 15.64 6.75
5.00 15.40 6.11
6.00 15.94 6.07
7.00 15.36 5.85
8.00 15.59 5.58
Libc bcopy aligned
FreeBSD Linux
0.0625 28.05 19.33
0.1250 21.36 24.10
0.5000 16.17 16.57
1.0000 15.27 15.41
2.0000 15.17 15.45
4.0000 15.25 15.49
8.0000 2.57 0.17
Libc bcopy unaligned
FreeBSD Linux
0.0625 27.26 24.32
0.1250 23.61 21.08
0.5000 16.97 17.57
1.0000 16.37 16.64
2.0000 16.33 16.64
4.0000 16.33 16.50
8.0000 2.84 0.18
Unrolled bcopy aligned
FreeBSD Linux
0.0625 11.93 26.68
0.1250 11.83 21.33
0.5000 11.54 16.78
^^^^^ DATA PROBLEM, PROBABLY SOME DAEMON DECIDED TO DO SOMETHING
ADDITIONAL MEASUREMENTS HAVEN'T SHOWN THIS PROBLEM.
1.0000 15.41 15.58
2.0000 15.37 15.58
4.0000 14.78 15.56
8.0000 1.34 0.18
Unrolled bcopy unaligned
FreeBSD Linux
0.0625 25.48 7.34
0.1250 21.44 7.46
0.5000 16.50 6.61
1.0000 15.41 6.53
2.0000 15.13 6.50
4.0000 14.73 6.51
^^^^ REALLY ODD... I DON'T KNOW ABOUT
THESE NUMBERS..., SHOULD BE ABOUT
15MB/sec. Subsequent runs verify
that these measurements are wrong.
8.0000 1.29 0.18
Note that Linux does not appear to be able to run as nicely when memory starved.
The 8MB cp results show the effects of paging (thrashing.) FreeBSD can
manifest similar behaviour, but usually at a higher memory load.
-----------------------------------------------------------
File system latency
FreeBSD:
0k 1000 39 90
1k 1000 35 36
4k 1000 34 36
10k 1000 27 36
FreeBSD-2.2-experimental with async meta-writes turned on, much safer than
pure async in Linux case below:
0k 1000 219 228
1k 1000 133 86
4k 1000 99 186
10k 1000 59 161
Linux:
0k 1000 243 2065
1k 1000 197 2265
4k 1000 163 2025
10k 1000 117 1637
Note that the ASYNC METADATA and FILE I/O REALLY SHOWS UP HERE!!!!
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199508230524.WAA10428>