Date: Thu, 8 Mar 2018 17:04:11 -0500 From: Arshan Khanifar <arshan@freebsdfoundation.org> To: freebsd-current@freebsd.org Subject: Performance Benchmark for PTI (aka Meltdown mitigation) Message-ID: <CADMyxqs93mbwxJB4C03LMfz5g3T45=n9NWBet_Wa22v4BO4LLA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Executive Summary:
- The PTI feature increases the system call times by more than 100%.
- As a macrobenchmark, buildworld was used. Wall clock and user time
showed no statistically-significant changes, while system time
increased by less than 5%.
This email contains the results for benchmarking the performance of the
PTI patch on FreeBSD 12-current. As a microbenchmark, timing of
getppid(2) system call was used, and as a macrobenchmark, a number of
buildworld tasks were timed.
The benchmark cases are as follows:
1. PTI off, PCID off
2. PTI off, PCID on
3. PTI on , PCID off
4. PTI on , PCID on
Since PCID is an optimization and is usually enabled by default, the
two most interesting cases above are the cases (2) and (4). Other cases
were measured for demonstration of PCID's effect on overall time.
The results section for each test is the output of ministat(1) for all
the data gathered for that test. Ministat compares the
first input file with all the other input files and reports the
difference. For more information, check out the man page of
ministat(1).
**********************************
REVISION: HEAD (b21ccf63f28)
**********************************
**************************
Benchmark:
Syscall microbenchmark from tools/tools/syscall_timing
./syscall_timing getppid
**************************
****************
Hardware: Packet's baremetal type 2
2 x Intel(R) Xeon(TM) CPU E5-2650 v4 (24 cores @ 2.20GHz)
256GB DDR4 ECC RAM
2.8TB SSD
****************
********
results:
********
x pti-off-pcid-on.log
+ pti-off-pcid-off.log
* pti-on-pcid-off.log
% pti-on-pcid-on.log
+------------------------------------------------------------------------+
| * % * |
| * % * |
| * % * |
| * % * |
| * %% ** |
| * +x + x %% % *O * *|
||_MA_|| |__M__A____| |___M___A_______| |
+------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 10 118 162 118 123.9 14.192721
+ 10 118 154 118 122.5 11.42366
No difference proven at 95.0% confidence
* 10 339 454 340 356.4 37.883447
Difference at 95.0% confidence
232.5 +/- 26.8779
187.651% +/- 29.8653%
(Student's t, pooled s = 28.6058)
% 10 262 342 262 272.2 25.301735
Difference at 95.0% confidence
148.3 +/- 19.2744
119.693% +/- 21.5323%
(Student's t, pooled s = 20.5135)
**********************************
REVISION: HEAD (c9cf79445478)
**********************************
**************************
Benchmark:
buildworld, invoked with this command:
make -C /usr/src -j48 buildworld > build.log 2>&1
**************************
****************
Hardware: Packet's baremetal type 2
2 x Intel(R) Xeon(TM) CPU E5-2650 v4 (24 cores @ 2.20GHz)
256GB DDR4 ECC RAM
2.8TB SSD
****************
********
results:
********
x real-pti-off-pcid-on.log
+ real-pti-off-pcid-off.log
* real-pti-on-pcid-off.log
% real-pti-on-pcid-on.log
+------------------------------------------------------------------------+
| x + % x % + % * * + x *|
||______________|_|___M_A__AMAM|___|____M_|_____|_A__________________| |
+------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 1677.08 1712.18 1691.4 1693.5533 17.6488
+ 3 1686.9 1706.72 1697.84 1697.1533 9.9278262
No difference proven at 95.0% confidence
* 3 1703.57 1729.55 1705.3 1712.8067 14.52593
No difference proven at 95.0% confidence
% 3 1688.69 1701.44 1695.96 1695.3633 6.3959075
No difference proven at 95.0% confidence
x user-pti-off-pcid-on.log
+ user-pti-off-pcid-off.log
* user-pti-on-pcid-off.log
% user-pti-on-pcid-on.log
+--------------------------------------------------------------------------+
| *+ x % % +x % * * *|
||_|__M____MA_A_|______M_|_A__________| |___MA_____||
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 50223.23 50409.49 50273.89 50302.203 96.303845
+ 3 50226.37 50401.75 50230.85 50286.323 99.98752
No difference proven at 95.0% confidence
* 3 50754.05 50842.26 50781.66 50792.657 45.121459
Difference at 95.0% confidence
490.453 +/- 170.45
0.975014% +/- 0.341564%
(Student's t, pooled s = 75.201)
% 3 50342.4 50535.65 50380.41 50419.487 102.37983
No difference proven at 95.0% confidence
x sys-pti-off-pcid-on.log
+ sys-pti-off-pcid-off.log
* sys-pti-on-pcid-off.log
% sys-pti-on-pcid-on.log
+--------------------------------------------------------------------------+
|+x + x+x % %% * * * |
||_|____A__A___M|__| |_____A_M__| |__________A_M_______||
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 2623.5 2658.68 2652.31 2644.83 18.74489
+ 3 2620.79 2654.58 2637.55 2637.64 16.89518
No difference proven at 95.0% confidence
* 3 2740.31 2787.22 2772.59 2766.7067 24.002026
Difference at 95.0% confidence
121.877 +/- 48.8099
4.60811% +/- 1.87816%
(Student's t, pooled s = 21.5345)
% 3 2710.22 2733.57 2730.79 2724.86 12.75458
Difference at 95.0% confidence
80.03 +/- 36.338
3.0259% +/- 1.40248%
(Student's t, pooled s = 16.032)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CADMyxqs93mbwxJB4C03LMfz5g3T45=n9NWBet_Wa22v4BO4LLA>