Date: Thu, 23 Nov 2006 14:25:16 +1100 (EST) From: Bruce Evans <bde@zeta.org.au> To: Alan Cox <alc@cs.rice.edu> Cc: Kip Macy <kip.macy@gmail.com>, arch@FreeBSD.org Subject: Re: superpage plans Message-ID: <20061123124725.P35210@delplex.bde.org> In-Reply-To: <45649E42.70409@cs.rice.edu> References: <b1fa29170611220939g32469638ncf3a3ddd4bba3670@mail.gmail.com> <45649E42.70409@cs.rice.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 22 Nov 2006, Alan Cox wrote:
> There is only one caveat. Idle-time page prezeroing is not supported.
> However, ever since the VM system emerged from the Giant kernel lock, I've
> seen little or no benefit from it. ...
That's probably because PREEMPTION is broken and the brokenness turns
idle-time page prezeroing into a pessimization. Without PREEMPTION I
see much the same benefits from idle-time page prezeroing as in RELENG_4
-- a speedup of a few percent for makeworld. E.g., for makeworld -j4
of a RELENG_4 with a -current i386 SMP kernel and a ~5.2 userland on
a Turion X2 2GHz:
%%%
vm.idlezero_enable=0:
--------------------------------------------------------------
>>> elf make world completed on Thu Nov 23 12:01:56 EST 2006
(started Thu Nov 23 11:50:59 EST 2006)
--------------------------------------------------------------
656.36 real 815.59 user 194.92 sys
23572 maximum resident set size
1164 average shared memory size
1212 average unshared data size
128 average unshared stack size
14202993 page reclaims
6911 page faults
0 swaps
14686 block input operations
4647 block output operations
77645 messages sent
0 messages received
35459 signals received
838638 voluntary context switches
391631 involuntary context switches
vm.idlezero_enable=1:
--------------------------------------------------------------
>>> elf make world completed on Thu Nov 23 12:35:54 EST 2006
(started Thu Nov 23 12:25:07 EST 2006)
--------------------------------------------------------------
647.19 real 814.16 user 185.69 sys
23572 maximum resident set size
1168 average shared memory size
1220 average unshared data size
128 average unshared stack size
14202807 page reclaims
6958 page faults
0 swaps
14534 block input operations
4689 block output operations
77466 messages sent
0 messages received
35456 signals received
847575 voluntary context switches
397783 involuntary context switches
%%%
With idlezero enabled and PREEMPTION not enabled in the above, pgzero
runs in actual idle time for 14 seconds and reduces both the real and
sys times by 9 seconds (1.5% of real time and 5% of system time).
With idlezero enabled and PREEMPTION enabled (details not shown),
PREEMPTION doesn't actually work but pgzero depends on it working, so
pgzero runs for much longer than in the above, with all the extra time
stolen from non-idle time. In my makeworld benchmarks, this gives
total benefits that are negative and about the same magnitude as the
postive ones without PREEMPTION. PREEMPTION gives some other negative
benefits for makeworld, but the others are smaller, at least without
any userland idle priority threads that want to run all the time.
The system for the above tests has a fairly large write bandwidtth
(5GB/sec for movnt*) so it benefits from idle-time page prezeroing
less than most systems. I've seen it taking and saving 3% of the time
for makeworld (60 seconds out of 1800) on UP systems with similar CPU
speeds but slower memory and pagezero() not optimized to use movnt*.
UP systems benefit less than SMP ones since they have a lower percentage
of idle time.
Of course the possible savings are less if the system is less often
idle, but makeworld -j4 on an SMP system leaves a lot of time idle,
especially when it runs mkdep and perl serially. I don't know if
pgzero is running mainly in bursts in the time left idle by mkdep in
the above, but guess not since it limits itself to not zeroing very
many pages to avoid thrashing caches. Perhaps it should not limit
itself so much when the zeroing is nontemporal.
Bruce
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20061123124725.P35210>