Date: Sun, 12 Nov 2006 18:27:16 -0800 From: "Kip Macy" <kip.macy@gmail.com> To: "Alan Cox" <alc@freebsd.org> Cc: cvs-src@freebsd.org, src-committers@freebsd.org, cvs-all@freebsd.org Subject: Re: cvs commit: src/sys/amd64/amd64 pmap.c src/sys/arm/arm pmap.c src/sys/i386/i386 pmap.c src/sys/ia64/ia64 pmap.c src/sys/powerpc/powerpc mmu_oea.c src/sys/sparc64/sparc64 pmap.c src/sys/sun4v/sun4v pmap.c src/sys/vm vm_fault.c vm_kern.c Message-ID: <b1fa29170611121827r2d84ce69jc328a08b5cdd1fb0@mail.gmail.com> In-Reply-To: <200611122148.kACLmYDV006108@repoman.freebsd.org> References: <200611122148.kACLmYDV006108@repoman.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Thanks Alan, this is a definite improvement.
For the benefit of other readers: below I have the ten locks with longest
max_wait (total time in microseconds spent waiting for the lock) during the
compile phase of a "make -j32 buildkernel" on a T2000 (8 cores w/ 4 threads
per core). The reduction in turnstile contention is a direct result of
reducing contention on the page queue mutex.
before:
max total wait_total count avg wait_avg cnt_hold
cnt_lock name
5 7266196 206805560 7522452 0 27
48266619 0
/usr/flatstor/shared/freebsd/kmacy/src/sys/kern/subr_turnstile.c:487
(turnstile chain)
457 528521 180592127 550284 0 328
1469872 0
/usr/flatstor/shared/freebsd/kmacy/src/sys/vm/vm_fault.c:844 (vm page queue
mutex)
15057461 1679582934 117520488 87978 19090 1335
0 0
/usr/flatstor/shared/freebsd/kmacy/src/sys/ufs/ffs/ffs_vnops.c:366 (ufs)
214 1076256 112489341 559032 1 201
1520471 0
/usr/flatstor/shared/freebsd/kmacy/src/sys/vm/vm_fault.c:902 (vm page queue
mutex)
424 8250360 105249196 559031 14 188
1767340 0
/usr/flatstor/shared/freebsd/kmacy/src/sys/sun4v/sun4v/pmap.c:956 (vm page
queue mutex)
72563452121 218316084315 94216669 452713 482239 208
0 0
/usr/flatstor/shared/freebsd/kmacy/src/sys/kern/vfs_default.c:263 (nfs)
23 1349030 14049785 280685 4 50
923679 0
/usr/flatstor/shared/freebsd/kmacy/src/sys/kern/kern_idle.c:121 (sched lock)
73 214117 11078161 63944 3 173
2505 0
/usr/flatstor/shared/freebsd/kmacy/src/sys/nfsclient/nfs_socket.c:1235
(Giant)
42 92768 10431233 40012 2 260
122966 0
/usr/flatstor/shared/freebsd/kmacy/src/sys/vm/vm_fault.c:342 (vm page queue
mutex)
6429 2155581 6645086 18297 117 363
105550 0
/usr/flatstor/shared/freebsd/kmacy/src/sys/vm/vm_object.c:651 (vm page queue
mutex)
after:
15311133 1125368615 119287635 94194 11947 1266
0 0
/usr/flatstor/shared/freebsd/kmacy/src/sys/ufs/ffs/ffs_vnops.c:366
(lockmgr:ufs)
546 4708767 93794515 326286 14 287 533239
713529 /usr/flatstor/shared/freebsd/kmacy/src/sys/sun4v/sun4v/pmap.c:955
(sleep mutex:vm page queue mutex)
36217304 520567907 53857789 243928 2134 220
0 0
/usr/flatstor/shared/freebsd/kmacy/src/sys/kern/vfs_default.c:263
(lockmgr:nfs)
210 590989 38380260 326286 1 117 335955
160913 /usr/flatstor/shared/freebsd/kmacy/src/sys/vm/vm_fault.c:899 (sleep
mutex:vm page queue mutex)
16 1702987 25490112 2068283 0 12
7329498 0
/usr/flatstor/shared/freebsd/kmacy/src/sys/kern/subr_turnstile.c:487 (spin
mutex:turnstile chain)
22 962236 7498290 194839 4 38
665502 0
/usr/flatstor/shared/freebsd/kmacy/src/sys/kern/kern_idle.c:121 (spin
mutex:sched lock)
9 28157 4130053 7242 3 570
9959 3040
/usr/flatstor/shared/freebsd/kmacy/src/sys/sun4v/sun4v/pmap.c:1162 (sleep
mutex:vm page queue mutex)
39 44908 2629172 21967 2 119 21879
11174 /usr/flatstor/shared/freebsd/kmacy/src/sys/vm/vm_fault.c:342 (sleep
mutex:vm page queue mutex)
4 20673 2265592 21708 0 104
88367 0
/usr/flatstor/shared/freebsd/kmacy/src/sys/kern/subr_trap.c:191 (spin
mutex:sched lock)
31 11117 2231209 7243 1 308
7419 3023
/usr/flatstor/shared/freebsd/kmacy/src/sys/kern/sys_pipe.c:827 (sleep
mutex:vm page queue mutex)
On 11/12/06, Alan Cox <alc@freebsd.org> wrote:
>
> alc 2006-11-12 21:48:34 UTC
>
> FreeBSD src repository
>
> Modified files:
> sys/amd64/amd64 pmap.c
> sys/arm/arm pmap.c
> sys/i386/i386 pmap.c
> sys/ia64/ia64 pmap.c
> sys/powerpc/powerpc mmu_oea.c
> sys/sparc64/sparc64 pmap.c
> sys/sun4v/sun4v pmap.c
> sys/vm vm_fault.c vm_kern.c
> Log:
> Make pmap_enter() responsible for setting PG_WRITEABLE instead
> of its caller. (As a beneficial side-effect, a high-contention
> acquisition of the page queues lock in vm_fault() is eliminated.)
>
> Revision Changes Path
> 1.573 +3 -1 src/sys/amd64/amd64/pmap.c
> 1.73 +4 -1 src/sys/arm/arm/pmap.c
> 1.577 +3 -1 src/sys/i386/i386/pmap.c
> 1.180 +2 -0 src/sys/ia64/ia64/pmap.c
> 1.112 +3 -2 src/sys/powerpc/powerpc/mmu_oea.c
> 1.160 +4 -1 src/sys/sparc64/sparc64/pmap.c
> 1.6 +3 -1 src/sys/sun4v/sun4v/pmap.c
> 1.223 +2 -7 src/sys/vm/vm_fault.c
> 1.124 +1 -1 src/sys/vm/vm_kern.c
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?b1fa29170611121827r2d84ce69jc328a08b5cdd1fb0>