Date: Fri, 19 Sep 2003 15:59:15 -0700 From: Sam Leffler <sam@errno.com> To: freebsd-arch@freebsd.org Subject: interrupt latency and driver locking Message-ID: <606095639.1063987155@melange.errno.com>
next in thread | raw e-mail | index | archive | help
I enabled MUTEX_PROFILING on a fast machine I've got setup as a
firewall/router. The machine had an fxp device on one side and an em on
the other. I then ran a bunch of netperf tests through the machine.
Unfortunately for the moment I'm stuck with 100baseT on the fxp side of the
machine so my tests were all limited by the media. But one thing stuck out:
max total count avg name
282 786 3 262 kern/kern_exec.c:853 (vm page queue
mute
775 7450 33 225 kern/vfs_subr.c:3218 (mntvnode)
1119 2789 15 185 kern/kern_fork.c:218 (Giant)
222 2344 14 167 vm/uma_core.c:1186 (UMA lock)
460 2972 19 156 kern/kern_exit.c:282 (vm page queue
mute
508 2415 19 127 kern/kern_exit.c:101 (Giant)
194 1801 16 112 kern/kern_exec.c:261 (Giant)
332 31113 277 112 dev/fxp/if_fxp.c:1798 (fxp0)
2435 31848 366 87 kern/kern_sysctl.c:1210 (Giant)
901 5306 66 80 vm/vm_map.c:1256 (Giant)
77 77 1 77 kern/uipc_syscalls.c:472 (Giant)
182 2901 39 74 vm/vm_kern.c:328 (system map)
507 841090 12508 67 dev/random/yarrow.c:172 (random
reseed)
7005 68335 1136 60 kern/kern_intr.c:533 (Giant)
66 964 16 60 vm/vm_map.c:2241 (system map)
88 7471 139 53 dev/em/if_em.c:1531 (em0)
881 8032 191 42 i386/i386/trap.c:997 (Giant)
591 10568722 271157 38 net/netisr.c:215 (netisr lock)
38 292 8 36 vm/vm_fault.c:997 (Giant)
Note that the fxp driver holds its driver lock for an average 112 us in one
spot. This turns out to be in fxp_tick and the likely culprit is the call
to mii_tick which is done with the lock held. Since there's only one lock
in the driver this means interrupts are blocked during this time which is
likely to do bad things. (The em driver comparison has a similar
bottleneck though it's <1/2 as long.)
I'm not sure if it's simple to move MII operations outside of the driver
lock but it might be worth investigating.
Sam
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?606095639.1063987155>