Date: Wed, 17 Nov 2010 16:14:50 +0100 From: Ivan Voras <ivoras@freebsd.org> To: freebsd-fs@freebsd.org Subject: Kind of slow IO under pressure? Message-ID: <ic0rh5$4bb$1@dough.gmane.org>
next in thread | raw e-mail | index | archive | help
I'm running bonnie++ on UFS hosted on a FC SAN, and during the "rewrite"
phase of the benchmark (bonnie++ is deliberately trying to trash disk
caches), the situation from "top" is like this:
21 root 76 - 0K 16K qsleep 1 2:48 44.38% bufdaemon
1274 root 76 0 11296K 1716K getblk 7 3:35 41.06% bonnie++
3 root -8 - 0K 16K - 2 1:44 21.78% g_up
With disk rates of about 40 MB/s in either direction and around 300 IOPS.
Together these three processes take up a whole CPU core and bonnie++ is
single-threaded, which sort of suggests that this supposedly IO
benchmark is actually CPU-bound on this 8-core 2.4 GHz system.
Transaction sizes are curiously not-quite-64KiB:
tty da0 da1 da2
cpu
tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy
in id
0 380 15.93 28 0.44 63.75 1242 77.30 1.93 0 0.00 0 0 9
0 91
0 1510 16.00 3 0.05 63.86 1148 71.56 0.00 0 0.00 0 0 10
0 90
0 1063 16.00 2 0.03 63.96 1093 68.24 0.00 0 0.00 0 0 14
0 86
0 357 0.00 0 0.00 63.95 1024 63.95 0.00 0 0.00 0 0 15
0 85
Interestingly, binding (with cpuset) all three processes to the same CPU
makes things very strange, with bufdaemon climbing to 90% CPU usage and
bandwidth dropping to ~~ 5 MB/s (but not completely stalling). Relaxing
bufdeamon binds to two CPUs makes things normal again.
On the "read" phase the situation is:
1274 root 61 0 11296K 1704K getblk 0 7:05 26.56% bonnie++
18 root 44 - 0K 16K psleep 4 0:09 2.88% pagedaemon
3 root -8 - 0K 16K - 0 3:31 1.66% g_up
4 root -8 - 0K 16K - 2 0:17 1.66% g_down
12 root -40 - 0K 432K WAIT 5 0:11 1.27% {swi2:
cambio}
i.e. no CPU hogging and the performance is ~~ 140 MB/s. The write phase
is similar, nothing unusual.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ic0rh5$4bb$1>