Date: Thu, 02 Sep 1999 22:58:44 +0200 From: "P.C. Uiterlinden" <puiterl@hacom.nl> To: AIC7xxx@FreeBSD.ORG Subject: Tagged queueing = lower write throughput? Message-ID: <37CEE504.B84D6E75@hacom.nl>
next in thread | raw e-mail | index | archive | help
While playing with RAID0, I found out that enabling tagged queueing
deteriorates write throughput considerably. The MD driver seems not
the one to blame: with a single disk it also happens. I did not
expect this behavior nor can I explain it. What I would expect was
a slight improvement (especially with multiple processes), but
certainly not a rather big deterioration! I hope someone can explain
this. Any help would be highly appreciated.
Below, I have included three reports. The first one is from bonnie,
on a single disk. The second one is generated with bonnie on a MD
device (two disks). The third one is on one disk, generated with
dd.
My systems looks like:
CPU : Intel Pentium II, 350 MHz, 512 kB cache
RAM : 128 MB
HD : 2* SEAGATE ST34520W
SCSI : Adaptec AIC-7880 Ultra Wide (on mobo)
===============================================================================
Bonnie output of /dev/sdb11 for different tagged queue depths.
Note the drop in write speed (both per char and per block) when
queue depth >= 3. Read speed is unaffected.
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
qdepth_1 1500 11009 74.8 11501 16.2 4167 12.0 11293 56.4 11671 10.8 50.1 0.8
qdepth_2 1500 10749 73.0 11257 16.1 3445 9.9 11139 56.7 11336 10.8 52.8 1.0
qdepth_3 1500 6596 44.3 7532 10.8 3252 9.3 10861 54.1 11502 11.4 56.0 1.1
qdepth_4 1500 6745 45.2 7380 10.6 3300 9.6 10899 54.5 11434 10.8 56.9 1.1
qdepth_8 1500 6909 46.4 7426 10.5 3237 9.5 10888 54.0 11368 11.4 57.2 1.2
===============================================================================
Bonnie output of /dev/md4 (/dev/sda11 + /dev/sdb11, chunk size 32k)
for different tagged queue depths. The per block read and write speed
has doubled compared to the single disk situation. However, again when
queue depth >= 3, things get worse: the per block write speed drops 40%.
Strange: this is the only thing that seems to be affected.
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
qdepth_1 1500 13305 90.9 22619 34.3 7107 21.7 12821 65.0 23834 26.4 83.9 1.3
qdepth_2 1500 13232 89.8 21828 33.8 5599 16.9 13003 66.1 22517 22.1 86.8 1.5
qdepth_3 1500 12491 84.8 14262 19.8 5541 16.8 12625 64.1 22963 23.8 87.4 1.9
qdepth_4 1500 12381 83.8 14271 19.8 5529 16.9 12542 64.2 22134 23.2 89.3 2.0
qdepth_8 1500 10460 70.8 13573 19.7 5249 15.8 12877 65.6 22440 23.7 85.6 1.7
qdepth16 1500 12097 81.7 14025 20.1 5580 16.6 12822 65.8 22308 24.4 90.6 2.0
qdepth32 1500 11880 79.6 12818 18.3 5515 16.3 12642 64.6 22137 22.9 88.9 1.9
===============================================================================
Transfer rate numbers obtained with dd (on /dev/sdb11) do confirm
the numbers obtained with bonnie: only write throughput is affected
if the queue depth is larger than 2. Note the constantness of the
user+sys time, while the real time makes a jump at queue depth = 3.
Write : time dd if=/dev/zero of=/mnt/t bs=1024k count=1500
Read : time dd if=/mnt/t of=/dev/null bs=1024k
Remove : time rm /mnt/t
Queue depth 1 2 3 4 8 16
Write 133.808 137.218 218.119 217.751 207.189 216.199 real time (sec)
23.190 23.180 23.590 23.550 23.520 23.370 user+sys time (sec)
11.21 10.93 6.87 6.88 7.23 6.93 transfer rate (Mbyte/sec)
Read 135.385 137.304 139.691 140.149 132.901 140.170 real time (sec)
22.360 23.120 22.310 22.430 23.090 23.240 user+sys time (sec)
11.07 10.92 10.73 10.70 11.28 10.70 transfer rate (Mbyte/sec)
Remove 29.396 29.892 29.527 29.564 29.273 29.526 real time (sec)
0.530 0.590 0.530 0.670 0.510 0.590 user+sys time (sec)
===============================================================================
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe aic7xxx" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?37CEE504.B84D6E75>