Date: Thu, 29 Mar 2007 10:41:23 +0200 From: =?iso-8859-2?B?U3phYvMgUOl0ZXI=?= <matyee@mail.alba.hu> To: <freebsd-geom@freebsd.org> Subject: Re: raid3 is slow Message-ID: <003301c771de$08d03650$6502a8c0@peteruj> References: <008701c77139$613694b0$6502a8c0@peteruj><657228.54534.qm@web30313.mail.mud.yahoo.com> <20070328134411.GC35749@garage.freebsd.pl>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Wed, Mar 28, 2007 at 06:21:34AM -0700, R. B. Riddick wrote:
> > > # atacontrol mode ad0
> > > current mode = UDMA100
> > > # atacontrol mode ad1
> > > current mode = UDMA100
> > > # atacontrol mode ad2
> > > current mode = UDMA100
> > > # atacontrol mode ad3
> > > current mode = UDMA100
> > > # atacontrol mode ad6
> > > current mode = SATA150
> > >
> > Looks good...
>
> Does it? It seems there are two disks per ata channel, which is very bad
> for performance.
>
> I suggesting trying this:
>
> # apply "dd if=/dev/ad%1 if=/dev/null bs=1m count=250 &" 0 1 2 3 6
>
> and once it completes try this:
> # dd if=/dev/raid3/<name> of=/dev/null bs=1m count=1000
> and send the results here.
A SATA cable was wrong, and the 5th disk of the raid was not connected. Now
i fix it.
Geom name: nmivol
State: COMPLETE
Components: 5
Flags: NONE
GenID: 0
SyncID: 77
ID: 2396346217
Zone64kFailed: 0
Zone64kRequested: 19808642
Zone16kFailed: 0
Zone16kRequested: 4629259
Zone4kFailed: 28270
Zone4kRequested: 2027159
# apply "dd if=/dev/ad%1 of=/dev/null bs=1m count=250 &" 0 1 2 3 4
250+0 records out
262144000 bytes transferred in 4.324466 secs (60618814 bytes/sec)
250+0 records in
250+0 records out
262144000 bytes transferred in 6.289833 secs (41677418 bytes/sec)
250+0 records in
250+0 records out
262144000 bytes transferred in 6.262564 secs (41858894 bytes/sec)
250+0 records in
250+0 records out
262144000 bytes transferred in 7.871292 secs (33303808 bytes/sec)
250+0 records in
250+0 records out
262144000 bytes transferred in 7.857484 secs (33362333 bytes/sec)
I forgot to tell, one label of the raid is a gbde encoded partition. If i do
a read test from the raid, the load seems normal?!?! I don't know how high
is normal.
# dd if=/dev/raid3/nmivol of=/dev/null bs=1m count=30000
30000+0 records in
30000+0 records out
31457280000 bytes transferred in 336.545001 secs (93471244 bytes/sec)
# top
last pid: 16191; load averages: 0.32, 0.36, 0.49
up 0+15:16:03 10:14:12
58 processes: 1 running, 57 sleeping
CPU states: 0.0% user, 0.0% nice, 35.9% system, 3.5% interrupt, 60.5%
idle
Mem: 50M Active, 306M Inact, 165M Wired, 20K Cache, 111M Buf, 475M Free
# systat -vmstat
7 users Load 0.45 0.39 0.50 Mar 29 10:13
Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER
Tot Share Tot Share Free in out in out
Act 60900 8108 143488 9684 486056 count
All 533396 12740622626264 15932 pages
zfod Interrupts
Proc:r p d s w Csw Trp Sys Int Sof Flt cow 3986 total
1 56 14559 2 418 4171 10 9 169268 wire 1000 0: clk
51560 act 128 8: rtc
38.9%Sys 3.3%Intr 0.0%User 0.0%Nice 57.8%Idl 313632 inact 9 10:
fxp
| | | | | | | | | | 20 cache 1 11:
ata
===================++ 486036 free 1424 14:
ata
daefr 1424 15:
ata
Namei Name-cache Dir-cache prcfr
Calls hits % hits % react
pdwake
pdpgs
Disks ad0 ad1 ad2 ad3 ad4 intrn
KB/t 31.97 31.97 31.97 31.97 3.12 113904 buf
tps 712 712 712 712 1 3 dirtybuf
MB/s 22.22 22.22 22.22 22.22 0.00 69984 desiredvnodes
% busy 33 71 32 59 0 59556 numvnodes
17488 freevnodes
The same test from the encoded partition:
# dd if=/dev/raid3/nmivole.bde of=/dev/null bs=1m count=30000
30000+0 records in
30000+0 records out
31457280000 bytes transferred in 1282.804938 secs (24522263 bytes/sec)
# top
last pid: 16322; load averages: 1.07, 0.66, 0.55
up 0+15:22:28 10:20:37
61 processes: 1 running, 60 sleeping
CPU states: 0.4% user, 0.0% nice, 90.3% system, 0.4% interrupt, 8.9%
idle
Mem: 58M Active, 306M Inact, 166M Wired, 20K Cache, 111M Buf, 466M Free
# systat -vmstat
7 users Load 1.04 0.70 0.57 Mar 29 10:21
Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER
Tot Share Tot Share Free in out in out
Act 60912 8108 143504 9684 486044 count
All 533408 12740622626280 15932 pages
zfod Interrupts
Proc:r p d s w Csw Trp Sys Int Sof Flt cow 3014 total
1 56 8575 5 275 3181 20 13 169268 wire 999 0: clk
51572 act 128 8: rtc
81.8%Sys 2.5%Intr 0.2%User 0.0%Nice 15.5%Idl 313632 inact 8 10:
fxp
| | | | | | | | | | 20 cache 1 11:
ata
=========================================+ 486024 free 939 14:
ata
daefr 939 15:
ata
Namei Name-cache Dir-cache prcfr
Calls hits % hits % react
pdwake
pdpgs
Disks ad0 ad1 ad2 ad3 ad4 intrn
KB/t 12.88 12.88 12.88 12.88 0.50 113904 buf
tps 470 470 470 470 1 22 dirtybuf
MB/s 5.92 5.91 5.92 5.92 0.00 69984 desiredvnodes
% busy 24 40 20 35 0 59556 numvnodes
17489 freevnodes
Maybe is what the hardware can do :( , isn't it?
Matyee
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?003301c771de$08d03650$6502a8c0>