From owner-freebsd-geom@FreeBSD.ORG Thu Mar 29 08:41:26 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 576AF16A401 for ; Thu, 29 Mar 2007 08:41:26 +0000 (UTC) (envelope-from matyee@mail.alba.hu) Received: from mta02.mail.t-online.hu (mta02.mail.t-online.hu [195.228.240.51]) by mx1.freebsd.org (Postfix) with ESMTP id EFD4E13C45A for ; Thu, 29 Mar 2007 08:41:25 +0000 (UTC) (envelope-from matyee@mail.alba.hu) Received: from peteruj (dsl51B6FBF9.fixip.t-online.hu [81.182.251.249]) by mail.axelero.hu (Postfix) with SMTP id BC0E133C45B for ; Thu, 29 Mar 2007 10:41:24 +0200 (CEST) Message-ID: <003301c771de$08d03650$6502a8c0@peteruj> From: =?iso-8859-2?B?U3phYvMgUOl0ZXI=?= To: References: <008701c77139$613694b0$6502a8c0@peteruj><657228.54534.qm@web30313.mail.mud.yahoo.com> <20070328134411.GC35749@garage.freebsd.pl> Date: Thu, 29 Mar 2007 10:41:23 +0200 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-2"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.3028 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3028 Subject: Re: raid3 is slow X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Mar 2007 08:41:26 -0000 > On Wed, Mar 28, 2007 at 06:21:34AM -0700, R. B. Riddick wrote: > > > # atacontrol mode ad0 > > > current mode = UDMA100 > > > # atacontrol mode ad1 > > > current mode = UDMA100 > > > # atacontrol mode ad2 > > > current mode = UDMA100 > > > # atacontrol mode ad3 > > > current mode = UDMA100 > > > # atacontrol mode ad6 > > > current mode = SATA150 > > > > > Looks good... > > Does it? It seems there are two disks per ata channel, which is very bad > for performance. > > I suggesting trying this: > > # apply "dd if=/dev/ad%1 if=/dev/null bs=1m count=250 &" 0 1 2 3 6 > > and once it completes try this: > # dd if=/dev/raid3/ of=/dev/null bs=1m count=1000 > and send the results here. A SATA cable was wrong, and the 5th disk of the raid was not connected. Now i fix it. Geom name: nmivol State: COMPLETE Components: 5 Flags: NONE GenID: 0 SyncID: 77 ID: 2396346217 Zone64kFailed: 0 Zone64kRequested: 19808642 Zone16kFailed: 0 Zone16kRequested: 4629259 Zone4kFailed: 28270 Zone4kRequested: 2027159 # apply "dd if=/dev/ad%1 of=/dev/null bs=1m count=250 &" 0 1 2 3 4 250+0 records out 262144000 bytes transferred in 4.324466 secs (60618814 bytes/sec) 250+0 records in 250+0 records out 262144000 bytes transferred in 6.289833 secs (41677418 bytes/sec) 250+0 records in 250+0 records out 262144000 bytes transferred in 6.262564 secs (41858894 bytes/sec) 250+0 records in 250+0 records out 262144000 bytes transferred in 7.871292 secs (33303808 bytes/sec) 250+0 records in 250+0 records out 262144000 bytes transferred in 7.857484 secs (33362333 bytes/sec) I forgot to tell, one label of the raid is a gbde encoded partition. If i do a read test from the raid, the load seems normal?!?! I don't know how high is normal. # dd if=/dev/raid3/nmivol of=/dev/null bs=1m count=30000 30000+0 records in 30000+0 records out 31457280000 bytes transferred in 336.545001 secs (93471244 bytes/sec) # top last pid: 16191; load averages: 0.32, 0.36, 0.49 up 0+15:16:03 10:14:12 58 processes: 1 running, 57 sleeping CPU states: 0.0% user, 0.0% nice, 35.9% system, 3.5% interrupt, 60.5% idle Mem: 50M Active, 306M Inact, 165M Wired, 20K Cache, 111M Buf, 475M Free # systat -vmstat 7 users Load 0.45 0.39 0.50 Mar 29 10:13 Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER Tot Share Tot Share Free in out in out Act 60900 8108 143488 9684 486056 count All 533396 12740622626264 15932 pages zfod Interrupts Proc:r p d s w Csw Trp Sys Int Sof Flt cow 3986 total 1 56 14559 2 418 4171 10 9 169268 wire 1000 0: clk 51560 act 128 8: rtc 38.9%Sys 3.3%Intr 0.0%User 0.0%Nice 57.8%Idl 313632 inact 9 10: fxp | | | | | | | | | | 20 cache 1 11: ata ===================++ 486036 free 1424 14: ata daefr 1424 15: ata Namei Name-cache Dir-cache prcfr Calls hits % hits % react pdwake pdpgs Disks ad0 ad1 ad2 ad3 ad4 intrn KB/t 31.97 31.97 31.97 31.97 3.12 113904 buf tps 712 712 712 712 1 3 dirtybuf MB/s 22.22 22.22 22.22 22.22 0.00 69984 desiredvnodes % busy 33 71 32 59 0 59556 numvnodes 17488 freevnodes The same test from the encoded partition: # dd if=/dev/raid3/nmivole.bde of=/dev/null bs=1m count=30000 30000+0 records in 30000+0 records out 31457280000 bytes transferred in 1282.804938 secs (24522263 bytes/sec) # top last pid: 16322; load averages: 1.07, 0.66, 0.55 up 0+15:22:28 10:20:37 61 processes: 1 running, 60 sleeping CPU states: 0.4% user, 0.0% nice, 90.3% system, 0.4% interrupt, 8.9% idle Mem: 58M Active, 306M Inact, 166M Wired, 20K Cache, 111M Buf, 466M Free # systat -vmstat 7 users Load 1.04 0.70 0.57 Mar 29 10:21 Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER Tot Share Tot Share Free in out in out Act 60912 8108 143504 9684 486044 count All 533408 12740622626280 15932 pages zfod Interrupts Proc:r p d s w Csw Trp Sys Int Sof Flt cow 3014 total 1 56 8575 5 275 3181 20 13 169268 wire 999 0: clk 51572 act 128 8: rtc 81.8%Sys 2.5%Intr 0.2%User 0.0%Nice 15.5%Idl 313632 inact 8 10: fxp | | | | | | | | | | 20 cache 1 11: ata =========================================+ 486024 free 939 14: ata daefr 939 15: ata Namei Name-cache Dir-cache prcfr Calls hits % hits % react pdwake pdpgs Disks ad0 ad1 ad2 ad3 ad4 intrn KB/t 12.88 12.88 12.88 12.88 0.50 113904 buf tps 470 470 470 470 1 22 dirtybuf MB/s 5.92 5.91 5.92 5.92 0.00 69984 desiredvnodes % busy 24 40 20 35 0 59556 numvnodes 17489 freevnodes Maybe is what the hardware can do :( , isn't it? Matyee