From owner-freebsd-scsi@FreeBSD.ORG Mon May 26 18:26:23 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E7D6106567B for ; Mon, 26 May 2008 18:26:23 +0000 (UTC) (envelope-from os@rsu.ru) Received: from mail.r61.net (mail.r61.net [195.208.245.249]) by mx1.freebsd.org (Postfix) with ESMTP id BC3788FC1F for ; Mon, 26 May 2008 18:26:22 +0000 (UTC) (envelope-from os@rsu.ru) Received: from mind.local (os.adsl.r61.net [195.208.243.95]) (authenticated bits=0) by mail.r61.net (8.14.1/8.14.1) with ESMTP id m4QI1pw9005785 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 26 May 2008 22:01:52 +0400 (MSD) (envelope-from os@rsu.ru) Message-ID: <483AFB0A.6050401@rsu.ru> Date: Mon, 26 May 2008 22:01:46 +0400 From: Oleg Sharoiko User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Alexander Goncharov References: <5c29cc10805260541r1f90f516q8ac7d45d2bb15702@mail.gmail.com> <20080526141005.GC1142@in-addr.com> <5c29cc10805260836rb8ccac0ic722c255e60286f8@mail.gmail.com> In-Reply-To: <5c29cc10805260836rb8ccac0ic722c255e60286f8@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-scsi@freebsd.org Subject: Re: Strange behavior of SCSI RAID 10 (FreeBSD, LSILogic MegaRAID) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2008 18:26:23 -0000 Hi! Alexander Goncharov wrote: > On Tue, May 27, 2008 at 1:10 AM, Gary Palmer wrote: > > >> On Mon, May 26, 2008 at 11:41:56PM +1100, Alexander Goncharov wrote: >> >>> Hello world, I have faced with following issue on my dedicated server: >>> >>> 8x Opteron 885, 32gm RAM, 8x36 GM 15k rpm SCSI with RAID 10 >>> >>> FreeBSD 7.0-generic, 64 bit version >>> >>> >>> >>> IO performance and behavior is very strange: >>> >>> 1) No other process are running: >>> >>> Memory stat: >>> >>> Mem: 8796K Active, 9372K Inact, 80M Wired, 36K Cache, 12M Buf, 31G Free >>> >>> Copy 3GB file first time >>> >>> dd if=/home/3gb_file of=/home/3gb_file2 >>> 6291456+0 records in >>> 6291456+0 records out >>> 3221225472 bytes transferred in 138.842926 secs (23200501 bytes/sec) >>> >>> 20MBS is very poor? >>> >>> Alexander, you're using default block size of 512 bytes. In this case your hardware has to: while (!eof(source_file)) { locate next 512 byte block read it locate writing position write block } So it does 6291456*2 searches. Moreover you're working with files and thus dd has to go throug filesystem layer which involves additional overhead. Please try increasing block size to, for example, 8 mb (add bs=8m to the arguments of dd) >>> Memory stat now: >>> >>> Mem: 8940K Active, 5951M Inact, 287M Wired, 36K Cache, 214M Buf, 25G Free >>> >>> 2) Copy the same file again: >>> >>> dd if=/home/3gb_file of=/home/3gb_file2 >>> 6291456+0 records in >>> 6291456+0 records out >>> 3221225472 bytes transferred in 30.433515 secs (105844674 bytes/sec) >>> >>> 100MBs ? much better >>> >>> >> The "Inact" (Inactive) went up - the 3GiB file is now cached in >> memory. So the second (and subsequent) runs are going from cached >> memory so your 100MiB/sec transfer is actually just testing write >> speed, not read/write speed. This is the same for your other >> tests too >> >> "Inact" is memory that has been used and is being kept around incase >> it is used again, in other words its caching file data in the >> "Inact" region in top. It'll be reused if something else needs >> the memory, but until then it sticks around. >> >> Remember - you are copying the file from and to the same >> filesystem - this is always going to appear slow relative to >> pure read or pure write tests. A pure write test is effectively >> what you have when you're getting your 100MiB/sec test result since >> its just writing out from cache memory. >> >> Regards, >> >> Gary >> >> > > Hi Gary, > > Big thanks for your quick reply. Read only and write only speed is good > ~100MBs. But I am not sure if it really good for hw RAID 10 8x 15k rpm > drives. > I am worried about read/write speed usecase which is most used at real tasks > (data base). I was surprised 20 MBs speed, this value is likely to one drive > speed. But I have 8 high speed drives. Something isn't right here. > > I am waiting for freebsd community help. > > Thanks in advance. > Alexander > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" >