From owner-freebsd-fs@FreeBSD.ORG Fri Jul 23 13:23:47 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2CF041065689; Fri, 23 Jul 2010 13:23:47 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id 677B58FC1D; Fri, 23 Jul 2010 13:23:45 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id 629AA39B1A8; Fri, 23 Jul 2010 15:23:44 +0200 (CEST) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 13.9824] X-CRM114-CacheID: sfid-20100723_15230_DDD0F203 X-CRM114-Status: Good ( pR: 13.9824 ) X-DSPAM-Result: Whitelisted X-DSPAM-Processed: Fri Jul 23 15:23:44 2010 X-DSPAM-Confidence: 0.9916 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 4c4997df898121049039089 X-DSPAM-Factors: 27, From*Attila Nagy , 0.00116, wrote, 0.00274, wrote, 0.00274, wrote+>, 0.00304, wrote+>, 0.00304, 8+0, 0.00631, 0+8, 0.00871, >+On, 0.00943, 9+0, 0.01000, >+It, 0.01000, data+is, 0.01000, >+There, 0.01000, stuff, 0.01000, 0+4, 0.01000, I+understand, 0.01000, reading, 0.01000, wrote+>>, 0.01000, benchmark, 0.01000, stream, 0.01000, saying, 0.01000, understand+your, 0.01000, but+I'm, 0.01000, 23+2010, 0.01000, >+it, 0.01000, I'm+not, 0.01000, 4+1, 0.01000, X-Spambayes-Classification: ham; 0.00 Received: from japan.t-online.private (japan.t-online.co.hu [195.228.243.99]) by people.fsn.hu (Postfix) with ESMTPSA id 7FAF139B16E; Fri, 23 Jul 2010 15:20:52 +0200 (CEST) Message-ID: <4C499733.5000104@fsn.hu> Date: Fri, 23 Jul 2010 15:20:51 +0200 From: Attila Nagy User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.10) Gecko/20100629 Thunderbird/3.0.5 MIME-Version: 1.0 To: ticso@cicely.de References: <4C496EB0.7050004@fsn.hu> <20100723125051.GM53114@cicely7.cicely.de> In-Reply-To: <20100723125051.GM53114@cicely7.cicely.de> Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Bernd Walter , Ivan Voras Subject: Re: ZFS makes SSDs faster than memory! X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jul 2010 13:23:47 -0000 On 07/23/10 14:50, Bernd Walter wrote: > On Fri, Jul 23, 2010 at 02:15:44PM +0200, Ivan Voras wrote: > >> On 07/23/10 12:28, Attila Nagy wrote: >> >>> Hi, >>> >>> I've came across a strange issue. On a file server (ftp/http/rsync) >>> there is a dual SSD based L2ARC configured for a pool of 24 disks: >>> >> >>> fetch -o /dev/null -4 >>> http://ftp.fsn.hu/pub/CDROM-Images/opensolaris/osol-0906-106a-ai-sparc.iso >>> /dev/null 100% of 493 MB 11 MBps >>> >> If I understand your setup and your benchmark correctly, you are saying >> you have achieved 11 megabytes / s performance out of a volume of 24 >> RAIDZ2 drives split into two parts (so it's like RAID 60). Doesn't this >> number seem extremely low to you, considering that (if recent models) >> each of your drives can probably pull at least 70 MB/s? >> > It is also quite strange that a linear read file gets stored in L2ARC, > which usually holds random accessed data. > Maybe it is very fragmented on disks. > L2ARC with MLC drives usually is much slower than modern disks when > it comes to linear reads. > There is no linear reads here from the PoV of the disks. Exactly one stream of linear read is linear read, but two streams are not. :) Maybe I should have written this first, but I'm not the only one reading from the machine. For random reads even the cheapest MLC outperforms a 7k2 SATA disk (only reads), and this is an Intel stuff, which can do 3000 RIOPS easily. > Are there any facts backup your assumption that data is really > read from memory, SSD, disk in the named cases? > E.g. by ARC/L2ARC and IO statistics. > Yes. When downloading from L2ARC: L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 174 174 21505 0.8 0 0 0.0 13.3| ad4 0 169 169 21479 0.9 0 0 0.0 15.0| ad6 when downloading from ARC: L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 26 19 1129 0.6 7 78 0.4 1.3| ad4 0 19 12 1436 1.1 7 78 0.3 1.4| ad6