From owner-freebsd-fs@FreeBSD.ORG Fri Jul 23 13:43:35 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 440AB106566B; Fri, 23 Jul 2010 13:43:35 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id 8C2AE8FC08; Fri, 23 Jul 2010 13:43:33 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id E6CC539B3A4; Fri, 23 Jul 2010 15:43:32 +0200 (CEST) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 10.9541] X-CRM114-CacheID: sfid-20100723_15432_4291F35F X-CRM114-Status: Good ( pR: 10.9541 ) X-DSPAM-Result: Whitelisted X-DSPAM-Processed: Fri Jul 23 15:43:32 2010 X-DSPAM-Confidence: 0.9915 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 4c499c84976601156211266 X-DSPAM-Factors: 27, From*Attila Nagy , 0.00114, >+>, 0.00271, wrote, 0.00274, wrote, 0.00274, wrote+>, 0.00304, wrote+>, 0.00304, >+On, 0.00943, it+doesn't, 0.01000, previously, 0.01000, mean+that, 0.01000, >+a, 0.01000, in+>, 0.01000, of+course, 0.01000, 15+15, 0.01000, case+of, 0.01000, worse, 0.01000, of+>, 0.01000, >+This, 0.01000, Nagy, 0.01000, )+>, 0.01000, checksum, 0.01000, the+machine, 0.01000, much+>, 0.01000, I've, 0.01076, >+>>, 0.01408, >+>>, 0.01408, X-Spambayes-Classification: ham; 0.00 Received: from japan.t-online.private (japan.t-online.co.hu [195.228.243.99]) by people.fsn.hu (Postfix) with ESMTPSA id A5FB239B396; Fri, 23 Jul 2010 15:43:21 +0200 (CEST) Message-ID: <4C499C79.8040001@fsn.hu> Date: Fri, 23 Jul 2010 15:43:21 +0200 From: Attila Nagy User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.10) Gecko/20100629 Thunderbird/3.0.5 MIME-Version: 1.0 To: Ivan Voras References: <4C496EB0.7050004@fsn.hu> <4C4995F7.2080107@fsn.hu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: ZFS makes SSDs faster than memory! X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jul 2010 13:43:35 -0000 On 07/23/10 15:21, Ivan Voras wrote: > On 07/23/10 15:15, Attila Nagy wrote: > > >> When pulling this amount of data out of the machine, the disks aren't >> saturated, they are at around 10-20% of utilization according to gstat. >> BTW, remember that two RAIDZ2 in stripe isn't RAID60. In RAIDZ2 every >> read involves a full stripe (er, block) read for checksum validation, >> which means at a 128 kiB blocksize and with 12 disks in a RAIDZ2 pool, >> all disks provide their part of that 128k read. >> That's why a RAIDZ2 pool's IO performance equals of one disk's. >> > Yes, in case of random IOPS you are correct - and in your case it would > mean that the files are horribly fragmented (torrent downloads? :)). For > sequential IO, even RAIDZ/1/2 will give N-1/2/3 times the performance of > a single drive because prefetching will kick in. > Especially when prefetching is disabled. :) I've had problems with it previously and it was left in this state. Turning it off makes that 11 MiBps 30 MiBps, and the disk utilization increase, but of course it doesn't affect the fact that SSDs are faster than RAM with FreeBSD/zfs. In fact, it made things worse. SSDs AND HDDs are now faster than RAM. :-O Damn. ;) > >> The disks in a normal 20-30 MiBps network load do about 30-40 read IOPS, >> you are right that they are capable of more (around 100-120). >> > Except for the possible fragmentation issue, I think you should get much > better throughput even with 30-40 IOPS per drive. > This is an FTP/HTTP server, like ftp.freebsd.org (well, exactly like that). If there is 20-30 MiBps network load that may be because nobody wants to fetch more data. :)