From owner-freebsd-fs@FreeBSD.ORG Sun Jun 28 17:54:28 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68B661065672 for ; Sun, 28 Jun 2009 17:54:28 +0000 (UTC) (envelope-from nhoyle@hoyletech.com) Received: from mout.perfora.net (mout.perfora.net [74.208.4.195]) by mx1.freebsd.org (Postfix) with ESMTP id 316B18FC13 for ; Sun, 28 Jun 2009 17:54:27 +0000 (UTC) (envelope-from nhoyle@hoyletech.com) Received: from [192.168.1.10] (pool-96-231-140-65.washdc.fios.verizon.net [96.231.140.65]) by mrelay.perfora.net (node=mrus1) with ESMTP (Nemesis) id 0MKpCa-1MKya40ser-000CC0; Sun, 28 Jun 2009 13:54:24 -0400 Message-ID: <4A47AE4A.6090705@hoyletech.com> Date: Sun, 28 Jun 2009 13:54:18 -0400 From: Nathanael Hoyle User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) MIME-Version: 1.0 To: Dan Naumov References: <4A4725FA.80505@modulus.org> <4A4747A0.6040902@modulus.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V01U2FsdGVkX18OqrS/n6+DW0YwrUV34ZJjxLP9sIBs3HnnEv8 gIEF28/Woti+JS0pKezxF3OwoVFap6dSTcGyhNAIX1Q5YHtgkx vmtrZtm3SBSbt/dx9qDMpgOyi0UzgZQ Cc: freebsd-fs@freebsd.org Subject: Re: read/write benchmarking: UFS2 vs ZFS vs EXT3 vs ZFS RAIDZ vs Linux MDRAID X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Jun 2009 17:54:28 -0000 The clear distinction between the two sets of performance tests you two have done is that Dan's are highly random short i/o's, and Andrew's are large sequential transfers. Large sequential transfers necessarily engage all of the disks in the pool, regardless of the parity strategy, therefore the implied penalty for ZFS to read the parity data from all drives is mostly theoretical, and actually performs more like RAID 5 typically would. In the case of Dan's highly random, short i/o's, the read itself is trivial, making the overhead of spinning/seeking all the disks to calculate the full checksum and validate it inordinately high. The implication of these two benchmarks is clear as well: ZFS RAIDZ may be an excellent choice for large storage capacity with reasonable performance characteristics for large sequential workloads, but should be avoided where many small transfers will be occurring. -Nathanael