From owner-freebsd-fs@FreeBSD.ORG Sun May 16 00:51:19 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC07C106566C for ; Sun, 16 May 2010 00:51:19 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) by mx1.freebsd.org (Postfix) with ESMTP id A12A18FC08 for ; Sun, 16 May 2010 00:51:18 +0000 (UTC) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.13.8+Sun/8.13.8) with ESMTP id o4G0pHZZ019886; Sat, 15 May 2010 19:51:17 -0500 (CDT) Date: Sat, 15 May 2010 19:51:17 -0500 (CDT) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: Jeremy Chadwick In-Reply-To: <20100516001351.GA50879@icarus.home.lan> Message-ID: References: <4BEF2F9C.7080409@netscape.net> <4BEF3137.4080203@netscape.net> <20100516001351.GA50879@icarus.home.lan> User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Sat, 15 May 2010 19:51:17 -0500 (CDT) Cc: freebsd-fs@freebsd.org Subject: Re: Quick ZFS mirroring question for non-mirrored pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2010 00:51:19 -0000 On Sat, 15 May 2010, Jeremy Chadwick wrote: > > What you have here is the equivalent of RAID-10. It might be more > helpful to look at the above as a "stripe of mirrors". > > In this situation, you might be better off with raidz1 (RAID-5 in > concept). You should get better actual I/O performance due to ZFS > distributing the I/O workload across 4 disks rather than 2. At least > that's how I understand it. That would be a reasonable assumption but actual evidence suggests otherwise. For sequential I/O, mirrors and raidz1 seem to offer roughly similar performance, except that mirrors win for reads and raidz1 often win for writes. The mirror configuration definitely wins as soon as there are many seeks or multi-user activity. The reason why mirrors still do well for sequential I/O is that there is still load-sharing across the vdevs (smart "striping") but in full 128K blocks whereas the raidz1 config needs to break the 128K blocks into smaller blocks which are striped across the disks in the vdev. Breaking the data into smaller chunks for raidz multiplies the disk IOPS required. Disk seeks are slow. The main reason to choose raidz1 is for better space efficiency but mirrors offer more performance. For an interesting set of results, see the results summary of "Bob's method" at "http://www.nedharvey.com/". The only way to be sure for your own system is to create various pool configurations and test with something which represents your expected work load. As long as the pool is not the boot pool, zfs makes such testing quite easy. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/