From owner-freebsd-fs@FreeBSD.ORG  Fri Dec 19 05:37:42 2008
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 665871065679
	for <freebsd-fs@freebsd.org>; Fri, 19 Dec 2008 05:37:42 +0000 (UTC)
	(envelope-from matt@corp.spry.com)
Received: from wf-out-1314.google.com (wf-out-1314.google.com [209.85.200.175])
	by mx1.freebsd.org (Postfix) with ESMTP id 440518FC21
	for <freebsd-fs@freebsd.org>; Fri, 19 Dec 2008 05:37:42 +0000 (UTC)
	(envelope-from matt@corp.spry.com)
Received: by wf-out-1314.google.com with SMTP id 24so1125473wfg.7
	for <freebsd-fs@freebsd.org>; Thu, 18 Dec 2008 21:37:41 -0800 (PST)
Received: by 10.142.125.9 with SMTP id x9mr1154440wfc.236.1229665061840;
	Thu, 18 Dec 2008 21:37:41 -0800 (PST)
Received: from imac24.simerson.net (c-67-168-10-190.hsd1.wa.comcast.net
	[67.168.10.190])
	by mx.google.com with ESMTPS id 32sm15184476wfc.39.2008.12.18.21.37.40
	(version=TLSv1/SSLv3 cipher=RC4-MD5);
	Thu, 18 Dec 2008 21:37:41 -0800 (PST)
Message-Id: <D18EBA53-704F-4C21-9BF9-CDBB2AF918D2@corp.spry.com>
From: Matt Simerson <matt@corp.spry.com>
To: freebsd-fs@freebsd.org
In-Reply-To: <alpine.BSF.2.00.0812181732440.14585@ibyngvyr.purzvxnyf.bet>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v930.3)
Date: Thu, 18 Dec 2008 21:37:39 -0800
References: <22C8092E-210F-4E91-AA09-CFD38966975C@spry.com>
	<alpine.BSF.2.00.0812181732440.14585@ibyngvyr.purzvxnyf.bet>
X-Mailer: Apple Mail (2.930.3)
Subject: Re: ZFS performance gains real or imaginary?
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 19 Dec 2008 05:37:42 -0000


On Dec 18, 2008, at 4:48 PM, Wes Morgan wrote:

>> On the two systems above (amd64 with 16GB of RAM and 24 1TB disks)  
>> I get about 30 days of uptime before the system hangs with a ZFS  
>> error.  They write backups to disk 24x7 and never stop. I could not  
>> anything near that level of stability with back03 (below) which was  
>> much older hardware maxed out at 4GB of RAM.  I finally resolved  
>> the stability issues on back03 by ditching ZFS and using  
>> geom_stripe across the two hardware RAID arrays.
>
> Were you doing a zfs mirror across two hardware raid arrays? The  
> performance of that type of setup would probably be sub-optimal  
> versus a zpool with two raidz volumes.

I haven't benchmarked it with -HEAD  but with FreeBSD 7, using a ZFS  
mirror across two 12-disk hardware RAID arrays (Areca 1231ML) was  
significantly (not quite double) faster than using JBOD and raidz. I  
tested a few variations (four disk pools, six disk zpools, 8 disk  
zpools, etc).

I'll be getting another 24 disk system to add to my backup pool in a  
month or two. When it arrives, I'll run some additional benchmarks  
with -HEAD and see where the numbers fall.  I'll be quite surprised if  
raidz can outrun a hardware RAID controller with 512MB of BBWC.

Matt