From owner-freebsd-fs@FreeBSD.ORG Thu Nov 13 06:20:02 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 397D21065672; Thu, 13 Nov 2008 06:20:02 +0000 (UTC) (envelope-from danny@dannysplace.net) Received: from mail.dannysplace.net (mail.dannysplace.net [213.133.54.210]) by mx1.freebsd.org (Postfix) with ESMTP id DB0EE8FC12; Thu, 13 Nov 2008 06:20:01 +0000 (UTC) (envelope-from danny@dannysplace.net) Received: from 203-206-171-212.perm.iinet.net.au ([203.206.171.212] helo=[192.168.10.10]) by mail.dannysplace.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1L0V25-000IUu-PC; Thu, 13 Nov 2008 15:46:32 +1000 Message-ID: <491BBF38.9010908@dannysplace.net> Date: Thu, 13 Nov 2008 15:46:32 +1000 From: Danny Carroll User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: fbsd@dannysplace.net References: <490A782F.9060406@dannysplace.net> <20081031033208.GA21220@icarus.home.lan> <490A849C.7030009@dannysplace.net> <20081031043412.GA22289@icarus.home.lan> <490A8FAD.8060009@dannysplace.net> In-Reply-To: <490A8FAD.8060009@dannysplace.net> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Authenticated-User: danny X-Authenticator: plain X-Sender-Verify: SUCCEEDED (sender exists & accepts mail) X-Exim-Version: 4.69 (build at 08-Jul-2008 08:59:40) X-Date: 2008-11-13 15:46:22 X-Connected-IP: 203.206.171.212:3073 X-Message-Linecount: 85 X-Body-Linecount: 70 X-Message-Size: 2967 X-Body-Size: 2246 X-Received-Count: 1 X-Recipient-Count: 4 X-Local-Recipient-Count: 4 X-Local-Recipient-Defer-Count: 0 X-Local-Recipient-Fail-Count: 0 X-SA-Exim-Connect-IP: 203.206.171.212 X-SA-Exim-Rcpt-To: fbsd@dannysplace.net, koitsu@FreeBSD.org, freebsd-fs@freebsd.org, freebsd-hardware@freebsd.org X-SA-Exim-Mail-From: danny@dannysplace.net X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on ferrari.dannysplace.net X-Spam-Level: X-Spam-Status: No, score=0.2 required=8.0 tests=ALL_TRUSTED,AWL,TVD_RCVD_IP autolearn=disabled version=3.2.5 X-SA-Exim-Version: 4.2 X-SA-Exim-Scanned: Yes (on mail.dannysplace.net) Cc: freebsd-fs@freebsd.org, Jeremy Chadwick , freebsd-hardware@freebsd.org Subject: Re: Areca vs. ZFS performance testing. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2008 06:20:02 -0000 Danny Carroll wrote: > Jeremy Chadwick wrote: >> I'd like to see the performance difference between these scenarios: >> >> - Memory cache enabled on Areca, write caching enabled on disks >> - Memory cache enabled on Areca, write caching disabled on disks >> - Memory cache disabled on Areca, write caching enabled on disks >> - Memory cache disabled on Areca, write caching disabled on disks >> The initial results for a ICH9 vs Areca in JBod mode can be found here: http://www.dannysplace.net/ZFS-JBODTests.html Summary: 5 Disk ZFS RaidZ array with atime turned off. ICH9 - block reads avg 400MByte/Sec ICH9 - block writes avg 150MByte/Sec ArecaJBOD - block reads avg 300MByte/Sec ArecaJBOD - block writes avg 160MByte/Sec The Areca seems to be in all except char and block writes. Block reads are 75% as fast as the ICH9 and rewrites are about 85% as fast. There seems to be little difference between enabling and disabling the disk cache on the Areca. This leads me to two conclusions: 1. Disabling the write cache does nothing on Seagate drives. 2. IO to the drives is so slow that a write cache is irrelevant. These are just some quick tests that I started with, mainly to compare the areca bus versus the ich9 bus. If someone has any tuning suggestions, then now is the time to make them before I migrate the ICH9 drives to the Areca bus. -D p.s. My OS details are: FreeBSD 7.1-PRERELEASE #3: Tue Nov 4 13:58:49 EST 2008 localhost# cat /etc/sysctl.conf kern.maxvnodes=400000 net.key.preferred_oldsa=0 net.key.blockacq_count=0 kern.ipc.maxsockbuf=400000 net.inet.ip.fastforwarding=1 net.inet.tcp.rfc1323=1 kern.ipc.maxsockbuf=16777216 net.local.stream.sendspace=82320 net.local.stream.recvspace=82320 net.inet.tcp.local_slowstart_flightsize=10 net.inet.tcp.nolocaltimewait=1 net.inet.tcp.delayed_ack=1 net.inet.tcp.delacktime=100 net.inet.tcp.mssdflt=1460 net.inet.tcp.sendspace=78840 net.inet.tcp.recvspace=78840 net.inet.tcp.slowstart_flightsize=54 net.inet.tcp.inflight.enable=1 net.inet.tcp.inflight.min=6144 net.inet.tcp.hostcache.expire=3900 localhost# cat /boot/loader.conf hw.em.rxd=4096 hw.em.txd=4096 vm.kmem_size="1536M" vm.kmem_size_max="1536M" smb_load="YES" smbus_load="YES" ichsmb_load="YES"