From owner-freebsd-fs@FreeBSD.ORG Mon Oct 6 05:25:48 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2AB1E10656A2 for ; Mon, 6 Oct 2008 05:25:48 +0000 (UTC) (envelope-from dimitar.vassilev@gmail.com) Received: from mail-gx0-f21.google.com (mail-gx0-f21.google.com [209.85.217.21]) by mx1.freebsd.org (Postfix) with ESMTP id C1FD88FC30 for ; Mon, 6 Oct 2008 05:25:47 +0000 (UTC) (envelope-from dimitar.vassilev@gmail.com) Received: by gxk14 with SMTP id 14so4432511gxk.19 for ; Sun, 05 Oct 2008 22:25:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type:references; bh=JYPCOIn0jrv6E6/jtdjIJ3pLYc8uL7/Eo1f4leB9ouQ=; b=iKzAu6jc0StUvKXfHvH5s3P4Y/B6faAx0ZTaXyLHQtzl8pDqtk7St0RDambXQcZ5e6 vEnYs00BFzeEEjt/7kI7Uk6F2e/bcsYv8qrD21+5TacPWHkgxPGcXsHWVmi6ezRRhVLg yZ2h9RneBalpaw81MqNzwtDvpWo9gOIuEzSzo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:references; b=UcP+dgPdDGGcHLn6D7qf+HV/qdcM1lrmP7dl0LQKZQvBHYL9bItE1CCd1xuIiJlrqs 6OWE3s0ltI+zIsIM2dphJV2CeUDKG/jcMy2lia4ESaheHLZ5BlumlG1HLfHkBVjAcS/p r2j6hbrZaO/1GT8bH1TBj5aaW5fAW5NdZzKbU= Received: by 10.151.108.13 with SMTP id k13mr6954123ybm.112.1223270747064; Sun, 05 Oct 2008 22:25:47 -0700 (PDT) Received: by 10.150.182.11 with HTTP; Sun, 5 Oct 2008 22:25:47 -0700 (PDT) Message-ID: <59adc1a0810052225w6d8c1b78r226ffe8ca4ccf35d@mail.gmail.com> Date: Mon, 6 Oct 2008 08:25:47 +0300 From: "Dimitar Vasilev" To: "Andrew Snow" In-Reply-To: <48E9556C.9060004@modulus.org> MIME-Version: 1.0 References: <59adc1a0810051210t4a3503aci2bc06ba0aa5376c3@mail.gmail.com> <48E9556C.9060004@modulus.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: zfs as layer distributor X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2008 05:25:48 -0000 2008/10/6 Andrew Snow > Dimitar Vasilev wrote: > >> Hi all, >> Does someone use zfs as layer distributor on the top of hardware raid - >> (RAID10,RAID6,etc)? >> > > I've found ZFS works faster when given more than one disk device. The > reason being, it is smart about writing journal logs and metadata copies to > different devices, resulting in higher performance by using idle disks. It > also provides more "channels" for write clustering so higher throughput on > write-heavy loads. > > Secondly if you use ZFS to provide RAID1 or RAID5, due to checksumming it > can be smarter about which data it chooses in the event of a checksum > failure. Hardware RAID can only do this with RAID6. > > Finally, when ZFS issues "flush cache" command to the disk for metadata and > journal logs, there is less data to flush when you give it multiple smaller > devices. If you have a single monolithic RAID device with a large (eg. > 256mb) cache, it can ruin performance while the RAID card flushes its entire > cache. (This can be disabled with a sysctl). > > - Andrew Thanks Andrew, I have an Areca 1120 with RAID-6 and on the top of it a zfs as a layer distributor. So far I can tell the following: 1)works nice and fast 2)can be pain in the rear if your controller spits one of the disks due to power surge/etc. 3) zfs snapshots caused some crashes and bad descriptors on 7.0-stable as of 3 months behind- but it's somewhat expected. I'm thinking of raidz2 and setting the disks as pass-through. Would love if someone to hear if someone has tested hardware raid6 and zfs over it. Best regards, Dimitar