From owner-freebsd-fs@FreeBSD.ORG Wed Dec 16 20:43:27 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4761C1065670 for ; Wed, 16 Dec 2009 20:43:27 +0000 (UTC) (envelope-from matt@corp.spry.com) Received: from mail-px0-f182.google.com (mail-px0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 25B2C8FC18 for ; Wed, 16 Dec 2009 20:43:26 +0000 (UTC) Received: by pxi12 with SMTP id 12so956361pxi.3 for ; Wed, 16 Dec 2009 12:43:26 -0800 (PST) Received: by 10.115.81.24 with SMTP id i24mr1024550wal.194.1260996206371; Wed, 16 Dec 2009 12:43:26 -0800 (PST) Received: from mattintosh.spry.com (isaid.donotdelete.com [64.79.222.10]) by mx.google.com with ESMTPS id 22sm937640pzk.10.2009.12.16.12.43.25 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 16 Dec 2009 12:43:25 -0800 (PST) From: Matt Simerson To: Solon Lutz In-Reply-To: <957649379.20091216005253@pyro.de> X-Priority: 3 (Normal) References: <568624531.20091215163420@pyro.de> <42952D86-6B4D-49A3-8E4F-7A1A53A954C2@spry.com> <957649379.20091216005253@pyro.de> Message-Id: <26F8D203-A923-47D3-9935-BE4BC6DA09B7@corp.spry.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v936) Date: Wed, 16 Dec 2009 12:43:24 -0800 X-Mailer: Apple Mail (2.936) Cc: freebsd-fs@freebsd.org Subject: Re: ZFS RaidZ2 with 24 drives? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Dec 2009 20:43:27 -0000 On Dec 15, 2009, at 3:52 PM, Solon Lutz wrote: >> I deployed using the two configurations you see above. Both machines >> have a pair of Areca 1231ML RAID controllers with super-sized BBWC >> (battery backed write cache). On back01, each controller presents a >> 12- >> disk RAID-5 array and ZFS concatenates them into the zpool you see >> above. On back02, the RAID controller is configured in JBOD mode and >> disks are pooled as shown. > > Why concatenate them into one pool and give up the redundancy? I didn't need redundant redundancy. > I have the same setup: Areca 24-port RAID6 (24x 500gb) > > NAME STATE READ WRITE CKSUM > temp ONLINE 0 0 24 > da0 ONLINE 0 0 48 > > And it very nearly killed itself after 28 months of flawless duty... > All went fine until 4 drives disconnected themselves from the Areca > due > to faulty SATA-cables. This crashed the Areca in such a way, that I > had > to disconnect the battery module from the controller in order to get > it > initialized during boot-up. > > Cache gone - ZFS unable to mount 10TB pool - scrub failed - I/O errors > > This was three months ago and if I hadn't found an extremly skilled > person > who was able to manually find and distinguish between good and > corrupted > meta-data sets, replicate them in their proper spots and zero out > corrupt > transaction ids - I would have lost 10TB of data. (No backups - to > expensive) > > Why do you use JBOD? You can configure a passthrough for all drives, > explicitly degrading the Areca to a dumb sata controller... Why would I bother? Both ways present each disk to FreeBSD. Based on my understanding (and an answer received from Areca support), the only reason I'd bother manually configuring some disks for passthrough is if I wanted to use some disks in a RAID array and others as raw disks. Configuring JBOD mode configures ALL the disks on the controller as passthrough devices. Matt