From owner-freebsd-fs@FreeBSD.ORG Fri Jun 22 08:08:22 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 95D4D106566C for ; Fri, 22 Jun 2012 08:08:22 +0000 (UTC) (envelope-from daniel@digsys.bg) Received: from smtp-sofia.digsys.bg (smtp-sofia.digsys.bg [193.68.3.230]) by mx1.freebsd.org (Postfix) with ESMTP id 172178FC15 for ; Fri, 22 Jun 2012 08:08:21 +0000 (UTC) Received: from dcave.digsys.bg (dcave.digsys.bg [192.92.129.5]) (authenticated bits=0) by smtp-sofia.digsys.bg (8.14.5/8.14.5) with ESMTP id q5M7e6oX086975 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Fri, 22 Jun 2012 10:40:06 +0300 (EEST) (envelope-from daniel@digsys.bg) Message-ID: <4FE42156.3090006@digsys.bg> Date: Fri, 22 Jun 2012 10:40:06 +0300 From: Daniel Kalchev User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:10.0.5) Gecko/20120607 Thunderbird/10.0.5 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <1953965235.30115.1340315339964.JavaMail.root@sz0192a.westchester.pa.mail.comcast.net> In-Reply-To: <1953965235.30115.1340315339964.JavaMail.root@sz0192a.westchester.pa.mail.comcast.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: ZFS Checksum errors X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Jun 2012 08:08:22 -0000 On 22.06.12 00:48, rondzierwa@comcast.net wrote: > its these kind of antics that make me resistant to the thought of allowing ZFS to manage the raid. it seems to be having problems just managing a big file system. I don't want it to correct anything, or restore anything, just let me delete the files that hurt, fix up the free space list so it doesn't point outside the bounds of the disk, and get on with life. Been there, done that... I too believed the hype that "hardware RAID" is the better, more "reliable" solution. But, when a third 3ware RAID array failed on me (I have lots) and had to spend few days and nights to reassemble the pieces, I finally decided to migrate everything to ZFS. In some cases I still do use the expensive 3ware RAID controllers as SLOW multi-port SATA controllers... I have discovered that my disks aren't actually that slow by just using normal HBA instead of "hardware RAID" controllers. I also had few cases, where a disk was considered just perfect, by both the 3ware controller (via any kind of test or verification) and S.M.A.R.T. but produced checksum errors when used with ZFS. I keep one or two such lying around to show to unbelievers. ZFS is way, way, WAY, more reliable for your data than any RAID controller could ever be. The reason is that ZFS checksums each and every block (metadata and data) in memory, before sending it down the pipe to disks and verifies those checksums when data comes back into memory. This is not done by any other system. If your memory is reliable, then you can trust ZFS if it tells you there are checksum errors: these happened somewhere between memory and disks, most probably corrupted RAID controller or on-disk caches, or some flaky bus. If your memory is unreliable, bad luck -- no file system can help. With ZFS over RAID you can only know there are problems "somewhere". With ZFS directly managing your disks, you know exactly which disk or the bus to it is failing and ZFS will automatically correct things. If you have enough redundancy, no data will be damaged. ZFS doesn't really have FAT table or such and free space is managed differently. But you are correct -- there should be tools to fix this kind of corruption. There is instrumentation for this, via zdb, but not enough good documentation and no one-click tool. In any case, you should be more concerned how you got to that corruption. ZFS does not have problem managing big file system. In fact, if anything can manage BIG file system that is ZFS. Your 12TB is in fact, an moderately small filesystem for ZFS -- it's used by way larger installations. Just let ZFS manage the disks directly. Daniel