From owner-freebsd-fs@FreeBSD.ORG Mon Aug 20 18:14:11 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2820D16A41A for ; Mon, 20 Aug 2007 18:14:11 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (ns1.bitblocks.com [64.142.15.60]) by mx1.freebsd.org (Postfix) with ESMTP id 0A1C113C4A6 for ; Mon, 20 Aug 2007 18:14:10 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 861295B30; Mon, 20 Aug 2007 10:53:47 -0700 (PDT) To: Kenneth Vestergaard Schmidt In-reply-to: Your message of "Mon, 20 Aug 2007 09:28:00 +0200." Date: Mon, 20 Aug 2007 10:53:47 -0700 From: Bakul Shah Message-Id: <20070820175347.861295B30@mail.bitblocks.com> Cc: freebsd-fs@freebsd.org Subject: Re: ZFS: 'checksum mismatch' all over the place X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Aug 2007 18:14:11 -0000 > Aug 20 01:00:24 leibniz root: ZFS: checksum mismatch, zpool=pil path=/dev/da0 > offset=58350080 size=512 > Aug 20 01:00:24 leibniz root: ZFS: checksum mismatch, zpool=pil path=/dev/da1 > offset=58350080 size=512 > Aug 20 01:00:24 leibniz root: ZFS: checksum mismatch, zpool=pil path=/dev/da2 > offset=58350080 size=512 > Aug 20 01:00:24 leibniz root: ZFS: checksum mismatch, zpool=pil path=/dev/da3 > offset=58350080 size=512 > > Aug 20 01:00:24 leibniz root: ZFS: checksum mismatch, zpool=pil path=/dev/da2 > offset=38010880 size=512 > Aug 20 01:00:24 leibniz root: ZFS: checksum mismatch, zpool=pil path=/dev/da3 > offset=38010880 size=512 > Aug 20 01:00:24 leibniz root: ZFS: checksum mismatch, zpool=pil path=/dev/da4 > offset=38010880 size=512 > Aug 20 01:00:24 leibniz root: ZFS: checksum mismatch, zpool=pil path=/dev/da5 > offset=38010880 size=512 > Aug 20 01:00:24 leibniz root: ZFS: checksum mismatch, zpool=pil path=/dev/da6 > offset=38010880 size=512 > Aug 20 01:00:24 leibniz root: ZFS: checksum mismatch, zpool=pil path=/dev/da7 > offset=38010880 size=512 > Aug 20 01:00:24 leibniz root: ZFS: checksum mismatch, zpool=pil path=/dev/da8 > offset=38010880 size=512 > Aug 20 01:00:24 leibniz root: ZFS: checksum mismatch, zpool=pil path=/dev/da9 > offset=38010880 size=512 > Aug 20 01:00:24 leibniz root: ZFS: checksum mismatch, zpool=pil path=/dev/da1 > 0 offset=38010880 size=512 > > Can anybody offer anything to help me with this? I'm pretty much at a > loss as to how I can find the cause of this. This probably means the more than two blocks in zraid2 were bad so zpool can't correct the error. Just speculating here but may be the controller or disk writes there "behind your back" (assuming the offset reported is correct -- you can check zfs logic for that)? Can you map the offset to a disk block number? You can try writing/reading that block (after disabling zfs) and see if it changes in an unexpected way. This may not show any error if the problem is some complex interaction. If the disks are all the same and new, check the vendor website to see if there is a firmware upgrade. See if replacing one disk with another type of disk changes the error. 38010880 is 0x2440000 -- don't know if that is magic in any way but sometime a hex value can reveal a pattern. Always look at the binary or hex representation of any reported number in an error message!