From owner-freebsd-fs@FreeBSD.ORG Mon Oct 13 07:39:20 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A8CDB724 for ; Mon, 13 Oct 2014 07:39:20 +0000 (UTC) Received: from mail-vc0-x22a.google.com (mail-vc0-x22a.google.com [IPv6:2607:f8b0:400c:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A962BFA for ; Mon, 13 Oct 2014 07:39:20 +0000 (UTC) Received: by mail-vc0-f170.google.com with SMTP id hy10so5401541vcb.29 for ; Mon, 13 Oct 2014 00:39:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=OdRfZlSwQdEM4uyEQgxmh65TS8sYd+PnZ+BmIHihYdc=; b=CGEr5Wr1UuylXpcB6oMAqjSFJP2QWLmv4w3PxNUB4kSTH0gV7Xey+0mjjiBo+lazKY Meeb8BImAKaA+9mmB/TVZiyqh15JRNgQAQeC5LTJgceEJ+nl+vvVr0sDAg821l7EnMyO u4VjHomi7iOtD/RvlGzfAZce4c/g5ieHDzdZ9+zfnmFkfb54FVRQ1zO2rQltb1d+G1iv i/S7WYJS0Np3WalC2ufra/3/5fM3bk0VfMUQZFr8x/Tpi3K01jvFdoaqWwu5bI1mAvU9 CUQiB0Hc47D3AgO23VXPLc2tAA7hlyxo1FimYB4Qk/oMoU1uwTrJiaY5VXiJstWzt+pg 8upA== MIME-Version: 1.0 X-Received: by 10.220.168.74 with SMTP id t10mr4265294vcy.35.1413185959383; Mon, 13 Oct 2014 00:39:19 -0700 (PDT) Received: by 10.220.238.14 with HTTP; Mon, 13 Oct 2014 00:39:19 -0700 (PDT) Date: Mon, 13 Oct 2014 03:39:19 -0400 Message-ID: Subject: ZFS Checksum errors without any Checksum errors? From: Zaphod Beeblebrox To: freebsd-fs Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Oct 2014 07:39:20 -0000 This bit confuses me. The errors are on 'vr2' and 'raidz1-0' ... but not on any of the disks. I can let this complete and there's some 400 data errors spread all around, but the disks will all read zero errors. What do errors on the array the the vdev mean, then? How did I get here? vr2-d2c (in my notation, dxy is the x'th drive in the array, version y --- so d2c is the third time I've replaced d2) replaced d2b... which is now refusing to spin up. Stay away from 1.5T drives. Anyways... If I let the resilver finish, it will finish without finding errors on any particular drive --- but it will still report hundreds of errors on vr2 (the array) and raidz1-0 (the 1st vdev). NAME STATE READ WRITE CKSUM vr2 ONLINE 0 0 7 raidz1-0 ONLINE 0 0 14 label/vr2-d0 ONLINE 0 0 0 label/vr2-d1 ONLINE 0 0 0 gpt/vr2-d2c ONLINE 0 0 0 (resilvering) gpt/vr2-d3b ONLINE 0 0 0 block size: 512B configured, 4096B native gpt/vr2-d4a ONLINE 0 0 0 block size: 512B configured, 4096B native ada14 ONLINE 0 0 0 label/vr2-d6 ONLINE 0 0 0 label/vr2-d7c ONLINE 0 0 0 label/vr2-d8 ONLINE 0 0 0 raidz1-1 ONLINE 0 0 0 gpt/vr2-e0 ONLINE 0 0 0 block size: 512B configured, 4096B native gpt/vr2-e1 ONLINE 0 0 0 block size: 512B configured, 4096B native gpt/vr2-e2 ONLINE 0 0 0 block size: 512B configured, 4096B native gpt/vr2-e3 ONLINE 0 0 0 gpt/vr2-e4 ONLINE 0 0 0 block size: 512B configured, 4096B native gpt/vr2-e5 ONLINE 0 0 0 block size: 512B configured, 4096B native gpt/vr2-e6 ONLINE 0 0 0 block size: 512B configured, 4096B native gpt/vr2-e7 ONLINE 0 0 0 block size: 512B configured, 4096B native logs gpt/vr2log ONLINE 0 0 0 cache gpt/vr2cache ONLINE 0 0 0 errors: 427 data errors, use '-v' for a list