From owner-freebsd-fs@FreeBSD.ORG Sun Jun 15 05:04:19 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5CDBBD0F for ; Sun, 15 Jun 2014 05:04:19 +0000 (UTC) Received: from mail-vc0-f178.google.com (mail-vc0-f178.google.com [209.85.220.178]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1A6F72149 for ; Sun, 15 Jun 2014 05:04:18 +0000 (UTC) Received: by mail-vc0-f178.google.com with SMTP id ij19so3726503vcb.23 for ; Sat, 14 Jun 2014 22:04:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=gcnODdki8v2jCjyttjjQ6JeHUfjw2y2lr7wTUX3jJdk=; b=be+yxo7Q2A35o+Sw3FNzS6HU2kQ+p2wRKu2NoLY3p6/fE2JFCyiU2m6/LV/2rM5vhG fuYmYcWVP3EBm72/AMenqZC2oaryE1RlllHytgZy1nXXFLuZuPiABdTkoyxb88T0GxXZ f/J9ZMIDt/1gkHhn5c/JrVMqo/P7vVNh0W0Rj8r+vH0oi8NOIt1aWplDDKBEocDCiZ1e BAMG+hVfEwXVodjRcLREeuA8uEA4Jmlk7h9kvI17V1xhTQG+S7tku5eU4AosM0A5AX0g Y/MPBn1pC6Tvyrc01BMwUPqkoOWCGrMAFcb+HxZNhhXdkRBrhGm3Gvd3qk2cFWsGCPk3 U5Pw== X-Gm-Message-State: ALoCoQn5obxlg60GToppAaQkmnNi9AZCCPdXUSIvPy0XlKs13MoWFLyBF5klOatogHhtVT0t9IvS MIME-Version: 1.0 X-Received: by 10.220.167.2 with SMTP id o2mr9914847vcy.8.1402808656986; Sat, 14 Jun 2014 22:04:16 -0700 (PDT) Received: by 10.58.187.162 with HTTP; Sat, 14 Jun 2014 22:04:16 -0700 (PDT) X-Originating-IP: [60.242.115.240] Date: Sun, 15 Jun 2014 15:04:16 +1000 Message-ID: Subject: ZFS pool permanent error question -- errors: Permanent errors have been detected in the following files: storage: <0x0> From: Anders Jensen-Waud To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 05:04:19 -0000 Hi all, My main zfs storage pool (named ``storage'') has recently started displaying a very odd error: root@beastie> zpool status -v / pool: backup state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM backup ONLINE 0 0 0 da1 ONLINE 0 0 0 errors: No known data errors pool: storage state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://illumos.org/msg/ZFS-8000-8A scan: scrub in progress since Sun Jun 15 14:18:45 2014 34.3G scanned out of 839G at 19.3M/s, 11h50m to go 72K repaired, 4.08% done config: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 0 da0 ONLINE 0 0 0 (repairing) errors: Permanent errors have been detected in the following files: storage:<0x0> My dmesg: Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-RELEASE-p1 #0: Tue Apr 8 06:45:06 UTC 2014 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 CPU: Intel(R) Core(TM)2 Duo CPU T7300 @ 2.00GHz (1995.04-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x6fa Family = 0x6 Model = 0xf Stepping = 10 Features=0xbfebfbff Features2=0xe3bd AMD Features=0x20100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 3221225472 (3072 MB) avail memory = 3074908160 (2932 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 ACPI BIOS Warning (bug): 32/64X length mismatch in FADT/Gpe1Block: 0/32 (20130823/tbfadt-601) ACPI BIOS Warning (bug): Optional FADT field Gpe1Block has zero address or length: 0x000000000000102C/0x0 (20130823/tbfadt-630) ioapic0: Changing APIC ID to 1 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 random: initialized acpi0: on motherboard CPU0: local APIC error 0x40 acpi_ec0: port 0x62,0x66 on acpi0 acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, bef00000 (3) failed cpu0: on acpi0 cpu1: on acpi0 attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 450 Event timer "HPET1" frequency 14318180 Hz quality 440 Event timer "HPET2" frequency 14318180 Hz quality 440 atrtc0: port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 acpi_lid0: on acpi0 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 16 at device 1.0 on pci0 pci1: on pcib1 vgapci0: port 0x2000-0x207f mem 0xd6000000-0xd6ffffff,0xe0000000-0xefffffff,0xd4000000-0xd5ffffff irq 16 at device 0.0 on pci1 vgapci0: Boot video device em0: port 0x1840-0x185f mem 0xfe200000-0xfe21ffff,0xfe225000-0xfe225fff irq 20 at device 25.0 on pci0 em0: Using an MSI interrupt em0: Ethernet address: 00:15:58:c6:c3:3f uhci0: port 0x1860-0x187f irq 20 at device 26.0 on pci0 usbus0 on uhci0 uhci1: port 0x1880-0x189f irq 21 at device 26.1 on pci0 usbus1 on uhci1 ehci0: mem 0xfe226c00-0xfe226fff irq 22 at device 26.7 on pci0 usbus2: EHCI version 1.0 usbus2 on ehci0 hdac0: mem 0xfe220000-0xfe223fff irq 17 at device 27.0 on pci0 pcib2: irq 20 at device 28.0 on pci0 pci2: on pcib2 pcib3: irq 21 at device 28.1 on pci0 pci3: on pcib3 iwn0: mem 0xdf2fe000-0xdf2fffff irq 17 at device 0.0 on pci3 pcib4: irq 22 at device 28.2 on pci0 pci4: on pcib4 pcib5: irq 23 at device 28.3 on pci0 pci5: on pcib5 pcib6: irq 20 at device 28.4 on pci0 pci13: on pcib6 uhci2: port 0x18a0-0x18bf irq 16 at device 29.0 on pci0 usbus3 on uhci2 uhci3: port 0x18c0-0x18df irq 17 at device 29.1 on pci0 usbus4 on uhci3 uhci4: port 0x18e0-0x18ff irq 18 at device 29.2 on pci0 usbus5 on uhci4 ehci1: mem 0xfe227000-0xfe2273ff irq 19 at device 29.7 on pci0 usbus6: EHCI version 1.0 usbus6 on ehci1 pcib7: at device 30.0 on pci0 pci21: on pcib7 cbb0: mem 0xf8100000-0xf8100fff irq 16 at device 0.0 on pci21 cardbus0: on cbb0 pccard0: <16-bit PCCard bus> on cbb0 pci21: at device 0.1 (no driver attached) sdhci_pci0: mem 0xf8101800-0xf81018ff irq 18 at device 0.2 on pci21 sdhci_pci0: 1 slot(s) allocated pci21: at device 0.3 (no driver attached) pci21: at device 0.4 (no driver attached) pci21: at device 0.5 (no driver attached) isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1830-0x183f at device 31.1 on pci0 ata0: at channel 0 on atapci0 ahci0: port 0x1c48-0x1c4f,0x1c1c-0x1c1f,0x1c40-0x1c47,0x1c18-0x1c1b,0x1c20-0x1c3f mem 0xfe226000-0xfe2267ff irq 16 at device 31.2 on pci0 ahci0: AHCI v1.10 with 3 1.5Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich2: at channel 2 on ahci0 pci0: at device 31.3 (no driver attached) acpi_tz0: on acpi0 acpi_tz1: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model Generic PS/2 mouse, device ID 0psm0: model Generic PS/2 mouse, device ID 0 battery0: on acpi0 acpi_acad0: on acpi0 orm0: at iomem 0xc0000-0xcefff,0xcf000-0xcffff,0xd0000-0xd0fff,0xe0000-0xeffff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ppc0: cannot reserve I/O port range est0: on cpu0 p4tcc0: on cpu0 est1: on cpu1 p4tcc1: on cpu1 Timecounters tick every 1.000 msec hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 pcm0: at nid 18,17 and 28,20,21 on hdaa0 pcm1: at nid 27 on hdaa0 hdacc1: at cad 1 on hdac0 unknown: at nid 2 on hdacc1 (no driver attached) random: unblocking device. usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 480Mbps High Speed USB v2.0 usbus3: 12Mbps Full Speed USB v1.0 usbus4: 12Mbps Full Speed USB v1.0 usbus5: 12Mbps Full Speed USB v1.0 usbus6: 480Mbps High Speed USB v2.0 ugen0.1: at usbus0 uhub0: on usbus0 ugen2.1: at usbus2 uhub1: on usbus2 ugen1.1: at usbus1 uhub2: on usbus1 ugen5.1: at usbus5 uhub3: on usbus5 ugen4.1: at usbus4 uhub4: on usbus4 ugen3.1: at usbus3 uhub5: on usbus3 ugen6.1: at usbus6 uhub6: on usbus6 ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ATA-8 SATA 1.x device ada0: Serial Number 071201DP0410DTG7P9AP ada0: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 190782MB (390721968 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad4 cd0 at ata0 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: Serial Number HC43 045371 cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present Netvsc initializing... SMP: AP CPU #1 Launched! Root mount waiting for: usbus6 usbus5 usbus4 usbus3 usbus2 usbus1 usbus0 uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered uhub4: 2 ports with 2 removable, self powered Root mount waiting for: usbus6 usbus2 uhub1: 4 ports with 4 removable, self powered Root mount waiting for: usbus6 usbus2 uhub6: 6 ports with 6 removable, self powered ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ATA-8 SATA 1.x device ada0: Serial Number 071201DP0410DTG7P9AP ada0: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 190782MB (390721968 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad4 cd0 at ata0 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: Serial Number HC43 045371 cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present Netvsc initializing... SMP: AP CPU #1 Launched! Root mount waiting for: usbus6 usbus5 usbus4 usbus3 usbus2 usbus1 usbus0 uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered uhub4: 2 ports with 2 removable, self powered Root mount waiting for: usbus6 usbus2 uhub1: 4 ports with 4 removable, self powered Root mount waiting for: usbus6 usbus2 uhub6: 6 ports with 6 removable, self powered Root mount waiting for: usbus6 ugen0.2: at usbus0 Root mount waiting for: usbus6 ugen6.2: at usbus6 umass0: on usbus6 umass0: SCSI over Bulk-Only; quirks = 0x0100 umass0:3:0:-1: Attached to scbus3 da0 at umass-sim0 bus 0 scbus3 target 0 lun 0 da0: Fixed Direct Access SCSI-6 device da0: Serial Number NA46H44R da0: 40.000MB/s transfers da0: 953869MB (1953525167 512 byte sectors: 255H 63S/T 121601C) da0: quirks=0x2 GEOM: da0: the primary GPT table is corrupt or invalid. GEOM: da0: using the secondary instead -- recovery strongly advised. GEOM: diskid/DISK-NA46H44R: the primary GPT table is corrupt or invalid. GEOM: diskid/DISK-NA46H44R: using the secondary instead -- recovery strongly advised. Root mount waiting for: usbus6 ugen6.3: at usbus6 umass1: on usbus6 umass1: SCSI over Bulk-Only; quirks = 0x0100 umass1:4:1:-1: Attached to scbus4 Trying to mount root from ufs:/dev/ada0p2 [rw]... da1 at umass-sim1 bus 1 scbus4 target 0 lun 0 da1: Fixed Direct Access SCSI-4 device da1: Serial Number 2GE1GTVM da1: 40.000MB/s transfers da1: 476940MB (976773168 512 byte sectors: 255H 63S/T 60801C) da1: quirks=0x2 GEOM: da1: the primary GPT table is corrupt or invalid. GEOM: da1: using the secondary instead -- recovery strongly advised. GEOM: diskid/DISK-2GE1GTVM: the primary GPT table is corrupt or invalid. GEOM: diskid/DISK-2GE1GTVM: using the secondary instead -- recovery strongly advised. ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present; to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Cheers Anders