Date: Sat, 16 Feb 2013 11:55:05 +0200 From: Alexandr Krivulya <shuriku@shurik.kiev.ua> To: freebsd-fs@freebsd.org Subject: Re: error destroying zfs filesystem Message-ID: <511F5779.5030805@shurik.kiev.ua> In-Reply-To: <511E30B3.2070302@brockmann-consult.de> References: <511E1C6B.50101@shurik.kiev.ua> <CAJDksDTnMNUo06Exivr4-tcCcGCNuRZnvP4jP7XTc-ydekKGvQ@mail.gmail.com> <511E30B3.2070302@brockmann-consult.de>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format. --------------050809020106070702000508 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 15.02.2013 14:57, Peter Maloney пишет: > On 2013-02-15 13:44, Alexandr Kovalenko wrote: >> On Fri, Feb 15, 2013 at 11:30 AM, Alexandr Krivulya >> <shuriku@shurik.kiev.ua> wrote: >>> Hello everyone! >>> >>> After upgrading my zfs-only system from 8.2 to 9.1 I have many errors >>> related to zfs in my /var/log/messages: >>> >>> Feb 15 13:12:44 gw kernel: metaslab_free_dva(): bad DVA >>> 0:264842321920Solaris: WARNING: metaslab_free_dva(): bad DVA 0:338480095232 >>> Feb 15 13:12:44 gw kernel: Solaris: WARNING: metaslab_free_dva(): bad >>> DVA 0:277633901056Solaris: WARNING: >>> Feb 15 13:12:45 gw kernel: metaslab_free_dva(): bad DVA >>> 0:277263710208Solaris: WARNING: metaslab_free_dva(): bad DVA >>> 0:277633606144Solaris: WARNING: metaslab_free_dva(): bad DVA >>> 0:278349642240Solaris: WARNING: metaslab_free_dva(): bad DVA >>> 0:278429099008Solaris: WARNING: metaslab_free_dva(): bad DVA >>> 0:278349926400Solaris: WARNING: metaslab_free_dva(): bad DVA >>> 0:278245378560Solaris: WARNING: metaslab_free_dva(): bad DVA >>> 0:256838777344Solaris: WARNING: metaslab_free_dva(): bad DVA 0:327364684800 >>> Feb 15 13:12:45 gw kernel: Solaris: WARNING: metaslab_free_dva(): bad >>> DVA 0:312373604864 >>> >>> root@gw:/ # zpool status -v >>> pool: zmirror >>> state: ONLINE >>> status: One or more devices has experienced an error resulting in data >>> corruption. Applications may be affected. >>> action: Restore the file in question if possible. Otherwise restore the >>> entire pool from backup. >>> see: http://illumos.org/msg/ZFS-8000-8A >>> scan: scrub repaired 0 in 1h39m with 1 errors on Thu Feb 14 17:48:53 2013 >>> config: >>> >>> NAME STATE READ WRITE CKSUM >>> zmirror ONLINE 0 0 2 >>> mirror-0 ONLINE 0 0 8 >>> gpt/disk01 ONLINE 0 0 8 >>> gpt/disk02 ONLINE 0 0 8 >>> >>> errors: Permanent errors have been detected in the following files: >>> >>> zmirror/usr:<0x0> >>> <0xc8>:<0x0> >> [dd] >>> How can I solve this issue? >> Make smartctl -t long /dev/<your_physical_drive_here> and then take a >> look if there any pending sectors/errors in output of smartctl -a >> /dev/<your_physical_drive_here> ? (for both of drives used) All tests seems to be fine: root@gw:/usr/home/support # smartctl -l selftest /dev/ada0 smartctl 6.0 2012-10-10 r3643 [FreeBSD 9.1-RELEASE amd64] (local build) Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org === START OF READ SMART DATA SECTION === SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed without error 00% 1446 - root@gw:/usr/home/support # smartctl -l selftest /dev/ada1 smartctl 6.0 2012-10-10 r3643 [FreeBSD 9.1-RELEASE amd64] (local build) Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org === START OF READ SMART DATA SECTION === SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed without error 00% 1630 smartctl also didn't show any problems, see attached file > You could also try going in /usr and "rm" or "truncate" some files > until the "Permanent errors have been detected" list is empty. And > this assumes you already ran a full scrub, which you must do to remove > the files. Now I cannot mount this filesystem to remove files: root@gw:/usr/home/support # zfs mount zmirror/usr cannot mount 'zmirror/usr': mountpoint or dataset is busy The only way I see is to backup entire pool, destroy and recreate it, and restore from a backup. --------------050809020106070702000508 Content-Type: text/plain; charset=UTF-8; name="smartctl.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smartctl.txt" cm9vdEBndzovdXNyL2hvbWUvc3VwcG9ydCAjIHNtYXJ0Y3RsIC1pQUggL2Rldi9hZGEwCnNt YXJ0Y3RsIDYuMCAyMDEyLTEwLTEwIHIzNjQzIFtGcmVlQlNEIDkuMS1SRUxFQVNFIGFtZDY0 XSAobG9jYWwgYnVpbGQpCkNvcHlyaWdodCAoQykgMjAwMi0xMiwgQnJ1Y2UgQWxsZW4sIENo cmlzdGlhbiBGcmFua2UsIHd3dy5zbWFydG1vbnRvb2xzLm9yZwoKPT09IFNUQVJUIE9GIElO Rk9STUFUSU9OIFNFQ1RJT04gPT09Ck1vZGVsIEZhbWlseTogICAgIFdlc3Rlcm4gRGlnaXRh bCBSRTQgU2VyaWFsIEFUQQpEZXZpY2UgTW9kZWw6ICAgICBXREMgV0Q1MDAzQUJZWC0wMVdF UkExClNlcmlhbCBOdW1iZXI6ICAgIFdELVdNQVlQMzI1MTM0MApMVSBXV04gRGV2aWNlIElk OiA1IDAwMTRlZSAwMDMyYWQ1M2IKRmlybXdhcmUgVmVyc2lvbjogMDEuMDFTMDIKVXNlciBD YXBhY2l0eTogICAgNTAwIDEwNyA4NjIgMDE2IGJ5dGVzIFs1MDAgR0JdClNlY3RvciBTaXpl OiAgICAgIDUxMiBieXRlcyBsb2dpY2FsL3BoeXNpY2FsClJvdGF0aW9uIFJhdGU6ICAgIDcy MDAgcnBtCkRldmljZSBpczogICAgICAgIEluIHNtYXJ0Y3RsIGRhdGFiYXNlIFtmb3IgZGV0 YWlscyB1c2U6IC1QIHNob3ddCkFUQSBWZXJzaW9uIGlzOiAgIEFUQTgtQUNTIChtaW5vciBy ZXZpc2lvbiBub3QgaW5kaWNhdGVkKQpTQVRBIFZlcnNpb24gaXM6ICBTQVRBIDMuMCwgMy4w IEdiL3MgKGN1cnJlbnQ6IDMuMCBHYi9zKQpMb2NhbCBUaW1lIGlzOiAgICBTYXQgRmViIDE2 IDExOjQ5OjMyIDIwMTMgRUVUClNNQVJUIHN1cHBvcnQgaXM6IEF2YWlsYWJsZSAtIGRldmlj ZSBoYXMgU01BUlQgY2FwYWJpbGl0eS4KU01BUlQgc3VwcG9ydCBpczogRW5hYmxlZAoKPT09 IFNUQVJUIE9GIFJFQUQgU01BUlQgREFUQSBTRUNUSU9OID09PQpTTUFSVCBvdmVyYWxsLWhl YWx0aCBzZWxmLWFzc2Vzc21lbnQgdGVzdCByZXN1bHQ6IFBBU1NFRAoKU01BUlQgQXR0cmli dXRlcyBEYXRhIFN0cnVjdHVyZSByZXZpc2lvbiBudW1iZXI6IDE2ClZlbmRvciBTcGVjaWZp YyBTTUFSVCBBdHRyaWJ1dGVzIHdpdGggVGhyZXNob2xkczoKSUQjIEFUVFJJQlVURV9OQU1F ICAgICAgICAgIEZMQUcgICAgIFZBTFVFIFdPUlNUIFRIUkVTSCBUWVBFICAgICAgVVBEQVRF RCAgV0hFTl9GQUlMRUQgUkFXX1ZBTFVFCiAgMSBSYXdfUmVhZF9FcnJvcl9SYXRlICAgICAw eDAwMmYgICAyMDAgICAyMDAgICAwNTEgICAgUHJlLWZhaWwgIEFsd2F5cyAgICAgICAtICAg ICAgIDAKICAzIFNwaW5fVXBfVGltZSAgICAgICAgICAgIDB4MDAyNyAgIDEzOSAgIDEzOSAg IDAyMSAgICBQcmUtZmFpbCAgQWx3YXlzICAgICAgIC0gICAgICAgNDAzMwogIDQgU3RhcnRf U3RvcF9Db3VudCAgICAgICAgMHgwMDMyICAgMTAwICAgMTAwICAgMDAwICAgIE9sZF9hZ2Ug ICBBbHdheXMgICAgICAgLSAgICAgICAxOAogIDUgUmVhbGxvY2F0ZWRfU2VjdG9yX0N0ICAg MHgwMDMzICAgMjAwICAgMjAwICAgMTQwICAgIFByZS1mYWlsICBBbHdheXMgICAgICAgLSAg ICAgICAwCiAgNyBTZWVrX0Vycm9yX1JhdGUgICAgICAgICAweDAwMmUgICAyMDAgICAyMDAg ICAwMDAgICAgT2xkX2FnZSAgIEFsd2F5cyAgICAgICAtICAgICAgIDAKICA5IFBvd2VyX09u X0hvdXJzICAgICAgICAgIDB4MDAzMiAgIDA5OCAgIDA5OCAgIDAwMCAgICBPbGRfYWdlICAg QWx3YXlzICAgICAgIC0gICAgICAgMTQ2NQogMTAgU3Bpbl9SZXRyeV9Db3VudCAgICAgICAg MHgwMDMyICAgMTAwICAgMjUzICAgMDAwICAgIE9sZF9hZ2UgICBBbHdheXMgICAgICAgLSAg ICAgICAwCiAxMSBDYWxpYnJhdGlvbl9SZXRyeV9Db3VudCAweDAwMzIgICAxMDAgICAyNTMg ICAwMDAgICAgT2xkX2FnZSAgIEFsd2F5cyAgICAgICAtICAgICAgIDAKIDEyIFBvd2VyX0N5 Y2xlX0NvdW50ICAgICAgIDB4MDAzMiAgIDEwMCAgIDEwMCAgIDAwMCAgICBPbGRfYWdlICAg QWx3YXlzICAgICAgIC0gICAgICAgMTYKMTkyIFBvd2VyLU9mZl9SZXRyYWN0X0NvdW50IDB4 MDAzMiAgIDIwMCAgIDIwMCAgIDAwMCAgICBPbGRfYWdlICAgQWx3YXlzICAgICAgIC0gICAg ICAgMTUKMTkzIExvYWRfQ3ljbGVfQ291bnQgICAgICAgIDB4MDAzMiAgIDIwMCAgIDIwMCAg IDAwMCAgICBPbGRfYWdlICAgQWx3YXlzICAgICAgIC0gICAgICAgMgoxOTQgVGVtcGVyYXR1 cmVfQ2Vsc2l1cyAgICAgMHgwMDIyICAgMTE2ICAgMDk1ICAgMDAwICAgIE9sZF9hZ2UgICBB bHdheXMgICAgICAgLSAgICAgICAyNwoxOTYgUmVhbGxvY2F0ZWRfRXZlbnRfQ291bnQgMHgw MDMyICAgMjAwICAgMjAwICAgMDAwICAgIE9sZF9hZ2UgICBBbHdheXMgICAgICAgLSAgICAg ICAwCjE5NyBDdXJyZW50X1BlbmRpbmdfU2VjdG9yICAweDAwMzIgICAyMDAgICAyMDAgICAw MDAgICAgT2xkX2FnZSAgIEFsd2F5cyAgICAgICAtICAgICAgIDAKMTk4IE9mZmxpbmVfVW5j b3JyZWN0YWJsZSAgIDB4MDAzMCAgIDEwMCAgIDI1MyAgIDAwMCAgICBPbGRfYWdlICAgT2Zm bGluZSAgICAgIC0gICAgICAgMAoxOTkgVURNQV9DUkNfRXJyb3JfQ291bnQgICAgMHgwMDMy ICAgMjAwICAgMjAwICAgMDAwICAgIE9sZF9hZ2UgICBBbHdheXMgICAgICAgLSAgICAgICAw CjIwMCBNdWx0aV9ab25lX0Vycm9yX1JhdGUgICAweDAwMDggICAyMDAgICAyMDAgICAwMDAg ICAgT2xkX2FnZSAgIE9mZmxpbmUgICAgICAtICAgICAgIDAKCgoKCgpyb290QGd3Oi91c3Iv aG9tZS9zdXBwb3J0ICMgc21hcnRjdGwgLWlBSCAvZGV2L2FkYTEKc21hcnRjdGwgNi4wIDIw MTItMTAtMTAgcjM2NDMgW0ZyZWVCU0QgOS4xLVJFTEVBU0UgYW1kNjRdIChsb2NhbCBidWls ZCkKQ29weXJpZ2h0IChDKSAyMDAyLTEyLCBCcnVjZSBBbGxlbiwgQ2hyaXN0aWFuIEZyYW5r ZSwgd3d3LnNtYXJ0bW9udG9vbHMub3JnCgo9PT0gU1RBUlQgT0YgSU5GT1JNQVRJT04gU0VD VElPTiA9PT0KTW9kZWwgRmFtaWx5OiAgICAgV2VzdGVybiBEaWdpdGFsIFJFNCBTZXJpYWwg QVRBCkRldmljZSBNb2RlbDogICAgIFdEQyBXRDUwMDNBQllYLTAxV0VSQTEKU2VyaWFsIE51 bWJlcjogICAgV0QtV01BWVAzMjY1NjQ1CkxVIFdXTiBEZXZpY2UgSWQ6IDUgMDAxNGVlIDAw MzJjMmIxNApGaXJtd2FyZSBWZXJzaW9uOiAwMS4wMVMwMgpVc2VyIENhcGFjaXR5OiAgICA1 MDAgMTA3IDg2MiAwMTYgYnl0ZXMgWzUwMCBHQl0KU2VjdG9yIFNpemU6ICAgICAgNTEyIGJ5 dGVzIGxvZ2ljYWwvcGh5c2ljYWwKUm90YXRpb24gUmF0ZTogICAgNzIwMCBycG0KRGV2aWNl IGlzOiAgICAgICAgSW4gc21hcnRjdGwgZGF0YWJhc2UgW2ZvciBkZXRhaWxzIHVzZTogLVAg c2hvd10KQVRBIFZlcnNpb24gaXM6ICAgQVRBOC1BQ1MgKG1pbm9yIHJldmlzaW9uIG5vdCBp bmRpY2F0ZWQpClNBVEEgVmVyc2lvbiBpczogIFNBVEEgMy4wLCAzLjAgR2IvcyAoY3VycmVu dDogMy4wIEdiL3MpCkxvY2FsIFRpbWUgaXM6ICAgIFNhdCBGZWIgMTYgMTE6NDg6NDAgMjAx MyBFRVQKU01BUlQgc3VwcG9ydCBpczogQXZhaWxhYmxlIC0gZGV2aWNlIGhhcyBTTUFSVCBj YXBhYmlsaXR5LgpTTUFSVCBzdXBwb3J0IGlzOiBFbmFibGVkCgo9PT0gU1RBUlQgT0YgUkVB RCBTTUFSVCBEQVRBIFNFQ1RJT04gPT09ClNNQVJUIG92ZXJhbGwtaGVhbHRoIHNlbGYtYXNz ZXNzbWVudCB0ZXN0IHJlc3VsdDogUEFTU0VECgpTTUFSVCBBdHRyaWJ1dGVzIERhdGEgU3Ry dWN0dXJlIHJldmlzaW9uIG51bWJlcjogMTYKVmVuZG9yIFNwZWNpZmljIFNNQVJUIEF0dHJp YnV0ZXMgd2l0aCBUaHJlc2hvbGRzOgpJRCMgQVRUUklCVVRFX05BTUUgICAgICAgICAgRkxB RyAgICAgVkFMVUUgV09SU1QgVEhSRVNIIFRZUEUgICAgICBVUERBVEVEICBXSEVOX0ZBSUxF RCBSQVdfVkFMVUUKICAxIFJhd19SZWFkX0Vycm9yX1JhdGUgICAgIDB4MDAyZiAgIDIwMCAg IDIwMCAgIDA1MSAgICBQcmUtZmFpbCAgQWx3YXlzICAgICAgIC0gICAgICAgMAogIDMgU3Bp bl9VcF9UaW1lICAgICAgICAgICAgMHgwMDI3ICAgMTQyICAgMTQyICAgMDIxICAgIFByZS1m YWlsICBBbHdheXMgICAgICAgLSAgICAgICAzODc1CiAgNCBTdGFydF9TdG9wX0NvdW50ICAg ICAgICAweDAwMzIgICAxMDAgICAxMDAgICAwMDAgICAgT2xkX2FnZSAgIEFsd2F5cyAgICAg ICAtICAgICAgIDE1CiAgNSBSZWFsbG9jYXRlZF9TZWN0b3JfQ3QgICAweDAwMzMgICAyMDAg ICAyMDAgICAxNDAgICAgUHJlLWZhaWwgIEFsd2F5cyAgICAgICAtICAgICAgIDAKICA3IFNl ZWtfRXJyb3JfUmF0ZSAgICAgICAgIDB4MDAyZSAgIDIwMCAgIDIwMCAgIDAwMCAgICBPbGRf YWdlICAgQWx3YXlzICAgICAgIC0gICAgICAgMAogIDkgUG93ZXJfT25fSG91cnMgICAgICAg ICAgMHgwMDMyICAgMDk4ICAgMDk4ICAgMDAwICAgIE9sZF9hZ2UgICBBbHdheXMgICAgICAg LSAgICAgICAxNjQ5CiAxMCBTcGluX1JldHJ5X0NvdW50ICAgICAgICAweDAwMzIgICAxMDAg ICAyNTMgICAwMDAgICAgT2xkX2FnZSAgIEFsd2F5cyAgICAgICAtICAgICAgIDAKIDExIENh bGlicmF0aW9uX1JldHJ5X0NvdW50IDB4MDAzMiAgIDEwMCAgIDI1MyAgIDAwMCAgICBPbGRf YWdlICAgQWx3YXlzICAgICAgIC0gICAgICAgMAogMTIgUG93ZXJfQ3ljbGVfQ291bnQgICAg ICAgMHgwMDMyICAgMTAwICAgMTAwICAgMDAwICAgIE9sZF9hZ2UgICBBbHdheXMgICAgICAg LSAgICAgICAxMwoxOTIgUG93ZXItT2ZmX1JldHJhY3RfQ291bnQgMHgwMDMyICAgMjAwICAg MjAwICAgMDAwICAgIE9sZF9hZ2UgICBBbHdheXMgICAgICAgLSAgICAgICAxMgoxOTMgTG9h ZF9DeWNsZV9Db3VudCAgICAgICAgMHgwMDMyICAgMjAwICAgMjAwICAgMDAwICAgIE9sZF9h Z2UgICBBbHdheXMgICAgICAgLSAgICAgICAyCjE5NCBUZW1wZXJhdHVyZV9DZWxzaXVzICAg ICAweDAwMjIgICAxMTYgICAwOTUgICAwMDAgICAgT2xkX2FnZSAgIEFsd2F5cyAgICAgICAt ICAgICAgIDI3CjE5NiBSZWFsbG9jYXRlZF9FdmVudF9Db3VudCAweDAwMzIgICAyMDAgICAy MDAgICAwMDAgICAgT2xkX2FnZSAgIEFsd2F5cyAgICAgICAtICAgICAgIDAKMTk3IEN1cnJl bnRfUGVuZGluZ19TZWN0b3IgIDB4MDAzMiAgIDIwMCAgIDIwMCAgIDAwMCAgICBPbGRfYWdl ICAgQWx3YXlzICAgICAgIC0gICAgICAgMAoxOTggT2ZmbGluZV9VbmNvcnJlY3RhYmxlICAg MHgwMDMwICAgMTAwICAgMjUzICAgMDAwICAgIE9sZF9hZ2UgICBPZmZsaW5lICAgICAgLSAg ICAgICAwCjE5OSBVRE1BX0NSQ19FcnJvcl9Db3VudCAgICAweDAwMzIgICAyMDAgICAyMDAg ICAwMDAgICAgT2xkX2FnZSAgIEFsd2F5cyAgICAgICAtICAgICAgIDAKMjAwIE11bHRpX1pv bmVfRXJyb3JfUmF0ZSAgIDB4MDAwOCAgIDIwMCAgIDIwMCAgIDAwMCAgICBPbGRfYWdlICAg T2ZmbGluZSAgICAgIC0gICAgICAgMAogCg== --------------050809020106070702000508--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?511F5779.5030805>