From owner-freebsd-fs@freebsd.org Fri Sep 2 16:06:48 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 87574BCD82F for ; Fri, 2 Sep 2016 16:06:48 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: from mail-it0-x230.google.com (mail-it0-x230.google.com [IPv6:2607:f8b0:4001:c0b::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 58CE5791 for ; Fri, 2 Sep 2016 16:06:48 +0000 (UTC) (envelope-from lkateley@kateley.com) Received: by mail-it0-x230.google.com with SMTP id e124so45385605ith.0 for ; Fri, 02 Sep 2016 09:06:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kateley-com.20150623.gappssmtp.com; s=20150623; h=reply-to:from:organization:to:subject:message-id:date:user-agent :mime-version:content-transfer-encoding; bh=PbOF7vreYwNuVG0tht9pj6pMdo57J+pHA8ySIZ5BOkA=; b=ToFN0ZT/Icwshj/glxyxzGobyWPQi3i1GJn2BWUP2IELZVay7+NqPek0OLcegpGXR7 fAYWm4rKynCJyHZ2O0VTdhTmk4+nGVv9X3I+560tLDMEbg4qzfTBE9jg9DwuUkNn6X8d k20V1r0jFllcc5Ir78Bud2adn0HA/evka+YBWQDubYpexkjwotfLGYeQQlx3NO20uLkI Zz1H7t7tWxp6A6J/tigw46u+WJ4fTpaTY++xfUgRydJmkiJuumldYkiCBU7jEUEZihx9 CU93EXHrQMiTq1OKaiNigHM0ToJOhnQ6Ey4bFVQM4teZLcjT979NTeOTQfRs+erTWoLI YoDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:reply-to:from:organization:to:subject:message-id :date:user-agent:mime-version:content-transfer-encoding; bh=PbOF7vreYwNuVG0tht9pj6pMdo57J+pHA8ySIZ5BOkA=; b=QI/6jN+c/W+D6Z80VjnV0JYLzMfx5NNyyihGEMmHQOOjbJvli3ODxsLogCdXKOr/vy QqXiIRld7H/QMWqZgrBFPRAKA6q58uhH36UgwnHQs0MHfUnUz2yz4EH3PM8c1849mEvD EKpJxzbS0EiB3+7jNd34fo4mvyT4RugFadIWGEg/ed8Zk9c72RpIjq5wTnLOfmQQ/8Mb HuNk4SHyxvs8IXFJP7s5AtdHl9b4Yf+GLWX1u8RxZYkMtRkDUifVTOUt/XMnxxvU62d/ 5EMIZFu092Egdn9AXQnjHUJxoxt4qhI3+sYnCs4mPLAzgwNuYV6bmTsJ9aVAEHWTLLbQ Rb8g== X-Gm-Message-State: AE9vXwNWkqyhQWdQcBktsjWYQrR0EHlAKBmkfhfo0AxswALb8Z3GRAv5neJXwJminxTSNw== X-Received: by 10.36.237.6 with SMTP id r6mr5657182ith.62.1472832407363; Fri, 02 Sep 2016 09:06:47 -0700 (PDT) Received: from Kateleyco-iMac.local (c-50-188-36-30.hsd1.mn.comcast.net. [50.188.36.30]) by smtp.googlemail.com with ESMTPSA id 194sm2823781itu.4.2016.09.02.09.06.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Sep 2016 09:06:46 -0700 (PDT) Reply-To: linda@kateley.com From: Linda Kateley Organization: Kateley Company To: FreeBSD Filesystems Subject: phantom snapshots Message-ID: <58816ff0-3ab2-cbae-3d50-c4d5e89d9773@kateley.com> Date: Fri, 2 Sep 2016 11:06:46 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Sep 2016 16:06:48 -0000 So I think I have isolated this to a phantom snapshot. At the end of this zpool status although the names change everytime I run scrub, it show unrecoverable error of files in a snapshot that doesn't exit. It also has in that zfs dataset it has a .zfs directory with a name of old snaps. Anyone know how to view or clear these phantom snaps? I tried to remove the files in the snapshot directory, but it is read-only. Ideas? Linda pool: pool state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Fri Sep 2 10:42:48 2016 46.8T scanned out of 144T at 2.48G/s, 11h11m to go 46.5G resilvered, 32.40% done config: NAME STATE READ WRITE CKSUM pool ONLINE 0 0 12 raidz3-0 ONLINE 0 0 0 gptid/ac108605-265c-11e5-9a02-0cc47a599098 ONLINE 0 0 0 gptid/ac591d4e-265c-11e5-9a02-0cc47a599098 ONLINE 0 0 0 gptid/ac92fd0d-265c-11e5-9a02-0cc47a599098 ONLINE 0 0 0 gptid/accd3076-265c-11e5-9a02-0cc47a599098 ONLINE 0 0 0 gptid/ad067e97-265c-11e5-9a02-0cc47a599098 ONLINE 0 0 0 gptid/ad46cbee-265c-11e5-9a02-0cc47a599098 ONLINE 0 0 0 gptid/ad91ba17-265c-11e5-9a02-0cc47a599098 ONLINE 0 0 0 gptid/adcbdd0a-265c-11e5-9a02-0cc47a599098 ONLINE 0 0 0 gptid/ae07dc0d-265c-11e5-9a02-0cc47a599098 ONLINE 0 0 0 gptid/ae494d10-265c-11e5-9a02-0cc47a599098 ONLINE 0 0 0 gptid/ae93a3a5-265c-11e5-9a02-0cc47a599098 ONLINE 0 0 0 raidz3-1 ONLINE 0 0 0 gptid/12f6a4c5-c929-11e5-8075-0cc47a599098 ONLINE 0 0 0 gptid/511ea1f9-1932-11e6-9b1e-0cc47a599098 ONLINE 0 0 0 gptid/14436fcf-c929-11e5-8075-0cc47a599098 ONLINE 0 0 0 gptid/14f50aa3-c929-11e5-8075-0cc47a599098 ONLINE 0 0 0 gptid/159b5654-c929-11e5-8075-0cc47a599098 ONLINE 0 0 0 gptid/163d682b-c929-11e5-8075-0cc47a599098 ONLINE 0 0 0 gptid/16ee624e-c929-11e5-8075-0cc47a599098 ONLINE 0 0 0 gptid/1799dde3-c929-11e5-8075-0cc47a599098 ONLINE 0 0 0 gptid/184c2ea4-c929-11e5-8075-0cc47a599098 ONLINE 0 0 0 gptid/18f51c30-c929-11e5-8075-0cc47a599098 ONLINE 0 0 0 gptid/19a861ea-c929-11e5-8075-0cc47a599098 ONLINE 0 0 0 raidz3-2 ONLINE 0 0 24 gptid/5f80fc42-4e00-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/60369e0f-4e00-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/60e8234a-4e00-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/61a235f2-4e00-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/62580471-4e00-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/6316a38a-4e00-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/63d4bce8-4e00-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/ebfc2b99-6893-11e6-9b09-0cc47a599098 ONLINE 0 0 0 (resilvering) gptid/654f143a-4e00-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/66236b33-4e00-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/66eda3f6-4e00-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 raidz3-3 ONLINE 0 0 24 gptid/c77a9da9-4e02-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/c83e100e-4e02-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/c8fd9ced-4e02-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/c9bb21ba-4e02-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/ca7a48db-4e02-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/cb422329-4e02-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/cbfe4c21-4e02-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/ccc43528-4e02-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/cd93a34c-4e02-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/ce622f51-4e02-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 gptid/cf2591d3-4e02-11e6-b7cf-0cc47a599098 ONLINE 0 0 0 cache gptid/aedd3872-265c-11e5-9a02-0cc47a599098 ONLINE 0 0 0 gptid/af559c10-265c-11e5-9a02-0cc47a599098 ONLINE 0 0 0 errors: Permanent errors have been detected in the following files: <0x597>:<0x1246> <0x597>:<0x1249> <0x597>:<0x125e> <0x597>:<0x1261> <0x597>:<0x127d> <0x597>:<0x1280> <0x599>:<0xe2c> <0x599>:<0xe2f> <0x599>:<0xe40> <0x599>:<0xe43> <0x599>:<0xe59> <0x599>:<0xe62> <0x5bc>:<0x2ad06> <0x5bc>:<0x2aef3> <0x5bc>:<0x2aef6> pool/.system@auto-20160720.2300-2d:<0x0> <0x5dc>:<0x397411> <0x5dc>:<0x397414> <0x5dc>:<0x38ee26> <0x5dc>:<0x38ee29> <0x5dc>:<0x38ee2c> <0x5dc>:<0x45b2e> <0x5dc>:<0x38ee2f> <0x5dc>:<0x38ee32> <0x5dc>:<0x394e58> <0x5dc>:<0x394e5b> <0x5dc>:<0x394e5e> <0x5dc>:<0x394e61> <0x5dc>:<0x394e64> <0x5dc>:<0x394e67> <0x5dc>:<0x394e6a> <0x5dc>:<0x397285>