From owner-freebsd-fs@FreeBSD.ORG Wed Apr 25 02:44:34 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6FF2D1065672 for ; Wed, 25 Apr 2012 02:44:34 +0000 (UTC) (envelope-from areilly@bigpond.net.au) Received: from nskntqsrv02p.mx.bigpond.com (nskntqsrv02p.mx.bigpond.com [61.9.168.234]) by mx1.freebsd.org (Postfix) with ESMTP id 00D5F8FC12 for ; Wed, 25 Apr 2012 02:44:33 +0000 (UTC) Received: from nskntcmgw06p ([61.9.169.166]) by nskntmtas03p.mx.bigpond.com with ESMTP id <20120424232137.HEJW10464.nskntmtas03p.mx.bigpond.com@nskntcmgw06p>; Tue, 24 Apr 2012 23:21:37 +0000 Received: from johnny.reilly.home ([124.188.161.100]) by nskntcmgw06p with BigPond Outbound id 1zMc1j0082AGJ5o01zMccA; Tue, 24 Apr 2012 23:21:37 +0000 X-Authority-Analysis: v=2.0 cv=MaHuSuDf c=1 sm=1 a=+rWFdGQzZE3xDYVtG1Y/Og==:17 a=z1TLwsU0kBEA:10 a=ea6dOSa9tC4A:10 a=kj9zAlcOel0A:10 a=RBzxiYmM5o6yayzINFUA:9 a=PZ93UOqz-Kex-gf7B3AA:7 a=CjuIK1q_8ugA:10 a=+rWFdGQzZE3xDYVtG1Y/Og==:117 Date: Wed, 25 Apr 2012 09:21:36 +1000 From: Andrew Reilly To: Peter Maloney Message-ID: <20120424232136.GA1441@johnny.reilly.home> References: <20120424143014.GA2865@johnny.reilly.home> <4F96BAB9.9080303@brockmann-consult.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4F96BAB9.9080303@brockmann-consult.de> User-Agent: Mutt/1.4.2.3i Cc: freebsd-fs@freebsd.org Subject: Re: Odd file system corruption in ZFS pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 02:44:34 -0000 On Tue, Apr 24, 2012 at 04:37:45PM +0200, Peter Maloney wrote: > On 04/24/2012 04:30 PM, Andrew Reilly wrote: > >ZFS filesystem version 5 > >ZFS storage pool version 28 > Is there anything special about your pool? Was it created in an old > FreeBSD and upgraded? Was it send/recv'd from OpenSolaris? etc. I don't know enough about zfs to know whether there's anything special about it, I'm afraid. The pool "tank" is a raidz across four 1T Seagate NS series drives. The first incarnation died from corruption (boot panic loop after a zpool scrub) a year or so ago, so the current system is new since then. The first had been upgraded at least once, not sure about the current. Has only ever been attached to this (regularly upgraded) _STABLE system. It isn't protected by a UPS, and the power has been going out without warning fairly regularly, so IMO that is sufficient to explain the cause of the corruption. Setting up a UPS is my next project. Interesting update to last night's message: the corruption is rubust under send/receive of snapshots: the last version of my backup exhibits exactly the same problem. (That is: a directory that shows up in glob expansion but can't be removed or touched, and a directory full of files that find returns: Unknown error: 122.) > So far the only corruption I had was the result of installing FreeBSD on > a 4 GB USB flash stick. It had no redundancy, and within a few months, > some files were spontaneously broken. > > And in that one instance I found that move, copy, etc. on broken files > reported by zpool status -v will always fail. Only "rm" worked for me. > So I suggest you try rmdir or rm -r. Rm and rm -r doesn't work. Even as root, rm -rf Maildir.bad returns a lot of messages of the form: foo/bar: no such file or directory. The result is that I now have a directory that contains no "good" files, but a concentrated collection of breakage. I have another zpool scrub running at the moment. We'll see if that is able to clean it up, but it hasn't had much luck in the past. Note that none of these broken files or directories show up in the zpool status -v error list. That just contains the one entry for the zfs root directory: tank/home:<0x0> Cheers, -- Andrew