From owner-freebsd-fs@FreeBSD.ORG Fri Nov 25 06:08:12 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89E14106564A; Fri, 25 Nov 2011 06:08:12 +0000 (UTC) (envelope-from areilly@bigpond.net.au) Received: from nskntqsrv02p.mx.bigpond.com (nskntqsrv02p.mx.bigpond.com [61.9.168.234]) by mx1.freebsd.org (Postfix) with ESMTP id 0E2DA8FC13; Fri, 25 Nov 2011 06:08:11 +0000 (UTC) Received: from nskntcmgw07p ([61.9.169.167]) by nskntmtas06p.mx.bigpond.com with ESMTP id <20111124230825.KUAE28461.nskntmtas06p.mx.bigpond.com@nskntcmgw07p>; Thu, 24 Nov 2011 23:08:25 +0000 Received: from johnny.reilly.home ([124.188.161.100]) by nskntcmgw07p with BigPond Outbound id 1B8N1i00K2AGJ5o01B8Rhe; Thu, 24 Nov 2011 23:08:25 +0000 X-Authority-Analysis: v=2.0 cv=N56r5hBB c=1 sm=1 a=+rWFdGQzZE3xDYVtG1Y/Og==:17 a=z1TLwsU0kBEA:10 a=R0KOSDEtWRsA:10 a=kj9zAlcOel0A:10 a=69ks7eD_NYYmXlrLNYgA:9 a=dd4w7-GCHr3Z5rRXV2QA:7 a=CjuIK1q_8ugA:10 a=+rWFdGQzZE3xDYVtG1Y/Og==:117 Date: Fri, 25 Nov 2011 10:08:22 +1100 From: Andrew Reilly To: John Message-ID: <20111124230822.GA96603@johnny.reilly.home> References: <20111101034118.GA73746@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111101034118.GA73746@FreeBSD.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-fs@freebsd.org Subject: Re: (Yet Another) Damaged directory on ZFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 06:08:12 -0000 On Tue, Nov 01, 2011 at 03:41:18AM +0000, John wrote: > Hi Folks, > > We have a zfs fileserver running 9.0-RC1 which appears to have locked > up in a manner similar to the "Damanged directory on ZFS" thread. > > It started after a zfs snapshat which appears to have hung. At > that point, an "ls" command on a directory which is either empty > or contains only other directories works correctly. An "ls" on > a directory containing a file will hang. Just want to add a sort-of "me too", to this question, in the hope that I will learn something useful. For a couple of weeks I've been seeing messages in my daily security reports along the lines of: find: /usr/src/.zfs/snapshot: bad file descriptor I can get the same message now by just cd'ing into one of the two "broken" .zfs directories and comparing ls -F output with a version that doesn't check the inode data: ls: snapshot: Bad file descriptor shares/ "zfs list -r -t snapshot" shows all of the snapshots created by my nightly backups to still exist, seemingly. A zpool scrub tank did not change or make them go away. No processes appear to be stuck, wedged or otherwise broken. FWIW I'm running: FreeBSD johnny.reilly.home 9.0-RC1 FreeBSD 9.0-RC1 #4: Sat Nov 5 14:52:15 EST 2011 root@johnny.reilly.home:/usr/obj/usr/src/sys/GENERIC amd64 Cheers, -- Andrew