From owner-freebsd-fs@FreeBSD.ORG Thu Feb 17 00:56:02 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 99A071065673 for ; Thu, 17 Feb 2011 00:56:02 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 51A1E8FC0A for ; Thu, 17 Feb 2011 00:56:02 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1PpsA4-00060M-Ra for freebsd-fs@freebsd.org; Thu, 17 Feb 2011 01:56:00 +0100 Received: from cpe-188-129-78-195.dynamic.amis.hr ([188.129.78.195]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 17 Feb 2011 01:56:00 +0100 Received: from ivoras by cpe-188-129-78-195.dynamic.amis.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 17 Feb 2011 01:56:00 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Thu, 17 Feb 2011 01:55:46 +0100 Lines: 27 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: cpe-188-129-78-195.dynamic.amis.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20101102 Thunderbird/3.1.6 In-Reply-To: Subject: Re: zfs directory listing X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Feb 2011 00:56:02 -0000 On 16/02/2011 23:52, Andrew Thompson wrote: > Hi, > > > I have a zfs file system on 8.1-RELEASE amd64 which as a large number > of files in /var/spool/mqueue. loader.conf has vfs.zfs.arc_max=2G for > an 8G box. > mqueue has a link count of 522824. I can not list the contents of this > directory, when I do the number of read IOPS sits> 100 and it will > never complete Your problem is probably not FreeBSD-specific and possibly not even ZFS-specific. That is a fairly large number of files in a directory for any file system; The rule of thumb is usually to start sharding as soon as the number of files gets even two orders of magnitude lower than what you have there. As others said, try "cd /var/spool/mqueue && find ." - the find utility just reads the directory, it doesn't try to gather other metadata which "ls" uses and is usually one of the rare utilities which can work with gigantic directories (usually in the form "find . -delete" :) ). The reason for this is that filenames and file metadata are separate objects on the drives and the drives need to seek between them to get both if they are not cached.