From owner-freebsd-current@FreeBSD.ORG  Wed Apr 21 07:19:32 2004
Return-Path: <owner-freebsd-current@FreeBSD.ORG>
Delivered-To: freebsd-current@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 7CAEC16A4CE
	for <freebsd-current@freebsd.org>;
	Wed, 21 Apr 2004 07:19:32 -0700 (PDT)
Received: from smtp01.syd.iprimus.net.au (smtp01.syd.iprimus.net.au
	[210.50.30.52])	by mx1.FreeBSD.org (Postfix) with ESMTP id 98FC843D1D
	for <freebsd-current@freebsd.org>;
	Wed, 21 Apr 2004 07:19:31 -0700 (PDT)
	(envelope-from tim@robbins.dropbear.id.au)
Received: from robbins.dropbear.id.au (210.50.251.51) by
	smtp01.syd.iprimus.net.au (7.0.024)
	id 402BA927019F9725; Thu, 22 Apr 2004 00:19:26 +1000
Received: by robbins.dropbear.id.au (Postfix, from userid 1000)
	id 7879A41DF; Thu, 22 Apr 2004 01:22:33 +1000 (EST)
Date: Thu, 22 Apr 2004 01:22:33 +1000
From: Tim Robbins <tjr@freebsd.org>
To: Eric Anderson <anderson@centtech.com>
Message-ID: <20040421152233.GA23501@cat.robbins.dropbear.id.au>
References: <40867A5D.9010600@centtech.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <40867A5D.9010600@centtech.com>
User-Agent: Mutt/1.4.1i
cc: freebsd-current@freebsd.org
Subject: Re: Directories with 2million files
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
	<freebsd-current.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
	<mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
	<mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Apr 2004 14:19:32 -0000

On Wed, Apr 21, 2004 at 08:42:53AM -0500, Eric Anderson wrote:

> First, let me say that I am impressed (but not shocked) - FreeBSD 
> quietly handled my building of a directory with 2055476 files in it.  
> I'm not sure if there is a limit to this number, but at least we know it 
> works to 2million.  I'm running 5.2.1-RELEASE.
> 
> However, several tools seem to choke on that many files - mainly ls and 
> du.  Find works just fine.  Here's what my directory looks like (from 
> the parent):
> 
> drwxr-xr-x   2 anderson  anderson  50919936 Apr 21 08:25 data
> 
> and when I cd into that directory, and do an ls:
> 
> $ ls -al | wc -l
> ls: fts_read: Cannot allocate memory
>       0

The problem here is likely to be that ls is trying to store all the
filenames in memory in order to sort them. Try using the -f option
to disable sorting. If you really do need a sorted list of filenames,
pipe the output through 'sort'.


Tim