From owner-freebsd-current@FreeBSD.ORG Wed Apr 21 14:02:06 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0E07316A4CE for ; Wed, 21 Apr 2004 14:02:06 -0700 (PDT) Received: from gabby.gsicomp.on.ca (CPE00062566c7bb-CM000039c69a66.cpe.net.cable.rogers.com [69.197.202.178]) by mx1.FreeBSD.org (Postfix) with ESMTP id 783C543D53 for ; Wed, 21 Apr 2004 14:02:05 -0700 (PDT) (envelope-from matt@gsicomp.on.ca) Received: from hermes (hermes.gsicomp.on.ca [192.168.0.18]) by gabby.gsicomp.on.ca (8.12.9p2/8.12.9) with ESMTP id i3LLaAXf032258; Wed, 21 Apr 2004 17:36:10 -0400 (EDT) (envelope-from matt@gsicomp.on.ca) Message-ID: <003801c427e3$6e5dff90$1200a8c0@gsicomp.on.ca> From: "Matt Emmerton" To: "Eric Anderson" , "Garance A Drosihn" Date: Wed, 21 Apr 2004 16:58:41 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1409 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409 cc: freebsd-current@freebsd.org Subject: Re: Directories with 2million files [ CORRECTION ] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Apr 2004 21:02:06 -0000 > > At 3:09 PM -0500 4/21/04, Eric Anderson wrote: > > >Garance A Drosihn wrote: > > > > > >>... If you really do want the -l (lowercase L) > > >>instead of -1 (digit one), it *might* help to add the -h option. > > > > > >Used 263MB, before returning the correct number.. It's functional, > > >but only if you have a lot of ram. > > > > Darn. Well, that was a bit of a long-shot, but worth a try. > > > > >>Another option is to use the `stat' command instead of `ls'. > > >>One advantage is that you'd have much better control over > > >>what information is printed. > > > > > >I'm not sure how to use stat to get that same info. > > > > Oops. My fault. I thought the `stat' command had an option to > > list all files in a given directory. I guess you'd have to combine > > it with `find' to do that. > > > > >>>du does the exact same thing. > > >> > > >>Just a plain `du'? If all you want is the total, did you > > >>try `du -s'? I would not expect any problem from `du -s'. > > > > > >$ du -s > > >du: fts_read: Cannot allocate memory > > > > Huh. Well, that seems pretty broken... > > > > >>>I'd work on some patches, but I'm not worth much when it comes > > >>>to C/C++. If someone has some patches, or code to try, let me > > >>>know - I'd be more than willing to test, possibly even give out > > >>>an account on the machine. > > >> > > >> > > >>It is probably possible to make `ls' behave better in this > > >>situation, though I don't know how much of a special-case > > >>we would need to make it. > > > > > > > > >I suppose this is one of those "who needs files bigger than 2gb?" > > >things.. > > > > Perhaps, but as a general rule we'd like our system utilities to > > at least *work* in extreme situations. This is something I'd > > love to dig into if I had the time, but I'm not sure I have the > > time right now. > > A quick inspection of the code (on my 4.9-REL-p2 machine) shows that there's > a memory leak. > The following malloc() does not have a matching free(). I looked at the > latest version in CVS (rev 1.76) and this hasn't been fixed. > > 537 /* Fill-in "::" as "0:0:0" for the sake of scanf. */ > 538 jinitmax = initmax2 = malloc(strlen(initmax) * 2 + > 2); > > We need to add a free() for this; here's my suggestion. > > 596 maxinode = makenines(maxinode); > 597 maxblock = makenines(maxblock); > 598 maxnlink = makenines(maxnlink); > 599 maxsize = makenines(maxsize); > 600 > 601 free(jinitmax); /* MTE */ > 602 > 603 } Reading the code a bit more carefully, this won't fix the problem Eric is seeing, but it still is a problem for anyone who has "LS_COLWIDTHS" defined in their environment. -- Matt Emmerton