From owner-freebsd-current@FreeBSD.ORG Wed Apr 21 13:57:59 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E7A4716A4CE for ; Wed, 21 Apr 2004 13:57:59 -0700 (PDT) Received: from gabby.gsicomp.on.ca (CPE00062566c7bb-CM000039c69a66.cpe.net.cable.rogers.com [69.197.202.178]) by mx1.FreeBSD.org (Postfix) with ESMTP id 22A9B43D4C for ; Wed, 21 Apr 2004 13:57:59 -0700 (PDT) (envelope-from matt@gsicomp.on.ca) Received: from hermes (hermes.gsicomp.on.ca [192.168.0.18]) by gabby.gsicomp.on.ca (8.12.9p2/8.12.9) with ESMTP id i3LLW2Xf032251; Wed, 21 Apr 2004 17:32:03 -0400 (EDT) (envelope-from matt@gsicomp.on.ca) Message-ID: <001801c427e2$da2f4680$1200a8c0@gsicomp.on.ca> From: "Matt Emmerton" To: "Eric Anderson" , "Garance A Drosihn" References: <40867A5D.9010600@centtech.com> <4086D513.9010605@centtech.com> Date: Wed, 21 Apr 2004 16:54:33 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1409 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409 cc: freebsd-current@freebsd.org Subject: Re: Directories with 2million files X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Apr 2004 20:58:00 -0000 > At 3:09 PM -0500 4/21/04, Eric Anderson wrote: > >Garance A Drosihn wrote: > > > >>... If you really do want the -l (lowercase L) > >>instead of -1 (digit one), it *might* help to add the -h option. > > > >Used 263MB, before returning the correct number.. It's functional, > >but only if you have a lot of ram. > > Darn. Well, that was a bit of a long-shot, but worth a try. > > >>Another option is to use the `stat' command instead of `ls'. > >>One advantage is that you'd have much better control over > >>what information is printed. > > > >I'm not sure how to use stat to get that same info. > > Oops. My fault. I thought the `stat' command had an option to > list all files in a given directory. I guess you'd have to combine > it with `find' to do that. > > >>>du does the exact same thing. > >> > >>Just a plain `du'? If all you want is the total, did you > >>try `du -s'? I would not expect any problem from `du -s'. > > > >$ du -s > >du: fts_read: Cannot allocate memory > > Huh. Well, that seems pretty broken... > > >>>I'd work on some patches, but I'm not worth much when it comes > >>>to C/C++. If someone has some patches, or code to try, let me > >>>know - I'd be more than willing to test, possibly even give out > >>>an account on the machine. > >> > >> > >>It is probably possible to make `ls' behave better in this > >>situation, though I don't know how much of a special-case > >>we would need to make it. > > > > > >I suppose this is one of those "who needs files bigger than 2gb?" > >things.. > > Perhaps, but as a general rule we'd like our system utilities to > at least *work* in extreme situations. This is something I'd > love to dig into if I had the time, but I'm not sure I have the > time right now. A quick inspection of the code (on my 4.9-REL-p2 machine) shows that there's a memory leak. The following malloc() does not have a matching free(). I looked at the latest version in CVS (rev 1.76) and this hasn't been fixed. 537 /* Fill-in "::" as "0:0:0" for the sake of scanf. */ 538 jinitmax = initmax2 = malloc(strlen(initmax) * 2 + 2); We need to add a free() for this; here's my suggestion. 596 maxinode = makenines(maxinode); 597 maxblock = makenines(maxblock); 598 maxnlink = makenines(maxnlink); 599 maxsize = makenines(maxsize); 600 601 free(jinitmax); /* MTE */ 602 603 } -- Matt Emmerton