Date: Wed, 21 Apr 2004 15:54:14 -0400 From: Garance A Drosihn <drosih@rpi.edu> To: Eric Anderson <anderson@centtech.com>, freebsd-current@freebsd.org Subject: Re: Directories with 2million files Message-ID: <p06020415bcac7bcec60c@[128.113.24.47]> In-Reply-To: <40867A5D.9010600@centtech.com> References: <40867A5D.9010600@centtech.com>
next in thread | previous in thread | raw e-mail | index | archive | help
At 8:42 AM -0500 4/21/04, Eric Anderson wrote: >... I'm not sure if there is a limit to this number, but at >least we know it works to 2million. I'm running 5.2.1-RELEASE. > >However, several tools seem to choke on that many files - mainly >ls and du. Find works just fine. Here's what my directory looks >like (from the parent): > >drwxr-xr-x 2 anderson anderson 50919936 Apr 21 08:25 data > >and when I cd into that directory, and do an ls: > >$ ls -al | wc -l >ls: fts_read: Cannot allocate memory > 0 > >Watching memory usage, it goes up to about 515Mb, and runs out >of memory (can't swap it), and then dies. (I only have 768Mb in >this machine). An `ls -al' is going to be doing a lot of work, most of which you probably do not care about. (Certainly not if you're just piping it to `wc'!). Depending on what you are looking for, an `ls -1Af' might work better. If you really do want the -l (lowercase L) instead of -1 (digit one), it *might* help to add the -h option. I probably should look at the source code to see if that's really true, but it's so much easier to just have you type in the command and see what happens... Another option is to use the `stat' command instead of `ls'. (I don't know if `stat' will work any better, I'm just saying it's another option you might want to try...). One advantage is that you'd have much better control over what information is printed. >du does the exact same thing. Just a plain `du'? If all you want is the total, did you try `du -s'? I would not expect any problem from `du -s'. >I'd work on some patches, but I'm not worth much when it comes >to C/C++. If someone has some patches, or code to try, let me >know - I'd be more than willing to test, possibly even give out >an account on the machine. It is probably possible to make `ls' behave better in this situation, though I don't know how much of a special-case we would need to make it. -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?p06020415bcac7bcec60c>