From owner-freebsd-stable@FreeBSD.ORG Thu Jan 20 17:57:54 2005 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5C6D216A4CE for ; Thu, 20 Jan 2005 17:57:54 +0000 (GMT) Received: from lora.pns.networktel.net (lora.pns.networktel.net [216.83.236.238]) by mx1.FreeBSD.org (Postfix) with ESMTP id CB6B743D1F for ; Thu, 20 Jan 2005 17:57:53 +0000 (GMT) (envelope-from phill@sysctl.net) Received: from MESE (wifi-ipnet.sysctl.net [216.83.242.102]) j0KHxkTB011455; Thu, 20 Jan 2005 17:59:47 GMT (envelope-from phill@sysctl.net) From: "Phillip Salzman" To: "'Peter Jeremy'" Date: Thu, 20 Jan 2005 11:58:19 -0600 Message-ID: <008001c4ff19$a38deff0$6745a8c0@MESE> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.6626 In-Reply-To: <20050120094551.GK79646@cirb503493.alcatel.com.au> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Importance: Normal X-AntiVirus: checked by Vexira Milter 1.0.6; VAE 6.29.0.5; VDF 6.29.0.52 cc: stable@freebsd.org Subject: RE: Very large directory X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jan 2005 17:57:54 -0000 Thanks Peter (along with everyone else who's responded.) I've received a couple perl scripts from a few different people. I can't use any of them until this evening due to the current load of the machines though. Last night I ended up doing a strings on the directory, and taking its output into a for loop removing the files. -- Phillip Salzman > -----Original Message----- > From: Peter Jeremy [mailto:PeterJeremy@optushome.com.au] > Sent: January 20, 2005 3:46 AM > To: Phillip Salzman > Cc: stable@freebsd.org > Subject: Re: Very large directory > > > On Wed, 2005-Jan-19 21:30:53 -0600, Phillip Salzman wrote: > >They've been running for a little while now - and recently we've > >noticed a lot of disk space disappearing. Shortly after > that, a simple > >du into our /var/spool returned a not so nice error: > > > > du: fts_read: Cannot allocate memory > > > >No matter what command I run on that directory, I just don't seem to > >have enough available resources to show the files let alone delete > >them (echo *, ls, find, rm -rf, etc.) > > I suspect you will need to write something that uses > dirent(3) to scan the offending directory and delete (or > whatever) the files one by one. > > Skeleton code (in perl) would look like: > > chdir $some_dir or die "Can't cd $some_dir: $!"; > opendir(DIR, ".") or die "Can't opendir: $!"; > while (my $file = readdir(DIR)) { > next if ($file eq '.' || $file eq '..'); > next if (&this_file_is_still_needed($file)); > unlink $file or warn "Unable to delete $file: $!"; > } > closedir DIR; > > If you've reached the point where you can't actually read the > entire directory into user memory, expect the cleanup to take > quite a while. > > Once you've finished the cleanup, you should confirm that the > directory has shrunk to a sensible size. If not, you need to > re-create the directory and move the remaining files into the > new directory. > > -- > Peter Jeremy >