From owner-freebsd-questions  Thu Oct 23 02:01:51 1997
Return-Path: <owner-freebsd-questions>
Received: (from root@localhost)
          by hub.freebsd.org (8.8.7/8.8.7) id CAA12197
          for questions-outgoing; Thu, 23 Oct 1997 02:01:51 -0700 (PDT)
          (envelope-from owner-freebsd-questions)
Received: from unicorn.uk1.vbc.net (unicorn.uk1.vbc.net [194.207.2.11])
          by hub.freebsd.org (8.8.7/8.8.7) with ESMTP id CAA12189
          for <questions@FreeBSD.ORG>; Thu, 23 Oct 1997 02:01:47 -0700 (PDT)
          (envelope-from gordon@drogon.net)
Received: from localhost (gordon@localhost)
	by unicorn.uk1.vbc.net (8.8.5/8.8.5) with SMTP id KAA14251;
	Thu, 23 Oct 1997 10:01:50 +0100
Date: Thu, 23 Oct 1997 10:01:46 +0100 (BST)
From: Gordon Henderson <gordon@drogon.net>
X-Sender: gordon@unicorn
To: Greg Lehey <grog@lemis.com>
cc: Shawn Ramsey <shawn@luke.cpl.net>, questions@FreeBSD.ORG
Subject: Re: deleted huge directory
In-Reply-To: <19971023181950.01356@lemis.com>
Message-ID: <Pine.LNX.3.95.971023095452.14037D-100000@unicorn>
Distribution: world
Organization: Home for lost Drogons
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: owner-freebsd-questions@FreeBSD.ORG
X-Loop: FreeBSD.org
Precedence: bulk

On Thu, 23 Oct 1997, Greg Lehey wrote:

> On Thu, Oct 23, 1997 at 09:39:55AM +0100, Gordon Henderson wrote:
> > On Thu, 23 Oct 1997, Greg Lehey wrote:
> >
> >> On Wed, Oct 22, 1997 at 09:38:52PM -0700, Shawn Ramsey wrote:
> >>> Does anyone have any suggestions for deleting a huge directory? The
> >>> direcory is /var/news/spool/control/cancel. The directory is too large to
> >>> even do a ls. If I try doing ls, it eats up all the avail. RAM(140+mb) and
> >>> hangs. Anyone? I would try a wildcard, but I don't know what the filenames
> >>> are. (This is a very large news server, and this directory has never been
> >>> touch as far as I know.)
> >>
> >> Doesn't rm -rf work?  Otherwise you could try
> >>
> >>   find dir | xargs rm -rf
> >
> > I do the following to clear this from time to time
> >
> >   cd /var/news/spool/control
> >   mv cancel cancel.old
> >
> > (then a new one gets created and news keeps on going)
> >
> >   cd cancel.old
> >   ls -f > /var/tmp/cancel.rm
> >   xargs rm -f < /var/tmp/cancel.rm
> 
> This is unnecessarily complicated.  Together, they're approximately
> the same as what I suggested above.  You certainly don't need to store
> the output of ls in a file.

Find will try to sort the directory first before printing it - I belive
it's the sort that causes the machine to run out of memory. The cancel
directory is typically huge: eg: 

  news @ news.uk0: ls -ld cancel
  drwxrwxr-x  2 news  news  847360 Oct 23 07:52 cancel
  news @ news.uk0: cd cancel
  news @ news.uk0: ls -f | wc -l
     20412

Only 20K articles in there today so far, but my expire has just finished
and I've seen it over 100K.

And yes, it's probably true that you don't have to store it in a file, but
I always wonder the wisdom of modifying the thing you are reading from!

Gordon