Date: Sun, 15 Aug 2004 00:39:33 +0100 From: Matthew Seaman <m.seaman@infracaninophile.co.uk> To: "Paul A. Hoadley" <paulh@logicsquad.net> Cc: freebsd-questions@freebsd.org Subject: Re: find -exec surprisingly slow Message-ID: <20040814233933.GA14322@happy-idiot-talk.infracaninophile.co.uk> In-Reply-To: <20040814230143.GB8610@grover.logicsquad.net> References: <20040814230143.GB8610@grover.logicsquad.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--tKW2IUtsqtDRztdT
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Sun, Aug 15, 2004 at 08:31:43AM +0930, Paul A. Hoadley wrote:
> Hello,
>=20
> I'm in the process of cleaning a Maildir full of spam. It has
> somewhere in the vicinity of 400K files in it. I started running
> this yesterday:
>=20
> find . -atime +1 -exec mv {} /home/paulh/tmp/spam/sne/ \;
>=20
> It's been running for well over 12 hours. It certainly is
> working---the spams are slowly moving to their new home---but it is
> taking a long time. It's a very modest system, running 4.8-R on a
> P2-350. I assume this is all overhead for spawning a shell and
> running mv 400K times. Is there a better way to move all files based
> on some characteristic of their date stamp? Maybe separating the find
> and the move, piping it through xargs? It's mostly done now, but I
> will know better for next time.
Yup. Invoking mv 40,000 times is not particularly efficient.
Something like this would have been better:
find . -atime +1 -print0 | xargs -0 -J % mv % /home/paulh/tmp/spam/sne/
xargs defaults to taking up to 5,000 arguments from it's stdin to
generate the mv commands (or up to ARG_MAX - 4096 =3D 61440 bytes), so
that would have done the job with only 8 or so invocations of mv.
Cheers,
Matthew
--=20
Dr Matthew J Seaman MA, D.Phil. 26 The Paddocks
Savill Way
PGP: http://www.infracaninophile.co.uk/pgpkey Marlow
Tel: +44 1628 476614 Bucks., SL7 1TH UK
--tKW2IUtsqtDRztdT
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (FreeBSD)
iD8DBQFBHqK1iD657aJF7eIRAo8CAJ90US/h4zGI8Re1A8rIt58kCzwr1ACfTLqw
rweM21V1+MW5pZppe9pO+s8=
=SZHL
-----END PGP SIGNATURE-----
--tKW2IUtsqtDRztdT--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040814233933.GA14322>