Date: Wed, 15 Jun 2011 13:14:32 +0200 From: dennis berger <db@bsdsystems.de> To: vadim_nuclight@mail.ru Cc: freebsd-performance@freebsd.org Subject: Re: strange differencies: find -exec \+ vs find | xargs Message-ID: <4E648192-7D2C-4556-B001-FDD812E821F8@bsdsystems.de> In-Reply-To: <slrnivh2kd.1g2d.vadim_nuclight@kernblitz.nuclight.avtf.net> References: <slrnivh2kd.1g2d.vadim_nuclight@kernblitz.nuclight.avtf.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Dear,
I suspect that you're hitting a cache limit for that machine.
I had similar behaviour when rsyncing huge directories. I usually raise =
vfs.ufs.dirhash_maxmem sysctl variable, which solves the problem.
best,
-dennis
Am 15.06.2011 um 12:33 schrieb Vadim Goncharov:
> Hi,
>=20
> There was a flamewar in a local maillist about the fastest way to do =
chmod 644
> on files and chmod 755 on directories in a sbubtree. It suddenly =
revealed the
> strange thing: on the same subtree, find -type d -exec \+ on 76000 =
directories
> has the same as find | xargs, but on 246000 files it was 4 times =
slower - while
> there must be almost identical number of forks.
>=20
> I've repeated the tests on deliberately old machine, a Pentium I 233 =
MHz with
> 80 Mb RAM, 7.4R UFS2. Subtree for tests was copy of /usr/src with =
several
> categories from /usr/ports. The output is from ZSH's builtin 'time' =
(also
> set up to automatically output if cmd was run more than 30 secs).
>=20
> Below is the ooutput, first with 80K files, it doesn't differ, than =
added
> several more /usr/ports categories and got slow result on 100K. Why is
> the difference at all? This machine has maxvnodes about 4600 and too =
small
> RAM for caches, if that's buffers, I'd expect the difference to begin =
on
> mush slow number of files. But it runs the same 80K normally...
>=20
> kernblitz:/home/vadim# find /warehouse/garbage -type f | wc -l; find =
/warehouse/garbage -type d | wc -l
> 80208
> find /warehouse/garbage -type f 5,01s user 53,12s system 53% cpu =
1:48,52 total
> wc -l 0,54s user 0,31s system 0% cpu 1:48,49 total
> 13989
> find /warehouse/garbage -type d 4,52s user 52,00s system 53% cpu =
1:45,63 total
> wc -l 0,11s user 0,06s system 0% cpu 1:45,61 total
>=20
> kernblitz:/home/vadim# time chmod -R u+rwX,go+rX,go-w =
/warehouse/garbage
> chmod -R u+rwX,go+rX,go-w /warehouse/garbage 4,39s user 54,94s system =
53% cpu 1:50,73 total
>=20
> kernblitz:/home/vadim# time find /warehouse/garbage -type f -exec =
chmod 644 {} \+
> find /warehouse/garbage -type f -exec chmod 644 {} \+ 7,29s user =
100,30s system 65% cpu 2:43,29 total
>=20
> kernblitz:/home/vadim# time find /warehouse/garbage -type d -exec =
chmod 755 {} \+
> find /warehouse/garbage -type d -exec chmod 755 {} \+ 5,04s user =
58,91s system 54% cpu 1:58,05 total
>=20
> kernblitz:/home/vadim# time find /warehouse/garbage -type f -print0 | =
xargs -0 chmod 644
> find /warehouse/garbage -type f -print0 4,78s user 55,14s system 37% =
cpu 2:40,27 total
> xargs -0 chmod 644 3,26s user 46,70s system 30% cpu 2:41,26 total
>=20
> kernblitz:/home/vadim# cp -r /usr/ports/[e-k]* /warehouse/garbage
> cp -r /usr/ports/[e-k]* /warehouse/garbage 4,57s user 115,70s system =
29% cpu 6:54,47 total
>=20
> kernblitz:/home/vadim# find /warehouse/garbage -type f | wc -l; find =
/warehouse/garbage -type d | wc -l
> 102014
> find /warehouse/garbage -type f 6,46s user 69,75s system 48% cpu =
2:38,31 total
> wc -l 0,78s user 0,27s system 0% cpu 2:38,29 total
> 19714
> find /warehouse/garbage -type d 6,11s user 72,32s system 48% cpu =
2:43,26 total
> wc -l 0,13s user 0,09s system 0% cpu 2:43,24 total
>=20
> kernblitz:/home/vadim# time chmod -R u+rwX,go+rX,go-w =
/warehouse/garbage
> chmod -R u+rwX,go+rX,go-w /warehouse/garbage 5,59s user 69,97s system =
48% cpu 2:36,53 total
>=20
> kernblitz:/home/vadim# time find /warehouse/garbage -type d -exec =
chmod 755 {} \+
> find /warehouse/garbage -type d -exec chmod 755 {} \+ 6,72s user =
83,05s system 23% cpu 6:15,22 total
>=20
> kernblitz:/home/vadim# time find /warehouse/garbage -type f -exec =
chmod 644 {} \+
> find /warehouse/garbage -type f -exec chmod 644 {} \+ 10,49s user =
138,30s system 20% cpu 12:14,49 total
>=20
> kernblitz:/home/vadim# time find /warehouse/garbage -type f -print0 | =
xargs -0 chmod 644
> find /warehouse/garbage -type f -print0 6,58s user 70,84s system 35% =
cpu 3:36,67 total
> xargs -0 chmod 644 3,93s user 59,30s system 28% cpu 3:39,50 total
>=20
> --=20
> WBR, Vadim Goncharov. ICQ#166852181 =
mailto:vadim_nuclight@mail.ru
> [Moderator of =
RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight]
>=20
> _______________________________________________
> freebsd-performance@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to =
"freebsd-performance-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4E648192-7D2C-4556-B001-FDD812E821F8>