From owner-freebsd-performance@FreeBSD.ORG Wed Jun 15 11:36:12 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3817106566C for ; Wed, 15 Jun 2011 11:36:12 +0000 (UTC) (envelope-from db@bsdsystems.de) Received: from fop.bsdsystems.de (mx.bsdsystems.de [88.198.57.43]) by mx1.freebsd.org (Postfix) with ESMTP id 48E598FC0C for ; Wed, 15 Jun 2011 11:36:12 +0000 (UTC) Received: from [IPv6:::1] (fop.bsdsystems.de [88.198.57.43]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by fop.bsdsystems.de (Postfix) with ESMTP id 5DC654FC38; Wed, 15 Jun 2011 13:13:22 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: dennis berger In-Reply-To: Date: Wed, 15 Jun 2011 13:14:32 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <4E648192-7D2C-4556-B001-FDD812E821F8@bsdsystems.de> References: To: vadim_nuclight@mail.ru X-Mailer: Apple Mail (2.1084) X-Mailman-Approved-At: Wed, 15 Jun 2011 13:02:19 +0000 Cc: freebsd-performance@freebsd.org Subject: Re: strange differencies: find -exec \+ vs find | xargs X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Jun 2011 11:36:12 -0000 Dear, I suspect that you're hitting a cache limit for that machine. I had similar behaviour when rsyncing huge directories. I usually raise = vfs.ufs.dirhash_maxmem sysctl variable, which solves the problem. best, -dennis Am 15.06.2011 um 12:33 schrieb Vadim Goncharov: > Hi, >=20 > There was a flamewar in a local maillist about the fastest way to do = chmod 644 > on files and chmod 755 on directories in a sbubtree. It suddenly = revealed the > strange thing: on the same subtree, find -type d -exec \+ on 76000 = directories > has the same as find | xargs, but on 246000 files it was 4 times = slower - while > there must be almost identical number of forks. >=20 > I've repeated the tests on deliberately old machine, a Pentium I 233 = MHz with > 80 Mb RAM, 7.4R UFS2. Subtree for tests was copy of /usr/src with = several > categories from /usr/ports. The output is from ZSH's builtin 'time' = (also > set up to automatically output if cmd was run more than 30 secs). >=20 > Below is the ooutput, first with 80K files, it doesn't differ, than = added > several more /usr/ports categories and got slow result on 100K. Why is > the difference at all? This machine has maxvnodes about 4600 and too = small > RAM for caches, if that's buffers, I'd expect the difference to begin = on > mush slow number of files. But it runs the same 80K normally... >=20 > kernblitz:/home/vadim# find /warehouse/garbage -type f | wc -l; find = /warehouse/garbage -type d | wc -l > 80208 > find /warehouse/garbage -type f 5,01s user 53,12s system 53% cpu = 1:48,52 total > wc -l 0,54s user 0,31s system 0% cpu 1:48,49 total > 13989 > find /warehouse/garbage -type d 4,52s user 52,00s system 53% cpu = 1:45,63 total > wc -l 0,11s user 0,06s system 0% cpu 1:45,61 total >=20 > kernblitz:/home/vadim# time chmod -R u+rwX,go+rX,go-w = /warehouse/garbage > chmod -R u+rwX,go+rX,go-w /warehouse/garbage 4,39s user 54,94s system = 53% cpu 1:50,73 total >=20 > kernblitz:/home/vadim# time find /warehouse/garbage -type f -exec = chmod 644 {} \+ > find /warehouse/garbage -type f -exec chmod 644 {} \+ 7,29s user = 100,30s system 65% cpu 2:43,29 total >=20 > kernblitz:/home/vadim# time find /warehouse/garbage -type d -exec = chmod 755 {} \+ > find /warehouse/garbage -type d -exec chmod 755 {} \+ 5,04s user = 58,91s system 54% cpu 1:58,05 total >=20 > kernblitz:/home/vadim# time find /warehouse/garbage -type f -print0 | = xargs -0 chmod 644 > find /warehouse/garbage -type f -print0 4,78s user 55,14s system 37% = cpu 2:40,27 total > xargs -0 chmod 644 3,26s user 46,70s system 30% cpu 2:41,26 total >=20 > kernblitz:/home/vadim# cp -r /usr/ports/[e-k]* /warehouse/garbage > cp -r /usr/ports/[e-k]* /warehouse/garbage 4,57s user 115,70s system = 29% cpu 6:54,47 total >=20 > kernblitz:/home/vadim# find /warehouse/garbage -type f | wc -l; find = /warehouse/garbage -type d | wc -l > 102014 > find /warehouse/garbage -type f 6,46s user 69,75s system 48% cpu = 2:38,31 total > wc -l 0,78s user 0,27s system 0% cpu 2:38,29 total > 19714 > find /warehouse/garbage -type d 6,11s user 72,32s system 48% cpu = 2:43,26 total > wc -l 0,13s user 0,09s system 0% cpu 2:43,24 total >=20 > kernblitz:/home/vadim# time chmod -R u+rwX,go+rX,go-w = /warehouse/garbage > chmod -R u+rwX,go+rX,go-w /warehouse/garbage 5,59s user 69,97s system = 48% cpu 2:36,53 total >=20 > kernblitz:/home/vadim# time find /warehouse/garbage -type d -exec = chmod 755 {} \+ > find /warehouse/garbage -type d -exec chmod 755 {} \+ 6,72s user = 83,05s system 23% cpu 6:15,22 total >=20 > kernblitz:/home/vadim# time find /warehouse/garbage -type f -exec = chmod 644 {} \+ > find /warehouse/garbage -type f -exec chmod 644 {} \+ 10,49s user = 138,30s system 20% cpu 12:14,49 total >=20 > kernblitz:/home/vadim# time find /warehouse/garbage -type f -print0 | = xargs -0 chmod 644 > find /warehouse/garbage -type f -print0 6,58s user 70,84s system 35% = cpu 3:36,67 total > xargs -0 chmod 644 3,93s user 59,30s system 28% cpu 3:39,50 total >=20 > --=20 > WBR, Vadim Goncharov. ICQ#166852181 = mailto:vadim_nuclight@mail.ru > [Moderator of = RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight] >=20 > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to = "freebsd-performance-unsubscribe@freebsd.org"