From owner-freebsd-performance@FreeBSD.ORG Wed Jun 15 10:34:08 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F5211065672 for ; Wed, 15 Jun 2011 10:34:08 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id E5E038FC17 for ; Wed, 15 Jun 2011 10:34:07 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QWnQ9-0005yk-Ij for freebsd-performance@freebsd.org; Wed, 15 Jun 2011 12:34:01 +0200 Received: from nuclight.avtf.net ([82.117.70.99]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 15 Jun 2011 12:34:01 +0200 Received: from vadim_nuclight by nuclight.avtf.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 15 Jun 2011 12:34:01 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: Vadim Goncharov Date: Wed, 15 Jun 2011 10:33:49 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 67 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: nuclight.avtf.net X-Comment-To: All User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: strange differencies: find -exec \+ vs find | xargs X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Jun 2011 10:34:08 -0000 Hi, There was a flamewar in a local maillist about the fastest way to do chmod 644 on files and chmod 755 on directories in a sbubtree. It suddenly revealed the strange thing: on the same subtree, find -type d -exec \+ on 76000 directories has the same as find | xargs, but on 246000 files it was 4 times slower - while there must be almost identical number of forks. I've repeated the tests on deliberately old machine, a Pentium I 233 MHz with 80 Mb RAM, 7.4R UFS2. Subtree for tests was copy of /usr/src with several categories from /usr/ports. The output is from ZSH's builtin 'time' (also set up to automatically output if cmd was run more than 30 secs). Below is the ooutput, first with 80K files, it doesn't differ, than added several more /usr/ports categories and got slow result on 100K. Why is the difference at all? This machine has maxvnodes about 4600 and too small RAM for caches, if that's buffers, I'd expect the difference to begin on mush slow number of files. But it runs the same 80K normally... kernblitz:/home/vadim# find /warehouse/garbage -type f | wc -l; find /warehouse/garbage -type d | wc -l 80208 find /warehouse/garbage -type f 5,01s user 53,12s system 53% cpu 1:48,52 total wc -l 0,54s user 0,31s system 0% cpu 1:48,49 total 13989 find /warehouse/garbage -type d 4,52s user 52,00s system 53% cpu 1:45,63 total wc -l 0,11s user 0,06s system 0% cpu 1:45,61 total kernblitz:/home/vadim# time chmod -R u+rwX,go+rX,go-w /warehouse/garbage chmod -R u+rwX,go+rX,go-w /warehouse/garbage 4,39s user 54,94s system 53% cpu 1:50,73 total kernblitz:/home/vadim# time find /warehouse/garbage -type f -exec chmod 644 {} \+ find /warehouse/garbage -type f -exec chmod 644 {} \+ 7,29s user 100,30s system 65% cpu 2:43,29 total kernblitz:/home/vadim# time find /warehouse/garbage -type d -exec chmod 755 {} \+ find /warehouse/garbage -type d -exec chmod 755 {} \+ 5,04s user 58,91s system 54% cpu 1:58,05 total kernblitz:/home/vadim# time find /warehouse/garbage -type f -print0 | xargs -0 chmod 644 find /warehouse/garbage -type f -print0 4,78s user 55,14s system 37% cpu 2:40,27 total xargs -0 chmod 644 3,26s user 46,70s system 30% cpu 2:41,26 total kernblitz:/home/vadim# cp -r /usr/ports/[e-k]* /warehouse/garbage cp -r /usr/ports/[e-k]* /warehouse/garbage 4,57s user 115,70s system 29% cpu 6:54,47 total kernblitz:/home/vadim# find /warehouse/garbage -type f | wc -l; find /warehouse/garbage -type d | wc -l 102014 find /warehouse/garbage -type f 6,46s user 69,75s system 48% cpu 2:38,31 total wc -l 0,78s user 0,27s system 0% cpu 2:38,29 total 19714 find /warehouse/garbage -type d 6,11s user 72,32s system 48% cpu 2:43,26 total wc -l 0,13s user 0,09s system 0% cpu 2:43,24 total kernblitz:/home/vadim# time chmod -R u+rwX,go+rX,go-w /warehouse/garbage chmod -R u+rwX,go+rX,go-w /warehouse/garbage 5,59s user 69,97s system 48% cpu 2:36,53 total kernblitz:/home/vadim# time find /warehouse/garbage -type d -exec chmod 755 {} \+ find /warehouse/garbage -type d -exec chmod 755 {} \+ 6,72s user 83,05s system 23% cpu 6:15,22 total kernblitz:/home/vadim# time find /warehouse/garbage -type f -exec chmod 644 {} \+ find /warehouse/garbage -type f -exec chmod 644 {} \+ 10,49s user 138,30s system 20% cpu 12:14,49 total kernblitz:/home/vadim# time find /warehouse/garbage -type f -print0 | xargs -0 chmod 644 find /warehouse/garbage -type f -print0 6,58s user 70,84s system 35% cpu 3:36,67 total xargs -0 chmod 644 3,93s user 59,30s system 28% cpu 3:39,50 total -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight]