From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 13 00:55:07 2006 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 74FB716A407; Fri, 13 Oct 2006 00:55:07 +0000 (UTC) (envelope-from pieter@degoeje.nl) Received: from smtp.utwente.nl (smtp1.utsp.utwente.nl [130.89.2.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id B46D443D58; Fri, 13 Oct 2006 00:55:06 +0000 (GMT) (envelope-from pieter@degoeje.nl) Received: from nox.student.utwente.nl (nox.student.utwente.nl [130.89.165.91]) by smtp.utwente.nl (8.12.10/SuSE Linux 0.7) with ESMTP id k9D0t1ZS008252; Fri, 13 Oct 2006 02:55:01 +0200 From: Pieter de Goeje To: Oliver Fromme Date: Fri, 13 Oct 2006 02:55:01 +0200 User-Agent: KMail/1.9.4 References: <200610121524.k9CFOOmS069191@lurza.secnetix.de> In-Reply-To: <200610121524.k9CFOOmS069191@lurza.secnetix.de> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200610130255.01566.pieter@degoeje.nl> X-UTwente-MailScanner-Information: Scanned by MailScanner. Contact helpdesk@ITBE.utwente.nl for more information. X-UTwente-MailScanner: Found to be clean X-UTwente-MailScanner-From: pieter@degoeje.nl X-Spam-Status: No Cc: freebsd-hackers@freebsd.org, kientzle@freebsd.org Subject: Re: "tar -c|gzip" faster than "tar -cz"?!? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Oct 2006 00:55:07 -0000 On Thursday 12 October 2006 17:24, Oliver Fromme wrote: > Interestingly, in both tests the compressed size of the > "gzip" case was slightly larger than the "tar cz" case. > That's the opposite of what I got in my very first test > (when archiving the root file system). I also found that the difference in size is insignificant. > I'm not concerned about the difference in compression > sizes, because it's in the sub-percent range. But I'm > more concerned about the CPU times ("user" times). > It makes quite a clear difference in all of my tests. I can confirm, however in my opinion the difference isn't really significant. Maybe a different compiler or gcc with better optimization settings could produce executables that are equally fast. First test on an AMD Athlon64 with 1GB memory running -STABLE and ~3GB /usr/ports dir. Warmup caches with tar -cf /dev/null /usr/ports. tar -czf /dev/null /usr/ports 622.64 real 244.37 user 9.36 sys tar -cf - /usr/ports | gzip > /dev/null 565.15 real 195.65 user 12.38 sys The tar|gzip command uses 18% less CPU and is 10% faster. It is clear the HDD is the bottleneck. Second test on an AMD Sempron with 512MB memory and 240MB directory also running -STABLE. Warmup caches with tar -cf /dev/null /usr/ports/distfiles/gnome2 tar -czf - /usr/ports/distfiles/gnome2 > /dev/null 33.71 real 31.56 user 1.04 sys 32.98 real 31.00 user 0.96 sys tar -cf - /usr/ports/distfiles/gnome2 | gzip > /dev/null 29.09 real 26.65 user 1.52 sys 29.18 real 26.62 user 1.62 sys The tar|gzip command uses 15% less CPU and is 12% faster. Very little disk I/O occured during this test. -- Pieter