From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 14 01:14:03 2006 Return-Path: X-Original-To: freebsd-hackers@FreeBSD.ORG Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3E3E816A415 for ; Sat, 14 Oct 2006 01:14:03 +0000 (UTC) (envelope-from babkin@verizon.net) Received: from vms048pub.verizon.net (vms048pub.verizon.net [206.46.252.48]) by mx1.FreeBSD.org (Postfix) with ESMTP id EC96343D5A for ; Sat, 14 Oct 2006 01:14:02 +0000 (GMT) (envelope-from babkin@verizon.net) Received: from vms062.mailsrvcs.net ([192.168.1.3]) by vms048.mailsrvcs.net (Sun Java System Messaging Server 6.2-4.02 (built Sep 9 2005)) with ESMTPA id <0J73002INQ2R50J0@vms048.mailsrvcs.net> for freebsd-hackers@FreeBSD.ORG; Fri, 13 Oct 2006 20:13:39 -0500 (CDT) Received: from 198.190.8.100 ([198.190.8.100]) by vms062.mailsrvcs.net (Verizon Webmail) with HTTP; Fri, 13 Oct 2006 20:13:39 -0500 (CDT) Date: Fri, 13 Oct 2006 20:13:39 -0500 (CDT) From: Sergey Babkin X-Originating-IP: [198.190.8.100] To: Oliver Fromme , freebsd-hackers@FreeBSD.ORG Message-id: <392921.500871160788419415.JavaMail.root@vms062.mailsrvcs.net> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 7bit X-Mailman-Approved-At: Sat, 14 Oct 2006 01:38:45 +0000 Cc: Subject: Re: Re: "tar -c|gzip" faster than "tar -cz"?!? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: babkin@users.sf.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Oct 2006 01:14:03 -0000 >From: Oliver Fromme > > > The tar|gzip command uses 18% less CPU and is 10% faster. It > > > is clear the HDD is the bottleneck. > > > > Now it's clear to me :) > > > > This makes sense if tar is single-threaded: there's only one thread of > > execution, and it can either be waiting on the disk, or compressing > > data. With two processes, gzip can compress while tar blocks on disk > > IO. > >No. During my tests there was no physical disk I/O (the >disk LED was *OFF* all the time). So tar certainly wasn't >blocking on disk I/O. > >The difference in CPU time (and wall clock time) seems >simply to be caused by different compression code. gzip >is noticeably more efficient than libz, at least on the >OS/processor combination where I tested it (Athlon64 with >FreeBSD/i386 6.2-PRERELEASE). Any chance that gzip uses a different version of libz? Or maybe the buffer size is different? Yet another possibility could be if tar calls zlib with the SYNC (or is that FLUSH? something like that) flag on each chunk, this would kill both the performance and the compression rate. Then again, the default compression level may be different (but it should be making the speed higher if the ratio falls lower). -SB