From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 10 17:28:02 2006 Return-Path: X-Original-To: freebsd-hackers@FreeBSD.ORG Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DE3B916A40F for ; Tue, 10 Oct 2006 17:28:02 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [83.120.8.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id D2E5A43D60 for ; Tue, 10 Oct 2006 17:28:00 +0000 (GMT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (zuvqlw@localhost [127.0.0.1]) by lurza.secnetix.de (8.13.4/8.13.4) with ESMTP id k9AHRrfM039775 for ; Tue, 10 Oct 2006 19:27:59 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.13.4/8.13.1/Submit) id k9AHRrYo039774; Tue, 10 Oct 2006 19:27:53 +0200 (CEST) (envelope-from olli) Date: Tue, 10 Oct 2006 19:27:53 +0200 (CEST) Message-Id: <200610101727.k9AHRrYo039774@lurza.secnetix.de> From: Oliver Fromme To: freebsd-hackers@FreeBSD.ORG X-Newsgroups: list.freebsd-hackers User-Agent: tin/1.8.2-20060425 ("Shillay") (UNIX) (FreeBSD/4.11-STABLE (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Tue, 10 Oct 2006 19:27:59 +0200 (CEST) X-Mailman-Approved-At: Tue, 10 Oct 2006 18:56:13 +0000 Cc: Subject: "tar -c|gzip" faster than "tar -cz"?!? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd-hackers@FreeBSD.ORG List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Oct 2006 17:28:02 -0000 Hi, While doing some performance tuning of a backup script I noticed that the -z option of our (bsd)tar behaves in a very suboptimal way. It's not only a lot slower than using gzip separately, it also compresses worse. I compared the following two commands (cwd=/): A. tar -cz --one-file-system -f- . | wc -c B. tar -c --one-file-system -f- . | gzip | wc -c In order to measure the time of the whole command pipes, I encapsulated them into subshell calls like this: /usr/bin/time sh -c 'tar ... | wc -c' These are results for multiple invocations of A (tar -cz): 7.30 real 7.15 user 0.09 sys 7.28 real 7.13 user 0.12 sys 7.29 real 7.14 user 0.09 sys And these are the numbers for B (tar -c | gzip): 5.54 real 5.37 user 0.15 sys 5.54 real 5.34 user 0.18 sys 5.55 real 5.40 user 0.12 sys My first thought was that "tar -z" would use a better compression level (e.g. -9) vs. the gzip default of -6, which would explain why it is slower. Therefore I expected the resulting backup to be smaller -- but just the opposite is the case. Command A resulted in a compressed size of 25364480 bytes, while B was a bit smaller (25306059 bytes). I'm surprised because I expected "tar -z" to be faster than a separate gzip process (at the same compression level), or at least as fast. But it's 30% slower. Is that a known problem? Is someone working on it? (BTW, I'm using 6.2-PRERELEASE about 1 week old.) Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. "Emacs ist für mich kein Editor. Für mich ist das genau das gleiche, als wenn ich nach einem Fahrrad (für die Sonntagbrötchen) frage und einen pangalaktischen Raumkreuzer mit 10 km Gesamtlänge bekomme. Ich weiß nicht, was ich damit soll." -- Frank Klemm, de.comp.os.unix.discussion