From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 13 13:45:42 2006 Return-Path: X-Original-To: freebsd-hackers@FreeBSD.ORG Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E49ED16A403; Fri, 13 Oct 2006 13:45:42 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [83.120.8.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3ACCB43D5A; Fri, 13 Oct 2006 13:45:41 +0000 (GMT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (azwtcf@localhost [127.0.0.1]) by lurza.secnetix.de (8.13.4/8.13.4) with ESMTP id k9DDjYfc030562; Fri, 13 Oct 2006 15:45:40 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.13.4/8.13.1/Submit) id k9DDjYkD030561; Fri, 13 Oct 2006 15:45:34 +0200 (CEST) (envelope-from olli) Date: Fri, 13 Oct 2006 15:45:34 +0200 (CEST) Message-Id: <200610131345.k9DDjYkD030561@lurza.secnetix.de> From: Oliver Fromme To: freebsd-hackers@FreeBSD.ORG, kientzle@FreeBSD.ORG, dougb@FreeBSD.ORG In-Reply-To: <452E8FDF.4050003@FreeBSD.org> X-Newsgroups: list.freebsd-hackers User-Agent: tin/1.8.2-20060425 ("Shillay") (UNIX) (FreeBSD/4.11-STABLE (i386)) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Fri, 13 Oct 2006 15:45:40 +0200 (CEST) Cc: Subject: Re: "tar -c|gzip" faster than "tar -cz"?!? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd-hackers@FreeBSD.ORG, kientzle@FreeBSD.ORG, dougb@FreeBSD.ORG List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Oct 2006 13:45:43 -0000 Doug Barton wrote: > Oliver Fromme wrote: > > In this case, the "real" time is much larger than the > > "user" time. I guess that's the overhead of 85677 files > > and 23399 directories (according to find(1)). :-) > > Did you perform your tests once only with each method, and one right > after the other? If so, the effect you saw might be due to file system > caching. I performed each test several times in succession. If the first run was much different from the rest, I ignored it, so the caches were filled the same on all runs. But 100,000 files still cause some I/O overhead, even if the data is cached and the I/O requests don't actually hit the physical disks. Anyway, my point is not about caching and I/O. The numbers are pretty normal in that regard. My point is about the difference in CPU ("user") time when using "tar -cz" vs. "tar -c | gzip". Meanwhile I had a quick look at the code: gzip uses some optimized assembler code (for x86 and 680x0), while libz doesn't have such a thing. Maybe that's the reason why gzip is noticeably faster. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. "The last good thing written in C was Franz Schubert's Symphony number 9." -- Erwin Dieterich