From owner-freebsd-current@FreeBSD.ORG Mon Aug 20 10:31:58 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4388916A41A for ; Mon, 20 Aug 2007 10:31:58 +0000 (UTC) (envelope-from keramida@ceid.upatras.gr) Received: from igloo.linux.gr (igloo.linux.gr [62.1.205.36]) by mx1.freebsd.org (Postfix) with ESMTP id B049713C468 for ; Mon, 20 Aug 2007 10:31:57 +0000 (UTC) (envelope-from keramida@ceid.upatras.gr) Received: from kobe.laptop (vader.bytemobile.ondsl.gr [83.235.244.135]) (authenticated bits=128) by igloo.linux.gr (8.14.1/8.14.1/Debian-8) with ESMTP id l7KAVCgU017198 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 20 Aug 2007 13:31:21 +0300 Received: from kobe.laptop (kobe.laptop [127.0.0.1]) by kobe.laptop (8.14.1/8.14.1) with ESMTP id l7KAUtmg003025; Mon, 20 Aug 2007 13:31:11 +0300 (EEST) (envelope-from keramida@ceid.upatras.gr) Received: (from keramida@localhost) by kobe.laptop (8.14.1/8.14.1/Submit) id l7KAUrZE003024; Mon, 20 Aug 2007 13:30:53 +0300 (EEST) (envelope-from keramida@ceid.upatras.gr) Date: Mon, 20 Aug 2007 13:30:52 +0300 From: Giorgos Keramidas To: Jeff Roberson Message-ID: <20070820103052.GA2997@kobe.laptop> References: <200708170939.l7H9diEk054469@lurza.secnetix.de> <20070819163934.V568@10.0.0.1> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070819163934.V568@10.0.0.1> X-Hellug-MailScanner: Found to be clean X-Hellug-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-3.926, required 5, autolearn=not spam, ALL_TRUSTED -1.80, AWL 0.47, BAYES_00 -2.60) X-Hellug-MailScanner-From: keramida@ceid.upatras.gr X-Spam-Status: No Cc: freebsd-current@freebsd.org, d@delphij.net Subject: Re: Why we don't use bzip2 in sysinstall/rescue? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Aug 2007 10:31:58 -0000 On 2007-08-19 16:46, Jeff Roberson wrote: >On Fri, 17 Aug 2007, Oliver Fromme wrote: >>LI Xin wrote: >>> As a side note. For networked installation, using bzip2 would >>> reduce traffic by ~11%. >> >> And increase local installation time by 900% (except maybe on >> high-end machines). >> >> I just tested extracting a 10 MB .bz2 file to /dev/null on our 800 >> MHz server: It took 57 seconds. Recompressing the result to .gz, >> extracting that took only 5 seconds. The installation data is >> roughly 30 times that much. > > I tried this on my 1.8ghz pentium M laptop with 5.6MB of jpg data. > > I did: > > tar cvf foo.tar foo > cat foo.tar >> /dev/null > time bzip2/gzip foo.tar > > I removed and recreated the tar each time. The cat was to make sure > it was in cache, although it certainly was from the creation step > before. > > Anyway, the results are: > > bzip2 > 2.452u 0.026s 0:07.65 32.2% 92+3227k 5+43io 0pf+0w 1849c/6w > > gzip > 0.539u 0.020s 0:01.75 31.4% 109+3268k 2+44io 0pf+0w 493c/3w > > So only 4.6x slower here although my processor is twice as fast. > Still, I imagine local installation is actually dominated by transfer > rates from the cd and file creation time on the new volume. Making > lots of little files is relatively slow, and I bet we don't use > softupdates on the target volume during sysinstall. A better test > would be to actually extract a bzip and a gzip from a cd to a local > filesystem and measure the times. My own tests with compressing stuff yielded similar results to what Oliver and Jeff have. bzip2 results in 5-10% more compression, but at the expense of 4x or 5x CPU time to compress everything. A more interesting test would be what Jeff suggests, i.e. to find out what the CPU-time penalty of *decompressing* bzip2 archives would be, and writing them back on disk. - Giorgos