From owner-freebsd-current@FreeBSD.ORG Mon Aug 20 18:59:01 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7264C16A419 for ; Mon, 20 Aug 2007 18:59:01 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.171]) by mx1.freebsd.org (Postfix) with ESMTP id EE97813C458 for ; Mon, 20 Aug 2007 18:59:00 +0000 (UTC) (envelope-from max@love2party.net) Received: from dslb-088-064-187-186.pools.arcor-ip.net [88.64.187.186] (helo=amd64.laiers.local) by mrelayeu.kundenserver.de (node=mrelayeu5) with ESMTP (Nemesis), id 0ML25U-1INCSZ3sMp-0007lZ; Mon, 20 Aug 2007 20:58:45 +0200 From: Max Laier Organization: FreeBSD To: freebsd-current@freebsd.org Date: Mon, 20 Aug 2007 20:58:23 +0200 User-Agent: KMail/1.9.7 References: <200708170939.l7H9diEk054469@lurza.secnetix.de> <20070819163934.V568@10.0.0.1> <20070820103424.GG1164@turion.vk2pj.dyndns.org> In-Reply-To: <20070820103424.GG1164@turion.vk2pj.dyndns.org> X-Face: ,,8R(x[kmU]tKN@>gtH1yQE4aslGdu+2]; R]*pL,U>^H?)gW@49@wdJ`H<=?utf-8?q?=25=7D*=5FBD=0A=09U=5For=3D=5CmOZf764=26nYj=3DJYbR1PW0ud?=>|!~,,CPC.1-D$FG@0h3#'5"k{V]a~.<=?utf-8?q?mZ=7D44=23Se=7Em=0A=09Fe=7E=5C=5DX5B=5D=5Fxj?=(ykz9QKMw_l0C2AQ]}Ym8)fU MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart2870256.GJF9jbluCR"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200708202058.39565.max@love2party.net> X-Provags-ID: V01U2FsdGVkX19cOq6FX6Su6MfQNIFoWhSQ0d9ypoidADUX6Cq eBWUZ4uqaCmeNv2GYe0FZz33EnQKYv6v6nCxF9rzl6k7vFr5nT AXzogWOPyv5i6jPnzZieCDZ+TJHs1MuIQxifYoIGmc= Cc: Peter Jeremy , Jeff Roberson Subject: Re: Why we don't use bzip2 in sysinstall/rescue? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Aug 2007 18:59:01 -0000 --nextPart2870256.GJF9jbluCR Content-Type: text/plain; charset="iso-8859-6" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Monday 20 August 2007, Peter Jeremy wrote: > On 2007-Aug-19 16:46:10 -0700, Jeff Roberson =20 wrote: > >I tried this on my 1.8ghz pentium M laptop with 5.6MB of jpg data. > > > >I did: > > > >tar cvf foo.tar foo > >cat foo.tar >> /dev/null > >time bzip2/gzip foo.tar > > > >I removed and recreated the tar each time. The cat was to make sure > > it was in cache, although it certainly was from the creation step > > before. > > > >Anyway, the results are: > > > >bzip2 > >2.452u 0.026s 0:07.65 32.2% 92+3227k 5+43io 0pf+0w 1849c/6w > > > >gzip > >0.539u 0.020s 0:01.75 31.4% 109+3268k 2+44io 0pf+0w 493c/3w > > I don't believe this is a reasonable test because: > 1) You are measuring compression time, whilst it's decompression time > that is relevant to installation. > 2) jpeg images should not be compressible and are not representative > of the type of data in a FreeBSD release. > > I've tried what I believe is a more reasonable benchmark on an > Athlon XP-1800, running a recent 7-CURRENT using all the installation > images in 6.2-RELEASE-i386-disk1.iso. > > I concatenated all the 6.2-RELEASE/*/*.?? parts into */*.tgz files as > well as copying ports.tgz (a total of 31 files). I also decompressed > each file and recompressed it into a bzip2 file. The total sizes > were: > */*.tbz: 237717490 > */*.tgz: 281754511 > > Like you, I used "cat */*.t{g,b}z >/dev/null" to cache the files > and use systat to verify that they were cached. > > Timing the gzcat and bzcat runs gives: > gzcat -v */*.tgz > /dev/null 12.01s user 0.88s system 98% cpu 13.115=20 > total=20 > gzcat -v */*.tgz > /dev/null 11.95s user 0.95s system 98% cpu 13.124=20 > total=20 > gzcat -v */*.tgz > /dev/null 11.96s user 0.91s system 98% cpu 13.092=20 > total=20 > bzcat -v */*.tbz > /dev/null 153.29s user 3.43s system 98% cpu 2:39.03 > total=20 > bzcat -v */*.tbz > /dev/null 153.32s user 3.26s system 98% cpu 2:39.14 > total=20 > bzcat -v */*.tbz > /dev/null 153.16s user 3.48s system 98% cpu 2:39.02 > total=20 > > This is nearly 13:1 slower for bzcat, with a size reduction of about > 15%. > > As for the CPU vs I/O tradeoff, I believe that gzcat will be I/O bound > whilst bzcat will be CPU bound in most situations, though I haven't > actually verified this. With an amd64 world(236M) tar'ed together with -czf / -cyf respectively I=20 get the following input bandwidth numbers (gathered via dd=20 if=3Damd64.t{g,b}z of=3D/dev/stdio | {g,b}zcat > /dev/null): hw.model: Intel(R) Pentium(R) 4 CPU 2.00GHz: x bz + gz N Min Max Median Avg Stddev x 13 1411432 1554463 1544446 1521445.7 50394.705 + 13 13391136 13847045 13804007 13726781 138363.27 Difference at 95.0% confidence 1.22053e+07 +/- 84296.2 802.22% +/- 5.54053% (Student's t, pooled s =3D 104125) hw.model=3DAMD Opteron(tm) Processor 275: x fast.bz + fast.gz N Min Max Median Avg Stddev x 10 3429556 3889574 3449869 3525725.6 169675.46 + 10 41967910 46046387 45944662 45490300 1257435.8 Difference at 95.0% confidence 4.19646e+07 +/- 843005 1190.24% +/- 23.9101% (Student's t, pooled s =3D 897200) So it seems that bzip2 will indeed be bound to CPU - at least when=20 installing from CD. netinst over the internet is a different story,=20 though. =2D-=20 /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --nextPart2870256.GJF9jbluCR Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQBGyeRfXyyEoT62BG0RAkwVAJ997GbvWwS9meNewwnpJsyAAmlrKQCfcip8 nw2QMwF3JYszR6oXcr4JvwM= =Pezz -----END PGP SIGNATURE----- --nextPart2870256.GJF9jbluCR--