From owner-freebsd-questions@FreeBSD.ORG Sat Jul 21 01:24:56 2007 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A4CBC16A41B for ; Sat, 21 Jul 2007 01:24:56 +0000 (UTC) (envelope-from list@museum.rain.com) Received: from ns.umpquanet.com (ns.umpquanet.com [204.119.0.16]) by mx1.freebsd.org (Postfix) with ESMTP id 8583313C469 for ; Sat, 21 Jul 2007 01:24:56 +0000 (UTC) (envelope-from list@museum.rain.com) Received: from ns.umpquanet.com (localhost [127.0.0.1]) by ns.umpquanet.com (8.13.8/8.13.8) with ESMTP id l6L1Ouu6011742; Fri, 20 Jul 2007 18:24:56 -0700 (PDT) (envelope-from list@museum.rain.com) Received: (from james@localhost) by ns.umpquanet.com (8.13.8/8.13.8/Submit) id l6L1OtPs011741; Fri, 20 Jul 2007 18:24:55 -0700 (PDT) (envelope-from list@museum.rain.com) X-Authentication-Warning: ns.umpquanet.com: james set sender to list@museum.rain.com using -f Date: Fri, 20 Jul 2007 18:24:55 -0700 From: James Long To: Chuck Swiger Message-ID: <20070721012455.GA5012@ns.umpquanet.com> References: <20070720220337.GA87174@ns.umpquanet.com> <20070721103710.1e16a319@localhost> <2BF10D44-4FB5-4F07-B515-553BC705B900@mac.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2BF10D44-4FB5-4F07-B515-553BC705B900@mac.com> User-Agent: Mutt/1.5.16 (2007-06-09) Cc: Norberto Meijome , freebsd-questions@freebsd.org Subject: Re: speed of bzip2 versus gzip X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jul 2007 01:24:56 -0000 On Fri, Jul 20, 2007 at 05:50:20PM -0700, Chuck Swiger wrote: > On Jul 20, 2007, at 5:37 PM, Norberto Meijome wrote: >>> Is it normal for bzip2 to be significantly slower than gzip? >>> If not, where can I look for things that might be causing >>> "bzip2 --fast" to take 50-60 times longer to compress a >>> (sendmail log) file than gzip? >> >> i never measured it to see if it is 50-60 times slower, but yes, gzip >> blows >> bzip2 out of the water on speed. I wanted to use bzip2 to compress >> multi-GB >> weblog files, but gzip beat it my miles, and bzip2 wasn't THAT much better >> @ >> compressing it to make it worth it. > > Thanks for the feedback, Norberto. > > Of course, it all depends on what your priorities are, too-- if what you > want is a final tarball which is being mirrored and downloaded frequently, > then your goal is to obtain the absolute best compression, and how much CPU > --best takes isn't important. > > Comparing the default (-5 compression?) of gzip to bzip2 would probably be > more reasonable if you care about reasonably timely compression. If I read the man page correctly, bzip2 defaults to --best, which is why I compared gzip to bzip2 --fast. With the 1.5G sendmail log, bzip2 --fast compresses to just under 10M in about 55 minutes, give or take. bzip2 --best compresses 1.5G to 1.8M, but takes about 2.25 hours. gzip compresses almost as well (with 3% or so) as --fast, but does it in 1 minute instead of 55 on a dual P-III 1.4GHz (but of course, using only one CPU). Jim