Date: Thu, 12 Oct 2006 00:26:02 -0700 From: Tim Kientzle <kientzle@freebsd.org> To: freebsd-hackers@freebsd.org Subject: Re: "tar -c|gzip" faster than "tar -cz"?!? Message-ID: <452DEE0A.4060500@freebsd.org> In-Reply-To: <200610101727.k9AHRrYo039774@lurza.secnetix.de> References: <200610101727.k9AHRrYo039774@lurza.secnetix.de>
next in thread | previous in thread | raw e-mail | index | archive | help
Oliver Fromme wrote: > > While doing some performance tuning of a backup script > I noticed that the -z option of our (bsd)tar behaves in > a very suboptimal way. It's not only a lot slower than > using gzip separately, it also compresses worse. It seems that you and others have seen very different performance. I'd be very interested in knowing why. I suspect it may have to do with average file size. How big are the files you're archiving? Does the relative performance differ with larger or smaller files? Right now, libarchive calls the libz compression function for each small piece of data. I think that it might be possible to make it faster by combining blocks of data to make fewer calls to the compression routines in libz. (This is why I think the size of the files might matter; small files result in more calls to libz with small blocks of data.) I am very surprised that you see different sizes of output. There are small differences between the compression code in libz and gzip, but I've only ever seen very trivial size differences because of that. Tim Kientzle
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?452DEE0A.4060500>