Date: Fri, 13 Oct 2006 10:19:37 -0700 (PDT) From: Matthew Dillon <dillon@apollo.backplane.com> To: "Steven Hartland" <killing@multiplay.co.uk> Cc: freebsd-hackers@freebsd.org, Joerg Sonnenberger <joerg@britannica.bec.de> Subject: Re: "tar -c|gzip" faster than "tar -cz"?!? Message-ID: <200610131719.k9DHJak1025551@apollo.backplane.com> References: <17707.64434.913943.549852@bhuda.mired.org> <200610110934.k9B9YASW081294@lurza.secnetix.de> <20061011133250.GA483@britannica.bec.de> <01a301c6ede8$061e9160$b3db87d4@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
:
:Just a silly one but are you guys using the same
:version of gzip, would be worth just checking?
It could also simply be a piplining issue. If the pipe inbetween the
'tar' and the 'gzip' is too small (whether gzip is internal to tar
or not), then the 'tar' portion could wind up getting blocked by the
'gzip' portion and not do disk I/O in parallel with the cpu that the gzip
portion uses.
Here I am presuming that there is in fact a fork internal to tar when
using the built-in gzip. There had better be, or performance would be
horrible!
In anycase, the pipe buffer needs to be at least 2x the block size gzip
uses internally when compressing. I would even recommend making it
very large, like several hundred kilobytes (at least). It is the
same problem one faces when, say, streaming data to a slow device such
as a tape drive. You want a large pipe buffer to avoid unsightly stalls
of the code scanning the filesystem.
-Matt
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200610131719.k9DHJak1025551>