Date: Tue, 9 Oct 2012 21:54:03 -0700 From: Tim Kientzle <tim@kientzle.com> To: Wojciech Puchar <wojtek@wojtek.tensor.gdynia.pl> Cc: freebsd-hackers@freebsd.org, Brandon Falk <bfalk_bsd@brandonfa.lk> Subject: Re: SMP Version of tar Message-ID: <15DBA1A9-A4B6-4F7D-A9DC-3412C4BE3517@kientzle.com> In-Reply-To: <alpine.BSF.2.00.1210081219300.4673@wojtek.tensor.gdynia.pl> References: <5069C9FC.6020400@brandonfa.lk> <alpine.BSF.2.00.1210071859430.15957@wojtek.tensor.gdynia.pl> <324B736D-8961-4E44-A212-2ECF3E60F2A0@kientzle.com> <alpine.BSF.2.00.1210080838170.3664@wojtek.tensor.gdynia.pl> <20121008083814.GA5830@straylight.m.ringlet.net> <alpine.BSF.2.00.1210081219300.4673@wojtek.tensor.gdynia.pl>
next in thread | previous in thread | raw e-mail | index | archive | help
On Oct 8, 2012, at 3:21 AM, Wojciech Puchar wrote: >> Not necessarily. If I understand correctly what Tim means, he's = talking >> about an in-memory compression of several blocks by several separate >> threads, and then - after all the threads have compressed their >=20 > but gzip format is single stream. dictionary IMHO is not reset every X = kilobytes. >=20 > parallel gzip is possible but not with same data format. Yes, it is. The following creates a compressed file that is completely compatible with the standard gzip/gunzip tools: * Break file into blocks * Compress each block into a gzip file (with gzip header and trailer = information) * Concatenate the result. This can be correctly decoded by gunzip. In theory, you get slightly worse compression. In practice, if your = blocks are reasonably large (a megabyte or so each), the difference is = negligible. Tim
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?15DBA1A9-A4B6-4F7D-A9DC-3412C4BE3517>