[BBLISA] Compression Algorithms

[Edward Ned Harvey]

> From: bblisa-bounces at bblisa.org [mailto:bblisa-bounces at bblisa.org] On
> Behalf Of Jeff Wasilko
> 
> On Tue, Jun 29, 2010 at 09:15:43AM -0400, Edward Ned Harvey wrote:
> > .         bzip2 is soundly destroyed by 7-zip, if only 7-zip is made
> > available as an in-line filter.  Presently, in-line filtering is the
> only
> > reason to ever use bzip2 instead of 7-zip.
> 
> You might want to checkout pbzip2. It's a parallelized version of
> bzip2.
> Quite fast on multi-core systems!

I'm aware of pbzip2.  (I'm the author of threadzip, and the threadzip home
page gives you links to both pigz and pbzip2.)

7-zip (lzma) is also parallelizable (in fact, parallel by default.)  This
still soundly obsoletes bzip2 and pbzip2.  In my personal experience, the
reason I wanted to create threadzip, was because I have an 8-core machine,
where pbzip2 was still slow enough that the processor was the bottleneck in
streaming data to tape.  Presently, I use threadzip, which is a parallel
implementation of libz or zlib or gzip (like pigz) because I'm able to
compress the datastream in an inline filter realtime.

Now that I know what I know now ... I'll likely adapt that to use 7-zip
(lzma) because it's so much better.