lzip/plzip
1
0
Fork 0
Code Issues Activity

Merging upstream version 1.4.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-24 04:10:09 +01:00
parent e627412516
commit 8b0a7cb0e8
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
16 changed files with 182 additions and 181 deletions
Download patch file Download diff file Expand all files Collapse all files

93
doc/plzip.texi
View file

@ -6,8 +6,8 @@
@finalout
@c %**end of header
@set UPDATED 23 May 2015
@set VERSION 1.4-rc1
@set UPDATED 9 July 2015
@set VERSION 1.4
@dircategory Data Compression
@direntry
@ -36,8 +36,8 @@ This manual is for Plzip (version @value{VERSION}, @value{UPDATED}).
@menu
* Introduction:: Purpose and features of plzip
* Program design:: Internal structure of plzip
* Invoking plzip:: Command line interface
* Program design:: Internal structure of plzip
* File format:: Detailed format of the compressed file
* Memory requirements:: Memory required to compress and decompress
* Minimum file sizes:: Minimum file sizes required for full speed
@ -77,10 +77,14 @@ availability:
@itemize @bullet
@item
The lzip format provides very safe integrity checking and some data
recovery means. The lziprecover program can repair bit-flip errors (one
of the most common forms of data corruption) in lzip files, and provides
data recovery capabilities, including error-checked merging of damaged
copies of a file.
recovery means. The
@uref{http://www.nongnu.org/lzip/manual/lziprecover_manual.html#Data-safety,,lziprecover}
program can repair bit-flip errors (one of the most common forms of data
corruption) in lzip files, and provides data recovery capabilities,
including error-checked merging of damaged copies of a file.
@ifnothtml
@ref{Data safety,,,lziprecover}.
@end ifnothtml
@item
The lzip format is as simple as possible (but not simpler). The lzip
@ -148,38 +152,6 @@ you verify the compressed file with a command like
@w{@samp{plzip -cd file.lz | cmp file -}}.
@node Program design
@chapter Program design
@cindex program design
When compressing, plzip divides the input file into chunks and
compresses as many chunks simultaneously as worker threads are chosen,
creating a multi-member compressed file.
When decompressing, plzip decompresses as many members simultaneously as
worker threads are chosen. Files that were compressed with lzip will not
be decompressed faster than using lzip (unless the @samp{-b} option was
used) because lzip usually produces single-member files, which can't be
decompressed in parallel.
For each input file, a splitter thread and several worker threads are
created, acting the main thread as muxer (multiplexer) thread. A "packet
courier" takes care of data transfers among threads and limits the
maximum number of data blocks (packets) being processed simultaneously.
The splitter reads data blocks from the input file, and distributes them
to the workers. The workers (de)compress the blocks received from the
splitter. The muxer collects processed packets from the workers, and
writes them to the output file.
When decompressing from a regular file, the splitter is removed and the
workers read directly from the input file. If the output file is also a
regular file, the muxer is also removed and the workers write directly
to the output file. With these optimizations, the use of RAM is greatly
reduced and the decompression speed of large files with many members is
only limited by the number of processors available and by I/O speed.
@node Invoking plzip
@chapter Invoking plzip
@cindex invoking
@ -229,7 +201,7 @@ Force overwrite of output files.
@item -F
@itemx --recompress
Force recompression of files whose name already has the @samp{.lz} or
Force re-compression of files whose name already has the @samp{.lz} or
@samp{.tlz} suffix.
@item -k
@ -346,6 +318,38 @@ invalid input file, 3 for an internal consistency error (eg, bug) which
caused plzip to panic.
@node Program design
@chapter Program design
@cindex program design
When compressing, plzip divides the input file into chunks and
compresses as many chunks simultaneously as worker threads are chosen,
creating a multi-member compressed file.
When decompressing, plzip decompresses as many members simultaneously as
worker threads are chosen. Files that were compressed with lzip will not
be decompressed faster than using lzip (unless the @samp{-b} option was
used) because lzip usually produces single-member files, which can't be
decompressed in parallel.
For each input file, a splitter thread and several worker threads are
created, acting the main thread as muxer (multiplexer) thread. A "packet
courier" takes care of data transfers among threads and limits the
maximum number of data blocks (packets) being processed simultaneously.
The splitter reads data blocks from the input file, and distributes them
to the workers. The workers (de)compress the blocks received from the
splitter. The muxer collects processed packets from the workers, and
writes them to the output file.
When decompressing from a regular file, the splitter is removed and the
workers read directly from the input file. If the output file is also a
regular file, the muxer is also removed and the workers write directly
to the output file. With these optimizations, the use of RAM is greatly
reduced and the decompression speed of large files with many members is
only limited by the number of processors available and by I/O speed.
@node File format
@chapter File format
@cindex file format
@ -439,7 +443,7 @@ following:
@itemize @bullet
@item
For compression at level -0; 1.5 MiB plus 3 times the data size
(@pxref{--data-size}). About 4.5 MiB.
(@pxref{--data-size}). Default is 4.5 MiB.
@item
For compression at other levels; 11 times the dictionary size plus 3
@ -447,9 +451,10 @@ times the data size. Default is 136 MiB.
@item
For decompression of a regular (seekable) file to another regular file,
or for testing of a regular file; the dictionary size. Note that regular
files with more than 1024 bytes of trailing garbage are treated as
non-seekable.
or for testing of a regular file; the dictionary size.
(Note that regular files with more than 1024 bytes of trailing garbage
are treated as non-seekable).
@item
For testing of a non-seekable file or of standard input; the dictionary