320 lines
10 KiB
Text
320 lines
10 KiB
Text
\input texinfo @c -*-texinfo-*-
|
|
@c %**start of header
|
|
@setfilename plzip.info
|
|
@settitle Plzip Manual
|
|
@finalout
|
|
@c %**end of header
|
|
|
|
@set UPDATED 10 February 2010
|
|
@set VERSION 0.5
|
|
|
|
@dircategory Data Compression
|
|
@direntry
|
|
* Plzip: (plzip). Parallel compressor compatible with lzip
|
|
@end direntry
|
|
|
|
|
|
@titlepage
|
|
@title Plzip
|
|
@subtitle A data compressor based on the LZMA algorithm
|
|
@subtitle for Plzip version @value{VERSION}, @value{UPDATED}
|
|
@author by Antonio Diaz Diaz
|
|
|
|
@page
|
|
@vskip 0pt plus 1filll
|
|
@end titlepage
|
|
|
|
@contents
|
|
|
|
@node Top
|
|
@top
|
|
|
|
This manual is for Plzip (version @value{VERSION}, @value{UPDATED}).
|
|
|
|
@menu
|
|
* Introduction:: Purpose and features of plzip
|
|
* Invoking Plzip:: Command line interface
|
|
* File Format:: Detailed format of the compressed file
|
|
* Problems:: Reporting bugs
|
|
* Concept Index:: Index of concepts
|
|
@end menu
|
|
|
|
@sp 1
|
|
Copyright @copyright{} 2009, 2010 Antonio Diaz Diaz.
|
|
|
|
This manual is free documentation: you have unlimited permission
|
|
to copy, distribute and modify it.
|
|
|
|
|
|
@node Introduction
|
|
@chapter Introduction
|
|
@cindex introduction
|
|
|
|
Plzip is a massively parallel (multi-threaded), lossless data compressor
|
|
based on the LZMA algorithm, with very safe integrity checking and a
|
|
user interface similar to the one of gzip or bzip2. Plzip uses the lzip
|
|
file format; the files produced by plzip are fully compatible with
|
|
lzip-1.4 or newer.
|
|
|
|
Plzip is intended for faster compression/decompression of big files on
|
|
multiprocessor machines, which makes it specially well suited for
|
|
distribution of big software files and large scale data archiving. On
|
|
files big enough, plzip can use hundreds of processors.
|
|
|
|
Plzip replaces every file given in the command line with a compressed
|
|
version of itself, with the name "original_name.lz". Each compressed
|
|
file has the same modification date, permissions, and, when possible,
|
|
ownership as the corresponding original, so that these properties can be
|
|
correctly restored at decompression time. Plzip is able to read from some
|
|
types of non regular files if the @samp{--stdout} option is specified.
|
|
|
|
If no file names are specified, plzip compresses (or decompresses) from
|
|
standard input to standard output. In this case, plzip will decline to
|
|
write compressed output to a terminal, as this would be entirely
|
|
incomprehensible and therefore pointless.
|
|
|
|
Plzip will correctly decompress a file which is the concatenation of two
|
|
or more compressed files. The result is the concatenation of the
|
|
corresponding uncompressed files. Integrity testing of concatenated
|
|
compressed files is also supported.
|
|
|
|
When decompressing, plzip attempts to guess the name for the decompressed
|
|
file from that of the compressed file as follows:
|
|
|
|
@multitable {anyothername} {becomes} {anyothername.out}
|
|
@item filename.lz @tab becomes @tab filename
|
|
@item filename.tlz @tab becomes @tab filename.tar
|
|
@item anyothername @tab becomes @tab anyothername.out
|
|
@end multitable
|
|
|
|
As a self-check for your protection, plzip stores in the member trailer
|
|
the 32-bit CRC of the original data and the size of the original data,
|
|
to make sure that the decompressed version of the data is identical to
|
|
the original. This guards against corruption of the compressed data, and
|
|
against undetected bugs in plzip (hopefully very unlikely). The chances
|
|
of data corruption going undetected are microscopic, less than one
|
|
chance in 4000 million for each member processed. Be aware, though, that
|
|
the check occurs upon decompression, so it can only tell you that
|
|
something is wrong. It can't help you recover the original uncompressed
|
|
data.
|
|
|
|
Return values: 0 for a normal exit, 1 for environmental problems (file
|
|
not found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or
|
|
invalid input file, 3 for an internal consistency error (eg, bug) which
|
|
caused plzip to panic.
|
|
|
|
|
|
@node Invoking Plzip
|
|
@chapter Invoking Plzip
|
|
@cindex invoking
|
|
@cindex options
|
|
@cindex usage
|
|
@cindex version
|
|
|
|
The format for running plzip is:
|
|
|
|
@example
|
|
plzip [@var{options}] [@var{files}]
|
|
@end example
|
|
|
|
Plzip supports the following options:
|
|
|
|
@table @samp
|
|
@item --help
|
|
@itemx -h
|
|
Print an informative help message describing the options and exit.
|
|
|
|
@item --version
|
|
@itemx -V
|
|
Print the version number of plzip on the standard output and exit.
|
|
|
|
@item --data-size=@var{size}
|
|
@itemx -B
|
|
Set the input data block size in bytes. The input file will be divided
|
|
in chunks of this size before compression is performed. Valid values
|
|
range from 8KiB to 1GiB. Default value is two times the dictionary size.
|
|
It is a waste of memory to choose a data size smaller than the
|
|
dictionary size.
|
|
|
|
@item --stdout
|
|
@itemx -c
|
|
Compress or decompress to standard output. Needed when reading from a
|
|
named pipe (fifo) or from a device.
|
|
|
|
@item --decompress
|
|
@itemx -d
|
|
Decompress.
|
|
|
|
@item --force
|
|
@itemx -f
|
|
Force overwrite of output file.
|
|
|
|
@item --keep
|
|
@itemx -k
|
|
Keep (don't delete) input files during compression or decompression.
|
|
|
|
@item --match-length=@var{length}
|
|
@itemx -m @var{length}
|
|
Set the match length limit in bytes. Valid values range from 5 to 273.
|
|
Larger values usually give better compression ratios but longer
|
|
compression times.
|
|
|
|
@item --output=@var{file}
|
|
@itemx -o @var{file}
|
|
When reading from standard input and @samp{--stdout} has not been
|
|
specified, use @samp{@var{file}} as the virtual name of the uncompressed
|
|
file. This produces a file named @samp{@var{file}} when decompressing,
|
|
and a file named @samp{@var{file}.lz} when compressing.
|
|
|
|
@item --quiet
|
|
@itemx -q
|
|
Quiet operation. Suppress all messages.
|
|
|
|
@item --dictionary-size=@var{size}
|
|
@itemx -s @var{size}
|
|
Set the dictionary size limit in bytes. Valid values range from 4KiB to
|
|
512MiB. Note that dictionary sizes are quantized. If the specified size
|
|
does not match one of the valid sizes, it will be rounded upwards.
|
|
|
|
@item --test
|
|
@itemx -t
|
|
Check integrity of the specified file(s), but don't decompress them.
|
|
This really performs a trial decompression and throws away the result.
|
|
Use @samp{-tvv} or @samp{-tvvv} to see information about the file.
|
|
|
|
@item --verbose
|
|
@itemx -v
|
|
Verbose mode. Show the compression ratio for each file processed.
|
|
Further -v's increase the verbosity level.
|
|
|
|
@item -1 .. -9
|
|
Set the compression parameters (dictionary size and match length limit)
|
|
as shown in the table below. Note that @samp{-9} can be much slower than
|
|
@samp{-1}. These options have no effect when decompressing.
|
|
|
|
@multitable {Level} {Dictionary size} {Match length limit}
|
|
@item Level @tab Dictionary size @tab Match length limit
|
|
@item -1 @tab 1MiB @tab 10 bytes
|
|
@item -2 @tab 1MiB @tab 12 bytes
|
|
@item -3 @tab 1MiB @tab 17 bytes
|
|
@item -4 @tab 2MiB @tab 26 bytes
|
|
@item -5 @tab 4MiB @tab 44 bytes
|
|
@item -6 @tab 8MiB @tab 80 bytes
|
|
@item -7 @tab 16MiB @tab 108 bytes
|
|
@item -8 @tab 16MiB @tab 163 bytes
|
|
@item -9 @tab 32MiB @tab 273 bytes
|
|
@end multitable
|
|
|
|
@item --fast
|
|
@itemx --best
|
|
Aliases for GNU gzip compatibility.
|
|
|
|
@end table
|
|
|
|
@sp 1
|
|
Numbers given as arguments to options may be followed by a multiplier
|
|
and an optional @samp{B} for "byte".
|
|
|
|
Table of SI and binary prefixes (unit multipliers):
|
|
|
|
@multitable {Prefix} {kilobyte (10^3 = 1000)} {|} {Prefix} {kibibyte (2^10 = 1024)}
|
|
@item Prefix @tab Value @tab | @tab Prefix @tab Value
|
|
@item k @tab kilobyte (10^3 = 1000) @tab | @tab Ki @tab kibibyte (2^10 = 1024)
|
|
@item M @tab megabyte (10^6) @tab | @tab Mi @tab mebibyte (2^20)
|
|
@item G @tab gigabyte (10^9) @tab | @tab Gi @tab gibibyte (2^30)
|
|
@item T @tab terabyte (10^12) @tab | @tab Ti @tab tebibyte (2^40)
|
|
@item P @tab petabyte (10^15) @tab | @tab Pi @tab pebibyte (2^50)
|
|
@item E @tab exabyte (10^18) @tab | @tab Ei @tab exbibyte (2^60)
|
|
@item Z @tab zettabyte (10^21) @tab | @tab Zi @tab zebibyte (2^70)
|
|
@item Y @tab yottabyte (10^24) @tab | @tab Yi @tab yobibyte (2^80)
|
|
@end multitable
|
|
|
|
|
|
@node File Format
|
|
@chapter File Format
|
|
@cindex file format
|
|
|
|
In the diagram below, a box like this:
|
|
@verbatim
|
|
+---+
|
|
| | <-- the vertical bars might be missing
|
|
+---+
|
|
@end verbatim
|
|
|
|
represents one byte; a box like this:
|
|
@verbatim
|
|
+==============+
|
|
| |
|
|
+==============+
|
|
@end verbatim
|
|
|
|
represents a variable number of bytes.
|
|
|
|
@sp 1
|
|
A lzip file consists of a series of "members" (compressed data sets).
|
|
The members simply appear one after another in the file, with no
|
|
additional information before, between, or after them.
|
|
|
|
Each member has the following structure:
|
|
@verbatim
|
|
+--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
|
| ID string | VN | DS | Lzma stream | CRC32 | Data size | Member size |
|
|
+--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
|
@end verbatim
|
|
|
|
All multibyte values are stored in little endian order.
|
|
|
|
@table @samp
|
|
@item ID string
|
|
A four byte string, identifying the member type, with the value "LZIP".
|
|
|
|
@item VN (version number, 1 byte)
|
|
Just in case something needs to be modified in the future. Valid values
|
|
are 0 and 1. Version 0 files have only one member and lack @samp{Member
|
|
size}.
|
|
|
|
@item DS (coded dictionary size, 1 byte)
|
|
Bits 4-0 contain the base 2 logarithm of the base dictionary size.@*
|
|
Bits 7-5 contain the number of "wedges" to substract from the base
|
|
dictionary size to obtain the dictionary size. The size of a wedge is
|
|
(base dictionary size / 16).@*
|
|
Valid values for dictionary size range from 4KiB to 512MiB.
|
|
|
|
@item Lzma stream
|
|
The lzma stream, finished by an end of stream marker. Uses default values
|
|
for encoder properties.
|
|
|
|
@item CRC32 (4 bytes)
|
|
CRC of the uncompressed original data.
|
|
|
|
@item Data size (8 bytes)
|
|
Size of the uncompressed original data.
|
|
|
|
@item Member size (8 bytes)
|
|
Total size of the member, including header and trailer. This facilitates
|
|
safe recovery of undamaged members from multimember files.
|
|
|
|
@end table
|
|
|
|
|
|
@node Problems
|
|
@chapter Reporting Bugs
|
|
@cindex bugs
|
|
@cindex getting help
|
|
|
|
There are probably bugs in plzip. There are certainly errors and
|
|
omissions in this manual. If you report them, they will get fixed. If
|
|
you don't, no one will ever know about them and they will remain unfixed
|
|
for all eternity, if not longer.
|
|
|
|
If you find a bug in plzip, please send electronic mail to
|
|
@email{lzip-bug@@nongnu.org}. Include the version number, which you can
|
|
find by running @w{@samp{plzip --version}}.
|
|
|
|
|
|
@node Concept Index
|
|
@unnumbered Concept Index
|
|
|
|
@printindex cp
|
|
|
|
@bye
|