Merging upstream version 1.19.
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
parent
5eba8c739c
commit
9abb114ebc
GPG key ID:
FBB4F0E80A80222F
31 changed files with 1468 additions and 963 deletions
|
|
@ -6,8 +6,8 @@
|
|||
@finalout
|
||||
@c %**end of header
|
||||
|
||||
@set UPDATED 12 May 2016
|
||||
@set VERSION 1.18
|
||||
@set UPDATED 10 April 2017
|
||||
@set VERSION 1.19
|
||||
|
||||
@dircategory Data Compression
|
||||
@direntry
|
||||
|
|
@ -50,7 +50,7 @@ This manual is for Lziprecover (version @value{VERSION}, @value{UPDATED}).
|
|||
@end menu
|
||||
|
||||
@sp 1
|
||||
Copyright @copyright{} 2009-2016 Antonio Diaz Diaz.
|
||||
Copyright @copyright{} 2009-2017 Antonio Diaz Diaz.
|
||||
|
||||
This manual is free documentation: you have unlimited permission
|
||||
to copy, distribute and modify it.
|
||||
|
|
@ -61,10 +61,13 @@ to copy, distribute and modify it.
|
|||
@cindex introduction
|
||||
|
||||
Lziprecover is a data recovery tool and decompressor for files in the
|
||||
lzip compressed data format (.lz), able to repair slightly damaged
|
||||
files, produce a correct file by merging the good parts of two or more
|
||||
damaged copies, extract data from damaged files, decompress files and
|
||||
test integrity of files.
|
||||
lzip compressed data format (.lz). Lziprecover is able to repair
|
||||
slightly damaged files, produce a correct file by merging the good parts
|
||||
of two or more damaged copies, extract data from damaged files,
|
||||
decompress files and test integrity of files.
|
||||
|
||||
Lziprecover provides random access to the data in multimember files; it
|
||||
only decompresses the members containing the desired data.
|
||||
|
||||
Lziprecover is not a replacement for regular backups, but a last line of
|
||||
defense for the case where the backups are also damaged.
|
||||
|
|
@ -83,10 +86,10 @@ copies of a file. @xref{Data safety}.
|
|||
|
||||
@item
|
||||
The lzip format is as simple as possible (but not simpler). The lzip
|
||||
manual provides the code of a simple decompressor along with a detailed
|
||||
explanation of how it works, so that with the only help of the lzip
|
||||
manual it would be possible for a digital archaeologist to extract the
|
||||
data from a lzip file long after quantum computers eventually render
|
||||
manual provides the source code of a simple decompressor along with a
|
||||
detailed explanation of how it works, so that with the only help of the
|
||||
lzip manual it would be possible for a digital archaeologist to extract
|
||||
the data from a lzip file long after quantum computers eventually render
|
||||
LZMA obsolete.
|
||||
|
||||
@item
|
||||
|
|
@ -120,12 +123,6 @@ garbage data may be produced at the end of each member):
|
|||
lziprecover -D0 -i -o file -q file.lz
|
||||
@end example
|
||||
|
||||
Lziprecover provides random access to the data in multimember files; it
|
||||
only decompresses the members containing the desired data.
|
||||
|
||||
Lziprecover can print correct total file sizes and ratios even for
|
||||
multimember files.
|
||||
|
||||
When recovering data, lziprecover takes as arguments the names of the
|
||||
damaged files and writes zero or more recovered files depending on the
|
||||
operation selected and whether the recovery succeeded or not. The
|
||||
|
|
@ -134,6 +131,10 @@ damaged files themselves are never modified.
|
|||
When decompressing or testing file integrity, lziprecover behaves like
|
||||
lzip or lunzip.
|
||||
|
||||
LANGUAGE NOTE: Uncompressed = not compressed = plain data; it may never
|
||||
have been compressed. Decompressed is used to refer to data which have
|
||||
undergone the process of decompression.
|
||||
|
||||
|
||||
@node Invoking lziprecover
|
||||
@chapter Invoking lziprecover
|
||||
|
|
@ -235,9 +236,17 @@ Keep (don't delete) input files during decompression.
|
|||
|
||||
@item -l
|
||||
@itemx --list
|
||||
Print total file sizes and ratios. The values produced are correct even
|
||||
for multimember files. Use it together with @samp{-v} to see information
|
||||
about the members in the file.
|
||||
Print the uncompressed size, compressed size and percentage saved of the
|
||||
specified file(s). Trailing data are ignored. The values produced are
|
||||
correct even for multimember files. If more than one file is given, a
|
||||
final line containing the cumulative sizes is printed. With @samp{-v},
|
||||
the dictionary size, the number of members in the file, and the amount
|
||||
of trailing data (if any) are also printed. With @samp{-vv}, the
|
||||
positions and sizes of each member in multimember files are also
|
||||
printed. @samp{-lq} can be used to verify quickly (without
|
||||
decompressing) the structural integrity of the specified files. (Use
|
||||
@samp{--test} to verify the data integrity). @samp{-alq} additionally
|
||||
verifies that none of the specified files contain trailing data.
|
||||
|
||||
@item -m
|
||||
@itemx --merge
|
||||
|
|
@ -259,14 +268,13 @@ file. If converting a lzma-alone file from standard input and
|
|||
name of the converted file. (Or plain @samp{@var{file}} if it already
|
||||
ends in @samp{.lz} or @samp{.tlz}).
|
||||
|
||||
|
||||
@item -q
|
||||
@itemx --quiet
|
||||
Quiet operation. Suppress all messages.
|
||||
|
||||
@item -R
|
||||
@itemx --repair
|
||||
Try to repair a file with small errors (up to one byte error per
|
||||
Try to repair a file with small errors (up to one single-byte error per
|
||||
member). If successful, a repaired copy is written to the file
|
||||
@samp{@var{file}_fixed.lz}. @samp{@var{file}} is not modified at all.
|
||||
The exit status is 0 if the file could be repaired, 2 otherwise. See the
|
||||
|
|
@ -292,8 +300,8 @@ on the number of members in @samp{@var{file}}.
|
|||
Check integrity of the specified file(s), but don't decompress them.
|
||||
This really performs a trial decompression and throws away the result.
|
||||
Use it together with @samp{-v} to see information about the file(s). If
|
||||
a file fails the test, lziprecover continues checking the rest of the
|
||||
files.
|
||||
a file fails the test, does not exist, can't be opened, or is a
|
||||
terminal, lziprecover continues checking the rest of the files.
|
||||
|
||||
@item -v
|
||||
@itemx --verbose
|
||||
|
|
@ -301,7 +309,10 @@ Verbose mode.@*
|
|||
When decompressing or testing, further -v's (up to 4) increase the
|
||||
verbosity level, showing status, compression ratio, dictionary size,
|
||||
trailer contents (CRC, data size, member size), and up to 6 bytes of
|
||||
trailing data (if any).
|
||||
trailing data (if any) both in hexadecimal and as a string of printable
|
||||
ASCII characters.@*
|
||||
In other modes, increasing verbosity levels show final status, progress
|
||||
of operations, and extra information (for example, the failed areas).
|
||||
|
||||
@end table
|
||||
|
||||
|
|
@ -349,7 +360,7 @@ The only remedy for total device failure is storing backup copies in
|
|||
separate media.
|
||||
|
||||
How does lzip compare with gzip and bzip2 with respect to data safety?
|
||||
Lets suppose that you made a backup of your valuable scientific data,
|
||||
Let's suppose that you made a backup of your valuable scientific data,
|
||||
compressed it, and stored two copies on separate media. Years later you
|
||||
notice that both copies are corrupt.
|
||||
|
||||
|
|
@ -393,11 +404,12 @@ Bit-flip happens when one bit in the file is changed from 0 to 1 or vice
|
|||
versa. It may be caused by bad RAM or even by natural radiation. I have
|
||||
seen a case of bit-flip in a file stored on an USB flash drive.
|
||||
|
||||
One byte may seem small, but most file corruptions not produced by I/O
|
||||
errors just affect one byte, or even one bit, of the file. Also, unlike
|
||||
magnetic media, where errors usually affect a whole sector, solid-state
|
||||
storage devices tend to produce single-byte errors, making of lzip the
|
||||
perfect format for data stored on such devices.
|
||||
One byte may seem small, but most file corruptions not produced by
|
||||
transmission errors or I/O errors just affect one byte, or even one bit,
|
||||
of the file. Also, unlike magnetic media, where errors usually affect a
|
||||
whole sector, solid-state storage devices tend to produce single-byte
|
||||
errors, making of lzip the perfect format for data stored on such
|
||||
devices.
|
||||
|
||||
Repairing a file can take some time. Small files or files with the error
|
||||
located near the beginning can be repaired in a few seconds. But
|
||||
|
|
@ -426,11 +438,11 @@ The merge will fail if the damaged areas overlap (at least one byte is
|
|||
damaged in all copies), or are adjacent and the boundary can't be
|
||||
determined, or if the copies have too many damaged areas.
|
||||
|
||||
All the copies must have the same size. If any of them is larger or
|
||||
smaller than it should, either because it has been truncated or because
|
||||
it got some garbage data appended at the end, it can be brought to the
|
||||
correct size with the following command before merging it with the other
|
||||
copies:
|
||||
All the copies to be merged must have the same size. If any of them is
|
||||
larger or smaller than it should, either because it has been truncated
|
||||
or because it got some garbage data appended at the end, it can be
|
||||
brought to the correct size with the following command before merging it
|
||||
with the other copies:
|
||||
|
||||
@example
|
||||
ddrescue -s<correct_size> -x<correct_size> file.lz correct_size_file.lz
|
||||
|
|
@ -444,6 +456,31 @@ few MB) with small errors (one sector damaged per copy), the probability
|
|||
approaches 100 percent even with only two copies. (Supposing that the
|
||||
errors are randomly located inside each copy).
|
||||
|
||||
Some types of solid-state device (NAND flash, for example) can produce
|
||||
bursts of scattered single-bit errors. Lziprecover is able to merge
|
||||
files with thousands of such scattered errors by grouping the errors
|
||||
into clusters and then merging the files as if each cluster were a
|
||||
single error.
|
||||
|
||||
Here is a real case of successful merging. Two copies of the file
|
||||
@samp{icecat-3.5.3-x86.tar.lz} (compressed size 9 MB) became corrupt
|
||||
while stored on the same NAND flash device. One of the copies had 76
|
||||
single-bit errors scattered in an area of 1020 bytes, and the other had
|
||||
3028 such errors in an area of 31729 bytes. Lziprecover produced a
|
||||
correct file, identical to the original, in just 5 seconds:
|
||||
|
||||
@example
|
||||
$ lziprecover -vvm a/icecat-3.5.3-x86.tar.lz b/icecat-3.5.3-x86.tar.lz
|
||||
Merging member 1 of 1 (2552 errors)
|
||||
2552 errors have been grouped in 16 clusters.
|
||||
Trying variation 2 of 2, block 2
|
||||
Input files merged successfully.
|
||||
@end example
|
||||
|
||||
Note that the number of errors reported by lziprecover (2552) is lower
|
||||
than the number of corrupt bytes (3104) because contiguous corrupt bytes
|
||||
are counted as a single multibyte error.
|
||||
|
||||
|
||||
@node File names
|
||||
@chapter Names of the files produced by lziprecover
|
||||
|
|
@ -543,20 +580,24 @@ facilitates safe recovery of undamaged members from multimember files.
|
|||
@chapter Extra data appended to the file
|
||||
@cindex trailing data
|
||||
|
||||
Sometimes extra data is found appended to a lzip file after the last
|
||||
Sometimes extra data are found appended to a lzip file after the last
|
||||
member. Such trailing data may be:
|
||||
|
||||
@itemize @bullet
|
||||
@item
|
||||
Padding added to make the file size a multiple of some block size, for
|
||||
example when writing to a tape.
|
||||
|
||||
@item
|
||||
Garbage added by some not totally successful copy operation.
|
||||
example when writing to a tape. It is safe to append any amount of
|
||||
padding zero bytes to a lzip file.
|
||||
|
||||
@item
|
||||
Useful data added by the user; a cryptographically secure hash, a
|
||||
description of file contents, etc.
|
||||
description of file contents, etc. It is safe to append any amount of
|
||||
text to a lzip file as long as the text does not begin with the string
|
||||
"LZIP", and does not contain any zero bytes (null characters). Nonzero
|
||||
bytes and zero bytes can't be safely mixed in trailing data.
|
||||
|
||||
@item
|
||||
Garbage added by some not totally successful copy operation.
|
||||
|
||||
@item
|
||||
Malicious data added to the file in order to make its total size and
|
||||
|
|
@ -571,8 +612,12 @@ integrity information itself. Therefore it can be considered to be below
|
|||
the noise level.
|
||||
@end itemize
|
||||
|
||||
Trailing data are in no way part of the lzip file format, but tools
|
||||
reading lzip files are expected to behave as correctly and usefully as
|
||||
possible in the presence of trailing data.
|
||||
|
||||
Trailing data can be safely ignored in most cases. In some cases, like
|
||||
that of user-added data, it is expected to be ignored. In those cases
|
||||
that of user-added data, they are expected to be ignored. In those cases
|
||||
where a file containing trailing data must be rejected, the option
|
||||
@samp{--trailing-error} can be used. @xref{--trailing-error}.
|
||||
|
||||
|
|
@ -601,8 +646,8 @@ lziprecover -tv file.lz
|
|||
@sp 1
|
||||
@anchor{concat-example}
|
||||
@noindent
|
||||
Example 3: The right way of concatenating compressed files.
|
||||
@xref{Trailing data}.
|
||||
Example 3: The right way of concatenating the decompressed output of two
|
||||
or more compressed files. @xref{Trailing data}.
|
||||
|
||||
@example
|
||||
Don't do this
|
||||
|
|
@ -753,7 +798,6 @@ See
|
|||
@uref{http://www.nongnu.org/zutils/manual/zutils_manual.html#Zcmp,,zcmp}
|
||||
@end ifhtml
|
||||
|
||||
|
||||
The format for running unzcrash is:
|
||||
|
||||
@example
|
||||
|
|
@ -800,6 +844,16 @@ to 512 bytes. @var{value} defaults to 0.
|
|||
Test only one of every @var{n} bytes, blocks or truncation sizes,
|
||||
instead of all of them.
|
||||
|
||||
@item -e @var{position},@var{value}
|
||||
@itemx --set-byte=@var{position},@var{value}
|
||||
Set byte at @var{position} to @var{value} in the internal buffer after
|
||||
reading and testing @var{filename}.lz but before the first test call to
|
||||
the decompressor. If @var{value} is preceded by @samp{+}, it is added to
|
||||
the original value of the byte at @var{position}. If @var{value} is
|
||||
preceded by @samp{f} (flip), it is XORed with the original value of the
|
||||
byte at @var{position}. This option can be used to run tests with a
|
||||
changed dictionary size, for example.
|
||||
|
||||
@item -p @var{bytes}
|
||||
@itemx --position=@var{bytes}
|
||||
First byte position to test in the file. Defaults to 0. Negative values
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue