lzip/lziprecover
1
0
Fork 0
Code Issues Activity

Merging upstream version 1.24.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-21 11:32:42 +01:00
parent cefe4620fe
commit bbed90a132
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
35 changed files with 910 additions and 848 deletions
Download patch file Download diff file Expand all files Collapse all files

129
doc/lziprecover.texi
View file

@ -6,8 +6,8 @@
@finalout
@c %**end of header
@set UPDATED 14 June 2023
@set VERSION 1.24-pre1
@set UPDATED 20 January 2024
@set VERSION 1.24
@dircategory Compression
@direntry
@ -37,7 +37,7 @@ This manual is for Lziprecover (version @value{VERSION}, @value{UPDATED}).
@menu
* Introduction:: Purpose and features of lziprecover
* Invoking lziprecover:: Command line interface
* Invoking lziprecover:: Command-line interface
* Data safety:: Protecting data from accidental loss
* Repairing one byte:: Fixing bit flips and similar errors
* Merging files:: Fixing several damaged copies
@ -53,7 +53,7 @@ This manual is for Lziprecover (version @value{VERSION}, @value{UPDATED}).
@end menu
@sp 1
Copyright @copyright{} 2009-2023 Antonio Diaz Diaz.
Copyright @copyright{} 2009-2024 Antonio Diaz Diaz.
This manual is free documentation: you have unlimited permission to copy,
distribute, and modify it.
@ -254,12 +254,12 @@ unless the option @option{--output} is used.
@itemx --reproduce
Try to recover a missing (zeroed) sector in @var{file} using a reference
file and the same version of lzip that created @var{file}. If successful, a
repaired copy is written to the file @samp{@var{file}_fixed.lz}. @var{file}
is not modified at all. The exit status is 0 if the member containing the
zeroed sector could be repaired, 2 otherwise. Note that
@samp{@var{file}_fixed.lz} may still contain errors in the members following
the one repaired. @xref{Reproducing one sector}, for a complete description
of the reproduce mode.
repaired copy is written to the file @var{file}_fixed.lz. @var{file} is not
modified at all. The exit status is 0 if the member containing the zeroed
sector could be repaired, 2 otherwise. Note that @var{file}_fixed.lz may
still contain errors in the members following the one repaired.
@xref{Reproducing one sector}, for a complete description of the reproduce
mode.
@item --lzip-level=@var{digit}|a|m[@var{length}]
Try only the given compression level or match length limit when reproducing
@ -315,30 +315,30 @@ are ignored, and with @option{-ivv}, gaps between members are shown. The
member numbers shown coincide with the file numbers produced by @option{--split}.
If any file is damaged, does not exist, can't be opened, or is not regular,
the final exit status will be @w{> 0}. @option{-lq} can be used to check
quickly (without decompressing) the structural integrity of the files
specified. (Use @option{--test} to check the data integrity). @option{-alq}
the final exit status is @w{> 0}. @option{-lq} can be used to check quickly
(without decompressing) the structural integrity of the files specified.
(Use @option{--test} to check the data integrity). @option{-alq}
additionally checks that none of the files specified contain trailing data.
@item -m
@itemx --merge
Try to produce a correct file by merging the good parts of two or more
damaged copies. If successful, a repaired copy is written to the file
@samp{@var{file}_fixed.lz}. The exit status is 0 if a correct file could
be produced, 2 otherwise. @xref{Merging files}, for a complete
description of the merge mode.
@var{file}_fixed.lz. The exit status is 0 if a correct file could be
produced, 2 otherwise. @xref{Merging files}, for a complete description of
the merge mode.
@item -o @var{file}
@itemx --output=@var{file}
Place the output into @var{file} instead of into @samp{@var{file}_fixed.lz}.
If splitting, the names of the files produced are in the form
@samp{rec01@var{file}}, @samp{rec02@var{file}}, etc.
Place the repaired output into @var{file} instead of into
@var{file}_fixed.lz. If splitting, the names of the files produced are in
the form @samp{rec01@var{file}}, @samp{rec02@var{file}}, etc.
If decompressing, or converting lzma-alone files, and @option{-c} has not been
also specified, write the decompressed or converted output to @var{file};
keep input files unchanged. This option (or @option{-c}) is needed when
reading from a named pipe (fifo) or from a device. @w{@samp{-o -}} is
equivalent to @option{-c}. @option{-o} has no effect when testing or listing.
If @option{-c} has not been also specified, write the (de)compressed output
to @var{file}, automatically creating any missing parent directories; keep
input files unchanged. This option (or @option{-c}) is needed when reading
from a named pipe (fifo) or from a device. @w{@option{-o -}} is equivalent
to @option{-c}. @option{-o} has no effect when testing or listing.
@item -q
@itemx --quiet
@ -349,9 +349,9 @@ Quiet operation. Suppress all messages.
@itemx --byte-repair
Try to repair a @var{file} with small errors (up to one single-byte error
per member). If successful, a repaired copy is written to the file
@samp{@var{file}_fixed.lz}. @var{file} is not modified at all. The exit
status is 0 if the file could be repaired, 2 otherwise. @xref{Repairing one
byte}, for a complete description of the repair mode.
@var{file}_fixed.lz. @var{file} is not modified at all. The exit status is 0
if the file could be repaired, 2 otherwise. @xref{Repairing one byte}, for a
complete description of the repair mode.
@item -s
@itemx --split
@ -522,17 +522,18 @@ specified, print the frequency of repeated sequences of all possible byte
values. Print cumulative data for all the files, followed by the name of the
first file with the longest sequence.
@anchor{--unzcrash}
@item -U 1|B@var{size}
@itemx --unzcrash=1|B@var{size}
With argument @samp{1}, test 1-bit errors in the LZMA stream of the
compressed input @var{file} like the command
@w{@samp{unzcrash -b1 -p7 -s-20 'lzip -t' @var{file}}} but in memory, and
therefore much faster. @xref{Unzcrash}. This option tests all the members
independently in a multimember file, skipping headers and trailers. If a
decompression succeeds, the decompressed output is compared with the
decompressed output of the original @var{file} using MD5 digests. @var{file}
must not contain errors and must decompress correctly for the comparisons to
work.
therefore much faster (30 to 50 times faster). @xref{Unzcrash}. This option
tests all the members independently in a multimember file, skipping headers
and trailers. If a decompression succeeds, the decompressed output is
compared with the decompressed output of the original @var{file} using MD5
digests. @var{file} must not contain errors and must decompress correctly
for the comparisons to work.
With argument @samp{B}, test zeroed sectors (blocks of bytes) in the LZMA
stream of the compressed input @var{file} like the command
@ -553,8 +554,8 @@ block for zeroed blocks.
@itemx --debug-decompress=@var{position},@var{value}
Load the compressed @var{file} into memory, set the byte at @var{position}
to @var{value}, and decompress the modified compressed data to standard
output. If the damaged member is decompressed fully (just fails with a CRC
mismatch), the members following it are also decompressed.
output. If the damaged member can be decompressed to the end (just fails
with a CRC mismatch), the members following it are also decompressed.
@item -X[@var{position},@var{value}]
@itemx --show-packets[=@var{position},@var{value}]
@ -603,7 +604,7 @@ Table of SI and binary prefixes (unit multipliers):
@sp 1
Exit status: 0 for a normal exit, 1 for environmental problems
(file not found, invalid command line options, I/O errors, etc), 2 to
(file not found, invalid command-line options, I/O errors, etc), 2 to
indicate a corrupt or invalid input file, 3 for an internal consistency
error (e.g., bug) which caused lziprecover to panic.
@ -612,7 +613,7 @@ error (e.g., bug) which caused lziprecover to panic.
@chapter Protecting data from accidental loss
@cindex data safety
It is a fact of life that sometimes data will become corrupt. Software has
It is a fact of life that sometimes data becomes corrupt. Software has
errors. Hardware may misbehave or fail. RAM may be struck by a cosmic ray.
This is why a safe enough integrity checking is needed in compressed
formats, and the reason why a data recovery tool is sometimes needed.
@ -725,10 +726,10 @@ identical backups (@pxref{performance-of-merge}).
@cindex repairing one byte
Lziprecover can repair perfectly most files with small errors (up to one
single-byte error per member), without the need of any extra redundance
at all. If the reparation is successful, the repaired file will be
identical bit for bit to the original. This makes lzip files resistant
to bit flip, one of the most common forms of data corruption.
single-byte error per member), without the need of any extra redundance at
all. If the reparation is successful, the repaired file is identical bit for
bit to the original. This makes lzip files resistant to bit flip, one of the
most common forms of data corruption.
The file is repaired in memory. Therefore, enough virtual memory
@w{(RAM + swap)} to contain the largest damaged member is required.
@ -765,23 +766,22 @@ repairs more efficiently the worst errors.
@cindex merging files
If you have several copies of a file but all of them are too damaged to
repair them (@pxref{Repairing one byte}), lziprecover can try to produce a
correct file by merging the good parts of the damaged copies.
repair them individually (@pxref{Repairing one byte}), lziprecover can try
to produce a correct file by merging the good parts of the damaged copies.
The merge may succeed even if some copies of the file have all the
headers and trailers damaged, as long as there is at least one copy of
every header and trailer intact, even if they are in different copies of
the file.
The merge may succeed even if some copies of the file have all the headers
and trailers damaged, as long as there is at least one copy of every header
and trailer intact, even if they are in different copies of the file.
The merge will fail if the damaged areas overlap (at least one byte is
damaged in all copies), or are adjacent and the boundary can't be
determined, or if the copies have too many damaged areas.
The merge fails if the damaged areas overlap (at least one byte is damaged
in all copies), or are adjacent and the boundary can't be determined, or if
the copies have too many damaged areas.
All the copies to be merged must have the same size. If any of them is
larger or smaller than it should, either because it has been truncated
or because it got some garbage data appended at the end, it can be
brought to the correct size with the following command before merging it
with the other copies:
larger or smaller than it should, either because it has been truncated or
because it got some garbage data appended at the end, it can be brought to
the correct size with the following command before merging it with the other
copies:
@example
ddrescue -s<correct_size> -x<correct_size> file.lz correct_size_file.lz
@ -1080,7 +1080,7 @@ archiver and the
Tarlz creates tar archives using a simplified and safer variant of the POSIX
pax format compressed in lzip format, keeping the alignment between tar
members and lzip members. The resulting multimember tar.lz archive is fully
members and lzip members. The resulting multimember tar.lz archive is
backward compatible with standard tar tools like GNU tar, which treat it
like any other tar.lz archive.
@ifnothtml
@ -1199,7 +1199,7 @@ represents one byte; a box like this:
represents a variable number of bytes.
@sp 1
A lzip file consists of a series of independent "members" (compressed data
A lzip file consists of one or more independent "members" (compressed data
sets). The members simply appear one after another in the file, with no
additional information before, between, or after them. Each member can
encode in compressed form up to @w{16 EiB - 1 byte} of uncompressed data.
@ -1277,7 +1277,7 @@ padding zero bytes to a lzip file.
Useful data added by the user; an "End Of File" string (to check that the
file has not been truncated), a cryptographically secure hash, a description
of file contents, etc. It is safe to append any amount of text to a lzip
file as long as none of the first four bytes of the text match the
file as long as none of the first four bytes of the text matches the
corresponding byte in the string "LZIP", and the text does not contain any
zero bytes (null characters). Nonzero bytes and zero bytes can't be safely
mixed in trailing data.
@ -1299,7 +1299,7 @@ the noise level. Additionally, the test used by lziprecover to discriminate
trailing data from a corrupt header has a Hamming distance (HD) of 3,
and the 3 bit flips must happen in different magic bytes for the test to
fail. In any case, the option @option{--trailing-error} guarantees that
any corrupt header will be detected.
any corrupt header is detected.
@end itemize
Trailing data are in no way part of the lzip file format, but tools
@ -1435,6 +1435,8 @@ lziprecover -tv rec*file.lz
@chapter Testing the robustness of decompressors
@cindex unzcrash
@xref{--unzcrash}, for a faster way of testing the robustness of lzip.
The lziprecover package also includes unzcrash, a program written to test
robustness to decompression of corrupted data, inspired by unzcrash.c from
Julian Seward's bzip2. Type @samp{make unzcrash} in the lziprecover source
@ -1462,7 +1464,7 @@ first non-option argument, and then writes the file specified in the second
non-option argument to the standard input of the subprocess, modifying the
corresponding byte each time. Therefore unzcrash can be used to test any
decompressor (not only lzip), or even other decoder programs having a
suitable command line syntax.
suitable command-line syntax.
If the decompressor returns with zero status, unzcrash compares the output
of the decompressor for the original and corrupt files. If the outputs
@ -1476,9 +1478,8 @@ In order to compare the outputs, unzcrash needs a @samp{zcmp} program able
to understand the format being tested. For example the @samp{zcmp} provided
by @uref{http://www.nongnu.org/zutils/manual/zutils_manual.html#Zcmp,,zutils}.
If the @samp{zcmp} program used does not understand the format being tested,
all the comparisons will fail because the compressed files will be compared
without being decompressed first. Use @option{--zcmp=false} to disable
comparisons.
all the comparisons fail because the compressed files are compared without
being decompressed first. Use @option{--zcmp=false} to disable comparisons.
@ifnothtml
@xref{Zcmp,,,zutils}.
@end ifnothtml
@ -1535,7 +1536,7 @@ with the option @option{--delta}.
Test one byte, block, or truncation size every @var{n} bytes. If
@option{--delta} is not specified, unzcrash tests all the bytes,
non-overlapping blocks, or truncation sizes. Values of @var{n} smaller than
the block size will result in overlapping blocks. (Which is convenient for
the block size result in overlapping blocks. (Which is convenient for
testing because there are usually too few non-overlapping blocks in a file).
@item -e @var{position},@var{value}
@ -1588,7 +1589,7 @@ unzcrash and zcmp to use the same decompressor with a command like
@end table
Exit status: 0 for a normal exit, 1 for environmental problems
(file not found, invalid command line options, I/O errors, etc), 2 to
(file not found, invalid command-line options, I/O errors, etc), 2 to
indicate a corrupt or invalid input file, 3 for an internal consistency
error (e.g., bug) which caused unzcrash to panic.