lzip/lziprecover
1
0
Fork 0
Code Issues Activity

Merging upstream version 1.19.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-21 11:27:58 +01:00
parent 5eba8c739c
commit 9abb114ebc
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
31 changed files with 1468 additions and 963 deletions
Download patch file Download diff file Expand all files Collapse all files

180
doc/lziprecover.info
View file

@ -12,7 +12,7 @@ File: lziprecover.info, Node: Top, Next: Introduction, Up: (dir)
Lziprecover Manual
******************
This manual is for Lziprecover (version 1.18, 12 May 2016).
This manual is for Lziprecover (version 1.19, 10 April 2017).
* Menu:
@ -30,7 +30,7 @@ This manual is for Lziprecover (version 1.18, 12 May 2016).
* Concept index:: Index of concepts
Copyright (C) 2009-2016 Antonio Diaz Diaz.
Copyright (C) 2009-2017 Antonio Diaz Diaz.
This manual is free documentation: you have unlimited permission to
copy, distribute and modify it.
@ -42,10 +42,13 @@ File: lziprecover.info, Node: Introduction, Next: Invoking lziprecover, Prev:
**************
Lziprecover is a data recovery tool and decompressor for files in the
lzip compressed data format (.lz), able to repair slightly damaged
files, produce a correct file by merging the good parts of two or more
damaged copies, extract data from damaged files, decompress files and
test integrity of files.
lzip compressed data format (.lz). Lziprecover is able to repair
slightly damaged files, produce a correct file by merging the good parts
of two or more damaged copies, extract data from damaged files,
decompress files and test integrity of files.
Lziprecover provides random access to the data in multimember files;
it only decompresses the members containing the desired data.
Lziprecover is not a replacement for regular backups, but a last
line of defense for the case where the backups are also damaged.
@ -61,11 +64,11 @@ availability:
merging of damaged copies of a file. *Note Data safety::.
* The lzip format is as simple as possible (but not simpler). The
lzip manual provides the code of a simple decompressor along with
a detailed explanation of how it works, so that with the only help
of the lzip manual it would be possible for a digital
archaeologist to extract the data from a lzip file long after
quantum computers eventually render LZMA obsolete.
lzip manual provides the source code of a simple decompressor
along with a detailed explanation of how it works, so that with
the only help of the lzip manual it would be possible for a
digital archaeologist to extract the data from a lzip file long
after quantum computers eventually render LZMA obsolete.
* Additionally the lzip reference implementation is copylefted, which
guarantees that it will remain free forever.
@ -94,12 +97,6 @@ garbage data may be produced at the end of each member):
lziprecover -D0 -i -o file -q file.lz
Lziprecover provides random access to the data in multimember files;
it only decompresses the members containing the desired data.
Lziprecover can print correct total file sizes and ratios even for
multimember files.
When recovering data, lziprecover takes as arguments the names of the
damaged files and writes zero or more recovered files depending on the
operation selected and whether the recovery succeeded or not. The
@ -108,6 +105,10 @@ damaged files themselves are never modified.
When decompressing or testing file integrity, lziprecover behaves
like lzip or lunzip.
LANGUAGE NOTE: Uncompressed = not compressed = plain data; it may
never have been compressed. Decompressed is used to refer to data which
have undergone the process of decompression.

File: lziprecover.info, Node: Invoking lziprecover, Next: Data safety, Prev: Introduction, Up: Top
@ -204,9 +205,18 @@ the first time it appears in the command line.
'-l'
'--list'
Print total file sizes and ratios. The values produced are correct
even for multimember files. Use it together with '-v' to see
information about the members in the file.
Print the uncompressed size, compressed size and percentage saved
of the specified file(s). Trailing data are ignored. The values
produced are correct even for multimember files. If more than one
file is given, a final line containing the cumulative sizes is
printed. With '-v', the dictionary size, the number of members in
the file, and the amount of trailing data (if any) are also
printed. With '-vv', the positions and sizes of each member in
multimember files are also printed. '-lq' can be used to verify
quickly (without decompressing) the structural integrity of the
specified files. (Use '--test' to verify the data integrity).
'-alq' additionally verifies that none of the specified files
contain trailing data.
'-m'
'--merge'
@ -234,11 +244,11 @@ the first time it appears in the command line.
'-R'
'--repair'
Try to repair a file with small errors (up to one byte error per
member). If successful, a repaired copy is written to the file
'FILE_fixed.lz'. 'FILE' is not modified at all. The exit status
is 0 if the file could be repaired, 2 otherwise. See the chapter
'Repairing files' (*note Repairing files::) for a complete
Try to repair a file with small errors (up to one single-byte
error per member). If successful, a repaired copy is written to
the file 'FILE_fixed.lz'. 'FILE' is not modified at all. The exit
status is 0 if the file could be repaired, 2 otherwise. See the
chapter 'Repairing files' (*note Repairing files::) for a complete
description of the repair mode.
'-s'
@ -261,8 +271,9 @@ the first time it appears in the command line.
Check integrity of the specified file(s), but don't decompress
them. This really performs a trial decompression and throws away
the result. Use it together with '-v' to see information about
the file(s). If a file fails the test, lziprecover continues
checking the rest of the files.
the file(s). If a file fails the test, does not exist, can't be
opened, or is a terminal, lziprecover continues checking the rest
of the files.
'-v'
'--verbose'
@ -270,7 +281,11 @@ the first time it appears in the command line.
When decompressing or testing, further -v's (up to 4) increase the
verbosity level, showing status, compression ratio, dictionary
size, trailer contents (CRC, data size, member size), and up to 6
bytes of trailing data (if any).
bytes of trailing data (if any) both in hexadecimal and as a
string of printable ASCII characters.
In other modes, increasing verbosity levels show final status,
progress of operations, and extra information (for example, the
failed areas).
Numbers given as arguments to options may be followed by a multiplier
@ -316,7 +331,7 @@ files::), if at least one backup copy of the file is made.
separate media.
How does lzip compare with gzip and bzip2 with respect to data
safety? Lets suppose that you made a backup of your valuable
safety? Let's suppose that you made a backup of your valuable
scientific data, compressed it, and stored two copies on separate
media. Years later you notice that both copies are corrupt.
@ -362,10 +377,11 @@ vice versa. It may be caused by bad RAM or even by natural radiation. I
have seen a case of bit-flip in a file stored on an USB flash drive.
One byte may seem small, but most file corruptions not produced by
I/O errors just affect one byte, or even one bit, of the file. Also,
unlike magnetic media, where errors usually affect a whole sector,
solid-state storage devices tend to produce single-byte errors, making
of lzip the perfect format for data stored on such devices.
transmission errors or I/O errors just affect one byte, or even one bit,
of the file. Also, unlike magnetic media, where errors usually affect a
whole sector, solid-state storage devices tend to produce single-byte
errors, making of lzip the perfect format for data stored on such
devices.
Repairing a file can take some time. Small files or files with the
error located near the beginning can be repaired in a few seconds. But
@ -395,11 +411,11 @@ the file.
is damaged in all copies), or are adjacent and the boundary can't be
determined, or if the copies have too many damaged areas.
All the copies must have the same size. If any of them is larger or
smaller than it should, either because it has been truncated or because
it got some garbage data appended at the end, it can be brought to the
correct size with the following command before merging it with the other
copies:
All the copies to be merged must have the same size. If any of them
is larger or smaller than it should, either because it has been
truncated or because it got some garbage data appended at the end, it
can be brought to the correct size with the following command before
merging it with the other copies:
ddrescue -s<correct_size> -x<correct_size> file.lz correct_size_file.lz
@ -411,6 +427,29 @@ few MB) with small errors (one sector damaged per copy), the probability
approaches 100 percent even with only two copies. (Supposing that the
errors are randomly located inside each copy).
Some types of solid-state device (NAND flash, for example) can
produce bursts of scattered single-bit errors. Lziprecover is able to
merge files with thousands of such scattered errors by grouping the
errors into clusters and then merging the files as if each cluster were
a single error.
Here is a real case of successful merging. Two copies of the file
'icecat-3.5.3-x86.tar.lz' (compressed size 9 MB) became corrupt while
stored on the same NAND flash device. One of the copies had 76
single-bit errors scattered in an area of 1020 bytes, and the other had
3028 such errors in an area of 31729 bytes. Lziprecover produced a
correct file, identical to the original, in just 5 seconds:
$ lziprecover -vvm a/icecat-3.5.3-x86.tar.lz b/icecat-3.5.3-x86.tar.lz
Merging member 1 of 1 (2552 errors)
2552 errors have been grouped in 16 clusters.
Trying variation 2 of 2, block 2
Input files merged successfully.
Note that the number of errors reported by lziprecover (2552) is
lower than the number of corrupt bytes (3104) because contiguous
corrupt bytes are counted as a single multibyte error.

File: lziprecover.info, Node: File names, Next: File format, Prev: Merging files, Up: Top
@ -499,16 +538,21 @@ File: lziprecover.info, Node: Trailing data, Next: Examples, Prev: File forma
8 Extra data appended to the file
*********************************
Sometimes extra data is found appended to a lzip file after the last
Sometimes extra data are found appended to a lzip file after the last
member. Such trailing data may be:
* Padding added to make the file size a multiple of some block size,
for example when writing to a tape.
* Garbage added by some not totally successful copy operation.
for example when writing to a tape. It is safe to append any
amount of padding zero bytes to a lzip file.
* Useful data added by the user; a cryptographically secure hash, a
description of file contents, etc.
description of file contents, etc. It is safe to append any amount
of text to a lzip file as long as the text does not begin with the
string "LZIP", and does not contain any zero bytes (null
characters). Nonzero bytes and zero bytes can't be safely mixed in
trailing data.
* Garbage added by some not totally successful copy operation.
* Malicious data added to the file in order to make its total size
and hash value (for a chosen hash) coincide with those of another
@ -521,8 +565,12 @@ member. Such trailing data may be:
the corruption of the integrity information itself. Therefore it
can be considered to be below the noise level.
Trailing data are in no way part of the lzip file format, but tools
reading lzip files are expected to behave as correctly and usefully as
possible in the presence of trailing data.
Trailing data can be safely ignored in most cases. In some cases,
like that of user-added data, it is expected to be ignored. In those
like that of user-added data, they are expected to be ignored. In those
cases where a file containing trailing data must be rejected, the option
'--trailing-error' can be used. *Note --trailing-error::.
@ -544,8 +592,8 @@ show status.
lziprecover -tv file.lz
Example 3: The right way of concatenating compressed files. *Note
Trailing data::.
Example 3: The right way of concatenating the decompressed output of two
or more compressed files. *Note Trailing data::.
Don't do this
cat file1.lz file2.lz file3.lz | lziprecover -d
@ -703,6 +751,16 @@ by 'zutils'. *Note Zcmp: (zutils)Zcmp,
Test only one of every N bytes, blocks or truncation sizes,
instead of all of them.
'-e POSITION,VALUE'
'--set-byte=POSITION,VALUE'
Set byte at POSITION to VALUE in the internal buffer after reading
and testing FILENAME.lz but before the first test call to the
decompressor. If VALUE is preceded by '+', it is added to the
original value of the byte at POSITION. If VALUE is preceded by
'f' (flip), it is XORed with the original value of the byte at
POSITION. This option can be used to run tests with a changed
dictionary size, for example.
'-p BYTES'
'--position=BYTES'
First byte position to test in the file. Defaults to 0. Negative
@ -779,21 +837,21 @@ Concept index

Tag Table:
Node: Top231
Node: Introduction1267
Node: Invoking lziprecover4525
Ref: --trailing-error5175
Node: Data safety11779
Node: Repairing files13702
Node: Merging files15602
Node: File names17217
Node: File format17681
Node: Trailing data20109
Node: Examples21492
Ref: concat-example21923
Ref: ddrescue-example22986
Node: Unzcrash24276
Node: Problems28786
Node: Concept index29338
Node: Introduction1269
Node: Invoking lziprecover4646
Ref: --trailing-error5296
Node: Data safety12788
Node: Repairing files14712
Node: Merging files16635
Node: File names19397
Node: File format19861
Node: Trailing data22289
Node: Examples24195
Ref: concat-example24626
Ref: ddrescue-example25727
Node: Unzcrash27017
Node: Problems32021
Node: Concept index32573

End Tag Table