lzip/lziprecover
1
0
Fork 0
Code Issues Activity

Adding upstream version 1.15~rc1.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-21 11:17:33 +01:00
parent 0ff20a5602
commit 6a117924f6
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
20 changed files with 439 additions and 219 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
ChangeLog
View file

@ -1,10 +1,16 @@
2013-07-29 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.15-rc1 released.
* Minor changes.
* Added chapter 'Unzcrash' to the manual.
2013-06-17 Antonio Diaz Diaz <antonio@gnu.org> 2013-06-17 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.15-pre1 released. * Version 1.15-pre1 released.
* repair.cc: Repair multi-member files with up to one byte error * repair.cc: Repair multi-member files with up to one byte error
per member. per member.
* merge.cc: Merge multi-member files. * merge.cc: Merge multi-member files.
* Added chapters 'Repairing Files' and 'Merging Files' to the manual. * Added chapters 'Repairing files' and 'Merging files' to the manual.
2013-05-31 Antonio Diaz Diaz <antonio@gnu.org> 2013-05-31 Antonio Diaz Diaz <antonio@gnu.org>

4
INSTALL
View file

@ -10,9 +10,9 @@ Procedure
--------- ---------
1. Unpack the archive if you have not done so already: 1. Unpack the archive if you have not done so already:
lzip -cd lziprecover[version].tar.lz | tar -xf - tar -xf lziprecover[version].tar.lz
or or
gzip -cd lziprecover[version].tar.gz | tar -xf - lzip -cd lziprecover[version].tar.lz | tar -xf -
This creates the directory ./lziprecover[version] containing the source This creates the directory ./lziprecover[version] containing the source
from the main archive. from the main archive.

4
NEWS
View file

@ -6,5 +6,5 @@ per member, without having to split them first.
Lziprecover can now merge multi-member files without having to split Lziprecover can now merge multi-member files without having to split
them first even if some copies have the header and the trailer damaged. them first even if some copies have the header and the trailer damaged.
The chapters "Repairing Files" and "Merging Files" have been added to The chapters "Repairing files", "Merging files" and "Unzcrash" have been
the manual. added to the manual.

51
README
View file

@ -1,16 +1,29 @@
Description Description
Lziprecover is a data recovery tool and decompressor for files in the Lziprecover is a data recovery tool and decompressor for files in the
lzip compressed data format (.lz) able to repair slightly damaged files, lzip compressed data format (.lz), able to repair slightly damaged
recover badly damaged files from two or more copies, extract data from files, recover badly damaged files from two or more copies, extract data
damaged files, decompress files and test integrity of files. from damaged files, decompress files and test integrity of files.
The lzip file format is designed for long-term data archiving. It is
clean, provides very safe 4 factor integrity checking, and is backed by
the recovery capabilities of lziprecover.
Lziprecover is able to recover or decompress files produced by any of Lziprecover is able to recover or decompress files produced by any of
the compressors in the lzip family; lzip, plzip, minilzip/lzlib, clzip the compressors in the lzip family; lzip, plzip, minilzip/lzlib, clzip
and pdlzip. It makes lzip files resistant to bit-flip, one of the most and pdlzip.
common forms of data corruption, and its recovery capabilities
contribute to make of the lzip format one of the best options for Lziprecover makes lzip files resistant to bit-flip (one of the most
long-term data archiving. common forms of data corruption), and can safely merge multiple damaged
backup copies.
If the cause of file corruption is damaged media, the combination
GNU ddrescue + lziprecover is the best option for recovering data from
multiple damaged copies.
If a file is too damaged for lziprecover to repair it, all the
recoverable data in all members of the file can be extracted with the
'-D' option.
Lziprecover is able to efficiently extract a range of bytes from a Lziprecover is able to efficiently extract a range of bytes from a
multi-member file, because it only decompresses the members containing multi-member file, because it only decompresses the members containing
@ -27,24 +40,20 @@ damaged files themselves are never modified.
When decompressing or testing file integrity, lziprecover behaves like When decompressing or testing file integrity, lziprecover behaves like
lzip or lunzip. lzip or lunzip.
If the files are too damaged for lziprecover to repair them, data from To give you an idea of its possibilities, when merging two copies, each
damaged members can be partially recovered writing it to stdout.
To give you an idea of its possibilities, when merging two copies each
of them with one damaged area affecting 1 percent of the copy, the of them with one damaged area affecting 1 percent of the copy, the
probability of obtaining a correct file is about 98 percent. With three probability of obtaining a correct file is about 98 percent. With three
such copies the probability rises to 99.97 percent. For large files with such copies the probability rises to 99.97 percent. For large files (a
small errors, the probability approaches 100 percent even with only two few MB) with small errors (one sector damaged per copy), the probability
copies. approaches 100 percent even with only two copies.
If the cause of file corruption is damaged media, the combination Lziprecover is not a replacement for regular backups, but a last line of
GNU ddrescue + lziprecover is the best option for recovering data from defense for the case where the backups are also damaged.
multiple damaged copies.
This package also includes unzcrash, a program written to test The lziprecover package also includes unzcrash, a program written to
robustness to decompression of corrupted data, inspired by unzcrash.c test robustness to decompression of corrupted data, inspired by
from Julian Seward's bzip2. Type 'make unzcrash' in the lziprecover unzcrash.c from Julian Seward's bzip2. Type 'make unzcrash' in the
directory to build it. Then try 'unzcrash --help'. lziprecover source directory to build it. Then try 'unzcrash --help'.
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.

8
configure vendored
View file

@ -1,14 +1,14 @@
#! /bin/sh #! /bin/sh
# configure script for Lziprecover - Data recovery tool for lzipped files # configure script for Lziprecover - Data recovery tool for lzip files
# Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. # Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
# #
# This configure script is free software: you have unlimited permission # This configure script is free software: you have unlimited permission
# to copy, distribute and modify it. # to copy, distribute and modify it.
pkgname=lziprecover pkgname=lziprecover
pkgversion=1.15-pre1 pkgversion=1.15-rc1
progname=lziprecover progname=lziprecover
srctrigger=doc/lziprecover.texinfo srctrigger=doc/${pkgname}.texinfo
# clear some things potentially inherited from environment. # clear some things potentially inherited from environment.
LC_ALL=C LC_ALL=C
@ -164,7 +164,7 @@ echo "CXXFLAGS = ${CXXFLAGS}"
echo "LDFLAGS = ${LDFLAGS}" echo "LDFLAGS = ${LDFLAGS}"
rm -f Makefile rm -f Makefile
cat > Makefile << EOF cat > Makefile << EOF
# Makefile for Lziprecover - Data recovery tool for lzipped files # Makefile for Lziprecover - Data recovery tool for lzip files
# Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. # Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
# This file was generated automatically by configure. Do not edit. # This file was generated automatically by configure. Do not edit.
# #

16
decoder.cc
View file

@ -1,4 +1,4 @@
/* Lziprecover - Data recovery tool for lzipped files /* Lziprecover - Data recovery tool for lzip files
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
@ -104,10 +104,10 @@ bool Range_decoder::read_block()
void LZ_decoder::flush_data() void LZ_decoder::flush_data()
{ {
const int size = pos - stream_pos; if( pos > stream_pos )
if( size > 0 )
{ {
crc32.update( crc_, buffer + stream_pos, size ); const int size = pos - stream_pos;
crc32.update_buf( crc_, buffer + stream_pos, size );
if( outfd >= 0 ) if( outfd >= 0 )
{ {
const unsigned long long sp = stream_position(); const unsigned long long sp = stream_position();
@ -207,7 +207,6 @@ int LZ_decoder::decode_member( const Pretty_print & pp )
Bit_model bm_align[dis_align_size]; Bit_model bm_align[dis_align_size];
Len_model match_len_model; Len_model match_len_model;
Len_model rep_len_model; Len_model rep_len_model;
unsigned rep0 = 0; // rep[0-3] latest four distances unsigned rep0 = 0; // rep[0-3] latest four distances
unsigned rep1 = 0; // used for efficient coding of unsigned rep1 = 0; // used for efficient coding of
unsigned rep2 = 0; // repeated distances unsigned rep2 = 0; // repeated distances
@ -222,11 +221,16 @@ int LZ_decoder::decode_member( const Pretty_print & pp )
{ {
const uint8_t prev_byte = get_prev_byte(); const uint8_t prev_byte = get_prev_byte();
if( state.is_char() ) if( state.is_char() )
{
state.set_char1();
put_byte( rdec.decode_tree( bm_literal[get_lit_state(prev_byte)], 8 ) ); put_byte( rdec.decode_tree( bm_literal[get_lit_state(prev_byte)], 8 ) );
}
else else
{
state.set_char2();
put_byte( rdec.decode_matched( bm_literal[get_lit_state(prev_byte)], put_byte( rdec.decode_matched( bm_literal[get_lit_state(prev_byte)],
get_byte( rep0 ) ) ); get_byte( rep0 ) ) );
state.set_char(); }
} }
else else
{ {

3
decoder.h
View file

@ -1,4 +1,4 @@
/* Lziprecover - Data recovery tool for lzipped files /* Lziprecover - Data recovery tool for lzip files
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
@ -278,7 +278,6 @@ public:
~LZ_decoder() { delete[] buffer; } ~LZ_decoder() { delete[] buffer; }
unsigned crc() const { return crc_ ^ 0xFFFFFFFFU; } unsigned crc() const { return crc_ ^ 0xFFFFFFFFU; }
unsigned long long data_position() const { return partial_data_pos + pos; } unsigned long long data_position() const { return partial_data_pos + pos; }
int decode_member( const Pretty_print & pp ); int decode_member( const Pretty_print & pp );

4
doc/lziprecover.1
View file

@ -1,12 +1,12 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.37.1. .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.37.1.
.TH LZIPRECOVER "1" "June 2013" "Lziprecover 1.15-pre1" "User Commands" .TH LZIPRECOVER "1" "July 2013" "Lziprecover 1.15-rc1" "User Commands"
.SH NAME .SH NAME
Lziprecover \- recovers data from damaged lzip files Lziprecover \- recovers data from damaged lzip files
.SH SYNOPSIS .SH SYNOPSIS
.B lziprecover .B lziprecover
[\fIoptions\fR] [\fIfiles\fR] [\fIoptions\fR] [\fIfiles\fR]
.SH DESCRIPTION .SH DESCRIPTION
Lziprecover \- Data recovery tool and decompressor for lzipped files. Lziprecover \- Data recovery tool and decompressor for lzip files.
.SH OPTIONS .SH OPTIONS
.TP .TP
\fB\-h\fR, \fB\-\-help\fR \fB\-h\fR, \fB\-\-help\fR

230
doc/lziprecover.info
View file

@ -3,7 +3,7 @@ lziprecover.texinfo.
INFO-DIR-SECTION Data Compression INFO-DIR-SECTION Data Compression
START-INFO-DIR-ENTRY START-INFO-DIR-ENTRY
* Lziprecover: (lziprecover). Data recovery tool for lzipped files * Lziprecover: (lziprecover). Data recovery tool for lzip files
END-INFO-DIR-ENTRY END-INFO-DIR-ENTRY
 
@ -12,18 +12,19 @@ File: lziprecover.info, Node: Top, Next: Introduction, Up: (dir)
Lziprecover Manual Lziprecover Manual
****************** ******************
This manual is for Lziprecover (version 1.15-pre1, 17 June 2013). This manual is for Lziprecover (version 1.15-rc1, 29 July 2013).
* Menu: * Menu:
* Introduction:: Purpose and features of lziprecover * Introduction:: Purpose and features of lziprecover
* Invoking Lziprecover:: Command line interface * Invoking lziprecover:: Command line interface
* Repairing Files:: Fixing bit-flip and similar errors * Repairing files:: Fixing bit-flip and similar errors
* Merging Files:: Fixing several damaged copies * Merging files:: Fixing several damaged copies
* File Format:: Detailed format of the compressed file * File format:: Detailed format of the compressed file
* Examples:: A small tutorial with examples * Examples:: A small tutorial with examples
* Unzcrash:: Testing the robustness of decompressors
* Problems:: Reporting bugs * Problems:: Reporting bugs
* Concept Index:: Index of concepts * Concept index:: Index of concepts
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
@ -32,22 +33,38 @@ This manual is for Lziprecover (version 1.15-pre1, 17 June 2013).
copy, distribute and modify it. copy, distribute and modify it.
 
File: lziprecover.info, Node: Introduction, Next: Invoking Lziprecover, Prev: Top, Up: Top File: lziprecover.info, Node: Introduction, Next: Invoking lziprecover, Prev: Top, Up: Top
1 Introduction 1 Introduction
************** **************
Lziprecover is a data recovery tool and decompressor for files in the Lziprecover is a data recovery tool and decompressor for files in the
lzip compressed data format (.lz) able to repair slightly damaged files, lzip compressed data format (.lz), able to repair slightly damaged
recover badly damaged files from two or more copies, extract data from files, recover badly damaged files from two or more copies, extract data
damaged files, decompress files and test integrity of files. from damaged files, decompress files and test integrity of files.
The lzip file format is designed for long-term data archiving. It is
clean, provides very safe 4 factor integrity checking, and is backed by
the recovery capabilities of lziprecover.
Lziprecover is able to recover or decompress files produced by any of Lziprecover is able to recover or decompress files produced by any of
the compressors in the lzip family; lzip, plzip, minilzip/lzlib, clzip the compressors in the lzip family; lzip, plzip, minilzip/lzlib, clzip
and pdlzip. It makes lzip files resistant to bit-flip, one of the most and pdlzip.
common forms of data corruption, and its recovery capabilities
contribute to make of the lzip format one of the best options for Lziprecover makes lzip files resistant to bit-flip (one of the most
long-term data archiving. common forms of data corruption), and can safely merge multiple damaged
backup copies.
If the cause of file corruption is damaged media, the combination
GNU ddrescue + lziprecover is the best option for recovering data from
multiple damaged copies. *Note ddrescue-example::, for an example.
If a file is too damaged for lziprecover to repair it, all the
recoverable data in all members of the file can be extracted with the
following command (the resulting file may contain errors and some
garbage data may be produced at the end of each member):
lziprecover -D0 -i -o file -q file.lz
Lziprecover is able to efficiently extract a range of bytes from a Lziprecover is able to efficiently extract a range of bytes from a
multi-member file, because it only decompresses the members containing multi-member file, because it only decompresses the members containing
@ -64,24 +81,13 @@ damaged files themselves are never modified.
When decompressing or testing file integrity, lziprecover behaves When decompressing or testing file integrity, lziprecover behaves
like lzip or lunzip. like lzip or lunzip.
If a file is too damaged for lziprecover to repair it, all the
recoverable data in all members of the file can be extracted with the
following command (the resulting file may contain errors and some
garbage data may be produced at the end of each member):
lziprecover -D0 -i -o file -q file.lz
If the cause of file corruption is damaged media, the combination
GNU ddrescue + lziprecover is the best option for recovering data from
multiple damaged copies. *Note ddrescue-example::, for an example.
Lziprecover is not a replacement for regular backups, but a last Lziprecover is not a replacement for regular backups, but a last
line of defense for the case where the backups are also damaged. line of defense for the case where the backups are also damaged.
 
File: lziprecover.info, Node: Invoking Lziprecover, Next: Repairing Files, Prev: Introduction, Up: Top File: lziprecover.info, Node: Invoking lziprecover, Next: Repairing files, Prev: Introduction, Up: Top
2 Invoking Lziprecover 2 Invoking lziprecover
********************** **********************
The format for running lziprecover is: The format for running lziprecover is:
@ -149,8 +155,8 @@ The format for running lziprecover is:
Try to produce a correct file merging the good parts of two or more Try to produce a correct file merging the good parts of two or more
damaged copies. If successful, a repaired copy is written to the damaged copies. If successful, a repaired copy is written to the
file `FILE_fixed.lz'. The exit status is 0 if a correct file could file `FILE_fixed.lz'. The exit status is 0 if a correct file could
be produced, 2 otherwise. See the chapter Merging Files (*note be produced, 2 otherwise. See the chapter `Merging files' (*note
Merging Files::) for a complete description of the merge mode. Merging files::) for a complete description of the merge mode.
`-o FILE' `-o FILE'
`--output=FILE' `--output=FILE'
@ -167,10 +173,10 @@ The format for running lziprecover is:
`-R' `-R'
`--repair' `--repair'
Try to repair a file with small errors (up to one byte error per Try to repair a file with small errors (up to one byte error per
member). If successful, a repaired copy is written to the file member). If successful, a repaired copy is written to the file
`FILE_fixed.lz'. `FILE' is not modified at all. The exit status `FILE_fixed.lz'. `FILE' is not modified at all. The exit status
is 0 if the file could be repaired, 2 otherwise. See the chapter is 0 if the file could be repaired, 2 otherwise. See the chapter
Repairing Files (*note Repairing Files::) for a complete `Repairing files' (*note Repairing files::) for a complete
description of the repair mode. description of the repair mode.
`-s' `-s'
@ -199,12 +205,11 @@ The format for running lziprecover is:
`--verbose' `--verbose'
Verbose mode. Verbose mode.
When decompressing or testing, further -v's (up to 4) increase the When decompressing or testing, further -v's (up to 4) increase the
verbosity level, showing status, dictionary size, compression verbosity level, showing status, compression ratio, dictionary
ratio, trailer contents (CRC, data size, member size), and up to 6 size, trailer contents (CRC, data size, member size), and up to 6
bytes of trailing garbage (if any). bytes of trailing garbage (if any).
Numbers given as arguments to options may be followed by a multiplier Numbers given as arguments to options may be followed by a multiplier
and an optional `B' for "byte". and an optional `B' for "byte".
@ -227,9 +232,9 @@ invalid input file, 3 for an internal consistency error (eg, bug) which
caused lziprecover to panic. caused lziprecover to panic.
 
File: lziprecover.info, Node: Repairing Files, Next: Merging Files, Prev: Invoking Lziprecover, Up: Top File: lziprecover.info, Node: Repairing files, Next: Merging files, Prev: Invoking lziprecover, Up: Top
3 Repairing Files 3 Repairing files
***************** *****************
Lziprecover is able to repair files with small errors (up to one byte Lziprecover is able to repair files with small errors (up to one byte
@ -244,13 +249,13 @@ vice versa. It may be caused by bad RAM or even by natural radiation. I
have seen a case of bit-flip in a file stored in an USB flash drive. have seen a case of bit-flip in a file stored in an USB flash drive.
 
File: lziprecover.info, Node: Merging Files, Next: File Format, Prev: Repairing Files, Up: Top File: lziprecover.info, Node: Merging files, Next: File format, Prev: Repairing files, Up: Top
4 Merging Files 4 Merging files
*************** ***************
If you have several copies of a file but all of them are too damaged to If you have several copies of a file but all of them are too damaged to
repair them (*note Repairing Files::), lziprecover can try to produce a repair them (*note Repairing files::), lziprecover can try to produce a
correct file merging the good parts of the damaged copies. correct file merging the good parts of the damaged copies.
The merge may succeed even if some copies of the file have all the The merge may succeed even if some copies of the file have all the
@ -262,17 +267,31 @@ the file.
is damaged in all copies), or are adjacent and the boundary can't be is damaged in all copies), or are adjacent and the boundary can't be
determined, or if the copies have too many damaged areas. determined, or if the copies have too many damaged areas.
To give you an idea of its possibilities, when merging two copies All the copies must have the same size. If some of them have been
truncated and are therefore smaller than they should, you can extend
them to the correct size with the following command before merging them
with the other copies:
ddrescue --extend-outfile=<correct_size> small_file.lz extended_file.lz
If some of the copies have got garbage data at the end and are
therefore larger than they should, you can reduce their sizes to the
correct value with the following command before merging them with the
other copies:
ddrescue --size=<correct_size> large_file.lz reduced_file.lz
To give you an idea of its possibilities, when merging two copies,
each of them with one damaged area affecting 1 percent of the copy, the each of them with one damaged area affecting 1 percent of the copy, the
probability of obtaining a correct file is about 98 percent. With three probability of obtaining a correct file is about 98 percent. With three
such copies the probability rises to 99.97 percent. For large files with such copies the probability rises to 99.97 percent. For large files (a
small errors, the probability approaches 100 percent even with only two few MB) with small errors (one sector damaged per copy), the probability
copies. approaches 100 percent even with only two copies.
 
File: lziprecover.info, Node: File Format, Next: Examples, Prev: Merging Files, Up: Top File: lziprecover.info, Node: File format, Next: Examples, Prev: Merging files, Up: Top
5 File Format 5 File format
************* *************
Perfection is reached, not when there is no longer anything to add, but Perfection is reached, not when there is no longer anything to add, but
@ -343,7 +362,7 @@ additional information before, between, or after them.
 
File: lziprecover.info, Node: Examples, Next: Problems, Prev: File Format, Up: Top File: lziprecover.info, Node: Examples, Next: Unzcrash, Prev: File format, Up: Top
6 A small tutorial with examples 6 A small tutorial with examples
******************************** ********************************
@ -381,8 +400,8 @@ are abridged diagnostic messages from lziprecover).
Example 6: Split the multi-member file `file.lz' and write each member Example 6: Split the multi-member file `file.lz' and write each member
in its own `recXXXXXfile.lz' file. Then use `lziprecover -t' to test in its own `recXXXfile.lz' file. Then use `lziprecover -t' to test the
the integrity of the resulting files. integrity of the resulting files.
lziprecover -s file.lz lziprecover -s file.lz
lziprecover -tv rec*file.lz lziprecover -tv rec*file.lz
@ -401,7 +420,7 @@ error-checked merging of copies (*Note GNU ddrescue manual:
mount -t iso9660 -o loop,ro cdimage2 /mnt/cdimage mount -t iso9660 -o loop,ro cdimage2 /mnt/cdimage
cp /mnt/cdimage/backup.tar.lz rescued2.tar.lz cp /mnt/cdimage/backup.tar.lz rescued2.tar.lz
umount /mnt/cdimage umount /mnt/cdimage
lziprecover -m -v -o rescued.tar.lz rescued1.tar.lz rescued2.tar.lz lziprecover -m -v -o backup.tar.lz rescued1.tar.lz rescued2.tar.lz
Example 8: Recover the first volume of those created with the command Example 8: Recover the first volume of those created with the command
@ -414,9 +433,80 @@ correct file produced is saved in `big_db_00001.lz'.
Input files merged successfully Input files merged successfully
 
File: lziprecover.info, Node: Problems, Next: Concept Index, Prev: Examples, Up: Top File: lziprecover.info, Node: Unzcrash, Next: Problems, Prev: Examples, Up: Top
7 Reporting Bugs 7 Testing the robustness of decompressors
*****************************************
The lziprecover package also includes unzcrash, a program written to
test robustness to decompression of corrupted data, inspired by
unzcrash.c from Julian Seward's bzip2. Type `make unzcrash' in the
lziprecover source directory to build it.
Unzcrash reads the specified file and then repeatedly decompresses
it, increasing 256 times each byte of the compressed data, so as to
test all possible one-byte errors. This should not cause any invalid
memory accesses. If it does, please, report it as a bug.
Unzcrash really executes as a subprocess the shell command specified
in the first non-option argument, and then writes the file specified in
the second non-option argument to the standard input of the subprocess,
modifying the corresponding byte each time. Therefore you can use
unzcrash to test any decompressor (not only lzip), or even other decoder
programs with a suitable command line syntax.
The format for running unzcrash is:
unzcrash [OPTIONS] "lzip -tv" FILENAME.lz
Unzcrash supports the following options:
`-h'
`--help'
Print an informative help message describing the options and exit.
`-V'
`--version'
Print the version number of lziprecover on the standard output and
exit.
`-b RANGE'
`--bits=RANGE'
Test N-bit errors only, instead of testing all the 255 wrong
values for each byte. `N-bit error' means any value differing from
the original value in N bit positions, not a value differing from
the original value in the bit position N.
The number of N-bit errors per byte (N = 1 to 8) is: 8 28 56 70 56
28 8 1
Examples of RANGE: 1 1,2,3 1-4 1,3-5,8 1-3,5-8
`-p BYTES'
`--position=BYTES'
First byte position to test in the file. Defaults to 0.
`-q'
`--quiet'
Quiet operation. Suppress all messages.
`-s BYTES'
`--size=BYTES'
Number of byte positions to test. If not specified, the whole file
is tested.
`-v'
`--verbose'
Verbose mode.
Exit status: 0 for a normal exit, 1 for environmental problems (file
not found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or
invalid input file, 3 for an internal consistency error (eg, bug) which
caused unzcrash to panic.

File: lziprecover.info, Node: Problems, Next: Concept index, Prev: Unzcrash, Up: Top
8 Reporting bugs
**************** ****************
There are probably bugs in lziprecover. There are certainly errors and There are probably bugs in lziprecover. There are certainly errors and
@ -429,9 +519,9 @@ for all eternity, if not longer.
by running `lziprecover --version'. by running `lziprecover --version'.
 
File: lziprecover.info, Node: Concept Index, Prev: Problems, Up: Top File: lziprecover.info, Node: Concept index, Prev: Problems, Up: Top
Concept Index Concept index
************* *************
[index] [index]
@ -439,26 +529,28 @@ Concept Index
* bugs: Problems. (line 6) * bugs: Problems. (line 6)
* examples: Examples. (line 6) * examples: Examples. (line 6)
* file format: File Format. (line 6) * file format: File format. (line 6)
* getting help: Problems. (line 6) * getting help: Problems. (line 6)
* introduction: Introduction. (line 6) * introduction: Introduction. (line 6)
* invoking: Invoking Lziprecover. (line 6) * invoking: Invoking lziprecover. (line 6)
* merging files: Merging Files. (line 6) * merging files: Merging files. (line 6)
* repairing files: Repairing Files. (line 6) * repairing files: Repairing files. (line 6)
* unzcrash: Unzcrash. (line 6)
 
Tag Table: Tag Table:
Node: Top231 Node: Top228
Node: Introduction1032 Node: Introduction1095
Node: Invoking Lziprecover3052 Node: Invoking lziprecover3248
Node: Repairing Files8489 Node: Repairing files8686
Node: Merging Files9208 Node: Merging files9405
Node: File Format10338 Node: File format11176
Node: Examples12822 Node: Examples13660
Ref: ddrescue-example14024 Ref: ddrescue-example14860
Node: Problems15134 Node: Unzcrash15969
Node: Concept Index15684 Node: Problems18344
Node: Concept index18894
 
End Tag Table End Tag Table

206
doc/lziprecover.texinfo
View file

@ -6,19 +6,19 @@
@finalout @finalout
@c %**end of header @c %**end of header
@set UPDATED 17 June 2013 @set UPDATED 29 July 2013
@set VERSION 1.15-pre1 @set VERSION 1.15-rc1
@dircategory Data Compression @dircategory Data Compression
@direntry @direntry
* Lziprecover: (lziprecover). Data recovery tool for lzipped files * Lziprecover: (lziprecover). Data recovery tool for lzip files
@end direntry @end direntry
@ifnothtml @ifnothtml
@titlepage @titlepage
@title Lziprecover @title Lziprecover
@subtitle Data recovery tool for lzipped files @subtitle Data recovery tool for lzip files
@subtitle for Lziprecover version @value{VERSION}, @value{UPDATED} @subtitle for Lziprecover version @value{VERSION}, @value{UPDATED}
@author by Antonio Diaz Diaz @author by Antonio Diaz Diaz
@ -36,13 +36,14 @@ This manual is for Lziprecover (version @value{VERSION}, @value{UPDATED}).
@menu @menu
* Introduction:: Purpose and features of lziprecover * Introduction:: Purpose and features of lziprecover
* Invoking Lziprecover:: Command line interface * Invoking lziprecover:: Command line interface
* Repairing Files:: Fixing bit-flip and similar errors * Repairing files:: Fixing bit-flip and similar errors
* Merging Files:: Fixing several damaged copies * Merging files:: Fixing several damaged copies
* File Format:: Detailed format of the compressed file * File format:: Detailed format of the compressed file
* Examples:: A small tutorial with examples * Examples:: A small tutorial with examples
* Unzcrash:: Testing the robustness of decompressors
* Problems:: Reporting bugs * Problems:: Reporting bugs
* Concept Index:: Index of concepts * Concept index:: Index of concepts
@end menu @end menu
@sp 1 @sp 1
@ -57,16 +58,34 @@ to copy, distribute and modify it.
@cindex introduction @cindex introduction
Lziprecover is a data recovery tool and decompressor for files in the Lziprecover is a data recovery tool and decompressor for files in the
lzip compressed data format (.lz) able to repair slightly damaged files, lzip compressed data format (.lz), able to repair slightly damaged
recover badly damaged files from two or more copies, extract data from files, recover badly damaged files from two or more copies, extract data
damaged files, decompress files and test integrity of files. from damaged files, decompress files and test integrity of files.
The lzip file format is designed for long-term data archiving. It is
clean, provides very safe 4 factor integrity checking, and is backed by
the recovery capabilities of lziprecover.
Lziprecover is able to recover or decompress files produced by any of Lziprecover is able to recover or decompress files produced by any of
the compressors in the lzip family; lzip, plzip, minilzip/lzlib, clzip the compressors in the lzip family; lzip, plzip, minilzip/lzlib, clzip
and pdlzip. It makes lzip files resistant to bit-flip, one of the most and pdlzip.
common forms of data corruption, and its recovery capabilities
contribute to make of the lzip format one of the best options for Lziprecover makes lzip files resistant to bit-flip (one of the most
long-term data archiving. common forms of data corruption), and can safely merge multiple damaged
backup copies.
If the cause of file corruption is damaged media, the combination
@w{GNU ddrescue + lziprecover} is the best option for recovering data
from multiple damaged copies. @xref{ddrescue-example}, for an example.
If a file is too damaged for lziprecover to repair it, all the
recoverable data in all members of the file can be extracted with the
following command (the resulting file may contain errors and some
garbage data may be produced at the end of each member):
@example
lziprecover -D0 -i -o file -q file.lz
@end example
Lziprecover is able to efficiently extract a range of bytes from a Lziprecover is able to efficiently extract a range of bytes from a
multi-member file, because it only decompresses the members containing multi-member file, because it only decompresses the members containing
@ -83,25 +102,12 @@ damaged files themselves are never modified.
When decompressing or testing file integrity, lziprecover behaves like When decompressing or testing file integrity, lziprecover behaves like
lzip or lunzip. lzip or lunzip.
If a file is too damaged for lziprecover to repair it, all the
recoverable data in all members of the file can be extracted with the
following command (the resulting file may contain errors and some
garbage data may be produced at the end of each member):
@example
lziprecover -D0 -i -o file -q file.lz
@end example
If the cause of file corruption is damaged media, the combination
@w{GNU ddrescue + lziprecover} is the best option for recovering data
from multiple damaged copies. @xref{ddrescue-example}, for an example.
Lziprecover is not a replacement for regular backups, but a last line of Lziprecover is not a replacement for regular backups, but a last line of
defense for the case where the backups are also damaged. defense for the case where the backups are also damaged.
@node Invoking Lziprecover @node Invoking lziprecover
@chapter Invoking Lziprecover @chapter Invoking lziprecover
@cindex invoking @cindex invoking
The format for running lziprecover is: The format for running lziprecover is:
@ -172,8 +178,8 @@ information about the members in the file.
Try to produce a correct file merging the good parts of two or more Try to produce a correct file merging the good parts of two or more
damaged copies. If successful, a repaired copy is written to the file damaged copies. If successful, a repaired copy is written to the file
@samp{@var{file}_fixed.lz}. The exit status is 0 if a correct file could @samp{@var{file}_fixed.lz}. The exit status is 0 if a correct file could
be produced, 2 otherwise. See the chapter Merging Files (@pxref{Merging be produced, 2 otherwise. See the chapter @samp{Merging files}
Files}) for a complete description of the merge mode. (@pxref{Merging files}) for a complete description of the merge mode.
@item -o @var{file} @item -o @var{file}
@itemx --output=@var{file} @itemx --output=@var{file}
@ -190,11 +196,11 @@ Quiet operation. Suppress all messages.
@item -R @item -R
@itemx --repair @itemx --repair
Try to repair a file with small errors (up to one byte error per member). Try to repair a file with small errors (up to one byte error per
If successful, a repaired copy is written to the file member). If successful, a repaired copy is written to the file
@samp{@var{file}_fixed.lz}. @samp{@var{file}} is not modified at all. @samp{@var{file}_fixed.lz}. @samp{@var{file}} is not modified at all.
The exit status is 0 if the file could be repaired, 2 otherwise. The exit status is 0 if the file could be repaired, 2 otherwise. See the
See the chapter Repairing Files (@pxref{Repairing Files}) for a complete chapter @samp{Repairing files} (@pxref{Repairing files}) for a complete
description of the repair mode. description of the repair mode.
@item -s @item -s
@ -221,13 +227,12 @@ Use it together with @samp{-v} to see information about the file.
@itemx --verbose @itemx --verbose
Verbose mode.@* Verbose mode.@*
When decompressing or testing, further -v's (up to 4) increase the When decompressing or testing, further -v's (up to 4) increase the
verbosity level, showing status, dictionary size, compression ratio, verbosity level, showing status, compression ratio, dictionary size,
trailer contents (CRC, data size, member size), and up to 6 bytes of trailer contents (CRC, data size, member size), and up to 6 bytes of
trailing garbage (if any). trailing garbage (if any).
@end table @end table
@sp 1
Numbers given as arguments to options may be followed by a multiplier Numbers given as arguments to options may be followed by a multiplier
and an optional @samp{B} for "byte". and an optional @samp{B} for "byte".
@ -252,8 +257,8 @@ invalid input file, 3 for an internal consistency error (eg, bug) which
caused lziprecover to panic. caused lziprecover to panic.
@node Repairing Files @node Repairing files
@chapter Repairing Files @chapter Repairing files
@cindex repairing files @cindex repairing files
Lziprecover is able to repair files with small errors (up to one byte Lziprecover is able to repair files with small errors (up to one byte
@ -268,12 +273,12 @@ versa. It may be caused by bad RAM or even by natural radiation. I have
seen a case of bit-flip in a file stored in an USB flash drive. seen a case of bit-flip in a file stored in an USB flash drive.
@node Merging Files @node Merging files
@chapter Merging Files @chapter Merging files
@cindex merging files @cindex merging files
If you have several copies of a file but all of them are too damaged to If you have several copies of a file but all of them are too damaged to
repair them (@pxref{Repairing Files}), lziprecover can try to produce a repair them (@pxref{Repairing files}), lziprecover can try to produce a
correct file merging the good parts of the damaged copies. correct file merging the good parts of the damaged copies.
The merge may succeed even if some copies of the file have all the The merge may succeed even if some copies of the file have all the
@ -285,16 +290,33 @@ The merge will fail if the damaged areas overlap (at least one byte is
damaged in all copies), or are adjacent and the boundary can't be damaged in all copies), or are adjacent and the boundary can't be
determined, or if the copies have too many damaged areas. determined, or if the copies have too many damaged areas.
To give you an idea of its possibilities, when merging two copies each All the copies must have the same size. If some of them have been
truncated and are therefore smaller than they should, you can extend
them to the correct size with the following command before merging them
with the other copies:
@example
ddrescue --extend-outfile=<correct_size> small_file.lz extended_file.lz
@end example
If some of the copies have got garbage data at the end and are therefore
larger than they should, you can reduce their sizes to the correct value
with the following command before merging them with the other copies:
@example
ddrescue --size=<correct_size> large_file.lz reduced_file.lz
@end example
To give you an idea of its possibilities, when merging two copies, each
of them with one damaged area affecting 1 percent of the copy, the of them with one damaged area affecting 1 percent of the copy, the
probability of obtaining a correct file is about 98 percent. With three probability of obtaining a correct file is about 98 percent. With three
such copies the probability rises to 99.97 percent. For large files with such copies the probability rises to 99.97 percent. For large files (a
small errors, the probability approaches 100 percent even with only two few MB) with small errors (one sector damaged per copy), the probability
copies. approaches 100 percent even with only two copies.
@node File Format @node File format
@chapter File Format @chapter File format
@cindex file format @cindex file format
Perfection is reached, not when there is no longer anything to add, but Perfection is reached, not when there is no longer anything to add, but
@ -421,7 +443,7 @@ mv file_fixed.lz file.lz
@sp 1 @sp 1
@noindent @noindent
Example 6: Split the multi-member file @samp{file.lz} and write each Example 6: Split the multi-member file @samp{file.lz} and write each
member in its own @samp{recXXXXXfile.lz} file. Then use member in its own @samp{recXXXfile.lz} file. Then use
@w{@samp{lziprecover -t}} to test the integrity of the resulting files. @w{@samp{lziprecover -t}} to test the integrity of the resulting files.
@example @example
@ -453,13 +475,13 @@ ddrescue -b2048 /dev/cdrom cdimage2 logfile2
mount -t iso9660 -o loop,ro cdimage2 /mnt/cdimage mount -t iso9660 -o loop,ro cdimage2 /mnt/cdimage
cp /mnt/cdimage/backup.tar.lz rescued2.tar.lz cp /mnt/cdimage/backup.tar.lz rescued2.tar.lz
umount /mnt/cdimage umount /mnt/cdimage
lziprecover -m -v -o rescued.tar.lz rescued1.tar.lz rescued2.tar.lz lziprecover -m -v -o backup.tar.lz rescued1.tar.lz rescued2.tar.lz
@end example @end example
@sp 1 @sp 1
@noindent @noindent
Example 8: Recover the first volume of those created with the command Example 8: Recover the first volume of those created with the command
@w{@code{lzip -b 32MiB -S 650MB big_db}} from two copies, @w{@samp{lzip -b 32MiB -S 650MB big_db}} from two copies,
@samp{big_db1_00001.lz} and @samp{big_db2_00001.lz}, with member 07 @samp{big_db1_00001.lz} and @samp{big_db2_00001.lz}, with member 07
damaged in the first copy, member 18 damaged in the second copy, and damaged in the first copy, member 18 damaged in the second copy, and
member 12 damaged in both copies. The correct file produced is saved in member 12 damaged in both copies. The correct file produced is saved in
@ -471,8 +493,80 @@ lziprecover -m -v -o big_db_00001.lz big_db1_00001.lz big_db2_00001.lz
@end example @end example
@node Unzcrash
@chapter Testing the robustness of decompressors
@cindex unzcrash
The lziprecover package also includes unzcrash, a program written to
test robustness to decompression of corrupted data, inspired by
unzcrash.c from Julian Seward's bzip2. Type @samp{make unzcrash} in the
lziprecover source directory to build it.
Unzcrash reads the specified file and then repeatedly decompresses it,
increasing 256 times each byte of the compressed data, so as to test all
possible one-byte errors. This should not cause any invalid memory
accesses. If it does, please, report it as a bug.
Unzcrash really executes as a subprocess the shell command specified in
the first non-option argument, and then writes the file specified in the
second non-option argument to the standard input of the subprocess,
modifying the corresponding byte each time. Therefore you can use
unzcrash to test any decompressor (not only lzip), or even other decoder
programs with a suitable command line syntax.
The format for running unzcrash is:
@example
unzcrash [@var{options}] "lzip -tv" @var{filename}.lz
@end example
Unzcrash supports the following options:
@table @samp
@item -h
@itemx --help
Print an informative help message describing the options and exit.
@item -V
@itemx --version
Print the version number of lziprecover on the standard output and exit.
@item -b @var{range}
@itemx --bits=@var{range}
Test N-bit errors only, instead of testing all the 255 wrong values for
each byte. @samp{N-bit error} means any value differing from the
original value in N bit positions, not a value differing from the
original value in the bit position N.@*
The number of N-bit errors per byte (N = 1 to 8) is: 8 28 56 70 56 28 8 1@*
Examples of @var{range}: 1 1,2,3 1-4 1,3-5,8 1-3,5-8
@item -p @var{bytes}
@itemx --position=@var{bytes}
First byte position to test in the file. Defaults to 0.
@item -q
@itemx --quiet
Quiet operation. Suppress all messages.
@item -s @var{bytes}
@itemx --size=@var{bytes}
Number of byte positions to test. If not specified, the whole file is
tested.
@item -v
@itemx --verbose
Verbose mode.
@end table
Exit status: 0 for a normal exit, 1 for environmental problems (file not
found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or
invalid input file, 3 for an internal consistency error (eg, bug) which
caused unzcrash to panic.
@node Problems @node Problems
@chapter Reporting Bugs @chapter Reporting bugs
@cindex bugs @cindex bugs
@cindex getting help @cindex getting help
@ -486,8 +580,8 @@ If you find a bug in lziprecover, please send electronic mail to
find by running @w{@samp{lziprecover --version}}. find by running @w{@samp{lziprecover --version}}.
@node Concept Index @node Concept index
@unnumbered Concept Index @unnumbered Concept index
@printindex cp @printindex cp

2
file_index.cc
View file

@ -1,4 +1,4 @@
/* Lziprecover - Data recovery tool for lzipped files /* Lziprecover - Data recovery tool for lzip files
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify

2
file_index.h
View file

@ -1,4 +1,4 @@
/* Lziprecover - Data recovery tool for lzipped files /* Lziprecover - Data recovery tool for lzip files
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify

27
lzip.h
View file

@ -1,4 +1,4 @@
/* Lziprecover - Data recovery tool for lzipped files /* Lziprecover - Data recovery tool for lzip files
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
@ -30,10 +30,11 @@ public:
static const int next[states] = { 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 4, 5 }; static const int next[states] = { 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 4, 5 };
st = next[st]; st = next[st];
} }
void set_char1() { st -= ( st < 4 ) ? st : 3; } // for st < 7
void set_match() { st = ( ( st < 7 ) ? 7 : 10 ); } void set_char2() { st -= ( st < 10 ) ? 3 : 6; } // for st >= 7
void set_rep() { st = ( ( st < 7 ) ? 8 : 11 ); } void set_match() { st = ( st < 7 ) ? 7 : 10; }
void set_short_rep() { st = ( ( st < 7 ) ? 9 : 11 ); } void set_rep() { st = ( st < 7 ) ? 8 : 11; }
void set_short_rep() { st = ( st < 7 ) ? 9 : 11; }
}; };
@ -160,10 +161,11 @@ public:
uint32_t operator[]( const uint8_t byte ) const { return data[byte]; } uint32_t operator[]( const uint8_t byte ) const { return data[byte]; }
void update( uint32_t & crc, const uint8_t byte ) const void update_byte( uint32_t & crc, const uint8_t byte ) const
{ crc = data[(crc^byte)&0xFF] ^ ( crc >> 8 ); } { crc = data[(crc^byte)&0xFF] ^ ( crc >> 8 ); }
void update( uint32_t & crc, const uint8_t * const buffer, const int size ) const void update_buf( uint32_t & crc, const uint8_t * const buffer,
const int size ) const
{ {
for( int i = 0; i < size; ++i ) for( int i = 0; i < size; ++i )
crc = data[(crc^buffer[i])&0xFF] ^ ( crc >> 8 ); crc = data[(crc^buffer[i])&0xFF] ^ ( crc >> 8 );
@ -205,15 +207,15 @@ struct File_header
return sz; return sz;
} }
bool dictionary_size( const int sz ) bool dictionary_size( const unsigned sz )
{ {
if( sz >= min_dictionary_size && sz <= max_dictionary_size ) if( sz >= min_dictionary_size && sz <= max_dictionary_size )
{ {
data[5] = real_bits( sz - 1 ); data[5] = real_bits( sz - 1 );
if( sz > min_dictionary_size ) if( sz > min_dictionary_size )
{ {
const int base_size = 1 << data[5]; const unsigned base_size = 1 << data[5];
const int wedge = base_size / 16; const unsigned wedge = base_size / 16;
for( int i = 7; i >= 1; --i ) for( int i = 7; i >= 1; --i )
if( base_size - ( i * wedge ) >= sz ) if( base_size - ( i * wedge ) >= sz )
{ data[5] |= ( i << 5 ); break; } { data[5] |= ( i << 5 ); break; }
@ -287,9 +289,8 @@ int writeblock( const int fd, const uint8_t * const buf, const int size );
// defined in main.cc // defined in main.cc
int open_instream( const std::string & name, struct stat * const in_statsp, int open_instream( const std::string & name, struct stat * const in_statsp,
const bool to_stdout, const bool reg_only = false ); const bool no_ofile, const bool reg_only = false );
int open_outstream_rw( const std::string & output_filename, int open_outstream_rw( const std::string & output_filename, const bool force );
const bool force );
void show_header( const File_header & header ); void show_header( const File_header & header );
void show_error( const char * const msg, const int errcode = 0, void show_error( const char * const msg, const int errcode = 0,
const bool help = false ); const bool help = false );

23
main.cc
View file

@ -1,4 +1,4 @@
/* Lziprecover - Data recovery tool for lzipped files /* Lziprecover - Data recovery tool for lzip files
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
@ -80,7 +80,7 @@ struct { const char * from; const char * to; } const known_extensions[] = {
{ 0, 0 } }; { 0, 0 } };
enum Mode { m_none, m_decompress, m_generate, m_list, m_merge, m_range, enum Mode { m_none, m_decompress, m_generate, m_list, m_merge, m_range,
m_recover, m_repair, m_split, m_test, m_update }; m_recover, m_repair, m_split, m_test };
std::string output_filename; std::string output_filename;
int outfd = -1; int outfd = -1;
@ -93,7 +93,7 @@ bool delete_output_on_interrupt = false;
void show_help() void show_help()
{ {
std::printf( "%s - Data recovery tool and decompressor for lzipped files.\n", Program_name ); std::printf( "%s - Data recovery tool and decompressor for lzip files.\n", Program_name );
std::printf( "\nUsage: %s [options] [files]\n", invocation_name ); std::printf( "\nUsage: %s [options] [files]\n", invocation_name );
std::printf( "\nOptions:\n" std::printf( "\nOptions:\n"
" -h, --help display this help and exit\n" " -h, --help display this help and exit\n"
@ -149,8 +149,6 @@ void show_header( const File_header & header )
for( int i = 0; i < 8 && ( num > 9999 || ( exact && num >= factor ) ); ++i ) for( int i = 0; i < 8 && ( num > 9999 || ( exact && num >= factor ) ); ++i )
{ num /= factor; if( num % factor != 0 ) exact = false; { num /= factor; if( num % factor != 0 ) exact = false;
p = prefix[i]; np = ""; } p = prefix[i]; np = ""; }
if( verbosity >= 4 )
std::fprintf( stderr, "version %d, ", header.version() );
std::fprintf( stderr, "dictionary size %s%4u %sB. ", np, num, p ); std::fprintf( stderr, "dictionary size %s%4u %sB. ", np, num, p );
} }
@ -192,7 +190,7 @@ int extension_index( const std::string & name )
} // end namespace } // end namespace
int open_instream( const std::string & name, struct stat * const in_statsp, int open_instream( const std::string & name, struct stat * const in_statsp,
const bool to_stdout, const bool reg_only ) const bool no_ofile, const bool reg_only )
{ {
int infd = open( name.c_str(), O_RDONLY | o_binary ); int infd = open( name.c_str(), O_RDONLY | o_binary );
if( infd < 0 ) if( infd < 0 )
@ -208,12 +206,12 @@ int open_instream( const std::string & name, struct stat * const in_statsp,
const bool can_read = ( i == 0 && !reg_only && const bool can_read = ( i == 0 && !reg_only &&
( S_ISBLK( mode ) || S_ISCHR( mode ) || ( S_ISBLK( mode ) || S_ISCHR( mode ) ||
S_ISFIFO( mode ) || S_ISSOCK( mode ) ) ); S_ISFIFO( mode ) || S_ISSOCK( mode ) ) );
if( i != 0 || ( !S_ISREG( mode ) && ( !to_stdout || !can_read ) ) ) if( i != 0 || ( !S_ISREG( mode ) && ( !can_read || !no_ofile ) ) )
{ {
if( verbosity >= 0 ) if( verbosity >= 0 )
std::fprintf( stderr, "%s: Input file '%s' is not a regular file%s.\n", std::fprintf( stderr, "%s: Input file '%s' is not a regular file%s.\n",
program_name, name.c_str(), program_name, name.c_str(),
( can_read && !to_stdout ) ? ( can_read && !no_ofile ) ?
" and '--stdout' was not specified" : "" ); " and '--stdout' was not specified" : "" );
close( infd ); close( infd );
infd = -1; infd = -1;
@ -377,7 +375,7 @@ int decompress( const int infd, const Pretty_print & pp, const bool testing )
if( rdec.finished() ) // End Of File if( rdec.finished() ) // End Of File
{ {
if( first_member ) if( first_member )
{ pp( "Error reading member header" ); retval = 1; } { pp( "File ends unexpectedly at member header" ); retval = 2; }
else if( verbosity >= 4 && size > 0 ) else if( verbosity >= 4 && size > 0 )
show_trailing_garbage( header.data, size, pp, true ); show_trailing_garbage( header.data, size, pp, true );
break; break;
@ -455,8 +453,7 @@ void set_signals()
} // end namespace } // end namespace
int open_outstream_rw( const std::string & output_filename, int open_outstream_rw( const std::string & output_filename, const bool force )
const bool force )
{ {
int flags = O_CREAT | O_RDWR | o_binary; int flags = O_CREAT | O_RDWR | o_binary;
if( force ) flags |= O_TRUNC; else flags |= O_EXCL; if( force ) flags |= O_TRUNC; else flags |= O_EXCL;
@ -593,7 +590,6 @@ int main( const int argc, const char * const argv[] )
{ {
case m_generate: case m_generate:
case m_recover: case m_recover:
case m_update:
case m_none: internal_error( "invalid operation" ); break; case m_none: internal_error( "invalid operation" ); break;
case m_decompress: break; case m_decompress: break;
case m_list: case m_list:
@ -663,7 +659,8 @@ int main( const int argc, const char * const argv[] )
else else
{ {
input_filename = filenames[i]; input_filename = filenames[i];
infd = open_instream( input_filename, &in_stats, to_stdout ); infd = open_instream( input_filename, &in_stats,
to_stdout || program_mode == m_test );
if( infd < 0 ) { if( retval < 1 ) retval = 1; continue; } if( infd < 0 ) { if( retval < 1 ) retval = 1; continue; }
if( program_mode != m_test ) if( program_mode != m_test )
{ {

2
merge.cc
View file

@ -1,4 +1,4 @@
/* Lziprecover - Data recovery tool for lzipped files /* Lziprecover - Data recovery tool for lzip files
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify

4
range_dec.cc
View file

@ -1,4 +1,4 @@
/* Lziprecover - Data recovery tool for lzipped files /* Lziprecover - Data recovery tool for lzip files
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
@ -112,7 +112,7 @@ int decompress_member( const int infd, const int outfd,
File_header header; File_header header;
rdec.read_data( header.data, File_header::size ); rdec.read_data( header.data, File_header::size );
if( rdec.finished() ) // End Of File if( rdec.finished() ) // End Of File
{ pp( "Error reading member header" ); return 1; } { pp( "File ends unexpectedly at member header" ); return 2; }
if( !header.verify_magic() ) if( !header.verify_magic() )
{ pp( "Bad magic number (file not in lzip format)" ); return 2; } { pp( "Bad magic number (file not in lzip format)" ); return 2; }
if( !header.verify_version() ) if( !header.verify_version() )

2
repair.cc
View file

@ -1,4 +1,4 @@
/* Lziprecover - Data recovery tool for lzipped files /* Lziprecover - Data recovery tool for lzip files
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify

2
split.cc
View file

@ -1,4 +1,4 @@
/* Lziprecover - Data recovery tool for lzipped files /* Lziprecover - Data recovery tool for lzip files
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify

32
testsuite/check.sh
View file

@ -1,5 +1,5 @@
#! /bin/sh #! /bin/sh
# check script for Lziprecover - Data recovery tool for lzipped files # check script for Lziprecover - Data recovery tool for lzip files
# Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. # Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
# #
# This script is free software: you have unlimited permission # This script is free software: you have unlimited permission
@ -10,7 +10,7 @@ export LC_ALL
objdir=`pwd` objdir=`pwd`
testdir=`cd "$1" ; pwd` testdir=`cd "$1" ; pwd`
LZIP="${objdir}"/lziprecover LZIP="${objdir}"/lziprecover
LZIPRECOVER="${objdir}"/lziprecover LZIPRECOVER="${LZIP}"
framework_failure() { echo "failure in testing framework" ; exit 1 ; } framework_failure() { echo "failure in testing framework" ; exit 1 ; }
if [ ! -x "${LZIP}" ] ; then if [ ! -x "${LZIP}" ] ; then
@ -65,9 +65,21 @@ if [ $? = 1 ] ; then printf . ; else fail=1 ; printf - ; fi
if [ $? = 1 ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 1 ] ; then printf . ; else fail=1 ; printf - ; fi
"${LZIPRECOVER}" -sq "${LZIPRECOVER}" -sq
if [ $? = 1 ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 1 ] ; then printf . ; else fail=1 ; printf - ; fi
"${LZIP}" -tq "${in}"
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi
"${LZIP}" -tq < "${in}"
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi
"${LZIP}" -cdq "${in}"
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi
"${LZIP}" -cdq < "${in}"
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi
dd if="${in_lz}" bs=1 count=6 2> /dev/null | "${LZIP}" -tq
if [ $? = 2 ] ; then printf . ; else printf - ; fail=1 ; fi
dd if="${in_lz}" bs=1 count=20 2> /dev/null | "${LZIP}" -tq
if [ $? = 2 ] ; then printf . ; else printf - ; fail=1 ; fi
"${LZIP}" -t "${testdir}"/test.txt.lz || fail=1 "${LZIP}" -t "${in_lz}" || fail=1
"${LZIP}" -cd "${testdir}"/test.txt.lz > copy || fail=1 "${LZIP}" -cd "${in_lz}" > copy || fail=1
cmp "${in}" copy || fail=1 cmp "${in}" copy || fail=1
printf . printf .
@ -100,8 +112,8 @@ for i in "${f5b1_lz}" "${f5b3_lz}" "${f5b4_lz}" "${f5b5_lz}" ; do
cmp "${fox5_lz}" copy.lz || fail=1 cmp "${fox5_lz}" copy.lz || fail=1
"${LZIPRECOVER}" -mf -o copy.lz "${i}" "${f5b2_lz}" || fail=1 "${LZIPRECOVER}" -mf -o copy.lz "${i}" "${f5b2_lz}" || fail=1
cmp "${fox5_lz}" copy.lz || fail=1 cmp "${fox5_lz}" copy.lz || fail=1
printf .
done done
printf .
for i in "${f5b3_lz}" "${f5b4_lz}" "${f5b5_lz}" ; do for i in "${f5b3_lz}" "${f5b4_lz}" "${f5b5_lz}" ; do
"${LZIPRECOVER}" -mf -o copy.lz "${f5b1_lz}" "${f5b2_lz}" "${i}" || fail=1 "${LZIPRECOVER}" -mf -o copy.lz "${f5b1_lz}" "${f5b2_lz}" "${i}" || fail=1
@ -116,8 +128,8 @@ for i in "${f5b3_lz}" "${f5b4_lz}" "${f5b5_lz}" ; do
cmp "${fox5_lz}" copy.lz || fail=1 cmp "${fox5_lz}" copy.lz || fail=1
"${LZIPRECOVER}" -mf -o copy.lz "${i}" "${f5b2_lz}" "${f5b1_lz}" || fail=1 "${LZIPRECOVER}" -mf -o copy.lz "${i}" "${f5b2_lz}" "${f5b1_lz}" || fail=1
cmp "${fox5_lz}" copy.lz || fail=1 cmp "${fox5_lz}" copy.lz || fail=1
printf .
done done
printf .
"${LZIPRECOVER}" -mf -o copy.lz "${f5b3_lz}" "${f5b4_lz}" "${f5b5_lz}" || fail=1 "${LZIPRECOVER}" -mf -o copy.lz "${f5b3_lz}" "${f5b4_lz}" "${f5b5_lz}" || fail=1
cmp "${fox5_lz}" copy.lz || fail=1 cmp "${fox5_lz}" copy.lz || fail=1
@ -141,9 +153,9 @@ for i in "${bad1_lz}" "${bad2_lz}" ; do
cmp "${in_lz}" copy.lz || fail=1 cmp "${in_lz}" copy.lz || fail=1
"${LZIPRECOVER}" -mf -o copy.lz "${j}" "${i}" || fail=1 "${LZIPRECOVER}" -mf -o copy.lz "${j}" "${i}" || fail=1
cmp "${in_lz}" copy.lz || fail=1 cmp "${in_lz}" copy.lz || fail=1
printf .
done done
done done
printf .
"${LZIPRECOVER}" -mf -o copy.lz "${bad3_lz}" "${bad4_lz}" "${bad5_lz}" || fail=1 "${LZIPRECOVER}" -mf -o copy.lz "${bad3_lz}" "${bad4_lz}" "${bad5_lz}" || fail=1
cmp "${in_lz}" copy.lz || fail=1 cmp "${in_lz}" copy.lz || fail=1
@ -171,7 +183,6 @@ if [ $? = 0 ] && [ ! -e copy.lz ] ; then printf . ; else fail=1 ; printf - ; fi
if [ $? = 2 ] && [ ! -e copy.lz ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 2 ] && [ ! -e copy.lz ] ; then printf . ; else fail=1 ; printf - ; fi
"${LZIPRECOVER}" -Rf -o copy.lz "${f5b1_lz}" || fail=1 "${LZIPRECOVER}" -Rf -o copy.lz "${f5b1_lz}" || fail=1
cmp "${fox5_lz}" copy.lz || fail=1 cmp "${fox5_lz}" copy.lz || fail=1
printf .
"${LZIPRECOVER}" -Rf -o copy.lz "${bad1_lz}" || fail=1 "${LZIPRECOVER}" -Rf -o copy.lz "${bad1_lz}" || fail=1
cmp "${in_lz}" copy.lz || fail=1 cmp "${in_lz}" copy.lz || fail=1
printf . printf .
@ -182,8 +193,8 @@ printf "garbage" >> copy || fail=1
for i in 1 2 3 ; do for i in 1 2 3 ; do
"${LZIPRECOVER}" -cd rec${i}copy.lz > copy || fail=1 "${LZIPRECOVER}" -cd rec${i}copy.lz > copy || fail=1
cmp "${in}" copy || fail=1 cmp "${in}" copy || fail=1
printf .
done done
printf .
cat "${in_lz}" > anyothername || framework_failure cat "${in_lz}" > anyothername || framework_failure
"${LZIP}" -d anyothername || fail=1 "${LZIP}" -d anyothername || fail=1
@ -191,8 +202,7 @@ cmp "${in}" anyothername.out || fail=1
printf . printf .
cat "${in}" "${in}" > in2 || framework_failure cat "${in}" "${in}" > in2 || framework_failure
cat "${in_lz}" "${in_lz}" > out2 || framework_failure cat "${in_lz}" "${in_lz}" | "${LZIP}" -d > copy2 || fail=1
"${LZIP}" -d < out2 > copy2 || fail=1
cmp in2 copy2 || fail=1 cmp in2 copy2 || fail=1
printf . printf .

28
testsuite/unzcrash.cc
View file

@ -1,5 +1,4 @@
/* Unzcrash - A test program written to test robustness to /* Unzcrash - Tests robustness of decompressors to corrupted data.
decompression of corrupted data.
Inspired by unzcrash.c from Julian Seward's bzip2. Inspired by unzcrash.c from Julian Seward's bzip2.
Copyright (C) 2008, 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz. Copyright (C) 2008, 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
@ -16,6 +15,12 @@
You should have received a copy of the GNU General Public License You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>. along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
/*
Exit status: 0 for a normal exit, 1 for environmental problems
(file not found, invalid flags, I/O errors, etc), 2 to indicate a
corrupt or invalid input file, 3 for an internal consistency error
(eg, bug) which caused unzcrash to panic.
*/
#include <cerrno> #include <cerrno>
#include <climits> #include <climits>
@ -47,9 +52,8 @@ int verbosity = 0;
void show_help() void show_help()
{ {
std::printf( "%s - A test program written to test robustness to\n", Program_name ); std::printf( "%s - Tests robustness of decompressors to corrupted data.\n", Program_name );
std::printf( "decompression of corrupted data.\n" std::printf( "\nUsage: %s [options] \"lzip -tv\" filename.lz\n", invocation_name );
"\nUsage: %s [options] \"lzip -tv\" filename.lz\n", invocation_name );
std::printf( "\nThis program reads the specified file and then repeatedly decompresses\n" std::printf( "\nThis program reads the specified file and then repeatedly decompresses\n"
"it, increasing 256 times each byte of the compressed data, so as to test\n" "it, increasing 256 times each byte of the compressed data, so as to test\n"
"all possible one-byte errors. This should not cause any invalid memory\n" "all possible one-byte errors. This should not cause any invalid memory\n"
@ -63,8 +67,12 @@ void show_help()
" -s, --size=<bytes> number of byte positions to test [all]\n" " -s, --size=<bytes> number of byte positions to test [all]\n"
" -v, --verbose be verbose (a 2nd -v gives more)\n" " -v, --verbose be verbose (a 2nd -v gives more)\n"
"Examples of <range>: 1 1,2,3 1-4 1,3-5,8 1-3,5-8\n" "Examples of <range>: 1 1,2,3 1-4 1,3-5,8 1-3,5-8\n"
"\nExit status: 0 for a normal exit, 1 for environmental problems (file\n"
"not found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or\n"
"invalid input file, 3 for an internal consistency error (eg, bug) which\n"
"caused unzcrash to panic.\n"
"\nReport bugs to lzip-bug@nongnu.org\n" "\nReport bugs to lzip-bug@nongnu.org\n"
"Lzip home page: http://www.nongnu.org/lzip/lzip.html\n" ); "Lziprecover home page: http://www.nongnu.org/lzip/lziprecover.html\n" );
} }
@ -160,7 +168,7 @@ unsigned long long getnum( const char * const ptr,
} }
class Bitset8 // 8 value bitset (1..8) class Bitset8 // 8 value bitset (1 to 8)
{ {
bool data[8]; bool data[8];
static bool valid_digit( const unsigned char ch ) static bool valid_digit( const unsigned char ch )
@ -195,7 +203,7 @@ public:
return false; return false;
} }
// number of n-bit errors per byte (n=0..8): 1 8 28 56 70 56 28 8 1 // number of N-bit errors per byte (N=0 to 8): 1 8 28 56 70 56 28 8 1
void print() const void print() const
{ {
std::fflush( stderr ); std::fflush( stderr );
@ -295,9 +303,9 @@ int main( const int argc, const char * const argv[] )
if( size >= buffer_size ) if( size >= buffer_size )
{ {
if( verbosity >= 0 ) if( verbosity >= 0 )
std::fprintf( stderr, "input file '%s' is too big.\n", std::fprintf( stderr, "input file '%s' is too large.\n",
parser.argument( argind + 1 ).c_str() ); parser.argument( argind + 1 ).c_str() );
return 1; return 2;
} }
std::fclose( f ); std::fclose( f );