Adding upstream version 1.15~rc1.
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
parent
0ff20a5602
commit
6a117924f6
GPG key ID:
FBB4F0E80A80222F
20 changed files with 439 additions and 219 deletions
|
|
@ -1,10 +1,16 @@
|
|||
2013-07-29 Antonio Diaz Diaz <antonio@gnu.org>
|
||||
|
||||
* Version 1.15-rc1 released.
|
||||
* Minor changes.
|
||||
* Added chapter 'Unzcrash' to the manual.
|
||||
|
||||
2013-06-17 Antonio Diaz Diaz <antonio@gnu.org>
|
||||
|
||||
* Version 1.15-pre1 released.
|
||||
* repair.cc: Repair multi-member files with up to one byte error
|
||||
per member.
|
||||
* merge.cc: Merge multi-member files.
|
||||
* Added chapters 'Repairing Files' and 'Merging Files' to the manual.
|
||||
* Added chapters 'Repairing files' and 'Merging files' to the manual.
|
||||
|
||||
2013-05-31 Antonio Diaz Diaz <antonio@gnu.org>
|
||||
|
||||
|
|
|
|||
4
INSTALL
4
INSTALL
|
|
@ -10,9 +10,9 @@ Procedure
|
|||
---------
|
||||
1. Unpack the archive if you have not done so already:
|
||||
|
||||
lzip -cd lziprecover[version].tar.lz | tar -xf -
|
||||
tar -xf lziprecover[version].tar.lz
|
||||
or
|
||||
gzip -cd lziprecover[version].tar.gz | tar -xf -
|
||||
lzip -cd lziprecover[version].tar.lz | tar -xf -
|
||||
|
||||
This creates the directory ./lziprecover[version] containing the source
|
||||
from the main archive.
|
||||
|
|
|
|||
4
NEWS
4
NEWS
|
|
@ -6,5 +6,5 @@ per member, without having to split them first.
|
|||
Lziprecover can now merge multi-member files without having to split
|
||||
them first even if some copies have the header and the trailer damaged.
|
||||
|
||||
The chapters "Repairing Files" and "Merging Files" have been added to
|
||||
the manual.
|
||||
The chapters "Repairing files", "Merging files" and "Unzcrash" have been
|
||||
added to the manual.
|
||||
|
|
|
|||
51
README
51
README
|
|
@ -1,16 +1,29 @@
|
|||
Description
|
||||
|
||||
Lziprecover is a data recovery tool and decompressor for files in the
|
||||
lzip compressed data format (.lz) able to repair slightly damaged files,
|
||||
recover badly damaged files from two or more copies, extract data from
|
||||
damaged files, decompress files and test integrity of files.
|
||||
lzip compressed data format (.lz), able to repair slightly damaged
|
||||
files, recover badly damaged files from two or more copies, extract data
|
||||
from damaged files, decompress files and test integrity of files.
|
||||
|
||||
The lzip file format is designed for long-term data archiving. It is
|
||||
clean, provides very safe 4 factor integrity checking, and is backed by
|
||||
the recovery capabilities of lziprecover.
|
||||
|
||||
Lziprecover is able to recover or decompress files produced by any of
|
||||
the compressors in the lzip family; lzip, plzip, minilzip/lzlib, clzip
|
||||
and pdlzip. It makes lzip files resistant to bit-flip, one of the most
|
||||
common forms of data corruption, and its recovery capabilities
|
||||
contribute to make of the lzip format one of the best options for
|
||||
long-term data archiving.
|
||||
and pdlzip.
|
||||
|
||||
Lziprecover makes lzip files resistant to bit-flip (one of the most
|
||||
common forms of data corruption), and can safely merge multiple damaged
|
||||
backup copies.
|
||||
|
||||
If the cause of file corruption is damaged media, the combination
|
||||
GNU ddrescue + lziprecover is the best option for recovering data from
|
||||
multiple damaged copies.
|
||||
|
||||
If a file is too damaged for lziprecover to repair it, all the
|
||||
recoverable data in all members of the file can be extracted with the
|
||||
'-D' option.
|
||||
|
||||
Lziprecover is able to efficiently extract a range of bytes from a
|
||||
multi-member file, because it only decompresses the members containing
|
||||
|
|
@ -27,24 +40,20 @@ damaged files themselves are never modified.
|
|||
When decompressing or testing file integrity, lziprecover behaves like
|
||||
lzip or lunzip.
|
||||
|
||||
If the files are too damaged for lziprecover to repair them, data from
|
||||
damaged members can be partially recovered writing it to stdout.
|
||||
|
||||
To give you an idea of its possibilities, when merging two copies each
|
||||
To give you an idea of its possibilities, when merging two copies, each
|
||||
of them with one damaged area affecting 1 percent of the copy, the
|
||||
probability of obtaining a correct file is about 98 percent. With three
|
||||
such copies the probability rises to 99.97 percent. For large files with
|
||||
small errors, the probability approaches 100 percent even with only two
|
||||
copies.
|
||||
such copies the probability rises to 99.97 percent. For large files (a
|
||||
few MB) with small errors (one sector damaged per copy), the probability
|
||||
approaches 100 percent even with only two copies.
|
||||
|
||||
If the cause of file corruption is damaged media, the combination
|
||||
GNU ddrescue + lziprecover is the best option for recovering data from
|
||||
multiple damaged copies.
|
||||
Lziprecover is not a replacement for regular backups, but a last line of
|
||||
defense for the case where the backups are also damaged.
|
||||
|
||||
This package also includes unzcrash, a program written to test
|
||||
robustness to decompression of corrupted data, inspired by unzcrash.c
|
||||
from Julian Seward's bzip2. Type 'make unzcrash' in the lziprecover
|
||||
directory to build it. Then try 'unzcrash --help'.
|
||||
The lziprecover package also includes unzcrash, a program written to
|
||||
test robustness to decompression of corrupted data, inspired by
|
||||
unzcrash.c from Julian Seward's bzip2. Type 'make unzcrash' in the
|
||||
lziprecover source directory to build it. Then try 'unzcrash --help'.
|
||||
|
||||
|
||||
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
|
|
|
|||
8
configure
vendored
8
configure
vendored
|
|
@ -1,14 +1,14 @@
|
|||
#! /bin/sh
|
||||
# configure script for Lziprecover - Data recovery tool for lzipped files
|
||||
# configure script for Lziprecover - Data recovery tool for lzip files
|
||||
# Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
#
|
||||
# This configure script is free software: you have unlimited permission
|
||||
# to copy, distribute and modify it.
|
||||
|
||||
pkgname=lziprecover
|
||||
pkgversion=1.15-pre1
|
||||
pkgversion=1.15-rc1
|
||||
progname=lziprecover
|
||||
srctrigger=doc/lziprecover.texinfo
|
||||
srctrigger=doc/${pkgname}.texinfo
|
||||
|
||||
# clear some things potentially inherited from environment.
|
||||
LC_ALL=C
|
||||
|
|
@ -164,7 +164,7 @@ echo "CXXFLAGS = ${CXXFLAGS}"
|
|||
echo "LDFLAGS = ${LDFLAGS}"
|
||||
rm -f Makefile
|
||||
cat > Makefile << EOF
|
||||
# Makefile for Lziprecover - Data recovery tool for lzipped files
|
||||
# Makefile for Lziprecover - Data recovery tool for lzip files
|
||||
# Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
# This file was generated automatically by configure. Do not edit.
|
||||
#
|
||||
|
|
|
|||
16
decoder.cc
16
decoder.cc
|
|
@ -1,4 +1,4 @@
|
|||
/* Lziprecover - Data recovery tool for lzipped files
|
||||
/* Lziprecover - Data recovery tool for lzip files
|
||||
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
|
|
@ -104,10 +104,10 @@ bool Range_decoder::read_block()
|
|||
|
||||
void LZ_decoder::flush_data()
|
||||
{
|
||||
const int size = pos - stream_pos;
|
||||
if( size > 0 )
|
||||
if( pos > stream_pos )
|
||||
{
|
||||
crc32.update( crc_, buffer + stream_pos, size );
|
||||
const int size = pos - stream_pos;
|
||||
crc32.update_buf( crc_, buffer + stream_pos, size );
|
||||
if( outfd >= 0 )
|
||||
{
|
||||
const unsigned long long sp = stream_position();
|
||||
|
|
@ -207,7 +207,6 @@ int LZ_decoder::decode_member( const Pretty_print & pp )
|
|||
Bit_model bm_align[dis_align_size];
|
||||
Len_model match_len_model;
|
||||
Len_model rep_len_model;
|
||||
|
||||
unsigned rep0 = 0; // rep[0-3] latest four distances
|
||||
unsigned rep1 = 0; // used for efficient coding of
|
||||
unsigned rep2 = 0; // repeated distances
|
||||
|
|
@ -222,11 +221,16 @@ int LZ_decoder::decode_member( const Pretty_print & pp )
|
|||
{
|
||||
const uint8_t prev_byte = get_prev_byte();
|
||||
if( state.is_char() )
|
||||
{
|
||||
state.set_char1();
|
||||
put_byte( rdec.decode_tree( bm_literal[get_lit_state(prev_byte)], 8 ) );
|
||||
}
|
||||
else
|
||||
{
|
||||
state.set_char2();
|
||||
put_byte( rdec.decode_matched( bm_literal[get_lit_state(prev_byte)],
|
||||
get_byte( rep0 ) ) );
|
||||
state.set_char();
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
/* Lziprecover - Data recovery tool for lzipped files
|
||||
/* Lziprecover - Data recovery tool for lzip files
|
||||
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
|
|
@ -278,7 +278,6 @@ public:
|
|||
~LZ_decoder() { delete[] buffer; }
|
||||
|
||||
unsigned crc() const { return crc_ ^ 0xFFFFFFFFU; }
|
||||
|
||||
unsigned long long data_position() const { return partial_data_pos + pos; }
|
||||
|
||||
int decode_member( const Pretty_print & pp );
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.37.1.
|
||||
.TH LZIPRECOVER "1" "June 2013" "Lziprecover 1.15-pre1" "User Commands"
|
||||
.TH LZIPRECOVER "1" "July 2013" "Lziprecover 1.15-rc1" "User Commands"
|
||||
.SH NAME
|
||||
Lziprecover \- recovers data from damaged lzip files
|
||||
.SH SYNOPSIS
|
||||
.B lziprecover
|
||||
[\fIoptions\fR] [\fIfiles\fR]
|
||||
.SH DESCRIPTION
|
||||
Lziprecover \- Data recovery tool and decompressor for lzipped files.
|
||||
Lziprecover \- Data recovery tool and decompressor for lzip files.
|
||||
.SH OPTIONS
|
||||
.TP
|
||||
\fB\-h\fR, \fB\-\-help\fR
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ lziprecover.texinfo.
|
|||
|
||||
INFO-DIR-SECTION Data Compression
|
||||
START-INFO-DIR-ENTRY
|
||||
* Lziprecover: (lziprecover). Data recovery tool for lzipped files
|
||||
* Lziprecover: (lziprecover). Data recovery tool for lzip files
|
||||
END-INFO-DIR-ENTRY
|
||||
|
||||
|
||||
|
|
@ -12,18 +12,19 @@ File: lziprecover.info, Node: Top, Next: Introduction, Up: (dir)
|
|||
Lziprecover Manual
|
||||
******************
|
||||
|
||||
This manual is for Lziprecover (version 1.15-pre1, 17 June 2013).
|
||||
This manual is for Lziprecover (version 1.15-rc1, 29 July 2013).
|
||||
|
||||
* Menu:
|
||||
|
||||
* Introduction:: Purpose and features of lziprecover
|
||||
* Invoking Lziprecover:: Command line interface
|
||||
* Repairing Files:: Fixing bit-flip and similar errors
|
||||
* Merging Files:: Fixing several damaged copies
|
||||
* File Format:: Detailed format of the compressed file
|
||||
* Invoking lziprecover:: Command line interface
|
||||
* Repairing files:: Fixing bit-flip and similar errors
|
||||
* Merging files:: Fixing several damaged copies
|
||||
* File format:: Detailed format of the compressed file
|
||||
* Examples:: A small tutorial with examples
|
||||
* Unzcrash:: Testing the robustness of decompressors
|
||||
* Problems:: Reporting bugs
|
||||
* Concept Index:: Index of concepts
|
||||
* Concept index:: Index of concepts
|
||||
|
||||
|
||||
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
|
|
@ -32,22 +33,38 @@ This manual is for Lziprecover (version 1.15-pre1, 17 June 2013).
|
|||
copy, distribute and modify it.
|
||||
|
||||
|
||||
File: lziprecover.info, Node: Introduction, Next: Invoking Lziprecover, Prev: Top, Up: Top
|
||||
File: lziprecover.info, Node: Introduction, Next: Invoking lziprecover, Prev: Top, Up: Top
|
||||
|
||||
1 Introduction
|
||||
**************
|
||||
|
||||
Lziprecover is a data recovery tool and decompressor for files in the
|
||||
lzip compressed data format (.lz) able to repair slightly damaged files,
|
||||
recover badly damaged files from two or more copies, extract data from
|
||||
damaged files, decompress files and test integrity of files.
|
||||
lzip compressed data format (.lz), able to repair slightly damaged
|
||||
files, recover badly damaged files from two or more copies, extract data
|
||||
from damaged files, decompress files and test integrity of files.
|
||||
|
||||
The lzip file format is designed for long-term data archiving. It is
|
||||
clean, provides very safe 4 factor integrity checking, and is backed by
|
||||
the recovery capabilities of lziprecover.
|
||||
|
||||
Lziprecover is able to recover or decompress files produced by any of
|
||||
the compressors in the lzip family; lzip, plzip, minilzip/lzlib, clzip
|
||||
and pdlzip. It makes lzip files resistant to bit-flip, one of the most
|
||||
common forms of data corruption, and its recovery capabilities
|
||||
contribute to make of the lzip format one of the best options for
|
||||
long-term data archiving.
|
||||
and pdlzip.
|
||||
|
||||
Lziprecover makes lzip files resistant to bit-flip (one of the most
|
||||
common forms of data corruption), and can safely merge multiple damaged
|
||||
backup copies.
|
||||
|
||||
If the cause of file corruption is damaged media, the combination
|
||||
GNU ddrescue + lziprecover is the best option for recovering data from
|
||||
multiple damaged copies. *Note ddrescue-example::, for an example.
|
||||
|
||||
If a file is too damaged for lziprecover to repair it, all the
|
||||
recoverable data in all members of the file can be extracted with the
|
||||
following command (the resulting file may contain errors and some
|
||||
garbage data may be produced at the end of each member):
|
||||
|
||||
lziprecover -D0 -i -o file -q file.lz
|
||||
|
||||
Lziprecover is able to efficiently extract a range of bytes from a
|
||||
multi-member file, because it only decompresses the members containing
|
||||
|
|
@ -64,24 +81,13 @@ damaged files themselves are never modified.
|
|||
When decompressing or testing file integrity, lziprecover behaves
|
||||
like lzip or lunzip.
|
||||
|
||||
If a file is too damaged for lziprecover to repair it, all the
|
||||
recoverable data in all members of the file can be extracted with the
|
||||
following command (the resulting file may contain errors and some
|
||||
garbage data may be produced at the end of each member):
|
||||
|
||||
lziprecover -D0 -i -o file -q file.lz
|
||||
|
||||
If the cause of file corruption is damaged media, the combination
|
||||
GNU ddrescue + lziprecover is the best option for recovering data from
|
||||
multiple damaged copies. *Note ddrescue-example::, for an example.
|
||||
|
||||
Lziprecover is not a replacement for regular backups, but a last
|
||||
line of defense for the case where the backups are also damaged.
|
||||
|
||||
|
||||
File: lziprecover.info, Node: Invoking Lziprecover, Next: Repairing Files, Prev: Introduction, Up: Top
|
||||
File: lziprecover.info, Node: Invoking lziprecover, Next: Repairing files, Prev: Introduction, Up: Top
|
||||
|
||||
2 Invoking Lziprecover
|
||||
2 Invoking lziprecover
|
||||
**********************
|
||||
|
||||
The format for running lziprecover is:
|
||||
|
|
@ -149,8 +155,8 @@ The format for running lziprecover is:
|
|||
Try to produce a correct file merging the good parts of two or more
|
||||
damaged copies. If successful, a repaired copy is written to the
|
||||
file `FILE_fixed.lz'. The exit status is 0 if a correct file could
|
||||
be produced, 2 otherwise. See the chapter Merging Files (*note
|
||||
Merging Files::) for a complete description of the merge mode.
|
||||
be produced, 2 otherwise. See the chapter `Merging files' (*note
|
||||
Merging files::) for a complete description of the merge mode.
|
||||
|
||||
`-o FILE'
|
||||
`--output=FILE'
|
||||
|
|
@ -167,10 +173,10 @@ The format for running lziprecover is:
|
|||
`-R'
|
||||
`--repair'
|
||||
Try to repair a file with small errors (up to one byte error per
|
||||
member). If successful, a repaired copy is written to the file
|
||||
member). If successful, a repaired copy is written to the file
|
||||
`FILE_fixed.lz'. `FILE' is not modified at all. The exit status
|
||||
is 0 if the file could be repaired, 2 otherwise. See the chapter
|
||||
Repairing Files (*note Repairing Files::) for a complete
|
||||
is 0 if the file could be repaired, 2 otherwise. See the chapter
|
||||
`Repairing files' (*note Repairing files::) for a complete
|
||||
description of the repair mode.
|
||||
|
||||
`-s'
|
||||
|
|
@ -199,12 +205,11 @@ The format for running lziprecover is:
|
|||
`--verbose'
|
||||
Verbose mode.
|
||||
When decompressing or testing, further -v's (up to 4) increase the
|
||||
verbosity level, showing status, dictionary size, compression
|
||||
ratio, trailer contents (CRC, data size, member size), and up to 6
|
||||
verbosity level, showing status, compression ratio, dictionary
|
||||
size, trailer contents (CRC, data size, member size), and up to 6
|
||||
bytes of trailing garbage (if any).
|
||||
|
||||
|
||||
|
||||
Numbers given as arguments to options may be followed by a multiplier
|
||||
and an optional `B' for "byte".
|
||||
|
||||
|
|
@ -227,9 +232,9 @@ invalid input file, 3 for an internal consistency error (eg, bug) which
|
|||
caused lziprecover to panic.
|
||||
|
||||
|
||||
File: lziprecover.info, Node: Repairing Files, Next: Merging Files, Prev: Invoking Lziprecover, Up: Top
|
||||
File: lziprecover.info, Node: Repairing files, Next: Merging files, Prev: Invoking lziprecover, Up: Top
|
||||
|
||||
3 Repairing Files
|
||||
3 Repairing files
|
||||
*****************
|
||||
|
||||
Lziprecover is able to repair files with small errors (up to one byte
|
||||
|
|
@ -244,13 +249,13 @@ vice versa. It may be caused by bad RAM or even by natural radiation. I
|
|||
have seen a case of bit-flip in a file stored in an USB flash drive.
|
||||
|
||||
|
||||
File: lziprecover.info, Node: Merging Files, Next: File Format, Prev: Repairing Files, Up: Top
|
||||
File: lziprecover.info, Node: Merging files, Next: File format, Prev: Repairing files, Up: Top
|
||||
|
||||
4 Merging Files
|
||||
4 Merging files
|
||||
***************
|
||||
|
||||
If you have several copies of a file but all of them are too damaged to
|
||||
repair them (*note Repairing Files::), lziprecover can try to produce a
|
||||
repair them (*note Repairing files::), lziprecover can try to produce a
|
||||
correct file merging the good parts of the damaged copies.
|
||||
|
||||
The merge may succeed even if some copies of the file have all the
|
||||
|
|
@ -262,17 +267,31 @@ the file.
|
|||
is damaged in all copies), or are adjacent and the boundary can't be
|
||||
determined, or if the copies have too many damaged areas.
|
||||
|
||||
To give you an idea of its possibilities, when merging two copies
|
||||
All the copies must have the same size. If some of them have been
|
||||
truncated and are therefore smaller than they should, you can extend
|
||||
them to the correct size with the following command before merging them
|
||||
with the other copies:
|
||||
|
||||
ddrescue --extend-outfile=<correct_size> small_file.lz extended_file.lz
|
||||
|
||||
If some of the copies have got garbage data at the end and are
|
||||
therefore larger than they should, you can reduce their sizes to the
|
||||
correct value with the following command before merging them with the
|
||||
other copies:
|
||||
|
||||
ddrescue --size=<correct_size> large_file.lz reduced_file.lz
|
||||
|
||||
To give you an idea of its possibilities, when merging two copies,
|
||||
each of them with one damaged area affecting 1 percent of the copy, the
|
||||
probability of obtaining a correct file is about 98 percent. With three
|
||||
such copies the probability rises to 99.97 percent. For large files with
|
||||
small errors, the probability approaches 100 percent even with only two
|
||||
copies.
|
||||
such copies the probability rises to 99.97 percent. For large files (a
|
||||
few MB) with small errors (one sector damaged per copy), the probability
|
||||
approaches 100 percent even with only two copies.
|
||||
|
||||
|
||||
File: lziprecover.info, Node: File Format, Next: Examples, Prev: Merging Files, Up: Top
|
||||
File: lziprecover.info, Node: File format, Next: Examples, Prev: Merging files, Up: Top
|
||||
|
||||
5 File Format
|
||||
5 File format
|
||||
*************
|
||||
|
||||
Perfection is reached, not when there is no longer anything to add, but
|
||||
|
|
@ -343,7 +362,7 @@ additional information before, between, or after them.
|
|||
|
||||
|
||||
|
||||
File: lziprecover.info, Node: Examples, Next: Problems, Prev: File Format, Up: Top
|
||||
File: lziprecover.info, Node: Examples, Next: Unzcrash, Prev: File format, Up: Top
|
||||
|
||||
6 A small tutorial with examples
|
||||
********************************
|
||||
|
|
@ -381,8 +400,8 @@ are abridged diagnostic messages from lziprecover).
|
|||
|
||||
|
||||
Example 6: Split the multi-member file `file.lz' and write each member
|
||||
in its own `recXXXXXfile.lz' file. Then use `lziprecover -t' to test
|
||||
the integrity of the resulting files.
|
||||
in its own `recXXXfile.lz' file. Then use `lziprecover -t' to test the
|
||||
integrity of the resulting files.
|
||||
|
||||
lziprecover -s file.lz
|
||||
lziprecover -tv rec*file.lz
|
||||
|
|
@ -401,7 +420,7 @@ error-checked merging of copies (*Note GNU ddrescue manual:
|
|||
mount -t iso9660 -o loop,ro cdimage2 /mnt/cdimage
|
||||
cp /mnt/cdimage/backup.tar.lz rescued2.tar.lz
|
||||
umount /mnt/cdimage
|
||||
lziprecover -m -v -o rescued.tar.lz rescued1.tar.lz rescued2.tar.lz
|
||||
lziprecover -m -v -o backup.tar.lz rescued1.tar.lz rescued2.tar.lz
|
||||
|
||||
|
||||
Example 8: Recover the first volume of those created with the command
|
||||
|
|
@ -414,9 +433,80 @@ correct file produced is saved in `big_db_00001.lz'.
|
|||
Input files merged successfully
|
||||
|
||||
|
||||
File: lziprecover.info, Node: Problems, Next: Concept Index, Prev: Examples, Up: Top
|
||||
File: lziprecover.info, Node: Unzcrash, Next: Problems, Prev: Examples, Up: Top
|
||||
|
||||
7 Reporting Bugs
|
||||
7 Testing the robustness of decompressors
|
||||
*****************************************
|
||||
|
||||
The lziprecover package also includes unzcrash, a program written to
|
||||
test robustness to decompression of corrupted data, inspired by
|
||||
unzcrash.c from Julian Seward's bzip2. Type `make unzcrash' in the
|
||||
lziprecover source directory to build it.
|
||||
|
||||
Unzcrash reads the specified file and then repeatedly decompresses
|
||||
it, increasing 256 times each byte of the compressed data, so as to
|
||||
test all possible one-byte errors. This should not cause any invalid
|
||||
memory accesses. If it does, please, report it as a bug.
|
||||
|
||||
Unzcrash really executes as a subprocess the shell command specified
|
||||
in the first non-option argument, and then writes the file specified in
|
||||
the second non-option argument to the standard input of the subprocess,
|
||||
modifying the corresponding byte each time. Therefore you can use
|
||||
unzcrash to test any decompressor (not only lzip), or even other decoder
|
||||
programs with a suitable command line syntax.
|
||||
|
||||
The format for running unzcrash is:
|
||||
|
||||
unzcrash [OPTIONS] "lzip -tv" FILENAME.lz
|
||||
|
||||
Unzcrash supports the following options:
|
||||
|
||||
`-h'
|
||||
`--help'
|
||||
Print an informative help message describing the options and exit.
|
||||
|
||||
`-V'
|
||||
`--version'
|
||||
Print the version number of lziprecover on the standard output and
|
||||
exit.
|
||||
|
||||
`-b RANGE'
|
||||
`--bits=RANGE'
|
||||
Test N-bit errors only, instead of testing all the 255 wrong
|
||||
values for each byte. `N-bit error' means any value differing from
|
||||
the original value in N bit positions, not a value differing from
|
||||
the original value in the bit position N.
|
||||
The number of N-bit errors per byte (N = 1 to 8) is: 8 28 56 70 56
|
||||
28 8 1
|
||||
Examples of RANGE: 1 1,2,3 1-4 1,3-5,8 1-3,5-8
|
||||
|
||||
`-p BYTES'
|
||||
`--position=BYTES'
|
||||
First byte position to test in the file. Defaults to 0.
|
||||
|
||||
`-q'
|
||||
`--quiet'
|
||||
Quiet operation. Suppress all messages.
|
||||
|
||||
`-s BYTES'
|
||||
`--size=BYTES'
|
||||
Number of byte positions to test. If not specified, the whole file
|
||||
is tested.
|
||||
|
||||
`-v'
|
||||
`--verbose'
|
||||
Verbose mode.
|
||||
|
||||
|
||||
Exit status: 0 for a normal exit, 1 for environmental problems (file
|
||||
not found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or
|
||||
invalid input file, 3 for an internal consistency error (eg, bug) which
|
||||
caused unzcrash to panic.
|
||||
|
||||
|
||||
File: lziprecover.info, Node: Problems, Next: Concept index, Prev: Unzcrash, Up: Top
|
||||
|
||||
8 Reporting bugs
|
||||
****************
|
||||
|
||||
There are probably bugs in lziprecover. There are certainly errors and
|
||||
|
|
@ -429,9 +519,9 @@ for all eternity, if not longer.
|
|||
by running `lziprecover --version'.
|
||||
|
||||
|
||||
File: lziprecover.info, Node: Concept Index, Prev: Problems, Up: Top
|
||||
File: lziprecover.info, Node: Concept index, Prev: Problems, Up: Top
|
||||
|
||||
Concept Index
|
||||
Concept index
|
||||
*************
|
||||
|
||||
[index]
|
||||
|
|
@ -439,26 +529,28 @@ Concept Index
|
|||
|
||||
* bugs: Problems. (line 6)
|
||||
* examples: Examples. (line 6)
|
||||
* file format: File Format. (line 6)
|
||||
* file format: File format. (line 6)
|
||||
* getting help: Problems. (line 6)
|
||||
* introduction: Introduction. (line 6)
|
||||
* invoking: Invoking Lziprecover. (line 6)
|
||||
* merging files: Merging Files. (line 6)
|
||||
* repairing files: Repairing Files. (line 6)
|
||||
* invoking: Invoking lziprecover. (line 6)
|
||||
* merging files: Merging files. (line 6)
|
||||
* repairing files: Repairing files. (line 6)
|
||||
* unzcrash: Unzcrash. (line 6)
|
||||
|
||||
|
||||
|
||||
Tag Table:
|
||||
Node: Top231
|
||||
Node: Introduction1032
|
||||
Node: Invoking Lziprecover3052
|
||||
Node: Repairing Files8489
|
||||
Node: Merging Files9208
|
||||
Node: File Format10338
|
||||
Node: Examples12822
|
||||
Ref: ddrescue-example14024
|
||||
Node: Problems15134
|
||||
Node: Concept Index15684
|
||||
Node: Top228
|
||||
Node: Introduction1095
|
||||
Node: Invoking lziprecover3248
|
||||
Node: Repairing files8686
|
||||
Node: Merging files9405
|
||||
Node: File format11176
|
||||
Node: Examples13660
|
||||
Ref: ddrescue-example14860
|
||||
Node: Unzcrash15969
|
||||
Node: Problems18344
|
||||
Node: Concept index18894
|
||||
|
||||
End Tag Table
|
||||
|
||||
|
|
|
|||
|
|
@ -6,19 +6,19 @@
|
|||
@finalout
|
||||
@c %**end of header
|
||||
|
||||
@set UPDATED 17 June 2013
|
||||
@set VERSION 1.15-pre1
|
||||
@set UPDATED 29 July 2013
|
||||
@set VERSION 1.15-rc1
|
||||
|
||||
@dircategory Data Compression
|
||||
@direntry
|
||||
* Lziprecover: (lziprecover). Data recovery tool for lzipped files
|
||||
* Lziprecover: (lziprecover). Data recovery tool for lzip files
|
||||
@end direntry
|
||||
|
||||
|
||||
@ifnothtml
|
||||
@titlepage
|
||||
@title Lziprecover
|
||||
@subtitle Data recovery tool for lzipped files
|
||||
@subtitle Data recovery tool for lzip files
|
||||
@subtitle for Lziprecover version @value{VERSION}, @value{UPDATED}
|
||||
@author by Antonio Diaz Diaz
|
||||
|
||||
|
|
@ -36,13 +36,14 @@ This manual is for Lziprecover (version @value{VERSION}, @value{UPDATED}).
|
|||
|
||||
@menu
|
||||
* Introduction:: Purpose and features of lziprecover
|
||||
* Invoking Lziprecover:: Command line interface
|
||||
* Repairing Files:: Fixing bit-flip and similar errors
|
||||
* Merging Files:: Fixing several damaged copies
|
||||
* File Format:: Detailed format of the compressed file
|
||||
* Invoking lziprecover:: Command line interface
|
||||
* Repairing files:: Fixing bit-flip and similar errors
|
||||
* Merging files:: Fixing several damaged copies
|
||||
* File format:: Detailed format of the compressed file
|
||||
* Examples:: A small tutorial with examples
|
||||
* Unzcrash:: Testing the robustness of decompressors
|
||||
* Problems:: Reporting bugs
|
||||
* Concept Index:: Index of concepts
|
||||
* Concept index:: Index of concepts
|
||||
@end menu
|
||||
|
||||
@sp 1
|
||||
|
|
@ -57,16 +58,34 @@ to copy, distribute and modify it.
|
|||
@cindex introduction
|
||||
|
||||
Lziprecover is a data recovery tool and decompressor for files in the
|
||||
lzip compressed data format (.lz) able to repair slightly damaged files,
|
||||
recover badly damaged files from two or more copies, extract data from
|
||||
damaged files, decompress files and test integrity of files.
|
||||
lzip compressed data format (.lz), able to repair slightly damaged
|
||||
files, recover badly damaged files from two or more copies, extract data
|
||||
from damaged files, decompress files and test integrity of files.
|
||||
|
||||
The lzip file format is designed for long-term data archiving. It is
|
||||
clean, provides very safe 4 factor integrity checking, and is backed by
|
||||
the recovery capabilities of lziprecover.
|
||||
|
||||
Lziprecover is able to recover or decompress files produced by any of
|
||||
the compressors in the lzip family; lzip, plzip, minilzip/lzlib, clzip
|
||||
and pdlzip. It makes lzip files resistant to bit-flip, one of the most
|
||||
common forms of data corruption, and its recovery capabilities
|
||||
contribute to make of the lzip format one of the best options for
|
||||
long-term data archiving.
|
||||
and pdlzip.
|
||||
|
||||
Lziprecover makes lzip files resistant to bit-flip (one of the most
|
||||
common forms of data corruption), and can safely merge multiple damaged
|
||||
backup copies.
|
||||
|
||||
If the cause of file corruption is damaged media, the combination
|
||||
@w{GNU ddrescue + lziprecover} is the best option for recovering data
|
||||
from multiple damaged copies. @xref{ddrescue-example}, for an example.
|
||||
|
||||
If a file is too damaged for lziprecover to repair it, all the
|
||||
recoverable data in all members of the file can be extracted with the
|
||||
following command (the resulting file may contain errors and some
|
||||
garbage data may be produced at the end of each member):
|
||||
|
||||
@example
|
||||
lziprecover -D0 -i -o file -q file.lz
|
||||
@end example
|
||||
|
||||
Lziprecover is able to efficiently extract a range of bytes from a
|
||||
multi-member file, because it only decompresses the members containing
|
||||
|
|
@ -83,25 +102,12 @@ damaged files themselves are never modified.
|
|||
When decompressing or testing file integrity, lziprecover behaves like
|
||||
lzip or lunzip.
|
||||
|
||||
If a file is too damaged for lziprecover to repair it, all the
|
||||
recoverable data in all members of the file can be extracted with the
|
||||
following command (the resulting file may contain errors and some
|
||||
garbage data may be produced at the end of each member):
|
||||
|
||||
@example
|
||||
lziprecover -D0 -i -o file -q file.lz
|
||||
@end example
|
||||
|
||||
If the cause of file corruption is damaged media, the combination
|
||||
@w{GNU ddrescue + lziprecover} is the best option for recovering data
|
||||
from multiple damaged copies. @xref{ddrescue-example}, for an example.
|
||||
|
||||
Lziprecover is not a replacement for regular backups, but a last line of
|
||||
defense for the case where the backups are also damaged.
|
||||
|
||||
|
||||
@node Invoking Lziprecover
|
||||
@chapter Invoking Lziprecover
|
||||
@node Invoking lziprecover
|
||||
@chapter Invoking lziprecover
|
||||
@cindex invoking
|
||||
|
||||
The format for running lziprecover is:
|
||||
|
|
@ -172,8 +178,8 @@ information about the members in the file.
|
|||
Try to produce a correct file merging the good parts of two or more
|
||||
damaged copies. If successful, a repaired copy is written to the file
|
||||
@samp{@var{file}_fixed.lz}. The exit status is 0 if a correct file could
|
||||
be produced, 2 otherwise. See the chapter Merging Files (@pxref{Merging
|
||||
Files}) for a complete description of the merge mode.
|
||||
be produced, 2 otherwise. See the chapter @samp{Merging files}
|
||||
(@pxref{Merging files}) for a complete description of the merge mode.
|
||||
|
||||
@item -o @var{file}
|
||||
@itemx --output=@var{file}
|
||||
|
|
@ -190,11 +196,11 @@ Quiet operation. Suppress all messages.
|
|||
|
||||
@item -R
|
||||
@itemx --repair
|
||||
Try to repair a file with small errors (up to one byte error per member).
|
||||
If successful, a repaired copy is written to the file
|
||||
Try to repair a file with small errors (up to one byte error per
|
||||
member). If successful, a repaired copy is written to the file
|
||||
@samp{@var{file}_fixed.lz}. @samp{@var{file}} is not modified at all.
|
||||
The exit status is 0 if the file could be repaired, 2 otherwise.
|
||||
See the chapter Repairing Files (@pxref{Repairing Files}) for a complete
|
||||
The exit status is 0 if the file could be repaired, 2 otherwise. See the
|
||||
chapter @samp{Repairing files} (@pxref{Repairing files}) for a complete
|
||||
description of the repair mode.
|
||||
|
||||
@item -s
|
||||
|
|
@ -221,13 +227,12 @@ Use it together with @samp{-v} to see information about the file.
|
|||
@itemx --verbose
|
||||
Verbose mode.@*
|
||||
When decompressing or testing, further -v's (up to 4) increase the
|
||||
verbosity level, showing status, dictionary size, compression ratio,
|
||||
verbosity level, showing status, compression ratio, dictionary size,
|
||||
trailer contents (CRC, data size, member size), and up to 6 bytes of
|
||||
trailing garbage (if any).
|
||||
|
||||
@end table
|
||||
|
||||
@sp 1
|
||||
Numbers given as arguments to options may be followed by a multiplier
|
||||
and an optional @samp{B} for "byte".
|
||||
|
||||
|
|
@ -252,8 +257,8 @@ invalid input file, 3 for an internal consistency error (eg, bug) which
|
|||
caused lziprecover to panic.
|
||||
|
||||
|
||||
@node Repairing Files
|
||||
@chapter Repairing Files
|
||||
@node Repairing files
|
||||
@chapter Repairing files
|
||||
@cindex repairing files
|
||||
|
||||
Lziprecover is able to repair files with small errors (up to one byte
|
||||
|
|
@ -268,12 +273,12 @@ versa. It may be caused by bad RAM or even by natural radiation. I have
|
|||
seen a case of bit-flip in a file stored in an USB flash drive.
|
||||
|
||||
|
||||
@node Merging Files
|
||||
@chapter Merging Files
|
||||
@node Merging files
|
||||
@chapter Merging files
|
||||
@cindex merging files
|
||||
|
||||
If you have several copies of a file but all of them are too damaged to
|
||||
repair them (@pxref{Repairing Files}), lziprecover can try to produce a
|
||||
repair them (@pxref{Repairing files}), lziprecover can try to produce a
|
||||
correct file merging the good parts of the damaged copies.
|
||||
|
||||
The merge may succeed even if some copies of the file have all the
|
||||
|
|
@ -285,16 +290,33 @@ The merge will fail if the damaged areas overlap (at least one byte is
|
|||
damaged in all copies), or are adjacent and the boundary can't be
|
||||
determined, or if the copies have too many damaged areas.
|
||||
|
||||
To give you an idea of its possibilities, when merging two copies each
|
||||
All the copies must have the same size. If some of them have been
|
||||
truncated and are therefore smaller than they should, you can extend
|
||||
them to the correct size with the following command before merging them
|
||||
with the other copies:
|
||||
|
||||
@example
|
||||
ddrescue --extend-outfile=<correct_size> small_file.lz extended_file.lz
|
||||
@end example
|
||||
|
||||
If some of the copies have got garbage data at the end and are therefore
|
||||
larger than they should, you can reduce their sizes to the correct value
|
||||
with the following command before merging them with the other copies:
|
||||
|
||||
@example
|
||||
ddrescue --size=<correct_size> large_file.lz reduced_file.lz
|
||||
@end example
|
||||
|
||||
To give you an idea of its possibilities, when merging two copies, each
|
||||
of them with one damaged area affecting 1 percent of the copy, the
|
||||
probability of obtaining a correct file is about 98 percent. With three
|
||||
such copies the probability rises to 99.97 percent. For large files with
|
||||
small errors, the probability approaches 100 percent even with only two
|
||||
copies.
|
||||
such copies the probability rises to 99.97 percent. For large files (a
|
||||
few MB) with small errors (one sector damaged per copy), the probability
|
||||
approaches 100 percent even with only two copies.
|
||||
|
||||
|
||||
@node File Format
|
||||
@chapter File Format
|
||||
@node File format
|
||||
@chapter File format
|
||||
@cindex file format
|
||||
|
||||
Perfection is reached, not when there is no longer anything to add, but
|
||||
|
|
@ -421,7 +443,7 @@ mv file_fixed.lz file.lz
|
|||
@sp 1
|
||||
@noindent
|
||||
Example 6: Split the multi-member file @samp{file.lz} and write each
|
||||
member in its own @samp{recXXXXXfile.lz} file. Then use
|
||||
member in its own @samp{recXXXfile.lz} file. Then use
|
||||
@w{@samp{lziprecover -t}} to test the integrity of the resulting files.
|
||||
|
||||
@example
|
||||
|
|
@ -453,13 +475,13 @@ ddrescue -b2048 /dev/cdrom cdimage2 logfile2
|
|||
mount -t iso9660 -o loop,ro cdimage2 /mnt/cdimage
|
||||
cp /mnt/cdimage/backup.tar.lz rescued2.tar.lz
|
||||
umount /mnt/cdimage
|
||||
lziprecover -m -v -o rescued.tar.lz rescued1.tar.lz rescued2.tar.lz
|
||||
lziprecover -m -v -o backup.tar.lz rescued1.tar.lz rescued2.tar.lz
|
||||
@end example
|
||||
|
||||
@sp 1
|
||||
@noindent
|
||||
Example 8: Recover the first volume of those created with the command
|
||||
@w{@code{lzip -b 32MiB -S 650MB big_db}} from two copies,
|
||||
@w{@samp{lzip -b 32MiB -S 650MB big_db}} from two copies,
|
||||
@samp{big_db1_00001.lz} and @samp{big_db2_00001.lz}, with member 07
|
||||
damaged in the first copy, member 18 damaged in the second copy, and
|
||||
member 12 damaged in both copies. The correct file produced is saved in
|
||||
|
|
@ -471,8 +493,80 @@ lziprecover -m -v -o big_db_00001.lz big_db1_00001.lz big_db2_00001.lz
|
|||
@end example
|
||||
|
||||
|
||||
@node Unzcrash
|
||||
@chapter Testing the robustness of decompressors
|
||||
@cindex unzcrash
|
||||
|
||||
The lziprecover package also includes unzcrash, a program written to
|
||||
test robustness to decompression of corrupted data, inspired by
|
||||
unzcrash.c from Julian Seward's bzip2. Type @samp{make unzcrash} in the
|
||||
lziprecover source directory to build it.
|
||||
|
||||
Unzcrash reads the specified file and then repeatedly decompresses it,
|
||||
increasing 256 times each byte of the compressed data, so as to test all
|
||||
possible one-byte errors. This should not cause any invalid memory
|
||||
accesses. If it does, please, report it as a bug.
|
||||
|
||||
Unzcrash really executes as a subprocess the shell command specified in
|
||||
the first non-option argument, and then writes the file specified in the
|
||||
second non-option argument to the standard input of the subprocess,
|
||||
modifying the corresponding byte each time. Therefore you can use
|
||||
unzcrash to test any decompressor (not only lzip), or even other decoder
|
||||
programs with a suitable command line syntax.
|
||||
|
||||
The format for running unzcrash is:
|
||||
|
||||
@example
|
||||
unzcrash [@var{options}] "lzip -tv" @var{filename}.lz
|
||||
@end example
|
||||
|
||||
Unzcrash supports the following options:
|
||||
|
||||
@table @samp
|
||||
@item -h
|
||||
@itemx --help
|
||||
Print an informative help message describing the options and exit.
|
||||
|
||||
@item -V
|
||||
@itemx --version
|
||||
Print the version number of lziprecover on the standard output and exit.
|
||||
|
||||
@item -b @var{range}
|
||||
@itemx --bits=@var{range}
|
||||
Test N-bit errors only, instead of testing all the 255 wrong values for
|
||||
each byte. @samp{N-bit error} means any value differing from the
|
||||
original value in N bit positions, not a value differing from the
|
||||
original value in the bit position N.@*
|
||||
The number of N-bit errors per byte (N = 1 to 8) is: 8 28 56 70 56 28 8 1@*
|
||||
Examples of @var{range}: 1 1,2,3 1-4 1,3-5,8 1-3,5-8
|
||||
|
||||
@item -p @var{bytes}
|
||||
@itemx --position=@var{bytes}
|
||||
First byte position to test in the file. Defaults to 0.
|
||||
|
||||
@item -q
|
||||
@itemx --quiet
|
||||
Quiet operation. Suppress all messages.
|
||||
|
||||
@item -s @var{bytes}
|
||||
@itemx --size=@var{bytes}
|
||||
Number of byte positions to test. If not specified, the whole file is
|
||||
tested.
|
||||
|
||||
@item -v
|
||||
@itemx --verbose
|
||||
Verbose mode.
|
||||
|
||||
@end table
|
||||
|
||||
Exit status: 0 for a normal exit, 1 for environmental problems (file not
|
||||
found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or
|
||||
invalid input file, 3 for an internal consistency error (eg, bug) which
|
||||
caused unzcrash to panic.
|
||||
|
||||
|
||||
@node Problems
|
||||
@chapter Reporting Bugs
|
||||
@chapter Reporting bugs
|
||||
@cindex bugs
|
||||
@cindex getting help
|
||||
|
||||
|
|
@ -486,8 +580,8 @@ If you find a bug in lziprecover, please send electronic mail to
|
|||
find by running @w{@samp{lziprecover --version}}.
|
||||
|
||||
|
||||
@node Concept Index
|
||||
@unnumbered Concept Index
|
||||
@node Concept index
|
||||
@unnumbered Concept index
|
||||
|
||||
@printindex cp
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
/* Lziprecover - Data recovery tool for lzipped files
|
||||
/* Lziprecover - Data recovery tool for lzip files
|
||||
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
/* Lziprecover - Data recovery tool for lzipped files
|
||||
/* Lziprecover - Data recovery tool for lzip files
|
||||
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
|
|
|
|||
27
lzip.h
27
lzip.h
|
|
@ -1,4 +1,4 @@
|
|||
/* Lziprecover - Data recovery tool for lzipped files
|
||||
/* Lziprecover - Data recovery tool for lzip files
|
||||
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
|
|
@ -30,10 +30,11 @@ public:
|
|||
static const int next[states] = { 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 4, 5 };
|
||||
st = next[st];
|
||||
}
|
||||
|
||||
void set_match() { st = ( ( st < 7 ) ? 7 : 10 ); }
|
||||
void set_rep() { st = ( ( st < 7 ) ? 8 : 11 ); }
|
||||
void set_short_rep() { st = ( ( st < 7 ) ? 9 : 11 ); }
|
||||
void set_char1() { st -= ( st < 4 ) ? st : 3; } // for st < 7
|
||||
void set_char2() { st -= ( st < 10 ) ? 3 : 6; } // for st >= 7
|
||||
void set_match() { st = ( st < 7 ) ? 7 : 10; }
|
||||
void set_rep() { st = ( st < 7 ) ? 8 : 11; }
|
||||
void set_short_rep() { st = ( st < 7 ) ? 9 : 11; }
|
||||
};
|
||||
|
||||
|
||||
|
|
@ -160,10 +161,11 @@ public:
|
|||
|
||||
uint32_t operator[]( const uint8_t byte ) const { return data[byte]; }
|
||||
|
||||
void update( uint32_t & crc, const uint8_t byte ) const
|
||||
void update_byte( uint32_t & crc, const uint8_t byte ) const
|
||||
{ crc = data[(crc^byte)&0xFF] ^ ( crc >> 8 ); }
|
||||
|
||||
void update( uint32_t & crc, const uint8_t * const buffer, const int size ) const
|
||||
void update_buf( uint32_t & crc, const uint8_t * const buffer,
|
||||
const int size ) const
|
||||
{
|
||||
for( int i = 0; i < size; ++i )
|
||||
crc = data[(crc^buffer[i])&0xFF] ^ ( crc >> 8 );
|
||||
|
|
@ -205,15 +207,15 @@ struct File_header
|
|||
return sz;
|
||||
}
|
||||
|
||||
bool dictionary_size( const int sz )
|
||||
bool dictionary_size( const unsigned sz )
|
||||
{
|
||||
if( sz >= min_dictionary_size && sz <= max_dictionary_size )
|
||||
{
|
||||
data[5] = real_bits( sz - 1 );
|
||||
if( sz > min_dictionary_size )
|
||||
{
|
||||
const int base_size = 1 << data[5];
|
||||
const int wedge = base_size / 16;
|
||||
const unsigned base_size = 1 << data[5];
|
||||
const unsigned wedge = base_size / 16;
|
||||
for( int i = 7; i >= 1; --i )
|
||||
if( base_size - ( i * wedge ) >= sz )
|
||||
{ data[5] |= ( i << 5 ); break; }
|
||||
|
|
@ -287,9 +289,8 @@ int writeblock( const int fd, const uint8_t * const buf, const int size );
|
|||
|
||||
// defined in main.cc
|
||||
int open_instream( const std::string & name, struct stat * const in_statsp,
|
||||
const bool to_stdout, const bool reg_only = false );
|
||||
int open_outstream_rw( const std::string & output_filename,
|
||||
const bool force );
|
||||
const bool no_ofile, const bool reg_only = false );
|
||||
int open_outstream_rw( const std::string & output_filename, const bool force );
|
||||
void show_header( const File_header & header );
|
||||
void show_error( const char * const msg, const int errcode = 0,
|
||||
const bool help = false );
|
||||
|
|
|
|||
23
main.cc
23
main.cc
|
|
@ -1,4 +1,4 @@
|
|||
/* Lziprecover - Data recovery tool for lzipped files
|
||||
/* Lziprecover - Data recovery tool for lzip files
|
||||
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
|
|
@ -80,7 +80,7 @@ struct { const char * from; const char * to; } const known_extensions[] = {
|
|||
{ 0, 0 } };
|
||||
|
||||
enum Mode { m_none, m_decompress, m_generate, m_list, m_merge, m_range,
|
||||
m_recover, m_repair, m_split, m_test, m_update };
|
||||
m_recover, m_repair, m_split, m_test };
|
||||
|
||||
std::string output_filename;
|
||||
int outfd = -1;
|
||||
|
|
@ -93,7 +93,7 @@ bool delete_output_on_interrupt = false;
|
|||
|
||||
void show_help()
|
||||
{
|
||||
std::printf( "%s - Data recovery tool and decompressor for lzipped files.\n", Program_name );
|
||||
std::printf( "%s - Data recovery tool and decompressor for lzip files.\n", Program_name );
|
||||
std::printf( "\nUsage: %s [options] [files]\n", invocation_name );
|
||||
std::printf( "\nOptions:\n"
|
||||
" -h, --help display this help and exit\n"
|
||||
|
|
@ -149,8 +149,6 @@ void show_header( const File_header & header )
|
|||
for( int i = 0; i < 8 && ( num > 9999 || ( exact && num >= factor ) ); ++i )
|
||||
{ num /= factor; if( num % factor != 0 ) exact = false;
|
||||
p = prefix[i]; np = ""; }
|
||||
if( verbosity >= 4 )
|
||||
std::fprintf( stderr, "version %d, ", header.version() );
|
||||
std::fprintf( stderr, "dictionary size %s%4u %sB. ", np, num, p );
|
||||
}
|
||||
|
||||
|
|
@ -192,7 +190,7 @@ int extension_index( const std::string & name )
|
|||
} // end namespace
|
||||
|
||||
int open_instream( const std::string & name, struct stat * const in_statsp,
|
||||
const bool to_stdout, const bool reg_only )
|
||||
const bool no_ofile, const bool reg_only )
|
||||
{
|
||||
int infd = open( name.c_str(), O_RDONLY | o_binary );
|
||||
if( infd < 0 )
|
||||
|
|
@ -208,12 +206,12 @@ int open_instream( const std::string & name, struct stat * const in_statsp,
|
|||
const bool can_read = ( i == 0 && !reg_only &&
|
||||
( S_ISBLK( mode ) || S_ISCHR( mode ) ||
|
||||
S_ISFIFO( mode ) || S_ISSOCK( mode ) ) );
|
||||
if( i != 0 || ( !S_ISREG( mode ) && ( !to_stdout || !can_read ) ) )
|
||||
if( i != 0 || ( !S_ISREG( mode ) && ( !can_read || !no_ofile ) ) )
|
||||
{
|
||||
if( verbosity >= 0 )
|
||||
std::fprintf( stderr, "%s: Input file '%s' is not a regular file%s.\n",
|
||||
program_name, name.c_str(),
|
||||
( can_read && !to_stdout ) ?
|
||||
( can_read && !no_ofile ) ?
|
||||
" and '--stdout' was not specified" : "" );
|
||||
close( infd );
|
||||
infd = -1;
|
||||
|
|
@ -377,7 +375,7 @@ int decompress( const int infd, const Pretty_print & pp, const bool testing )
|
|||
if( rdec.finished() ) // End Of File
|
||||
{
|
||||
if( first_member )
|
||||
{ pp( "Error reading member header" ); retval = 1; }
|
||||
{ pp( "File ends unexpectedly at member header" ); retval = 2; }
|
||||
else if( verbosity >= 4 && size > 0 )
|
||||
show_trailing_garbage( header.data, size, pp, true );
|
||||
break;
|
||||
|
|
@ -455,8 +453,7 @@ void set_signals()
|
|||
} // end namespace
|
||||
|
||||
|
||||
int open_outstream_rw( const std::string & output_filename,
|
||||
const bool force )
|
||||
int open_outstream_rw( const std::string & output_filename, const bool force )
|
||||
{
|
||||
int flags = O_CREAT | O_RDWR | o_binary;
|
||||
if( force ) flags |= O_TRUNC; else flags |= O_EXCL;
|
||||
|
|
@ -593,7 +590,6 @@ int main( const int argc, const char * const argv[] )
|
|||
{
|
||||
case m_generate:
|
||||
case m_recover:
|
||||
case m_update:
|
||||
case m_none: internal_error( "invalid operation" ); break;
|
||||
case m_decompress: break;
|
||||
case m_list:
|
||||
|
|
@ -663,7 +659,8 @@ int main( const int argc, const char * const argv[] )
|
|||
else
|
||||
{
|
||||
input_filename = filenames[i];
|
||||
infd = open_instream( input_filename, &in_stats, to_stdout );
|
||||
infd = open_instream( input_filename, &in_stats,
|
||||
to_stdout || program_mode == m_test );
|
||||
if( infd < 0 ) { if( retval < 1 ) retval = 1; continue; }
|
||||
if( program_mode != m_test )
|
||||
{
|
||||
|
|
|
|||
2
merge.cc
2
merge.cc
|
|
@ -1,4 +1,4 @@
|
|||
/* Lziprecover - Data recovery tool for lzipped files
|
||||
/* Lziprecover - Data recovery tool for lzip files
|
||||
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
/* Lziprecover - Data recovery tool for lzipped files
|
||||
/* Lziprecover - Data recovery tool for lzip files
|
||||
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
|
|
@ -112,7 +112,7 @@ int decompress_member( const int infd, const int outfd,
|
|||
File_header header;
|
||||
rdec.read_data( header.data, File_header::size );
|
||||
if( rdec.finished() ) // End Of File
|
||||
{ pp( "Error reading member header" ); return 1; }
|
||||
{ pp( "File ends unexpectedly at member header" ); return 2; }
|
||||
if( !header.verify_magic() )
|
||||
{ pp( "Bad magic number (file not in lzip format)" ); return 2; }
|
||||
if( !header.verify_version() )
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
/* Lziprecover - Data recovery tool for lzipped files
|
||||
/* Lziprecover - Data recovery tool for lzip files
|
||||
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
|
|
|
|||
2
split.cc
2
split.cc
|
|
@ -1,4 +1,4 @@
|
|||
/* Lziprecover - Data recovery tool for lzipped files
|
||||
/* Lziprecover - Data recovery tool for lzip files
|
||||
Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
#! /bin/sh
|
||||
# check script for Lziprecover - Data recovery tool for lzipped files
|
||||
# check script for Lziprecover - Data recovery tool for lzip files
|
||||
# Copyright (C) 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
#
|
||||
# This script is free software: you have unlimited permission
|
||||
|
|
@ -10,7 +10,7 @@ export LC_ALL
|
|||
objdir=`pwd`
|
||||
testdir=`cd "$1" ; pwd`
|
||||
LZIP="${objdir}"/lziprecover
|
||||
LZIPRECOVER="${objdir}"/lziprecover
|
||||
LZIPRECOVER="${LZIP}"
|
||||
framework_failure() { echo "failure in testing framework" ; exit 1 ; }
|
||||
|
||||
if [ ! -x "${LZIP}" ] ; then
|
||||
|
|
@ -65,9 +65,21 @@ if [ $? = 1 ] ; then printf . ; else fail=1 ; printf - ; fi
|
|||
if [ $? = 1 ] ; then printf . ; else fail=1 ; printf - ; fi
|
||||
"${LZIPRECOVER}" -sq
|
||||
if [ $? = 1 ] ; then printf . ; else fail=1 ; printf - ; fi
|
||||
"${LZIP}" -tq "${in}"
|
||||
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi
|
||||
"${LZIP}" -tq < "${in}"
|
||||
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi
|
||||
"${LZIP}" -cdq "${in}"
|
||||
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi
|
||||
"${LZIP}" -cdq < "${in}"
|
||||
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi
|
||||
dd if="${in_lz}" bs=1 count=6 2> /dev/null | "${LZIP}" -tq
|
||||
if [ $? = 2 ] ; then printf . ; else printf - ; fail=1 ; fi
|
||||
dd if="${in_lz}" bs=1 count=20 2> /dev/null | "${LZIP}" -tq
|
||||
if [ $? = 2 ] ; then printf . ; else printf - ; fail=1 ; fi
|
||||
|
||||
"${LZIP}" -t "${testdir}"/test.txt.lz || fail=1
|
||||
"${LZIP}" -cd "${testdir}"/test.txt.lz > copy || fail=1
|
||||
"${LZIP}" -t "${in_lz}" || fail=1
|
||||
"${LZIP}" -cd "${in_lz}" > copy || fail=1
|
||||
cmp "${in}" copy || fail=1
|
||||
printf .
|
||||
|
||||
|
|
@ -100,8 +112,8 @@ for i in "${f5b1_lz}" "${f5b3_lz}" "${f5b4_lz}" "${f5b5_lz}" ; do
|
|||
cmp "${fox5_lz}" copy.lz || fail=1
|
||||
"${LZIPRECOVER}" -mf -o copy.lz "${i}" "${f5b2_lz}" || fail=1
|
||||
cmp "${fox5_lz}" copy.lz || fail=1
|
||||
printf .
|
||||
done
|
||||
printf .
|
||||
|
||||
for i in "${f5b3_lz}" "${f5b4_lz}" "${f5b5_lz}" ; do
|
||||
"${LZIPRECOVER}" -mf -o copy.lz "${f5b1_lz}" "${f5b2_lz}" "${i}" || fail=1
|
||||
|
|
@ -116,8 +128,8 @@ for i in "${f5b3_lz}" "${f5b4_lz}" "${f5b5_lz}" ; do
|
|||
cmp "${fox5_lz}" copy.lz || fail=1
|
||||
"${LZIPRECOVER}" -mf -o copy.lz "${i}" "${f5b2_lz}" "${f5b1_lz}" || fail=1
|
||||
cmp "${fox5_lz}" copy.lz || fail=1
|
||||
printf .
|
||||
done
|
||||
printf .
|
||||
|
||||
"${LZIPRECOVER}" -mf -o copy.lz "${f5b3_lz}" "${f5b4_lz}" "${f5b5_lz}" || fail=1
|
||||
cmp "${fox5_lz}" copy.lz || fail=1
|
||||
|
|
@ -141,9 +153,9 @@ for i in "${bad1_lz}" "${bad2_lz}" ; do
|
|||
cmp "${in_lz}" copy.lz || fail=1
|
||||
"${LZIPRECOVER}" -mf -o copy.lz "${j}" "${i}" || fail=1
|
||||
cmp "${in_lz}" copy.lz || fail=1
|
||||
printf .
|
||||
done
|
||||
done
|
||||
printf .
|
||||
|
||||
"${LZIPRECOVER}" -mf -o copy.lz "${bad3_lz}" "${bad4_lz}" "${bad5_lz}" || fail=1
|
||||
cmp "${in_lz}" copy.lz || fail=1
|
||||
|
|
@ -171,7 +183,6 @@ if [ $? = 0 ] && [ ! -e copy.lz ] ; then printf . ; else fail=1 ; printf - ; fi
|
|||
if [ $? = 2 ] && [ ! -e copy.lz ] ; then printf . ; else fail=1 ; printf - ; fi
|
||||
"${LZIPRECOVER}" -Rf -o copy.lz "${f5b1_lz}" || fail=1
|
||||
cmp "${fox5_lz}" copy.lz || fail=1
|
||||
printf .
|
||||
"${LZIPRECOVER}" -Rf -o copy.lz "${bad1_lz}" || fail=1
|
||||
cmp "${in_lz}" copy.lz || fail=1
|
||||
printf .
|
||||
|
|
@ -182,8 +193,8 @@ printf "garbage" >> copy || fail=1
|
|||
for i in 1 2 3 ; do
|
||||
"${LZIPRECOVER}" -cd rec${i}copy.lz > copy || fail=1
|
||||
cmp "${in}" copy || fail=1
|
||||
printf .
|
||||
done
|
||||
printf .
|
||||
|
||||
cat "${in_lz}" > anyothername || framework_failure
|
||||
"${LZIP}" -d anyothername || fail=1
|
||||
|
|
@ -191,8 +202,7 @@ cmp "${in}" anyothername.out || fail=1
|
|||
printf .
|
||||
|
||||
cat "${in}" "${in}" > in2 || framework_failure
|
||||
cat "${in_lz}" "${in_lz}" > out2 || framework_failure
|
||||
"${LZIP}" -d < out2 > copy2 || fail=1
|
||||
cat "${in_lz}" "${in_lz}" | "${LZIP}" -d > copy2 || fail=1
|
||||
cmp in2 copy2 || fail=1
|
||||
printf .
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,4 @@
|
|||
/* Unzcrash - A test program written to test robustness to
|
||||
decompression of corrupted data.
|
||||
/* Unzcrash - Tests robustness of decompressors to corrupted data.
|
||||
Inspired by unzcrash.c from Julian Seward's bzip2.
|
||||
Copyright (C) 2008, 2009, 2010, 2011, 2012, 2013 Antonio Diaz Diaz.
|
||||
|
||||
|
|
@ -16,6 +15,12 @@
|
|||
You should have received a copy of the GNU General Public License
|
||||
along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
*/
|
||||
/*
|
||||
Exit status: 0 for a normal exit, 1 for environmental problems
|
||||
(file not found, invalid flags, I/O errors, etc), 2 to indicate a
|
||||
corrupt or invalid input file, 3 for an internal consistency error
|
||||
(eg, bug) which caused unzcrash to panic.
|
||||
*/
|
||||
|
||||
#include <cerrno>
|
||||
#include <climits>
|
||||
|
|
@ -47,9 +52,8 @@ int verbosity = 0;
|
|||
|
||||
void show_help()
|
||||
{
|
||||
std::printf( "%s - A test program written to test robustness to\n", Program_name );
|
||||
std::printf( "decompression of corrupted data.\n"
|
||||
"\nUsage: %s [options] \"lzip -tv\" filename.lz\n", invocation_name );
|
||||
std::printf( "%s - Tests robustness of decompressors to corrupted data.\n", Program_name );
|
||||
std::printf( "\nUsage: %s [options] \"lzip -tv\" filename.lz\n", invocation_name );
|
||||
std::printf( "\nThis program reads the specified file and then repeatedly decompresses\n"
|
||||
"it, increasing 256 times each byte of the compressed data, so as to test\n"
|
||||
"all possible one-byte errors. This should not cause any invalid memory\n"
|
||||
|
|
@ -63,8 +67,12 @@ void show_help()
|
|||
" -s, --size=<bytes> number of byte positions to test [all]\n"
|
||||
" -v, --verbose be verbose (a 2nd -v gives more)\n"
|
||||
"Examples of <range>: 1 1,2,3 1-4 1,3-5,8 1-3,5-8\n"
|
||||
"\nExit status: 0 for a normal exit, 1 for environmental problems (file\n"
|
||||
"not found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or\n"
|
||||
"invalid input file, 3 for an internal consistency error (eg, bug) which\n"
|
||||
"caused unzcrash to panic.\n"
|
||||
"\nReport bugs to lzip-bug@nongnu.org\n"
|
||||
"Lzip home page: http://www.nongnu.org/lzip/lzip.html\n" );
|
||||
"Lziprecover home page: http://www.nongnu.org/lzip/lziprecover.html\n" );
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -160,7 +168,7 @@ unsigned long long getnum( const char * const ptr,
|
|||
}
|
||||
|
||||
|
||||
class Bitset8 // 8 value bitset (1..8)
|
||||
class Bitset8 // 8 value bitset (1 to 8)
|
||||
{
|
||||
bool data[8];
|
||||
static bool valid_digit( const unsigned char ch )
|
||||
|
|
@ -195,7 +203,7 @@ public:
|
|||
return false;
|
||||
}
|
||||
|
||||
// number of n-bit errors per byte (n=0..8): 1 8 28 56 70 56 28 8 1
|
||||
// number of N-bit errors per byte (N=0 to 8): 1 8 28 56 70 56 28 8 1
|
||||
void print() const
|
||||
{
|
||||
std::fflush( stderr );
|
||||
|
|
@ -295,9 +303,9 @@ int main( const int argc, const char * const argv[] )
|
|||
if( size >= buffer_size )
|
||||
{
|
||||
if( verbosity >= 0 )
|
||||
std::fprintf( stderr, "input file '%s' is too big.\n",
|
||||
std::fprintf( stderr, "input file '%s' is too large.\n",
|
||||
parser.argument( argind + 1 ).c_str() );
|
||||
return 1;
|
||||
return 2;
|
||||
}
|
||||
std::fclose( f );
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue