lzip/lziprecover
1
0
Fork 0
Code Issues Activity

Merging upstream version 1.13~rc2.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-21 10:12:48 +01:00
parent 51862e9dc7
commit 7215c2ed30
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
17 changed files with 679 additions and 132 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
ChangeLog
View file

@ -1,3 +1,14 @@
2011-11-20 Antonio Diaz Diaz <ant_diaz@teleline.es>
* Version 1.13-rc2 released.
* Added new option `-D, --range-decompress' which extracts a
range of bytes decompressing only the members containing the
desired data.
* Added new option `-l, --list' which prints correct total file
sizes and ratios even for multimember files.
* New file range_dec.cc.
* testsuite/check.sh: Do not use `ln'. `ln' doesn't work on OS/2.
2011-11-12 Antonio Diaz Diaz <ant_diaz@teleline.es>
* Version 1.13-rc1 released.

4
Makefile.in
View file

@ -6,7 +6,7 @@ INSTALL_DATA = $(INSTALL) -p -m 644
INSTALL_DIR = $(INSTALL) -d -m 755
SHELL = /bin/sh
objs = arg_parser.o decoder.o merge.o repair.o split.o main.o
objs = arg_parser.o decoder.o merge.o range_dec.o repair.o split.o main.o
.PHONY : all install install-info install-man install-strip \
@ -32,6 +32,7 @@ arg_parser.o : arg_parser.h
decoder.o : lzip.h decoder.h
main.o : arg_parser.h lzip.h decoder.h
merge.o : lzip.h decoder.h
range_dec.o : lzip.h decoder.h
repair.o : lzip.h
split.o : lzip.h
@ -97,6 +98,7 @@ dist : doc
$(DISTNAME)/doc/$(pkgname).texinfo \
$(DISTNAME)/testsuite/check.sh \
$(DISTNAME)/testsuite/test.txt \
$(DISTNAME)/testsuite/test921-1921.txt \
$(DISTNAME)/testsuite/test_bad[1-5].lz \
$(DISTNAME)/testsuite/test_v[01].lz \
$(DISTNAME)/*.h \

7
NEWS
View file

@ -7,4 +7,11 @@ Decompressor options (-c, -d, -k, -t) have been implemented in
lziprecover so that a external decompressor is not needed for recovery
nor for "make check".
The new option "-D, --range-decompress" which extracts a range of bytes
decompressing only the members containing the desired data, has been
added.
The new option "-l, --list" which prints correct total file sizes and
ratios even for multimember files, has been added.
"--merge" and "--repair" now remove the output file if recovery fails.

2
configure vendored
View file

@ -8,7 +8,7 @@
args=
no_create=
pkgname=lziprecover
pkgversion=1.13-rc1
pkgversion=1.13-rc2
progname=lziprecover
srctrigger=lzip.h

9
decoder.cc
View file

@ -108,9 +108,14 @@ void LZ_decoder::flush_data()
if( size > 0 )
{
crc32.update( crc_, buffer + stream_pos, size );
if( outfd >= 0 &&
writeblock( outfd, buffer + stream_pos, size ) != size )
if( outfd >= 0 )
{
const long long i = std::max( 0LL, outskip - stream_position() );
const long long s =
std::min( outend - stream_position(), (long long)size ) - i;
if( s > 0 && writeblock( outfd, buffer + stream_pos + i, s ) != s )
throw Error( "Write error" );
}
if( pos >= buffer_size ) { partial_data_pos += pos; pos = 0; }
stream_pos = pos;
}

9
decoder.h
View file

@ -192,6 +192,8 @@ public:
class LZ_decoder
{
const long long outskip;
const long long outend;
long long partial_data_pos;
const int dictionary_size;
const int buffer_size;
@ -203,6 +205,8 @@ class LZ_decoder
const int member_version;
Range_decoder & range_decoder;
long long stream_position() const throw()
{ return partial_data_pos + stream_pos; }
void flush_data();
bool verify_trailer( const Pretty_print & pp ) const;
@ -243,8 +247,11 @@ class LZ_decoder
}
public:
LZ_decoder( const File_header & header, Range_decoder & rdec, const int ofd )
LZ_decoder( const File_header & header, Range_decoder & rdec, const int ofd,
const long long oskip = 0, const long long oend = LLONG_MAX )
:
outskip( oskip ),
outend( oend ),
partial_data_pos( 0 ),
dictionary_size( header.dictionary_size() ),
buffer_size( std::max( 65536, dictionary_size ) ),

13
doc/lziprecover.1
View file

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.37.1.
.TH LZIPRECOVER "1" "November 2011" "Lziprecover 1.13-rc1" "User Commands"
.TH LZIPRECOVER "1" "November 2011" "Lziprecover 1.13-rc2" "User Commands"
.SH NAME
Lziprecover \- recovers data from damaged lzip files
.SH SYNOPSIS
@ -21,12 +21,18 @@ send decompressed output to standard output
\fB\-d\fR, \fB\-\-decompress\fR
decompress
.TP
\fB\-D\fR, \fB\-\-range\-decompress=\fR<range>
decompress only a range of bytes (N\-M)
.TP
\fB\-f\fR, \fB\-\-force\fR
overwrite existing output files
.TP
\fB\-k\fR, \fB\-\-keep\fR
keep (don't delete) input files
.TP
\fB\-l\fR, \fB\-\-list\fR
print total file sizes and ratios
.TP
\fB\-m\fR, \fB\-\-merge\fR
correct errors in file using several copies
.TP
@ -40,13 +46,16 @@ suppress all messages
try to repair a small error in file
.TP
\fB\-s\fR, \fB\-\-split\fR
split a multimember file in single\-member files
split multimember file in single\-member files
.TP
\fB\-t\fR, \fB\-\-test\fR
test compressed file integrity
.TP
\fB\-v\fR, \fB\-\-verbose\fR
be verbose (a 2nd \fB\-v\fR gives more)
.PP
Numbers may be followed by a multiplier: k = kB = 10^3 = 1000,
Ki = KiB = 2^10 = 1024, M = 10^6, Mi = 2^20, G = 10^9, Gi = 2^30, etc...
.SH "REPORTING BUGS"
Report bugs to lzip\-bug@nongnu.org
.br

46
doc/lziprecover.info
View file

@ -12,7 +12,7 @@ File: lziprecover.info, Node: Top, Next: Introduction, Up: (dir)
Lziprecover Manual
******************
This manual is for Lziprecover (version 1.13-rc1, 12 November 2011).
This manual is for Lziprecover (version 1.13-rc2, 20 November 2011).
* Menu:
@ -101,6 +101,19 @@ The format for running lziprecover is:
`--decompress'
Decompress.
`-D RANGE'
`--range-decompress=RANGE'
Decompress only a range of bytes starting at decompressed byte
position `BEGIN' and up to byte position `END - 1'. Three formats
of RANGE are recognized, `BEGIN', `BEGIN-END', and `BEGIN,SIZE'.
If only BEGIN is specified, END is taken as the end of the file.
The produced bytes are sent to standard output unless the
`--output' option is used. In order to guarantee the correctness
of the data produced, all members containing any part of the
desired data are decompressed and their integrity is verified.
This operation is more efficient in multimember files because it
only decompresses the members containing the desired data.
`-f'
`--force'
Force overwrite of output files.
@ -109,6 +122,11 @@ The format for running lziprecover is:
`--keep'
Keep (don't delete) input files during decompression.
`-l'
`--list'
Print total file sizes and ratios. The values produced are correct
even for multimember files.
`-m'
`--merge'
Try to produce a correct file merging the good parts of two or more
@ -174,6 +192,22 @@ The format for running lziprecover is:
bytes of trailing garbage (if any).
Numbers given as arguments to options may be followed by a multiplier
and an optional `B' for "byte".
Table of SI and binary prefixes (unit multipliers):
Prefix Value | Prefix Value
k kilobyte (10^3 = 1000) | Ki kibibyte (2^10 = 1024)
M megabyte (10^6) | Mi mebibyte (2^20)
G gigabyte (10^9) | Gi gibibyte (2^30)
T terabyte (10^12) | Ti tebibyte (2^40)
P petabyte (10^15) | Pi pebibyte (2^50)
E exabyte (10^18) | Ei exbibyte (2^60)
Z zettabyte (10^21) | Zi zebibyte (2^70)
Y yottabyte (10^24) | Yi yobibyte (2^80)

File: lziprecover.info, Node: File Format, Next: Examples, Prev: Invoking Lziprecover, Up: Top
@ -359,10 +393,10 @@ Tag Table:
Node: Top231
Node: Introduction898
Node: Invoking Lziprecover2684
Node: File Format6150
Node: Examples8156
Ref: ddrescue-example9409
Node: Problems11188
Node: Concept Index11738
Node: File Format7727
Node: Examples9733
Ref: ddrescue-example10986
Node: Problems12765
Node: Concept Index13315

End Tag Table

41
doc/lziprecover.texinfo
View file

@ -5,8 +5,8 @@
@finalout
@c %**end of header
@set UPDATED 12 November 2011
@set VERSION 1.13-rc1
@set UPDATED 20 November 2011
@set VERSION 1.13-rc2
@dircategory Data Compression
@direntry
@ -122,6 +122,20 @@ data as possible when decompressing a corrupt file.
@itemx --decompress
Decompress.
@item -D @var{range}
@itemx --range-decompress=@var{range}
Decompress only a range of bytes starting at decompressed byte position
@samp{@var{begin}} and up to byte position @w{@samp{@var{end} - 1}}.
Three formats of @var{range} are recognized, @samp{@var{begin}},
@samp{@var{begin}-@var{end}}, and @samp{@var{begin},@var{size}}. If only
@var{begin} is specified, @var{end} is taken as the end of the file. The
produced bytes are sent to standard output unless the @samp{--output}
option is used. In order to guarantee the correctness of the data
produced, all members containing any part of the desired data are
decompressed and their integrity is verified. This operation is more
efficient in multimember files because it only decompresses the members
containing the desired data.
@item -f
@itemx --force
Force overwrite of output files.
@ -130,6 +144,11 @@ Force overwrite of output files.
@itemx --keep
Keep (don't delete) input files during decompression.
@item -l
@itemx --list
Print total file sizes and ratios. The values produced are correct even
for multimember files.
@item -m
@itemx --merge
Try to produce a correct file merging the good parts of two or more
@ -195,6 +214,24 @@ trailing garbage (if any).
@end table
@sp 1
Numbers given as arguments to options may be followed by a multiplier
and an optional @samp{B} for "byte".
Table of SI and binary prefixes (unit multipliers):
@multitable {Prefix} {kilobyte (10^3 = 1000)} {|} {Prefix} {kibibyte (2^10 = 1024)}
@item Prefix @tab Value @tab | @tab Prefix @tab Value
@item k @tab kilobyte (10^3 = 1000) @tab | @tab Ki @tab kibibyte (2^10 = 1024)
@item M @tab megabyte (10^6) @tab | @tab Mi @tab mebibyte (2^20)
@item G @tab gigabyte (10^9) @tab | @tab Gi @tab gibibyte (2^30)
@item T @tab terabyte (10^12) @tab | @tab Ti @tab tebibyte (2^40)
@item P @tab petabyte (10^15) @tab | @tab Pi @tab pebibyte (2^50)
@item E @tab exabyte (10^18) @tab | @tab Ei @tab exbibyte (2^60)
@item Z @tab zettabyte (10^21) @tab | @tab Zi @tab zebibyte (2^70)
@item Y @tab yottabyte (10^24) @tab | @tab Yi @tab yobibyte (2^80)
@end multitable
@node File Format
@chapter File Format

40
lzip.h
View file

@ -60,6 +60,7 @@ enum {
min_dictionary_size = 1 << min_dictionary_bits,
max_dictionary_bits = 29,
max_dictionary_size = 1 << max_dictionary_bits,
min_member_size = 36,
literal_context_bits = 3,
pos_state_bits = 2,
pos_states = 1 << pos_state_bits,
@ -128,6 +129,15 @@ public:
if( longest_name == 0 ) longest_name = stdin_name_len;
}
Pretty_print( const std::string & filename, const int v )
: stdin_name( "(stdin)" ), verbosity_( v ), first_post( false )
{
const unsigned int stdin_name_len = std::strlen( stdin_name );
longest_name = ( ( filename == "-" ) ? stdin_name_len : filename.size() );
if( longest_name == 0 ) longest_name = stdin_name_len;
set_name( filename );
}
void set_name( const std::string & filename )
{
if( filename.size() && filename != "-" ) name_ = filename;
@ -288,12 +298,35 @@ struct Error
#endif
class Block
{
long long pos_, size_; // pos + size <= LLONG_MAX
public:
Block( const long long p, const long long s ) throw()
: pos_( p ), size_( s ) {}
long long pos() const throw() { return pos_; }
long long size() const throw() { return size_; }
long long end() const throw() { return pos_ + size_; }
void pos( const long long p ) throw() { pos_ = p; }
void size( const long long s ) throw() { size_ = s; }
bool overlaps( const Block & b ) const throw()
{ return ( pos_ < b.end() && b.pos_ < end() ); }
void shift( Block & b ) throw() { ++size_; ++b.pos_; --b.size_; }
};
// defined in decoder.cc
int readblock( const int fd, uint8_t * const buf, const int size ) throw();
int writeblock( const int fd, const uint8_t * const buf, const int size ) throw();
// defined in main.cc
extern int verbosity;
const char * format_num( long long num, long long limit = LLONG_MAX,
const int set_prefix = 0 ) throw();
int open_instream( const std::string & name, struct stat * const in_statsp,
const bool to_stdout, const bool reg_only = false ) throw();
int open_outstream_rw( const std::string & output_filename,
@ -314,6 +347,13 @@ bool verify_single_member( const int fd, const long long file_size );
int merge_files( const std::vector< std::string > & filenames,
const std::string & output_filename, const bool force );
// defined in range_dec.cc
int list_file( const std::string & input_filename );
int range_decompress( const std::string & input_filename,
const std::string & default_output_filename,
const std::string & range_string,
const bool to_stdout, const bool force );
// defined in repair.cc
int repair_file( const std::string & input_filename,
const std::string & output_filename, const bool force );

85
main.cc
View file

@ -71,8 +71,8 @@ struct { const char * from; const char * to; } const known_extensions[] = {
{ ".tlz", ".tar" },
{ 0, 0 } };
enum Mode { m_none, m_decompress, m_generate, m_merge, m_recover, m_repair,
m_split, m_test, m_update };
enum Mode { m_none, m_decompress, m_generate, m_list, m_merge, m_range,
m_recover, m_repair, m_split, m_test, m_update };
std::string output_filename;
int outfd = -1;
@ -91,18 +91,22 @@ void show_help() throw()
" -V, --version output version information and exit\n"
" -c, --stdout send decompressed output to standard output\n"
" -d, --decompress decompress\n"
" -D, --range-decompress=<range> decompress only a range of bytes (N-M)\n"
" -f, --force overwrite existing output files\n"
// " -g, --generate-recover-file generate a recover file\n"
" -k, --keep keep (don't delete) input files\n"
" -l, --list print total file sizes and ratios\n"
" -m, --merge correct errors in file using several copies\n"
" -o, --output=<file> place the output into <file>\n"
" -q, --quiet suppress all messages\n"
// " -r, --recover correct errors in file using a recover file\n"
" -R, --repair try to repair a small error in file\n"
" -s, --split split a multimember file in single-member files\n"
" -s, --split split multimember file in single-member files\n"
" -t, --test test compressed file integrity\n"
// " -u, --update convert file from version 0 to version 1\n"
" -v, --verbose be verbose (a 2nd -v gives more)\n"
"Numbers may be followed by a multiplier: k = kB = 10^3 = 1000,\n"
"Ki = KiB = 2^10 = 1024, M = 10^6, Mi = 2^20, G = 10^9, Gi = 2^30, etc...\n"
"\nReport bugs to lzip-bug@nongnu.org\n"
"Lziprecover home page: http://www.nongnu.org/lzip/lziprecover.html\n" );
}
@ -118,20 +122,13 @@ void show_version() throw()
}
const char * format_num( long long num ) throw()
void one_file( const int argind, const int arguments ) throw()
{
const char * const prefix[8] =
{ "Ki", "Mi", "Gi", "Ti", "Pi", "Ei", "Zi", "Yi" };
enum { buf_size = 16, factor = 1024 };
static char buf[buf_size];
const char *p = "";
bool exact = ( num % factor == 0 );
for( int i = 0; i < 8 && ( llabs( num ) > 9999 ||
( exact && llabs( num ) >= factor ) ); ++i )
{ num /= factor; if( num % factor != 0 ) exact = false; p = prefix[i]; }
snprintf( buf, buf_size, "%lld %s", num, p );
return buf;
if( argind + 1 != arguments )
{
show_error( "You must specify exactly 1 file.", 0, true );
std::exit( 1 );
}
}
@ -345,7 +342,7 @@ int decompress( const int infd, const Pretty_print & pp, const bool testing )
if( verbosity >= 2 )
std::fprintf( stderr, "version %d, dictionary size %7sB. ",
header.version(),
format_num( header.dictionary_size() ) );
format_num( header.dictionary_size(), 9999, -1 ) );
}
LZ_decoder decoder( header, rdec, outfd );
@ -403,6 +400,30 @@ void set_signals() throw()
int verbosity = 0;
const char * format_num( long long num, long long limit,
const int set_prefix ) throw()
{
const char * const si_prefix[8] =
{ "k", "M", "G", "T", "P", "E", "Z", "Y" };
const char * const binary_prefix[8] =
{ "Ki", "Mi", "Gi", "Ti", "Pi", "Ei", "Zi", "Yi" };
static bool si = true;
static char buf[32];
if( set_prefix ) si = ( set_prefix > 0 );
const int factor = ( si ? 1000 : 1024 );
const char * const * prefix = ( si ? si_prefix : binary_prefix );
const char * p = "";
bool exact = ( num % factor == 0 );
for( int i = 0; i < 8 && ( llabs( num ) > limit ||
( exact && llabs( num ) >= factor ) ); ++i )
{ num /= factor; if( num % factor != 0 ) exact = false; p = prefix[i]; }
snprintf( buf, sizeof buf, "%lld %s", num, p );
return buf;
}
int open_instream( const std::string & name, struct stat * const in_statsp,
const bool to_stdout, const bool reg_only ) throw()
{
@ -491,6 +512,7 @@ int main( const int argc, const char * const argv[] )
bool to_stdout = false;
std::string input_filename;
std::string default_output_filename;
std::string range_string;
std::vector< std::string > filenames;
invocation_name = argv[0];
@ -498,9 +520,11 @@ int main( const int argc, const char * const argv[] )
{
{ 'c', "stdout", Arg_parser::no },
{ 'd', "decompress", Arg_parser::no },
{ 'D', "range-decompress", Arg_parser::yes },
{ 'f', "force", Arg_parser::no },
{ 'h', "help", Arg_parser::no },
{ 'k', "keep", Arg_parser::no },
{ 'l', "list", Arg_parser::no },
{ 'm', "merge", Arg_parser::no },
{ 'o', "output", Arg_parser::yes },
{ 'q', "quiet", Arg_parser::no },
@ -520,15 +544,19 @@ int main( const int argc, const char * const argv[] )
{
const int code = parser.code( argind );
if( !code ) break; // no more options
const std::string & arg = parser.argument( argind ).c_str();
switch( code )
{
case 'c': to_stdout = true; break;
case 'd': set_mode( program_mode, m_decompress ); break;
case 'D': set_mode( program_mode, m_range );
range_string = arg; break;
case 'f': force = true; break;
case 'h': show_help(); return 0;
case 'k': keep_input_files = true; break;
case 'l': set_mode( program_mode, m_list ); break;
case 'm': set_mode( program_mode, m_merge ); break;
case 'o': default_output_filename = parser.argument( argind ); break;
case 'o': default_output_filename = arg; break;
case 'q': verbosity = -1; break;
case 'R': set_mode( program_mode, m_repair ); break;
case 's': set_mode( program_mode, m_split ); break;
@ -552,9 +580,10 @@ int main( const int argc, const char * const argv[] )
case m_update:
case m_none: internal_error( "invalid operation" ); break;
case m_decompress: break;
case m_list:
one_file( argind, parser.arguments() );
return list_file( parser.argument( argind ) );
case m_merge:
{
std::vector< std::string > filenames;
for( ; argind < parser.arguments(); ++argind )
filenames.push_back( parser.argument( argind ) );
if( filenames.size() < 2 )
@ -562,21 +591,19 @@ int main( const int argc, const char * const argv[] )
if( !default_output_filename.size() )
default_output_filename = insert_fixed( filenames[0] );
return merge_files( filenames, default_output_filename, force );
} break;
case m_range:
one_file( argind, parser.arguments() );
return range_decompress( parser.argument( argind ),
default_output_filename, range_string,
to_stdout, force );
case m_repair:
{
if( argind + 1 != parser.arguments() )
{ show_error( "You must specify exactly 1 file.", 0, true ); return 1; }
one_file( argind, parser.arguments() );
if( !default_output_filename.size() )
default_output_filename = insert_fixed( parser.argument( argind ) );
return repair_file( parser.argument( argind ), default_output_filename, force );
} break;
case m_split:
{
if( argind + 1 != parser.arguments() )
{ show_error( "You must specify exactly 1 file.", 0, true ); return 1; }
one_file( argind, parser.arguments() );
return split_file( parser.argument( argind ), default_output_filename, force );
} break;
case m_test: break;
}

25
merge.cc
View file

@ -34,24 +34,6 @@
namespace {
class Block
{
long long pos_, size_; // pos + size <= LLONG_MAX
public:
Block( const long long p, const long long s ) throw()
: pos_( p ), size_( s ) {}
long long pos() const throw() { return pos_; }
long long size() const throw() { return size_; }
long long end() const throw() { return pos_ + size_; }
void pos( const long long p ) throw() { pos_ = p; }
void size( const long long s ) throw() { size_ = s; }
void shift( Block & b ) throw() { ++size_; ++b.pos_; --b.size_; }
};
bool copy_and_diff_file( const std::vector< int > & infd_vector,
const int outfd, std::vector< Block > & block_vector )
{
@ -166,7 +148,7 @@ int open_input_files( const std::vector< std::string > & filenames,
if( i == 0 )
{
isize = tmp;
if( isize < 36 ) { show_error( "Input file is too short." ); return 2; }
if( isize < min_member_size ) { show_error( "Input file is too short." ); return 2; }
}
else if( isize != tmp )
{ show_error( "Sizes of input files are different." ); return 1; }
@ -251,8 +233,7 @@ bool try_decompress( const int fd, const long long file_size,
header.dictionary_size() <= max_dictionary_size )
{
LZ_decoder decoder( header, rdec, -1 );
std::vector< std::string > dummy_filenames;
Pretty_print dummy( dummy_filenames, -1 );
Pretty_print dummy( "", -1 );
if( decoder.decode_member( dummy ) == 0 &&
rdec.member_position() == file_size ) return true;
@ -336,7 +317,7 @@ int merge_files( const std::vector< std::string > & filenames,
if( !copy_and_diff_file( infd_vector, outfd, block_vector ) )
cleanup_and_fail( output_filename, outfd, 1 );
if( !block_vector.size() )
if( block_vector.size() == 0 )
{ show_error( "Input files are identical. Recovery is not possible." );
cleanup_and_fail( output_filename, outfd, 2 ); }

359
range_dec.cc Normal file
View file

@ -0,0 +1,359 @@
/* Lziprecover - Data recovery tool for lzipped files
Copyright (C) 2009, 2010, 2011 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#define _FILE_OFFSET_BITS 64
#include <algorithm>
#include <cerrno>
#include <climits>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <string>
#include <vector>
#include <stdint.h>
#include <unistd.h>
#include <sys/stat.h>
#include "lzip.h"
#include "decoder.h"
namespace {
class Member
{
Block dblock_, mblock_; // data block, member block
public:
Member( const long long dp, const long long ds,
const long long mp, const long long ms )
: dblock_( dp, ds ), mblock_( mp, ms ) {}
const Block & dblock() const throw() { return dblock_; }
Block & dblock() throw() { return dblock_; }
const Block & mblock() const throw() { return mblock_; }
Block & mblock() throw() { return mblock_; }
};
int seek_read( const int fd, uint8_t * const buf, const int size,
const long long pos ) throw()
{
if( lseek( fd, pos, SEEK_SET ) == pos )
return readblock( fd, buf, size );
return 0;
}
class Member_index
{
std::vector< Member > member_vector;
public:
Member_index( const int infd, const long long isize )
{
long long pos = isize; // always points to a header or EOF
File_header header;
File_trailer trailer;
while( pos >= min_member_size )
{
if( seek_read( infd, trailer.data, File_trailer::size(),
pos - File_trailer::size() ) != File_trailer::size() )
{ show_error( "Read error", errno ); std::exit( 1 ); }
const long long member_size = trailer.member_size();
if( member_size < min_member_size || pos < member_size ) break;
if( seek_read( infd, header.data, File_header::size,
pos - member_size ) != File_header::size )
{ show_error( "Read error", errno ); std::exit( 1 ); }
if( !header.verify_magic() || !header.verify_version() ) break;
pos -= member_size;
member_vector.push_back( Member( 0, trailer.data_size(),
pos, member_size ) );
}
if( pos != 0 || member_vector.size() == 0 )
{
show_error( "Member size in input file trailer is corrupt." );
std::exit( 1 );
}
std::reverse( member_vector.begin(), member_vector.end() );
for( unsigned int i = 0; i < member_vector.size() - 1; ++i )
member_vector[i+1].dblock().pos( member_vector[i].dblock().end() );
}
long long data_end() const throw()
{ if( member_vector.size() ) return member_vector.back().dblock().end();
else return 0; }
const Member & member( const int i ) const throw()
{ return member_vector[i]; }
const Block & dblock( const int i ) const throw()
{ return member_vector[i].dblock(); }
const Block & mblock( const int i ) const throw()
{ return member_vector[i].mblock(); }
int members() const throw() { return (int)member_vector.size(); }
};
// Returns the number of chars read, or 0 if error.
//
int parse_long_long( const char * const ptr, long long & value ) throw()
{
char * tail;
int c = 0;
errno = 0;
value = strtoll( ptr, &tail, 0 );
if( tail == ptr || errno ) return 0;
c = tail - ptr;
if( ptr[c] )
{
const int factor = ( ptr[c+1] == 'i' ) ? 1024 : 1000;
int exponent = 0;
switch( ptr[c] )
{
case 'Y': exponent = 8; break;
case 'Z': exponent = 7; break;
case 'E': exponent = 6; break;
case 'P': exponent = 5; break;
case 'T': exponent = 4; break;
case 'G': exponent = 3; break;
case 'M': exponent = 2; break;
case 'K': if( factor == 1024 ) exponent = 1; break;
case 'k': if( factor == 1000 ) exponent = 1; break;
}
if( exponent > 0 )
{
++c;
if( ptr[c] == 'i' ) { ++c; if( value ) format_num( 0, 0, -1 ); }
if( ptr[c] == 'B' ) ++c;
for( int i = 0; i < exponent; ++i )
{
if( LLONG_MAX / factor >= llabs( value ) ) value *= factor;
else return 0;
}
}
}
return c;
}
// Recognized formats: <begin> <begin>-<end> <begin>,<size>
//
void parse_range( const char * const ptr, Block & range ) throw()
{
long long value = 0;
int c = parse_long_long( ptr, value ); // pos
if( c && value >= 0 && value < LLONG_MAX &&
( ptr[c] == 0 || ptr[c] == ',' || ptr[c] == '-' ) )
{
range.pos( value );
if( ptr[c] == 0 ) { range.size( LLONG_MAX - value ); return; }
const bool issize = ( ptr[c] == ',' );
c = parse_long_long( ptr + c + 1, value ); // size
if( c && value > 0 && ( issize || value > range.pos() ) )
{
if( !issize ) value -= range.pos();
if( LLONG_MAX - range.pos() >= value ) { range.size( value ); return; }
}
}
show_error( "Bad decompression range.", 0, true );
std::exit( 1 );
}
bool safe_seek( const int fd, const long long pos ) throw()
{
if( lseek( fd, pos, SEEK_SET ) == pos ) return true;
show_error( "Seek error", errno ); return false;
}
int decompress_member( const int infd, const int outfd,
const Pretty_print & pp, const Member & member,
const long long outskip, const long long outend )
{
int retval = 0;
try {
Range_decoder rdec( infd );
File_header header;
int size;
for( size = 0; size < File_header::size && !rdec.finished(); ++size )
header.data[size] = rdec.get_byte();
if( rdec.finished() ) // End Of File
{ pp( "Error reading member header" ); retval = 1; }
if( !header.verify_magic() )
{ pp( "Bad magic number (file not in lzip format)" ); retval = 2; }
if( !header.verify_version() )
{
if( verbosity >= 0 )
{ pp();
std::fprintf( stderr, "Version %d member format not supported.\n",
header.version() ); }
retval = 2;
}
if( header.dictionary_size() < min_dictionary_size ||
header.dictionary_size() > max_dictionary_size )
{ pp( "Invalid dictionary size in member header" ); retval = 2; }
if( pp.verbosity() >= 2 )
{
pp();
std::fprintf( stderr, "version %d, dictionary size %7sB. ",
header.version(),
format_num( header.dictionary_size(), 9999, -1 ) );
}
LZ_decoder decoder( header, rdec, outfd, outskip, outend );
const int result = decoder.decode_member( pp );
if( result != 0 )
{
if( verbosity >= 0 && result <= 2 )
{
pp();
if( result == 2 )
std::fprintf( stderr, "File ends unexpectedly at pos %lld\n",
member.mblock().pos() + rdec.member_position() );
else
std::fprintf( stderr, "Decoder error at pos %lld\n",
member.mblock().pos() + rdec.member_position() );
}
retval = 2;
}
if( pp.verbosity() >= 2 ) std::fprintf( stderr, "done\n" );
}
catch( std::bad_alloc )
{
pp( "Not enough memory. Find a machine with more memory" );
retval = 1;
}
catch( Error e ) { pp(); show_error( e.msg, errno ); retval = 1; }
return retval;
}
} // end namespace
int list_file( const std::string & input_filename )
{
struct stat in_stats;
const int infd = open_instream( input_filename, &in_stats, true, true );
if( infd < 0 ) return 1;
const long long isize = lseek( infd, 0, SEEK_END );
if( isize < 0 )
{ show_error( "Input file is not seekable", errno ); return 1; }
if( isize < min_member_size )
{ show_error( "Input file is too short." ); return 2; }
Member_index member_index( infd, isize );
if( verbosity >= 0 )
{
if( verbosity >= 1 )
{
std::printf( "Total members in file = %d.\n", member_index.members() );
for( int i = 0; i < member_index.members(); ++i )
{
const Block & db = member_index.dblock( i );
const Block & mb = member_index.mblock( i );
std::printf( "Member %3d data pos %9lld data size %7lld "
"member pos %9lld member size %7lld.\n", i,
db.pos(), db.size(), mb.pos(), mb.size() );
}
std::printf( "\n" );
}
const long long data_size = member_index.data_end();
if( data_size > 0 && isize > 0 )
std::printf( "%6.3f:1, %6.3f bits/byte, %5.2f%% saved.\n",
(double)data_size / isize,
( 8.0 * isize ) / data_size,
100.0 * ( 1.0 - ( (double)isize / data_size ) ) );
std::printf( "decompressed size %9lld, compressed size %8lld.\n",
data_size, isize );
}
return 0;
}
int range_decompress( const std::string & input_filename,
const std::string & output_filename,
const std::string & range_string,
const bool to_stdout, const bool force )
{
Block range( 0, 0 );
parse_range( range_string.c_str(), range );
struct stat in_stats;
const int infd = open_instream( input_filename, &in_stats, true, true );
if( infd < 0 ) return 1;
const long long isize = lseek( infd, 0, SEEK_END );
if( isize < 0 )
{ show_error( "Input file is not seekable", errno ); return 1; }
if( isize < min_member_size )
{ show_error( "Input file is too short." ); return 2; }
Member_index member_index( infd, isize );
if( range.end() > member_index.data_end() )
range.size( std::max( 0LL, member_index.data_end() - range.pos() ) );
if( range.size() <= 0 )
{ if( verbosity >= 1 ) show_error( "Nothing to do." ); return 0; }
if( verbosity >= 1 )
{
if( verbosity >= 2 )
std::fprintf( stderr, "Decompressed file size = %sB\n",
format_num( member_index.data_end() ) );
std::fprintf( stderr, "Decompressing range %sB", format_num( range.pos() ) );
std::fprintf( stderr, " to %sB ", format_num( range.pos() + range.size() ) );
std::fprintf( stderr, "(%sBytes)\n", format_num( range.size() ) );
}
int outfd = -1;
if( to_stdout || !output_filename.size() )
outfd = STDOUT_FILENO;
else
{ outfd = open_outstream_rw( output_filename, force );
if( outfd < 0 ) return 1; }
Pretty_print pp( input_filename, 0 );
int retval = 0;
for( int i = 0; i < member_index.members(); ++i )
{
const Block & db = member_index.dblock( i );
if( range.overlaps( db ) )
{
if( verbosity >= 3 )
std::fprintf( stderr, "Decompressing member %3d\n", i );
const long long outskip = std::max( 0LL, range.pos() - db.pos() );
const long long outend = std::min( db.end(), range.end() - db.pos() );
if( !safe_seek( infd, member_index.mblock( i ).pos() ) )
{ retval = 1; break; }
retval = decompress_member( infd, outfd, pp, member_index.member( i ),
outskip, outend );
if( retval ) cleanup_and_fail( output_filename, outfd, retval );
pp.reset();
}
}
if( close( outfd ) != 0 )
{
show_error( "Error closing output file", errno );
cleanup_and_fail( output_filename, -1, 1 );
}
if( verbosity >= 2 )
std::fprintf( stderr, "Byte range decompressed successfully.\n" );
return retval;
}

3
repair.cc
View file

@ -24,6 +24,7 @@
#include <string>
#include <vector>
#include <stdint.h>
#include <unistd.h>
#include <sys/stat.h>
#include "lzip.h"
@ -38,7 +39,7 @@ int repair_file( const std::string & input_filename,
const long long isize = lseek( infd, 0, SEEK_END );
if( isize < 0 )
{ show_error( "Input file is not seekable", errno ); return 1; }
if( isize < 36 )
if( isize < min_member_size )
{ show_error( "Input file is too short." ); return 2; }
if( !verify_single_member( infd, isize ) ) return 2;
if( lseek( infd, 0, SEEK_SET ) < 0 )

1
split.cc
View file

@ -24,6 +24,7 @@
#include <string>
#include <vector>
#include <stdint.h>
#include <unistd.h>
#include <sys/stat.h>
#include "lzip.h"

73
testsuite/check.sh
View file

@ -21,12 +21,14 @@ if [ -d tmp ] ; then rm -rf tmp ; fi
mkdir tmp
cd "${objdir}"/tmp
ln "${testdir}"/test.txt in || framework_failure
ln "${testdir}"/test_v1.lz in.lz || framework_failure
cat in.lz in.lz in.lz > in3.lz || framework_failure
for i in 1 2 3 4 5 ; do
ln "${testdir}"/test_bad${i}.lz bad${i}.lz
done
in="${testdir}"/test.txt
in_lz="${testdir}"/test_v1.lz
inD="${testdir}"/test921-1921.txt
bad1_lz="${testdir}"/test_bad1.lz
bad2_lz="${testdir}"/test_bad2.lz
bad3_lz="${testdir}"/test_bad3.lz
bad4_lz="${testdir}"/test_bad4.lz
bad5_lz="${testdir}"/test_bad5.lz
fail=0
# Description of test files for lziprecover:
@ -38,58 +40,65 @@ fail=0
printf "testing lziprecover-%s..." "$2"
"${LZIPRECOVER}" -m -o copy.lz bad1.lz bad2.lz bad1.lz -q
"${LZIPRECOVER}" -D 921-1921 -o copy ${in_lz} || fail=1
cmp ${inD} copy || fail=1
printf .
"${LZIPRECOVER}" -D 921,1000 ${in_lz} > copy || fail=1
cmp ${inD} copy || fail=1
printf .
"${LZIPRECOVER}" -m -o copy.lz ${bad1_lz} ${bad2_lz} ${bad1_lz} -q
if [ $? != 1 ] ; then fail=1 ; printf - ; else printf . ; fi
"${LZIPRECOVER}" -m -o copy.lz bad1.lz bad2.lz || fail=1
"${LZIPRECOVER}" -m -o copy.lz ${bad1_lz} ${bad2_lz} || fail=1
"${LZIPRECOVER}" -df copy.lz || fail=1
cmp in copy || fail=1
cmp ${in} copy || fail=1
printf .
"${LZIPRECOVER}" -m -o copy.lz bad2.lz bad1.lz || fail=1
"${LZIPRECOVER}" -m -o copy.lz ${bad2_lz} ${bad1_lz} || fail=1
"${LZIPRECOVER}" -df copy.lz || fail=1
cmp in copy || fail=1
cmp ${in} copy || fail=1
printf .
for i in 1 2 ; do
for j in 3 4 5 ; do
"${LZIPRECOVER}" -m -o copy.lz bad${i}.lz bad${j}.lz || fail=1
for i in ${bad1_lz} ${bad2_lz} ; do
for j in ${bad3_lz} ${bad4_lz} ${bad5_lz} ; do
"${LZIPRECOVER}" -m -o copy.lz ${i} ${j} || fail=1
"${LZIPRECOVER}" -df copy.lz || fail=1
cmp in copy || fail=1
cmp ${in} copy || fail=1
printf .
"${LZIPRECOVER}" -m -o copy.lz bad${j}.lz bad${i}.lz || fail=1
"${LZIPRECOVER}" -m -o copy.lz ${j} ${i} || fail=1
"${LZIPRECOVER}" -df copy.lz || fail=1
cmp in copy || fail=1
cmp ${in} copy || fail=1
printf .
done
done
"${LZIPRECOVER}" -m -o copy.lz bad3.lz bad4.lz bad5.lz || fail=1
"${LZIPRECOVER}" -m -o copy.lz ${bad3_lz} ${bad4_lz} ${bad5_lz} || fail=1
"${LZIPRECOVER}" -df copy.lz || fail=1
cmp in copy || fail=1
cmp ${in} copy || fail=1
printf .
"${LZIPRECOVER}" -m -o copy.lz bad4.lz bad5.lz bad3.lz || fail=1
"${LZIPRECOVER}" -m -o copy.lz ${bad4_lz} ${bad5_lz} ${bad3_lz} || fail=1
"${LZIPRECOVER}" -df copy.lz || fail=1
cmp in copy || fail=1
cmp ${in} copy || fail=1
printf .
"${LZIPRECOVER}" -m -o copy.lz bad5.lz bad3.lz bad4.lz || fail=1
"${LZIPRECOVER}" -m -o copy.lz ${bad5_lz} ${bad3_lz} ${bad4_lz} || fail=1
"${LZIPRECOVER}" -df copy.lz || fail=1
cmp in copy || fail=1
cmp ${in} copy || fail=1
printf .
"${LZIPRECOVER}" -R in.lz || fail=1
"${LZIPRECOVER}" -R ${in_lz} || fail=1
printf .
"${LZIPRECOVER}" -R -o copy.lz bad2.lz -q
"${LZIPRECOVER}" -R -o copy.lz ${bad2_lz} -q
if [ $? != 2 ] ; then fail=1 ; printf - ; else printf . ; fi
"${LZIPRECOVER}" -R -o copy.lz bad1.lz || fail=1
"${LZIPRECOVER}" -R -o copy.lz ${bad1_lz} || fail=1
"${LZIPRECOVER}" -df copy.lz || fail=1
cmp in copy || fail=1
cmp ${in} copy || fail=1
printf .
cat in3.lz > out || framework_failure
printf "garbage" >> out || fail=1
"${LZIPRECOVER}" -s out -o out.lz || fail=1
cat ${in_lz} ${in_lz} ${in_lz} > copy || framework_failure
printf "garbage" >> copy || fail=1
"${LZIPRECOVER}" -s -o copy.lz copy || fail=1
for i in 1 2 3 ; do
"${LZIPRECOVER}" -cd rec0000${i}out.lz > copy || fail=1
cmp in copy || fail=1
"${LZIPRECOVER}" -cd rec0000${i}copy.lz > copy || fail=1
cmp ${in} copy || fail=1
printf .
done

17
testsuite/test921-1921.txt Normal file
View file

@ -0,0 +1,17 @@
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.