lzip/plzip
1
0
Fork 0
Code Issues Activity

Adding upstream version 1.2~rc1.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-24 04:04:02 +01:00
parent 79610bce87
commit 945344fb87
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
17 changed files with 288 additions and 217 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
ChangeLog
View file

@ -1,3 +1,8 @@
2014-05-08 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.2-rc1 released.
* Minor changes.
2014-01-20 Antonio Diaz Diaz <antonio@gnu.org> 2014-01-20 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.2-pre1 released. * Version 1.2-pre1 released.

12
INSTALL
View file

@ -45,12 +45,12 @@ the main archive.
Another way Another way
----------- -----------
You can also compile plzip into a separate directory. To do this, you You can also compile plzip into a separate directory.
must use a version of 'make' that supports the 'VPATH' variable, such To do this, you must use a version of 'make' that supports the 'VPATH'
as GNU 'make'. 'cd' to the directory where you want the object files variable, such as GNU 'make'. 'cd' to the directory where you want the
and executables to go and run the 'configure' script. 'configure' object files and executables to go and run the 'configure' script.
automatically checks for the source code in '.', in '..' and in the 'configure' automatically checks for the source code in '.', in '..' and
directory that 'configure' is in. in the directory that 'configure' is in.
'configure' recognizes the option '--srcdir=DIR' to control where to 'configure' recognizes the option '--srcdir=DIR' to control where to
look for the sources. Usually 'configure' can determine that directory look for the sources. Usually 'configure' can determine that directory

8
Makefile.in
View file

@ -18,16 +18,16 @@ objs = arg_parser.o file_index.o compress.o dec_stdout.o dec_stream.o \
all : $(progname) all : $(progname)
$(progname) : $(objs) $(progname) : $(objs)
$(CXX) $(LDFLAGS) -o $@ $(objs) $(LIBS) $(CXX) $(CXXFLAGS) $(LDFLAGS) -o $@ $(objs) $(LIBS)
$(progname)_profiled : $(objs) $(progname)_profiled : $(objs)
$(CXX) $(LDFLAGS) -pg -o $@ $(objs) $(LIBS) $(CXX) $(CXXFLAGS) $(LDFLAGS) -pg -o $@ $(objs) $(LIBS)
main.o : main.cc main.o : main.cc
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -DPROGVERSION=\"$(pkgversion)\" -c -o $@ $< $(CXX) $(CXXFLAGS) $(CPPFLAGS) -DPROGVERSION=\"$(pkgversion)\" -c -o $@ $<
%.o : %.cc %.o : %.cc
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -c -o $@ $< $(CXX) $(CXXFLAGS) $(CPPFLAGS) -c -o $@ $<
$(objs) : Makefile $(objs) : Makefile
arg_parser.o : arg_parser.h arg_parser.o : arg_parser.h

39
README
View file

@ -13,29 +13,34 @@ but on files of only a few MB plzip is no faster than lzip.
Plzip uses the lzip file format; the files produced by plzip are fully Plzip uses the lzip file format; the files produced by plzip are fully
compatible with lzip-1.4 or newer, and can be rescued with lziprecover. compatible with lzip-1.4 or newer, and can be rescued with lziprecover.
The lzip file format is designed for long-term data archiving and The lzip file format is designed for long-term data archiving, taking
provides very safe integrity checking. The member trailer stores the into account both data integrity and decoder availability:
32-bit CRC of the original data, the size of the original data and the
size of the member. These values, together with the value remaining in
the range decoder and the end-of-stream marker, provide a 4 factor
integrity checking which guarantees that the decompressed version of the
data is identical to the original. This guards against corruption of the
compressed data, and against undetected bugs in plzip (hopefully very
unlikely). The chances of data corruption going undetected are
microscopic. Be aware, though, that the check occurs upon decompression,
so it can only tell you that something is wrong. It can't help you
recover the original uncompressed data.
If you ever need to recover data from a damaged lzip file, try the * The lzip format provides very safe integrity checking and some data
lziprecover program. Lziprecover makes lzip files resistant to bit-flip recovery means. The lziprecover program can repair bit-flip errors
(one of the most common forms of data corruption), and provides data (one of the most common forms of data corruption) in lzip files,
recovery capabilities, including error-checked merging of damaged copies and provides data recovery capabilities, including error-checked
of a file. merging of damaged copies of a file.
* The lzip format is as simple as possible (but not simpler). The
lzip manual provides the code of a simple decompressor along with a
detailed explanation of how it works, so that with the only help of
the lzip manual it would be possible for a digital archaeologist to
extract the data from a lzip file long after quantum computers
eventually render LZMA obsolete.
* Additionally lzip is copylefted, which guarantees that it will
remain free forever.
Plzip uses the same well-defined exit status values used by lzip and Plzip uses the same well-defined exit status values used by lzip and
bzip2, which makes it safer than compressors returning ambiguous warning bzip2, which makes it safer than compressors returning ambiguous warning
values (like gzip) when it is used as a back end for tar or zutils. values (like gzip) when it is used as a back end for tar or zutils.
Plzip will automatically use the smallest possible dictionary size for
each file without exceeding the given limit. Keep in mind that the
decompression memory requirement is affected at compression time by the
choice of dictionary size limit.
When compressing, plzip replaces every file given in the command line When compressing, plzip replaces every file given in the command line
with a compressed version of itself, with the name "original_name.lz". with a compressed version of itself, with the name "original_name.lz".
When decompressing, plzip attempts to guess the name for the decompressed When decompressing, plzip attempts to guess the name for the decompressed

5
arg_parser.cc
View file

@ -120,7 +120,7 @@ bool Arg_parser::parse_short_option( const char * const opt, const char * const
if( index < 0 ) if( index < 0 )
{ {
error_ = "invalid option -- "; error_ += c; error_ = "invalid option -- '"; error_ += c; error_ += '\'';
return false; return false;
} }
@ -135,7 +135,8 @@ bool Arg_parser::parse_short_option( const char * const opt, const char * const
{ {
if( !arg || !arg[0] ) if( !arg || !arg[0] )
{ {
error_ = "option requires an argument -- "; error_ += c; error_ = "option requires an argument -- '"; error_ += c;
error_ += '\'';
return false; return false;
} }
data.back().argument = arg; ++argind; cind = 0; data.back().argument = arg; ++argind; cind = 0;

8
compress.cc
View file

@ -52,7 +52,7 @@ int readblock( const int fd, uint8_t * const buf, const int size )
const int n = read( fd, buf + size - rest, rest ); const int n = read( fd, buf + size - rest, rest );
if( n > 0 ) rest -= n; if( n > 0 ) rest -= n;
else if( n == 0 ) break; // EOF else if( n == 0 ) break; // EOF
else if( errno != EINTR && errno != EAGAIN ) break; else if( errno != EINTR ) break;
errno = 0; errno = 0;
} }
return size - rest; return size - rest;
@ -70,7 +70,7 @@ int writeblock( const int fd, const uint8_t * const buf, const int size )
{ {
const int n = write( fd, buf + size - rest, rest ); const int n = write( fd, buf + size - rest, rest );
if( n > 0 ) rest -= n; if( n > 0 ) rest -= n;
else if( n < 0 && errno != EINTR && errno != EAGAIN ) break; else if( n < 0 && errno != EINTR ) break;
errno = 0; errno = 0;
} }
return size - rest; return size - rest;
@ -421,7 +421,7 @@ extern "C" void * cworker( void * arg )
} }
if( LZ_compress_close( encoder ) < 0 ) if( LZ_compress_close( encoder ) < 0 )
{ pp( "LZ_compress_close failed" ); cleanup_and_fail(); } { pp( "LZ_compress_close failed." ); cleanup_and_fail(); }
if( verbosity >= 2 && packet->size > 0 ) show_progress( packet->size ); if( verbosity >= 2 && packet->size > 0 ) show_progress( packet->size );
packet->data = new_data; packet->data = new_data;
@ -440,7 +440,7 @@ void muxer( Packet_courier & courier, const Pretty_print & pp, const int outfd )
while( true ) while( true )
{ {
courier.deliver_packets( packet_vector ); courier.deliver_packets( packet_vector );
if( packet_vector.size() == 0 ) break; // all workers exited if( packet_vector.empty() ) break; // all workers exited
for( unsigned i = 0; i < packet_vector.size(); ++i ) for( unsigned i = 0; i < packet_vector.size(); ++i )
{ {

2
configure vendored
View file

@ -6,7 +6,7 @@
# to copy, distribute and modify it. # to copy, distribute and modify it.
pkgname=plzip pkgname=plzip
pkgversion=1.2-pre1 pkgversion=1.2-rc1
progname=plzip progname=plzip
srctrigger=doc/${pkgname}.texi srctrigger=doc/${pkgname}.texi

18
dec_stdout.cc
View file

@ -172,10 +172,10 @@ extern "C" void * dworker_o( void * arg )
LZ_Decoder * const decoder = LZ_decompress_open(); LZ_Decoder * const decoder = LZ_decompress_open();
if( !new_data || !ibuffer || !decoder || if( !new_data || !ibuffer || !decoder ||
LZ_decompress_errno( decoder ) != LZ_ok ) LZ_decompress_errno( decoder ) != LZ_ok )
{ pp( "Not enough memory" ); cleanup_and_fail(); } { pp( "Not enough memory." ); cleanup_and_fail(); }
int new_pos = 0; int new_pos = 0;
for( int i = worker_id; i < file_index.members(); i += num_workers ) for( long i = worker_id; i < file_index.members(); i += num_workers )
{ {
long long member_pos = file_index.mblock( i ).pos(); long long member_pos = file_index.mblock( i ).pos();
long long member_rest = file_index.mblock( i ).size(); long long member_rest = file_index.mblock( i ).size();
@ -193,7 +193,7 @@ extern "C" void * dworker_o( void * arg )
member_pos += size; member_pos += size;
member_rest -= size; member_rest -= size;
if( LZ_decompress_write( decoder, ibuffer, size ) != size ) if( LZ_decompress_write( decoder, ibuffer, size ) != size )
internal_error( "library error (LZ_decompress_write)" ); internal_error( "library error (LZ_decompress_write)." );
} }
if( member_rest <= 0 ) { LZ_decompress_finish( decoder ); break; } if( member_rest <= 0 ) { LZ_decompress_finish( decoder ); break; }
} }
@ -205,7 +205,7 @@ extern "C" void * dworker_o( void * arg )
cleanup_and_fail( decompress_read_error( decoder, pp, worker_id ) ); cleanup_and_fail( decompress_read_error( decoder, pp, worker_id ) );
new_pos += rd; new_pos += rd;
if( new_pos > max_packet_size ) if( new_pos > max_packet_size )
internal_error( "opacket size exceeded in worker" ); internal_error( "opacket size exceeded in worker." );
if( new_pos == max_packet_size || if( new_pos == max_packet_size ||
LZ_decompress_finished( decoder ) == 1 ) LZ_decompress_finished( decoder ) == 1 )
{ {
@ -217,7 +217,7 @@ extern "C" void * dworker_o( void * arg )
courier.collect_packet( opacket, worker_id ); courier.collect_packet( opacket, worker_id );
new_pos = 0; new_pos = 0;
new_data = new( std::nothrow ) uint8_t[max_packet_size]; new_data = new( std::nothrow ) uint8_t[max_packet_size];
if( !new_data ) { pp( "Not enough memory" ); cleanup_and_fail(); } if( !new_data ) { pp( "Not enough memory." ); cleanup_and_fail(); }
} }
if( LZ_decompress_finished( decoder ) == 1 ) if( LZ_decompress_finished( decoder ) == 1 )
{ {
@ -236,9 +236,9 @@ extern "C" void * dworker_o( void * arg )
delete[] ibuffer; delete[] new_data; delete[] ibuffer; delete[] new_data;
if( LZ_decompress_member_position( decoder ) != 0 ) if( LZ_decompress_member_position( decoder ) != 0 )
{ pp( "Error, some data remains in decoder" ); cleanup_and_fail(); } { pp( "Error, some data remains in decoder." ); cleanup_and_fail(); }
if( LZ_decompress_close( decoder ) < 0 ) if( LZ_decompress_close( decoder ) < 0 )
{ pp( "LZ_decompress_close failed" ); cleanup_and_fail(); } { pp( "LZ_decompress_close failed." ); cleanup_and_fail(); }
courier.worker_finished(); courier.worker_finished();
return 0; return 0;
} }
@ -278,7 +278,7 @@ int dec_stdout( const int num_workers, const int infd, const int outfd,
Worker_arg * worker_args = new( std::nothrow ) Worker_arg[num_workers]; Worker_arg * worker_args = new( std::nothrow ) Worker_arg[num_workers];
pthread_t * worker_threads = new( std::nothrow ) pthread_t[num_workers]; pthread_t * worker_threads = new( std::nothrow ) pthread_t[num_workers];
if( !worker_args || !worker_threads ) if( !worker_args || !worker_threads )
{ pp( "Not enough memory" ); cleanup_and_fail(); } { pp( "Not enough memory." ); cleanup_and_fail(); }
for( int i = 0; i < num_workers; ++i ) for( int i = 0; i < num_workers; ++i )
{ {
worker_args[i].file_index = &file_index; worker_args[i].file_index = &file_index;
@ -324,6 +324,6 @@ int dec_stdout( const int num_workers, const int infd, const int outfd,
courier.ocheck_counter, courier.ocheck_counter,
courier.owait_counter ); courier.owait_counter );
if( !courier.finished() ) internal_error( "courier not finished" ); if( !courier.finished() ) internal_error( "courier not finished." );
return 0; return 0;
} }

30
dec_stream.cc
View file

@ -249,7 +249,7 @@ extern "C" void * dsplitter_s( void * arg )
const int buffer_size = max_packet_size; const int buffer_size = max_packet_size;
const int base_buffer_size = tsize + buffer_size + hsize; const int base_buffer_size = tsize + buffer_size + hsize;
uint8_t * const base_buffer = new( std::nothrow ) uint8_t[base_buffer_size]; uint8_t * const base_buffer = new( std::nothrow ) uint8_t[base_buffer_size];
if( !base_buffer ) { pp( "Not enough memory" ); cleanup_and_fail(); } if( !base_buffer ) { pp( "Not enough memory." ); cleanup_and_fail(); }
uint8_t * const buffer = base_buffer + tsize; uint8_t * const buffer = base_buffer + tsize;
int size = readblock( infd, buffer, buffer_size + hsize ) - hsize; int size = readblock( infd, buffer, buffer_size + hsize ) - hsize;
@ -257,10 +257,10 @@ extern "C" void * dsplitter_s( void * arg )
if( size != buffer_size && errno ) if( size != buffer_size && errno )
{ pp(); show_error( "Read error", errno ); cleanup_and_fail(); } { pp(); show_error( "Read error", errno ); cleanup_and_fail(); }
if( size + hsize < min_member_size ) if( size + hsize < min_member_size )
{ pp( "Input file is too short" ); cleanup_and_fail( 2 ); } { pp( "Input file is too short." ); cleanup_and_fail( 2 ); }
const File_header & header = *(File_header *)buffer; const File_header & header = *(File_header *)buffer;
if( !header.verify_magic() ) if( !header.verify_magic() )
{ pp( "Bad magic number (file not in lzip format)" ); cleanup_and_fail( 2 ); } { pp( "Bad magic number (file not in lzip format)." ); cleanup_and_fail( 2 ); }
if( !header.verify_version() ) if( !header.verify_version() )
{ {
if( verbosity >= 0 ) if( verbosity >= 0 )
@ -293,7 +293,7 @@ extern "C" void * dsplitter_s( void * arg )
cleanup_and_fail( 2 ); cleanup_and_fail( 2 );
} }
uint8_t * const data = new( std::nothrow ) uint8_t[newpos - pos]; uint8_t * const data = new( std::nothrow ) uint8_t[newpos - pos];
if( !data ) { pp( "Not enough memory" ); cleanup_and_fail(); } if( !data ) { pp( "Not enough memory." ); cleanup_and_fail(); }
std::memcpy( data, buffer + pos, newpos - pos ); std::memcpy( data, buffer + pos, newpos - pos );
courier.receive_packet( data, newpos - pos ); courier.receive_packet( data, newpos - pos );
courier.receive_packet( 0, 0 ); // end of member token courier.receive_packet( 0, 0 ); // end of member token
@ -306,7 +306,7 @@ extern "C" void * dsplitter_s( void * arg )
if( at_stream_end ) if( at_stream_end )
{ {
uint8_t * data = new( std::nothrow ) uint8_t[size + hsize - pos]; uint8_t * data = new( std::nothrow ) uint8_t[size + hsize - pos];
if( !data ) { pp( "Not enough memory" ); cleanup_and_fail(); } if( !data ) { pp( "Not enough memory." ); cleanup_and_fail(); }
std::memcpy( data, buffer + pos, size + hsize - pos ); std::memcpy( data, buffer + pos, size + hsize - pos );
courier.receive_packet( data, size + hsize - pos ); courier.receive_packet( data, size + hsize - pos );
courier.receive_packet( 0, 0 ); // end of member token courier.receive_packet( 0, 0 ); // end of member token
@ -316,7 +316,7 @@ extern "C" void * dsplitter_s( void * arg )
{ {
partial_member_size += buffer_size - pos; partial_member_size += buffer_size - pos;
uint8_t * data = new( std::nothrow ) uint8_t[buffer_size - pos]; uint8_t * data = new( std::nothrow ) uint8_t[buffer_size - pos];
if( !data ) { pp( "Not enough memory" ); cleanup_and_fail(); } if( !data ) { pp( "Not enough memory." ); cleanup_and_fail(); }
std::memcpy( data, buffer + pos, buffer_size - pos ); std::memcpy( data, buffer + pos, buffer_size - pos );
courier.receive_packet( data, buffer_size - pos ); courier.receive_packet( data, buffer_size - pos );
} }
@ -352,7 +352,7 @@ extern "C" void * dworker_s( void * arg )
uint8_t * new_data = new( std::nothrow ) uint8_t[max_packet_size]; uint8_t * new_data = new( std::nothrow ) uint8_t[max_packet_size];
LZ_Decoder * const decoder = LZ_decompress_open(); LZ_Decoder * const decoder = LZ_decompress_open();
if( !new_data || !decoder || LZ_decompress_errno( decoder ) != LZ_ok ) if( !new_data || !decoder || LZ_decompress_errno( decoder ) != LZ_ok )
{ pp( "Not enough memory" ); cleanup_and_fail(); } { pp( "Not enough memory." ); cleanup_and_fail(); }
int new_pos = 0; int new_pos = 0;
bool trailing_garbage_found = false; bool trailing_garbage_found = false;
@ -369,10 +369,10 @@ extern "C" void * dworker_s( void * arg )
{ {
const int wr = LZ_decompress_write( decoder, ipacket->data + written, const int wr = LZ_decompress_write( decoder, ipacket->data + written,
ipacket->size - written ); ipacket->size - written );
if( wr < 0 ) internal_error( "library error (LZ_decompress_write)" ); if( wr < 0 ) internal_error( "library error (LZ_decompress_write)." );
written += wr; written += wr;
if( written > ipacket->size ) if( written > ipacket->size )
internal_error( "ipacket size exceeded in worker" ); internal_error( "ipacket size exceeded in worker." );
} }
while( !trailing_garbage_found ) // read and pack decompressed data while( !trailing_garbage_found ) // read and pack decompressed data
{ {
@ -387,7 +387,7 @@ extern "C" void * dworker_s( void * arg )
} }
else new_pos += rd; else new_pos += rd;
if( new_pos > max_packet_size ) if( new_pos > max_packet_size )
internal_error( "opacket size exceeded in worker" ); internal_error( "opacket size exceeded in worker." );
if( new_pos == max_packet_size || trailing_garbage_found || if( new_pos == max_packet_size || trailing_garbage_found ||
LZ_decompress_finished( decoder ) == 1 ) LZ_decompress_finished( decoder ) == 1 )
{ {
@ -399,7 +399,7 @@ extern "C" void * dworker_s( void * arg )
courier.collect_packet( opacket, worker_id ); courier.collect_packet( opacket, worker_id );
new_pos = 0; new_pos = 0;
new_data = new( std::nothrow ) uint8_t[max_packet_size]; new_data = new( std::nothrow ) uint8_t[max_packet_size];
if( !new_data ) { pp( "Not enough memory" ); cleanup_and_fail(); } if( !new_data ) { pp( "Not enough memory." ); cleanup_and_fail(); }
} }
if( trailing_garbage_found || if( trailing_garbage_found ||
LZ_decompress_finished( decoder ) == 1 ) LZ_decompress_finished( decoder ) == 1 )
@ -422,9 +422,9 @@ extern "C" void * dworker_s( void * arg )
delete[] new_data; delete[] new_data;
if( LZ_decompress_member_position( decoder ) != 0 ) if( LZ_decompress_member_position( decoder ) != 0 )
{ pp( "Error, some data remains in decoder" ); cleanup_and_fail(); } { pp( "Error, some data remains in decoder." ); cleanup_and_fail(); }
if( LZ_decompress_close( decoder ) < 0 ) if( LZ_decompress_close( decoder ) < 0 )
{ pp( "LZ_decompress_close failed" ); cleanup_and_fail(); } { pp( "LZ_decompress_close failed." ); cleanup_and_fail(); }
return 0; return 0;
} }
@ -481,7 +481,7 @@ int dec_stream( const int num_workers, const int infd, const int outfd,
Worker_arg * worker_args = new( std::nothrow ) Worker_arg[num_workers]; Worker_arg * worker_args = new( std::nothrow ) Worker_arg[num_workers];
pthread_t * worker_threads = new( std::nothrow ) pthread_t[num_workers]; pthread_t * worker_threads = new( std::nothrow ) pthread_t[num_workers];
if( !worker_args || !worker_threads ) if( !worker_args || !worker_threads )
{ pp( "Not enough memory" ); cleanup_and_fail(); } { pp( "Not enough memory." ); cleanup_and_fail(); }
for( int i = 0; i < num_workers; ++i ) for( int i = 0; i < num_workers; ++i )
{ {
worker_args[i].courier = &courier; worker_args[i].courier = &courier;
@ -529,6 +529,6 @@ int dec_stream( const int num_workers, const int infd, const int outfd,
courier.ocheck_counter, courier.ocheck_counter,
courier.owait_counter ); courier.owait_counter );
if( !courier.finished() ) internal_error( "courier not finished" ); if( !courier.finished() ) internal_error( "courier not finished." );
return 0; return 0;
} }

20
decompress.cc
View file

@ -50,7 +50,7 @@ int preadblock( const int fd, uint8_t * const buf, const int size,
const int n = pread( fd, buf + size - rest, rest, pos + size - rest ); const int n = pread( fd, buf + size - rest, rest, pos + size - rest );
if( n > 0 ) rest -= n; if( n > 0 ) rest -= n;
else if( n == 0 ) break; // EOF else if( n == 0 ) break; // EOF
else if( errno != EINTR && errno != EAGAIN ) break; else if( errno != EINTR ) break;
errno = 0; errno = 0;
} }
return size - rest; return size - rest;
@ -69,7 +69,7 @@ int pwriteblock( const int fd, const uint8_t * const buf, const int size,
{ {
const int n = pwrite( fd, buf + size - rest, rest, pos + size - rest ); const int n = pwrite( fd, buf + size - rest, rest, pos + size - rest );
if( n > 0 ) rest -= n; if( n > 0 ) rest -= n;
else if( n < 0 && errno != EINTR && errno != EAGAIN ) break; else if( n < 0 && errno != EINTR ) break;
errno = 0; errno = 0;
} }
return size - rest; return size - rest;
@ -122,9 +122,9 @@ extern "C" void * dworker( void * arg )
LZ_Decoder * const decoder = LZ_decompress_open(); LZ_Decoder * const decoder = LZ_decompress_open();
if( !ibuffer || !obuffer || !decoder || if( !ibuffer || !obuffer || !decoder ||
LZ_decompress_errno( decoder ) != LZ_ok ) LZ_decompress_errno( decoder ) != LZ_ok )
{ pp( "Not enough memory" ); cleanup_and_fail(); } { pp( "Not enough memory." ); cleanup_and_fail(); }
for( int i = worker_id; i < file_index.members(); i += num_workers ) for( long i = worker_id; i < file_index.members(); i += num_workers )
{ {
long long data_pos = file_index.dblock( i ).pos(); long long data_pos = file_index.dblock( i ).pos();
long long data_rest = file_index.dblock( i ).size(); long long data_rest = file_index.dblock( i ).size();
@ -144,7 +144,7 @@ extern "C" void * dworker( void * arg )
member_pos += size; member_pos += size;
member_rest -= size; member_rest -= size;
if( LZ_decompress_write( decoder, ibuffer, size ) != size ) if( LZ_decompress_write( decoder, ibuffer, size ) != size )
internal_error( "library error (LZ_decompress_write)" ); internal_error( "library error (LZ_decompress_write)." );
} }
if( member_rest <= 0 ) { LZ_decompress_finish( decoder ); break; } if( member_rest <= 0 ) { LZ_decompress_finish( decoder ); break; }
} }
@ -160,7 +160,7 @@ extern "C" void * dworker( void * arg )
{ {
pp(); pp();
if( verbosity >= 0 ) if( verbosity >= 0 )
std::fprintf( stderr, "Write error in worker %d: %s\n", std::fprintf( stderr, "Write error in worker %d: %s.\n",
worker_id, std::strerror( errno ) ); worker_id, std::strerror( errno ) );
cleanup_and_fail(); cleanup_and_fail();
} }
@ -173,7 +173,7 @@ extern "C" void * dworker( void * arg )
if( LZ_decompress_finished( decoder ) == 1 ) if( LZ_decompress_finished( decoder ) == 1 )
{ {
if( data_rest != 0 ) if( data_rest != 0 )
internal_error( "final data_rest != 0" ); internal_error( "final data_rest is not zero." );
LZ_decompress_reset( decoder ); // prepare for new member LZ_decompress_reset( decoder ); // prepare for new member
break; break;
} }
@ -184,9 +184,9 @@ extern "C" void * dworker( void * arg )
delete[] obuffer; delete[] ibuffer; delete[] obuffer; delete[] ibuffer;
if( LZ_decompress_member_position( decoder ) != 0 ) if( LZ_decompress_member_position( decoder ) != 0 )
{ pp( "Error, some data remains in decoder" ); cleanup_and_fail(); } { pp( "Error, some data remains in decoder." ); cleanup_and_fail(); }
if( LZ_decompress_close( decoder ) < 0 ) if( LZ_decompress_close( decoder ) < 0 )
{ pp( "LZ_decompress_close failed" ); cleanup_and_fail(); } { pp( "LZ_decompress_close failed." ); cleanup_and_fail(); }
return 0; return 0;
} }
@ -224,7 +224,7 @@ int decompress( int num_workers, const int infd, const int outfd,
Worker_arg * worker_args = new( std::nothrow ) Worker_arg[num_workers]; Worker_arg * worker_args = new( std::nothrow ) Worker_arg[num_workers];
pthread_t * worker_threads = new( std::nothrow ) pthread_t[num_workers]; pthread_t * worker_threads = new( std::nothrow ) pthread_t[num_workers];
if( !worker_args || !worker_threads ) if( !worker_args || !worker_threads )
{ pp( "Not enough memory" ); cleanup_and_fail(); } { pp( "Not enough memory." ); cleanup_and_fail(); }
for( int i = 0; i < num_workers; ++i ) for( int i = 0; i < num_workers; ++i )
{ {
worker_args[i].file_index = &file_index; worker_args[i].file_index = &file_index;

12
doc/plzip.1
View file

@ -1,7 +1,7 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.37.1. .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.37.1.
.TH PLZIP "1" "January 2014" "Plzip 1.2-pre1" "User Commands" .TH PLZIP "1" "May 2014" "plzip 1.2-rc1" "User Commands"
.SH NAME .SH NAME
Plzip \- reduces the size of files plzip \- reduces the size of files
.SH SYNOPSIS .SH SYNOPSIS
.B plzip .B plzip
[\fIoptions\fR] [\fIfiles\fR] [\fIoptions\fR] [\fIfiles\fR]
@ -85,20 +85,20 @@ Plzip home page: http://www.nongnu.org/lzip/plzip.html
Copyright \(co 2009 Laszlo Ersek. Copyright \(co 2009 Laszlo Ersek.
.br .br
Copyright \(co 2014 Antonio Diaz Diaz. Copyright \(co 2014 Antonio Diaz Diaz.
Using Lzlib 1.6\-pre1 Using Lzlib 1.6\-rc1
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
.br .br
This is free software: you are free to change and redistribute it. This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. There is NO WARRANTY, to the extent permitted by law.
.SH "SEE ALSO" .SH "SEE ALSO"
The full documentation for The full documentation for
.B Plzip .B plzip
is maintained as a Texinfo manual. If the is maintained as a Texinfo manual. If the
.B info .B info
and and
.B Plzip .B plzip
programs are properly installed at your site, the command programs are properly installed at your site, the command
.IP .IP
.B info Plzip .B info plzip
.PP .PP
should give you access to the complete manual. should give you access to the complete manual.

69
doc/plzip.info
View file

@ -11,7 +11,7 @@ File: plzip.info, Node: Top, Next: Introduction, Up: (dir)
Plzip Manual Plzip Manual
************ ************
This manual is for Plzip (version 1.2-pre1, 20 January 2014). This manual is for Plzip (version 1.2-rc1, 8 May 2014).
* Menu: * Menu:
@ -48,29 +48,46 @@ but on files of only a few MB plzip is no faster than lzip.
fully compatible with lzip-1.4 or newer, and can be rescued with fully compatible with lzip-1.4 or newer, and can be rescued with
lziprecover. lziprecover.
The lzip file format is designed for long-term data archiving and The lzip file format is designed for long-term data archiving, taking
provides very safe integrity checking. The member trailer stores the into account both data integrity and decoder availability:
32-bit CRC of the original data, the size of the original data and the
size of the member. These values, together with the value remaining in
the range decoder and the end-of-stream marker, provide a 4 factor
integrity checking which guarantees that the decompressed version of the
data is identical to the original. This guards against corruption of the
compressed data, and against undetected bugs in plzip (hopefully very
unlikely). The chances of data corruption going undetected are
microscopic. Be aware, though, that the check occurs upon decompression,
so it can only tell you that something is wrong. It can't help you
recover the original uncompressed data.
If you ever need to recover data from a damaged lzip file, try the * The lzip format provides very safe integrity checking and some data
lziprecover program. Lziprecover makes lzip files resistant to bit-flip recovery means. The lziprecover program can repair bit-flip errors
(one of the most common forms of data corruption), and provides data (one of the most common forms of data corruption) in lzip files,
recovery capabilities, including error-checked merging of damaged copies and provides data recovery capabilities, including error-checked
of a file. merging of damaged copies of a file.
* The lzip format is as simple as possible (but not simpler). The
lzip manual provides the code of a simple decompressor along with
a detailed explanation of how it works, so that with the only help
of the lzip manual it would be possible for a digital
archaeologist to extract the data from a lzip file long after
quantum computers eventually render LZMA obsolete.
* Additionally lzip is copylefted, which guarantees that it will
remain free forever.
The member trailer stores the 32-bit CRC of the original data, the
size of the original data and the size of the member. These values,
together with the value remaining in the range decoder and the
end-of-stream marker, provide a 4 factor integrity checking which
guarantees that the decompressed version of the data is identical to
the original. This guards against corruption of the compressed data,
and against undetected bugs in plzip (hopefully very unlikely). The
chances of data corruption going undetected are microscopic. Be aware,
though, that the check occurs upon decompression, so it can only tell
you that something is wrong. It can't help you recover the original
uncompressed data.
Plzip uses the same well-defined exit status values used by lzip and Plzip uses the same well-defined exit status values used by lzip and
bzip2, which makes it safer than compressors returning ambiguous warning bzip2, which makes it safer than compressors returning ambiguous warning
values (like gzip) when it is used as a back end for tar or zutils. values (like gzip) when it is used as a back end for tar or zutils.
Plzip will automatically use the smallest possible dictionary size
for each file without exceeding the given limit. Keep in mind that the
decompression memory requirement is affected at compression time by the
choice of dictionary size limit.
When compressing, plzip replaces every file given in the command line When compressing, plzip replaces every file given in the command line
with a compressed version of itself, with the name "original_name.lz". with a compressed version of itself, with the name "original_name.lz".
When decompressing, plzip attempts to guess the name for the When decompressing, plzip attempts to guess the name for the
@ -215,7 +232,7 @@ The format for running plzip is:
'--dictionary-size=BYTES' '--dictionary-size=BYTES'
Set the dictionary size limit in bytes. Valid values range from 4 Set the dictionary size limit in bytes. Valid values range from 4
KiB to 512 MiB. Plzip will use the smallest possible dictionary KiB to 512 MiB. Plzip will use the smallest possible dictionary
size for each member without exceeding this limit. Note that size for each file without exceeding this limit. Note that
dictionary sizes are quantized. If the specified size does not dictionary sizes are quantized. If the specified size does not
match one of the valid sizes, it will be rounded upwards by adding match one of the valid sizes, it will be rounded upwards by adding
up to (BYTES / 16) to it. up to (BYTES / 16) to it.
@ -401,13 +418,13 @@ Concept index
 
Tag Table: Tag Table:
Node: Top221 Node: Top221
Node: Introduction878 Node: Introduction872
Node: Program design4650 Node: Program design5441
Node: Invoking plzip5704 Node: Invoking plzip6495
Ref: --data-size6149 Ref: --data-size6940
Node: File format11300 Node: File format12089
Node: Problems13805 Node: Problems14594
Node: Concept index14334 Node: Concept index15123
 
End Tag Table End Tag Table

61
doc/plzip.texi
View file

@ -6,8 +6,8 @@
@finalout @finalout
@c %**end of header @c %**end of header
@set UPDATED 20 January 2014 @set UPDATED 8 May 2014
@set VERSION 1.2-pre1 @set VERSION 1.2-rc1
@dircategory Data Compression @dircategory Data Compression
@direntry @direntry
@ -68,29 +68,50 @@ but on files of only a few MB plzip is no faster than lzip.
Plzip uses the lzip file format; the files produced by plzip are fully Plzip uses the lzip file format; the files produced by plzip are fully
compatible with lzip-1.4 or newer, and can be rescued with lziprecover. compatible with lzip-1.4 or newer, and can be rescued with lziprecover.
The lzip file format is designed for long-term data archiving and The lzip file format is designed for long-term data archiving, taking
provides very safe integrity checking. The member trailer stores the into account both data integrity and decoder availability:
32-bit CRC of the original data, the size of the original data and the
size of the member. These values, together with the value remaining in
the range decoder and the end-of-stream marker, provide a 4 factor
integrity checking which guarantees that the decompressed version of the
data is identical to the original. This guards against corruption of the
compressed data, and against undetected bugs in plzip (hopefully very
unlikely). The chances of data corruption going undetected are
microscopic. Be aware, though, that the check occurs upon decompression,
so it can only tell you that something is wrong. It can't help you
recover the original uncompressed data.
If you ever need to recover data from a damaged lzip file, try the @itemize @bullet
lziprecover program. Lziprecover makes lzip files resistant to bit-flip @item
(one of the most common forms of data corruption), and provides data The lzip format provides very safe integrity checking and some data
recovery capabilities, including error-checked merging of damaged copies recovery means. The lziprecover program can repair bit-flip errors (one
of a file. of the most common forms of data corruption) in lzip files, and provides
data recovery capabilities, including error-checked merging of damaged
copies of a file.
@item
The lzip format is as simple as possible (but not simpler). The lzip
manual provides the code of a simple decompressor along with a detailed
explanation of how it works, so that with the only help of the lzip
manual it would be possible for a digital archaeologist to extract the
data from a lzip file long after quantum computers eventually render
LZMA obsolete.
@item
Additionally lzip is copylefted, which guarantees that it will remain
free forever.
@end itemize
The member trailer stores the 32-bit CRC of the original data, the size
of the original data and the size of the member. These values, together
with the value remaining in the range decoder and the end-of-stream
marker, provide a 4 factor integrity checking which guarantees that the
decompressed version of the data is identical to the original. This
guards against corruption of the compressed data, and against undetected
bugs in plzip (hopefully very unlikely). The chances of data corruption
going undetected are microscopic. Be aware, though, that the check
occurs upon decompression, so it can only tell you that something is
wrong. It can't help you recover the original uncompressed data.
Plzip uses the same well-defined exit status values used by lzip and Plzip uses the same well-defined exit status values used by lzip and
bzip2, which makes it safer than compressors returning ambiguous warning bzip2, which makes it safer than compressors returning ambiguous warning
values (like gzip) when it is used as a back end for tar or zutils. values (like gzip) when it is used as a back end for tar or zutils.
Plzip will automatically use the smallest possible dictionary size for
each file without exceeding the given limit. Keep in mind that the
decompression memory requirement is affected at compression time by the
choice of dictionary size limit.
When compressing, plzip replaces every file given in the command line When compressing, plzip replaces every file given in the command line
with a compressed version of itself, with the name "original_name.lz". with a compressed version of itself, with the name "original_name.lz".
When decompressing, plzip attempts to guess the name for the decompressed When decompressing, plzip attempts to guess the name for the decompressed
@ -238,7 +259,7 @@ Quiet operation. Suppress all messages.
@itemx --dictionary-size=@var{bytes} @itemx --dictionary-size=@var{bytes}
Set the dictionary size limit in bytes. Valid values range from 4 KiB to Set the dictionary size limit in bytes. Valid values range from 4 KiB to
512 MiB. Plzip will use the smallest possible dictionary size for each 512 MiB. Plzip will use the smallest possible dictionary size for each
member without exceeding this limit. Note that dictionary sizes are file without exceeding this limit. Note that dictionary sizes are
quantized. If the specified size does not match one of the valid sizes, quantized. If the specified size does not match one of the valid sizes,
it will be rounded upwards by adding up to (@var{bytes} / 16) to it. it will be rounded upwards by adding up to (@var{bytes} / 16) to it.

89
file_index.cc
View file

@ -38,95 +38,84 @@ int seek_read( const int fd, uint8_t * const buf, const int size,
} }
const char * format_num( unsigned long long num, void File_index::set_errno_error( const char * const msg )
unsigned long long limit = -1ULL,
const int set_prefix = 0 )
{ {
const char * const si_prefix[8] = error_ = msg; error_ += std::strerror( errno ); error_ += '.';
{ "k", "M", "G", "T", "P", "E", "Z", "Y" }; retval_ = 1;
const char * const binary_prefix[8] = }
{ "Ki", "Mi", "Gi", "Ti", "Pi", "Ei", "Zi", "Yi" };
static bool si = true;
static char buf[32];
if( set_prefix ) si = ( set_prefix > 0 ); void File_index::set_num_error( const char * const msg1, unsigned long long num,
const unsigned factor = ( si ? 1000 : 1024 ); const char * const msg2 )
const char * const * prefix = ( si ? si_prefix : binary_prefix ); {
const char * p = ""; char buf[80];
bool exact = ( num % factor == 0 ); snprintf( buf, sizeof buf, "%s%llu%s", msg1, num, msg2 );
error_ = buf;
for( int i = 0; i < 8 && ( num > limit || ( exact && num >= factor ) ); ++i ) retval_ = 2;
{ num /= factor; if( num % factor != 0 ) exact = false; p = prefix[i]; }
snprintf( buf, sizeof buf, "%llu %s", num, p );
return buf;
} }
File_index::File_index( const int infd ) : retval_( 0 ) File_index::File_index( const int infd )
: retval_( 0 )
{ {
const long long isize = lseek( infd, 0, SEEK_END ); const long long isize = lseek( infd, 0, SEEK_END );
if( isize < 0 ) if( isize < 0 )
{ error_ = "Input file is not seekable :"; { set_errno_error( "Input file is not seekable :" ); return; }
error_ += std::strerror( errno ); retval_ = 1; return; }
if( isize > INT64_MAX )
{ error_ = "Input file is too long (2^63 bytes or more)";
retval_ = 2; return; }
long long pos = isize; // always points to a header or EOF
File_header header;
File_trailer trailer;
if( isize < min_member_size ) if( isize < min_member_size )
{ error_ = "Input file is too short"; retval_ = 2; return; } { error_ = "Input file is too short."; retval_ = 2; return; }
if( isize > INT64_MAX )
{ error_ = "Input file is too long (2^63 bytes or more).";
retval_ = 2; return; }
File_header header;
if( seek_read( infd, header.data, File_header::size, 0 ) != File_header::size ) if( seek_read( infd, header.data, File_header::size, 0 ) != File_header::size )
{ error_ = "Error reading member header :"; { set_errno_error( "Error reading member header :" ); return; }
error_ += std::strerror( errno ); retval_ = 1; return; }
if( !header.verify_magic() ) if( !header.verify_magic() )
{ error_ = "Bad magic number (file not in lzip format)"; { error_ = "Bad magic number (file not in lzip format).";
retval_ = 2; return; } retval_ = 2; return; }
if( !header.verify_version() ) if( !header.verify_version() )
{ error_ = "Version "; error_ += format_num( header.version() ); { set_num_error( "Version ", header.version(),
error_ += "member format not supported"; retval_ = 2; return; } " member format not supported." ); return; }
long long pos = isize; // always points to a header or to EOF
while( pos >= min_member_size ) while( pos >= min_member_size )
{ {
File_trailer trailer;
if( seek_read( infd, trailer.data, File_trailer::size, if( seek_read( infd, trailer.data, File_trailer::size,
pos - File_trailer::size ) != File_trailer::size ) pos - File_trailer::size ) != File_trailer::size )
{ error_ = "Error reading member trailer :"; { set_errno_error( "Error reading member trailer :" ); break; }
error_ += std::strerror( errno ); retval_ = 1; break; }
const long long member_size = trailer.member_size(); const long long member_size = trailer.member_size();
if( member_size < min_member_size || member_size > pos ) if( member_size < min_member_size || member_size > pos )
{ {
if( member_vector.size() == 0 ) // maybe trailing garbage if( member_vector.empty() ) // maybe trailing garbage
{ --pos; continue; } { --pos; continue; }
error_ = "Member size in trailer is corrupt at pos "; set_num_error( "Member size in trailer is corrupt at pos ", pos - 8 );
error_ += format_num( pos - 8 ); retval_ = 2; break; break;
} }
if( seek_read( infd, header.data, File_header::size, if( seek_read( infd, header.data, File_header::size,
pos - member_size ) != File_header::size ) pos - member_size ) != File_header::size )
{ error_ = "Error reading member header :"; { set_errno_error( "Error reading member header :" ); break; }
error_ += std::strerror( errno ); retval_ = 1; break; }
if( !header.verify_magic() || !header.verify_version() ) if( !header.verify_magic() || !header.verify_version() )
{ {
if( member_vector.size() == 0 ) // maybe trailing garbage if( member_vector.empty() ) // maybe trailing garbage
{ --pos; continue; } { --pos; continue; }
error_ = "Bad header at pos "; set_num_error( "Bad header at pos ", pos - member_size );
error_ += format_num( pos - member_size ); retval_ = 2; break; break;
} }
if( member_vector.size() == 0 && isize - pos > File_header::size && if( member_vector.empty() && isize - pos > File_header::size &&
seek_read( infd, header.data, File_header::size, pos ) == File_header::size && seek_read( infd, header.data, File_header::size, pos ) == File_header::size &&
header.verify_magic() && header.verify_version() ) header.verify_magic() && header.verify_version() )
{ {
error_ = "Last member in input file is truncated or corrupt"; error_ = "Last member in input file is truncated or corrupt.";
retval_ = 2; break; retval_ = 2; break;
} }
pos -= member_size; pos -= member_size;
member_vector.push_back( Member( 0, trailer.data_size(), member_vector.push_back( Member( 0, trailer.data_size(),
pos, member_size ) ); pos, member_size ) );
} }
if( pos != 0 || member_vector.size() == 0 ) if( pos != 0 || member_vector.empty() )
{ {
member_vector.clear(); member_vector.clear();
if( retval_ == 0 ) { error_ = "Can't create file index"; retval_ = 2; } if( retval_ == 0 ) { error_ = "Can't create file index."; retval_ = 2; }
return; return;
} }
std::reverse( member_vector.begin(), member_vector.end() ); std::reverse( member_vector.begin(), member_vector.end() );
@ -136,7 +125,7 @@ File_index::File_index( const int infd ) : retval_( 0 )
if( end < 0 || end > INT64_MAX ) if( end < 0 || end > INT64_MAX )
{ {
member_vector.clear(); member_vector.clear();
error_ = "Data in input file is too long (2^63 bytes or more)"; error_ = "Data in input file is too long (2^63 bytes or more).";
retval_ = 2; return; retval_ = 2; return;
} }
member_vector[i+1].dblock.pos( end ); member_vector[i+1].dblock.pos( end );

14
file_index.h
View file

@ -55,9 +55,14 @@ class File_index
std::string error_; std::string error_;
int retval_; int retval_;
public: void set_errno_error( const char * const msg );
File_index( const int infd ); void set_num_error( const char * const msg1, unsigned long long num,
const char * const msg2 = "." );
public:
explicit File_index( const int infd );
long members() const { return member_vector.size(); }
const std::string & error() const { return error_; } const std::string & error() const { return error_; }
int retval() const { return retval_; } int retval() const { return retval_; }
@ -69,9 +74,8 @@ public:
{ if( member_vector.size() ) return member_vector.back().mblock.end(); { if( member_vector.size() ) return member_vector.back().mblock.end();
else return 0; } else return 0; }
const Block & dblock( const int i ) const const Block & dblock( const long i ) const
{ return member_vector[i].dblock; } { return member_vector[i].dblock; }
const Block & mblock( const int i ) const const Block & mblock( const long i ) const
{ return member_vector[i].mblock; } { return member_vector[i].mblock; }
int members() const { return (int)member_vector.size(); }
}; };

27
main.cc
View file

@ -143,7 +143,7 @@ void show_help( const long num_online )
void show_version() void show_version()
{ {
std::printf( "%s %s\n", Program_name, PROGVERSION ); std::printf( "%s %s\n", program_name, PROGVERSION );
std::printf( "Copyright (C) 2009 Laszlo Ersek.\n" std::printf( "Copyright (C) 2009 Laszlo Ersek.\n"
"Copyright (C) %s Antonio Diaz Diaz.\n", program_year ); "Copyright (C) %s Antonio Diaz Diaz.\n", program_year );
std::printf( "Using Lzlib %s\n", LZ_version() ); std::printf( "Using Lzlib %s\n", LZ_version() );
@ -245,8 +245,7 @@ int open_instream( const char * const name, struct stat * const in_statsp,
} }
else else
{ {
do infd = open( name, O_RDONLY | O_BINARY ); infd = open( name, O_RDONLY | O_BINARY );
while( infd < 0 && errno == EINTR );
if( infd < 0 ) if( infd < 0 )
{ {
if( verbosity >= 0 ) if( verbosity >= 0 )
@ -308,8 +307,7 @@ bool open_outstream( const bool force )
int flags = O_CREAT | O_WRONLY | O_BINARY; int flags = O_CREAT | O_WRONLY | O_BINARY;
if( force ) flags |= O_TRUNC; else flags |= O_EXCL; if( force ) flags |= O_TRUNC; else flags |= O_EXCL;
do outfd = open( output_filename.c_str(), flags, outfd_mode ); outfd = open( output_filename.c_str(), flags, outfd_mode );
while( outfd < 0 && errno == EINTR );
if( outfd < 0 && verbosity >= 0 ) if( outfd < 0 && verbosity >= 0 )
{ {
if( errno == EEXIST ) if( errno == EEXIST )
@ -402,7 +400,7 @@ void Pretty_print::operator()( const char * const msg ) const
std::fprintf( stderr, " " ); std::fprintf( stderr, " " );
if( !msg ) std::fflush( stderr ); if( !msg ) std::fflush( stderr );
} }
if( msg ) std::fprintf( stderr, "%s.\n", msg ); if( msg ) std::fprintf( stderr, "%s\n", msg );
} }
} }
@ -415,7 +413,7 @@ void show_error( const char * const msg, const int errcode, const bool help )
{ {
std::fprintf( stderr, "%s: %s", program_name, msg ); std::fprintf( stderr, "%s: %s", program_name, msg );
if( errcode > 0 ) if( errcode > 0 )
std::fprintf( stderr, ": %s", std::strerror( errcode ) ); std::fprintf( stderr, ": %s.", std::strerror( errcode ) );
std::fprintf( stderr, "\n" ); std::fprintf( stderr, "\n" );
} }
if( help ) if( help )
@ -428,7 +426,7 @@ void show_error( const char * const msg, const int errcode, const bool help )
void internal_error( const char * const msg ) void internal_error( const char * const msg )
{ {
if( verbosity >= 0 ) if( verbosity >= 0 )
std::fprintf( stderr, "%s: internal error: %s.\n", program_name, msg ); std::fprintf( stderr, "%s: internal error: %s\n", program_name, msg );
std::exit( 3 ); std::exit( 3 );
} }
@ -467,7 +465,7 @@ void show_progress( const int packet_size,
static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
if( p ) // initialize static vars if( p ) // initialize static vars
{ csize = cfile_size; pos = 0; pp = p; return; } { csize = cfile_size; pos = 0; pp = p; }
if( pp ) if( pp )
{ {
xlock( &mutex ); xlock( &mutex );
@ -513,7 +511,7 @@ int main( const int argc, const char * const argv[] )
invocation_name = argv[0]; invocation_name = argv[0];
if( LZ_version()[0] != LZ_version_string[0] ) if( LZ_version()[0] != LZ_version_string[0] )
internal_error( "bad library version" ); internal_error( "bad library version." );
const long num_online = std::max( 1L, sysconf( _SC_NPROCESSORS_ONLN ) ); const long num_online = std::max( 1L, sysconf( _SC_NPROCESSORS_ONLN ) );
long max_workers = sysconf( _SC_THREAD_THREADS_MAX ); long max_workers = sysconf( _SC_THREAD_THREADS_MAX );
@ -589,7 +587,7 @@ int main( const int argc, const char * const argv[] )
case 't': program_mode = m_test; break; case 't': program_mode = m_test; break;
case 'v': if( verbosity < 4 ) ++verbosity; break; case 'v': if( verbosity < 4 ) ++verbosity; break;
case 'V': show_version(); return 0; case 'V': show_version(); return 0;
default : internal_error( "uncaught option" ); default : internal_error( "uncaught option." );
} }
} // end process options } // end process options
@ -604,7 +602,8 @@ int main( const int argc, const char * const argv[] )
if( data_size <= 0 ) if( data_size <= 0 )
data_size = 2 * std::max( 65536, encoder_options.dictionary_size ); data_size = 2 * std::max( 65536, encoder_options.dictionary_size );
else if( data_size < encoder_options.dictionary_size ) else if( data_size < encoder_options.dictionary_size )
encoder_options.dictionary_size = std::max( data_size, LZ_min_dictionary_size() ); encoder_options.dictionary_size =
std::max( data_size, LZ_min_dictionary_size() );
if( num_workers <= 0 ) if( num_workers <= 0 )
num_workers = std::min( num_online, max_workers ); num_workers = std::min( num_online, max_workers );
@ -629,13 +628,13 @@ int main( const int argc, const char * const argv[] )
struct stat in_stats; struct stat in_stats;
output_filename.clear(); output_filename.clear();
if( !filenames[i].size() || filenames[i] == "-" ) if( filenames[i].empty() || filenames[i] == "-" )
{ {
input_filename.clear(); input_filename.clear();
infd = STDIN_FILENO; infd = STDIN_FILENO;
if( program_mode != m_test ) if( program_mode != m_test )
{ {
if( to_stdout || !default_output_filename.size() ) if( to_stdout || default_output_filename.empty() )
outfd = STDOUT_FILENO; outfd = STDOUT_FILENO;
else else
{ {

86
testsuite/check.sh
View file

@ -12,7 +12,7 @@ testdir=`cd "$1" ; pwd`
LZIP="${objdir}"/plzip LZIP="${objdir}"/plzip
framework_failure() { echo "failure in testing framework" ; exit 1 ; } framework_failure() { echo "failure in testing framework" ; exit 1 ; }
if [ ! -x "${LZIP}" ] ; then if [ ! -f "${LZIP}" ] || [ ! -x "${LZIP}" ] ; then
echo "${LZIP}: cannot execute" echo "${LZIP}: cannot execute"
exit 1 exit 1
fi fi
@ -27,22 +27,29 @@ fail=0
printf "testing plzip-%s..." "$2" printf "testing plzip-%s..." "$2"
"${LZIP}" -cqs-1 in > /dev/null "${LZIP}" -cqm4 in > /dev/null
if [ $? = 1 ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 1 ] ; then printf . ; else printf - ; fail=1 ; fi
"${LZIP}" -cqs0 in > /dev/null
if [ $? = 1 ] ; then printf . ; else fail=1 ; printf - ; fi
"${LZIP}" -cqs4095 in > /dev/null
if [ $? = 1 ] ; then printf . ; else fail=1 ; printf - ; fi
"${LZIP}" -cqm274 in > /dev/null "${LZIP}" -cqm274 in > /dev/null
if [ $? = 1 ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 1 ] ; then printf . ; else printf - ; fail=1 ; fi
"${LZIP}" -tq in "${LZIP}" -cqs-1 in > /dev/null
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 1 ] ; then printf . ; else printf - ; fail=1 ; fi
"${LZIP}" -tq < in "${LZIP}" -cqs0 in > /dev/null
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 1 ] ; then printf . ; else printf - ; fail=1 ; fi
"${LZIP}" -cqs4095 in > /dev/null
if [ $? = 1 ] ; then printf . ; else printf - ; fail=1 ; fi
"${LZIP}" -cqs513MiB in > /dev/null
if [ $? = 1 ] ; then printf . ; else printf - ; fail=1 ; fi
printf " in: Bad magic number (file not in lzip format).\n" > msg
"${LZIP}" -t in 2> out
if [ $? = 2 ] && cmp out msg ; then printf . ; else printf - ; fail=1 ; fi
printf " (stdin): Bad magic number (file not in lzip format).\n" > msg
"${LZIP}" -t < in 2> out
if [ $? = 2 ] && cmp out msg ; then printf . ; else printf - ; fail=1 ; fi
rm -f out msg
"${LZIP}" -cdq in "${LZIP}" -cdq in
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 2 ] ; then printf . ; else printf - ; fail=1 ; fi
"${LZIP}" -cdq < in "${LZIP}" -cdq < in
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 2 ] ; then printf . ; else printf - ; fail=1 ; fi
dd if="${in_lz}" bs=1 count=6 2> /dev/null | "${LZIP}" -tq dd if="${in_lz}" bs=1 count=6 2> /dev/null | "${LZIP}" -tq
if [ $? = 2 ] ; then printf . ; else printf - ; fail=1 ; fi if [ $? = 2 ] ; then printf . ; else printf - ; fail=1 ; fi
dd if="${in_lz}" bs=1 count=20 2> /dev/null | "${LZIP}" -tq dd if="${in_lz}" bs=1 count=20 2> /dev/null | "${LZIP}" -tq
@ -53,8 +60,38 @@ if [ $? = 2 ] ; then printf . ; else printf - ; fail=1 ; fi
cmp in copy || fail=1 cmp in copy || fail=1
printf . printf .
cat "${in_lz}" > copy.lz || framework_failure
printf "to be overwritten" > copy || framework_failure
"${LZIP}" -df copy.lz || fail=1
cmp in copy || fail=1
printf .
printf "to be overwritten" > copy || framework_failure
"${LZIP}" -df -o copy < "${in_lz}" || fail=1
cmp in copy || fail=1
printf .
"${LZIP}" < in > anyothername || fail=1
"${LZIP}" -d anyothername || fail=1
cmp in anyothername.out || fail=1
printf .
cat in in > in2 || framework_failure
"${LZIP}" -o copy2 < in2 || fail=1
"${LZIP}" -t copy2.lz || fail=1
printf .
"${LZIP}" -cd copy2.lz > copy2 || fail=1
cmp in2 copy2 || fail=1
printf .
printf "garbage" >> copy2.lz || framework_failure
printf "to be overwritten" > copy2 || framework_failure
"${LZIP}" -df copy2.lz || fail=1
cmp in2 copy2 || fail=1
printf .
"${LZIP}" -cfq "${in_lz}" > out "${LZIP}" -cfq "${in_lz}" > out
if [ $? = 1 ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 1 ] ; then printf . ; else printf - ; fail=1 ; fi
"${LZIP}" -cF "${in_lz}" > out || fail=1 "${LZIP}" -cF "${in_lz}" > out || fail=1
"${LZIP}" -cd out | "${LZIP}" -d > copy || fail=1 "${LZIP}" -cd out | "${LZIP}" -d > copy || fail=1
cmp in copy || fail=1 cmp in copy || fail=1
@ -79,7 +116,6 @@ printf .
for i in s4Ki 0 1 2 3 4 5 6 7 8 9 ; do for i in s4Ki 0 1 2 3 4 5 6 7 8 9 ; do
"${LZIP}" -$i < in > out || fail=1 "${LZIP}" -$i < in > out || fail=1
printf "garbage" >> out || fail=1
"${LZIP}" -d < out > copy || fail=1 "${LZIP}" -d < out > copy || fail=1
cmp in copy || fail=1 cmp in copy || fail=1
done done
@ -87,17 +123,11 @@ printf .
for i in s4Ki 0 1 2 3 4 5 6 7 8 9 ; do for i in s4Ki 0 1 2 3 4 5 6 7 8 9 ; do
"${LZIP}" -f -$i -o out < in || fail=1 "${LZIP}" -f -$i -o out < in || fail=1
printf "g" >> out.lz || fail=1
"${LZIP}" -df -o copy < out.lz || fail=1 "${LZIP}" -df -o copy < out.lz || fail=1
cmp in copy || fail=1 cmp in copy || fail=1
done done
printf . printf .
"${LZIP}" < in > anyothername || fail=1
"${LZIP}" -d anyothername || fail=1
cmp in anyothername.out || fail=1
printf .
cat in in in in > in4 || framework_failure cat in in in in > in4 || framework_failure
for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 ; do for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 ; do
"${LZIP}" -c -s4Ki -B8Ki -n$i in4 > out4.lz || fail=1 "${LZIP}" -c -s4Ki -B8Ki -n$i in4 > out4.lz || fail=1
@ -122,9 +152,9 @@ cat "${in_lz}" > ingin.lz || framework_failure
printf "g" >> ingin.lz || framework_failure printf "g" >> ingin.lz || framework_failure
cat "${in_lz}" >> ingin.lz || framework_failure cat "${in_lz}" >> ingin.lz || framework_failure
"${LZIP}" -tq ingin.lz "${LZIP}" -tq ingin.lz
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 2 ] ; then printf . ; else printf - ; fail=1 ; fi
"${LZIP}" -cdq ingin.lz > out "${LZIP}" -cdq ingin.lz > out
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 2 ] ; then printf . ; else printf - ; fail=1 ; fi
"${LZIP}" -t < ingin.lz || fail=1 "${LZIP}" -t < ingin.lz || fail=1
"${LZIP}" -d < ingin.lz > copy || fail=1 "${LZIP}" -d < ingin.lz > copy || fail=1
cmp in copy || fail=1 cmp in copy || fail=1
@ -132,13 +162,13 @@ printf .
dd if="${in_lz}" bs=1024 count=10 > trunc.lz 2> /dev/null || framework_failure dd if="${in_lz}" bs=1024 count=10 > trunc.lz 2> /dev/null || framework_failure
"${LZIP}" -tq trunc.lz "${LZIP}" -tq trunc.lz
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 2 ] ; then printf . ; else printf - ; fail=1 ; fi
"${LZIP}" -cdq trunc.lz > out "${LZIP}" -cdq trunc.lz > out
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 2 ] ; then printf . ; else printf - ; fail=1 ; fi
"${LZIP}" -tq < trunc.lz "${LZIP}" -tq < trunc.lz
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 2 ] ; then printf . ; else printf - ; fail=1 ; fi
"${LZIP}" -dq < trunc.lz > out "${LZIP}" -dq < trunc.lz > out
if [ $? = 2 ] ; then printf . ; else fail=1 ; printf - ; fi if [ $? = 2 ] ; then printf . ; else printf - ; fail=1 ; fi
echo echo
if [ ${fail} = 0 ] ; then if [ ${fail} = 0 ] ; then