lzip/plzip
1
0
Fork 0
Code Issues Activity

Merging upstream version 0.7.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-24 03:33:34 +01:00
parent fe9dba7d15
commit 238d2661e5
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
18 changed files with 317 additions and 287 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
AUTHORS
View file

@ -1,4 +1 @@
Plzip was written by Laszlo Ersek and Antonio Diaz Diaz.
Plzip uses a simplified version of the LZMA algorithm.
The original LZMA algorithm was designed by Igor Pavlov.

30
ChangeLog
View file

@ -1,9 +1,21 @@
2010-12-03 Antonio Diaz Diaz <ant_diaz@teleline.es>
* Version 0.7 released.
* Match length limits set by options -1 to -9 have been changed
to match those of lzip 1.11.
* decompress.cc: A limit has been set on the number of packets
produced by workers to limit the amount of memory used.
* main.cc (open_instream): Do not show the message
" and `--stdout' was not specified" for directories, etc.
* main.cc: Fixed warning about fchown return value being ignored.
* testsuite: `test1' renamed to `test.txt'. Added new tests.
2010-03-20 Antonio Diaz Diaz <ant_diaz@teleline.es>
* Version 0.6 released.
* Small portability fixes.
* Added chapter "Program Design" and description of option
"--threads" to manual.
* Added chapter `Program Design' and description of option
`--threads' to manual.
* Debug stats have been fixed.
2010-02-10 Antonio Diaz Diaz <ant_diaz@teleline.es>
@ -21,19 +33,19 @@
2010-01-24 Antonio Diaz Diaz <ant_diaz@teleline.es>
* Version 0.3 released.
* Implemented option "--data-size".
* Implemented option `--data-size'.
* Output file is now removed if plzip is interrupted.
* This version automatically chooses the smallest possible
dictionary size for each member during compression, saving
memory during decompression.
* main.cc: New constant "o_binary".
* main.cc: New constant `o_binary'.
2010-01-17 Antonio Diaz Diaz <ant_diaz@teleline.es>
* Version 0.2 released.
* Implemented option "--dictionary-size".
* Implemented option "--match-length".
* "lacos_rbtree" has been replaced with a circular buffer.
* Implemented option `--dictionary-size'.
* Implemented option `--match-length'.
* `lacos_rbtree' has been replaced with a circular buffer.
2009-12-05 Antonio Diaz Diaz <ant_diaz@teleline.es>
@ -54,5 +66,5 @@
Copyright (C) 2009, 2010 Antonio Diaz Diaz.
This file is a collection of facts, and thus it is not copyrightable,
but just in case, I give you unlimited permission to copy, distribute
and modify it.
but just in case, you have unlimited permission to copy, distribute and
modify it.

4
INSTALL
View file

@ -1,9 +1,9 @@
Requirements
------------
You will need a C++ compiler and the lzlib compression library installed.
I use gcc 4.3.4 and 3.3.6, but the code should compile with any
I use gcc 4.3.5 and 3.3.6, but the code should compile with any
standards compliant compiler.
Lzlib must be version 0.8 or newer.
Lzlib must be version 1.0 or newer.
Gcc is available at http://gcc.gnu.org.
Lzlib is available at http://www.nongnu.org/lzip/lzlib.html.

14
Makefile.in
View file

@ -17,10 +17,10 @@ objs = arg_parser.o compress.o decompress.o main.o
all : $(progname)
$(progname) : $(objs)
$(CXX) $(LDFLAGS) -o $(progname) $(objs) $(LIBS)
$(CXX) $(LDFLAGS) -o $@ $^ $(LIBS)
$(progname)_profiled : $(objs)
$(CXX) $(LDFLAGS) -pg -o $(progname)_profiled $(objs) $(LIBS)
$(CXX) $(LDFLAGS) -pg -o $@ $^ $(LIBS)
main.o : main.cc
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -DPROGVERSION=\"$(pkgversion)\" -c -o $@ $<
@ -44,14 +44,14 @@ $(VPATH)/doc/$(pkgname).info : $(VPATH)/doc/$(pkgname).texinfo
man : $(VPATH)/doc/$(progname).1
$(VPATH)/doc/$(progname).1 : $(progname)
help2man -n 'data compressor based on the LZMA algorithm' \
-o $(VPATH)/doc/$(progname).1 ./$(progname)
help2man -n 'reduces the size of files' \
-o $@ ./$(progname)
Makefile : $(VPATH)/configure $(VPATH)/Makefile.in
./config.status
check : all
@$(VPATH)/testsuite/check.sh $(VPATH)/testsuite
@$(VPATH)/testsuite/check.sh $(VPATH)/testsuite $(pkgversion)
install : all install-info install-man
if [ ! -d "$(DESTDIR)$(bindir)" ] ; then $(INSTALL_DIR) "$(DESTDIR)$(bindir)" ; fi
@ -94,8 +94,8 @@ dist : doc
$(DISTNAME)/doc/$(pkgname).info \
$(DISTNAME)/doc/$(pkgname).texinfo \
$(DISTNAME)/testsuite/check.sh \
$(DISTNAME)/testsuite/test1 \
$(DISTNAME)/testsuite/test1.lz \
$(DISTNAME)/testsuite/test.txt \
$(DISTNAME)/testsuite/test_v[01].lz \
$(DISTNAME)/*.h \
$(DISTNAME)/*.cc
rm -f $(DISTNAME)

14
NEWS
View file

@ -1,6 +1,12 @@
Changes in version 0.6:
Changes in version 0.7:
Some small portability problems have been fixed.
Match length limits set by options -1 to -9 have been changed to match
those of lzip 1.11.
The chapter "Program Design" and a description of option "--threads"
have been added to the manual.
A limit has been set on the number of packets produced by decompresor
worker threads to limit the amount of memory used.
Do not show the message "and `--stdout' was not specified" for file
types that can't be read (directories, etc).
A warning about fchown's return value being ignored has been fixed.

12
README
View file

@ -1,10 +1,10 @@
Description
Plzip is a massively parallel (multi-threaded), lossless data compressor
based on the LZMA algorithm, with very safe integrity checking and a
user interface similar to the one of gzip or bzip2. Plzip uses the lzip
file format; the files produced by plzip are fully compatible with
lzip-1.4 or newer.
based on the lzlib compression library, with very safe integrity
checking and a user interface similar to the one of bzip2, gzip or lzip.
Plzip uses the lzip file format; the files produced by plzip are fully
compatible with lzip-1.4 or newer.
Plzip is intended for faster compression/decompression of big files on
multiprocessor machines, which makes it specially well suited for
@ -18,5 +18,5 @@ This file is free documentation: you have unlimited permission to copy,
distribute and modify it.
The file Makefile.in is a data file used by configure to produce the
Makefile. It has the same copyright owner and permissions that this
file.
Makefile. It has the same copyright owner and permissions that configure
itself.

55
compress.cc
View file

@ -48,11 +48,14 @@
void xinit( pthread_cond_t * cond, pthread_mutex_t * mutex )
{
int errcode = pthread_mutex_init( mutex, 0 );
if( errcode ) { show_error( "pthread_mutex_init", errcode ); fatal(); }
errcode = pthread_cond_init( cond, 0 );
int errcode = pthread_cond_init( cond, 0 );
if( errcode ) { show_error( "pthread_cond_init", errcode ); fatal(); }
if( mutex )
{
errcode = pthread_mutex_init( mutex, 0 );
if( errcode ) { show_error( "pthread_mutex_init", errcode ); fatal(); }
}
}
@ -61,8 +64,11 @@ void xdestroy( pthread_cond_t * cond, pthread_mutex_t * mutex )
int errcode = pthread_cond_destroy( cond );
if( errcode ) { show_error( "pthread_cond_destroy", errcode ); fatal(); }
errcode = pthread_mutex_destroy( mutex );
if( errcode ) { show_error( "pthread_mutex_destroy", errcode ); fatal(); }
if( mutex )
{
errcode = pthread_mutex_destroy( mutex );
if( errcode ) { show_error( "pthread_mutex_destroy", errcode ); fatal(); }
}
}
@ -125,16 +131,16 @@ public:
private:
unsigned long long receive_id; // id assigned to next packet received
unsigned long long deliver_id; // id of next packet to be delivered
Slot_tally slot_tally;
Slot_tally slot_tally; // limits the number of input packets
std::queue< Packet * > packet_queue;
std::vector< Packet * > circular_buffer;
int num_working; // Number of workers still running
int num_working; // number of workers still running
const int num_slots; // max packets in circulation
pthread_mutex_t imutex;
pthread_cond_t iav_or_eof; // input packet available or splitter done
pthread_mutex_t omutex;
pthread_cond_t oav_or_exit; // output packet available or all workers exited
bool eof; // splitter done
bool eof; // splitter done
public:
Packet_courier( const int num_workers, const int slots )
@ -148,6 +154,8 @@ public:
~Packet_courier()
{ xdestroy( &iav_or_eof, &imutex ); xdestroy( &oav_or_exit, &omutex ); }
const Slot_tally & tally() const { return slot_tally; }
// make a packet with data received from splitter
void receive_packet( uint8_t * const data, const int size )
{
@ -182,10 +190,9 @@ public:
xunlock( &imutex );
if( ipacket == 0 )
{
// Notify muxer when last worker exits
// notify muxer when last worker exits
xlock( &omutex );
if( --num_working == 0 )
xsignal( &oav_or_exit );
if( --num_working == 0 ) xsignal( &oav_or_exit );
xunlock( &omutex );
}
return ipacket;
@ -198,7 +205,7 @@ public:
// id collision shouldn't happen
if( circular_buffer[opacket->id%num_slots] != 0 )
internal_error( "id collision in collect_packet" );
// Merge packet into circular buffer
// merge packet into circular buffer
circular_buffer[opacket->id%num_slots] = opacket;
if( opacket->id == deliver_id ) xsignal( &oav_or_exit );
xunlock( &omutex );
@ -240,8 +247,6 @@ public:
if( circular_buffer[i] != 0 ) return false;
return true;
}
const Slot_tally & tally() const { return slot_tally; }
};
@ -267,10 +272,10 @@ extern "C" void * csplitter( void * arg )
for( bool first_post = true; ; first_post = false )
{
uint8_t * data = new( std::nothrow ) uint8_t[data_size];
if( data == 0 ) { pp( "not enough memory" ); fatal(); }
if( data == 0 ) { pp( "Not enough memory" ); fatal(); }
const int size = readblock( infd, data, data_size );
if( size != data_size && errno )
{ pp(); show_error( "read error", errno ); fatal(); }
{ pp(); show_error( "Read error", errno ); fatal(); }
if( size > 0 || first_post ) // first packet can be empty
{
@ -314,7 +319,7 @@ extern "C" void * cworker( void * arg )
const int compr_size = 42 + packet->size + ( ( packet->size + 7 ) / 8 );
uint8_t * const new_data = new( std::nothrow ) uint8_t[compr_size];
if( new_data == 0 ) { pp( "not enough memory" ); fatal(); }
if( new_data == 0 ) { pp( "Not enough memory" ); fatal(); }
const int dict_size = std::max( LZ_min_dictionary_size(),
std::min( dictionary_size, packet->size ) );
LZ_Encoder * const encoder =
@ -322,7 +327,7 @@ extern "C" void * cworker( void * arg )
if( !encoder || LZ_compress_errno( encoder ) != LZ_ok )
{
if( !encoder || LZ_compress_errno( encoder ) == LZ_mem_error )
pp( "not enough memory. Try a smaller dictionary size" );
pp( "Not enough memory. Try a smaller dictionary size" );
else
internal_error( "invalid argument to encoder" );
fatal();
@ -386,7 +391,7 @@ void muxer( Packet_courier & courier, const Pretty_print & pp, const int outfd )
{
const int wr = writeblock( outfd, opacket->data, opacket->size );
if( wr != opacket->size )
{ pp(); show_error( "write error", errno ); fatal(); }
{ pp(); show_error( "Write error", errno ); fatal(); }
}
delete[] opacket->data;
delete opacket;
@ -416,7 +421,7 @@ int compress( const int data_size, const int dictionary_size,
pthread_t splitter_thread;
int errcode = pthread_create( &splitter_thread, 0, csplitter, &splitter_arg );
if( errcode )
{ show_error( "can't create splitter thread", errcode ); fatal(); }
{ show_error( "Can't create splitter thread", errcode ); fatal(); }
Worker_arg worker_arg;
worker_arg.courier = &courier;
@ -426,12 +431,12 @@ int compress( const int data_size, const int dictionary_size,
pthread_t * worker_threads = new( std::nothrow ) pthread_t[num_workers];
if( worker_threads == 0 )
{ pp( "not enough memory" ); fatal(); }
{ pp( "Not enough memory" ); fatal(); }
for( int i = 0; i < num_workers; ++i )
{
errcode = pthread_create( &worker_threads[i], 0, cworker, &worker_arg );
if( errcode )
{ show_error( "can't create worker threads", errcode ); fatal(); }
{ show_error( "Can't create worker threads", errcode ); fatal(); }
}
muxer( courier, pp, outfd );
@ -440,13 +445,13 @@ int compress( const int data_size, const int dictionary_size,
{
errcode = pthread_join( worker_threads[i], 0 );
if( errcode )
{ show_error( "can't join worker threads", errcode ); fatal(); }
{ show_error( "Can't join worker threads", errcode ); fatal(); }
}
delete[] worker_threads; worker_threads = 0;
errcode = pthread_join( splitter_thread, 0 );
if( errcode )
{ show_error( "can't join splitter thread", errcode ); fatal(); }
{ show_error( "Can't join splitter thread", errcode ); fatal(); }
if( verbosity >= 1 )
{

6
configure vendored
View file

@ -5,12 +5,12 @@
# This configure script is free software: you have unlimited permission
# to copy, distribute and modify it.
#
# Date of this version: 2010-03-20
# Date of this version: 2010-12-03
args=
no_create=
pkgname=plzip
pkgversion=0.6
pkgversion=0.7
progname=plzip
srctrigger=plzip.h
@ -80,7 +80,7 @@ while [ -n "$1" ] ; do
bindir=`echo ${optarg} | sed -e 's,/$,,'` ;;
--datadir* | --da*)
datadir=`echo ${optarg} | sed -e 's,/$,,'` ;;
--infodir* | --in*)
--infodir* | --inf*)
infodir=`echo ${optarg} | sed -e 's,/$,,'` ;;
--mandir* | --ma*)
mandir=`echo ${optarg} | sed -e 's,/$,,'` ;;

90
decompress.cc
View file

@ -38,6 +38,7 @@
namespace {
enum { max_packet_size = 1 << 20 };
long long in_size = 0;
long long out_size = 0;
@ -59,17 +60,18 @@ public:
private:
int receive_worker_id; // worker queue currently receiving packets
int deliver_worker_id; // worker queue currently delivering packets
Slot_tally slot_tally;
Slot_tally slot_tally; // limits the number of input packets
std::vector< std::queue< Packet * > > ipacket_queues;
std::vector< std::queue< Packet * > > opacket_queues;
int num_working; // Number of workers still running
const int num_workers; // Number of workers
const int num_slots; // max packets in circulation
int num_working; // number of workers still running
const int num_workers; // number of workers
int num_free; // remaining free output slots
pthread_mutex_t imutex;
pthread_cond_t iav_or_eof; // input packet available or splitter done
pthread_mutex_t omutex;
pthread_cond_t oav_or_exit; // output packet available or all workers exited
bool eof; // splitter done
pthread_cond_t slot_av; // free output slot available
bool eof; // splitter done
public:
Packet_courier( const int workers, const int slots )
@ -78,11 +80,19 @@ public:
receive_worker_id( 0 ), deliver_worker_id( 0 ),
slot_tally( slots ), ipacket_queues( workers ),
opacket_queues( workers ), num_working( workers ),
num_workers( workers ), num_slots( slots ), eof( false )
{ xinit( &iav_or_eof, &imutex ); xinit( &oav_or_exit, &omutex ); }
num_workers( workers ), num_free( 8 * slots ), eof( false )
{
xinit( &iav_or_eof, &imutex );
xinit( &oav_or_exit, &omutex ); xinit( &slot_av, 0 );
}
~Packet_courier()
{ xdestroy( &iav_or_eof, &imutex ); xdestroy( &oav_or_exit, &omutex ); }
{
xdestroy( &iav_or_eof, &imutex );
xdestroy( &oav_or_exit, &omutex ); xdestroy( &slot_av, 0 );
}
const Slot_tally & tally() const { return slot_tally; }
// make a packet with data received from splitter
// if data == 0, move to next queue
@ -123,10 +133,9 @@ public:
{ if( ipacket->data != 0 ) slot_tally.leave_slot(); }
else
{
// Notify muxer when last worker exits
// notify muxer when last worker exits
xlock( &omutex );
if( --num_working == 0 )
xsignal( &oav_or_exit );
if( --num_working == 0 ) xsignal( &oav_or_exit );
xunlock( &omutex );
}
return ipacket;
@ -136,6 +145,12 @@ public:
void collect_packet( Packet * const opacket, const int worker_id )
{
xlock( &omutex );
if( opacket->data != 0 )
{
while( worker_id != deliver_worker_id && num_free <= 0 )
xwait( &slot_av, &omutex );
--num_free;
}
opacket_queues[worker_id].push( opacket );
if( worker_id == deliver_worker_id ) xsignal( &oav_or_exit );
xunlock( &omutex );
@ -159,12 +174,14 @@ public:
if( opacket_queues[deliver_worker_id].empty() ) break;
opacket = opacket_queues[deliver_worker_id].front();
opacket_queues[deliver_worker_id].pop();
if( opacket->data != 0 ) break;
else
if( opacket->data != 0 )
{
if( ++deliver_worker_id >= num_workers ) deliver_worker_id = 0;
delete opacket; opacket = 0;
if( ++num_free == 1 ) xsignal( &slot_av );
break;
}
if( ++deliver_worker_id >= num_workers ) deliver_worker_id = 0;
xbroadcast( &slot_av ); // restart deliver_worker_id thread
delete opacket; opacket = 0;
}
xunlock( &omutex );
return opacket;
@ -187,8 +204,6 @@ public:
if( !opacket_queues[i].empty() ) return false;
return true;
}
const Slot_tally & tally() const { return slot_tally; }
};
@ -197,7 +212,6 @@ struct Splitter_arg
Packet_courier * courier;
const Pretty_print * pp;
int infd;
int packet_size;
};
@ -211,19 +225,19 @@ extern "C" void * dsplitter( void * arg )
const int infd = tmp.infd;
const int hsize = 6; // header size
const int tsize = 20; // trailer size
const int buffer_size = tmp.packet_size;
const int buffer_size = max_packet_size;
const int base_buffer_size = tsize + buffer_size + hsize;
uint8_t * const base_buffer = new( std::nothrow ) uint8_t[base_buffer_size];
if( base_buffer == 0 ) { pp( "not enough memory" ); fatal(); }
if( base_buffer == 0 ) { pp( "Not enough memory" ); fatal(); }
uint8_t * const buffer = base_buffer + tsize;
int size = readblock( infd, buffer, buffer_size + hsize ) - hsize;
bool at_stream_end = ( size < buffer_size );
if( size != buffer_size && errno )
{ pp(); show_error( "read error", errno ); fatal(); }
{ pp(); show_error( "Read error", errno ); fatal(); }
if( size <= tsize || buffer[0] != 'L' || buffer[1] != 'Z' ||
buffer[2] != 'I' || buffer[3] != 'P' )
{ pp( "bad magic number (file not in lzip format)" ); fatal(); }
{ pp( "Bad magic number (file not in lzip format)" ); fatal(); }
long long partial_member_size = 0;
while( true )
@ -239,7 +253,7 @@ extern "C" void * dsplitter( void * arg )
if( partial_member_size + newpos - pos == member_size )
{ // header found
uint8_t * data = new( std::nothrow ) uint8_t[newpos - pos];
if( data == 0 ) { pp( "not enough memory" ); fatal(); }
if( data == 0 ) { pp( "Not enough memory" ); fatal(); }
std::memcpy( data, buffer + pos, newpos - pos );
courier.receive_packet( data, newpos - pos );
courier.receive_packet( 0, 0 ); // end of member token
@ -251,7 +265,7 @@ extern "C" void * dsplitter( void * arg )
if( at_stream_end )
{
uint8_t * data = new( std::nothrow ) uint8_t[size + hsize - pos];
if( data == 0 ) { pp( "not enough memory" ); fatal(); }
if( data == 0 ) { pp( "Not enough memory" ); fatal(); }
std::memcpy( data, buffer + pos, size + hsize - pos );
courier.receive_packet( data, size + hsize - pos );
courier.receive_packet( 0, 0 ); // end of member token
@ -261,7 +275,7 @@ extern "C" void * dsplitter( void * arg )
{
partial_member_size += buffer_size - pos;
uint8_t * data = new( std::nothrow ) uint8_t[buffer_size - pos];
if( data == 0 ) { pp( "not enough memory" ); fatal(); }
if( data == 0 ) { pp( "Not enough memory" ); fatal(); }
std::memcpy( data, buffer + pos, buffer_size - pos );
courier.receive_packet( data, buffer_size - pos );
}
@ -269,7 +283,7 @@ extern "C" void * dsplitter( void * arg )
size = readblock( infd, buffer + hsize, buffer_size );
at_stream_end = ( size < buffer_size );
if( size != buffer_size && errno )
{ pp(); show_error( "read error", errno ); fatal(); }
{ pp(); show_error( "Read error", errno ); fatal(); }
}
delete[] base_buffer;
courier.finish(); // no more packets to send
@ -282,7 +296,6 @@ struct Worker_arg
Packet_courier * courier;
const Pretty_print * pp;
int worker_id;
int packet_size;
};
@ -294,12 +307,12 @@ extern "C" void * dworker( void * arg )
Packet_courier & courier = *tmp.courier;
const Pretty_print & pp = *tmp.pp;
const int worker_id = tmp.worker_id;
const int new_data_size = tmp.packet_size;
const int new_data_size = max_packet_size;
uint8_t * new_data = new( std::nothrow ) uint8_t[new_data_size];
LZ_Decoder * const decoder = LZ_decompress_open();
if( !new_data || !decoder || LZ_decompress_errno( decoder ) != LZ_ok )
{ pp( "not enough memory" ); fatal(); }
{ pp( "Not enough memory" ); fatal(); }
int new_pos = 0;
while( true )
@ -345,7 +358,7 @@ extern "C" void * dworker( void * arg )
courier.collect_packet( opacket, worker_id );
new_pos = 0;
new_data = new( std::nothrow ) uint8_t[new_data_size];
if( new_data == 0 ) { pp( "not enough memory" ); fatal(); }
if( new_data == 0 ) { pp( "Not enough memory" ); fatal(); }
}
if( LZ_decompress_finished( decoder ) == 1 )
{
@ -367,7 +380,7 @@ extern "C" void * dworker( void * arg )
delete[] new_data;
if( LZ_decompress_total_in_size( decoder ) != 0 )
{ pp( "error, remaining data in decoder" ); fatal(); }
{ pp( "Error, some data remains in decoder" ); fatal(); }
LZ_decompress_close( decoder );
return 0;
}
@ -388,7 +401,7 @@ void muxer( Packet_courier & courier, const Pretty_print & pp, const int outfd )
{
const int wr = writeblock( outfd, opacket->data, opacket->size );
if( wr != opacket->size )
{ pp(); show_error( "write error", errno ); fatal(); }
{ pp(); show_error( "Write error", errno ); fatal(); }
}
delete[] opacket->data;
delete opacket;
@ -406,33 +419,30 @@ int decompress( const int num_workers, const int num_slots,
{
in_size = 0;
out_size = 0;
const int packet_size = 1 << 20;
Packet_courier courier( num_workers, num_slots );
Splitter_arg splitter_arg;
splitter_arg.courier = &courier;
splitter_arg.pp = &pp;
splitter_arg.infd = infd;
splitter_arg.packet_size = packet_size;
pthread_t splitter_thread;
int errcode = pthread_create( &splitter_thread, 0, dsplitter, &splitter_arg );
if( errcode )
{ show_error( "can't create splitter thread", errcode ); fatal(); }
{ show_error( "Can't create splitter thread", errcode ); fatal(); }
Worker_arg * worker_args = new( std::nothrow ) Worker_arg[num_workers];
pthread_t * worker_threads = new( std::nothrow ) pthread_t[num_workers];
if( worker_args == 0 || worker_threads == 0 )
{ pp( "not enough memory" ); fatal(); }
{ pp( "Not enough memory" ); fatal(); }
for( int i = 0; i < num_workers; ++i )
{
worker_args[i].courier = &courier;
worker_args[i].pp = &pp;
worker_args[i].worker_id = i;
worker_args[i].packet_size = packet_size;
errcode = pthread_create( &worker_threads[i], 0, dworker, &worker_args[i] );
if( errcode )
{ show_error( "can't create worker threads", errcode ); fatal(); }
{ show_error( "Can't create worker threads", errcode ); fatal(); }
}
muxer( courier, pp, outfd );
@ -441,14 +451,14 @@ int decompress( const int num_workers, const int num_slots,
{
errcode = pthread_join( worker_threads[i], 0 );
if( errcode )
{ show_error( "can't join worker threads", errcode ); fatal(); }
{ show_error( "Can't join worker threads", errcode ); fatal(); }
}
delete[] worker_threads; worker_threads = 0;
delete[] worker_args; worker_args = 0;
errcode = pthread_join( splitter_thread, 0 );
if( errcode )
{ show_error( "can't join splitter thread", errcode ); fatal(); }
{ show_error( "Can't join splitter thread", errcode ); fatal(); }
if( verbosity >= 2 )
std::fprintf( stderr, "decompressed size %9lld, size %9lld. ",

8
doc/plzip.1
View file

@ -1,7 +1,7 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.37.1.
.TH PLZIP "1" "March 2010" "Plzip 0.6" "User Commands"
.TH PLZIP "1" "December 2010" "Plzip 0.7" "User Commands"
.SH NAME
Plzip \- data compressor based on the LZMA algorithm
Plzip \- reduces the size of files
.SH SYNOPSIS
.B plzip
[\fIoptions\fR] [\fIfiles\fR]
@ -31,7 +31,7 @@ overwrite existing output files
keep (don't delete) input files
.TP
\fB\-m\fR, \fB\-\-match\-length=\fR<n>
set match length limit in bytes [80]
set match length limit in bytes [36]
.TP
\fB\-n\fR, \fB\-\-threads=\fR<n>
set the number of (de)compression threads
@ -72,7 +72,7 @@ Plzip home page: http://www.nongnu.org/lzip/plzip.html
Copyright \(co 2009 Laszlo Ersek.
.br
Copyright \(co 2010 Antonio Diaz Diaz.
Using Lzlib 1.0\-rc1
Using Lzlib 1.1\-rc1
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
.br
This is free software: you are free to change and redistribute it.

84
doc/plzip.info
View file

@ -12,16 +12,16 @@ File: plzip.info, Node: Top, Next: Introduction, Up: (dir)
Plzip Manual
************
This manual is for Plzip (version 0.6, 20 March 2010).
This manual is for Plzip (version 0.7, 3 December 2010).
* Menu:
* Introduction:: Purpose and features of plzip
* Invoking Plzip:: Command line interface
* Program Design:: Internal structure of plzip
* File Format:: Detailed format of the compressed file
* Problems:: Reporting bugs
* Concept Index:: Index of concepts
* Introduction:: Purpose and features of plzip
* Invoking Plzip:: Command line interface
* Program Design:: Internal structure of plzip
* File Format:: Detailed format of the compressed file
* Problems:: Reporting bugs
* Concept Index:: Index of concepts
Copyright (C) 2009, 2010 Antonio Diaz Diaz.
@ -36,10 +36,10 @@ File: plzip.info, Node: Introduction, Next: Invoking Plzip, Prev: Top, Up: T
**************
Plzip is a massively parallel (multi-threaded), lossless data compressor
based on the LZMA algorithm, with very safe integrity checking and a
user interface similar to the one of gzip or bzip2. Plzip uses the lzip
file format; the files produced by plzip are fully compatible with
lzip-1.4 or newer.
based on the lzlib compression library, with very safe integrity
checking and a user interface similar to the one of bzip2, gzip or lzip.
Plzip uses the lzip file format; the files produced by plzip are fully
compatible with lzip-1.4 or newer.
Plzip is intended for faster compression/decompression of big files
on multiprocessor machines, which makes it specially well suited for
@ -98,80 +98,80 @@ The format for running plzip is:
Plzip supports the following options:
`--help'
`-h'
`--help'
Print an informative help message describing the options and exit.
`--version'
`-V'
`--version'
Print the version number of plzip on the standard output and exit.
`--data-size=SIZE'
`-B'
`--data-size=SIZE'
Set the input data block size in bytes. The input file will be
divided in chunks of this size before compression is performed.
Valid values range from 8KiB to 1GiB. Default value is two times
the dictionary size. Plzip will reduce the dictionary size if it
is larger than the chosen data size.
`--stdout'
`-c'
`--stdout'
Compress or decompress to standard output. Needed when reading
from a named pipe (fifo) or from a device.
`--decompress'
`-d'
`--decompress'
Decompress.
`--force'
`-f'
`--force'
Force overwrite of output file.
`--keep'
`-k'
`--keep'
Keep (don't delete) input files during compression or
decompression.
`--match-length=LENGTH'
`-m LENGTH'
`--match-length=LENGTH'
Set the match length limit in bytes. Valid values range from 5 to
273. Larger values usually give better compression ratios but
longer compression times.
`--threads=THREADS'
`-n THREADS'
`--threads=THREADS'
Set the number of worker threads. Valid values range from 1 to "as
many as your system can support". If this option is not used,
plzip tries to detect the number of processors in the system and
use it as default value.
`--output=FILE'
`-o FILE'
`--output=FILE'
When reading from standard input and `--stdout' has not been
specified, use `FILE' as the virtual name of the uncompressed
file. This produces a file named `FILE' when decompressing, and a
file named `FILE.lz' when compressing.
`--quiet'
`-q'
`--quiet'
Quiet operation. Suppress all messages.
`--dictionary-size=SIZE'
`-s SIZE'
`--dictionary-size=SIZE'
Set the dictionary size limit in bytes. Valid values range from
4KiB to 512MiB. Note that dictionary sizes are quantized. If the
specified size does not match one of the valid sizes, it will be
rounded upwards.
`--test'
`-t'
`--test'
Check integrity of the specified file(s), but don't decompress
them. This really performs a trial decompression and throws away
the result. Use `-tvv' or `-tvvv' to see information about the
file.
`--verbose'
`-v'
`--verbose'
Verbose mode. Show the compression ratio for each file processed.
Further -v's increase the verbosity level.
@ -180,15 +180,21 @@ The format for running plzip is:
limit) as shown in the table below. Note that `-9' can be much
slower than `-1'. These options have no effect when decompressing.
The bidimensional parameter space of LZMA can't be mapped to a
linear scale optimal for all files. If your files are large, very
repetitive, etc, you may need to use the `--match-length' and
`--dictionary-size' options directly to achieve optimal
performance.
Level Dictionary size Match length limit
-1 1 MiB 10 bytes
-2 1.5 MiB 12 bytes
-3 2 MiB 17 bytes
-4 3 MiB 26 bytes
-5 4 MiB 44 bytes
-6 8 MiB 80 bytes
-7 16 MiB 108 bytes
-8 24 MiB 163 bytes
-1 1 MiB 5 bytes
-2 1.5 MiB 6 bytes
-3 2 MiB 8 bytes
-4 3 MiB 12 bytes
-5 4 MiB 20 bytes
-6 8 MiB 36 bytes
-7 16 MiB 68 bytes
-8 24 MiB 132 bytes
-9 32 MiB 273 bytes
`--fast'
@ -328,11 +334,11 @@ Concept Index

Tag Table:
Node: Top223
Node: Introduction791
Node: Invoking Plzip3533
Node: Program Design7499
Node: File Format8161
Node: Problems10117
Node: Concept Index10646
Node: Introduction833
Node: Invoking Plzip3592
Node: Program Design7840
Node: File Format8502
Node: Problems10458
Node: Concept Index10987

End Tag Table

106
doc/plzip.texinfo
View file

@ -5,8 +5,8 @@
@finalout
@c %**end of header
@set UPDATED 20 March 2010
@set VERSION 0.6
@set UPDATED 3 December 2010
@set VERSION 0.7
@dircategory Data Compression
@direntry
@ -14,9 +14,10 @@
@end direntry
@ifnothtml
@titlepage
@title Plzip
@subtitle A data compressor based on the LZMA algorithm
@subtitle Parallel compressor compatible with lzip
@subtitle for Plzip version @value{VERSION}, @value{UPDATED}
@author by Antonio Diaz Diaz
@ -25,6 +26,7 @@
@end titlepage
@contents
@end ifnothtml
@node Top
@top
@ -32,12 +34,12 @@
This manual is for Plzip (version @value{VERSION}, @value{UPDATED}).
@menu
* Introduction:: Purpose and features of plzip
* Invoking Plzip:: Command line interface
* Program Design:: Internal structure of plzip
* File Format:: Detailed format of the compressed file
* Problems:: Reporting bugs
* Concept Index:: Index of concepts
* Introduction:: Purpose and features of plzip
* Invoking Plzip:: Command line interface
* Program Design:: Internal structure of plzip
* File Format:: Detailed format of the compressed file
* Problems:: Reporting bugs
* Concept Index:: Index of concepts
@end menu
@sp 1
@ -52,10 +54,10 @@ to copy, distribute and modify it.
@cindex introduction
Plzip is a massively parallel (multi-threaded), lossless data compressor
based on the LZMA algorithm, with very safe integrity checking and a
user interface similar to the one of gzip or bzip2. Plzip uses the lzip
file format; the files produced by plzip are fully compatible with
lzip-1.4 or newer.
based on the lzlib compression library, with very safe integrity
checking and a user interface similar to the one of bzip2, gzip or lzip.
Plzip uses the lzip file format; the files produced by plzip are fully
compatible with lzip-1.4 or newer.
Plzip is intended for faster compression/decompression of big files on
multiprocessor machines, which makes it specially well suited for
@ -121,77 +123,77 @@ plzip [@var{options}] [@var{files}]
Plzip supports the following options:
@table @samp
@item --help
@itemx -h
@item -h
@itemx --help
Print an informative help message describing the options and exit.
@item --version
@itemx -V
@item -V
@itemx --version
Print the version number of plzip on the standard output and exit.
@item --data-size=@var{size}
@itemx -B
@item -B
@itemx --data-size=@var{size}
Set the input data block size in bytes. The input file will be divided
in chunks of this size before compression is performed. Valid values
range from 8KiB to 1GiB. Default value is two times the dictionary size.
Plzip will reduce the dictionary size if it is larger than the chosen
data size.
@item --stdout
@itemx -c
@item -c
@itemx --stdout
Compress or decompress to standard output. Needed when reading from a
named pipe (fifo) or from a device.
@item --decompress
@itemx -d
@item -d
@itemx --decompress
Decompress.
@item --force
@itemx -f
@item -f
@itemx --force
Force overwrite of output file.
@item --keep
@itemx -k
@item -k
@itemx --keep
Keep (don't delete) input files during compression or decompression.
@item --match-length=@var{length}
@itemx -m @var{length}
@item -m @var{length}
@itemx --match-length=@var{length}
Set the match length limit in bytes. Valid values range from 5 to 273.
Larger values usually give better compression ratios but longer
compression times.
@item --threads=@var{threads}
@itemx -n @var{threads}
@item -n @var{threads}
@itemx --threads=@var{threads}
Set the number of worker threads. Valid values range from 1 to "as many
as your system can support". If this option is not used, plzip tries to
detect the number of processors in the system and use it as default
value.
@item --output=@var{file}
@itemx -o @var{file}
@item -o @var{file}
@itemx --output=@var{file}
When reading from standard input and @samp{--stdout} has not been
specified, use @samp{@var{file}} as the virtual name of the uncompressed
file. This produces a file named @samp{@var{file}} when decompressing,
and a file named @samp{@var{file}.lz} when compressing.
@item --quiet
@itemx -q
@item -q
@itemx --quiet
Quiet operation. Suppress all messages.
@item --dictionary-size=@var{size}
@itemx -s @var{size}
@item -s @var{size}
@itemx --dictionary-size=@var{size}
Set the dictionary size limit in bytes. Valid values range from 4KiB to
512MiB. Note that dictionary sizes are quantized. If the specified size
does not match one of the valid sizes, it will be rounded upwards.
@item --test
@itemx -t
@item -t
@itemx --test
Check integrity of the specified file(s), but don't decompress them.
This really performs a trial decompression and throws away the result.
Use @samp{-tvv} or @samp{-tvvv} to see information about the file.
@item --verbose
@itemx -v
@item -v
@itemx --verbose
Verbose mode. Show the compression ratio for each file processed.
Further -v's increase the verbosity level.
@ -200,16 +202,22 @@ Set the compression parameters (dictionary size and match length limit)
as shown in the table below. Note that @samp{-9} can be much slower than
@samp{-1}. These options have no effect when decompressing.
The bidimensional parameter space of LZMA can't be mapped to a linear
scale optimal for all files. If your files are large, very repetitive,
etc, you may need to use the @samp{--match-length} and
@samp{--dictionary-size} options directly to achieve optimal
performance.
@multitable {Level} {Dictionary size} {Match length limit}
@item Level @tab Dictionary size @tab Match length limit
@item -1 @tab 1 MiB @tab 10 bytes
@item -2 @tab 1.5 MiB @tab 12 bytes
@item -3 @tab 2 MiB @tab 17 bytes
@item -4 @tab 3 MiB @tab 26 bytes
@item -5 @tab 4 MiB @tab 44 bytes
@item -6 @tab 8 MiB @tab 80 bytes
@item -7 @tab 16 MiB @tab 108 bytes
@item -8 @tab 24 MiB @tab 163 bytes
@item -1 @tab 1 MiB @tab 5 bytes
@item -2 @tab 1.5 MiB @tab 6 bytes
@item -3 @tab 2 MiB @tab 8 bytes
@item -4 @tab 3 MiB @tab 12 bytes
@item -5 @tab 4 MiB @tab 20 bytes
@item -6 @tab 8 MiB @tab 36 bytes
@item -7 @tab 16 MiB @tab 68 bytes
@item -8 @tab 24 MiB @tab 132 bytes
@item -9 @tab 32 MiB @tab 273 bytes
@end multitable

149
main.cc
View file

@ -61,10 +61,10 @@
namespace {
const char * const Program_name = "Plzip";
const char * const program_name = "plzip";
const char * const program_year = "2010";
const char * invocation_name = 0;
const char * const Program_name = "Plzip";
const char * const program_name = "plzip";
const char * const program_year = "2010";
#ifdef O_BINARY
const int o_binary = O_BINARY;
@ -83,7 +83,7 @@ struct Lzma_options
int match_len_limit; // 5..273
};
enum Mode { m_compress = 0, m_decompress, m_test };
enum Mode { m_compress, m_decompress, m_test };
std::string output_filename;
int outfd = -1;
@ -105,7 +105,7 @@ void show_help() throw()
std::printf( " -d, --decompress decompress\n" );
std::printf( " -f, --force overwrite existing output files\n" );
std::printf( " -k, --keep keep (don't delete) input files\n" );
std::printf( " -m, --match-length=<n> set match length limit in bytes [80]\n" );
std::printf( " -m, --match-length=<n> set match length limit in bytes [36]\n" );
std::printf( " -n, --threads=<n> set the number of (de)compression threads\n" );
std::printf( " -o, --output=<file> if reading stdin, place the output into <file>\n" );
std::printf( " -q, --quiet suppress all messages\n" );
@ -140,30 +140,6 @@ void show_version() throw()
}
const char * format_num( long long num, long long limit = 9999,
const int set_prefix = 0 ) throw()
{
const char * const si_prefix[8] =
{ "k", "M", "G", "T", "P", "E", "Z", "Y" };
const char * const binary_prefix[8] =
{ "Ki", "Mi", "Gi", "Ti", "Pi", "Ei", "Zi", "Yi" };
static bool si = false;
static char buf[16];
if( set_prefix ) si = ( set_prefix > 0 );
const int factor = ( si ) ? 1000 : 1024;
const char * const *prefix = ( si ) ? si_prefix : binary_prefix;
const char *p = "";
limit = std::max( 999LL, std::min( 999999LL, limit ) );
for( int i = 0; i < 8 && ( llabs( num ) > limit ||
( llabs( num ) >= factor && num % factor == 0 ) ); ++i )
{ num /= factor; p = prefix[i]; }
snprintf( buf, sizeof buf, "%lld %s", num, p );
return buf;
}
long long getnum( const char * const ptr, const int bs = 0,
const long long llimit = LLONG_MIN + 1,
const long long ulimit = LLONG_MAX ) throw()
@ -173,7 +149,7 @@ long long getnum( const char * const ptr, const int bs = 0,
long long result = strtoll( ptr, &tail, 0 );
if( tail == ptr )
{
show_error( "bad or missing numerical argument", 0, true );
show_error( "Bad or missing numerical argument.", 0, true );
std::exit( 1 );
}
@ -203,7 +179,7 @@ long long getnum( const char * const ptr, const int bs = 0,
}
if( bad_multiplier )
{
show_error( "bad multiplier in numerical argument", 0, true );
show_error( "Bad multiplier in numerical argument.", 0, true );
std::exit( 1 );
}
for( int i = 0; i < exponent; ++i )
@ -215,7 +191,7 @@ long long getnum( const char * const ptr, const int bs = 0,
if( !errno && ( result < llimit || result > ulimit ) ) errno = ERANGE;
if( errno )
{
show_error( "numerical argument out of limits" );
show_error( "Numerical argument out of limits." );
std::exit( 1 );
}
return result;
@ -254,7 +230,7 @@ int open_instream( const std::string & name, struct stat * const in_statsp,
if( program_mode == m_compress && !force && eindex >= 0 )
{
if( verbosity >= 0 )
std::fprintf( stderr, "%s: input file `%s' already has `%s' suffix.\n",
std::fprintf( stderr, "%s: Input file `%s' already has `%s' suffix.\n",
program_name, name.c_str(),
known_extensions[eindex].from );
}
@ -271,14 +247,16 @@ int open_instream( const std::string & name, struct stat * const in_statsp,
{
const int i = fstat( infd, in_statsp );
const mode_t & mode = in_statsp->st_mode;
if( i < 0 || !( S_ISREG( mode ) || ( to_stdout &&
( S_ISFIFO( mode ) || S_ISSOCK( mode ) ||
S_ISBLK( mode ) || S_ISCHR( mode ) ) ) ) )
const bool can_read = ( i == 0 &&
( S_ISBLK( mode ) || S_ISCHR( mode ) ||
S_ISFIFO( mode ) || S_ISSOCK( mode ) ) );
if( i != 0 || ( !S_ISREG( mode ) && ( !to_stdout || !can_read ) ) )
{
if( verbosity >= 0 )
std::fprintf( stderr, "%s: input file `%s' is not a regular file%s.\n",
std::fprintf( stderr, "%s: Input file `%s' is not a regular file%s.\n",
program_name, name.c_str(),
to_stdout ? "" : " and `--stdout' was not specified" );
( can_read && !to_stdout ) ?
" and `--stdout' was not specified" : "" );
close( infd );
infd = -1;
}
@ -309,7 +287,7 @@ void set_d_outname( const std::string & name, const int i ) throw()
}
output_filename = name; output_filename += ".out";
if( verbosity >= 0 )
std::fprintf( stderr, "%s: can't guess original name for `%s' -- using `%s'.\n",
std::fprintf( stderr, "%s: Can't guess original name for `%s' -- using `%s'.\n",
program_name, name.c_str(), output_filename.c_str() );
}
@ -320,18 +298,14 @@ bool open_outstream( const bool force ) throw()
if( force ) flags |= O_TRUNC; else flags |= O_EXCL;
outfd = open( output_filename.c_str(), flags, outfd_mode );
if( outfd < 0 )
if( outfd < 0 && verbosity >= 0 )
{
if( errno == EEXIST ) outfd = -2; else outfd = -1;
if( verbosity >= 0 )
{
if( outfd == -2 )
std::fprintf( stderr, "%s: Output file %s already exists, skipping.\n",
program_name, output_filename.c_str() );
else
std::fprintf( stderr, "%s: Can't create output file `%s': %s.\n",
program_name, output_filename.c_str(), std::strerror( errno ) );
}
if( errno == EEXIST )
std::fprintf( stderr, "%s: Output file `%s' already exists, skipping.\n",
program_name, output_filename.c_str() );
else
std::fprintf( stderr, "%s: Can't create output file `%s': %s.\n",
program_name, output_filename.c_str(), std::strerror( errno ) );
}
return ( outfd >= 0 );
}
@ -339,7 +313,7 @@ bool open_outstream( const bool force ) throw()
bool check_tty( const int infd, const Mode program_mode ) throw()
{
if( program_mode == m_compress && isatty( outfd ) )
if( program_mode == m_compress && outfd >= 0 && isatty( outfd ) )
{
show_error( "I won't write compressed data to a terminal.", 0, true );
return false;
@ -376,8 +350,9 @@ void close_and_set_permissions( const struct stat * const in_statsp )
bool error = false;
if( in_statsp )
{
if( fchmod( outfd, in_statsp->st_mode ) != 0 ) error = true;
else (void)fchown( outfd, in_statsp->st_uid, in_statsp->st_gid );
if( fchmod( outfd, in_statsp->st_mode ) != 0 ||
( fchown( outfd, in_statsp->st_uid, in_statsp->st_gid ) != 0 &&
errno != EPERM ) ) error = true;
// fchown will in many cases return with EPERM, which can be safely ignored.
}
if( close( outfd ) == 0 ) outfd = -1;
@ -393,7 +368,7 @@ void close_and_set_permissions( const struct stat * const in_statsp )
}
if( error )
{
show_error( "I can't change output file attributes." );
show_error( "Can't change output file attributes." );
cleanup_and_fail( 1 );
}
}
@ -409,15 +384,11 @@ extern "C" void signal_handler( int sig ) throw()
}
void set_signals( const bool to_file ) throw()
void set_signals() throw()
{
if( to_file )
{
std::signal( SIGHUP, signal_handler );
std::signal( SIGINT, signal_handler );
std::signal( SIGTERM, signal_handler );
}
std::signal( SIGUSR1, signal_handler );
std::signal( SIGHUP, signal_handler );
std::signal( SIGINT, signal_handler );
std::signal( SIGTERM, signal_handler );
}
} // end namespace
@ -453,22 +424,24 @@ void show_error( const char * const msg, const int errcode, const bool help ) th
{
if( verbosity >= 0 )
{
if( msg && msg[0] != 0 )
if( msg && msg[0] )
{
std::fprintf( stderr, "%s: %s", program_name, msg );
if( errcode > 0 ) std::fprintf( stderr, ": %s", std::strerror( errcode ) );
if( errcode > 0 )
std::fprintf( stderr, ": %s", std::strerror( errcode ) );
std::fprintf( stderr, "\n" );
}
if( help && invocation_name && invocation_name[0] != 0 )
std::fprintf( stderr, "Try `%s --help' for more information.\n", invocation_name );
if( help && invocation_name && invocation_name[0] )
std::fprintf( stderr, "Try `%s --help' for more information.\n",
invocation_name );
}
}
void internal_error( const char * const msg )
{
std::string s( "internal error: " ); s += msg;
show_error( s.c_str() );
if( verbosity >= 0 )
std::fprintf( stderr, "%s: internal error: %s.\n", program_name, msg );
std::exit( 3 );
}
@ -516,21 +489,21 @@ int main( const int argc, const char * const argv[] )
// to the corresponding LZMA compression modes.
const Lzma_options option_mapping[] =
{
{ 1 << 16, 5 }, // -0
{ 1 << 20, 10 }, // -1
{ 3 << 19, 12 }, // -2
{ 1 << 21, 17 }, // -3
{ 3 << 20, 26 }, // -4
{ 1 << 22, 44 }, // -5
{ 1 << 23, 80 }, // -6
{ 1 << 24, 108 }, // -7
{ 3 << 23, 163 }, // -8
{ 1 << 20, 5 }, // -0
{ 1 << 20, 5 }, // -1
{ 3 << 19, 6 }, // -2
{ 1 << 21, 8 }, // -3
{ 3 << 20, 12 }, // -4
{ 1 << 22, 20 }, // -5
{ 1 << 23, 36 }, // -6
{ 1 << 24, 68 }, // -7
{ 3 << 23, 132 }, // -8
{ 1 << 25, 273 } }; // -9
Lzma_options encoder_options = option_mapping[6]; // default = "-6"
int data_size = 0;
int debug_level = 0;
int infd = -1;
int num_workers = 0; // Start this many worker threads
int num_workers = 0; // start this many worker threads
Mode program_mode = m_compress;
bool force = false;
bool keep_input_files = false;
@ -554,8 +527,8 @@ int main( const int argc, const char * const argv[] )
const Arg_parser::Option options[] =
{
{ '0', 0, Arg_parser::no },
{ '1', "fast", Arg_parser::no },
{ '0', "fast", Arg_parser::no },
{ '1', 0, Arg_parser::no },
{ '2', 0, Arg_parser::no },
{ '3', 0, Arg_parser::no },
{ '4', 0, Arg_parser::no },
@ -584,7 +557,7 @@ int main( const int argc, const char * const argv[] )
{ 'V', "version", Arg_parser::no },
{ 0 , 0, Arg_parser::no } };
Arg_parser parser( argc, argv, options );
const Arg_parser parser( argc, argv, options );
if( parser.error().size() ) // bad option
{ show_error( parser.error().c_str(), 0, true ); return 1; }
@ -605,7 +578,7 @@ int main( const int argc, const char * const argv[] )
case 'c': to_stdout = true; break;
case 'd': program_mode = m_decompress; break;
case 'D': debug_level = getnum( arg, 0, 0, 3 ); break;
case 'e': break;
case 'e': break; // ignored by now
case 'f': force = true; break;
case 'h': show_help(); return 0;
case 'k': keep_input_files = true; break;
@ -623,7 +596,7 @@ int main( const int argc, const char * const argv[] )
case 'V': show_version(); return 0;
default : internal_error( "uncaught option" );
}
}
} // end process options
if( data_size <= 0 )
data_size = 2 * std::max( 65536, encoder_options.dictionary_size );
@ -646,8 +619,10 @@ int main( const int argc, const char * const argv[] )
}
if( filenames.empty() ) filenames.push_back("-");
set_signals( !to_stdout && program_mode != m_test &&
( filenames_given || default_output_filename.size() ) );
if( !to_stdout && program_mode != m_test &&
( filenames_given || default_output_filename.size() ) )
set_signals();
std::signal( SIGUSR1, signal_handler );
Pretty_print pp( filenames );
if( program_mode == m_test )
@ -737,9 +712,7 @@ int main( const int argc, const char * const argv[] )
}
if( outfd >= 0 && close( outfd ) != 0 )
{
if( verbosity >= 0 )
std::fprintf( stderr, "%s: Can't close stdout: %s.\n",
program_name, std::strerror( errno ) );
show_error( "Can't close stdout", errno );
if( retval < 1 ) retval = 1;
}
return retval;

7
plzip.h
View file

@ -19,16 +19,15 @@
class Pretty_print
{
const char * const stdin_name;
const unsigned int stdin_name_len;
unsigned int longest_name;
std::string name_;
mutable bool first_post;
public:
Pretty_print( const std::vector< std::string > & filenames )
: stdin_name( "(stdin)" ), stdin_name_len( std::strlen( stdin_name ) ),
longest_name( 0 ), first_post( false )
: stdin_name( "(stdin)" ), longest_name( 0 ), first_post( false )
{
const unsigned int stdin_name_len = std::strlen( stdin_name );
for( unsigned int i = 0; i < filenames.size(); ++i )
{
const std::string & s = filenames[i];
@ -119,7 +118,7 @@ int decompress( const int num_workers, const int num_slots,
extern int verbosity;
void fatal(); // Terminate the process
void fatal(); // terminate the program
void show_error( const char * const msg, const int errcode = 0, const bool help = false ) throw();
void internal_error( const char * const msg );

22
testsuite/check.sh
View file

@ -19,15 +19,24 @@ fi
if [ -d tmp ] ; then rm -rf tmp ; fi
mkdir tmp
printf "testing plzip..."
printf "testing plzip-%s..." "$2"
cd "${objdir}"/tmp
cat "${testdir}"/test1 > in || framework_failure
cat "${testdir}"/test.txt > in || framework_failure
cat in in in in > in4 || framework_failure
fail=0
"${LZIP}" -cd "${testdir}"/test1.lz > copy || fail=1
"${LZIP}" -t "${testdir}"/test_v0.lz || fail=1
printf .
"${LZIP}" -cd "${testdir}"/test_v0.lz > copy || fail=1
cmp in copy || fail=1
printf .
"${LZIP}" -t "${testdir}"/test_v1.lz || fail=1
printf .
"${LZIP}" -cd "${testdir}"/test_v1.lz > copy || fail=1
cmp in copy || fail=1
printf .
for i in s4Ki 0 1 2 3 4 5 6 7 8 9 ; do
"${LZIP}" -k -$i in || fail=1
@ -54,12 +63,17 @@ for i in s4Ki 0 1 2 3 4 5 6 7 8 9 ; do
done
for i in s4Ki 0 1 2 3 4 5 6 7 8 9 ; do
"${LZIP}" -fe -$i -o out < in || fail=1
"${LZIP}" -f -$i -o out < in || fail=1
"${LZIP}" -df -o copy < out.lz || fail=1
cmp in copy || fail=1
printf .
done
"${LZIP}" -$i < in > anyothername || fail=1
"${LZIP}" -dq anyothername || fail=1
cmp in anyothername.out || fail=1
printf .
for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 ; do
"${LZIP}" -s4Ki -B8Ki -n$i < in4 > out4 || fail=1
"${LZIP}" -d -n$i < out4 > copy4 || fail=1

0
testsuite/test1 → testsuite/test.txt
View file

0
testsuite/test1.lz → testsuite/test_v0.lz
View file

BIN
testsuite/test_v1.lz Normal file
View file

Binary file not shown.