lzip/plzip
1
0
Fork 0
Code Issues Activity

Adding upstream version 1.4~pre1.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-24 04:08:30 +01:00
parent c74f1a4d51
commit 8473fa783f
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
9 changed files with 99 additions and 75 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
ChangeLog
View file

@ -1,3 +1,8 @@
2015-02-25 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.4-pre1 released.
* Option '-0' now uses the fast encoder of lzlib 1.7-pre1.
2015-01-22 Antonio Diaz Diaz <antonio@gnu.org> 2015-01-22 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.3 released. * Version 1.3 released.

17
NEWS
View file

@ -1,14 +1,5 @@
Changes in version 1.3: Changes in version 1.4:
Testing of a non-seekable file or of standard input now uses up to 30 The option "-0" has been modified to use the new fast encoder of lzlib
MiB less memory per thread. 1.7-pre1, achieving a compression speed and ratio comparable to those of
pigz's default compression level.
"-dvvv" and "-tvvv" now show the dictionary size of the first member,
producing the same output as lzip for single-member files.
Chapters "Memory requirements" and "Minimum file sizes" have been added
to the manual.
The targets "install-compress", "install-strip-compress",
"install-info-compress" and "install-man-compress" have been added to
the Makefile.

7
compress.cc
View file

@ -375,8 +375,11 @@ extern "C" void * cworker( void * arg )
Packet * const packet = courier.distribute_packet(); Packet * const packet = courier.distribute_packet();
if( !packet ) break; // no more packets to process if( !packet ) break; // no more packets to process
const int dict_size = std::max( LZ_min_dictionary_size(), int dict_size;
std::min( dictionary_size, packet->size ) ); if( dictionary_size == 65535 && match_len_limit == 16 )
dict_size = dictionary_size;
else dict_size = std::max( LZ_min_dictionary_size(),
std::min( dictionary_size, packet->size ) );
LZ_Encoder * const encoder = LZ_Encoder * const encoder =
LZ_compress_open( dict_size, match_len_limit, LLONG_MAX ); LZ_compress_open( dict_size, match_len_limit, LLONG_MAX );
if( !encoder || LZ_compress_errno( encoder ) != LZ_ok ) if( !encoder || LZ_compress_errno( encoder ) != LZ_ok )

2
configure vendored
View file

@ -6,7 +6,7 @@
# to copy, distribute and modify it. # to copy, distribute and modify it.
pkgname=plzip pkgname=plzip
pkgversion=1.3 pkgversion=1.4-pre1
progname=plzip progname=plzip
srctrigger=doc/${pkgname}.texi srctrigger=doc/${pkgname}.texi

8
doc/plzip.1
View file

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.46.1. .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.46.1.
.TH PLZIP "1" "January 2015" "plzip 1.3" "User Commands" .TH PLZIP "1" "February 2015" "plzip 1.4-pre1" "User Commands"
.SH NAME .SH NAME
plzip \- reduces the size of files plzip \- reduces the size of files
.SH SYNOPSIS .SH SYNOPSIS
@ -54,11 +54,11 @@ test compressed file integrity
\fB\-v\fR, \fB\-\-verbose\fR \fB\-v\fR, \fB\-\-verbose\fR
be verbose (a 2nd \fB\-v\fR gives more) be verbose (a 2nd \fB\-v\fR gives more)
.TP .TP
\fB\-1\fR .. \fB\-9\fR \fB\-0\fR .. \fB\-9\fR
set compression level [default 6] set compression level [default 6]
.TP .TP
\fB\-\-fast\fR \fB\-\-fast\fR
alias for \fB\-1\fR alias for \fB\-0\fR
.TP .TP
\fB\-\-best\fR \fB\-\-best\fR
alias for \fB\-9\fR alias for \fB\-9\fR
@ -84,7 +84,7 @@ Plzip home page: http://www.nongnu.org/lzip/plzip.html
Copyright \(co 2009 Laszlo Ersek. Copyright \(co 2009 Laszlo Ersek.
.br .br
Copyright \(co 2015 Antonio Diaz Diaz. Copyright \(co 2015 Antonio Diaz Diaz.
Using Lzlib 1.6 Using Lzlib 1.7\-pre1
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html> License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>
.br .br
This is free software: you are free to change and redistribute it. This is free software: you are free to change and redistribute it.

44
doc/plzip.info
View file

@ -11,7 +11,7 @@ File: plzip.info, Node: Top, Next: Introduction, Up: (dir)
Plzip Manual Plzip Manual
************ ************
This manual is for Plzip (version 1.3, 22 January 2015). This manual is for Plzip (version 1.4-pre1, 25 February 2015).
* Menu: * Menu:
@ -116,7 +116,7 @@ compressed files is also supported.
WARNING! Even if plzip is bug-free, other causes may result in a WARNING! Even if plzip is bug-free, other causes may result in a
corrupt compressed file (bugs in the system libraries, memory errors, corrupt compressed file (bugs in the system libraries, memory errors,
etc). Therefore, if the data you are going to compress is important, etc). Therefore, if the data you are going to compress are important,
give the '--keep' option to plzip and do not remove the original file give the '--keep' option to plzip and do not remove the original file
until you verify the compressed file with a command like until you verify the compressed file with a command like
'plzip -cd file.lz | cmp file -'. 'plzip -cd file.lz | cmp file -'.
@ -180,8 +180,9 @@ The format for running plzip is:
Set the size of the input data blocks, in bytes. The input file Set the size of the input data blocks, in bytes. The input file
will be divided in chunks of this size before compression is will be divided in chunks of this size before compression is
performed. Valid values range from 8 KiB to 1 GiB. Default value performed. Valid values range from 8 KiB to 1 GiB. Default value
is two times the dictionary size. Plzip will reduce the dictionary is two times the dictionary size, except for option '-0' where it
size if it is larger than the chosen data size. defaults to 1 MiB. Plzip will reduce the dictionary size if it is
larger than the chosen data size.
'-c' '-c'
'--stdout' '--stdout'
@ -267,10 +268,10 @@ The format for running plzip is:
verbosity level, showing status, compression ratio, dictionary verbosity level, showing status, compression ratio, dictionary
size, decompressed size, and compressed size. size, decompressed size, and compressed size.
'-1 .. -9' '-0 .. -9'
Set the compression parameters (dictionary size and match length Set the compression parameters (dictionary size and match length
limit) as shown in the table below. Note that '-9' can be much limit) as shown in the table below. Note that '-9' can be much
slower than '-1'. These options have no effect when decompressing. slower than '-0'. These options have no effect when decompressing.
The bidimensional parameter space of LZMA can't be mapped to a The bidimensional parameter space of LZMA can't be mapped to a
linear scale optimal for all files. If your files are large, very linear scale optimal for all files. If your files are large, very
@ -279,6 +280,7 @@ The format for running plzip is:
performance. performance.
Level Dictionary size Match length limit Level Dictionary size Match length limit
-0 64 KiB 16 bytes
-1 1 MiB 5 bytes -1 1 MiB 5 bytes
-2 1.5 MiB 6 bytes -2 1.5 MiB 6 bytes
-3 2 MiB 8 bytes -3 2 MiB 8 bytes
@ -372,8 +374,8 @@ additional information before, between, or after them.
'Lzma stream' 'Lzma stream'
The lzma stream, finished by an end of stream marker. Uses default The lzma stream, finished by an end of stream marker. Uses default
values for encoder properties. See the lzip manual for a full values for encoder properties. *Note Stream format: (lzip)Stream
description. format, for a complete description.
'CRC32 (4 bytes)' 'CRC32 (4 bytes)'
CRC of the uncompressed original data. CRC of the uncompressed original data.
@ -397,8 +399,11 @@ File: plzip.info, Node: Memory requirements, Next: Minimum file sizes, Prev:
The amount of memory required *per thread* is approximately the The amount of memory required *per thread* is approximately the
following: following:
* For compression; 11 times the dictionary size plus 3 times the * For compression at level -0; 1.5 MiB plus 3 times the data size
data size (*note --data-size::). (*note --data-size::).
* For compression at other levels; 11 times the dictionary size plus
3 times the data size.
* For decompression of a regular (seekable) file to another regular * For decompression of a regular (seekable) file to another regular
file, or for testing of a regular file; the dictionary size. Note file, or for testing of a regular file; the dictionary size. Note
@ -439,6 +444,7 @@ default data size for each level:
Processors 2 4 8 16 64 256 Processors 2 4 8 16 64 256
------------------------------------------------------------------------- -------------------------------------------------------------------------
Level Level
-0 2 MiB 4 MiB 8 MiB 16 MiB 64 MiB 256 MiB
-1 4 MiB 8 MiB 16 MiB 32 MiB 128 MiB 512 MiB -1 4 MiB 8 MiB 16 MiB 32 MiB 128 MiB 512 MiB
-2 6 MiB 12 MiB 24 MiB 48 MiB 192 MiB 768 MiB -2 6 MiB 12 MiB 24 MiB 48 MiB 192 MiB 768 MiB
-3 8 MiB 16 MiB 32 MiB 64 MiB 256 MiB 1 GiB -3 8 MiB 16 MiB 32 MiB 64 MiB 256 MiB 1 GiB
@ -489,15 +495,15 @@ Concept index
 
Tag Table: Tag Table:
Node: Top221 Node: Top221
Node: Introduction988 Node: Introduction994
Node: Program design5284 Node: Program design5291
Node: Invoking plzip6873 Node: Invoking plzip6880
Ref: --data-size7287 Ref: --data-size7294
Node: File format12414 Node: File format12518
Node: Memory requirements14930 Node: Memory requirements15061
Node: Minimum file sizes15907 Node: Minimum file sizes16130
Node: Problems17758 Node: Problems18052
Node: Concept index18294 Node: Concept index18588
 
End Tag Table End Tag Table

35
doc/plzip.texi
View file

@ -6,8 +6,8 @@
@finalout @finalout
@c %**end of header @c %**end of header
@set UPDATED 22 January 2015 @set UPDATED 25 February 2015
@set VERSION 1.3 @set VERSION 1.4-pre1
@dircategory Data Compression @dircategory Data Compression
@direntry @direntry
@ -142,7 +142,7 @@ compressed files is also supported.
WARNING! Even if plzip is bug-free, other causes may result in a corrupt WARNING! Even if plzip is bug-free, other causes may result in a corrupt
compressed file (bugs in the system libraries, memory errors, etc). compressed file (bugs in the system libraries, memory errors, etc).
Therefore, if the data you are going to compress is important, give the Therefore, if the data you are going to compress are important, give the
@samp{--keep} option to plzip and do not remove the original file until @samp{--keep} option to plzip and do not remove the original file until
you verify the compressed file with a command like you verify the compressed file with a command like
@w{@samp{plzip -cd file.lz | cmp file -}}. @w{@samp{plzip -cd file.lz | cmp file -}}.
@ -210,8 +210,9 @@ Print the version number of plzip on the standard output and exit.
Set the size of the input data blocks, in bytes. The input file will be Set the size of the input data blocks, in bytes. The input file will be
divided in chunks of this size before compression is performed. Valid divided in chunks of this size before compression is performed. Valid
values range from 8 KiB to 1 GiB. Default value is two times the values range from 8 KiB to 1 GiB. Default value is two times the
dictionary size. Plzip will reduce the dictionary size if it is larger dictionary size, except for option @samp{-0} where it defaults to 1 MiB.
than the chosen data size. Plzip will reduce the dictionary size if it is larger than the chosen
data size.
@item -c @item -c
@itemx --stdout @itemx --stdout
@ -290,10 +291,10 @@ When decompressing or testing, further -v's (up to 4) increase the
verbosity level, showing status, compression ratio, dictionary size, verbosity level, showing status, compression ratio, dictionary size,
decompressed size, and compressed size. decompressed size, and compressed size.
@item -1 .. -9 @item -0 .. -9
Set the compression parameters (dictionary size and match length limit) Set the compression parameters (dictionary size and match length limit)
as shown in the table below. Note that @samp{-9} can be much slower than as shown in the table below. Note that @samp{-9} can be much slower than
@samp{-1}. These options have no effect when decompressing. @samp{-0}. These options have no effect when decompressing.
The bidimensional parameter space of LZMA can't be mapped to a linear The bidimensional parameter space of LZMA can't be mapped to a linear
scale optimal for all files. If your files are large, very repetitive, scale optimal for all files. If your files are large, very repetitive,
@ -303,6 +304,7 @@ performance.
@multitable {Level} {Dictionary size} {Match length limit} @multitable {Level} {Dictionary size} {Match length limit}
@item Level @tab Dictionary size @tab Match length limit @item Level @tab Dictionary size @tab Match length limit
@item -0 @tab 64 KiB @tab 16 bytes
@item -1 @tab 1 MiB @tab 5 bytes @item -1 @tab 1 MiB @tab 5 bytes
@item -2 @tab 1.5 MiB @tab 6 bytes @item -2 @tab 1.5 MiB @tab 6 bytes
@item -3 @tab 2 MiB @tab 8 bytes @item -3 @tab 2 MiB @tab 8 bytes
@ -403,8 +405,16 @@ Example: 0xD3 = 2^19 - 6 * 2^15 = 512 KiB - 6 * 32 KiB = 320 KiB@*
Valid values for dictionary size range from 4 KiB to 512 MiB. Valid values for dictionary size range from 4 KiB to 512 MiB.
@item Lzma stream @item Lzma stream
The lzma stream, finished by an end of stream marker. Uses default values The lzma stream, finished by an end of stream marker. Uses default
for encoder properties. See the lzip manual for a full description. values for encoder properties.
@ifnothtml
@xref{Stream format,,,lzip},
@end ifnothtml
@ifhtml
See
@uref{http://www.nongnu.org/lzip/manual/lzip_manual.html#Stream-format,,Stream format}
@end ifhtml
for a complete description.
@item CRC32 (4 bytes) @item CRC32 (4 bytes)
CRC of the uncompressed original data. CRC of the uncompressed original data.
@ -429,9 +439,13 @@ following:
@itemize @bullet @itemize @bullet
@item @item
For compression; 11 times the dictionary size plus 3 times the data size For compression at level -0; 1.5 MiB plus 3 times the data size
(@pxref{--data-size}). (@pxref{--data-size}).
@item
For compression at other levels; 11 times the dictionary size plus 3
times the data size.
@item @item
For decompression of a regular (seekable) file to another regular file, For decompression of a regular (seekable) file to another regular file,
or for testing of a regular file; the dictionary size. Note that regular or for testing of a regular file; the dictionary size. Note that regular
@ -475,6 +489,7 @@ data size for each level:
@multitable {Processors} {512 MiB} {512 MiB} {512 MiB} {512 MiB} {512 MiB} {512 MiB} @multitable {Processors} {512 MiB} {512 MiB} {512 MiB} {512 MiB} {512 MiB} {512 MiB}
@headitem Processors @tab 2 @tab 4 @tab 8 @tab 16 @tab 64 @tab 256 @headitem Processors @tab 2 @tab 4 @tab 8 @tab 16 @tab 64 @tab 256
@item Level @item Level
@item -0 @tab 2 MiB @tab 4 MiB @tab 8 MiB @tab 16 MiB @tab 64 MiB @tab 256 MiB
@item -1 @tab 4 MiB @tab 8 MiB @tab 16 MiB @tab 32 MiB @tab 128 MiB @tab 512 MiB @item -1 @tab 4 MiB @tab 8 MiB @tab 16 MiB @tab 32 MiB @tab 128 MiB @tab 512 MiB
@item -2 @tab 6 MiB @tab 12 MiB @tab 24 MiB @tab 48 MiB @tab 192 MiB @tab 768 MiB @item -2 @tab 6 MiB @tab 12 MiB @tab 24 MiB @tab 48 MiB @tab 192 MiB @tab 768 MiB
@item -3 @tab 8 MiB @tab 16 MiB @tab 32 MiB @tab 64 MiB @tab 256 MiB @tab 1 GiB @item -3 @tab 8 MiB @tab 16 MiB @tab 32 MiB @tab 64 MiB @tab 256 MiB @tab 1 GiB

12
lzip.h
View file

@ -69,9 +69,9 @@ const uint8_t magic_string[4] = { 0x4C, 0x5A, 0x49, 0x50 }; // "LZIP"
struct File_header struct File_header
{ {
uint8_t data[6]; // 0-3 magic bytes uint8_t data[6]; /* 0-3 magic bytes */
// 4 version /* 4 version */
// 5 coded_dict_size /* 5 coded_dict_size */
enum { size = 6 }; enum { size = 6 };
void set_magic() { std::memcpy( data, magic_string, 4 ); data[4] = 1; } void set_magic() { std::memcpy( data, magic_string, 4 ); data[4] = 1; }
@ -111,9 +111,9 @@ struct File_header
struct File_trailer struct File_trailer
{ {
uint8_t data[20]; // 0-3 CRC32 of the uncompressed data uint8_t data[20]; /* 0-3 CRC32 of the uncompressed data */
// 4-11 size of the uncompressed data /* 4-11 size of the uncompressed data */
// 12-19 member size including header and trailer /* 12-19 member size including header and trailer */
enum { size = 20 }; enum { size = 20 };

44
main.cc
View file

@ -82,8 +82,8 @@ struct { const char * from; const char * to; } const known_extensions[] = {
struct Lzma_options struct Lzma_options
{ {
int dictionary_size; // 4 KiB .. 512 MiB int dictionary_size; /* 4 KiB .. 512 MiB */
int match_len_limit; // 5 .. 273 int match_len_limit; /* 5 .. 273 */
}; };
enum Mode { m_compress, m_decompress, m_test }; enum Mode { m_compress, m_decompress, m_test };
@ -116,8 +116,8 @@ void show_help( const long num_online )
" -s, --dictionary-size=<bytes> set dictionary size limit in bytes [8 MiB]\n" " -s, --dictionary-size=<bytes> set dictionary size limit in bytes [8 MiB]\n"
" -t, --test test compressed file integrity\n" " -t, --test test compressed file integrity\n"
" -v, --verbose be verbose (a 2nd -v gives more)\n" " -v, --verbose be verbose (a 2nd -v gives more)\n"
" -1 .. -9 set compression level [default 6]\n" " -0 .. -9 set compression level [default 6]\n"
" --fast alias for -1\n" " --fast alias for -0\n"
" --best alias for -9\n", num_online ); " --best alias for -9\n", num_online );
if( verbosity >= 1 ) if( verbosity >= 1 )
{ {
@ -385,14 +385,14 @@ void cleanup_and_fail( const int retval )
namespace { namespace {
// Set permissions, owner and times. /* Set permissions, owner and times. */
void close_and_set_permissions( const struct stat * const in_statsp ) void close_and_set_permissions( const struct stat * const in_statsp )
{ {
bool warning = false; bool warning = false;
if( in_statsp ) if( in_statsp )
{ {
const mode_t mode = in_statsp->st_mode; const mode_t mode = in_statsp->st_mode;
// fchown will in many cases return with EPERM, which can be safely ignored. /* fchown will in many cases return with EPERM, which can be safely ignored. */
if( fchown( outfd, in_statsp->st_uid, in_statsp->st_gid ) == 0 ) if( fchown( outfd, in_statsp->st_uid, in_statsp->st_gid ) == 0 )
{ if( fchmod( outfd, mode ) != 0 ) warning = true; } { if( fchmod( outfd, mode ) != 0 ) warning = true; }
else else
@ -482,7 +482,7 @@ void show_progress( const int packet_size,
const Pretty_print * const p, const Pretty_print * const p,
const unsigned long long cfile_size ) const unsigned long long cfile_size )
{ {
static unsigned long long csize = 0; // file_size / 100 static unsigned long long csize = 0; /* file_size / 100 */
static unsigned long long pos = 0; static unsigned long long pos = 0;
static const Pretty_print * pp = 0; static const Pretty_print * pp = 0;
static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
@ -511,16 +511,16 @@ int main( const int argc, const char * const argv[] )
to the corresponding LZMA compression modes. */ to the corresponding LZMA compression modes. */
const Lzma_options option_mapping[] = const Lzma_options option_mapping[] =
{ {
{ 1 << 20, 5 }, // -0 { 65535, 16 }, /* -0 */
{ 1 << 20, 5 }, // -1 { 1 << 20, 5 }, /* -1 */
{ 3 << 19, 6 }, // -2 { 3 << 19, 6 }, /* -2 */
{ 1 << 21, 8 }, // -3 { 1 << 21, 8 }, /* -3 */
{ 3 << 20, 12 }, // -4 { 3 << 20, 12 }, /* -4 */
{ 1 << 22, 20 }, // -5 { 1 << 22, 20 }, /* -5 */
{ 1 << 23, 36 }, // -6 { 1 << 23, 36 }, /* -6 */
{ 1 << 24, 68 }, // -7 { 1 << 24, 68 }, /* -7 */
{ 3 << 23, 132 }, // -8 { 3 << 23, 132 }, /* -8 */
{ 1 << 25, 273 } }; // -9 { 1 << 25, 273 } }; /* -9 */
Lzma_options encoder_options = option_mapping[6]; // default = "-6" Lzma_options encoder_options = option_mapping[6]; // default = "-6"
std::string input_filename; std::string input_filename;
std::string default_output_filename; std::string default_output_filename;
@ -584,7 +584,7 @@ int main( const int argc, const char * const argv[] )
for( ; argind < parser.arguments(); ++argind ) for( ; argind < parser.arguments(); ++argind )
{ {
const int code = parser.code( argind ); const int code = parser.code( argind );
if( !code ) break; // no more options if( !code ) break; /* no more options */
const char * const arg = parser.argument( argind ).c_str(); const char * const arg = parser.argument( argind ).c_str();
switch( code ) switch( code )
{ {
@ -615,7 +615,7 @@ int main( const int argc, const char * const argv[] )
case 'V': show_version(); return 0; case 'V': show_version(); return 0;
default : internal_error( "uncaught option." ); default : internal_error( "uncaught option." );
} }
} // end process options } /* end process options */
#if defined(__MSVCRT__) || defined(__OS2__) #if defined(__MSVCRT__) || defined(__OS2__)
setmode( STDIN_FILENO, O_BINARY ); setmode( STDIN_FILENO, O_BINARY );
@ -626,7 +626,11 @@ int main( const int argc, const char * const argv[] )
outfd = -1; outfd = -1;
if( data_size <= 0 ) if( data_size <= 0 )
data_size = 2 * std::max( 65536, encoder_options.dictionary_size ); {
if( encoder_options.dictionary_size == 65535 &&
encoder_options.match_len_limit == 16 ) data_size = 1 << 20;
else data_size = 2 * std::max( 65536, encoder_options.dictionary_size );
}
else if( data_size < encoder_options.dictionary_size ) else if( data_size < encoder_options.dictionary_size )
encoder_options.dictionary_size = encoder_options.dictionary_size =
std::max( data_size, LZ_min_dictionary_size() ); std::max( data_size, LZ_min_dictionary_size() );