lzip/plzip
1
0
Fork 0
Code Issues Activity

Merging upstream version 1.12.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-24 04:20:02 +01:00
parent d2621fb521
commit 3708015695
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
19 changed files with 83 additions and 95 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
ChangeLog
View file

@ -1,6 +1,6 @@
2024-11-19 Antonio Diaz Diaz <antonio@gnu.org>
2025-01-09 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.12-rc1 released.
* Version 1.12 released.
* decompress.cc (decompress), list.cc (list_files):
Return 2 if any empty member is found in a multimember file.
* dec_stdout.cc, dec_stream.cc:
@ -237,7 +237,7 @@
until something better appears on the net.
Copyright (C) 2009-2024 Antonio Diaz Diaz.
Copyright (C) 2009-2025 Antonio Diaz Diaz.
This file is a collection of facts, and thus it is not copyrightable, but just
in case, you have unlimited permission to copy, distribute, and modify it.

7
INSTALL
View file

@ -4,8 +4,9 @@ You will need a C++98 compiler with support for 'long long', and the
compression library lzlib installed. (gcc 3.3.6 or newer is recommended).
I use gcc 6.1.0 and 3.3.6, but the code should compile with any standards
compliant compiler.
Gcc is available at http://gcc.gnu.org.
Lzlib is available at http://www.nongnu.org/lzip/lzlib.html.
Gcc is available at http://gcc.gnu.org
Lzip is available at http://www.nongnu.org/lzip/lzip.html
Lzlib is available at http://www.nongnu.org/lzip/lzlib.html
Lzlib must be version 1.0 or newer, but the fast encoder requires lzlib 1.7
or newer, the Hamming distance (HD) = 3 detection of corrupt headers in
@ -86,7 +87,7 @@ After running 'configure', you can run 'make' and 'make install' as
explained above.
Copyright (C) 2009-2024 Antonio Diaz Diaz.
Copyright (C) 2009-2025 Antonio Diaz Diaz.
This file is free documentation: you have unlimited permission to copy,
distribute, and modify it.

23
README
View file

@ -1,3 +1,5 @@
See the file INSTALL for compilation and installation instructions.
Description
Plzip is a massively parallel (multi-threaded) implementation of lzip. Plzip
@ -20,7 +22,7 @@ faster than lzip, at the cost of a slightly reduced compression ratio (0.4
to 2 percent larger compressed files). Note that the number of usable
threads is limited by file size; on files larger than a few GB plzip can use
hundreds of processors, but on files smaller than 1 MiB plzip is no faster
than lzip (even at compression level -0).
than lzip (not even at compression level -0).
For creation and manipulation of compressed tar archives tarlz can be more
efficient than using tar and plzip because tarlz is able to keep the
@ -40,11 +42,10 @@ decompressed in parallel.
The lzip file format is designed for data sharing and long-term archiving,
taking into account both data integrity and decoder availability:
* The lzip format provides very safe integrity checking and some data
recovery means. The program lziprecover can repair bit flip errors
(one of the most common forms of data corruption) in lzip files, and
provides data recovery capabilities, including error-checked merging
of damaged copies of a file.
* The program lziprecover can repair bit flip errors (one of the most
common forms of data corruption) in lzip files, and provides data
recovery capabilities, including error-checked merging of damaged
copies of a file.
* The lzip format is as simple as possible (but not simpler). The lzip
manual provides the source code of a simple decompressor along with a
@ -66,9 +67,8 @@ makes it safer than compressors returning ambiguous warning values (like
gzip) when it is used as a back end for other programs like tar or zutils.
Plzip automatically uses for each file the largest dictionary size that does
not exceed neither the file size nor the limit given. Keep in mind that the
decompression memory requirement is affected at compression time by the
choice of dictionary size limit.
not exceed neither the file size nor the limit given. The dictionary size
used for decompression is the same dictionary size used for compression.
When compressing, plzip replaces every file given in the command line
with a compressed version of itself, with the name "original_name.lz".
@ -102,8 +102,11 @@ LANGUAGE NOTE: Uncompressed = not compressed = plain data; it may never have
been compressed. Decompressed is used to refer to data which have undergone
the process of decompression.
Plzip uses Arg_parser for command-line argument parsing:
http://www.nongnu.org/arg-parser/arg_parser.html
Copyright (C) 2009-2024 Antonio Diaz Diaz.
Copyright (C) 2009-2025 Antonio Diaz Diaz.
This file is free documentation: you have unlimited permission to copy,
distribute, and modify it.

2
arg_parser.cc
View file

@ -1,5 +1,5 @@
/* Arg_parser - POSIX/GNU command-line argument parser. (C++ version)
Copyright (C) 2006-2024 Antonio Diaz Diaz.
Copyright (C) 2006-2025 Antonio Diaz Diaz.
This library is free software. Redistribution and use in source and
binary forms, with or without modification, are permitted provided

2
arg_parser.h
View file

@ -1,5 +1,5 @@
/* Arg_parser - POSIX/GNU command-line argument parser. (C++ version)
Copyright (C) 2006-2024 Antonio Diaz Diaz.
Copyright (C) 2006-2025 Antonio Diaz Diaz.
This library is free software. Redistribution and use in source and
binary forms, with or without modification, are permitted provided

4
compress.cc
View file

@ -1,6 +1,6 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009 Laszlo Ersek.
Copyright (C) 2009-2024 Antonio Diaz Diaz.
Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -465,7 +465,7 @@ void muxer( Packet_courier & courier, const Pretty_print & pp, const int outfd )
out_size += opacket->size;
if( writeblock( outfd, opacket->data, opacket->size ) != opacket->size )
{ pp(); show_error( write_error_msg, errno ); cleanup_and_fail(); }
{ pp(); show_error( wr_err_msg, errno ); cleanup_and_fail(); }
delete[] opacket->data;
courier.return_empty_packet();
}

6
configure vendored
View file

@ -1,12 +1,12 @@
#! /bin/sh
# configure script for Plzip - Massively parallel implementation of lzip
# Copyright (C) 2009-2024 Antonio Diaz Diaz.
# Copyright (C) 2009-2025 Antonio Diaz Diaz.
#
# This configure script is free software: you have unlimited permission
# to copy, distribute, and modify it.
pkgname=plzip
pkgversion=1.12-rc1
pkgversion=1.12
progname=plzip
with_mingw=
srctrigger=doc/${pkgname}.texi
@ -179,7 +179,7 @@ echo "MAKEINFO = ${MAKEINFO}"
rm -f Makefile
cat > Makefile << EOF
# Makefile for Plzip - Massively parallel implementation of lzip
# Copyright (C) 2009-2024 Antonio Diaz Diaz.
# Copyright (C) 2009-2025 Antonio Diaz Diaz.
# This file was generated automatically by configure. Don't edit.
#
# This Makefile is free software: you have unlimited permission

7
dec_stdout.cc
View file

@ -1,6 +1,6 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009 Laszlo Ersek.
Copyright (C) 2009-2024 Antonio Diaz Diaz.
Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -129,9 +129,8 @@ out: xunlock( &omutex );
do {
while( opacket_queues[deliver_id].empty() && num_working > 0 )
{ ++owait_counter; xwait( &oav_or_exit, &omutex ); }
while( true )
while( !opacket_queues[deliver_id].empty() )
{
if( opacket_queues[deliver_id].empty() ) break;
Packet opacket = opacket_queues[deliver_id].front();
opacket_queues[deliver_id].pop();
if( opacket_queues[deliver_id].size() + 1 == out_slots )
@ -275,7 +274,7 @@ void muxer( Packet_courier & courier, const Pretty_print & pp,
if( shared_retval() == 0 &&
writeblock( outfd, opacket.data, opacket.size ) != opacket.size &&
shared_retval.set_value( 1 ) )
{ pp(); show_error( write_error_msg, errno ); }
{ pp(); show_error( wr_err_msg, errno ); }
opacket.delete_data();
}
}

9
dec_stream.cc
View file

@ -1,6 +1,6 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009 Laszlo Ersek.
Copyright (C) 2009-2024 Antonio Diaz Diaz.
Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -185,9 +185,8 @@ out: xunlock( &omutex );
do {
while( opacket_queues[deliver_id].empty() && num_working > 0 )
{ ++owait_counter; xwait( &oav_or_exit, &omutex ); }
while( true )
while( !opacket_queues[deliver_id].empty() )
{
if( opacket_queues[deliver_id].empty() ) break;
Packet opacket = opacket_queues[deliver_id].front();
opacket_queues[deliver_id].pop();
if( opacket_queues[deliver_id].size() + 1 == out_slots )
@ -446,7 +445,7 @@ fail:
if( size + hsize < min_member_size )
{ if( shared_retval.set_value( 2 ) ) show_file_error( pp.name(),
( size <= 0 ) ? "File ends unexpectedly at member header." :
"Input file is too short." ); goto fail; }
"Input file is truncated." ); goto fail; }
const Lzip_header & header = *(const Lzip_header *)buffer;
if( !header.check_magic() )
{ if( shared_retval.set_value( 2 ) )
@ -563,7 +562,7 @@ void muxer( Packet_courier & courier, const Pretty_print & pp,
if( shared_retval() == 0 &&
writeblock( outfd, opacket.data, opacket.size ) != opacket.size &&
shared_retval.set_value( 1 ) )
{ pp(); show_error( write_error_msg, errno ); }
{ pp(); show_error( wr_err_msg, errno ); }
opacket.delete_data();
}
}

2
decompress.cc
View file

@ -1,6 +1,6 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009 Laszlo Ersek.
Copyright (C) 2009-2024 Antonio Diaz Diaz.
Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by

8
doc/plzip.1
View file

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.2.
.TH PLZIP "1" "November 2024" "plzip 1.12-rc1" "User Commands"
.TH PLZIP "1" "January 2025" "plzip 1.12" "User Commands"
.SH NAME
plzip \- reduces the size of files
.SH SYNOPSIS
@ -26,7 +26,7 @@ faster than lzip, at the cost of a slightly reduced compression ratio (0.4
to 2 percent larger compressed files). Note that the number of usable
threads is limited by file size; on files larger than a few GB plzip can use
hundreds of processors, but on files smaller than 1 MiB plzip is no faster
than lzip (even at compression level \fB\-0\fR).
than lzip (not even at compression level \fB\-0\fR).
The number of threads defaults to the number of processors.
.SH OPTIONS
.TP
@ -128,12 +128,12 @@ Plzip home page: http://www.nongnu.org/lzip/plzip.html
.SH COPYRIGHT
Copyright \(co 2009 Laszlo Ersek.
.br
Copyright \(co 2024 Antonio Diaz Diaz.
Copyright \(co 2025 Antonio Diaz Diaz.
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>
.br
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Using lzlib 1.15\-rc1
Using lzlib 1.15
Using LZ_API_VERSION = 1015
.SH "SEE ALSO"
The full documentation for

57
doc/plzip.info
View file

@ -11,7 +11,7 @@ File: plzip.info, Node: Top, Next: Introduction, Up: (dir)
Plzip Manual
************
This manual is for Plzip (version 1.12-rc1, 19 November 2024).
This manual is for Plzip (version 1.12, 9 January 2025).
* Menu:
@ -29,7 +29,7 @@ This manual is for Plzip (version 1.12-rc1, 19 November 2024).
* Concept index:: Index of concepts
Copyright (C) 2009-2024 Antonio Diaz Diaz.
Copyright (C) 2009-2025 Antonio Diaz Diaz.
This manual is free documentation: you have unlimited permission to copy,
distribute, and modify it.
@ -60,7 +60,7 @@ faster than lzip, at the cost of a slightly reduced compression ratio (0.4
to 2 percent larger compressed files). Note that the number of usable
threads is limited by file size; on files larger than a few GB plzip can use
hundreds of processors, but on files smaller than 1 MiB plzip is no faster
than lzip (even at compression level -0). *Note Minimum file sizes::.
than lzip (not even at compression level -0). *Note Minimum file sizes::.
For creation and manipulation of compressed tar archives tarlz can be
more efficient than using tar and plzip because tarlz is able to keep the
@ -70,12 +70,10 @@ alignment between tar members and lzip members. *Note tarlz manual:
The lzip file format is designed for data sharing and long-term
archiving, taking into account both data integrity and decoder availability:
* The lzip format provides very safe integrity checking and some data
recovery means. The program lziprecover can repair bit flip errors
(one of the most common forms of data corruption) in lzip files, and
provides data recovery capabilities, including error-checked merging
of damaged copies of a file. *Note Data safety: (lziprecover)Data
safety.
* The program lziprecover can repair bit flip errors (one of the most
common forms of data corruption) in lzip files, and provides data
recovery capabilities, including error-checked merging of damaged
copies of a file. *Note Data safety: (lziprecover)Data safety.
* The lzip format is as simple as possible (but not simpler). The lzip
manual provides the source code of a simple decompressor along with a
@ -458,9 +456,8 @@ POSIX recommends these conventions for command-line arguments.
* Certain options require an argument.
* An option and its argument may or may not appear as separate tokens.
(In other words, the whitespace separating them is optional, unless the
argument is the empty string). Thus, '-o foo' and '-ofoo' are
equivalent.
(In other words, the whitespace separating them is optional). Thus,
'-o foo' and '-ofoo' are equivalent.
* One or more options without arguments, followed by at most one option
that takes an argument, may follow a hyphen in a single token. Thus,
@ -487,10 +484,6 @@ GNU adds "long options" to these conventions:
tokens. In the latter case they must be separated by an equal sign '='.
Thus, '--foo bar' and '--foo=bar' are equivalent.
The syntax of options with an optional argument is
'-<short_option><argument>' (without whitespace), or
'--<long_option>=<argument>'.

File: plzip.info, Node: File format, Next: Program design, Prev: Argument syntax, Up: Top
@ -853,22 +846,22 @@ Concept index

Tag Table:
Node: Top217
Node: Introduction1207
Node: Output5956
Node: Invoking plzip7519
Ref: --trailing-error8365
Ref: --data-size8603
Node: Argument syntax19941
Node: File format21886
Ref: coded-dict-size23386
Node: Program design24637
Node: Memory requirements26933
Node: Minimum file sizes28643
Node: Trailing data30648
Node: Examples32991
Ref: concat-example34420
Node: Problems35174
Node: Concept index35729
Node: Introduction1201
Node: Output5860
Node: Invoking plzip7423
Ref: --trailing-error8269
Ref: --data-size8507
Node: Argument syntax19845
Node: File format21609
Ref: coded-dict-size23109
Node: Program design24360
Node: Memory requirements26656
Node: Minimum file sizes28366
Node: Trailing data30371
Node: Examples32714
Ref: concat-example34143
Node: Problems34897
Node: Concept index35452

End Tag Table

19
doc/plzip.texi
View file

@ -6,8 +6,8 @@
@finalout
@c %**end of header
@set UPDATED 19 November 2024
@set VERSION 1.12-rc1
@set UPDATED 9 January 2025
@set VERSION 1.12
@dircategory Compression
@direntry
@ -51,7 +51,7 @@ This manual is for Plzip (version @value{VERSION}, @value{UPDATED}).
@end menu
@sp 1
Copyright @copyright{} 2009-2024 Antonio Diaz Diaz.
Copyright @copyright{} 2009-2025 Antonio Diaz Diaz.
This manual is free documentation: you have unlimited permission to copy,
distribute, and modify it.
@ -84,7 +84,7 @@ faster than lzip, at the cost of a slightly reduced compression ratio (0.4
to 2 percent larger compressed files). Note that the number of usable
threads is limited by file size; on files larger than a few GB plzip can use
hundreds of processors, but on files smaller than @w{1 MiB} plzip is no faster
than lzip (even at compression level -0). @xref{Minimum file sizes}.
than lzip (not even at compression level -0). @xref{Minimum file sizes}.
For creation and manipulation of compressed tar archives
@uref{http://www.nongnu.org/lzip/manual/tarlz_manual.html,,tarlz} can be more
@ -99,8 +99,7 @@ taking into account both data integrity and decoder availability:
@itemize @bullet
@item
The lzip format provides very safe integrity checking and some data
recovery means. The program
The program
@uref{http://www.nongnu.org/lzip/manual/lziprecover_manual.html#Data-safety,,lziprecover}
can repair bit flip errors (one of the most common forms of data corruption)
in lzip files, and provides data recovery capabilities, including
@ -509,8 +508,7 @@ POSIX recommends these conventions for command-line arguments.
@item Certain options require an argument.
@item An option and its argument may or may not appear as separate tokens.
(In other words, the whitespace separating them is optional, unless the
argument is the empty string).
(In other words, the whitespace separating them is optional).
Thus, @w{@option{-o foo}} and @option{-ofoo} are equivalent.
@item One or more options without arguments, followed by at most one option
@ -541,11 +539,6 @@ tokens. In the latter case they must be separated by an equal sign @samp{=}.
Thus, @w{@option{--foo bar}} and @option{--foo=bar} are equivalent.
@end itemize
@noindent
The syntax of options with an optional argument is
@option{-<short_option><argument>} (without whitespace), or
@option{--<long_option>=<argument>}.
@node File format
@chapter File format

4
list.cc
View file

@ -1,5 +1,5 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009-2024 Antonio Diaz Diaz.
Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -117,7 +117,7 @@ int list_files( const std::vector< std::string > & filenames,
std::fflush( stdout );
}
if( verbosity >= 0 && ( std::ferror( stdout ) || std::fclose( stdout ) != 0 ) )
{ show_file_error( "(stdout)", write_error_msg, errno );
{ show_file_error( "(stdout)", wr_err_msg, errno );
set_retval( retval, 1 ); }
return retval;
}

4
lzip.h
View file

@ -1,5 +1,5 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009-2024 Antonio Diaz Diaz.
Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -210,7 +210,7 @@ const char * const corrupt_mm_msg = "Corrupt header in multimember file.";
const char * const empty_msg = "Empty member not allowed.";
const char * const mem_msg = "Not enough memory.";
const char * const trailing_msg = "Trailing data not allowed.";
const char * const write_error_msg = "Write error";
const char * const wr_err_msg = "Write error";
// defined in compress.cc
int readblock( const int fd, uint8_t * const buf, const int size );

4
lzip_index.cc
View file

@ -1,5 +1,5 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009-2024 Antonio Diaz Diaz.
Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -153,7 +153,7 @@ Lzip_index::Lzip_index( const int infd, const Cl_options & cl_opts )
( !read_header( infd, header, 0 ) ||
!check_header( header, true ) ) ) return;
if( insize < min_member_size )
{ error_ = "Input file is too short."; retval_ = 2; return; }
{ error_ = "Input file is truncated."; retval_ = 2; return; }
if( insize > INT64_MAX )
{ error_ = "Input file is too long (2^63 bytes or more).";
retval_ = 2; return; }

2
lzip_index.h
View file

@ -1,5 +1,5 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009-2024 Antonio Diaz Diaz.
Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by

10
main.cc
View file

@ -1,6 +1,6 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009 Laszlo Ersek.
Copyright (C) 2009-2024 Antonio Diaz Diaz.
Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -79,7 +79,7 @@ int verbosity = 0;
namespace {
const char * const program_name = "plzip";
const char * const program_year = "2024";
const char * const program_year = "2025";
const char * invocation_name = program_name; // default value
const struct { const char * from; const char * to; } known_extensions[] = {
@ -122,7 +122,7 @@ void show_help( const long num_online )
"to 2 percent larger compressed files). Note that the number of usable\n"
"threads is limited by file size; on files larger than a few GB plzip can use\n"
"hundreds of processors, but on files smaller than 1 MiB plzip is no faster\n"
"than lzip (even at compression level -0).\n"
"than lzip (not even at compression level -0).\n"
"The number of threads defaults to the number of processors.\n"
"\nUsage: %s [options] [files]\n", invocation_name );
std::printf( "\nOptions:\n"
@ -297,7 +297,7 @@ void show_header( const unsigned dictionary_size )
namespace {
// separate numbers of 6 or more digits in groups of 3 digits using '_'
// separate numbers of 5 or more digits in groups of 3 digits using '_'
const char * format_num3( unsigned long long num )
{
enum { buffers = 8, bufsize = 4 * sizeof num, n = 10 };
@ -320,7 +320,7 @@ const char * format_num3( unsigned long long num )
{ num /= 1000; prefix = si_prefix[i]; }
if( prefix ) *(--p) = prefix;
}
const bool split = num >= 100000;
const bool split = num >= 10000;
for( int i = 0; ; )
{

2
testsuite/check.sh
View file

@ -1,6 +1,6 @@
#! /bin/sh
# check script for Plzip - Massively parallel implementation of lzip
# Copyright (C) 2009-2024 Antonio Diaz Diaz.
# Copyright (C) 2009-2025 Antonio Diaz Diaz.
#
# This script is free software: you have unlimited permission
# to copy, distribute, and modify it.