lzip/plzip
1
0
Fork 0
Code Issues Activity

Adding upstream version 1.10.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-24 04:18:11 +01:00
parent e28a4525c4
commit bc90ea9112
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
22 changed files with 565 additions and 472 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
ChangeLog
View file

@ -1,3 +1,13 @@
2022-01-24 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.10 released.
* main.cc (getnum): Show option name and valid range if error.
(check_lib): Check that LZ_API_VERSION and LZ_version_string match.
* Set variable LIBS from configure.
* Improve several descriptions in manual, '--help', and man page.
* plzip.texi: Change GNU Texinfo category to 'Compression'.
(Reported by Alfred M. Szmidt).
2021-01-03 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.9 released.
@ -37,7 +47,7 @@
* main.cc (main): Check return value of close( infd ).
* plzip.texi: Improve description of '-0..-9', '-m', and '-s'.
* configure: New option '--with-mingw'.
* configure: Accept appending to CXXFLAGS, 'CXXFLAGS+=OPTIONS'.
* configure: Accept appending to CXXFLAGS; 'CXXFLAGS+=OPTIONS'.
* INSTALL: Document use of CXXFLAGS+='-D __USE_MINGW_ANSI_STDIO'.
2018-02-07 Antonio Diaz Diaz <antonio@gnu.org>
@ -164,7 +174,7 @@
* Small portability fixes.
* plzip.texinfo: New chapter 'Program Design'.
Add missing description of option '-n, --threads'.
* Debug stats have been fixed.
* Fix debug statistics.
2010-02-10 Antonio Diaz Diaz <ant_diaz@teleline.es>
@ -210,7 +220,7 @@
until something better appears on the net.
Copyright (C) 2009-2021 Antonio Diaz Diaz.
Copyright (C) 2009-2022 Antonio Diaz Diaz.
This file is a collection of facts, and thus it is not copyrightable,
but just in case, you have unlimited permission to copy, distribute, and

8
INSTALL
View file

@ -1,8 +1,8 @@
Requirements
------------
You will need a C++11 compiler and the compression library lzlib installed.
(gcc 3.3.6 or newer is recommended).
I use gcc 6.1.0 and 4.1.2, but the code should compile with any standards
You will need a C++98 compiler with suport for 'long long', and the
compression library lzlib installed. (gcc 3.3.6 or newer is recommended).
I use gcc 6.1.0 and 3.3.6, but the code should compile with any standards
compliant compiler.
Lzlib must be version 1.0 or newer, but the fast encoder requires lzlib 1.7
@ -86,7 +86,7 @@ After running 'configure', you can run 'make' and 'make install' as
explained above.
Copyright (C) 2009-2021 Antonio Diaz Diaz.
Copyright (C) 2009-2022 Antonio Diaz Diaz.
This file is free documentation: you have unlimited permission to copy,
distribute, and modify it.

4
Makefile.in
View file

@ -4,7 +4,6 @@ INSTALL = install
INSTALL_PROGRAM = $(INSTALL) -m 755
INSTALL_DATA = $(INSTALL) -m 644
INSTALL_DIR = $(INSTALL) -d -m 755
LIBS = -llz -lpthread
SHELL = /bin/sh
CAN_RUN_INSTALLINFO = $(SHELL) -c "install-info --version" > /dev/null 2>&1
@ -22,7 +21,7 @@ objs = arg_parser.o lzip_index.o list.o compress.o dec_stdout.o \
all : $(progname)
$(progname) : $(objs)
$(CXX) $(LDFLAGS) $(CXXFLAGS) -o $@ $(objs) $(LIBS)
$(CXX) $(CXXFLAGS) $(LDFLAGS) -o $@ $(objs) $(LIBS)
decompress.o : decompress.cc
$(CXX) $(CPPFLAGS) $(CXXFLAGS) $(with_mingw) -c -o $@ $<
@ -129,6 +128,7 @@ dist : doc
$(DISTNAME)/*.cc \
$(DISTNAME)/testsuite/check.sh \
$(DISTNAME)/testsuite/test.txt \
$(DISTNAME)/testsuite/fox.lz \
$(DISTNAME)/testsuite/fox_*.lz \
$(DISTNAME)/testsuite/test.txt.lz \
$(DISTNAME)/testsuite/test_em.txt.lz

61
NEWS
View file

@ -1,58 +1,13 @@
Changes in version 1.9:
Changes in version 1.10:
Plzip now reports an error if a file name is empty (plzip -t "").
In case of error in a numerical argument to a command line option, plzip
now shows the name of the option and the range of valid values.
Option '-o, --output' now behaves like '-c, --stdout', but sending the
output unconditionally to a file instead of to standard output. See the new
description of '-o' in the manual. This change is backwards compatible only
when (de)compressing from standard input alone. Therefore commands like:
plzip -o foo.lz - bar < foo
must now be split into:
plzip -o foo.lz - < foo
plzip bar
or rewritten as:
plzip - bar < foo > foo.lz
'--check-lib' now checks that LZ_API_VERSION and LZ_version_string match.
When using '-c' or '-o', plzip now checks whether the output is a terminal
only once.
The variable LIBS can now be set from configure.
Plzip now does not even open the output file if the input file is a terminal.
Several descriptions have been improved in manual, '--help', and man page.
The new option '--check-lib', which compares the version of lzlib used to
compile plzip with the version actually being used at run time, has been added.
The words 'decompressed' and 'compressed' have been replaced with the
shorter 'out' and 'in' in the verbose output when decompressing or testing.
When checking the integrity of multiple files, plzip is now able to continue
checking the rest of the files (instead of exiting) if some of them fail the
test, allowing 'plzip --test' to show a final diagnostic with the number of
files that failed (just as 'lzip --test').
Testing is now slightly (1.6%) faster when using lzlib 1.12.
When compressing, or when decompressing or testing from a non-seekable file
or from standard input, plzip now starts only the number of worker threads
required.
When decompressing or testing from a non-seekable file or from standard
input, trailing data are now not counted in the compressed size shown.
When decompressing or testing a multimember file, plzip now shows the
largest dictionary size of all members in the file instead of showing the
dictionary size of the first member.
Option '--list' now reports corruption or truncation of the last header in a
multimenber file specifically instead of showing the generic message "Last
member in input file is truncated or corrupt."
The error messages for 'Data error' and 'Unexpected EOF' have been shortened.
The commands needed to extract files from a tar.lz archive have been
documented in the manual, in the output of '--help', and in the man page.
Tarlz is mentioned in the manual as an alternative to tar + plzip.
Several fixes and improvements have been made to the manual.
8 new test files have been added to the testsuite.
The texinfo category of the manual has been changed from 'Data Compression'
to 'Compression' to match that of gzip. (Reported by Alfred M. Szmidt).

19
README
View file

@ -5,13 +5,14 @@ compatible with lzip 1.4 or newer. Plzip uses the compression library lzlib.
Lzip is a lossless data compressor with a user interface similar to the one
of gzip or bzip2. Lzip uses a simplified form of the 'Lempel-Ziv-Markov
chain-Algorithm' (LZMA) stream format, chosen to maximize safety and
interoperability. Lzip can compress about as fast as gzip (lzip -0) or
compress most files more than bzip2 (lzip -9). Decompression speed is
intermediate between gzip and bzip2. Lzip is better than gzip and bzip2 from
a data recovery perspective. Lzip has been designed, written, and tested
with great care to replace gzip and bzip2 as the standard general-purpose
compressed format for unix-like systems.
chain-Algorithm' (LZMA) stream format and provides a 3 factor integrity
checking to maximize interoperability and optimize safety. Lzip can compress
about as fast as gzip (lzip -0) or compress most files more than bzip2
(lzip -9). Decompression speed is intermediate between gzip and bzip2.
Lzip is better than gzip and bzip2 from a data recovery perspective. Lzip
has been designed, written, and tested with great care to replace gzip and
bzip2 as the standard general-purpose compressed format for unix-like
systems.
Plzip can compress/decompress large files on multiprocessor machines much
faster than lzip, at the cost of a slightly reduced compression ratio (0.4
@ -76,7 +77,7 @@ filename.lz becomes filename
filename.tlz becomes filename.tar
anyothername becomes anyothername.out
(De)compressing a file is much like copying or moving it; therefore plzip
(De)compressing a file is much like copying or moving it. Therefore plzip
preserves the access and modification dates, permissions, and, when
possible, ownership of the file just as 'cp -p' does. (If the user ID or
the group ID can't be duplicated, the file permission bits S_ISUID and
@ -100,7 +101,7 @@ been compressed. Decompressed is used to refer to data which have undergone
the process of decompression.
Copyright (C) 2009-2021 Antonio Diaz Diaz.
Copyright (C) 2009-2022 Antonio Diaz Diaz.
This file is free documentation: you have unlimited permission to copy,
distribute, and modify it.

15
arg_parser.cc
View file

@ -1,5 +1,5 @@
/* Arg_parser - POSIX/GNU command line argument parser. (C++ version)
Copyright (C) 2006-2021 Antonio Diaz Diaz.
Copyright (C) 2006-2022 Antonio Diaz Diaz.
This library is free software. Redistribution and use in source and
binary forms, with or without modification, are permitted provided
@ -35,9 +35,10 @@ bool Arg_parser::parse_long_option( const char * const opt, const char * const a
// Test all long options for either exact match or abbreviated matches.
for( int i = 0; options[i].code != 0; ++i )
if( options[i].name && std::strncmp( options[i].name, &opt[2], len ) == 0 )
if( options[i].long_name &&
std::strncmp( options[i].long_name, &opt[2], len ) == 0 )
{
if( std::strlen( options[i].name ) == len ) // Exact match found
if( std::strlen( options[i].long_name ) == len ) // Exact match found
{ index = i; exact = true; break; }
else if( index < 0 ) index = i; // First nonexact match found
else if( options[index].code != options[i].code ||
@ -58,19 +59,19 @@ bool Arg_parser::parse_long_option( const char * const opt, const char * const a
}
++argind;
data.push_back( Record( options[index].code ) );
data.push_back( Record( options[index].code, options[index].long_name ) );
if( opt[len+2] ) // '--<long_option>=<argument>' syntax
{
if( options[index].has_arg == no )
{
error_ = "option '--"; error_ += options[index].name;
error_ = "option '--"; error_ += options[index].long_name;
error_ += "' doesn't allow an argument";
return false;
}
if( options[index].has_arg == yes && !opt[len+3] )
{
error_ = "option '--"; error_ += options[index].name;
error_ = "option '--"; error_ += options[index].long_name;
error_ += "' requires an argument";
return false;
}
@ -82,7 +83,7 @@ bool Arg_parser::parse_long_option( const char * const opt, const char * const a
{
if( !arg || !arg[0] )
{
error_ = "option '--"; error_ += options[index].name;
error_ = "option '--"; error_ += options[index].long_name;
error_ += "' requires an argument";
return false;
}

23
arg_parser.h
View file

@ -1,5 +1,5 @@
/* Arg_parser - POSIX/GNU command line argument parser. (C++ version)
Copyright (C) 2006-2021 Antonio Diaz Diaz.
Copyright (C) 2006-2022 Antonio Diaz Diaz.
This library is free software. Redistribution and use in source and
binary forms, with or without modification, are permitted provided
@ -23,9 +23,9 @@
In case of error, 'error' returns a non-empty error message.
'options' is an array of 'struct Option' terminated by an element
containing a code which is zero. A null name means a short-only
option. A code value outside the unsigned char range means a
long-only option.
containing a code which is zero. A null long_name means a short-only
option. A code value outside the unsigned char range means a long-only
option.
Arg_parser normally makes it appear as if all the option arguments
were specified before all the non-option arguments for the purposes
@ -48,7 +48,7 @@ public:
struct Option
{
int code; // Short option letter or code ( code != 0 )
const char * name; // Long option name (maybe null)
const char * long_name; // Long option name (maybe null)
Has_arg has_arg;
};
@ -56,8 +56,12 @@ private:
struct Record
{
int code;
std::string parsed_name;
std::string argument;
explicit Record( const int c ) : code( c ) {}
explicit Record( const unsigned char c )
: code( c ), parsed_name( "-" ) { parsed_name += c; }
Record( const int c, const char * const long_name )
: code( c ), parsed_name( "--" ) { parsed_name += long_name; }
explicit Record( const char * const arg ) : code( 0 ), argument( arg ) {}
};
@ -91,6 +95,13 @@ public:
else return 0;
}
// Full name of the option parsed (short or long).
const std::string & parsed_name( const int i ) const
{
if( i >= 0 && i < arguments() ) return data[i].parsed_name;
else return empty_arg;
}
const std::string & argument( const int i ) const
{
if( i >= 0 && i < arguments() ) return data[i].argument;

12
compress.cc
View file

@ -1,6 +1,6 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009 Laszlo Ersek.
Copyright (C) 2009-2021 Antonio Diaz Diaz.
Copyright (C) 2009-2022 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -38,8 +38,8 @@
#endif
/* Returns the number of bytes really read.
If (returned value < size) and (errno == 0), means EOF was reached.
/* Return the number of bytes really read.
If (value returned < size) and (errno == 0), means EOF was reached.
*/
int readblock( const int fd, uint8_t * const buf, const int size )
{
@ -57,8 +57,8 @@ int readblock( const int fd, uint8_t * const buf, const int size )
}
/* Returns the number of bytes really written.
If (returned value < size), it is always an error.
/* Return the number of bytes really written.
If (value returned < size), it is always an error.
*/
int writeblock( const int fd, const uint8_t * const buf, const int size )
{
@ -483,7 +483,7 @@ int compress( const unsigned long long cfile_size,
const int infd, const int outfd,
const Pretty_print & pp, const int debug_level )
{
const int offset = data_size / 8;
const int offset = data_size / 8; // offset for compression in-place
const int slots_per_worker = 2;
const int num_slots =
( ( num_workers > 1 ) ? num_workers * slots_per_worker : 1 );

11
configure vendored
View file

@ -1,12 +1,12 @@
#! /bin/sh
# configure script for Plzip - Massively parallel implementation of lzip
# Copyright (C) 2009-2021 Antonio Diaz Diaz.
# Copyright (C) 2009-2022 Antonio Diaz Diaz.
#
# This configure script is free software: you have unlimited permission
# to copy, distribute, and modify it.
pkgname=plzip
pkgversion=1.9
pkgversion=1.10
progname=plzip
with_mingw=
srctrigger=doc/${pkgname}.texi
@ -25,6 +25,7 @@ CXX=g++
CPPFLAGS=
CXXFLAGS='-Wall -W -O2'
LDFLAGS=
LIBS='-llz -lpthread'
# checking whether we are using GNU C++.
/bin/sh -c "${CXX} --version" > /dev/null 2>&1 || { CXX=c++ ; CXXFLAGS=-O2 ; }
@ -71,6 +72,7 @@ while [ $# != 0 ] ; do
echo " CXXFLAGS=OPTIONS command line options for the C++ compiler [${CXXFLAGS}]"
echo " CXXFLAGS+=OPTIONS append options to the current value of CXXFLAGS"
echo " LDFLAGS=OPTIONS command line options for the linker [${LDFLAGS}]"
echo " LIBS=OPTIONS libraries to pass to the linker [${LIBS}]"
echo
exit 0 ;;
--version | -V)
@ -99,6 +101,7 @@ while [ $# != 0 ] ; do
CXXFLAGS=*) CXXFLAGS=${optarg} ;;
CXXFLAGS+=*) CXXFLAGS="${CXXFLAGS} ${optarg}" ;;
LDFLAGS=*) LDFLAGS=${optarg} ;;
LIBS=*) LIBS="${optarg} ${LIBS}" ;;
--*)
echo "configure: WARNING: unrecognized option: '${option}'" 1>&2 ;;
@ -168,10 +171,11 @@ echo "CXX = ${CXX}"
echo "CPPFLAGS = ${CPPFLAGS}"
echo "CXXFLAGS = ${CXXFLAGS}"
echo "LDFLAGS = ${LDFLAGS}"
echo "LIBS = ${LIBS}"
rm -f Makefile
cat > Makefile << EOF
# Makefile for Plzip - Massively parallel implementation of lzip
# Copyright (C) 2009-2021 Antonio Diaz Diaz.
# Copyright (C) 2009-2022 Antonio Diaz Diaz.
# This file was generated automatically by configure. Don't edit.
#
# This Makefile is free software: you have unlimited permission
@ -192,6 +196,7 @@ CXX = ${CXX}
CPPFLAGS = ${CPPFLAGS}
CXXFLAGS = ${CXXFLAGS}
LDFLAGS = ${LDFLAGS}
LIBS = ${LIBS}
EOF
cat "${srcdir}/Makefile.in" >> Makefile

2
dec_stdout.cc
View file

@ -1,6 +1,6 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009 Laszlo Ersek.
Copyright (C) 2009-2021 Antonio Diaz Diaz.
Copyright (C) 2009-2022 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by

2
dec_stream.cc
View file

@ -1,6 +1,6 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009 Laszlo Ersek.
Copyright (C) 2009-2021 Antonio Diaz Diaz.
Copyright (C) 2009-2022 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by

12
decompress.cc
View file

@ -1,6 +1,6 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009 Laszlo Ersek.
Copyright (C) 2009-2021 Antonio Diaz Diaz.
Copyright (C) 2009-2022 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -39,7 +39,7 @@
/* This code is based on a patch by Hannes Domani, <ssbssa@yahoo.de> to make
possible compiling plzip under MS Windows (with MINGW compiler).
*/
#if defined(__MSVCRT__) && defined(WITH_MINGW)
#if defined __MSVCRT__ && defined WITH_MINGW
#include <windows.h>
#warning "Parallel I/O is not guaranteed to work on Windows."
@ -76,8 +76,8 @@ ssize_t pwrite( int fd, const void *buf, size_t count, uint64_t offset )
#endif // __MSVCRT__
/* Returns the number of bytes really read.
If (returned value < size) and (errno == 0), means EOF was reached.
/* Return the number of bytes really read.
If (value returned < size) and (errno == 0), means EOF was reached.
*/
int preadblock( const int fd, uint8_t * const buf, const int size,
const long long pos )
@ -96,8 +96,8 @@ int preadblock( const int fd, uint8_t * const buf, const int size,
}
/* Returns the number of bytes really written.
If (returned value < size), it is always an error.
/* Return the number of bytes really written.
If (value returned < size), it is always an error.
*/
int pwriteblock( const int fd, const uint8_t * const buf, const int size,
const long long pos )

23
doc/plzip.1
View file

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.16.
.TH PLZIP "1" "January 2021" "plzip 1.9" "User Commands"
.TH PLZIP "1" "January 2022" "plzip 1.10" "User Commands"
.SH NAME
plzip \- reduces the size of files
.SH SYNOPSIS
@ -11,13 +11,14 @@ compatible with lzip 1.4 or newer. Plzip uses the compression library lzlib.
.PP
Lzip is a lossless data compressor with a user interface similar to the one
of gzip or bzip2. Lzip uses a simplified form of the 'Lempel\-Ziv\-Markov
chain\-Algorithm' (LZMA) stream format, chosen to maximize safety and
interoperability. Lzip can compress about as fast as gzip (lzip \fB\-0\fR) or
compress most files more than bzip2 (lzip \fB\-9\fR). Decompression speed is
intermediate between gzip and bzip2. Lzip is better than gzip and bzip2 from
a data recovery perspective. Lzip has been designed, written, and tested
with great care to replace gzip and bzip2 as the standard general\-purpose
compressed format for unix\-like systems.
chain\-Algorithm' (LZMA) stream format and provides a 3 factor integrity
checking to maximize interoperability and optimize safety. Lzip can compress
about as fast as gzip (lzip \fB\-0\fR) or compress most files more than bzip2
(lzip \fB\-9\fR). Decompression speed is intermediate between gzip and bzip2.
Lzip is better than gzip and bzip2 from a data recovery perspective. Lzip
has been designed, written, and tested with great care to replace gzip and
bzip2 as the standard general\-purpose compressed format for unix\-like
systems.
.PP
Plzip can compress/decompress large files on multiprocessor machines much
faster than lzip, at the cost of a slightly reduced compression ratio (0.4
@ -116,7 +117,7 @@ To extract all the files from archive 'foo.tar.lz', use the commands
.PP
Exit status: 0 for a normal exit, 1 for environmental problems (file
not found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or
invalid input file, 3 for an internal consistency error (eg, bug) which
invalid input file, 3 for an internal consistency error (e.g., bug) which
caused plzip to panic.
.SH "REPORTING BUGS"
Report bugs to lzip\-bug@nongnu.org
@ -125,8 +126,8 @@ Plzip home page: http://www.nongnu.org/lzip/plzip.html
.SH COPYRIGHT
Copyright \(co 2009 Laszlo Ersek.
.br
Copyright \(co 2021 Antonio Diaz Diaz.
Using lzlib 1.12
Copyright \(co 2022 Antonio Diaz Diaz.
Using lzlib 1.13
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>
.br
This is free software: you are free to change and redistribute it.

283
doc/plzip.info
View file

@ -1,6 +1,6 @@
This is plzip.info, produced by makeinfo version 4.13+ from plzip.texi.
INFO-DIR-SECTION Data Compression
INFO-DIR-SECTION Compression
START-INFO-DIR-ENTRY
* Plzip: (plzip). Massively parallel implementation of lzip
END-INFO-DIR-ENTRY
@ -11,7 +11,7 @@ File: plzip.info, Node: Top, Next: Introduction, Up: (dir)
Plzip Manual
************
This manual is for Plzip (version 1.9, 3 January 2021).
This manual is for Plzip (version 1.10, 24 January 2022).
* Menu:
@ -19,16 +19,16 @@ This manual is for Plzip (version 1.9, 3 January 2021).
* Output:: Meaning of plzip's output
* Invoking plzip:: Command line interface
* Program design:: Internal structure of plzip
* File format:: Detailed format of the compressed file
* Memory requirements:: Memory required to compress and decompress
* Minimum file sizes:: Minimum file sizes required for full speed
* File format:: Detailed format of the compressed file
* Trailing data:: Extra data appended to the file
* Examples:: A small tutorial with examples
* Problems:: Reporting bugs
* Concept index:: Index of concepts
Copyright (C) 2009-2021 Antonio Diaz Diaz.
Copyright (C) 2009-2022 Antonio Diaz Diaz.
This manual is free documentation: you have unlimited permission to copy,
distribute, and modify it.
@ -44,13 +44,14 @@ compatible with lzip 1.4 or newer. Plzip uses the compression library lzlib.
Lzip is a lossless data compressor with a user interface similar to the
one of gzip or bzip2. Lzip uses a simplified form of the 'Lempel-Ziv-Markov
chain-Algorithm' (LZMA) stream format, chosen to maximize safety and
interoperability. Lzip can compress about as fast as gzip (lzip -0) or
compress most files more than bzip2 (lzip -9). Decompression speed is
intermediate between gzip and bzip2. Lzip is better than gzip and bzip2 from
a data recovery perspective. Lzip has been designed, written, and tested
with great care to replace gzip and bzip2 as the standard general-purpose
compressed format for unix-like systems.
chain-Algorithm' (LZMA) stream format and provides a 3 factor integrity
checking to maximize interoperability and optimize safety. Lzip can compress
about as fast as gzip (lzip -0) or compress most files more than bzip2
(lzip -9). Decompression speed is intermediate between gzip and bzip2. Lzip
is better than gzip and bzip2 from a data recovery perspective. Lzip has
been designed, written, and tested with great care to replace gzip and
bzip2 as the standard general-purpose compressed format for unix-like
systems.
Plzip can compress/decompress large files on multiprocessor machines much
faster than lzip, at the cost of a slightly reduced compression ratio (0.4
@ -107,7 +108,7 @@ filename.lz becomes filename
filename.tlz becomes filename.tar
anyothername becomes anyothername.out
(De)compressing a file is much like copying or moving it; therefore plzip
(De)compressing a file is much like copying or moving it. Therefore plzip
preserves the access and modification dates, permissions, and, when
possible, ownership of the file just as 'cp -p' does. (If the user ID or
the group ID can't be duplicated, the file permission bits S_ISUID and
@ -206,7 +207,7 @@ once, the first time it appears in the command line.
'-B BYTES'
'--data-size=BYTES'
When compressing, set the size of the input data blocks in bytes. The
When compressing, set the size in bytes of the input data blocks. The
input file will be divided in chunks of this size before compression is
performed. Valid values range from 8 KiB to 1 GiB. Default value is
two times the dictionary size, except for option '-0' where it
@ -224,10 +225,13 @@ once, the first time it appears in the command line.
'-d'
'--decompress'
Decompress the files specified. If a file does not exist or can't be
opened, plzip continues decompressing the rest of the files. If a file
fails to decompress, or is a terminal, plzip exits immediately without
decompressing the rest of the files.
Decompress the files specified. If a file does not exist, can't be
opened, or the destination file already exists and '--force' has not
been specified, plzip continues decompressing the rest of the files
and exits with error status 1. If a file fails to decompress, or is a
terminal, plzip exits immediately with error status 2 without
decompressing the rest of the files. A terminal is considered an
uncompressed file, and therefore invalid.
'-f'
'--force'
@ -253,10 +257,12 @@ once, the first time it appears in the command line.
positions and sizes of each member in multimember files are also
printed.
'-lq' can be used to verify quickly (without decompressing) the
structural integrity of the files specified. (Use '--test' to verify
the data integrity). '-alq' additionally verifies that none of the
files specified contain trailing data.
If any file is damaged, does not exist, can't be opened, or is not
regular, the final exit status will be > 0. '-lq' can be used to verify
quickly (without decompressing) the structural integrity of the files
specified. (Use '--test' to verify the data integrity). '-alq'
additionally verifies that none of the files specified contain
trailing data.
'-m BYTES'
'--match-length=BYTES'
@ -395,9 +401,10 @@ once, the first time it appears in the command line.
actually being used at run time and exit. Report any differences
found. Exit with error status 1 if differences are found. A mismatch
may indicate that lzlib is not correctly installed or that a different
version of lzlib has been installed after compiling plzip.
version of lzlib has been installed after compiling plzip. Exit with
error status 2 if LZ_API_VERSION and LZ_version_string don't match.
'plzip -v --check-lib' shows the version of lzlib being used and the
value of 'LZ_API_VERSION' (if defined). *Note Library version:
value of LZ_API_VERSION (if defined). *Note Library version:
(lzlib)Library version.
@ -419,18 +426,19 @@ Y yottabyte (10^24) | Yi yobibyte (2^80)
Exit status: 0 for a normal exit, 1 for environmental problems (file not
found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or invalid
input file, 3 for an internal consistency error (eg, bug) which caused
input file, 3 for an internal consistency error (e.g., bug) which caused
plzip to panic.

File: plzip.info, Node: Program design, Next: File format, Prev: Invoking plzip, Up: Top
File: plzip.info, Node: Program design, Next: Memory requirements, Prev: Invoking plzip, Up: Top
4 Internal structure of plzip
*****************************
When compressing, plzip divides the input file into chunks and compresses as
many chunks simultaneously as worker threads are chosen, creating a
multimember compressed file.
multimember compressed file. Each chunk is compressed in-place (using the
same buffer for input and output), reducing the amount of RAM required.
When decompressing, plzip decompresses as many members simultaneously as
worker threads are chosen. Files that were compressed with lzip will not be
@ -448,14 +456,14 @@ to the workers. The workers (de)compress the blocks received from the
splitter. The muxer collects processed packets from the workers, and writes
them to the output file.
,------------,
.------------.
,-->| worker 0 |--,
| `------------' |
,-------, ,----------, | ,------------, | ,-------, ,--------,
.-------. .----------. | .------------. | .-------. .--------.
| input |-->| splitter |-+-->| worker 1 |--+-->| muxer |-->| output |
| file | `----------' | `------------' | `-------' | file |
`-------' | ... | `--------'
| ,------------, |
| .------------. |
`-->| worker N-1 |--'
`------------'
@ -467,82 +475,9 @@ reduced and the decompression speed of large files with many members is
only limited by the number of processors available and by I/O speed.

File: plzip.info, Node: File format, Next: Memory requirements, Prev: Program design, Up: Top
File: plzip.info, Node: Memory requirements, Next: Minimum file sizes, Prev: Program design, Up: Top
5 File format
*************
Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away.
-- Antoine de Saint-Exupery
In the diagram below, a box like this:
+---+
| | <-- the vertical bars might be missing
+---+
represents one byte; a box like this:
+==============+
| |
+==============+
represents a variable number of bytes.
A lzip file consists of a series of "members" (compressed data sets).
The members simply appear one after another in the file, with no additional
information before, between, or after them.
Each member has the following structure:
+--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ID string | VN | DS | LZMA stream | CRC32 | Data size | Member size |
+--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
All multibyte values are stored in little endian order.
'ID string (the "magic" bytes)'
A four byte string, identifying the lzip format, with the value "LZIP"
(0x4C, 0x5A, 0x49, 0x50).
'VN (version number, 1 byte)'
Just in case something needs to be modified in the future. 1 for now.
'DS (coded dictionary size, 1 byte)'
The dictionary size is calculated by taking a power of 2 (the base
size) and subtracting from it a fraction between 0/16 and 7/16 of the
base size.
Bits 4-0 contain the base 2 logarithm of the base size (12 to 29).
Bits 7-5 contain the numerator of the fraction (0 to 7) to subtract
from the base size to obtain the dictionary size.
Example: 0xD3 = 2^19 - 6 * 2^15 = 512 KiB - 6 * 32 KiB = 320 KiB
Valid values for dictionary size range from 4 KiB to 512 MiB.
'LZMA stream'
The LZMA stream, finished by an end of stream marker. Uses default
values for encoder properties. *Note Stream format: (lzip)Stream
format, for a complete description.
'CRC32 (4 bytes)'
Cyclic Redundancy Check (CRC) of the uncompressed original data.
'Data size (8 bytes)'
Size of the uncompressed original data.
'Member size (8 bytes)'
Total size of the member, including header and trailer. This field acts
as a distributed index, allows the verification of stream integrity,
and facilitates safe recovery of undamaged members from multimember
files.

File: plzip.info, Node: Memory requirements, Next: Minimum file sizes, Prev: File format, Up: Top
6 Memory required to compress and decompress
5 Memory required to compress and decompress
********************************************
The amount of memory required *per worker thread* for decompression or
@ -588,9 +523,9 @@ Level Memory required
-9 568 MiB

File: plzip.info, Node: Minimum file sizes, Next: Trailing data, Prev: Memory requirements, Up: Top
File: plzip.info, Node: Minimum file sizes, Next: File format, Prev: Memory requirements, Up: Top
7 Minimum file sizes required for full compression speed
6 Minimum file sizes required for full compression speed
********************************************************
When compressing, plzip divides the input file into chunks and compresses
@ -625,7 +560,83 @@ Level
-9 128 MiB 256 MiB 512 MiB 1 GiB 4 GiB 16 GiB

File: plzip.info, Node: Trailing data, Next: Examples, Prev: Minimum file sizes, Up: Top
File: plzip.info, Node: File format, Next: Trailing data, Prev: Minimum file sizes, Up: Top
7 File format
*************
Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away.
-- Antoine de Saint-Exupery
In the diagram below, a box like this:
+---+
| | <-- the vertical bars might be missing
+---+
represents one byte; a box like this:
+==============+
| |
+==============+
represents a variable number of bytes.
A lzip file consists of a series of independent "members" (compressed
data sets). The members simply appear one after another in the file, with no
additional information before, between, or after them. Each member can
encode in compressed form up to 16 EiB - 1 byte of uncompressed data. The
size of a multimember file is unlimited.
Each member has the following structure:
+--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ID string | VN | DS | LZMA stream | CRC32 | Data size | Member size |
+--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
All multibyte values are stored in little endian order.
'ID string (the "magic" bytes)'
A four byte string, identifying the lzip format, with the value "LZIP"
(0x4C, 0x5A, 0x49, 0x50).
'VN (version number, 1 byte)'
Just in case something needs to be modified in the future. 1 for now.
'DS (coded dictionary size, 1 byte)'
The dictionary size is calculated by taking a power of 2 (the base
size) and subtracting from it a fraction between 0/16 and 7/16 of the
base size.
Bits 4-0 contain the base 2 logarithm of the base size (12 to 29).
Bits 7-5 contain the numerator of the fraction (0 to 7) to subtract
from the base size to obtain the dictionary size.
Example: 0xD3 = 2^19 - 6 * 2^15 = 512 KiB - 6 * 32 KiB = 320 KiB
Valid values for dictionary size range from 4 KiB to 512 MiB.
'LZMA stream'
The LZMA stream, finished by an "End Of Stream" marker. Uses default
values for encoder properties. *Note Stream format: (lzip)Stream
format, for a complete description.
'CRC32 (4 bytes)'
Cyclic Redundancy Check (CRC) of the original uncompressed data.
'Data size (8 bytes)'
Size of the original uncompressed data.
'Member size (8 bytes)'
Total size of the member, including header and trailer. This field acts
as a distributed index, allows the verification of stream integrity,
and facilitates the safe recovery of undamaged members from
multimember files. Member size should be limited to 2 PiB to prevent
the data size field from overflowing.

File: plzip.info, Node: Trailing data, Next: Examples, Prev: File format, Up: Top
8 Extra data appended to the file
*********************************
@ -699,7 +710,7 @@ show the compression ratio.
plzip -v file
Example 3: Like example 1 but the created 'file.lz' has a block size of
Example 3: Like example 2 but the created 'file.lz' has a block size of
1 MiB. The compression ratio is not shown.
plzip -B 1MiB file
@ -717,15 +728,7 @@ status.
plzip -tv file.lz
Example 6: Compress a whole device in /dev/sdc and send the output to
'file.lz'.
plzip -c /dev/sdc > file.lz
or
plzip /dev/sdc -o file.lz
Example 7: The right way of concatenating the decompressed output of two or
Example 6: The right way of concatenating the decompressed output of two or
more compressed files. *Note Trailing data::.
Don't do this
@ -734,17 +737,25 @@ more compressed files. *Note Trailing data::.
plzip -cd file1.lz file2.lz file3.lz
Example 8: Decompress 'file.lz' partially until 10 KiB of decompressed data
Example 7: Decompress 'file.lz' partially until 10 KiB of decompressed data
are produced.
plzip -cd file.lz | dd bs=1024 count=10
Example 9: Decompress 'file.lz' partially from decompressed byte at offset
Example 8: Decompress 'file.lz' partially from decompressed byte at offset
10000 to decompressed byte at offset 14999 (5000 bytes are produced).
plzip -cd file.lz | dd bs=1000 skip=10 count=5
Example 9: Compress a whole device in /dev/sdc and send the output to
'file.lz'.
plzip -c /dev/sdc > file.lz
or
plzip /dev/sdc -o file.lz

File: plzip.info, Node: Problems, Next: Concept index, Prev: Examples, Up: Top
@ -758,7 +769,7 @@ eternity, if not longer.
If you find a bug in plzip, please send electronic mail to
<lzip-bug@nongnu.org>. Include the version number, which you can find by
running 'plzip --version'.
running 'plzip --version' and 'plzip -v --check-lib'.

File: plzip.info, Node: Concept index, Prev: Problems, Up: Top
@ -787,22 +798,22 @@ Concept index

Tag Table:
Node: Top222
Node: Introduction1159
Node: Output5788
Node: Invoking plzip7351
Ref: --trailing-error8146
Ref: --data-size8384
Node: Program design18364
Node: File format20542
Ref: coded-dict-size21840
Node: Memory requirements22995
Node: Minimum file sizes24677
Node: Trailing data26693
Node: Examples28961
Ref: concat-example30556
Node: Problems31153
Node: Concept index31681
Node: Top217
Node: Introduction1156
Node: Output5829
Node: Invoking plzip7392
Ref: --trailing-error8187
Ref: --data-size8425
Node: Program design18819
Node: Memory requirements21122
Node: Minimum file sizes22807
Node: File format24821
Ref: coded-dict-size26260
Node: Trailing data27514
Node: Examples29775
Ref: concat-example31210
Node: Problems31967
Node: Concept index32522

End Tag Table

287
doc/plzip.texi
View file

@ -6,10 +6,10 @@
@finalout
@c %**end of header
@set UPDATED 3 January 2021
@set VERSION 1.9
@set UPDATED 24 January 2022
@set VERSION 1.10
@dircategory Data Compression
@dircategory Compression
@direntry
* Plzip: (plzip). Massively parallel implementation of lzip
@end direntry
@ -40,9 +40,9 @@ This manual is for Plzip (version @value{VERSION}, @value{UPDATED}).
* Output:: Meaning of plzip's output
* Invoking plzip:: Command line interface
* Program design:: Internal structure of plzip
* File format:: Detailed format of the compressed file
* Memory requirements:: Memory required to compress and decompress
* Minimum file sizes:: Minimum file sizes required for full speed
* File format:: Detailed format of the compressed file
* Trailing data:: Extra data appended to the file
* Examples:: A small tutorial with examples
* Problems:: Reporting bugs
@ -50,7 +50,7 @@ This manual is for Plzip (version @value{VERSION}, @value{UPDATED}).
@end menu
@sp 1
Copyright @copyright{} 2009-2021 Antonio Diaz Diaz.
Copyright @copyright{} 2009-2022 Antonio Diaz Diaz.
This manual is free documentation: you have unlimited permission to copy,
distribute, and modify it.
@ -69,13 +69,14 @@ compatible with lzip 1.4 or newer. Plzip uses the compression library
@uref{http://www.nongnu.org/lzip/lzip.html,,Lzip}
is a lossless data compressor with a user interface similar to the one
of gzip or bzip2. Lzip uses a simplified form of the 'Lempel-Ziv-Markov
chain-Algorithm' (LZMA) stream format, chosen to maximize safety and
interoperability. Lzip can compress about as fast as gzip @w{(lzip -0)} or
compress most files more than bzip2 @w{(lzip -9)}. Decompression speed is
intermediate between gzip and bzip2. Lzip is better than gzip and bzip2 from
a data recovery perspective. Lzip has been designed, written, and tested
with great care to replace gzip and bzip2 as the standard general-purpose
compressed format for unix-like systems.
chain-Algorithm' (LZMA) stream format and provides a 3 factor integrity
checking to maximize interoperability and optimize safety. Lzip can compress
about as fast as gzip @w{(lzip -0)} or compress most files more than bzip2
@w{(lzip -9)}. Decompression speed is intermediate between gzip and bzip2.
Lzip is better than gzip and bzip2 from a data recovery perspective. Lzip
has been designed, written, and tested with great care to replace gzip and
bzip2 as the standard general-purpose compressed format for unix-like
systems.
Plzip can compress/decompress large files on multiprocessor machines much
faster than lzip, at the cost of a slightly reduced compression ratio (0.4
@ -85,8 +86,8 @@ hundreds of processors, but on files of only a few MB plzip is no faster
than lzip. @xref{Minimum file sizes}.
For creation and manipulation of compressed tar archives
@uref{http://www.nongnu.org/lzip/manual/tarlz_manual.html,,tarlz} can be
more efficient than using tar and plzip because tarlz is able to keep the
@uref{http://www.nongnu.org/lzip/manual/tarlz_manual.html,,tarlz} can be more
efficient than using tar and plzip because tarlz is able to keep the
alignment between tar members and lzip members.
@ifnothtml
@xref{Top,tarlz manual,,tarlz}.
@ -112,8 +113,8 @@ The lzip format is as simple as possible (but not simpler). The lzip
manual provides the source code of a simple decompressor along with a
detailed explanation of how it works, so that with the only help of the
lzip manual it would be possible for a digital archaeologist to extract
the data from a lzip file long after quantum computers eventually render
LZMA obsolete.
the data from a lzip file long after quantum computers eventually
render LZMA obsolete.
@item
Additionally the lzip reference implementation is copylefted, which
@ -145,9 +146,9 @@ file from that of the compressed file as follows:
@item anyothername @tab becomes @tab anyothername.out
@end multitable
(De)compressing a file is much like copying or moving it; therefore plzip
(De)compressing a file is much like copying or moving it. Therefore plzip
preserves the access and modification dates, permissions, and, when
possible, ownership of the file just as @samp{cp -p} does. (If the user ID or
possible, ownership of the file just as @w{@samp{cp -p}} does. (If the user ID or
the group ID can't be duplicated, the file permission bits S_ISUID and
S_ISGID are cleared).
@ -258,7 +259,7 @@ garbage that can be safely ignored. @xref{concat-example}.
@anchor{--data-size}
@item -B @var{bytes}
@itemx --data-size=@var{bytes}
When compressing, set the size of the input data blocks in bytes. The
When compressing, set the size in bytes of the input data blocks. The
input file will be divided in chunks of this size before compression is
performed. Valid values range from @w{8 KiB} to @w{1 GiB}. Default value
is two times the dictionary size, except for option @samp{-0} where it
@ -276,10 +277,12 @@ overrides @samp{-o}. @samp{-c} has no effect when testing or listing.
@item -d
@itemx --decompress
Decompress the files specified. If a file does not exist or can't be
opened, plzip continues decompressing the rest of the files. If a file
fails to decompress, or is a terminal, plzip exits immediately without
decompressing the rest of the files.
Decompress the files specified. If a file does not exist, can't be opened,
or the destination file already exists and @samp{--force} has not been
specified, plzip continues decompressing the rest of the files and exits with
error status 1. If a file fails to decompress, or is a terminal, plzip exits
immediately with error status 2 without decompressing the rest of the files.
A terminal is considered an uncompressed file, and therefore invalid.
@item -f
@itemx --force
@ -304,10 +307,11 @@ size, the number of members in the file, and the amount of trailing data (if
any) are also printed. With @samp{-vv}, the positions and sizes of each
member in multimember files are also printed.
@samp{-lq} can be used to verify quickly (without decompressing) the
structural integrity of the files specified. (Use @samp{--test} to verify
the data integrity). @samp{-alq} additionally verifies that none of the
files specified contain trailing data.
If any file is damaged, does not exist, can't be opened, or is not regular,
the final exit status will be @w{> 0}. @samp{-lq} can be used to verify
quickly (without decompressing) the structural integrity of the files
specified. (Use @samp{--test} to verify the data integrity). @samp{-alq}
additionally verifies that none of the files specified contain trailing data.
@item -m @var{bytes}
@itemx --match-length=@var{bytes}
@ -448,8 +452,9 @@ used to compile plzip with the version actually being used at run time and
exit. Report any differences found. Exit with error status 1 if differences
are found. A mismatch may indicate that lzlib is not correctly installed or
that a different version of lzlib has been installed after compiling plzip.
@w{@samp{plzip -v --check-lib}} shows the version of lzlib being used and
the value of @samp{LZ_API_VERSION} (if defined).
Exit with error status 2 if LZ_API_VERSION and LZ_version_string don't
match. @w{@samp{plzip -v --check-lib}} shows the version of lzlib being used
and the value of LZ_API_VERSION (if defined).
@ifnothtml
@xref{Library version,,,lzlib}.
@end ifnothtml
@ -475,9 +480,9 @@ Table of SI and binary prefixes (unit multipliers):
@sp 1
Exit status: 0 for a normal exit, 1 for environmental problems (file not
found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or
invalid input file, 3 for an internal consistency error (eg, bug) which
caused plzip to panic.
found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or invalid
input file, 3 for an internal consistency error (e.g., bug) which caused
plzip to panic.
@node Program design
@ -486,7 +491,8 @@ caused plzip to panic.
When compressing, plzip divides the input file into chunks and compresses as
many chunks simultaneously as worker threads are chosen, creating a
multimember compressed file.
multimember compressed file. Each chunk is compressed in-place (using the
same buffer for input and output), reducing the amount of RAM required.
When decompressing, plzip decompresses as many members simultaneously as
worker threads are chosen. Files that were compressed with lzip will not
@ -505,14 +511,14 @@ splitter. The muxer collects processed packets from the workers, and
writes them to the output file.
@verbatim
,------------,
.------------.
,-->| worker 0 |--,
| `------------' |
,-------, ,----------, | ,------------, | ,-------, ,--------,
.-------. .----------. | .------------. | .-------. .--------.
| input |-->| splitter |-+-->| worker 1 |--+-->| muxer |-->| output |
| file | `----------' | `------------' | `-------' | file |
`-------' | ... | `--------'
| ,------------, |
| .------------. |
`-->| worker N-1 |--'
`------------'
@end verbatim
@ -525,92 +531,6 @@ reduced and the decompression speed of large files with many members is
only limited by the number of processors available and by I/O speed.
@node File format
@chapter File format
@cindex file format
Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away.@*
--- Antoine de Saint-Exupery
@sp 1
In the diagram below, a box like this:
@verbatim
+---+
| | <-- the vertical bars might be missing
+---+
@end verbatim
represents one byte; a box like this:
@verbatim
+==============+
| |
+==============+
@end verbatim
represents a variable number of bytes.
@sp 1
A lzip file consists of a series of "members" (compressed data sets).
The members simply appear one after another in the file, with no
additional information before, between, or after them.
Each member has the following structure:
@verbatim
+--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ID string | VN | DS | LZMA stream | CRC32 | Data size | Member size |
+--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
@end verbatim
All multibyte values are stored in little endian order.
@table @samp
@item ID string (the "magic" bytes)
A four byte string, identifying the lzip format, with the value "LZIP"
(0x4C, 0x5A, 0x49, 0x50).
@item VN (version number, 1 byte)
Just in case something needs to be modified in the future. 1 for now.
@anchor{coded-dict-size}
@item DS (coded dictionary size, 1 byte)
The dictionary size is calculated by taking a power of 2 (the base size)
and subtracting from it a fraction between 0/16 and 7/16 of the base size.@*
Bits 4-0 contain the base 2 logarithm of the base size (12 to 29).@*
Bits 7-5 contain the numerator of the fraction (0 to 7) to subtract
from the base size to obtain the dictionary size.@*
Example: 0xD3 = 2^19 - 6 * 2^15 = 512 KiB - 6 * 32 KiB = 320 KiB@*
Valid values for dictionary size range from 4 KiB to 512 MiB.
@item LZMA stream
The LZMA stream, finished by an end of stream marker. Uses default values
for encoder properties.
@ifnothtml
@xref{Stream format,,,lzip},
@end ifnothtml
@ifhtml
See
@uref{http://www.nongnu.org/lzip/manual/lzip_manual.html#Stream-format,,Stream format}
@end ifhtml
for a complete description.
@item CRC32 (4 bytes)
Cyclic Redundancy Check (CRC) of the uncompressed original data.
@item Data size (8 bytes)
Size of the uncompressed original data.
@item Member size (8 bytes)
Total size of the member, including header and trailer. This field acts
as a distributed index, allows the verification of stream integrity, and
facilitates safe recovery of undamaged members from multimember files.
@end table
@node Memory requirements
@chapter Memory required to compress and decompress
@cindex memory requirements
@ -709,6 +629,96 @@ data size for each level:
@end multitable
@node File format
@chapter File format
@cindex file format
Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away.@*
--- Antoine de Saint-Exupery
@sp 1
In the diagram below, a box like this:
@verbatim
+---+
| | <-- the vertical bars might be missing
+---+
@end verbatim
represents one byte; a box like this:
@verbatim
+==============+
| |
+==============+
@end verbatim
represents a variable number of bytes.
@sp 1
A lzip file consists of a series of independent "members" (compressed data
sets). The members simply appear one after another in the file, with no
additional information before, between, or after them. Each member can
encode in compressed form up to @w{16 EiB - 1 byte} of uncompressed data.
The size of a multimember file is unlimited.
Each member has the following structure:
@verbatim
+--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ID string | VN | DS | LZMA stream | CRC32 | Data size | Member size |
+--+--+--+--+----+----+=============+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
@end verbatim
All multibyte values are stored in little endian order.
@table @samp
@item ID string (the "magic" bytes)
A four byte string, identifying the lzip format, with the value "LZIP"
(0x4C, 0x5A, 0x49, 0x50).
@item VN (version number, 1 byte)
Just in case something needs to be modified in the future. 1 for now.
@anchor{coded-dict-size}
@item DS (coded dictionary size, 1 byte)
The dictionary size is calculated by taking a power of 2 (the base size)
and subtracting from it a fraction between 0/16 and 7/16 of the base size.@*
Bits 4-0 contain the base 2 logarithm of the base size (12 to 29).@*
Bits 7-5 contain the numerator of the fraction (0 to 7) to subtract
from the base size to obtain the dictionary size.@*
Example: 0xD3 = 2^19 - 6 * 2^15 = 512 KiB - 6 * 32 KiB = 320 KiB@*
Valid values for dictionary size range from 4 KiB to 512 MiB.
@item LZMA stream
The LZMA stream, finished by an "End Of Stream" marker. Uses default values
for encoder properties.
@ifnothtml
@xref{Stream format,,,lzip},
@end ifnothtml
@ifhtml
See
@uref{http://www.nongnu.org/lzip/manual/lzip_manual.html#Stream-format,,Stream format}
@end ifhtml
for a complete description.
@item CRC32 (4 bytes)
Cyclic Redundancy Check (CRC) of the original uncompressed data.
@item Data size (8 bytes)
Size of the original uncompressed data.
@item Member size (8 bytes)
Total size of the member, including header and trailer. This field acts
as a distributed index, allows the verification of stream integrity, and
facilitates the safe recovery of undamaged members from multimember files.
Member size should be limited to @w{2 PiB} to prevent the data size field
from overflowing.
@end table
@node Trailing data
@chapter Extra data appended to the file
@cindex trailing data
@ -795,7 +805,7 @@ plzip -v file
@sp 1
@noindent
Example 3: Like example 1 but the created @samp{file.lz} has a block size of
Example 3: Like example 2 but the created @samp{file.lz} has a block size of
@w{1 MiB}. The compression ratio is not shown.
@example
@ -820,21 +830,10 @@ show status.
plzip -tv file.lz
@end example
@sp 1
@noindent
Example 6: Compress a whole device in /dev/sdc and send the output to
@samp{file.lz}.
@example
plzip -c /dev/sdc > file.lz
or
plzip /dev/sdc -o file.lz
@end example
@sp 1
@anchor{concat-example}
@noindent
Example 7: The right way of concatenating the decompressed output of two or
Example 6: The right way of concatenating the decompressed output of two or
more compressed files. @xref{Trailing data}.
@example
@ -846,7 +845,7 @@ Do this instead
@sp 1
@noindent
Example 8: Decompress @samp{file.lz} partially until @w{10 KiB} of
Example 7: Decompress @samp{file.lz} partially until @w{10 KiB} of
decompressed data are produced.
@example
@ -855,13 +854,24 @@ plzip -cd file.lz | dd bs=1024 count=10
@sp 1
@noindent
Example 9: Decompress @samp{file.lz} partially from decompressed byte at
Example 8: Decompress @samp{file.lz} partially from decompressed byte at
offset 10000 to decompressed byte at offset 14999 (5000 bytes are produced).
@example
plzip -cd file.lz | dd bs=1000 skip=10 count=5
@end example
@sp 1
@noindent
Example 9: Compress a whole device in /dev/sdc and send the output to
@samp{file.lz}.
@example
plzip -c /dev/sdc > file.lz
or
plzip /dev/sdc -o file.lz
@end example
@node Problems
@chapter Reporting bugs
@ -875,7 +885,8 @@ for all eternity, if not longer.
If you find a bug in plzip, please send electronic mail to
@email{lzip-bug@@nongnu.org}. Include the version number, which you can
find by running @w{@samp{plzip --version}}.
find by running @w{@samp{plzip --version}} and
@w{@samp{plzip -v --check-lib}}.
@node Concept index

59
list.cc
View file

@ -1,5 +1,5 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009-2021 Antonio Diaz Diaz.
Copyright (C) 2009-2022 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -73,38 +73,35 @@ int list_files( const std::vector< std::string > & filenames,
set_retval( retval, lzip_index.retval() );
continue;
}
if( verbosity >= 0 )
if( verbosity < 0 ) continue;
const unsigned long long udata_size = lzip_index.udata_size();
const unsigned long long cdata_size = lzip_index.cdata_size();
total_comp += cdata_size; total_uncomp += udata_size; ++files;
const long members = lzip_index.members();
if( first_post )
{
const unsigned long long udata_size = lzip_index.udata_size();
const unsigned long long cdata_size = lzip_index.cdata_size();
total_comp += cdata_size; total_uncomp += udata_size; ++files;
const long members = lzip_index.members();
if( first_post )
{
first_post = false;
if( verbosity >= 1 ) std::fputs( " dict memb trail ", stdout );
std::fputs( " uncompressed compressed saved name\n", stdout );
}
if( verbosity >= 1 )
std::printf( "%s %5ld %6lld ",
format_ds( lzip_index.dictionary_size() ), members,
lzip_index.file_size() - cdata_size );
list_line( udata_size, cdata_size, input_filename );
if( verbosity >= 2 && members > 1 )
{
std::fputs( " member data_pos data_size member_pos member_size\n", stdout );
for( long i = 0; i < members; ++i )
{
const Block & db = lzip_index.dblock( i );
const Block & mb = lzip_index.mblock( i );
std::printf( "%6ld %14llu %14llu %14llu %14llu\n",
i + 1, db.pos(), db.size(), mb.pos(), mb.size() );
}
first_post = true; // reprint heading after list of members
}
std::fflush( stdout );
first_post = false;
if( verbosity >= 1 ) std::fputs( " dict memb trail ", stdout );
std::fputs( " uncompressed compressed saved name\n", stdout );
}
if( verbosity >= 1 )
std::printf( "%s %5ld %6lld ", format_ds( lzip_index.dictionary_size() ),
members, lzip_index.file_size() - cdata_size );
list_line( udata_size, cdata_size, input_filename );
if( verbosity >= 2 && members > 1 )
{
std::fputs( " member data_pos data_size member_pos member_size\n", stdout );
for( long i = 0; i < members; ++i )
{
const Block & db = lzip_index.dblock( i );
const Block & mb = lzip_index.mblock( i );
std::printf( "%6ld %14llu %14llu %14llu %14llu\n",
i + 1, db.pos(), db.size(), mb.pos(), mb.size() );
}
first_post = true; // reprint heading after list of members
}
std::fflush( stdout );
}
if( verbosity >= 0 && files > 1 )
{

2
lzip.h
View file

@ -1,5 +1,5 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009-2021 Antonio Diaz Diaz.
Copyright (C) 2009-2022 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by

2
lzip_index.cc
View file

@ -1,5 +1,5 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009-2021 Antonio Diaz Diaz.
Copyright (C) 2009-2022 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by

2
lzip_index.h
View file

@ -1,5 +1,5 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009-2021 Antonio Diaz Diaz.
Copyright (C) 2009-2022 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by

166
main.cc
View file

@ -1,6 +1,6 @@
/* Plzip - Massively parallel implementation of lzip
Copyright (C) 2009 Laszlo Ersek.
Copyright (C) 2009-2021 Antonio Diaz Diaz.
Copyright (C) 2009-2022 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -19,7 +19,7 @@
Exit status: 0 for a normal exit, 1 for environmental problems
(file not found, invalid flags, I/O errors, etc), 2 to indicate a
corrupt or invalid input file, 3 for an internal consistency error
(eg, bug) which caused plzip to panic.
(e.g., bug) which caused plzip to panic.
*/
#define _FILE_OFFSET_BITS 64
@ -39,9 +39,9 @@
#include <utime.h>
#include <sys/stat.h>
#include <lzlib.h>
#if defined(__MSVCRT__) || defined(__OS2__)
#if defined __MSVCRT__ || defined __OS2__
#include <io.h>
#if defined(__MSVCRT__)
#if defined __MSVCRT__
#define fchmod(x,y) 0
#define fchown(x,y,z) 0
#define strtoull std::strtoul
@ -67,12 +67,17 @@
#error "Environments where CHAR_BIT != 8 are not supported."
#endif
#if ( defined SIZE_MAX && SIZE_MAX < UINT_MAX ) || \
( defined SSIZE_MAX && SSIZE_MAX < INT_MAX )
#error "Environments where 'size_t' is narrower than 'int' are not supported."
#endif
int verbosity = 0;
namespace {
const char * const program_name = "plzip";
const char * const program_year = "2021";
const char * const program_year = "2022";
const char * invocation_name = program_name; // default value
const struct { const char * from; const char * to; } known_extensions[] = {
@ -101,13 +106,14 @@ void show_help( const long num_online )
"compatible with lzip 1.4 or newer. Plzip uses the compression library lzlib.\n"
"\nLzip is a lossless data compressor with a user interface similar to the one\n"
"of gzip or bzip2. Lzip uses a simplified form of the 'Lempel-Ziv-Markov\n"
"chain-Algorithm' (LZMA) stream format, chosen to maximize safety and\n"
"interoperability. Lzip can compress about as fast as gzip (lzip -0) or\n"
"compress most files more than bzip2 (lzip -9). Decompression speed is\n"
"intermediate between gzip and bzip2. Lzip is better than gzip and bzip2 from\n"
"a data recovery perspective. Lzip has been designed, written, and tested\n"
"with great care to replace gzip and bzip2 as the standard general-purpose\n"
"compressed format for unix-like systems.\n"
"chain-Algorithm' (LZMA) stream format and provides a 3 factor integrity\n"
"checking to maximize interoperability and optimize safety. Lzip can compress\n"
"about as fast as gzip (lzip -0) or compress most files more than bzip2\n"
"(lzip -9). Decompression speed is intermediate between gzip and bzip2.\n"
"Lzip is better than gzip and bzip2 from a data recovery perspective. Lzip\n"
"has been designed, written, and tested with great care to replace gzip and\n"
"bzip2 as the standard general-purpose compressed format for unix-like\n"
"systems.\n"
"\nPlzip can compress/decompress large files on multiprocessor machines much\n"
"faster than lzip, at the cost of a slightly reduced compression ratio (0.4\n"
"to 2 percent larger compressed files). Note that the number of usable\n"
@ -159,7 +165,7 @@ void show_help( const long num_online )
"'tar -xf foo.tar.lz' or 'plzip -cd foo.tar.lz | tar -xf -'.\n"
"\nExit status: 0 for a normal exit, 1 for environmental problems (file\n"
"not found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or\n"
"invalid input file, 3 for an internal consistency error (eg, bug) which\n"
"invalid input file, 3 for an internal consistency error (e.g., bug) which\n"
"caused plzip to panic.\n"
"\nReport bugs to lzip-bug@nongnu.org\n"
"Plzip home page: http://www.nongnu.org/lzip/plzip.html\n" );
@ -178,17 +184,44 @@ void show_version()
}
int check_lzlib_ver() // <major>.<minor> or <major>.<minor>[a-z.-]*
{
#if defined LZ_API_VERSION && LZ_API_VERSION >= 1012
const unsigned char * p = (unsigned char *)LZ_version_string;
unsigned major = 0, minor = 0;
while( major < 100000 && isdigit( *p ) )
{ major *= 10; major += *p - '0'; ++p; }
if( *p == '.' ) ++p;
else
out: { show_error( "Invalid LZ_version_string in lzlib.h" ); return 2; }
while( minor < 100 && isdigit( *p ) )
{ minor *= 10; minor += *p - '0'; ++p; }
if( *p && *p != '-' && *p != '.' && !std::islower( *p ) ) goto out;
const unsigned version = major * 1000 + minor;
if( LZ_API_VERSION != version )
{
if( verbosity >= 0 )
std::fprintf( stderr, "%s: Version mismatch in lzlib.h: "
"LZ_API_VERSION = %u, should be %u.\n",
program_name, LZ_API_VERSION, version );
return 2;
}
#endif
return 0;
}
int check_lib()
{
bool warning = false;
int retval = check_lzlib_ver();
if( std::strcmp( LZ_version_string, LZ_version() ) != 0 )
{ warning = true;
{ set_retval( retval, 1 );
if( verbosity >= 0 )
std::printf( "warning: LZ_version_string != LZ_version() (%s vs %s)\n",
LZ_version_string, LZ_version() ); }
#if defined LZ_API_VERSION && LZ_API_VERSION >= 1012
if( LZ_API_VERSION != LZ_api_version() )
{ warning = true;
{ set_retval( retval, 1 );
if( verbosity >= 0 )
std::printf( "warning: LZ_API_VERSION != LZ_api_version() (%u vs %u)\n",
LZ_API_VERSION, LZ_api_version() ); }
@ -205,23 +238,21 @@ int check_lib()
"Using an unknown LZ_API_VERSION\n", LZ_API_VERSION );
#endif
}
return warning;
return retval;
}
} // end namespace
void Pretty_print::operator()( const char * const msg ) const
{
if( verbosity >= 0 )
if( verbosity < 0 ) return;
if( first_post )
{
if( first_post )
{
first_post = false;
std::fputs( padded_name.c_str(), stderr );
if( !msg ) std::fflush( stderr );
}
if( msg ) std::fprintf( stderr, "%s\n", msg );
first_post = false;
std::fputs( padded_name.c_str(), stderr );
if( !msg ) std::fflush( stderr );
}
if( msg ) std::fprintf( stderr, "%s\n", msg );
}
@ -260,16 +291,53 @@ void show_header( const unsigned dictionary_size )
namespace {
unsigned long long getnum( const char * const ptr,
// separate large numbers >= 100_000 in groups of 3 digits using '_'
const char * format_num3( unsigned long long num )
{
const char * const si_prefix = "kMGTPEZY";
const char * const binary_prefix = "KMGTPEZY";
enum { buffers = 8, bufsize = 4 * sizeof (long long) };
static char buffer[buffers][bufsize]; // circle of static buffers for printf
static int current = 0;
char * const buf = buffer[current++]; current %= buffers;
char * p = buf + bufsize - 1; // fill the buffer backwards
*p = 0; // terminator
if( num > 1024 )
{
char prefix = 0; // try binary first, then si
for( int i = 0; i < 8 && num >= 1024 && num % 1024 == 0; ++i )
{ num /= 1024; prefix = binary_prefix[i]; }
if( prefix ) *(--p) = 'i';
else
for( int i = 0; i < 8 && num >= 1000 && num % 1000 == 0; ++i )
{ num /= 1000; prefix = si_prefix[i]; }
if( prefix ) *(--p) = prefix;
}
const bool split = num >= 100000;
for( int i = 0; ; )
{
*(--p) = num % 10 + '0'; num /= 10; if( num == 0 ) break;
if( split && ++i >= 3 ) { i = 0; *(--p) = '_'; }
}
return p;
}
unsigned long long getnum( const char * const arg,
const char * const option_name,
const unsigned long long llimit,
const unsigned long long ulimit )
{
char * tail;
errno = 0;
unsigned long long result = strtoull( ptr, &tail, 0 );
if( tail == ptr )
unsigned long long result = strtoull( arg, &tail, 0 );
if( tail == arg )
{
show_error( "Bad or missing numerical argument.", 0, true );
if( verbosity >= 0 )
std::fprintf( stderr, "%s: Bad or missing numerical argument in "
"option '%s'.\n", program_name, option_name );
std::exit( 1 );
}
@ -291,7 +359,9 @@ unsigned long long getnum( const char * const ptr,
}
if( exponent <= 0 )
{
show_error( "Bad multiplier in numerical argument.", 0, true );
if( verbosity >= 0 )
std::fprintf( stderr, "%s: Bad multiplier in numerical argument of "
"option '%s'.\n", program_name, option_name );
std::exit( 1 );
}
for( int i = 0; i < exponent; ++i )
@ -303,22 +373,25 @@ unsigned long long getnum( const char * const ptr,
if( !errno && ( result < llimit || result > ulimit ) ) errno = ERANGE;
if( errno )
{
show_error( "Numerical argument out of limits." );
if( verbosity >= 0 )
std::fprintf( stderr, "%s: Numerical argument out of limits [%s,%s] "
"in option '%s'.\n", program_name, format_num3( llimit ),
format_num3( ulimit ), option_name );
std::exit( 1 );
}
return result;
}
int get_dict_size( const char * const arg )
int get_dict_size( const char * const arg, const char * const option_name )
{
char * tail;
const long bits = std::strtol( arg, &tail, 0 );
if( bits >= LZ_min_dictionary_bits() &&
bits <= LZ_max_dictionary_bits() && *tail == 0 )
return 1 << bits;
int dictionary_size = getnum( arg, LZ_min_dictionary_size(),
LZ_max_dictionary_size() );
int dictionary_size = getnum( arg, option_name, LZ_min_dictionary_size(),
LZ_max_dictionary_size() );
if( dictionary_size == 65535 ) ++dictionary_size; // no fast encoder
return dictionary_size;
}
@ -499,7 +572,7 @@ bool check_tty_in( const char * const input_filename, const int infd,
isatty( infd ) ) // for example /dev/tty
{ show_file_error( input_filename,
"I won't read compressed data from a terminal." );
close( infd ); set_retval( retval, 1 );
close( infd ); set_retval( retval, 2 );
if( program_mode != m_test ) cleanup_and_fail( retval );
return false; }
return true;
@ -613,7 +686,7 @@ void show_progress( const unsigned long long packet_size,
}
#if defined(__MSVCRT__)
#if defined __MSVCRT__
#include <windows.h>
#define _SC_NPROCESSORS_ONLN 1
#define _SC_THREAD_THREADS_MAX 2
@ -651,7 +724,6 @@ int main( const int argc, const char * const argv[] )
{ 1 << 25, 273 } }; // -9
Lzma_options encoder_options = option_mapping[6]; // default = "-6"
std::string default_output_filename;
std::vector< std::string > filenames;
int data_size = 0;
int debug_level = 0;
int num_workers = 0; // start this many worker threads
@ -719,6 +791,7 @@ int main( const int argc, const char * const argv[] )
{
const int code = parser.code( argind );
if( !code ) break; // no more options
const char * const pn = parser.parsed_name( argind ).c_str();
const std::string & sarg = parser.argument( argind );
const char * const arg = sarg.c_str();
switch( code )
@ -728,7 +801,7 @@ int main( const int argc, const char * const argv[] )
encoder_options = option_mapping[code-'0']; break;
case 'a': ignore_trailing = false; break;
case 'b': break;
case 'B': data_size = getnum( arg, 2 * LZ_min_dictionary_size(),
case 'B': data_size = getnum( arg, pn, 2 * LZ_min_dictionary_size(),
2 * LZ_max_dictionary_size() ); break;
case 'c': to_stdout = true; break;
case 'd': set_mode( program_mode, m_decompress ); break;
@ -738,23 +811,23 @@ int main( const int argc, const char * const argv[] )
case 'k': keep_input_files = true; break;
case 'l': set_mode( program_mode, m_list ); break;
case 'm': encoder_options.match_len_limit =
getnum( arg, LZ_min_match_len_limit(),
LZ_max_match_len_limit() ); break;
case 'n': num_workers = getnum( arg, 1, max_workers ); break;
getnum( arg, pn, LZ_min_match_len_limit(),
LZ_max_match_len_limit() ); break;
case 'n': num_workers = getnum( arg, pn, 1, max_workers ); break;
case 'o': if( sarg == "-" ) to_stdout = true;
else { default_output_filename = sarg; } break;
case 'q': verbosity = -1; break;
case 's': encoder_options.dictionary_size = get_dict_size( arg );
case 's': encoder_options.dictionary_size = get_dict_size( arg, pn );
break;
case 'S': break;
case 't': set_mode( program_mode, m_test ); break;
case 'v': if( verbosity < 4 ) ++verbosity; break;
case 'V': show_version(); return 0;
case opt_chk: return check_lib();
case opt_dbg: debug_level = getnum( arg, 0, 3 ); break;
case opt_in: in_slots = getnum( arg, 1, 64 ); break;
case opt_dbg: debug_level = getnum( arg, pn, 0, 3 ); break;
case opt_in: in_slots = getnum( arg, pn, 1, 64 ); break;
case opt_lt: loose_trailing = true; break;
case opt_out: out_slots = getnum( arg, 1, 1024 ); break;
case opt_out: out_slots = getnum( arg, pn, 1, 1024 ); break;
default : internal_error( "uncaught option." );
}
} // end process options
@ -763,11 +836,12 @@ int main( const int argc, const char * const argv[] )
{ show_error( "Wrong library version. At least lzlib 1.0 is required." );
return 1; }
#if defined(__MSVCRT__) || defined(__OS2__)
#if defined __MSVCRT__ || defined __OS2__
setmode( STDIN_FILENO, O_BINARY );
setmode( STDOUT_FILENO, O_BINARY );
#endif
std::vector< std::string > filenames;
bool filenames_given = false;
for( ; argind < parser.arguments(); ++argind )
{

28
testsuite/check.sh
View file

@ -1,6 +1,6 @@
#! /bin/sh
# check script for Plzip - Massively parallel implementation of lzip
# Copyright (C) 2009-2021 Antonio Diaz Diaz.
# Copyright (C) 2009-2022 Antonio Diaz Diaz.
#
# This script is free software: you have unlimited permission
# to copy, distribute, and modify it.
@ -31,6 +31,7 @@ cd "${objdir}"/tmp || framework_failure
cat "${testdir}"/test.txt > in || framework_failure
in_lz="${testdir}"/test.txt.lz
in_em="${testdir}"/test_em.txt.lz
fox_lz="${testdir}"/fox.lz
fail=0
lwarn8=0
lwarn10=0
@ -43,6 +44,7 @@ lzlib_1_10() { [ ${lwarn10} = 0 ] &&
lwarn10=1 ; }
"${LZIP}" --check-lib # just print warning
[ $? != 2 ] || test_failed $LINENO # unless bad lzlib.h
printf "testing plzip-%s..." "$2"
"${LZIP}" -fkqm4 in
@ -106,6 +108,7 @@ done
printf "LZIP\001-.............................." | "${LZIP}" -t 2> /dev/null
printf "LZIP\002-.............................." | "${LZIP}" -t 2> /dev/null
printf "LZIP\001+.............................." | "${LZIP}" -t 2> /dev/null
rm -f out || framework_failure
printf "\ntesting decompression..."
@ -129,15 +132,21 @@ lines=$("${LZIP}" -tvv "${in_em}" 2>&1 | wc -l) || test_failed $LINENO
lines=$("${LZIP}" -lvv "${in_em}" | wc -l) || test_failed $LINENO
[ "${lines}" -eq 11 ] || test_failed $LINENO "${lines}"
"${LZIP}" -cd "${fox_lz}" > fox || test_failed $LINENO
cat "${in_lz}" > copy.lz || framework_failure
"${LZIP}" -dk copy.lz || test_failed $LINENO
cmp in copy || test_failed $LINENO
printf "to be overwritten" > copy || framework_failure
"${LZIP}" -d copy.lz 2> /dev/null
cat fox > copy || framework_failure
cat "${in_lz}" > out.lz || framework_failure
rm -f out || framework_failure
"${LZIP}" -d copy.lz out.lz 2> /dev/null # skip copy, decompress out
[ $? = 1 ] || test_failed $LINENO
cmp fox copy || test_failed $LINENO
cmp in out || test_failed $LINENO
"${LZIP}" -df copy.lz || test_failed $LINENO
[ ! -e copy.lz ] || test_failed $LINENO
cmp in copy || test_failed $LINENO
rm -f fox out || framework_failure
printf "to be overwritten" > copy || framework_failure
"${LZIP}" -df -o copy < "${in_lz}" || test_failed $LINENO
@ -167,7 +176,7 @@ rm -f copy anyothername.out || framework_failure
[ $? = 1 ] || test_failed $LINENO
"${LZIP}" -cdq in "${in_lz}" > copy
[ $? = 2 ] || test_failed $LINENO
cat copy in | cmp in - || test_failed $LINENO
cat copy in | cmp in - || test_failed $LINENO # copy must be empty
"${LZIP}" -cdq nx_file.lz "${in_lz}" > copy
[ $? = 1 ] || test_failed $LINENO
cmp in copy || test_failed $LINENO
@ -220,10 +229,17 @@ printf "\ngarbage" >> copy2.lz || framework_failure
printf "to be overwritten" > copy2 || framework_failure
"${LZIP}" -df copy2.lz || test_failed $LINENO
cmp in2 copy2 || test_failed $LINENO
rm -f in2 copy2 || framework_failure
rm -f copy2 || framework_failure
printf "\ntesting compression..."
"${LZIP}" -c -0 in in in -o out3.lz > copy2.lz || test_failed $LINENO
[ ! -e out3.lz ] || test_failed $LINENO # override -o
"${LZIP}" -0f in in --output=copy2.lz || test_failed $LINENO
"${LZIP}" -d copy2.lz -o out2 || test_failed $LINENO
cmp in2 out2 || test_failed $LINENO
rm -f in2 out2 copy2.lz || framework_failure
"${LZIP}" -cf "${in_lz}" > out 2> /dev/null # /dev/null is a tty on OS/2
[ $? = 1 ] || test_failed $LINENO
"${LZIP}" -Fvvm36 -o - "${in_lz}" > out 2> /dev/null || test_failed $LINENO
@ -233,7 +249,7 @@ cmp in copy || test_failed $LINENO
"${LZIP}" -0 -o ./- in || test_failed $LINENO
"${LZIP}" -cd ./- | cmp in - || test_failed $LINENO
rm -f ./- || framework_failure
"${LZIP}" -0 -o ./- < in || test_failed $LINENO # add .lz
"${LZIP}" -0 -o ./- < in || test_failed $LINENO # add .lz
[ ! -e ./- ] || test_failed $LINENO
"${LZIP}" -cd -- -.lz | cmp in - || test_failed $LINENO
rm -f ./-.lz || framework_failure

BIN
testsuite/fox.lz Normal file
View file

Binary file not shown.