lzip/clzip
1
0
Fork 0
Code Issues Activity

Merging upstream version 1.15~rc1.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-17 20:54:18 +01:00
parent f99c7a8d1c
commit fe6e695444
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
31 changed files with 1415 additions and 1373 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
COPYING
View file

@ -1,8 +1,7 @@
GNU GENERAL PUBLIC LICENSE GNU GENERAL PUBLIC LICENSE
Version 2, June 1991 Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc., Copyright (C) 1989, 1991 Free Software Foundation, Inc. <http://fsf.org/>
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed. of this license document, but changing it is not allowed.

46
ChangeLog
View file

@ -1,3 +1,13 @@
2024-11-23 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.15-rc1 released.
* Remove options '--empty-error' and '--marking-error'.
* main.c (Pp_free): New function.
* decoder.c (LZd_decode_member): Remove support for Sync Flush marker.
* clzip.texi: New chapter 'Syntax of command-line arguments'.
* check.sh: Use 'cp' instead of 'cat'.
* testsuite: Add fox_nz.lz. Remove fox6.lz,fox6_mark.lz,test_em.txt.lz.
2024-01-22 Antonio Diaz Diaz <antonio@gnu.org> 2024-01-22 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.14 released. * Version 1.14 released.
@ -27,12 +37,12 @@
Make '-o' behave like '-c', but writing to file instead of stdout. Make '-o' behave like '-c', but writing to file instead of stdout.
Make '-c' and '-o' check whether the output is a terminal only once. Make '-c' and '-o' check whether the output is a terminal only once.
Do not open output if input is a terminal. Do not open output if input is a terminal.
Set a valid invocation_name even if argc == 0.
* Replace 'decompressed', 'compressed' with 'out', 'in' in output. * Replace 'decompressed', 'compressed' with 'out', 'in' in output.
* lzip_index.c: Improve messages for corruption in last header. * lzip_index.c: Improve messages for corruption in last header.
* main.c: Set a valid invocation_name even if argc == 0.
* Document extraction from tar.lz in manual, '--help', and man page. * Document extraction from tar.lz in manual, '--help', and man page.
* clzip.texi (Introduction): Mention plzip and tarlz as alternatives. * clzip.texi (Introduction): Mention plzip and tarlz as alternatives.
* clzip.texi: Several fixes and improvements. Several fixes and improvements.
* testsuite: Add 9 new test files. * testsuite: Add 9 new test files.
2019-01-03 Antonio Diaz Diaz <antonio@gnu.org> 2019-01-03 Antonio Diaz Diaz <antonio@gnu.org>
@ -42,7 +52,7 @@
* lzip.h (Lzip_trailer): New function 'Lt_verify_consistency'. * lzip.h (Lzip_trailer): New function 'Lt_verify_consistency'.
* lzip_index.c: Detect some kinds of corrupt trailers. * lzip_index.c: Detect some kinds of corrupt trailers.
* main.c (main): Check return value of close( infd ). * main.c (main): Check return value of close( infd ).
* main.c: Compile on DOS with DJGPP. Compile on DOS with DJGPP. (Reported by Robert Riebisch).
* clzip.texi: Improve descriptions of '-0..-9', '-m', and '-s'. * clzip.texi: Improve descriptions of '-0..-9', '-m', and '-s'.
* configure: Accept appending to CFLAGS; 'CFLAGS+=OPTIONS'. * configure: Accept appending to CFLAGS; 'CFLAGS+=OPTIONS'.
* INSTALL: Document use of CFLAGS+='-D __USE_MINGW_ANSI_STDIO'. * INSTALL: Document use of CFLAGS+='-D __USE_MINGW_ANSI_STDIO'.
@ -53,13 +63,13 @@
* New option '--loose-trailing'. * New option '--loose-trailing'.
* Improve corrupt header detection to HD=3. * Improve corrupt header detection to HD=3.
* main.c: Show corrupt or truncated header in multimember file. * main.c: Show corrupt or truncated header in multimember file.
* main.c (main): Option '-S, --volume-size' now keeps input files. (main): Make option '-S, --volume-size' keep input files.
(main): Show final diagnostic when testing multiple files.
(set_c_outname): Do not add a second '.lz' to the arg of '-o'.
* encoder_base.*: Adjust dictionary size for each member. * encoder_base.*: Adjust dictionary size for each member.
* Replace 'bits/byte' with inverse compression ratio in output. * Replace 'bits/byte' with inverse compression ratio in output.
* Show progress of decompression at verbosity level 2 (-vv). * Show progress of decompression at verbosity level 2 (-vv).
* Show progress of (de)compression only if stderr is a terminal. * Show progress of (de)compression only if stderr is a terminal.
* main.c: Show final diagnostic when testing multiple files.
* main.c: Do not add a second .lz extension to the arg of -o.
* decoder.c (LZd_verify_trailer): Show stored sizes also in hex. * decoder.c (LZd_verify_trailer): Show stored sizes also in hex.
Show dictionary size at verbosity level 4 (-vvvv). Show dictionary size at verbosity level 4 (-vvvv).
* clzip.texi: New chapter 'Meaning of clzip's output'. * clzip.texi: New chapter 'Meaning of clzip's output'.
@ -73,7 +83,7 @@
* Compression time of options -1 to -9 has been reduced by 1%. * Compression time of options -1 to -9 has been reduced by 1%.
* Decompression time has been reduced by 7%. * Decompression time has been reduced by 7%.
* main.c: Continue testing if any input file is a terminal. * main.c: Continue testing if any input file is a terminal.
* main.c: Show trailing data in both hexadecimal and ASCII. Show trailing data in both hexadecimal and ASCII.
* lzip_index.c: Improve detection of bad dict and trailing data. * lzip_index.c: Improve detection of bad dict and trailing data.
* lzip.h: Unify messages for bad magic, trailing data, etc. * lzip.h: Unify messages for bad magic, trailing data, etc.
* clzip.texi: Add missing chapters from lzip.texi. * clzip.texi: Add missing chapters from lzip.texi.
@ -82,16 +92,14 @@
* Version 1.8 released. * Version 1.8 released.
* New option '-a, --trailing-error'. * New option '-a, --trailing-error'.
* main.c (decompress): Print up to 6 bytes of trailing data when
'-vvvv' is specified.
* decoder.c (LZd_verify_trailer): Remove test of final code.
* main.c (main): Delete '--output' file if infd is a terminal. * main.c (main): Delete '--output' file if infd is a terminal.
* main.c (main): Don't use stdin more than once. (main): Don't use stdin more than once.
(decompress): Print 6 bytes of trailing data at verbosity level 4.
* decoder.c (LZd_verify_trailer): Remove test of final code.
* clzip.texi: New chapter 'Trailing data'. * clzip.texi: New chapter 'Trailing data'.
* configure: Avoid warning on some shells when testing for gcc. * configure: Avoid warning on some shells when testing for gcc.
* Makefile.in: Detect the existence of install-info. * Makefile.in: Detect the existence of install-info.
* check.sh: A POSIX shell is required to run the tests. * check.sh: Require a POSIX shell. Don't check error messages.
* check.sh: Don't check error messages.
2015-07-07 Antonio Diaz Diaz <antonio@gnu.org> 2015-07-07 Antonio Diaz Diaz <antonio@gnu.org>
@ -123,8 +131,8 @@
* Compression time has been reduced by 10%. * Compression time has been reduced by 10%.
* Decompression time has been reduced by 8%. * Decompression time has been reduced by 8%.
* Makefile.in: New targets 'install-as-lzip' and 'install-bin'. * Makefile.in: New targets 'install-as-lzip' and 'install-bin'.
* main.c: Use 'setmode' instead of '_setmode' on Windows and OS/2.
* main.c: Define 'strtoull' to 'strtoul' on Windows. * main.c: Define 'strtoull' to 'strtoul' on Windows.
(main): Use 'setmode' instead of '_setmode' on Windows and OS/2.
2012-02-25 Antonio Diaz Diaz <ant_diaz@teleline.es> 2012-02-25 Antonio Diaz Diaz <ant_diaz@teleline.es>
@ -141,29 +149,29 @@
* Version 1.2 released. * Version 1.2 released.
* New option '-F, --recompress'. * New option '-F, --recompress'.
* main.c (decompress): Print only one status line for each
multimember file when only one '-v' is specified.
* encoder.h (Lee_update_prices): Update high length symbol prices * encoder.h (Lee_update_prices): Update high length symbol prices
independently of the value of 'pos_state'. This gives better independently of the value of 'pos_state'. This gives better
compression for large values of '--match-length' without being compression for large values of '--match-length' without being
slower. slower.
* encoder.h, encoder.c: Optimize pair price calculations, reducing * encoder.h, encoder.c: Optimize pair price calculations, reducing
compression time for large values of '--match-length' by up to 6%. compression time for large values of '--match-length' by up to 6%.
* main.c (decompress): Print only one status line for each
multimember file when only one '-v' is specified.
2011-01-11 Antonio Diaz Diaz <ant_diaz@teleline.es> 2011-01-11 Antonio Diaz Diaz <ant_diaz@teleline.es>
* Version 1.1 released. * Version 1.1 released.
* Code has been converted to 'C89 + long long' from C99. * Code has been converted to 'C89 + long long' from C99.
* main.c: Fix warning about fchown return value being ignored.
* decoder.c: '-tvvvv' now shows compression ratio. * decoder.c: '-tvvvv' now shows compression ratio.
* main.c: Match length limit set by options -1 to -8 has been * main.c: Match length limit set by options -1 to -8 has been
reduced to extend range of use towards gzip. Lower numbers now reduced to extend range of use towards gzip. Lower numbers now
compress less but faster. (-1 now takes 43% less time for only 20% compress less but faster. (-1 now takes 43% less time for only 20%
larger compressed size). larger compressed size).
Exit with status 1 if any output file exists and is skipped. Exit with status 1 if any output file exists and is skipped.
* Compression ratio of option '-9' has been slightly increased. Fix warning about fchown's return value being ignored.
* main.c (open_instream): Don't show the message (open_instream): Don't show the message
" and '--stdout' was not specified" for directories, etc. " and '--stdout' was not specified" for directories, etc.
* Compression ratio of option '-9' has been slightly increased.
* New examples have been added to the manual. * New examples have been added to the manual.
2010-04-05 Antonio Diaz Diaz <ant_diaz@teleline.es> 2010-04-05 Antonio Diaz Diaz <ant_diaz@teleline.es>

10
Makefile.in
View file

@ -2,8 +2,8 @@
DISTNAME = $(pkgname)-$(pkgversion) DISTNAME = $(pkgname)-$(pkgversion)
INSTALL = install INSTALL = install
INSTALL_PROGRAM = $(INSTALL) -m 755 INSTALL_PROGRAM = $(INSTALL) -m 755
INSTALL_DATA = $(INSTALL) -m 644
INSTALL_DIR = $(INSTALL) -d -m 755 INSTALL_DIR = $(INSTALL) -d -m 755
INSTALL_DATA = $(INSTALL) -m 644
SHELL = /bin/sh SHELL = /bin/sh
CAN_RUN_INSTALLINFO = $(SHELL) -c "install-info --version" > /dev/null 2>&1 CAN_RUN_INSTALLINFO = $(SHELL) -c "install-info --version" > /dev/null 2>&1
@ -31,7 +31,8 @@ main.o : main.c
# prevent 'make' from trying to remake source files # prevent 'make' from trying to remake source files
$(VPATH)/configure $(VPATH)/Makefile.in $(VPATH)/doc/$(pkgname).texi : ; $(VPATH)/configure $(VPATH)/Makefile.in $(VPATH)/doc/$(pkgname).texi : ;
%.h %.c : ; MAKEFLAGS += -r
.SUFFIXES :
$(objs) : Makefile $(objs) : Makefile
carg_parser.o : carg_parser.h carg_parser.o : carg_parser.h
@ -130,10 +131,7 @@ dist : doc
$(DISTNAME)/testsuite/test.txt \ $(DISTNAME)/testsuite/test.txt \
$(DISTNAME)/testsuite/fox.lz \ $(DISTNAME)/testsuite/fox.lz \
$(DISTNAME)/testsuite/fox_*.lz \ $(DISTNAME)/testsuite/fox_*.lz \
$(DISTNAME)/testsuite/fox6.lz \ $(DISTNAME)/testsuite/test.txt.lz
$(DISTNAME)/testsuite/fox6_mark.lz \
$(DISTNAME)/testsuite/test.txt.lz \
$(DISTNAME)/testsuite/test_em.txt.lz
rm -f $(DISTNAME) rm -f $(DISTNAME)
lzip -v -9 $(DISTNAME).tar lzip -v -9 $(DISTNAME).tar

27
NEWS
View file

@ -1,24 +1,11 @@
Changes in version 1.14: Changes in version 1.15:
The option '--empty-error', which forces exit status 2 if any empty member clzip now exits with error status 2 if any empty member is found in a
is found, has been added. multimember file.
The option '--marking-error', which forces exit status 2 if the first LZMA clzip now exits with error status 2 if the first byte of the LZMA stream is
byte is non-zero in any member, has been added. not 0.
File diagnostics have been reformatted as 'PROGRAM: FILE: MESSAGE'. Options '--empty-error' and '--marking-error' have been removed.
Diagnostics caused by invalid arguments to command-line options now show the The chapter 'Syntax of command-line arguments' has been added to the manual.
argument and the name of the option.
The option '-o, --output' now preserves dates, permissions, and ownership of
the file when (de)compressing exactly one file.
The option '-o, --output' now creates missing intermediate directories when
writing to a file.
The variable MAKEINFO has been added to configure and Makefile.in.
It has been documented in INSTALL that when choosing a C standard, the POSIX
features need to be enabled explicitly:
./configure CFLAGS+='--std=c99 -D_XOPEN_SOURCE=500'

41
README
View file

@ -1,20 +1,21 @@
See the file INSTALL for compilation and installation instructions.
Description Description
Clzip is a C language version of lzip, compatible with lzip 1.4 or newer. As Clzip is a C language version of lzip intended for systems lacking a C++
clzip is written in C, it may be easier to integrate in applications like compiler.
package managers, embedded devices, or systems lacking a C++ compiler.
Lzip is a lossless data compressor with a user interface similar to the one Lzip is a lossless data compressor with a user interface similar to the one
of gzip or bzip2. Lzip uses a simplified form of the 'Lempel-Ziv-Markov of gzip or bzip2. Lzip uses a simplified form of LZMA (Lempel-Ziv-Markov
chain-Algorithm' (LZMA) stream format to maximize interoperability. The chain-Algorithm) designed to achieve complete interoperability between
maximum dictionary size is 512 MiB so that any lzip file can be decompressed implementations. The maximum dictionary size is 512 MiB so that any lzip
on 32-bit machines. Lzip provides accurate and robust 3-factor integrity file can be decompressed on 32-bit machines. Lzip provides accurate and
checking. Lzip can compress about as fast as gzip (lzip -0) or compress most robust 3-factor integrity checking. 'lzip -0' compresses about as fast as
files more than bzip2 (lzip -9). Decompression speed is intermediate between gzip, while 'lzip -9' compresses most files more than bzip2. Decompression
gzip and bzip2. Lzip is better than gzip and bzip2 from a data recovery speed is intermediate between gzip and bzip2. Lzip provides better data
perspective. Lzip has been designed, written, and tested with great care to recovery capabilities than gzip and bzip2. Lzip has been designed, written,
replace gzip and bzip2 as the standard general-purpose compressed format for and tested with great care to replace gzip and bzip2 as general-purpose
Unix-like systems. compressed format for Unix-like systems.
For compressing/decompressing large files on multiprocessor machines plzip For compressing/decompressing large files on multiprocessor machines plzip
can be much faster than lzip at the cost of a slightly reduced compression can be much faster than lzip at the cost of a slightly reduced compression
@ -95,7 +96,6 @@ also supported.
Clzip can produce multimember files, and lziprecover can safely recover the Clzip can produce multimember files, and lziprecover can safely recover the
undamaged members in case of file damage. Clzip can also split the compressed undamaged members in case of file damage. Clzip can also split the compressed
output in volumes of a given size, even when reading from standard input. output in volumes of a given size, even when reading from standard input.
This allows the direct creation of multivolume compressed tar archives.
Clzip is able to compress and decompress streams of unlimited size by Clzip is able to compress and decompress streams of unlimited size by
automatically creating multimember output. The members so created are large, automatically creating multimember output. The members so created are large,
@ -105,16 +105,16 @@ In spite of its name (Lempel-Ziv-Markov chain-Algorithm), LZMA is not a
concrete algorithm; it is more like "any algorithm using the LZMA coding concrete algorithm; it is more like "any algorithm using the LZMA coding
scheme". For example, the option '-0' of lzip uses the scheme in almost the scheme". For example, the option '-0' of lzip uses the scheme in almost the
simplest way possible; issuing the longest match it can find, or a literal simplest way possible; issuing the longest match it can find, or a literal
byte if it can't find a match. Inversely, a much more elaborated way of byte if it can't find a match. Inversely, a more elaborate way of finding
finding coding sequences of minimum size than the one currently used by lzip coding sequences of minimum size than the one currently used by lzip could
could be developed, and the resulting sequence could also be coded using the be developed, and the resulting sequence could also be coded using the LZMA
LZMA coding scheme. coding scheme.
Clzip currently implements two variants of the LZMA algorithm: fast Clzip currently implements two variants of the LZMA algorithm: fast
(used by option '-0') and normal (used by all other compression levels). (used by option '-0') and normal (used by all other compression levels).
The high compression of LZMA comes from combining two basic, well-proven The high compression of LZMA comes from combining two basic, well-proven
compression ideas: sliding dictionaries (LZ77) and markov models (the thing compression ideas: sliding dictionaries (LZ77) and Markov models (the thing
used by every compression algorithm that uses a range encoder or similar used by every compression algorithm that uses a range encoder or similar
order-0 entropy coder as its last stage) with segregation of contexts order-0 entropy coder as its last stage) with segregation of contexts
according to what the bits are used for. according to what the bits are used for.
@ -125,6 +125,9 @@ definition of Markov chains), G.N.N. Martin (for the definition of range
encoding), Igor Pavlov (for putting all the above together in LZMA), and encoding), Igor Pavlov (for putting all the above together in LZMA), and
Julian Seward (for bzip2's CLI). Julian Seward (for bzip2's CLI).
Clzip uses Arg_parser for command-line argument parsing:
http://www.nongnu.org/arg-parser/arg_parser.html
LANGUAGE NOTE: Uncompressed = not compressed = plain data; it may never have LANGUAGE NOTE: Uncompressed = not compressed = plain data; it may never have
been compressed. Decompressed is used to refer to data which have undergone been compressed. Decompressed is used to refer to data which have undergone
the process of decompression. the process of decompression.

60
carg_parser.c
View file

@ -32,15 +32,15 @@ static void * ap_resize_buffer( void * buf, const int min_size )
} }
static char push_back_record( struct Arg_parser * const ap, const int code, static char push_back_record( Arg_parser * const ap, const int code,
const char * const long_name, const char * const long_name,
const char * const argument ) const char * const argument )
{ {
struct ap_Record * p; ap_Record * p;
void * tmp = ap_resize_buffer( ap->data, void * tmp = ap_resize_buffer( ap->data,
( ap->data_size + 1 ) * sizeof (struct ap_Record) ); ( ap->data_size + 1 ) * sizeof (ap_Record) );
if( !tmp ) return 0; if( !tmp ) return 0;
ap->data = (struct ap_Record *)tmp; ap->data = (ap_Record *)tmp;
p = &(ap->data[ap->data_size]); p = &(ap->data[ap->data_size]);
p->code = code; p->code = code;
if( long_name ) if( long_name )
@ -71,7 +71,7 @@ static char push_back_record( struct Arg_parser * const ap, const int code,
} }
static char add_error( struct Arg_parser * const ap, const char * const msg ) static char add_error( Arg_parser * const ap, const char * const msg )
{ {
const int len = strlen( msg ); const int len = strlen( msg );
void * tmp = ap_resize_buffer( ap->error, ap->error_size + len + 1 ); void * tmp = ap_resize_buffer( ap->error, ap->error_size + len + 1 );
@ -83,7 +83,7 @@ static char add_error( struct Arg_parser * const ap, const char * const msg )
} }
static void free_data( struct Arg_parser * const ap ) static void free_data( Arg_parser * const ap )
{ {
int i; int i;
for( i = 0; i < ap->data_size; ++i ) for( i = 0; i < ap->data_size; ++i )
@ -94,10 +94,9 @@ static void free_data( struct Arg_parser * const ap )
/* Return 0 only if out of memory. */ /* Return 0 only if out of memory. */
static char parse_long_option( struct Arg_parser * const ap, static char parse_long_option( Arg_parser * const ap,
const char * const opt, const char * const arg, const char * const opt, const char * const arg,
const struct ap_Option options[], const ap_Option options[], int * const argindp )
int * const argindp )
{ {
unsigned len; unsigned len;
int index = -1, i; int index = -1, i;
@ -148,21 +147,21 @@ static char parse_long_option( struct Arg_parser * const ap,
add_error( ap, "' requires an argument" ); add_error( ap, "' requires an argument" );
return 1; return 1;
} }
return push_back_record( ap, options[index].code, return push_back_record( ap, options[index].code, options[index].long_name,
options[index].long_name, &opt[len+3] ); &opt[len+3] ); /* argument may be empty */
} }
if( options[index].has_arg == ap_yes ) if( options[index].has_arg == ap_yes || options[index].has_arg == ap_yme )
{ {
if( !arg || !arg[0] ) if( !arg || ( options[index].has_arg == ap_yes && !arg[0] ) )
{ {
add_error( ap, "option '--" ); add_error( ap, options[index].long_name ); add_error( ap, "option '--" ); add_error( ap, options[index].long_name );
add_error( ap, "' requires an argument" ); add_error( ap, "' requires an argument" );
return 1; return 1;
} }
++*argindp; ++*argindp;
return push_back_record( ap, options[index].code, return push_back_record( ap, options[index].code, options[index].long_name,
options[index].long_name, arg ); arg ); /* argument may be empty */
} }
return push_back_record( ap, options[index].code, return push_back_record( ap, options[index].code,
@ -171,10 +170,9 @@ static char parse_long_option( struct Arg_parser * const ap,
/* Return 0 only if out of memory. */ /* Return 0 only if out of memory. */
static char parse_short_option( struct Arg_parser * const ap, static char parse_short_option( Arg_parser * const ap,
const char * const opt, const char * const arg, const char * const opt, const char * const arg,
const struct ap_Option options[], const ap_Option options[], int * const argindp )
int * const argindp )
{ {
int cind = 1; /* character index in opt */ int cind = 1; /* character index in opt */
@ -204,15 +202,15 @@ static char parse_short_option( struct Arg_parser * const ap,
if( !push_back_record( ap, c, 0, &opt[cind] ) ) return 0; if( !push_back_record( ap, c, 0, &opt[cind] ) ) return 0;
++*argindp; cind = 0; ++*argindp; cind = 0;
} }
else if( options[index].has_arg == ap_yes ) else if( options[index].has_arg == ap_yes || options[index].has_arg == ap_yme )
{ {
if( !arg || !arg[0] ) if( !arg || ( options[index].has_arg == ap_yes && !arg[0] ) )
{ {
add_error( ap, "option requires an argument -- '" ); add_error( ap, "option requires an argument -- '" );
add_error( ap, code_str ); add_error( ap, "'" ); add_error( ap, code_str ); add_error( ap, "'" );
return 1; return 1;
} }
++*argindp; cind = 0; ++*argindp; cind = 0; /* argument may be empty */
if( !push_back_record( ap, c, 0, arg ) ) return 0; if( !push_back_record( ap, c, 0, arg ) ) return 0;
} }
else if( !push_back_record( ap, c, 0, 0 ) ) return 0; else if( !push_back_record( ap, c, 0, 0 ) ) return 0;
@ -221,9 +219,9 @@ static char parse_short_option( struct Arg_parser * const ap,
} }
char ap_init( struct Arg_parser * const ap, char ap_init( Arg_parser * const ap,
const int argc, const char * const argv[], const int argc, const char * const argv[],
const struct ap_Option options[], const char in_order ) const ap_Option options[], const char in_order )
{ {
const char ** non_options = 0; /* skipped non-options */ const char ** non_options = 0; /* skipped non-options */
int non_options_size = 0; /* number of skipped non-options */ int non_options_size = 0; /* number of skipped non-options */
@ -282,7 +280,7 @@ out: if( non_options ) free( non_options );
} }
void ap_free( struct Arg_parser * const ap ) void ap_free( Arg_parser * const ap )
{ {
free_data( ap ); free_data( ap );
if( ap->error ) { free( ap->error ); ap->error = 0; } if( ap->error ) { free( ap->error ); ap->error = 0; }
@ -290,29 +288,25 @@ void ap_free( struct Arg_parser * const ap )
} }
const char * ap_error( const struct Arg_parser * const ap ) const char * ap_error( const Arg_parser * const ap ) { return ap->error; }
{ return ap->error; }
int ap_arguments( const Arg_parser * const ap ) { return ap->data_size; }
int ap_arguments( const struct Arg_parser * const ap ) int ap_code( const Arg_parser * const ap, const int i )
{ return ap->data_size; }
int ap_code( const struct Arg_parser * const ap, const int i )
{ {
if( i < 0 || i >= ap_arguments( ap ) ) return 0; if( i < 0 || i >= ap_arguments( ap ) ) return 0;
return ap->data[i].code; return ap->data[i].code;
} }
const char * ap_parsed_name( const struct Arg_parser * const ap, const int i ) const char * ap_parsed_name( const Arg_parser * const ap, const int i )
{ {
if( i < 0 || i >= ap_arguments( ap ) || !ap->data[i].parsed_name ) return ""; if( i < 0 || i >= ap_arguments( ap ) || !ap->data[i].parsed_name ) return "";
return ap->data[i].parsed_name; return ap->data[i].parsed_name;
} }
const char * ap_argument( const struct Arg_parser * const ap, const int i ) const char * ap_argument( const Arg_parser * const ap, const int i )
{ {
if( i < 0 || i >= ap_arguments( ap ) || !ap->data[i].argument ) return ""; if( i < 0 || i >= ap_arguments( ap ) || !ap->data[i].argument ) return "";
return ap->data[i].argument; return ap->data[i].argument;

43
carg_parser.h
View file

@ -37,60 +37,65 @@
The argument '--' terminates all options; any following arguments are The argument '--' terminates all options; any following arguments are
treated as non-option arguments, even if they begin with a hyphen. treated as non-option arguments, even if they begin with a hyphen.
The syntax for optional option arguments is '-<short_option><argument>' The syntax of options with an optional argument is
(without whitespace), or '--<long_option>=<argument>'. '-<short_option><argument>' (without whitespace), or
'--<long_option>=<argument>'.
The syntax of options with an empty argument is '-<short_option> ""',
'--<long_option> ""', or '--<long_option>=""'.
*/ */
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {
#endif #endif
enum ap_Has_arg { ap_no, ap_yes, ap_maybe }; /* ap_yme = yes but maybe empty */
typedef enum ap_Has_arg { ap_no, ap_yes, ap_maybe, ap_yme } ap_Has_arg;
struct ap_Option typedef struct ap_Option
{ {
int code; /* Short option letter or code ( code != 0 ) */ int code; /* Short option letter or code ( code != 0 ) */
const char * long_name; /* Long option name (maybe null) */ const char * long_name; /* Long option name (maybe null) */
enum ap_Has_arg has_arg; ap_Has_arg has_arg;
}; } ap_Option;
struct ap_Record typedef struct ap_Record
{ {
int code; int code;
char * parsed_name; char * parsed_name;
char * argument; char * argument;
}; } ap_Record;
struct Arg_parser typedef struct Arg_parser
{ {
struct ap_Record * data; ap_Record * data;
char * error; char * error;
int data_size; int data_size;
int error_size; int error_size;
}; } Arg_parser;
char ap_init( struct Arg_parser * const ap, char ap_init( Arg_parser * const ap,
const int argc, const char * const argv[], const int argc, const char * const argv[],
const struct ap_Option options[], const char in_order ); const ap_Option options[], const char in_order );
void ap_free( struct Arg_parser * const ap ); void ap_free( Arg_parser * const ap );
const char * ap_error( const struct Arg_parser * const ap ); const char * ap_error( const Arg_parser * const ap );
/* The number of arguments parsed. May be different from argc. */ /* The number of arguments parsed. May be different from argc. */
int ap_arguments( const struct Arg_parser * const ap ); int ap_arguments( const Arg_parser * const ap );
/* If ap_code( i ) is 0, ap_argument( i ) is a non-option. /* If ap_code( i ) is 0, ap_argument( i ) is a non-option.
Else ap_argument( i ) is the option's argument (or empty). */ Else ap_argument( i ) is the option's argument (or empty). */
int ap_code( const struct Arg_parser * const ap, const int i ); int ap_code( const Arg_parser * const ap, const int i );
/* Full name of the option parsed (short or long). */ /* Full name of the option parsed (short or long). */
const char * ap_parsed_name( const struct Arg_parser * const ap, const int i ); const char * ap_parsed_name( const Arg_parser * const ap, const int i );
const char * ap_argument( const struct Arg_parser * const ap, const int i ); const char * ap_argument( const Arg_parser * const ap, const int i );
#ifdef __cplusplus #ifdef __cplusplus
} }

4
configure vendored
View file

@ -6,7 +6,7 @@
# to copy, distribute, and modify it. # to copy, distribute, and modify it.
pkgname=clzip pkgname=clzip
pkgversion=1.14 pkgversion=1.15-rc1
progname=clzip progname=clzip
srctrigger=doc/${pkgname}.texi srctrigger=doc/${pkgname}.texi
@ -109,7 +109,7 @@ while [ $# != 0 ] ; do
exit 1 ;; exit 1 ;;
esac esac
# Check if the option took a separate argument # Check whether the option took a separate argument
if [ "${arg2}" = yes ] ; then if [ "${arg2}" = yes ] ; then
if [ $# != 0 ] ; then args="${args} \"$1\"" ; shift if [ $# != 0 ] ; then args="${args} \"$1\"" ; shift
else echo "configure: Missing argument to '${option}'" 1>&2 else echo "configure: Missing argument to '${option}'" 1>&2

63
decoder.c
View file

@ -66,14 +66,14 @@ int writeblock( const int fd, const uint8_t * const buf, const int size )
} }
bool Rd_read_block( struct Range_decoder * const rdec ) bool Rd_read_block( Range_decoder * const rdec )
{ {
if( !rdec->at_stream_end ) if( !rdec->at_stream_end )
{ {
rdec->stream_pos = readblock( rdec->infd, rdec->buffer, rd_buffer_size ); rdec->stream_pos = readblock( rdec->infd, rdec->buffer, rd_buffer_size );
if( rdec->stream_pos != rd_buffer_size && errno ) if( rdec->stream_pos != rd_buffer_size && errno )
{ show_error( "Read error", errno, false ); cleanup_and_fail( 1 ); } { show_error( "Read error", errno, false ); cleanup_and_fail( 1 ); }
rdec->at_stream_end = ( rdec->stream_pos < rd_buffer_size ); rdec->at_stream_end = rdec->stream_pos < rd_buffer_size;
rdec->partial_member_pos += rdec->pos; rdec->partial_member_pos += rdec->pos;
rdec->pos = 0; rdec->pos = 0;
show_dprogress( 0, 0, 0, 0 ); show_dprogress( 0, 0, 0, 0 );
@ -82,7 +82,7 @@ bool Rd_read_block( struct Range_decoder * const rdec )
} }
void LZd_flush_data( struct LZ_decoder * const d ) void LZd_flush_data( LZ_decoder * const d )
{ {
if( d->pos > d->stream_pos ) if( d->pos > d->stream_pos )
{ {
@ -90,7 +90,7 @@ void LZd_flush_data( struct LZ_decoder * const d )
CRC32_update_buf( &d->crc, d->buffer + d->stream_pos, size ); CRC32_update_buf( &d->crc, d->buffer + d->stream_pos, size );
if( d->outfd >= 0 && if( d->outfd >= 0 &&
writeblock( d->outfd, d->buffer + d->stream_pos, size ) != size ) writeblock( d->outfd, d->buffer + d->stream_pos, size ) != size )
{ show_error( "Write error", errno, false ); cleanup_and_fail( 1 ); } { show_error( write_error_msg, errno, false ); cleanup_and_fail( 1 ); }
if( d->pos >= d->dictionary_size ) if( d->pos >= d->dictionary_size )
{ d->partial_data_pos += d->pos; d->pos = 0; d->pos_wrapped = true; } { d->partial_data_pos += d->pos; d->pos = 0; d->pos_wrapped = true; }
d->stream_pos = d->pos; d->stream_pos = d->pos;
@ -98,9 +98,7 @@ void LZd_flush_data( struct LZ_decoder * const d )
} }
static int LZd_check_trailer( struct LZ_decoder * const d, static bool LZd_check_trailer( LZ_decoder * const d, Pretty_print * const pp )
struct Pretty_print * const pp,
const bool ignore_empty )
{ {
Lzip_trailer trailer; Lzip_trailer trailer;
int size = Rd_read_data( d->rdec, trailer, Lt_size ); int size = Rd_read_data( d->rdec, trailer, Lt_size );
@ -145,8 +143,7 @@ static int LZd_check_trailer( struct LZ_decoder * const d,
fprintf( stderr, "Member size mismatch; stored %llu (0x%llX), computed %llu (0x%llX)\n", fprintf( stderr, "Member size mismatch; stored %llu (0x%llX), computed %llu (0x%llX)\n",
tm_size, tm_size, member_size, member_size ); } tm_size, tm_size, member_size, member_size ); }
} }
if( error ) return 3; if( error ) return false;
if( !ignore_empty && data_size == 0 ) return 5;
if( verbosity >= 2 ) if( verbosity >= 2 )
{ {
if( verbosity >= 4 ) show_header( d->dictionary_size ); if( verbosity >= 4 ) show_header( d->dictionary_size );
@ -161,18 +158,16 @@ static int LZd_check_trailer( struct LZ_decoder * const d,
if( verbosity >= 3 ) if( verbosity >= 3 )
fprintf( stderr, "%9llu out, %8llu in. ", data_size, member_size ); fprintf( stderr, "%9llu out, %8llu in. ", data_size, member_size );
} }
return 0; return true;
} }
/* Return value: 0 = OK, 1 = decoder error, 2 = unexpected EOF, /* Return value: 0 = OK, 1 = decoder error, 2 = unexpected EOF,
3 = trailer error, 4 = unknown marker found, 3 = trailer error, 4 = unknown marker found,
5 = empty member found, 6 = marked member found. */ 5 = nonzero first LZMA byte found. */
int LZd_decode_member( struct LZ_decoder * const d, int LZd_decode_member( LZ_decoder * const d, Pretty_print * const pp )
const struct Cl_options * const cl_opts,
struct Pretty_print * const pp )
{ {
struct Range_decoder * const rdec = d->rdec; Range_decoder * const rdec = d->rdec;
Bit_model bm_literal[1<<literal_context_bits][0x300]; Bit_model bm_literal[1<<literal_context_bits][0x300];
Bit_model bm_match[states][pos_states]; Bit_model bm_match[states][pos_states];
Bit_model bm_rep[states]; Bit_model bm_rep[states];
@ -183,8 +178,8 @@ int LZd_decode_member( struct LZ_decoder * const d,
Bit_model bm_dis_slot[len_states][1<<dis_slot_bits]; Bit_model bm_dis_slot[len_states][1<<dis_slot_bits];
Bit_model bm_dis[modeled_distances-end_dis_model+1]; Bit_model bm_dis[modeled_distances-end_dis_model+1];
Bit_model bm_align[dis_align_size]; Bit_model bm_align[dis_align_size];
struct Len_model match_len_model; Len_model match_len_model;
struct Len_model rep_len_model; Len_model rep_len_model;
unsigned rep0 = 0; /* rep[0-3] latest four distances */ unsigned rep0 = 0; /* rep[0-3] latest four distances */
unsigned rep1 = 0; /* used for efficient coding of */ unsigned rep1 = 0; /* used for efficient coding of */
unsigned rep2 = 0; /* repeated distances */ unsigned rep2 = 0; /* repeated distances */
@ -204,7 +199,7 @@ int LZd_decode_member( struct LZ_decoder * const d,
Lm_init( &match_len_model ); Lm_init( &match_len_model );
Lm_init( &rep_len_model ); Lm_init( &rep_len_model );
if( !Rd_load( rdec, cl_opts->ignore_marking ) ) return 6; if( !Rd_load( rdec ) ) return 5;
while( !Rd_finished( rdec ) ) while( !Rd_finished( rdec ) )
{ {
const int pos_state = LZd_data_position( d ) & pos_state_mask; const int pos_state = LZd_data_position( d ) & pos_state_mask;
@ -249,39 +244,33 @@ int LZd_decode_member( struct LZ_decoder * const d,
} }
else /* match */ else /* match */
{ {
rep3 = rep2; rep2 = rep1; rep1 = rep0;
len = Rd_decode_len( rdec, &match_len_model, pos_state ); len = Rd_decode_len( rdec, &match_len_model, pos_state );
unsigned distance = Rd_decode_tree6( rdec, bm_dis_slot[get_len_state(len)] ); rep0 = Rd_decode_tree6( rdec, bm_dis_slot[get_len_state(len)] );
if( distance >= start_dis_model ) if( rep0 >= start_dis_model )
{ {
const unsigned dis_slot = distance; const unsigned dis_slot = rep0;
const int direct_bits = ( dis_slot >> 1 ) - 1; const int direct_bits = ( dis_slot >> 1 ) - 1;
distance = ( 2 | ( dis_slot & 1 ) ) << direct_bits; rep0 = ( 2 | ( dis_slot & 1 ) ) << direct_bits;
if( dis_slot < end_dis_model ) if( dis_slot < end_dis_model )
distance += Rd_decode_tree_reversed( rdec, rep0 += Rd_decode_tree_reversed( rdec, bm_dis + ( rep0 - dis_slot ),
bm_dis + ( distance - dis_slot ), direct_bits ); direct_bits );
else else
{ {
distance += rep0 += Rd_decode( rdec, direct_bits - dis_align_bits ) << dis_align_bits;
Rd_decode( rdec, direct_bits - dis_align_bits ) << dis_align_bits; rep0 += Rd_decode_tree_reversed4( rdec, bm_align );
distance += Rd_decode_tree_reversed4( rdec, bm_align ); if( rep0 == 0xFFFFFFFFU ) /* marker found */
if( distance == 0xFFFFFFFFU ) /* marker found */
{ {
Rd_normalize( rdec ); Rd_normalize( rdec );
LZd_flush_data( d ); LZd_flush_data( d );
if( len == min_match_len ) /* End Of Stream marker */ if( len == min_match_len ) /* End Of Stream marker */
return LZd_check_trailer( d, pp, cl_opts->ignore_empty ); { if( LZd_check_trailer( d, pp ) ) return 0; else return 3; }
if( len == min_match_len + 1 ) /* Sync Flush marker */ if( verbosity >= 0 ) { Pp_show_msg( pp, 0 );
{ Rd_load( rdec, true ); continue; } fprintf( stderr, "Unsupported marker code '%d'\n", len ); }
if( verbosity >= 0 )
{
Pp_show_msg( pp, 0 );
fprintf( stderr, "Unsupported marker code '%d'\n", len );
}
return 4; return 4;
} }
} }
} }
rep3 = rep2; rep2 = rep1; rep1 = rep0; rep0 = distance;
state = St_set_match( state ); state = St_set_match( state );
if( rep0 >= d->dictionary_size || ( rep0 >= d->pos && !d->pos_wrapped ) ) if( rep0 >= d->dictionary_size || ( rep0 >= d->pos && !d->pos_wrapped ) )
{ LZd_flush_data( d ); return 1; } { LZd_flush_data( d ); return 1; }

95
decoder.h
View file

@ -29,9 +29,9 @@ struct Range_decoder
bool at_stream_end; bool at_stream_end;
}; };
bool Rd_read_block( struct Range_decoder * const rdec ); bool Rd_read_block( Range_decoder * const rdec );
static inline bool Rd_init( struct Range_decoder * const rdec, const int ifd ) static inline bool Rd_init( Range_decoder * const rdec, const int ifd )
{ {
rdec->partial_member_pos = 0; rdec->partial_member_pos = 0;
rdec->buffer = (uint8_t *)malloc( rd_buffer_size ); rdec->buffer = (uint8_t *)malloc( rd_buffer_size );
@ -45,27 +45,27 @@ static inline bool Rd_init( struct Range_decoder * const rdec, const int ifd )
return true; return true;
} }
static inline void Rd_free( struct Range_decoder * const rdec ) static inline void Rd_free( Range_decoder * const rdec )
{ free( rdec->buffer ); } { free( rdec->buffer ); }
static inline bool Rd_finished( struct Range_decoder * const rdec ) static inline bool Rd_finished( Range_decoder * const rdec )
{ return rdec->pos >= rdec->stream_pos && !Rd_read_block( rdec ); } { return rdec->pos >= rdec->stream_pos && !Rd_read_block( rdec ); }
static inline unsigned long long static inline unsigned long long
Rd_member_position( const struct Range_decoder * const rdec ) Rd_member_position( const Range_decoder * const rdec )
{ return rdec->partial_member_pos + rdec->pos; } { return rdec->partial_member_pos + rdec->pos; }
static inline void Rd_reset_member_position( struct Range_decoder * const rdec ) static inline void Rd_reset_member_position( Range_decoder * const rdec )
{ rdec->partial_member_pos = 0; rdec->partial_member_pos -= rdec->pos; } { rdec->partial_member_pos = 0; rdec->partial_member_pos -= rdec->pos; }
static inline uint8_t Rd_get_byte( struct Range_decoder * const rdec ) static inline uint8_t Rd_get_byte( Range_decoder * const rdec )
{ {
/* 0xFF avoids decoder error if member is truncated at EOS marker */ /* 0xFF avoids decoder error if member is truncated at EOS marker */
if( Rd_finished( rdec ) ) return 0xFF; if( Rd_finished( rdec ) ) return 0xFF;
return rdec->buffer[rdec->pos++]; return rdec->buffer[rdec->pos++];
} }
static inline int Rd_read_data( struct Range_decoder * const rdec, static inline int Rd_read_data( Range_decoder * const rdec,
uint8_t * const outbuf, const int size ) uint8_t * const outbuf, const int size )
{ {
int sz = 0; int sz = 0;
@ -79,25 +79,24 @@ static inline int Rd_read_data( struct Range_decoder * const rdec,
return sz; return sz;
} }
static inline bool Rd_load( struct Range_decoder * const rdec, static inline bool Rd_load( Range_decoder * const rdec )
const bool ignore_marking )
{ {
int i;
rdec->code = 0; rdec->code = 0;
rdec->range = 0xFFFFFFFFU; rdec->range = 0xFFFFFFFFU;
/* check and discard first byte of the LZMA stream */ /* check first byte of the LZMA stream */
if( Rd_get_byte( rdec ) != 0 && !ignore_marking ) return false; if( Rd_get_byte( rdec ) != 0 ) return false;
for( i = 0; i < 4; ++i ) rdec->code = (rdec->code << 8) | Rd_get_byte( rdec ); int i; for( i = 0; i < 4; ++i )
rdec->code = (rdec->code << 8) | Rd_get_byte( rdec );
return true; return true;
} }
static inline void Rd_normalize( struct Range_decoder * const rdec ) static inline void Rd_normalize( Range_decoder * const rdec )
{ {
if( rdec->range <= 0x00FFFFFFU ) if( rdec->range <= 0x00FFFFFFU )
{ rdec->range <<= 8; rdec->code = (rdec->code << 8) | Rd_get_byte( rdec ); } { rdec->range <<= 8; rdec->code = (rdec->code << 8) | Rd_get_byte( rdec ); }
} }
static inline unsigned Rd_decode( struct Range_decoder * const rdec, static inline unsigned Rd_decode( Range_decoder * const rdec,
const int num_bits ) const int num_bits )
{ {
unsigned symbol = 0; unsigned symbol = 0;
@ -108,14 +107,14 @@ static inline unsigned Rd_decode( struct Range_decoder * const rdec,
rdec->range >>= 1; rdec->range >>= 1;
/* symbol <<= 1; */ /* symbol <<= 1; */
/* if( rdec->code >= rdec->range ) { rdec->code -= rdec->range; symbol |= 1; } */ /* if( rdec->code >= rdec->range ) { rdec->code -= rdec->range; symbol |= 1; } */
const bool bit = ( rdec->code >= rdec->range ); const bool bit = rdec->code >= rdec->range;
symbol <<= 1; symbol += bit; symbol <<= 1; symbol += bit;
rdec->code -= rdec->range & ( 0U - bit ); rdec->code -= rdec->range & ( 0U - bit );
} }
return symbol; return symbol;
} }
static inline unsigned Rd_decode_bit( struct Range_decoder * const rdec, static inline unsigned Rd_decode_bit( Range_decoder * const rdec,
Bit_model * const probability ) Bit_model * const probability )
{ {
Rd_normalize( rdec ); Rd_normalize( rdec );
@ -135,7 +134,7 @@ static inline unsigned Rd_decode_bit( struct Range_decoder * const rdec,
} }
} }
static inline void Rd_decode_symbol_bit( struct Range_decoder * const rdec, static inline void Rd_decode_symbol_bit( Range_decoder * const rdec,
Bit_model * const probability, unsigned * symbol ) Bit_model * const probability, unsigned * symbol )
{ {
Rd_normalize( rdec ); Rd_normalize( rdec );
@ -155,7 +154,7 @@ static inline void Rd_decode_symbol_bit( struct Range_decoder * const rdec,
} }
} }
static inline void Rd_decode_symbol_bit_reversed( struct Range_decoder * const rdec, static inline void Rd_decode_symbol_bit_reversed( Range_decoder * const rdec,
Bit_model * const probability, unsigned * model, Bit_model * const probability, unsigned * model,
unsigned * symbol, const int i ) unsigned * symbol, const int i )
{ {
@ -177,7 +176,7 @@ static inline void Rd_decode_symbol_bit_reversed( struct Range_decoder * const r
} }
} }
static inline unsigned Rd_decode_tree6( struct Range_decoder * const rdec, static inline unsigned Rd_decode_tree6( Range_decoder * const rdec,
Bit_model bm[] ) Bit_model bm[] )
{ {
unsigned symbol = 1; unsigned symbol = 1;
@ -190,7 +189,7 @@ static inline unsigned Rd_decode_tree6( struct Range_decoder * const rdec,
return symbol & 0x3F; return symbol & 0x3F;
} }
static inline unsigned Rd_decode_tree8( struct Range_decoder * const rdec, static inline unsigned Rd_decode_tree8( Range_decoder * const rdec,
Bit_model bm[] ) Bit_model bm[] )
{ {
unsigned symbol = 1; unsigned symbol = 1;
@ -205,9 +204,8 @@ static inline unsigned Rd_decode_tree8( struct Range_decoder * const rdec,
return symbol & 0xFF; return symbol & 0xFF;
} }
static inline unsigned static inline unsigned Rd_decode_tree_reversed( Range_decoder * const rdec,
Rd_decode_tree_reversed( struct Range_decoder * const rdec, Bit_model bm[], const int num_bits )
Bit_model bm[], const int num_bits )
{ {
unsigned model = 1; unsigned model = 1;
unsigned symbol = 0; unsigned symbol = 0;
@ -218,7 +216,7 @@ Rd_decode_tree_reversed( struct Range_decoder * const rdec,
} }
static inline unsigned static inline unsigned
Rd_decode_tree_reversed4( struct Range_decoder * const rdec, Bit_model bm[] ) Rd_decode_tree_reversed4( Range_decoder * const rdec, Bit_model bm[] )
{ {
unsigned model = 1; unsigned model = 1;
unsigned symbol = 0; unsigned symbol = 0;
@ -229,7 +227,7 @@ Rd_decode_tree_reversed4( struct Range_decoder * const rdec, Bit_model bm[] )
return symbol; return symbol;
} }
static inline unsigned Rd_decode_matched( struct Range_decoder * const rdec, static inline unsigned Rd_decode_matched( Range_decoder * const rdec,
Bit_model bm[], unsigned match_byte ) Bit_model bm[], unsigned match_byte )
{ {
unsigned symbol = 1; unsigned symbol = 1;
@ -244,9 +242,8 @@ static inline unsigned Rd_decode_matched( struct Range_decoder * const rdec,
} }
} }
static inline unsigned Rd_decode_len( struct Range_decoder * const rdec, static inline unsigned Rd_decode_len( Range_decoder * const rdec,
struct Len_model * const lm, Len_model * const lm, const int pos_state )
const int pos_state )
{ {
Bit_model * bm; Bit_model * bm;
unsigned mask, offset, symbol = 1; unsigned mask, offset, symbol = 1;
@ -269,10 +266,10 @@ len3:
} }
struct LZ_decoder typedef struct LZ_decoder
{ {
unsigned long long partial_data_pos; unsigned long long partial_data_pos;
struct Range_decoder * rdec; Range_decoder * rdec;
unsigned dictionary_size; unsigned dictionary_size;
uint8_t * buffer; /* output buffer */ uint8_t * buffer; /* output buffer */
unsigned pos; /* current pos in buffer */ unsigned pos; /* current pos in buffer */
@ -280,14 +277,14 @@ struct LZ_decoder
uint32_t crc; uint32_t crc;
int outfd; /* output file descriptor */ int outfd; /* output file descriptor */
bool pos_wrapped; bool pos_wrapped;
}; } LZ_decoder;
void LZd_flush_data( struct LZ_decoder * const d ); void LZd_flush_data( LZ_decoder * const d );
static inline uint8_t LZd_peek_prev( const struct LZ_decoder * const d ) static inline uint8_t LZd_peek_prev( const LZ_decoder * const d )
{ return d->buffer[((d->pos > 0) ? d->pos : d->dictionary_size)-1]; } { return d->buffer[((d->pos > 0) ? d->pos : d->dictionary_size)-1]; }
static inline uint8_t LZd_peek( const struct LZ_decoder * const d, static inline uint8_t LZd_peek( const LZ_decoder * const d,
const unsigned distance ) const unsigned distance )
{ {
const unsigned i = ( ( d->pos > distance ) ? 0 : d->dictionary_size ) + const unsigned i = ( ( d->pos > distance ) ? 0 : d->dictionary_size ) +
@ -295,27 +292,27 @@ static inline uint8_t LZd_peek( const struct LZ_decoder * const d,
return d->buffer[i]; return d->buffer[i];
} }
static inline void LZd_put_byte( struct LZ_decoder * const d, const uint8_t b ) static inline void LZd_put_byte( LZ_decoder * const d, const uint8_t b )
{ {
d->buffer[d->pos] = b; d->buffer[d->pos] = b;
if( ++d->pos >= d->dictionary_size ) LZd_flush_data( d ); if( ++d->pos >= d->dictionary_size ) LZd_flush_data( d );
} }
static inline void LZd_copy_block( struct LZ_decoder * const d, static inline void LZd_copy_block( LZ_decoder * const d,
const unsigned distance, unsigned len ) const unsigned distance, unsigned len )
{ {
unsigned lpos = d->pos, i = lpos - distance - 1; unsigned lpos = d->pos, i = lpos - distance - 1;
bool fast, fast2; bool fast, fast2;
if( lpos > distance ) if( lpos > distance )
{ {
fast = ( len < d->dictionary_size - lpos ); fast = len < d->dictionary_size - lpos;
fast2 = ( fast && len <= lpos - i ); fast2 = fast && len <= lpos - i;
} }
else else
{ {
i += d->dictionary_size; i += d->dictionary_size;
fast = ( len < d->dictionary_size - i ); /* (i == pos) may happen */ fast = len < d->dictionary_size - i; /* (i == pos) may happen */
fast2 = ( fast && len <= i - lpos ); fast2 = fast && len <= i - lpos;
} }
if( fast ) /* no wrap */ if( fast ) /* no wrap */
{ {
@ -333,8 +330,7 @@ static inline void LZd_copy_block( struct LZ_decoder * const d,
} }
} }
static inline bool LZd_init( struct LZ_decoder * const d, static inline bool LZd_init( LZ_decoder * const d, Range_decoder * const rde,
struct Range_decoder * const rde,
const unsigned dict_size, const int ofd ) const unsigned dict_size, const int ofd )
{ {
d->partial_data_pos = 0; d->partial_data_pos = 0;
@ -352,16 +348,13 @@ static inline bool LZd_init( struct LZ_decoder * const d,
return true; return true;
} }
static inline void LZd_free( struct LZ_decoder * const d ) static inline void LZd_free( LZ_decoder * const d )
{ free( d->buffer ); } { free( d->buffer ); }
static inline unsigned LZd_crc( const struct LZ_decoder * const d ) static inline unsigned LZd_crc( const LZ_decoder * const d )
{ return d->crc ^ 0xFFFFFFFFU; } { return d->crc ^ 0xFFFFFFFFU; }
static inline unsigned long long static inline unsigned long long LZd_data_position( const LZ_decoder * const d )
LZd_data_position( const struct LZ_decoder * const d )
{ return d->partial_data_pos + d->pos; } { return d->partial_data_pos + d->pos; }
int LZd_decode_member( struct LZ_decoder * const d, int LZd_decode_member( LZ_decoder * const d, Pretty_print * const pp );
const struct Cl_options * const cl_opts,
struct Pretty_print * const pp );

35
doc/clzip.1
View file

@ -1,26 +1,25 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.2. .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.2.
.TH CLZIP "1" "January 2024" "clzip 1.14" "User Commands" .TH CLZIP "1" "November 2024" "clzip 1.15-rc1" "User Commands"
.SH NAME .SH NAME
clzip \- reduces the size of files clzip \- reduces the size of files
.SH SYNOPSIS .SH SYNOPSIS
.B clzip .B clzip
[\fI\,options\/\fR] [\fI\,files\/\fR] [\fI\,options\/\fR] [\fI\,files\/\fR]
.SH DESCRIPTION .SH DESCRIPTION
Clzip is a C language version of lzip, compatible with lzip 1.4 or newer. As Clzip is a C language version of lzip intended for systems lacking a C++
clzip is written in C, it may be easier to integrate in applications like compiler.
package managers, embedded devices, or systems lacking a C++ compiler.
.PP .PP
Lzip is a lossless data compressor with a user interface similar to the one Lzip is a lossless data compressor with a user interface similar to the one
of gzip or bzip2. Lzip uses a simplified form of the 'Lempel\-Ziv\-Markov of gzip or bzip2. Lzip uses a simplified form of LZMA (Lempel\-Ziv\-Markov
chain\-Algorithm' (LZMA) stream format to maximize interoperability. The chain\-Algorithm) designed to achieve complete interoperability between
maximum dictionary size is 512 MiB so that any lzip file can be decompressed implementations. The maximum dictionary size is 512 MiB so that any lzip
on 32\-bit machines. Lzip provides accurate and robust 3\-factor integrity file can be decompressed on 32\-bit machines. Lzip provides accurate and
checking. Lzip can compress about as fast as gzip (lzip \fB\-0\fR) or compress most robust 3\-factor integrity checking. 'lzip \fB\-0\fR' compresses about as fast as
files more than bzip2 (lzip \fB\-9\fR). Decompression speed is intermediate between gzip, while 'lzip \fB\-9\fR' compresses most files more than bzip2. Decompression
gzip and bzip2. Lzip is better than gzip and bzip2 from a data recovery speed is intermediate between gzip and bzip2. Lzip provides better data
perspective. Lzip has been designed, written, and tested with great care to recovery capabilities than gzip and bzip2. Lzip has been designed, written,
replace gzip and bzip2 as the standard general\-purpose compressed format for and tested with great care to replace gzip and bzip2 as general\-purpose
Unix\-like systems. compressed format for Unix\-like systems.
.SH OPTIONS .SH OPTIONS
.TP .TP
\fB\-h\fR, \fB\-\-help\fR \fB\-h\fR, \fB\-\-help\fR
@ -33,7 +32,7 @@ output version information and exit
exit with error status if trailing data exit with error status if trailing data
.TP .TP
\fB\-b\fR, \fB\-\-member\-size=\fR<bytes> \fB\-b\fR, \fB\-\-member\-size=\fR<bytes>
set member size limit in bytes set member size limit of multimember files
.TP .TP
\fB\-c\fR, \fB\-\-stdout\fR \fB\-c\fR, \fB\-\-stdout\fR
write to standard output, keep input files write to standard output, keep input files
@ -83,12 +82,6 @@ alias for \fB\-0\fR
\fB\-\-best\fR \fB\-\-best\fR
alias for \fB\-9\fR alias for \fB\-9\fR
.TP .TP
\fB\-\-empty\-error\fR
exit with error status if empty member in file
.TP
\fB\-\-marking\-error\fR
exit with error status if 1st LZMA byte not 0
.TP
\fB\-\-loose\-trailing\fR \fB\-\-loose\-trailing\fR
allow trailing data seeming corrupt header allow trailing data seeming corrupt header
.PP .PP

753
doc/clzip.info
View file

File diff suppressed because it is too large Load diff

751
doc/clzip.texi
View file

File diff suppressed because it is too large Load diff

23
encoder.c
View file

@ -31,7 +31,7 @@
CRC32 crc32; CRC32 crc32;
int LZe_get_match_pairs( struct LZ_encoder * const e, struct Pair * pairs ) int LZe_get_match_pairs( LZ_encoder * const e, Pair * pairs )
{ {
int len_limit = e->match_len_limit; int len_limit = e->match_len_limit;
if( len_limit > Mb_available_bytes( &e->eb.mb ) ) if( len_limit > Mb_available_bytes( &e->eb.mb ) )
@ -133,7 +133,7 @@ int LZe_get_match_pairs( struct LZ_encoder * const e, struct Pair * pairs )
} }
static void LZe_update_distance_prices( struct LZ_encoder * const e ) static void LZe_update_distance_prices( LZ_encoder * const e )
{ {
int dis, len_state; int dis, len_state;
for( dis = start_dis_model; dis < modeled_distances; ++dis ) for( dis = start_dis_model; dis < modeled_distances; ++dis )
@ -172,7 +172,7 @@ static void LZe_update_distance_prices( struct LZ_encoder * const e )
( trials[0].dis4 == -1 ) means literal. ( trials[0].dis4 == -1 ) means literal.
A match/rep longer or equal than match_len_limit finishes the sequence. A match/rep longer or equal than match_len_limit finishes the sequence.
*/ */
static int LZe_sequence_optimizer( struct LZ_encoder * const e, static int LZe_sequence_optimizer( LZ_encoder * const e,
const int reps[num_rep_distances], const int reps[num_rep_distances],
const State state ) const State state )
{ {
@ -291,7 +291,7 @@ static int LZe_sequence_optimizer( struct LZ_encoder * const e,
} }
/* give final values to current trial */ /* give final values to current trial */
struct Trial * cur_trial = &e->trials[cur]; Trial * cur_trial = &e->trials[cur];
State cur_state; State cur_state;
{ {
const int dis4 = cur_trial->dis4; const int dis4 = cur_trial->dis4;
@ -336,7 +336,7 @@ static int LZe_sequence_optimizer( struct LZ_encoder * const e,
next_price += LZeb_price_matched( &e->eb, prev_byte, cur_byte, match_byte ); next_price += LZeb_price_matched( &e->eb, prev_byte, cur_byte, match_byte );
/* try last updates to next trial */ /* try last updates to next trial */
struct Trial * next_trial = &e->trials[cur+1]; Trial * next_trial = &e->trials[cur+1];
Tr_update( next_trial, next_price, -1, cur ); /* literal */ Tr_update( next_trial, next_price, -1, cur ); /* literal */
@ -346,8 +346,7 @@ static int LZe_sequence_optimizer( struct LZ_encoder * const e,
if( match_byte == cur_byte && next_trial->dis4 != 0 && if( match_byte == cur_byte && next_trial->dis4 != 0 &&
next_trial->prev_index2 == single_step_trial ) next_trial->prev_index2 == single_step_trial )
{ {
const int price = rep_match_price + const int price = rep_match_price + LZeb_price_shortrep( &e->eb, cur_state, pos_state );
LZeb_price_shortrep( &e->eb, cur_state, pos_state );
if( price <= next_trial->price ) if( price <= next_trial->price )
{ {
next_trial->price = price; next_trial->price = price;
@ -478,12 +477,12 @@ static int LZe_sequence_optimizer( struct LZ_encoder * const e,
} }
bool LZe_encode_member( struct LZ_encoder * const e, bool LZe_encode_member( LZ_encoder * const e,
const unsigned long long member_size ) const unsigned long long member_size )
{ {
const unsigned long long member_size_limit = const unsigned long long member_size_limit =
member_size - Lt_size - max_marker_size; member_size - Lt_size - max_marker_size;
const bool best = ( e->match_len_limit > 12 ); const bool best = e->match_len_limit > 12;
const int dis_price_count = best ? 1 : 512; const int dis_price_count = best ? 1 : 512;
const int align_price_count = best ? 1 : dis_align_size; const int align_price_count = best ? 1 : dis_align_size;
const int price_count = ( e->match_len_limit > 36 ) ? 1013 : 4093; const int price_count = ( e->match_len_limit > 36 ) ? 1013 : 4093;
@ -537,7 +536,7 @@ bool LZe_encode_member( struct LZ_encoder * const e,
const int len = e->trials[i].price; const int len = e->trials[i].price;
int dis = e->trials[i].dis4; int dis = e->trials[i].dis4;
bool bit = ( dis < 0 ); bool bit = dis < 0;
Re_encode_bit( &e->eb.renc, &e->eb.bm_match[state][pos_state], !bit ); Re_encode_bit( &e->eb.renc, &e->eb.bm_match[state][pos_state], !bit );
if( bit ) /* literal byte */ if( bit ) /* literal byte */
{ {
@ -556,11 +555,11 @@ bool LZe_encode_member( struct LZ_encoder * const e,
{ {
CRC32_update_buf( &e->eb.crc, Mb_ptr_to_current_pos( &e->eb.mb ) - ahead, len ); CRC32_update_buf( &e->eb.crc, Mb_ptr_to_current_pos( &e->eb.mb ) - ahead, len );
mtf_reps( dis, reps ); mtf_reps( dis, reps );
bit = ( dis < num_rep_distances ); bit = dis < num_rep_distances;
Re_encode_bit( &e->eb.renc, &e->eb.bm_rep[state], bit ); Re_encode_bit( &e->eb.renc, &e->eb.bm_rep[state], bit );
if( bit ) /* repeated match */ if( bit ) /* repeated match */
{ {
bit = ( dis == 0 ); bit = dis == 0;
Re_encode_bit( &e->eb.renc, &e->eb.bm_rep0[state], !bit ); Re_encode_bit( &e->eb.renc, &e->eb.bm_rep0[state], !bit );
if( bit ) if( bit )
Re_encode_bit( &e->eb.renc, &e->eb.bm_len[state][pos_state], len > 1 ); Re_encode_bit( &e->eb.renc, &e->eb.bm_len[state][pos_state], len > 1 );

81
encoder.h
View file

@ -15,16 +15,16 @@
along with this program. If not, see <http://www.gnu.org/licenses/>. along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
struct Len_prices typedef struct Len_prices
{ {
const struct Len_model * lm; const Len_model * lm;
int len_symbols; int len_symbols;
int count; int count;
int prices[pos_states][max_len_symbols]; int prices[pos_states][max_len_symbols];
int counters[pos_states]; /* may decrement below 0 */ int counters[pos_states]; /* may decrement below 0 */
}; } Len_prices;
static inline void Lp_update_low_mid_prices( struct Len_prices * const lp, static inline void Lp_update_low_mid_prices( Len_prices * const lp,
const int pos_state ) const int pos_state )
{ {
int * const pps = lp->prices[pos_state]; int * const pps = lp->prices[pos_state];
@ -39,7 +39,7 @@ static inline void Lp_update_low_mid_prices( struct Len_prices * const lp,
price_symbol3( lp->lm->bm_mid[pos_state], len - len_low_symbols ); price_symbol3( lp->lm->bm_mid[pos_state], len - len_low_symbols );
} }
static inline void Lp_update_high_prices( struct Len_prices * const lp ) static inline void Lp_update_high_prices( Len_prices * const lp )
{ {
const int tmp = price1( lp->lm->choice1 ) + price1( lp->lm->choice2 ); const int tmp = price1( lp->lm->choice1 ) + price1( lp->lm->choice2 );
int len; int len;
@ -50,11 +50,10 @@ static inline void Lp_update_high_prices( struct Len_prices * const lp )
price_symbol8( lp->lm->bm_high, len - len_low_symbols - len_mid_symbols ); price_symbol8( lp->lm->bm_high, len - len_low_symbols - len_mid_symbols );
} }
static inline void Lp_reset( struct Len_prices * const lp ) static inline void Lp_reset( Len_prices * const lp )
{ int i; for( i = 0; i < pos_states; ++i ) lp->counters[i] = 0; } { int i; for( i = 0; i < pos_states; ++i ) lp->counters[i] = 0; }
static inline void Lp_init( struct Len_prices * const lp, static inline void Lp_init( Len_prices * const lp, const Len_model * const lm,
const struct Len_model * const lm,
const int match_len_limit ) const int match_len_limit )
{ {
lp->lm = lm; lp->lm = lm;
@ -63,11 +62,11 @@ static inline void Lp_init( struct Len_prices * const lp,
Lp_reset( lp ); Lp_reset( lp );
} }
static inline void Lp_decrement_counter( struct Len_prices * const lp, static inline void Lp_decrement_counter( Len_prices * const lp,
const int pos_state ) const int pos_state )
{ --lp->counters[pos_state]; } { --lp->counters[pos_state]; }
static inline void Lp_update_prices( struct Len_prices * const lp ) static inline void Lp_update_prices( Len_prices * const lp )
{ {
int pos_state; int pos_state;
bool high_pending = false; bool high_pending = false;
@ -79,23 +78,23 @@ static inline void Lp_update_prices( struct Len_prices * const lp )
Lp_update_high_prices( lp ); Lp_update_high_prices( lp );
} }
static inline int Lp_price( const struct Len_prices * const lp, static inline int Lp_price( const Len_prices * const lp,
const int len, const int pos_state ) const int len, const int pos_state )
{ return lp->prices[pos_state][len - min_match_len]; } { return lp->prices[pos_state][len - min_match_len]; }
struct Pair /* distance-length pair */ typedef struct Pair /* distance-length pair */
{ {
int dis; int dis;
int len; int len;
}; } Pair;
enum { infinite_price = 0x0FFFFFFF, enum { infinite_price = 0x0FFFFFFF,
max_num_trials = 1 << 13, max_num_trials = 1 << 13,
single_step_trial = -2, single_step_trial = -2,
dual_step_trial = -1 }; dual_step_trial = -1 };
struct Trial typedef struct Trial
{ {
State state; State state;
int price; /* dual use var; cumulative price, match length */ int price; /* dual use var; cumulative price, match length */
@ -105,9 +104,9 @@ struct Trial
/* -1 literal + rep0 */ /* -1 literal + rep0 */
/* >= 0 ( rep or match ) + literal + rep0 */ /* >= 0 ( rep or match ) + literal + rep0 */
int reps[num_rep_distances]; int reps[num_rep_distances];
}; } Trial;
static inline void Tr_update( struct Trial * const trial, const int pr, static inline void Tr_update( Trial * const trial, const int pr,
const int distance4, const int p_i ) const int distance4, const int p_i )
{ {
if( pr < trial->price ) if( pr < trial->price )
@ -115,7 +114,7 @@ static inline void Tr_update( struct Trial * const trial, const int pr,
trial->prev_index2 = single_step_trial; } trial->prev_index2 = single_step_trial; }
} }
static inline void Tr_update2( struct Trial * const trial, const int pr, static inline void Tr_update2( Trial * const trial, const int pr,
const int p_i ) const int p_i )
{ {
if( pr < trial->price ) if( pr < trial->price )
@ -123,7 +122,7 @@ static inline void Tr_update2( struct Trial * const trial, const int pr,
trial->prev_index2 = dual_step_trial; } trial->prev_index2 = dual_step_trial; }
} }
static inline void Tr_update3( struct Trial * const trial, const int pr, static inline void Tr_update3( Trial * const trial, const int pr,
const int distance4, const int p_i, const int distance4, const int p_i,
const int p_i2 ) const int p_i2 )
{ {
@ -133,25 +132,24 @@ static inline void Tr_update3( struct Trial * const trial, const int pr,
} }
struct LZ_encoder typedef struct LZ_encoder
{ {
struct LZ_encoder_base eb; LZ_encoder_base eb;
int cycles; int cycles;
int match_len_limit; int match_len_limit;
struct Len_prices match_len_prices; Len_prices match_len_prices;
struct Len_prices rep_len_prices; Len_prices rep_len_prices;
int pending_num_pairs; int pending_num_pairs;
struct Pair pairs[max_match_len+1]; Pair pairs[max_match_len+1];
struct Trial trials[max_num_trials]; Trial trials[max_num_trials];
int dis_slot_prices[len_states][2*max_dictionary_bits]; int dis_slot_prices[len_states][2*max_dictionary_bits];
int dis_prices[len_states][modeled_distances]; int dis_prices[len_states][modeled_distances];
int align_prices[dis_align_size]; int align_prices[dis_align_size];
int num_dis_slots; int num_dis_slots;
}; } LZ_encoder;
static inline bool Mb_dec_pos( struct Matchfinder_base * const mb, static inline bool Mb_dec_pos( Matchfinder_base * const mb, const int ahead )
const int ahead )
{ {
if( ahead < 0 || mb->pos < ahead ) return false; if( ahead < 0 || mb->pos < ahead ) return false;
mb->pos -= ahead; mb->pos -= ahead;
@ -160,7 +158,7 @@ static inline bool Mb_dec_pos( struct Matchfinder_base * const mb,
return true; return true;
} }
int LZe_get_match_pairs( struct LZ_encoder * const e, struct Pair * pairs ); int LZe_get_match_pairs( LZ_encoder * const e, Pair * pairs );
/* move-to-front dis in/into reps; do nothing if( dis4 <= 0 ) */ /* move-to-front dis in/into reps; do nothing if( dis4 <= 0 ) */
static inline void mtf_reps( const int dis4, int reps[num_rep_distances] ) static inline void mtf_reps( const int dis4, int reps[num_rep_distances] )
@ -178,13 +176,13 @@ static inline void mtf_reps( const int dis4, int reps[num_rep_distances] )
} }
} }
static inline int LZeb_price_shortrep( const struct LZ_encoder_base * const eb, static inline int LZeb_price_shortrep( const LZ_encoder_base * const eb,
const State state, const int pos_state ) const State state, const int pos_state )
{ {
return price0( eb->bm_rep0[state] ) + price0( eb->bm_len[state][pos_state] ); return price0( eb->bm_rep0[state] ) + price0( eb->bm_len[state][pos_state] );
} }
static inline int LZeb_price_rep( const struct LZ_encoder_base * const eb, static inline int LZeb_price_rep( const LZ_encoder_base * const eb,
const int rep, const State state, const int rep, const State state,
const int pos_state ) const int pos_state )
{ {
@ -201,7 +199,7 @@ static inline int LZeb_price_rep( const struct LZ_encoder_base * const eb,
return price; return price;
} }
static inline int LZe_price_rep0_len( const struct LZ_encoder * const e, static inline int LZe_price_rep0_len( const LZ_encoder * const e,
const int len, const State state, const int len, const State state,
const int pos_state ) const int pos_state )
{ {
@ -209,9 +207,8 @@ static inline int LZe_price_rep0_len( const struct LZ_encoder * const e,
Lp_price( &e->rep_len_prices, len, pos_state ); Lp_price( &e->rep_len_prices, len, pos_state );
} }
static inline int LZe_price_pair( const struct LZ_encoder * const e, static inline int LZe_price_pair( const LZ_encoder * const e, const int dis,
const int dis, const int len, const int len, const int pos_state )
const int pos_state )
{ {
const int price = Lp_price( &e->match_len_prices, len, pos_state ); const int price = Lp_price( &e->match_len_prices, len, pos_state );
const int len_state = get_len_state( len ); const int len_state = get_len_state( len );
@ -222,7 +219,7 @@ static inline int LZe_price_pair( const struct LZ_encoder * const e,
e->align_prices[dis & (dis_align_size - 1)]; e->align_prices[dis & (dis_align_size - 1)];
} }
static inline int LZe_read_match_distances( struct LZ_encoder * const e ) static inline int LZe_read_match_distances( LZ_encoder * const e )
{ {
const int num_pairs = LZe_get_match_pairs( e, e->pairs ); const int num_pairs = LZe_get_match_pairs( e, e->pairs );
if( num_pairs > 0 ) if( num_pairs > 0 )
@ -235,7 +232,7 @@ static inline int LZe_read_match_distances( struct LZ_encoder * const e )
return num_pairs; return num_pairs;
} }
static inline void LZe_move_and_update( struct LZ_encoder * const e, int n ) static inline void LZe_move_and_update( LZ_encoder * const e, int n )
{ {
while( true ) while( true )
{ {
@ -245,13 +242,13 @@ static inline void LZe_move_and_update( struct LZ_encoder * const e, int n )
} }
} }
static inline void LZe_backward( struct LZ_encoder * const e, int cur ) static inline void LZe_backward( LZ_encoder * const e, int cur )
{ {
int dis4 = e->trials[cur].dis4; int dis4 = e->trials[cur].dis4;
while( cur > 0 ) while( cur > 0 )
{ {
const int prev_index = e->trials[cur].prev_index; const int prev_index = e->trials[cur].prev_index;
struct Trial * const prev_trial = &e->trials[prev_index]; Trial * const prev_trial = &e->trials[prev_index];
if( e->trials[cur].prev_index2 != single_step_trial ) if( e->trials[cur].prev_index2 != single_step_trial )
{ {
@ -260,7 +257,7 @@ static inline void LZe_backward( struct LZ_encoder * const e, int cur )
prev_trial->prev_index2 = single_step_trial; prev_trial->prev_index2 = single_step_trial;
if( e->trials[cur].prev_index2 >= 0 ) if( e->trials[cur].prev_index2 >= 0 )
{ {
struct Trial * const prev_trial2 = &e->trials[prev_index-1]; Trial * const prev_trial2 = &e->trials[prev_index-1];
prev_trial2->dis4 = dis4; dis4 = 0; /* rep0 */ prev_trial2->dis4 = dis4; dis4 = 0; /* rep0 */
prev_trial2->prev_index = e->trials[cur].prev_index2; prev_trial2->prev_index = e->trials[cur].prev_index2;
prev_trial2->prev_index2 = single_step_trial; prev_trial2->prev_index2 = single_step_trial;
@ -275,7 +272,7 @@ static inline void LZe_backward( struct LZ_encoder * const e, int cur )
enum { num_prev_positions3 = 1 << 16, enum { num_prev_positions3 = 1 << 16,
num_prev_positions2 = 1 << 10 }; num_prev_positions2 = 1 << 10 };
static inline bool LZe_init( struct LZ_encoder * const e, static inline bool LZe_init( LZ_encoder * const e,
const int dict_size, const int len_limit, const int dict_size, const int len_limit,
const int ifd, const int outfd ) const int ifd, const int outfd )
{ {
@ -300,7 +297,7 @@ static inline bool LZe_init( struct LZ_encoder * const e,
return true; return true;
} }
static inline void LZe_reset( struct LZ_encoder * const e ) static inline void LZe_reset( LZ_encoder * const e )
{ {
LZeb_reset( &e->eb ); LZeb_reset( &e->eb );
Lp_reset( &e->match_len_prices ); Lp_reset( &e->match_len_prices );
@ -308,5 +305,5 @@ static inline void LZe_reset( struct LZ_encoder * const e )
e->pending_num_pairs = 0; e->pending_num_pairs = 0;
} }
bool LZe_encode_member( struct LZ_encoder * const e, bool LZe_encode_member( LZ_encoder * const e,
const unsigned long long member_size ); const unsigned long long member_size );

16
encoder_base.c
View file

@ -31,7 +31,7 @@ Dis_slots dis_slots;
Prob_prices prob_prices; Prob_prices prob_prices;
bool Mb_read_block( struct Matchfinder_base * const mb ) bool Mb_read_block( Matchfinder_base * const mb )
{ {
if( !mb->at_stream_end && mb->stream_pos < mb->buffer_size ) if( !mb->at_stream_end && mb->stream_pos < mb->buffer_size )
{ {
@ -46,7 +46,7 @@ bool Mb_read_block( struct Matchfinder_base * const mb )
} }
void Mb_normalize_pos( struct Matchfinder_base * const mb ) void Mb_normalize_pos( Matchfinder_base * const mb )
{ {
if( mb->pos > mb->stream_pos ) if( mb->pos > mb->stream_pos )
internal_error( "pos > stream_pos in Mb_normalize_pos." ); internal_error( "pos > stream_pos in Mb_normalize_pos." );
@ -69,7 +69,7 @@ void Mb_normalize_pos( struct Matchfinder_base * const mb )
} }
bool Mb_init( struct Matchfinder_base * const mb, const int before_size, bool Mb_init( Matchfinder_base * const mb, const int before_size,
const int dict_size, const int after_size, const int dict_size, const int after_size,
const int dict_factor, const int num_prev_positions23, const int dict_factor, const int num_prev_positions23,
const int pos_array_factor, const int ifd ) const int pos_array_factor, const int ifd )
@ -123,7 +123,7 @@ bool Mb_init( struct Matchfinder_base * const mb, const int before_size,
} }
void Mb_reset( struct Matchfinder_base * const mb ) void Mb_reset( Matchfinder_base * const mb )
{ {
int i; int i;
if( mb->stream_pos > mb->pos ) if( mb->stream_pos > mb->pos )
@ -147,13 +147,13 @@ void Mb_reset( struct Matchfinder_base * const mb )
} }
void Re_flush_data( struct Range_encoder * const renc ) void Re_flush_data( Range_encoder * const renc )
{ {
if( renc->pos > 0 ) if( renc->pos > 0 )
{ {
if( renc->outfd >= 0 && if( renc->outfd >= 0 &&
writeblock( renc->outfd, renc->buffer, renc->pos ) != renc->pos ) writeblock( renc->outfd, renc->buffer, renc->pos ) != renc->pos )
{ show_error( "Write error", errno, false ); cleanup_and_fail( 1 ); } { show_error( write_error_msg, errno, false ); cleanup_and_fail( 1 ); }
renc->partial_member_pos += renc->pos; renc->partial_member_pos += renc->pos;
renc->pos = 0; renc->pos = 0;
show_cprogress( 0, 0, 0, 0 ); show_cprogress( 0, 0, 0, 0 );
@ -162,7 +162,7 @@ void Re_flush_data( struct Range_encoder * const renc )
/* End Of Stream marker => (dis == 0xFFFFFFFFU, len == min_match_len) */ /* End Of Stream marker => (dis == 0xFFFFFFFFU, len == min_match_len) */
void LZeb_full_flush( struct LZ_encoder_base * const eb, const State state ) void LZeb_full_flush( LZ_encoder_base * const eb, const State state )
{ {
const int pos_state = Mb_data_position( &eb->mb ) & pos_state_mask; const int pos_state = Mb_data_position( &eb->mb ) & pos_state_mask;
Re_encode_bit( &eb->renc, &eb->bm_match[state][pos_state], 1 ); Re_encode_bit( &eb->renc, &eb->bm_match[state][pos_state], 1 );
@ -178,7 +178,7 @@ void LZeb_full_flush( struct LZ_encoder_base * const eb, const State state )
} }
void LZeb_reset( struct LZ_encoder_base * const eb ) void LZeb_reset( LZ_encoder_base * const eb )
{ {
Mb_reset( &eb->mb ); Mb_reset( &eb->mb );
eb->crc = 0xFFFFFFFFU; eb->crc = 0xFFFFFFFFU;

101
encoder_base.h
View file

@ -174,36 +174,36 @@ struct Matchfinder_base
bool at_stream_end; /* stream_pos shows real end of file */ bool at_stream_end; /* stream_pos shows real end of file */
}; };
bool Mb_read_block( struct Matchfinder_base * const mb ); bool Mb_read_block( Matchfinder_base * const mb );
void Mb_normalize_pos( struct Matchfinder_base * const mb ); void Mb_normalize_pos( Matchfinder_base * const mb );
bool Mb_init( struct Matchfinder_base * const mb, const int before_size, bool Mb_init( Matchfinder_base * const mb, const int before_size,
const int dict_size, const int after_size, const int dict_size, const int after_size,
const int dict_factor, const int num_prev_positions23, const int dict_factor, const int num_prev_positions23,
const int pos_array_factor, const int ifd ); const int pos_array_factor, const int ifd );
static inline void Mb_free( struct Matchfinder_base * const mb ) static inline void Mb_free( Matchfinder_base * const mb )
{ free( mb->prev_positions ); free( mb->buffer ); } { free( mb->prev_positions ); free( mb->buffer ); }
static inline uint8_t Mb_peek( const struct Matchfinder_base * const mb, static inline uint8_t Mb_peek( const Matchfinder_base * const mb,
const int distance ) const int distance )
{ return mb->buffer[mb->pos-distance]; } { return mb->buffer[mb->pos-distance]; }
static inline int Mb_available_bytes( const struct Matchfinder_base * const mb ) static inline int Mb_available_bytes( const Matchfinder_base * const mb )
{ return mb->stream_pos - mb->pos; } { return mb->stream_pos - mb->pos; }
static inline unsigned long long static inline unsigned long long
Mb_data_position( const struct Matchfinder_base * const mb ) Mb_data_position( const Matchfinder_base * const mb )
{ return mb->partial_data_pos + mb->pos; } { return mb->partial_data_pos + mb->pos; }
static inline bool Mb_data_finished( const struct Matchfinder_base * const mb ) static inline bool Mb_data_finished( const Matchfinder_base * const mb )
{ return mb->at_stream_end && mb->pos >= mb->stream_pos; } { return mb->at_stream_end && mb->pos >= mb->stream_pos; }
static inline const uint8_t * static inline const uint8_t *
Mb_ptr_to_current_pos( const struct Matchfinder_base * const mb ) Mb_ptr_to_current_pos( const Matchfinder_base * const mb )
{ return mb->buffer + mb->pos; } { return mb->buffer + mb->pos; }
static inline int Mb_true_match_len( const struct Matchfinder_base * const mb, static inline int Mb_true_match_len( const Matchfinder_base * const mb,
const int index, const int distance ) const int index, const int distance )
{ {
const uint8_t * const data = mb->buffer + mb->pos; const uint8_t * const data = mb->buffer + mb->pos;
@ -213,18 +213,18 @@ static inline int Mb_true_match_len( const struct Matchfinder_base * const mb,
return i; return i;
} }
static inline void Mb_move_pos( struct Matchfinder_base * const mb ) static inline void Mb_move_pos( Matchfinder_base * const mb )
{ {
if( ++mb->cyclic_pos > mb->dictionary_size ) mb->cyclic_pos = 0; if( ++mb->cyclic_pos > mb->dictionary_size ) mb->cyclic_pos = 0;
if( ++mb->pos >= mb->pos_limit ) Mb_normalize_pos( mb ); if( ++mb->pos >= mb->pos_limit ) Mb_normalize_pos( mb );
} }
void Mb_reset( struct Matchfinder_base * const mb ); void Mb_reset( Matchfinder_base * const mb );
enum { re_buffer_size = 65536 }; enum { re_buffer_size = 65536 };
struct Range_encoder typedef struct Range_encoder
{ {
uint64_t low; uint64_t low;
unsigned long long partial_member_pos; unsigned long long partial_member_pos;
@ -235,22 +235,21 @@ struct Range_encoder
int outfd; /* output file descriptor */ int outfd; /* output file descriptor */
uint8_t cache; uint8_t cache;
Lzip_header header; Lzip_header header;
}; } Range_encoder;
void Re_flush_data( struct Range_encoder * const renc ); void Re_flush_data( Range_encoder * const renc );
static inline void Re_put_byte( struct Range_encoder * const renc, static inline void Re_put_byte( Range_encoder * const renc, const uint8_t b )
const uint8_t b )
{ {
renc->buffer[renc->pos] = b; renc->buffer[renc->pos] = b;
if( ++renc->pos >= re_buffer_size ) Re_flush_data( renc ); if( ++renc->pos >= re_buffer_size ) Re_flush_data( renc );
} }
static inline void Re_shift_low( struct Range_encoder * const renc ) static inline void Re_shift_low( Range_encoder * const renc )
{ {
if( renc->low >> 24 != 0xFF ) if( renc->low >> 24 != 0xFF )
{ {
const bool carry = ( renc->low > 0xFFFFFFFFU ); const bool carry = renc->low > 0xFFFFFFFFU;
Re_put_byte( renc, renc->cache + carry ); Re_put_byte( renc, renc->cache + carry );
for( ; renc->ff_count > 0; --renc->ff_count ) for( ; renc->ff_count > 0; --renc->ff_count )
Re_put_byte( renc, 0xFF + carry ); Re_put_byte( renc, 0xFF + carry );
@ -260,7 +259,7 @@ static inline void Re_shift_low( struct Range_encoder * const renc )
renc->low = ( renc->low & 0x00FFFFFFU ) << 8; renc->low = ( renc->low & 0x00FFFFFFU ) << 8;
} }
static inline void Re_reset( struct Range_encoder * const renc, static inline void Re_reset( Range_encoder * const renc,
const unsigned dictionary_size ) const unsigned dictionary_size )
{ {
renc->low = 0; renc->low = 0;
@ -273,7 +272,7 @@ static inline void Re_reset( struct Range_encoder * const renc,
int i; for( i = 0; i < Lh_size; ++i ) Re_put_byte( renc, renc->header[i] ); int i; for( i = 0; i < Lh_size; ++i ) Re_put_byte( renc, renc->header[i] );
} }
static inline bool Re_init( struct Range_encoder * const renc, static inline bool Re_init( Range_encoder * const renc,
const unsigned dictionary_size, const int ofd ) const unsigned dictionary_size, const int ofd )
{ {
renc->buffer = (uint8_t *)malloc( re_buffer_size ); renc->buffer = (uint8_t *)malloc( re_buffer_size );
@ -284,17 +283,17 @@ static inline bool Re_init( struct Range_encoder * const renc,
return true; return true;
} }
static inline void Re_free( struct Range_encoder * const renc ) static inline void Re_free( Range_encoder * const renc )
{ free( renc->buffer ); } { free( renc->buffer ); }
static inline unsigned long long static inline unsigned long long
Re_member_position( const struct Range_encoder * const renc ) Re_member_position( const Range_encoder * const renc )
{ return renc->partial_member_pos + renc->pos + renc->ff_count; } { return renc->partial_member_pos + renc->pos + renc->ff_count; }
static inline void Re_flush( struct Range_encoder * const renc ) static inline void Re_flush( Range_encoder * const renc )
{ int i; for( i = 0; i < 5; ++i ) Re_shift_low( renc ); } { int i; for( i = 0; i < 5; ++i ) Re_shift_low( renc ); }
static inline void Re_encode( struct Range_encoder * const renc, static inline void Re_encode( Range_encoder * const renc,
const int symbol, const int num_bits ) const int symbol, const int num_bits )
{ {
unsigned mask; unsigned mask;
@ -306,7 +305,7 @@ static inline void Re_encode( struct Range_encoder * const renc,
} }
} }
static inline void Re_encode_bit( struct Range_encoder * const renc, static inline void Re_encode_bit( Range_encoder * const renc,
Bit_model * const probability, const bool bit ) Bit_model * const probability, const bool bit )
{ {
const uint32_t bound = ( renc->range >> bit_model_total_bits ) * *probability; const uint32_t bound = ( renc->range >> bit_model_total_bits ) * *probability;
@ -324,7 +323,7 @@ static inline void Re_encode_bit( struct Range_encoder * const renc,
if( renc->range <= 0x00FFFFFFU ) { renc->range <<= 8; Re_shift_low( renc ); } if( renc->range <= 0x00FFFFFFU ) { renc->range <<= 8; Re_shift_low( renc ); }
} }
static inline void Re_encode_tree3( struct Range_encoder * const renc, static inline void Re_encode_tree3( Range_encoder * const renc,
Bit_model bm[], const int symbol ) Bit_model bm[], const int symbol )
{ {
bool bit = ( symbol >> 2 ) & 1; bool bit = ( symbol >> 2 ) & 1;
@ -335,7 +334,7 @@ static inline void Re_encode_tree3( struct Range_encoder * const renc,
Re_encode_bit( renc, &bm[model], symbol & 1 ); Re_encode_bit( renc, &bm[model], symbol & 1 );
} }
static inline void Re_encode_tree6( struct Range_encoder * const renc, static inline void Re_encode_tree6( Range_encoder * const renc,
Bit_model bm[], const unsigned symbol ) Bit_model bm[], const unsigned symbol )
{ {
bool bit = ( symbol >> 5 ) & 1; bool bit = ( symbol >> 5 ) & 1;
@ -352,7 +351,7 @@ static inline void Re_encode_tree6( struct Range_encoder * const renc,
Re_encode_bit( renc, &bm[model], symbol & 1 ); Re_encode_bit( renc, &bm[model], symbol & 1 );
} }
static inline void Re_encode_tree8( struct Range_encoder * const renc, static inline void Re_encode_tree8( Range_encoder * const renc,
Bit_model bm[], const int symbol ) Bit_model bm[], const int symbol )
{ {
int model = 1; int model = 1;
@ -365,7 +364,7 @@ static inline void Re_encode_tree8( struct Range_encoder * const renc,
} }
} }
static inline void Re_encode_tree_reversed( struct Range_encoder * const renc, static inline void Re_encode_tree_reversed( Range_encoder * const renc,
Bit_model bm[], int symbol, const int num_bits ) Bit_model bm[], int symbol, const int num_bits )
{ {
int model = 1; int model = 1;
@ -379,7 +378,7 @@ static inline void Re_encode_tree_reversed( struct Range_encoder * const renc,
} }
} }
static inline void Re_encode_matched( struct Range_encoder * const renc, static inline void Re_encode_matched( Range_encoder * const renc,
Bit_model bm[], unsigned symbol, Bit_model bm[], unsigned symbol,
unsigned match_byte ) unsigned match_byte )
{ {
@ -395,17 +394,17 @@ static inline void Re_encode_matched( struct Range_encoder * const renc,
} }
} }
static inline void Re_encode_len( struct Range_encoder * const renc, static inline void Re_encode_len( Range_encoder * const renc,
struct Len_model * const lm, Len_model * const lm,
int symbol, const int pos_state ) int symbol, const int pos_state )
{ {
bool bit = ( ( symbol -= min_match_len ) >= len_low_symbols ); bool bit = ( symbol -= min_match_len ) >= len_low_symbols;
Re_encode_bit( renc, &lm->choice1, bit ); Re_encode_bit( renc, &lm->choice1, bit );
if( !bit ) if( !bit )
Re_encode_tree3( renc, lm->bm_low[pos_state], symbol ); Re_encode_tree3( renc, lm->bm_low[pos_state], symbol );
else else
{ {
bit = ( ( symbol -= len_low_symbols ) >= len_mid_symbols ); bit = ( symbol -= len_low_symbols ) >= len_mid_symbols;
Re_encode_bit( renc, &lm->choice2, bit ); Re_encode_bit( renc, &lm->choice2, bit );
if( !bit ) if( !bit )
Re_encode_tree3( renc, lm->bm_mid[pos_state], symbol ); Re_encode_tree3( renc, lm->bm_mid[pos_state], symbol );
@ -418,9 +417,9 @@ static inline void Re_encode_len( struct Range_encoder * const renc,
enum { max_marker_size = 16, enum { max_marker_size = 16,
num_rep_distances = 4 }; /* must be 4 */ num_rep_distances = 4 }; /* must be 4 */
struct LZ_encoder_base typedef struct LZ_encoder_base
{ {
struct Matchfinder_base mb; Matchfinder_base mb;
uint32_t crc; uint32_t crc;
Bit_model bm_literal[1<<literal_context_bits][0x300]; Bit_model bm_literal[1<<literal_context_bits][0x300];
@ -433,14 +432,14 @@ struct LZ_encoder_base
Bit_model bm_dis_slot[len_states][1<<dis_slot_bits]; Bit_model bm_dis_slot[len_states][1<<dis_slot_bits];
Bit_model bm_dis[modeled_distances-end_dis_model+1]; Bit_model bm_dis[modeled_distances-end_dis_model+1];
Bit_model bm_align[dis_align_size]; Bit_model bm_align[dis_align_size];
struct Len_model match_len_model; Len_model match_len_model;
struct Len_model rep_len_model; Len_model rep_len_model;
struct Range_encoder renc; Range_encoder renc;
}; } LZ_encoder_base;
void LZeb_reset( struct LZ_encoder_base * const eb ); void LZeb_reset( LZ_encoder_base * const eb );
static inline bool LZeb_init( struct LZ_encoder_base * const eb, static inline bool LZeb_init( LZ_encoder_base * const eb,
const int before_size, const int dict_size, const int before_size, const int dict_size,
const int after_size, const int dict_factor, const int after_size, const int dict_factor,
const int num_prev_positions23, const int num_prev_positions23,
@ -454,31 +453,31 @@ static inline bool LZeb_init( struct LZ_encoder_base * const eb,
return true; return true;
} }
static inline void LZeb_free( struct LZ_encoder_base * const eb ) static inline void LZeb_free( LZ_encoder_base * const eb )
{ Re_free( &eb->renc ); Mb_free( &eb->mb ); } { Re_free( &eb->renc ); Mb_free( &eb->mb ); }
static inline unsigned LZeb_crc( const struct LZ_encoder_base * const eb ) static inline unsigned LZeb_crc( const LZ_encoder_base * const eb )
{ return eb->crc ^ 0xFFFFFFFFU; } { return eb->crc ^ 0xFFFFFFFFU; }
static inline int LZeb_price_literal( const struct LZ_encoder_base * const eb, static inline int LZeb_price_literal( const LZ_encoder_base * const eb,
const uint8_t prev_byte, const uint8_t symbol ) const uint8_t prev_byte, const uint8_t symbol )
{ return price_symbol8( eb->bm_literal[get_lit_state(prev_byte)], symbol ); } { return price_symbol8( eb->bm_literal[get_lit_state(prev_byte)], symbol ); }
static inline int LZeb_price_matched( const struct LZ_encoder_base * const eb, static inline int LZeb_price_matched( const LZ_encoder_base * const eb,
const uint8_t prev_byte, const uint8_t symbol, const uint8_t match_byte ) const uint8_t prev_byte, const uint8_t symbol, const uint8_t match_byte )
{ return price_matched( eb->bm_literal[get_lit_state(prev_byte)], symbol, { return price_matched( eb->bm_literal[get_lit_state(prev_byte)], symbol,
match_byte ); } match_byte ); }
static inline void LZeb_encode_literal( struct LZ_encoder_base * const eb, static inline void LZeb_encode_literal( LZ_encoder_base * const eb,
const uint8_t prev_byte, const uint8_t symbol ) const uint8_t prev_byte, const uint8_t symbol )
{ Re_encode_tree8( &eb->renc, eb->bm_literal[get_lit_state(prev_byte)], symbol ); } { Re_encode_tree8( &eb->renc, eb->bm_literal[get_lit_state(prev_byte)], symbol ); }
static inline void LZeb_encode_matched( struct LZ_encoder_base * const eb, static inline void LZeb_encode_matched( LZ_encoder_base * const eb,
const uint8_t prev_byte, const uint8_t symbol, const uint8_t match_byte ) const uint8_t prev_byte, const uint8_t symbol, const uint8_t match_byte )
{ Re_encode_matched( &eb->renc, eb->bm_literal[get_lit_state(prev_byte)], { Re_encode_matched( &eb->renc, eb->bm_literal[get_lit_state(prev_byte)],
symbol, match_byte ); } symbol, match_byte ); }
static inline void LZeb_encode_pair( struct LZ_encoder_base * const eb, static inline void LZeb_encode_pair( LZ_encoder_base * const eb,
const unsigned dis, const int len, const unsigned dis, const int len,
const int pos_state ) const int pos_state )
{ {
@ -504,4 +503,4 @@ static inline void LZeb_encode_pair( struct LZ_encoder_base * const eb,
} }
} }
void LZeb_full_flush( struct LZ_encoder_base * const eb, const State state ); void LZeb_full_flush( LZ_encoder_base * const eb, const State state );

4
fast_encoder.c
View file

@ -28,7 +28,7 @@
#include "fast_encoder.h" #include "fast_encoder.h"
int FLZe_longest_match_len( struct FLZ_encoder * const fe, int * const distance ) int FLZe_longest_match_len( FLZ_encoder * const fe, int * const distance )
{ {
enum { len_limit = 16 }; enum { len_limit = 16 };
const int available = min( Mb_available_bytes( &fe->eb.mb ), max_match_len ); const int available = min( Mb_available_bytes( &fe->eb.mb ), max_match_len );
@ -69,7 +69,7 @@ int FLZe_longest_match_len( struct FLZ_encoder * const fe, int * const distance
} }
bool FLZe_encode_member( struct FLZ_encoder * const fe, bool FLZe_encode_member( FLZ_encoder * const fe,
const unsigned long long member_size ) const unsigned long long member_size )
{ {
const unsigned long long member_size_limit = const unsigned long long member_size_limit =

20
fast_encoder.h
View file

@ -15,13 +15,13 @@
along with this program. If not, see <http://www.gnu.org/licenses/>. along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
struct FLZ_encoder typedef struct FLZ_encoder
{ {
struct LZ_encoder_base eb; LZ_encoder_base eb;
unsigned key4; /* key made from latest 4 bytes */ unsigned key4; /* key made from latest 4 bytes */
}; } FLZ_encoder;
static inline void FLZe_reset_key4( struct FLZ_encoder * const fe ) static inline void FLZe_reset_key4( FLZ_encoder * const fe )
{ {
int i; int i;
fe->key4 = 0; fe->key4 = 0;
@ -29,11 +29,11 @@ static inline void FLZe_reset_key4( struct FLZ_encoder * const fe )
fe->key4 = ( fe->key4 << 4 ) ^ fe->eb.mb.buffer[i]; fe->key4 = ( fe->key4 << 4 ) ^ fe->eb.mb.buffer[i];
} }
int FLZe_longest_match_len( struct FLZ_encoder * const fe, int * const distance ); int FLZe_longest_match_len( FLZ_encoder * const fe, int * const distance );
static inline void FLZe_update_and_move( struct FLZ_encoder * const fe, int n ) static inline void FLZe_update_and_move( FLZ_encoder * const fe, int n )
{ {
struct Matchfinder_base * const mb = &fe->eb.mb; Matchfinder_base * const mb = &fe->eb.mb;
while( --n >= 0 ) while( --n >= 0 )
{ {
if( Mb_available_bytes( mb ) >= 4 ) if( Mb_available_bytes( mb ) >= 4 )
@ -46,7 +46,7 @@ static inline void FLZe_update_and_move( struct FLZ_encoder * const fe, int n )
} }
} }
static inline bool FLZe_init( struct FLZ_encoder * const fe, static inline bool FLZe_init( FLZ_encoder * const fe,
const int ifd, const int outfd ) const int ifd, const int outfd )
{ {
enum { before_size = 0, enum { before_size = 0,
@ -61,8 +61,8 @@ static inline bool FLZe_init( struct FLZ_encoder * const fe,
num_prev_positions23, pos_array_factor, ifd, outfd ); num_prev_positions23, pos_array_factor, ifd, outfd );
} }
static inline void FLZe_reset( struct FLZ_encoder * const fe ) static inline void FLZe_reset( FLZ_encoder * const fe )
{ LZeb_reset( &fe->eb ); } { LZeb_reset( &fe->eb ); }
bool FLZe_encode_member( struct FLZ_encoder * const fe, bool FLZe_encode_member( FLZ_encoder * const fe,
const unsigned long long member_size ); const unsigned long long member_size );

21
list.c
View file

@ -17,6 +17,7 @@
#define _FILE_OFFSET_BITS 64 #define _FILE_OFFSET_BITS 64
#include <errno.h>
#include <stdbool.h> #include <stdbool.h>
#include <stdint.h> #include <stdint.h>
#include <stdio.h> #include <stdio.h>
@ -43,7 +44,7 @@ static void list_line( const unsigned long long uncomp_size,
int list_files( const char * const filenames[], const int num_filenames, int list_files( const char * const filenames[], const int num_filenames,
const struct Cl_options * const cl_opts ) const Cl_options * const cl_opts )
{ {
unsigned long long total_comp = 0, total_uncomp = 0; unsigned long long total_comp = 0, total_uncomp = 0;
int files = 0, retval = 0; int files = 0, retval = 0;
@ -53,7 +54,7 @@ int list_files( const char * const filenames[], const int num_filenames,
for( i = 0; i < num_filenames; ++i ) for( i = 0; i < num_filenames; ++i )
{ {
const bool from_stdin = ( strcmp( filenames[i], "-" ) == 0 ); const bool from_stdin = strcmp( filenames[i], "-" ) == 0;
if( from_stdin ) { if( stdin_used ) continue; else stdin_used = true; } if( from_stdin ) { if( stdin_used ) continue; else stdin_used = true; }
const char * const input_filename = from_stdin ? "(stdin)" : filenames[i]; const char * const input_filename = from_stdin ? "(stdin)" : filenames[i];
struct stat in_stats; /* not used */ struct stat in_stats; /* not used */
@ -61,7 +62,7 @@ int list_files( const char * const filenames[], const int num_filenames,
open_instream( input_filename, &in_stats, false, true ); open_instream( input_filename, &in_stats, false, true );
if( infd < 0 ) { set_retval( &retval, 1 ); continue; } if( infd < 0 ) { set_retval( &retval, 1 ); continue; }
struct Lzip_index lzip_index; Lzip_index lzip_index;
Li_init( &lzip_index, infd, cl_opts ); Li_init( &lzip_index, infd, cl_opts );
close( infd ); close( infd );
if( lzip_index.retval != 0 ) if( lzip_index.retval != 0 )
@ -70,6 +71,8 @@ int list_files( const char * const filenames[], const int num_filenames,
set_retval( &retval, lzip_index.retval ); set_retval( &retval, lzip_index.retval );
Li_free( &lzip_index ); continue; Li_free( &lzip_index ); continue;
} }
const bool multi_empty = !from_stdin && Li_multi_empty( &lzip_index );
if( multi_empty ) set_retval( &retval, 2 );
if( verbosity < 0 ) { Li_free( &lzip_index ); continue; } if( verbosity < 0 ) { Li_free( &lzip_index ); continue; }
const unsigned long long udata_size = Li_udata_size( &lzip_index ); const unsigned long long udata_size = Li_udata_size( &lzip_index );
const unsigned long long cdata_size = Li_cdata_size( &lzip_index ); const unsigned long long cdata_size = Li_cdata_size( &lzip_index );
@ -81,6 +84,8 @@ int list_files( const char * const filenames[], const int num_filenames,
if( verbosity >= 1 ) fputs( " dict memb trail ", stdout ); if( verbosity >= 1 ) fputs( " dict memb trail ", stdout );
fputs( " uncompressed compressed saved name\n", stdout ); fputs( " uncompressed compressed saved name\n", stdout );
} }
if( multi_empty )
{ fflush( stdout ); show_file_error( input_filename, empty_msg, 0 ); }
if( verbosity >= 1 ) if( verbosity >= 1 )
printf( "%s %5ld %6lld ", format_ds( lzip_index.dictionary_size ), printf( "%s %5ld %6lld ", format_ds( lzip_index.dictionary_size ),
members, Li_file_size( &lzip_index ) - cdata_size ); members, Li_file_size( &lzip_index ) - cdata_size );
@ -92,8 +97,8 @@ int list_files( const char * const filenames[], const int num_filenames,
fputs( " member data_pos data_size member_pos member_size\n", stdout ); fputs( " member data_pos data_size member_pos member_size\n", stdout );
for( i = 0; i < members; ++i ) for( i = 0; i < members; ++i )
{ {
const struct Block * db = Li_dblock( &lzip_index, i ); const Block * db = Li_dblock( &lzip_index, i );
const struct Block * mb = Li_mblock( &lzip_index, i ); const Block * mb = Li_mblock( &lzip_index, i );
printf( "%6ld %14llu %14llu %14llu %14llu\n", printf( "%6ld %14llu %14llu %14llu %14llu\n",
i + 1, db->pos, db->size, mb->pos, mb->size ); i + 1, db->pos, db->size, mb->pos, mb->size );
} }
@ -101,12 +106,16 @@ int list_files( const char * const filenames[], const int num_filenames,
} }
fflush( stdout ); fflush( stdout );
Li_free( &lzip_index ); Li_free( &lzip_index );
if( ferror( stdout ) ) break;
} }
if( verbosity >= 0 && files > 1 ) if( verbosity >= 0 && files > 1 && !ferror( stdout ) )
{ {
if( verbosity >= 1 ) fputs( " ", stdout ); if( verbosity >= 1 ) fputs( " ", stdout );
list_line( total_uncomp, total_comp, "(totals)" ); list_line( total_uncomp, total_comp, "(totals)" );
fflush( stdout ); fflush( stdout );
} }
if( verbosity >= 0 && ( ferror( stdout ) || fclose( stdout ) != 0 ) )
{ show_file_error( "(stdout)", write_error_msg, errno );
set_retval( &retval, 1 ); }
return retval; return retval;
} }

40
lzip.h
View file

@ -92,16 +92,16 @@ static inline void Bm_init( Bit_model * const probability )
static inline void Bm_array_init( Bit_model bm[], const int size ) static inline void Bm_array_init( Bit_model bm[], const int size )
{ int i; for( i = 0; i < size; ++i ) Bm_init( &bm[i] ); } { int i; for( i = 0; i < size; ++i ) Bm_init( &bm[i] ); }
struct Len_model typedef struct Len_model
{ {
Bit_model choice1; Bit_model choice1;
Bit_model choice2; Bit_model choice2;
Bit_model bm_low[pos_states][len_low_symbols]; Bit_model bm_low[pos_states][len_low_symbols];
Bit_model bm_mid[pos_states][len_mid_symbols]; Bit_model bm_mid[pos_states][len_mid_symbols];
Bit_model bm_high[len_high_symbols]; Bit_model bm_high[len_high_symbols];
}; } Len_model;
static inline void Lm_init( struct Len_model * const lm ) static inline void Lm_init( Len_model * const lm )
{ {
Bm_init( &lm->choice1 ); Bm_init( &lm->choice1 );
Bm_init( &lm->choice2 ); Bm_init( &lm->choice2 );
@ -276,17 +276,14 @@ static inline bool Lt_check_consistency( const Lzip_trailer data )
} }
struct Cl_options /* command-line options */ typedef struct Cl_options /* command-line options */
{ {
bool ignore_empty;
bool ignore_marking;
bool ignore_trailing; bool ignore_trailing;
bool loose_trailing; bool loose_trailing;
}; } Cl_options;
static inline void Cl_options_init( struct Cl_options * cl_opts ) static inline void Cl_options_init( Cl_options * cl_opts )
{ cl_opts->ignore_empty = true; cl_opts->ignore_marking = true; { cl_opts->ignore_trailing = true; cl_opts->loose_trailing = false; }
cl_opts->ignore_trailing = true; cl_opts->loose_trailing = false; }
static inline void set_retval( int * retval, const int new_val ) static inline void set_retval( int * retval, const int new_val )
@ -296,9 +293,10 @@ static const char * const bad_magic_msg = "Bad magic number (file not in lzip fo
static const char * const bad_dict_msg = "Invalid dictionary size in member header."; static const char * const bad_dict_msg = "Invalid dictionary size in member header.";
static const char * const corrupt_mm_msg = "Corrupt header in multimember file."; static const char * const corrupt_mm_msg = "Corrupt header in multimember file.";
static const char * const empty_msg = "Empty member not allowed."; static const char * const empty_msg = "Empty member not allowed.";
static const char * const marking_msg = "Marking data not allowed.";
static const char * const trailing_msg = "Trailing data not allowed.";
static const char * const mem_msg = "Not enough memory."; static const char * const mem_msg = "Not enough memory.";
static const char * const nonzero_msg = "Nonzero first LZMA byte.";
static const char * const trailing_msg = "Trailing data not allowed.";
static const char * const write_error_msg = "Write error";
/* defined in decoder.c */ /* defined in decoder.c */
int readblock( const int fd, uint8_t * const buf, const int size ); int readblock( const int fd, uint8_t * const buf, const int size );
@ -306,14 +304,16 @@ int writeblock( const int fd, const uint8_t * const buf, const int size );
/* defined in list.c */ /* defined in list.c */
int list_files( const char * const filenames[], const int num_filenames, int list_files( const char * const filenames[], const int num_filenames,
const struct Cl_options * const cl_opts ); const Cl_options * const cl_opts );
/* defined in main.c */ /* defined in main.c */
struct stat; struct stat;
struct Pretty_print; typedef struct Matchfinder_base Matchfinder_base;
typedef struct Pretty_print Pretty_print;
typedef struct Range_decoder Range_decoder;
extern int verbosity; extern int verbosity;
void * resize_buffer( void * buf, const unsigned min_size ); void * resize_buffer( void * buf, const unsigned min_size );
void Pp_show_msg( struct Pretty_print * const pp, const char * const msg ); void Pp_show_msg( Pretty_print * const pp, const char * const msg );
const char * bad_version( const unsigned version ); const char * bad_version( const unsigned version );
const char * format_ds( const unsigned dictionary_size ); const char * format_ds( const unsigned dictionary_size );
void show_header( const unsigned dictionary_size ); void show_header( const unsigned dictionary_size );
@ -324,13 +324,11 @@ void show_error( const char * const msg, const int errcode, const bool help );
void show_file_error( const char * const filename, const char * const msg, void show_file_error( const char * const filename, const char * const msg,
const int errcode ); const int errcode );
void internal_error( const char * const msg ); void internal_error( const char * const msg );
struct Matchfinder_base;
void show_cprogress( const unsigned long long cfile_size, void show_cprogress( const unsigned long long cfile_size,
const unsigned long long partial_size, const unsigned long long partial_size,
const struct Matchfinder_base * const m, const Matchfinder_base * const m,
struct Pretty_print * const p ); Pretty_print * const p );
struct Range_decoder;
void show_dprogress( const unsigned long long cfile_size, void show_dprogress( const unsigned long long cfile_size,
const unsigned long long partial_size, const unsigned long long partial_size,
const struct Range_decoder * const d, const Range_decoder * const d,
struct Pretty_print * const p ); Pretty_print * const p );

78
lzip_index.c
View file

@ -38,7 +38,7 @@ static int seek_read( const int fd, uint8_t * const buf, const int size,
} }
static bool add_error( struct Lzip_index * const li, const char * const msg ) static bool add_error( Lzip_index * const li, const char * const msg )
{ {
const int len = strlen( msg ); const int len = strlen( msg );
void * tmp = resize_buffer( li->error, li->error_size + len + 1 ); void * tmp = resize_buffer( li->error, li->error_size + len + 1 );
@ -50,16 +50,15 @@ static bool add_error( struct Lzip_index * const li, const char * const msg )
} }
static bool push_back_member( struct Lzip_index * const li, static bool push_back_member( Lzip_index * const li, const long long dp,
const long long dp, const long long ds, const long long ds, const long long mp,
const long long mp, const long long ms, const long long ms, const unsigned dict_size )
const unsigned dict_size )
{ {
struct Member * p; Member * p;
void * tmp = resize_buffer( li->member_vector, void * tmp = resize_buffer( li->member_vector,
( li->members + 1 ) * sizeof li->member_vector[0] ); ( li->members + 1 ) * sizeof li->member_vector[0] );
if( !tmp ) { add_error( li, mem_msg ); li->retval = 1; return false; } if( !tmp ) { add_error( li, mem_msg ); li->retval = 1; return false; }
li->member_vector = (struct Member *)tmp; li->member_vector = (Member *)tmp;
p = &(li->member_vector[li->members]); p = &(li->member_vector[li->members]);
init_member( p, dp, ds, mp, ms, dict_size ); init_member( p, dp, ds, mp, ms, dict_size );
++li->members; ++li->members;
@ -67,7 +66,7 @@ static bool push_back_member( struct Lzip_index * const li,
} }
static void Li_free_member_vector( struct Lzip_index * const li ) static void Li_free_member_vector( Lzip_index * const li )
{ {
if( li->member_vector ) if( li->member_vector )
{ free( li->member_vector ); li->member_vector = 0; } { free( li->member_vector ); li->member_vector = 0; }
@ -75,9 +74,9 @@ static void Li_free_member_vector( struct Lzip_index * const li )
} }
static void Li_reverse_member_vector( struct Lzip_index * const li ) static void Li_reverse_member_vector( Lzip_index * const li )
{ {
struct Member tmp; Member tmp;
long i; long i;
for( i = 0; i < li->members / 2; ++i ) for( i = 0; i < li->members / 2; ++i )
{ {
@ -88,8 +87,7 @@ static void Li_reverse_member_vector( struct Lzip_index * const li )
} }
static bool Li_check_header( struct Lzip_index * const li, static bool Li_check_header( Lzip_index * const li, const Lzip_header header )
const Lzip_header header )
{ {
if( !Lh_check_magic( header ) ) if( !Lh_check_magic( header ) )
{ add_error( li, bad_magic_msg ); li->retval = 2; return false; } { add_error( li, bad_magic_msg ); li->retval = 2; return false; }
@ -101,15 +99,14 @@ static bool Li_check_header( struct Lzip_index * const li,
return true; return true;
} }
static void Li_set_errno_error( struct Lzip_index * const li, static void Li_set_errno_error( Lzip_index * const li, const char * const msg )
const char * const msg )
{ {
add_error( li, msg ); add_error( li, strerror( errno ) ); add_error( li, msg ); add_error( li, strerror( errno ) );
li->retval = 1; li->retval = 1;
} }
static void Li_set_num_error( struct Lzip_index * const li, static void Li_set_num_error( Lzip_index * const li, const char * const msg,
const char * const msg, unsigned long long num ) unsigned long long num )
{ {
char buf[80]; char buf[80];
snprintf( buf, sizeof buf, "%s%llu", msg, num ); snprintf( buf, sizeof buf, "%s%llu", msg, num );
@ -118,22 +115,19 @@ static void Li_set_num_error( struct Lzip_index * const li,
} }
static bool Li_read_header( struct Lzip_index * const li, const int fd, static bool Li_read_header( Lzip_index * const li, const int fd,
Lzip_header header, const long long pos, const bool ignore_marking ) Lzip_header header, const long long pos )
{ {
if( seek_read( fd, header, Lh_size, pos ) != Lh_size ) if( seek_read( fd, header, Lh_size, pos ) != Lh_size )
{ Li_set_errno_error( li, "Error reading member header: " ); return false; } { Li_set_errno_error( li, "Error reading member header: " ); return false; }
uint8_t byte;
if( !ignore_marking && readblock( fd, &byte, 1 ) == 1 && byte != 0 )
{ add_error( li, marking_msg ); li->retval = 2; return false; }
return true; return true;
} }
/* If successful, push last member and set pos to member header. */ /* If successful, push last member and set pos to member header. */
static bool Li_skip_trailing_data( struct Lzip_index * const li, const int fd, static bool Li_skip_trailing_data( Lzip_index * const li, const int fd,
unsigned long long * const pos, unsigned long long * const pos,
const struct Cl_options * const cl_opts ) const Cl_options * const cl_opts )
{ {
if( *pos < min_member_size ) return false; if( *pos < min_member_size ) return false;
enum { block_size = 16384, enum { block_size = 16384,
@ -162,8 +156,8 @@ static bool Li_skip_trailing_data( struct Lzip_index * const li, const int fd,
if( member_size > ipos + i || !Lt_check_consistency( *trailer ) ) if( member_size > ipos + i || !Lt_check_consistency( *trailer ) )
continue; continue;
Lzip_header header; Lzip_header header;
if( !Li_read_header( li, fd, header, ipos + i - member_size, if( !Li_read_header( li, fd, header, ipos + i - member_size ) )
cl_opts->ignore_marking ) ) return false; return false;
if( !Lh_check( header ) ) continue; if( !Lh_check( header ) ) continue;
const Lzip_header * header2 = (const Lzip_header *)( buffer + i ); const Lzip_header * header2 = (const Lzip_header *)( buffer + i );
const bool full_h2 = bsize - i >= Lh_size; const bool full_h2 = bsize - i >= Lh_size;
@ -178,15 +172,12 @@ static bool Li_skip_trailing_data( struct Lzip_index * const li, const int fd,
{ add_error( li, corrupt_mm_msg ); li->retval = 2; return false; } { add_error( li, corrupt_mm_msg ); li->retval = 2; return false; }
if( !cl_opts->ignore_trailing ) if( !cl_opts->ignore_trailing )
{ add_error( li, trailing_msg ); li->retval = 2; return false; } { add_error( li, trailing_msg ); li->retval = 2; return false; }
const unsigned long long data_size = Lt_get_data_size( *trailer );
if( !cl_opts->ignore_empty && data_size == 0 )
{ add_error( li, empty_msg ); li->retval = 2; return false; }
*pos = ipos + i - member_size; /* good member */ *pos = ipos + i - member_size; /* good member */
const unsigned dictionary_size = Lh_get_dictionary_size( header ); const unsigned dictionary_size = Lh_get_dictionary_size( header );
if( li->dictionary_size < dictionary_size ) if( li->dictionary_size < dictionary_size )
li->dictionary_size = dictionary_size; li->dictionary_size = dictionary_size;
return push_back_member( li, 0, data_size, *pos, member_size, return push_back_member( li, 0, Lt_get_data_size( *trailer ), *pos,
dictionary_size ); member_size, dictionary_size );
} }
if( ipos == 0 ) if( ipos == 0 )
{ Li_set_num_error( li, "Bad trailer at pos ", *pos - Lt_size ); { Li_set_num_error( li, "Bad trailer at pos ", *pos - Lt_size );
@ -200,8 +191,8 @@ static bool Li_skip_trailing_data( struct Lzip_index * const li, const int fd,
} }
bool Li_init( struct Lzip_index * const li, const int infd, bool Li_init( Lzip_index * const li, const int infd,
const struct Cl_options * const cl_opts ) const Cl_options * const cl_opts )
{ {
li->member_vector = 0; li->member_vector = 0;
li->error = 0; li->error = 0;
@ -212,6 +203,10 @@ bool Li_init( struct Lzip_index * const li, const int infd,
li->dictionary_size = 0; li->dictionary_size = 0;
if( li->insize < 0 ) if( li->insize < 0 )
{ Li_set_errno_error( li, "Input file is not seekable: " ); return false; } { Li_set_errno_error( li, "Input file is not seekable: " ); return false; }
Lzip_header header;
if( li->insize >= Lh_size &&
( !Li_read_header( li, infd, header, 0 ) ||
!Li_check_header( li, header ) ) ) return false;
if( li->insize < min_member_size ) if( li->insize < min_member_size )
{ add_error( li, "Input file is too short." ); li->retval = 2; { add_error( li, "Input file is too short." ); li->retval = 2;
return false; } return false; }
@ -219,10 +214,6 @@ bool Li_init( struct Lzip_index * const li, const int infd,
{ add_error( li, "Input file is too long (2^63 bytes or more)." ); { add_error( li, "Input file is too long (2^63 bytes or more)." );
li->retval = 2; return false; } li->retval = 2; return false; }
Lzip_header header;
if( !Li_read_header( li, infd, header, 0, cl_opts->ignore_marking ) ||
!Li_check_header( li, header ) ) return false;
unsigned long long pos = li->insize; /* always points to a header or to EOF */ unsigned long long pos = li->insize; /* always points to a header or to EOF */
while( pos >= min_member_size ) while( pos >= min_member_size )
{ {
@ -237,8 +228,7 @@ bool Li_init( struct Lzip_index * const li, const int infd,
return false; } return false; }
Li_set_num_error( li, "Bad trailer at pos ", pos - Lt_size ); break; Li_set_num_error( li, "Bad trailer at pos ", pos - Lt_size ); break;
} }
if( !Li_read_header( li, infd, header, pos - member_size, if( !Li_read_header( li, infd, header, pos - member_size ) ) break;
cl_opts->ignore_marking ) ) break;
if( !Lh_check( header ) ) /* bad header */ if( !Lh_check( header ) ) /* bad header */
{ {
if( li->members <= 0 ) if( li->members <= 0 )
@ -246,15 +236,12 @@ bool Li_init( struct Lzip_index * const li, const int infd,
return false; } return false; }
Li_set_num_error( li, "Bad header at pos ", pos - member_size ); break; Li_set_num_error( li, "Bad header at pos ", pos - member_size ); break;
} }
const unsigned long long data_size = Lt_get_data_size( trailer );
if( !cl_opts->ignore_empty && data_size == 0 )
{ add_error( li, empty_msg ); li->retval = 2; break; }
pos -= member_size; /* good member */ pos -= member_size; /* good member */
const unsigned dictionary_size = Lh_get_dictionary_size( header ); const unsigned dictionary_size = Lh_get_dictionary_size( header );
if( li->dictionary_size < dictionary_size ) if( li->dictionary_size < dictionary_size )
li->dictionary_size = dictionary_size; li->dictionary_size = dictionary_size;
if( !push_back_member( li, 0, data_size, pos, member_size, if( !push_back_member( li, 0, Lt_get_data_size( trailer ), pos,
dictionary_size ) ) return false; member_size, dictionary_size ) ) return false;
} }
if( pos != 0 || li->members <= 0 || li->retval != 0 ) if( pos != 0 || li->members <= 0 || li->retval != 0 )
{ {
@ -264,8 +251,7 @@ bool Li_init( struct Lzip_index * const li, const int infd,
return false; return false;
} }
Li_reverse_member_vector( li ); Li_reverse_member_vector( li );
long i; long i; for( i = 0; ; ++i )
for( i = 0; ; ++i )
{ {
const long long end = block_end( li->member_vector[i].dblock ); const long long end = block_end( li->member_vector[i].dblock );
if( end < 0 || end > INT64_MAX ) if( end < 0 || end > INT64_MAX )
@ -281,7 +267,7 @@ bool Li_init( struct Lzip_index * const li, const int infd,
} }
void Li_free( struct Lzip_index * const li ) void Li_free( Lzip_index * const li )
{ {
Li_free_member_vector( li ); Li_free_member_vector( li );
if( li->error ) { free( li->error ); li->error = 0; } if( li->error ) { free( li->error ); li->error = 0; }

64
lzip_index.h
View file

@ -20,72 +20,80 @@
#endif #endif
struct Block typedef struct Block
{ {
long long pos, size; /* pos >= 0, size >= 0, pos + size <= INT64_MAX */ long long pos, size; /* pos >= 0, size >= 0, pos + size <= INT64_MAX */
}; } Block;
static inline void init_block( struct Block * const b, static inline void init_block( Block * const b,
const long long p, const long long s ) const long long p, const long long s )
{ b->pos = p; b->size = s; } { b->pos = p; b->size = s; }
static inline long long block_end( const struct Block b ) static inline long long block_end( const Block b ) { return b.pos + b.size; }
{ return b.pos + b.size; }
struct Member typedef struct Member
{ {
struct Block dblock, mblock; /* data block, member block */ Block dblock, mblock; /* data block, member block */
unsigned dictionary_size; unsigned dictionary_size;
}; } Member;
static inline void init_member( struct Member * const m, static inline void init_member( Member * const m, const long long dpos,
const long long dpos, const long long dsize, const long long dsize, const long long mpos,
const long long mpos, const long long msize, const long long msize, const unsigned dict_size )
const unsigned dict_size ) { init_block( &m->dblock, dpos, dsize );
{ init_block( &m->dblock, dpos, dsize ); init_block( &m->mblock, mpos, msize ); init_block( &m->mblock, mpos, msize ); m->dictionary_size = dict_size; }
m->dictionary_size = dict_size; }
struct Lzip_index typedef struct Lzip_index
{ {
struct Member * member_vector; Member * member_vector;
char * error; char * error;
long long insize; long long insize;
long members; long members;
int error_size; int error_size;
int retval; int retval;
unsigned dictionary_size; /* largest dictionary size in the file */ unsigned dictionary_size; /* largest dictionary size in the file */
}; } Lzip_index;
bool Li_init( struct Lzip_index * const li, const int infd, bool Li_init( Lzip_index * const li, const int infd,
const struct Cl_options * const cl_opts ); const Cl_options * const cl_opts );
void Li_free( struct Lzip_index * const li ); void Li_free( Lzip_index * const li );
static inline long long Li_udata_size( const struct Lzip_index * const li ) /* multimember file with empty member(s) */
static inline bool Li_multi_empty( Lzip_index * const li )
{
long i;
if( li->members > 1 )
for( i = 0; i < li->members; ++i )
if( li->member_vector[i].dblock.size == 0 ) return true;
return false;
}
static inline long long Li_udata_size( const Lzip_index * const li )
{ {
if( li->members <= 0 ) return 0; if( li->members <= 0 ) return 0;
return block_end( li->member_vector[li->members-1].dblock ); return block_end( li->member_vector[li->members-1].dblock );
} }
static inline long long Li_cdata_size( const struct Lzip_index * const li ) static inline long long Li_cdata_size( const Lzip_index * const li )
{ {
if( li->members <= 0 ) return 0; if( li->members <= 0 ) return 0;
return block_end( li->member_vector[li->members-1].mblock ); return block_end( li->member_vector[li->members-1].mblock );
} }
/* total size including trailing data (if any) */ /* total size including trailing data (if any) */
static inline long long Li_file_size( const struct Lzip_index * const li ) static inline long long Li_file_size( const Lzip_index * const li )
{ if( li->insize >= 0 ) return li->insize; else return 0; } { if( li->insize >= 0 ) return li->insize; else return 0; }
static inline const struct Block * Li_dblock( const struct Lzip_index * const li, static inline const Block * Li_dblock( const Lzip_index * const li,
const long i ) const long i )
{ return &li->member_vector[i].dblock; } { return &li->member_vector[i].dblock; }
static inline const struct Block * Li_mblock( const struct Lzip_index * const li, static inline const Block * Li_mblock( const Lzip_index * const li,
const long i ) const long i )
{ return &li->member_vector[i].mblock; } { return &li->member_vector[i].mblock; }
static inline unsigned Li_dictionary_size( const struct Lzip_index * const li, static inline unsigned Li_dictionary_size( const Lzip_index * const li,
const long i ) const long i )
{ return li->member_vector[i].dictionary_size; } { return li->member_vector[i].dictionary_size; }

240
main.c
View file

@ -26,7 +26,7 @@
#include <ctype.h> #include <ctype.h>
#include <errno.h> #include <errno.h>
#include <fcntl.h> #include <fcntl.h>
#include <limits.h> /* SSIZE_MAX */ #include <limits.h> /* CHAR_BIT, SSIZE_MAX */
#include <signal.h> #include <signal.h>
#include <stdbool.h> #include <stdbool.h>
#include <stdint.h> /* SIZE_MAX */ #include <stdint.h> /* SIZE_MAX */
@ -39,8 +39,10 @@
#if defined __MSVCRT__ || defined __OS2__ || defined __DJGPP__ #if defined __MSVCRT__ || defined __OS2__ || defined __DJGPP__
#include <io.h> #include <io.h>
#if defined __MSVCRT__ #if defined __MSVCRT__
#include <direct.h>
#define fchmod(x,y) 0 #define fchmod(x,y) 0
#define fchown(x,y,z) 0 #define fchown(x,y,z) 0
#define mkdir(name,mode) _mkdir(name)
#define strtoull strtoul #define strtoull strtoul
#define SIGHUP SIGTERM #define SIGHUP SIGTERM
#define S_ISSOCK(x) 0 #define S_ISSOCK(x) 0
@ -88,13 +90,13 @@ static const struct { const char * from; const char * to; } known_extensions[] =
{ ".tlz", ".tar" }, { ".tlz", ".tar" },
{ 0, 0 } }; { 0, 0 } };
struct Lzma_options typedef struct Lzma_options
{ {
int dictionary_size; /* 4 KiB .. 512 MiB */ int dictionary_size; /* 4 KiB .. 512 MiB */
int match_len_limit; /* 5 .. 273 */ int match_len_limit; /* 5 .. 273 */
}; } Lzma_options;
enum Mode { m_compress, m_decompress, m_list, m_test }; typedef enum Mode { m_compress, m_decompress, m_list, m_test } Mode;
/* Variables used in signal handler context. /* Variables used in signal handler context.
They are not declared volatile because the handler never returns. */ They are not declared volatile because the handler never returns. */
@ -105,26 +107,25 @@ static bool delete_output_on_interrupt = false;
static void show_help( void ) static void show_help( void )
{ {
printf( "Clzip is a C language version of lzip, compatible with lzip 1.4 or newer. As\n" printf( "Clzip is a C language version of lzip intended for systems lacking a C++\n"
"clzip is written in C, it may be easier to integrate in applications like\n" "compiler.\n"
"package managers, embedded devices, or systems lacking a C++ compiler.\n"
"\nLzip is a lossless data compressor with a user interface similar to the one\n" "\nLzip is a lossless data compressor with a user interface similar to the one\n"
"of gzip or bzip2. Lzip uses a simplified form of the 'Lempel-Ziv-Markov\n" "of gzip or bzip2. Lzip uses a simplified form of LZMA (Lempel-Ziv-Markov\n"
"chain-Algorithm' (LZMA) stream format to maximize interoperability. The\n" "chain-Algorithm) designed to achieve complete interoperability between\n"
"maximum dictionary size is 512 MiB so that any lzip file can be decompressed\n" "implementations. The maximum dictionary size is 512 MiB so that any lzip\n"
"on 32-bit machines. Lzip provides accurate and robust 3-factor integrity\n" "file can be decompressed on 32-bit machines. Lzip provides accurate and\n"
"checking. Lzip can compress about as fast as gzip (lzip -0) or compress most\n" "robust 3-factor integrity checking. 'lzip -0' compresses about as fast as\n"
"files more than bzip2 (lzip -9). Decompression speed is intermediate between\n" "gzip, while 'lzip -9' compresses most files more than bzip2. Decompression\n"
"gzip and bzip2. Lzip is better than gzip and bzip2 from a data recovery\n" "speed is intermediate between gzip and bzip2. Lzip provides better data\n"
"perspective. Lzip has been designed, written, and tested with great care to\n" "recovery capabilities than gzip and bzip2. Lzip has been designed, written,\n"
"replace gzip and bzip2 as the standard general-purpose compressed format for\n" "and tested with great care to replace gzip and bzip2 as general-purpose\n"
"Unix-like systems.\n" "compressed format for Unix-like systems.\n"
"\nUsage: %s [options] [files]\n", invocation_name ); "\nUsage: %s [options] [files]\n", invocation_name );
printf( "\nOptions:\n" printf( "\nOptions:\n"
" -h, --help display this help and exit\n" " -h, --help display this help and exit\n"
" -V, --version output version information and exit\n" " -V, --version output version information and exit\n"
" -a, --trailing-error exit with error status if trailing data\n" " -a, --trailing-error exit with error status if trailing data\n"
" -b, --member-size=<bytes> set member size limit in bytes\n" " -b, --member-size=<bytes> set member size limit of multimember files\n"
" -c, --stdout write to standard output, keep input files\n" " -c, --stdout write to standard output, keep input files\n"
" -d, --decompress decompress, test compressed file integrity\n" " -d, --decompress decompress, test compressed file integrity\n"
" -f, --force overwrite existing output files\n" " -f, --force overwrite existing output files\n"
@ -141,8 +142,6 @@ static void show_help( void )
" -0 .. -9 set compression level [default 6]\n" " -0 .. -9 set compression level [default 6]\n"
" --fast alias for -0\n" " --fast alias for -0\n"
" --best alias for -9\n" " --best alias for -9\n"
" --empty-error exit with error status if empty member in file\n"
" --marking-error exit with error status if 1st LZMA byte not 0\n"
" --loose-trailing allow trailing data seeming corrupt header\n" " --loose-trailing allow trailing data seeming corrupt header\n"
"\nIf no file names are given, or if a file is '-', clzip compresses or\n" "\nIf no file names are given, or if a file is '-', clzip compresses or\n"
"decompresses from standard input to standard output.\n" "decompresses from standard input to standard output.\n"
@ -199,7 +198,7 @@ struct Pretty_print
bool first_post; bool first_post;
}; };
static void Pp_init( struct Pretty_print * const pp, static void Pp_init( Pretty_print * const pp,
const char * const filenames[], const int num_filenames ) const char * const filenames[], const int num_filenames )
{ {
pp->name = 0; pp->name = 0;
@ -220,8 +219,10 @@ static void Pp_init( struct Pretty_print * const pp,
if( pp->longest_name == 0 ) pp->longest_name = stdin_name_len; if( pp->longest_name == 0 ) pp->longest_name = stdin_name_len;
} }
static void Pp_set_name( struct Pretty_print * const pp, void Pp_free( Pretty_print * const pp )
const char * const filename ) { if( pp->padded_name ) { free( pp->padded_name ); pp->padded_name = 0; } }
static void Pp_set_name( Pretty_print * const pp, const char * const filename )
{ {
unsigned name_len, padded_name_len, i = 0; unsigned name_len, padded_name_len, i = 0;
@ -239,10 +240,10 @@ static void Pp_set_name( struct Pretty_print * const pp,
pp->first_post = true; pp->first_post = true;
} }
static void Pp_reset( struct Pretty_print * const pp ) static void Pp_reset( Pretty_print * const pp )
{ if( pp->name && pp->name[0] ) pp->first_post = true; } { if( pp->name && pp->name[0] ) pp->first_post = true; }
void Pp_show_msg( struct Pretty_print * const pp, const char * const msg ) void Pp_show_msg( Pretty_print * const pp, const char * const msg )
{ {
if( verbosity < 0 ) return; if( verbosity < 0 ) return;
if( pp->first_post ) if( pp->first_post )
@ -272,7 +273,7 @@ const char * format_ds( const unsigned dictionary_size )
const char * p = ""; const char * p = "";
const char * np = " "; const char * np = " ";
unsigned num = dictionary_size; unsigned num = dictionary_size;
bool exact = ( num % factor == 0 ); bool exact = num % factor == 0;
int i; for( i = 0; i < n && ( num > 9999 || ( exact && num >= factor ) ); ++i ) int i; for( i = 0; i < n && ( num > 9999 || ( exact && num >= factor ) ); ++i )
{ num /= factor; if( num % factor != 0 ) exact = false; { num /= factor; if( num % factor != 0 ) exact = false;
@ -288,7 +289,7 @@ void show_header( const unsigned dictionary_size )
} }
/* separate numbers of 5 or more digits in groups of 3 digits using '_' */ /* separate numbers of 6 or more digits in groups of 3 digits using '_' */
static const char * format_num3( unsigned long long num ) static const char * format_num3( unsigned long long num )
{ {
enum { buffers = 8, bufsize = 4 * sizeof num, n = 10 }; enum { buffers = 8, bufsize = 4 * sizeof num, n = 10 };
@ -300,7 +301,7 @@ static const char * format_num3( unsigned long long num )
char * const buf = buffer[current++]; current %= buffers; char * const buf = buffer[current++]; current %= buffers;
char * p = buf + bufsize - 1; /* fill the buffer backwards */ char * p = buf + bufsize - 1; /* fill the buffer backwards */
*p = 0; /* terminator */ *p = 0; /* terminator */
if( num > 1024 ) if( num > 9999 )
{ {
char prefix = 0; /* try binary first, then si */ char prefix = 0; /* try binary first, then si */
for( i = 0; i < n && num != 0 && num % 1024 == 0; ++i ) for( i = 0; i < n && num != 0 && num % 1024 == 0; ++i )
@ -311,7 +312,7 @@ static const char * format_num3( unsigned long long num )
{ num /= 1000; prefix = si_prefix[i]; } { num /= 1000; prefix = si_prefix[i]; }
if( prefix ) *(--p) = prefix; if( prefix ) *(--p) = prefix;
} }
const bool split = num >= 10000; const bool split = num >= 100000;
for( i = 0; ; ) for( i = 0; ; )
{ {
@ -346,7 +347,7 @@ static unsigned long long getnum( const char * const arg,
if( !errno && tail[0] ) if( !errno && tail[0] )
{ {
const unsigned factor = ( tail[1] == 'i' ) ? 1024 : 1000; const unsigned factor = (tail[1] == 'i') ? 1024 : 1000;
int exponent = 0; /* 0 = bad multiplier */ int exponent = 0; /* 0 = bad multiplier */
int i; int i;
switch( tail[0] ) switch( tail[0] )
@ -396,7 +397,7 @@ static int get_dict_size( const char * const arg, const char * const option_name
} }
static void set_mode( enum Mode * const program_modep, const enum Mode new_mode ) static void set_mode( Mode * const program_modep, const Mode new_mode )
{ {
if( *program_modep != m_compress && *program_modep != new_mode ) if( *program_modep != m_compress && *program_modep != new_mode )
{ {
@ -473,9 +474,9 @@ int open_instream( const char * const name, struct stat * const in_statsp,
{ {
const int i = fstat( infd, in_statsp ); const int i = fstat( infd, in_statsp );
const mode_t mode = in_statsp->st_mode; const mode_t mode = in_statsp->st_mode;
const bool can_read = ( i == 0 && !reg_only && const bool can_read = i == 0 && !reg_only &&
( S_ISBLK( mode ) || S_ISCHR( mode ) || ( S_ISBLK( mode ) || S_ISCHR( mode ) ||
S_ISFIFO( mode ) || S_ISSOCK( mode ) ) ); S_ISFIFO( mode ) || S_ISSOCK( mode ) );
if( i != 0 || ( !S_ISREG( mode ) && ( !can_read || one_to_one ) ) ) if( i != 0 || ( !S_ISREG( mode ) && ( !can_read || one_to_one ) ) )
{ {
if( verbosity >= 0 ) if( verbosity >= 0 )
@ -491,13 +492,13 @@ int open_instream( const char * const name, struct stat * const in_statsp,
static int open_instream2( const char * const name, struct stat * const in_statsp, static int open_instream2( const char * const name, struct stat * const in_statsp,
const enum Mode program_mode, const int eindex, const Mode program_mode, const int eindex,
const bool one_to_one, const bool recompress ) const bool one_to_one, const bool recompress )
{ {
if( program_mode == m_compress && !recompress && eindex >= 0 ) if( program_mode == m_compress && !recompress && eindex >= 0 )
{ {
if( verbosity >= 0 ) if( verbosity >= 0 )
fprintf( stderr, "%s: %s: Input file already has '%s' suffix.\n", fprintf( stderr, "%s: %s: Input file already has '%s' suffix, ignored.\n",
program_name, name, known_extensions[eindex].from ); program_name, name, known_extensions[eindex].from );
return -1; return -1;
} }
@ -519,7 +520,7 @@ static bool make_dirs( const char * const name )
while( i < dirsize && name[i] != '/' ) ++i; while( i < dirsize && name[i] != '/' ) ++i;
if( first < i ) if( first < i )
{ {
char partial[i+1]; memcpy( partial, name, i ); partial[i] = 0; char partial[i+1]; memcpy( partial, name, i ); partial[i] = 0; /* vla */
const mode_t mode = S_IRWXU | S_IRGRP | S_IXGRP | S_IROTH | S_IXOTH; const mode_t mode = S_IRWXU | S_IRGRP | S_IXGRP | S_IROTH | S_IXOTH;
struct stat st; struct stat st;
if( stat( partial, &st ) == 0 ) if( stat( partial, &st ) == 0 )
@ -590,7 +591,7 @@ static void signal_handler( int sig )
static bool check_tty_in( const char * const input_filename, const int infd, static bool check_tty_in( const char * const input_filename, const int infd,
const enum Mode program_mode, int * const retval ) const Mode program_mode, int * const retval )
{ {
if( ( program_mode == m_decompress || program_mode == m_test ) && if( ( program_mode == m_decompress || program_mode == m_test ) &&
isatty( infd ) ) /* for example /dev/tty */ isatty( infd ) ) /* for example /dev/tty */
@ -602,7 +603,7 @@ static bool check_tty_in( const char * const input_filename, const int infd,
return true; return true;
} }
static bool check_tty_out( const enum Mode program_mode ) static bool check_tty_out( const Mode program_mode )
{ {
if( program_mode == m_compress && isatty( outfd ) ) if( program_mode == m_compress && isatty( outfd ) )
{ show_file_error( output_filename[0] ? { show_file_error( output_filename[0] ?
@ -661,30 +662,26 @@ static bool next_filename( void )
} }
struct Poly_encoder
{
struct LZ_encoder_base * eb;
struct LZ_encoder * e;
struct FLZ_encoder * fe;
};
static int compress( const unsigned long long cfile_size, static int compress( const unsigned long long cfile_size,
const unsigned long long member_size, const unsigned long long member_size,
const unsigned long long volume_size, const int infd, const unsigned long long volume_size, const int infd,
const struct Lzma_options * const encoder_options, const Lzma_options * const encoder_options,
struct Pretty_print * const pp, Pretty_print * const pp,
const struct stat * const in_statsp, const bool zero ) const struct stat * const in_statsp, const bool zero )
{ {
int retval = 0; struct
struct Poly_encoder encoder = { 0, 0, 0 }; /* polymorphic encoder */ {
LZ_encoder_base * eb;
LZ_encoder * e;
FLZ_encoder * fe;
} encoder = { 0, 0, 0 }; /* polymorphic encoder */
if( verbosity >= 1 ) Pp_show_msg( pp, 0 ); if( verbosity >= 1 ) Pp_show_msg( pp, 0 );
{ {
bool error = false; bool error = false;
if( zero ) if( zero )
{ {
encoder.fe = (struct FLZ_encoder *)malloc( sizeof *encoder.fe ); encoder.fe = (FLZ_encoder *)malloc( sizeof *encoder.fe );
if( !encoder.fe || !FLZe_init( encoder.fe, infd, outfd ) ) error = true; if( !encoder.fe || !FLZe_init( encoder.fe, infd, outfd ) ) error = true;
else encoder.eb = &encoder.fe->eb; else encoder.eb = &encoder.fe->eb;
} }
@ -694,7 +691,7 @@ static int compress( const unsigned long long cfile_size,
if( Lh_set_dictionary_size( header, encoder_options->dictionary_size ) && if( Lh_set_dictionary_size( header, encoder_options->dictionary_size ) &&
encoder_options->match_len_limit >= min_match_len_limit && encoder_options->match_len_limit >= min_match_len_limit &&
encoder_options->match_len_limit <= max_match_len ) encoder_options->match_len_limit <= max_match_len )
encoder.e = (struct LZ_encoder *)malloc( sizeof *encoder.e ); encoder.e = (LZ_encoder *)malloc( sizeof *encoder.e );
else internal_error( "invalid argument to encoder." ); else internal_error( "invalid argument to encoder." );
if( !encoder.e || !LZe_init( encoder.e, Lh_get_dictionary_size( header ), if( !encoder.e || !LZe_init( encoder.e, Lh_get_dictionary_size( header ),
encoder_options->match_len_limit, infd, outfd ) ) encoder_options->match_len_limit, infd, outfd ) )
@ -709,9 +706,10 @@ static int compress( const unsigned long long cfile_size,
} }
unsigned long long in_size = 0, out_size = 0, partial_volume_size = 0; unsigned long long in_size = 0, out_size = 0, partial_volume_size = 0;
int retval = 0;
while( true ) /* encode one member per iteration */ while( true ) /* encode one member per iteration */
{ {
const unsigned long long size = ( volume_size > 0 ) ? const unsigned long long size = (volume_size > 0) ?
min( member_size, volume_size - partial_volume_size ) : member_size; min( member_size, volume_size - partial_volume_size ) : member_size;
show_cprogress( cfile_size, in_size, &encoder.eb->mb, pp ); /* init */ show_cprogress( cfile_size, in_size, &encoder.eb->mb, pp ); /* init */
if( ( zero && !FLZe_encode_member( encoder.fe, size ) ) || if( ( zero && !FLZe_encode_member( encoder.fe, size ) ) ||
@ -765,7 +763,7 @@ static unsigned char xdigit( const unsigned value ) /* hex digit for 'value' */
static bool show_trailing_data( const uint8_t * const data, const int size, static bool show_trailing_data( const uint8_t * const data, const int size,
struct Pretty_print * const pp, const bool all, Pretty_print * const pp, const bool all,
const int ignore_trailing ) /* -1 = show */ const int ignore_trailing ) /* -1 = show */
{ {
if( verbosity >= 4 || ignore_trailing <= 0 ) if( verbosity >= 4 || ignore_trailing <= 0 )
@ -793,15 +791,16 @@ static bool show_trailing_data( const uint8_t * const data, const int size,
static int decompress( const unsigned long long cfile_size, const int infd, static int decompress( const unsigned long long cfile_size, const int infd,
const struct Cl_options * const cl_opts, const Cl_options * const cl_opts, Pretty_print * const pp,
struct Pretty_print * const pp, const bool testing ) const bool from_stdin, const bool testing )
{ {
unsigned long long partial_file_pos = 0; unsigned long long partial_file_pos = 0;
struct Range_decoder rdec; Range_decoder rdec;
int retval = 0; int retval = 0;
bool first_member; bool first_member;
if( !Rd_init( &rdec, infd ) ) if( !Rd_init( &rdec, infd ) )
{ show_error( mem_msg, 0, false ); cleanup_and_fail( 1 ); } { show_error( mem_msg, 0, false ); cleanup_and_fail( 1 ); }
bool empty = false, multi = false;
for( first_member = true; ; first_member = false ) for( first_member = true; ; first_member = false )
{ {
@ -841,11 +840,11 @@ static int decompress( const unsigned long long cfile_size, const int infd,
if( verbosity >= 2 || ( verbosity == 1 && first_member ) ) if( verbosity >= 2 || ( verbosity == 1 && first_member ) )
Pp_show_msg( pp, 0 ); Pp_show_msg( pp, 0 );
struct LZ_decoder decoder; LZ_decoder decoder;
if( !LZd_init( &decoder, &rdec, dictionary_size, outfd ) ) if( !LZd_init( &decoder, &rdec, dictionary_size, outfd ) )
{ Pp_show_msg( pp, mem_msg ); retval = 1; break; } { Pp_show_msg( pp, mem_msg ); retval = 1; break; }
show_dprogress( cfile_size, partial_file_pos, &rdec, pp ); /* init */ show_dprogress( cfile_size, partial_file_pos, &rdec, pp ); /* init */
const int result = LZd_decode_member( &decoder, cl_opts, pp ); const int result = LZd_decode_member( &decoder, pp );
partial_file_pos += Rd_member_position( &rdec ); partial_file_pos += Rd_member_position( &rdec );
LZd_free( &decoder ); LZd_free( &decoder );
if( result != 0 ) if( result != 0 )
@ -857,16 +856,19 @@ static int decompress( const unsigned long long cfile_size, const int infd,
"File ends unexpectedly" : "Decoder error", "File ends unexpectedly" : "Decoder error",
partial_file_pos ); partial_file_pos );
} }
else if( result == 5 ) Pp_show_msg( pp, empty_msg ); else if( result == 5 ) Pp_show_msg( pp, nonzero_msg );
else if( result == 6 ) Pp_show_msg( pp, marking_msg );
retval = 2; break; retval = 2; break;
} }
if( !from_stdin ) { multi = !first_member;
if( LZd_data_position( &decoder ) == 0 ) empty = true; }
if( verbosity >= 2 ) if( verbosity >= 2 )
{ fputs( testing ? "ok\n" : "done\n", stderr ); Pp_reset( pp ); } { fputs( testing ? "ok\n" : "done\n", stderr ); Pp_reset( pp ); }
} }
Rd_free( &rdec ); Rd_free( &rdec );
if( verbosity == 1 && retval == 0 ) if( verbosity == 1 && retval == 0 )
fputs( testing ? "ok\n" : "done\n", stderr ); fputs( testing ? "ok\n" : "done\n", stderr );
if( empty && multi && retval == 0 )
{ show_file_error( pp->name, empty_msg, 0 ); retval = 2; }
return retval; return retval;
} }
@ -904,13 +906,13 @@ void internal_error( const char * const msg )
void show_cprogress( const unsigned long long cfile_size, void show_cprogress( const unsigned long long cfile_size,
const unsigned long long partial_size, const unsigned long long partial_size,
const struct Matchfinder_base * const m, const Matchfinder_base * const m,
struct Pretty_print * const p ) Pretty_print * const p )
{ {
static unsigned long long csize = 0; /* file_size / 100 */ static unsigned long long csize = 0; /* file_size / 100 */
static unsigned long long psize = 0; static unsigned long long psize = 0;
static const struct Matchfinder_base * mb = 0; static const Matchfinder_base * mb = 0;
static struct Pretty_print * pp = 0; static Pretty_print * pp = 0;
static bool enabled = true; static bool enabled = true;
if( !enabled ) return; if( !enabled ) return;
@ -933,13 +935,13 @@ void show_cprogress( const unsigned long long cfile_size,
void show_dprogress( const unsigned long long cfile_size, void show_dprogress( const unsigned long long cfile_size,
const unsigned long long partial_size, const unsigned long long partial_size,
const struct Range_decoder * const d, const Range_decoder * const d,
struct Pretty_print * const p ) Pretty_print * const p )
{ {
static unsigned long long csize = 0; /* file_size / 100 */ static unsigned long long csize = 0; /* file_size / 100 */
static unsigned long long psize = 0; static unsigned long long psize = 0;
static const struct Range_decoder * rdec = 0; static const Range_decoder * rdec = 0;
static struct Pretty_print * pp = 0; static Pretty_print * pp = 0;
static int counter = 0; static int counter = 0;
static bool enabled = true; static bool enabled = true;
@ -966,7 +968,7 @@ int main( const int argc, const char * const argv[] )
{ {
/* Mapping from gzip/bzip2 style 0..9 compression levels to the /* Mapping from gzip/bzip2 style 0..9 compression levels to the
corresponding LZMA compression parameters. */ corresponding LZMA compression parameters. */
const struct Lzma_options option_mapping[] = const Lzma_options option_mapping[] =
{ {
{ 1 << 16, 16 }, /* -0 */ { 1 << 16, 16 }, /* -0 */
{ 1 << 20, 5 }, /* -1 */ { 1 << 20, 5 }, /* -1 */
@ -978,15 +980,14 @@ int main( const int argc, const char * const argv[] )
{ 1 << 24, 68 }, /* -7 */ { 1 << 24, 68 }, /* -7 */
{ 3 << 23, 132 }, /* -8 */ { 3 << 23, 132 }, /* -8 */
{ 1 << 25, 273 } }; /* -9 */ { 1 << 25, 273 } }; /* -9 */
struct Lzma_options encoder_options = option_mapping[6]; /* default = "-6" */ Lzma_options encoder_options = option_mapping[6]; /* default = "-6" */
const unsigned long long max_member_size = 0x0008000000000000ULL; /* 2 PiB */ const unsigned long long max_member_size = 0x0008000000000000ULL; /* 2 PiB */
const unsigned long long max_volume_size = 0x4000000000000000ULL; /* 4 EiB */ const unsigned long long max_volume_size = 0x4000000000000000ULL; /* 4 EiB */
unsigned long long member_size = max_member_size; unsigned long long member_size = max_member_size;
unsigned long long volume_size = 0; unsigned long long volume_size = 0;
const char * default_output_filename = ""; const char * default_output_filename = "";
enum Mode program_mode = m_compress; Mode program_mode = m_compress;
int i; Cl_options cl_opts; /* command-line options */
struct Cl_options cl_opts; /* command-line options */
Cl_options_init( &cl_opts ); Cl_options_init( &cl_opts );
bool force = false; bool force = false;
bool keep_input_files = false; bool keep_input_files = false;
@ -995,46 +996,44 @@ int main( const int argc, const char * const argv[] )
bool zero = false; bool zero = false;
if( argc > 0 ) invocation_name = argv[0]; if( argc > 0 ) invocation_name = argv[0];
enum { opt_eer = 256, opt_lt, opt_mer }; enum { opt_lt = 256 };
const struct ap_Option options[] = const ap_Option options[] =
{ {
{ '0', "fast", ap_no }, { '0', "fast", ap_no },
{ '1', 0, ap_no }, { '1', 0, ap_no },
{ '2', 0, ap_no }, { '2', 0, ap_no },
{ '3', 0, ap_no }, { '3', 0, ap_no },
{ '4', 0, ap_no }, { '4', 0, ap_no },
{ '5', 0, ap_no }, { '5', 0, ap_no },
{ '6', 0, ap_no }, { '6', 0, ap_no },
{ '7', 0, ap_no }, { '7', 0, ap_no },
{ '8', 0, ap_no }, { '8', 0, ap_no },
{ '9', "best", ap_no }, { '9', "best", ap_no },
{ 'a', "trailing-error", ap_no }, { 'a', "trailing-error", ap_no },
{ 'b', "member-size", ap_yes }, { 'b', "member-size", ap_yes },
{ 'c', "stdout", ap_no }, { 'c', "stdout", ap_no },
{ 'd', "decompress", ap_no }, { 'd', "decompress", ap_no },
{ 'f', "force", ap_no }, { 'f', "force", ap_no },
{ 'F', "recompress", ap_no }, { 'F', "recompress", ap_no },
{ 'h', "help", ap_no }, { 'h', "help", ap_no },
{ 'k', "keep", ap_no }, { 'k', "keep", ap_no },
{ 'l', "list", ap_no }, { 'l', "list", ap_no },
{ 'm', "match-length", ap_yes }, { 'm', "match-length", ap_yes },
{ 'n', "threads", ap_yes }, { 'n', "threads", ap_yes },
{ 'o', "output", ap_yes }, { 'o', "output", ap_yes },
{ 'q', "quiet", ap_no }, { 'q', "quiet", ap_no },
{ 's', "dictionary-size", ap_yes }, { 's', "dictionary-size", ap_yes },
{ 'S', "volume-size", ap_yes }, { 'S', "volume-size", ap_yes },
{ 't', "test", ap_no }, { 't', "test", ap_no },
{ 'v', "verbose", ap_no }, { 'v', "verbose", ap_no },
{ 'V', "version", ap_no }, { 'V', "version", ap_no },
{ opt_eer, "empty-error", ap_no }, { opt_lt, "loose-trailing", ap_no },
{ opt_lt, "loose-trailing", ap_no }, { 0, 0, ap_no } };
{ opt_mer, "marking-error", ap_no },
{ 0, 0, ap_no } };
CRC32_init(); CRC32_init();
/* static because valgrind complains and memory management in C sucks */ /* static because valgrind complains and memory management in C sucks */
static struct Arg_parser parser; static Arg_parser parser;
if( !ap_init( &parser, argc, argv, options, 0 ) ) if( !ap_init( &parser, argc, argv, options, 0 ) )
{ show_error( mem_msg, 0, false ); return 1; } { show_error( mem_msg, 0, false ); return 1; }
if( ap_error( &parser ) ) /* bad option */ if( ap_error( &parser ) ) /* bad option */
@ -1049,9 +1048,8 @@ int main( const int argc, const char * const argv[] )
const char * const arg = ap_argument( &parser, argind ); const char * const arg = ap_argument( &parser, argind );
switch( code ) switch( code )
{ {
case '0': case '1': case '2': case '3': case '4': case '0': case '1': case '2': case '3': case '4': case '5':
case '5': case '6': case '7': case '8': case '9': case '6': case '7': case '8': case '9': zero = code == '0';
zero = ( code == '0' );
encoder_options = option_mapping[code-'0']; break; encoder_options = option_mapping[code-'0']; break;
case 'a': cl_opts.ignore_trailing = false; break; case 'a': cl_opts.ignore_trailing = false; break;
case 'b': member_size = getnum( arg, pn, 100000, max_member_size ); break; case 'b': member_size = getnum( arg, pn, 100000, max_member_size ); break;
@ -1065,7 +1063,7 @@ int main( const int argc, const char * const argv[] )
case 'm': encoder_options.match_len_limit = case 'm': encoder_options.match_len_limit =
getnum( arg, pn, min_match_len_limit, max_match_len ); getnum( arg, pn, min_match_len_limit, max_match_len );
zero = false; break; zero = false; break;
case 'n': break; case 'n': break; /* ignored */
case 'o': if( strcmp( arg, "-" ) == 0 ) to_stdout = true; case 'o': if( strcmp( arg, "-" ) == 0 ) to_stdout = true;
else { default_output_filename = arg; } break; else { default_output_filename = arg; } break;
case 'q': verbosity = -1; break; case 'q': verbosity = -1; break;
@ -1075,9 +1073,7 @@ int main( const int argc, const char * const argv[] )
case 't': set_mode( &program_mode, m_test ); break; case 't': set_mode( &program_mode, m_test ); break;
case 'v': if( verbosity < 4 ) ++verbosity; break; case 'v': if( verbosity < 4 ) ++verbosity; break;
case 'V': show_version(); return 0; case 'V': show_version(); return 0;
case opt_eer: cl_opts.ignore_empty = false; break; case opt_lt: cl_opts.loose_trailing = true; break;
case opt_lt: cl_opts.loose_trailing = true; break;
case opt_mer: cl_opts.ignore_marking = false; break;
default: internal_error( "uncaught option." ); default: internal_error( "uncaught option." );
} }
} /* end process options */ } /* end process options */
@ -1092,6 +1088,7 @@ int main( const int argc, const char * const argv[] )
filenames = resize_buffer( filenames, num_filenames * sizeof filenames[0] ); filenames = resize_buffer( filenames, num_filenames * sizeof filenames[0] );
filenames[0] = "-"; filenames[0] = "-";
int i;
bool filenames_given = false; bool filenames_given = false;
for( i = 0; argind + i < ap_arguments( &parser ); ++i ) for( i = 0; argind + i < ap_arguments( &parser ); ++i )
{ {
@ -1126,7 +1123,7 @@ int main( const int argc, const char * const argv[] )
if( !to_stdout && program_mode != m_test && ( filenames_given || to_file ) ) if( !to_stdout && program_mode != m_test && ( filenames_given || to_file ) )
set_signals( signal_handler ); set_signals( signal_handler );
static struct Pretty_print pp; static Pretty_print pp;
Pp_init( &pp, filenames, num_filenames ); Pp_init( &pp, filenames, num_filenames );
int failed_tests = 0; int failed_tests = 0;
@ -1138,9 +1135,10 @@ int main( const int argc, const char * const argv[] )
{ {
const char * input_filename = ""; const char * input_filename = "";
int infd; int infd;
const bool from_stdin = strcmp( filenames[i], "-" ) == 0;
Pp_set_name( &pp, filenames[i] ); Pp_set_name( &pp, filenames[i] );
if( strcmp( filenames[i], "-" ) == 0 ) if( from_stdin )
{ {
if( stdin_used ) continue; else stdin_used = true; if( stdin_used ) continue; else stdin_used = true;
infd = STDIN_FILENO; infd = STDIN_FILENO;
@ -1189,7 +1187,8 @@ int main( const int argc, const char * const argv[] )
tmp = compress( cfile_size, member_size, volume_size, infd, tmp = compress( cfile_size, member_size, volume_size, infd,
&encoder_options, &pp, in_statsp, zero ); &encoder_options, &pp, in_statsp, zero );
else else
tmp = decompress( cfile_size, infd, &cl_opts, &pp, program_mode == m_test ); tmp = decompress( cfile_size, infd, &cl_opts, &pp, from_stdin,
program_mode == m_test );
if( close( infd ) != 0 ) if( close( infd ) != 0 )
{ show_file_error( pp.name, "Error closing input file", errno ); { show_file_error( pp.name, "Error closing input file", errno );
set_retval( &tmp, 1 ); } set_retval( &tmp, 1 ); }
@ -1217,6 +1216,7 @@ int main( const int argc, const char * const argv[] )
program_name, failed_tests, program_name, failed_tests,
( failed_tests == 1 ) ? "file" : "files" ); ( failed_tests == 1 ) ? "file" : "files" );
free( output_filename ); free( output_filename );
Pp_free( &pp );
free( filenames ); free( filenames );
ap_free( &parser ); ap_free( &parser );
return retval; return retval;

163
testsuite/check.sh
View file

@ -28,12 +28,10 @@ if [ -d tmp ] ; then rm -rf tmp ; fi
mkdir tmp mkdir tmp
cd "${objdir}"/tmp || framework_failure cd "${objdir}"/tmp || framework_failure
cat "${testdir}"/test.txt > in || framework_failure cp "${testdir}"/test.txt in || framework_failure
in_lz="${testdir}"/test.txt.lz in_lz="${testdir}"/test.txt.lz
in_em="${testdir}"/test_em.txt.lz
fox_lz="${testdir}"/fox.lz fox_lz="${testdir}"/fox.lz
fox6_lz="${testdir}"/fox6.lz fnz_lz="${testdir}"/fox_nz.lz
f6mk_lz="${testdir}"/fox6_mark.lz
fail=0 fail=0
test_failed() { fail=1 ; printf " $1" ; [ -z "$2" ] || printf "($2)" ; } test_failed() { fail=1 ; printf " $1" ; [ -z "$2" ] || printf "($2)" ; }
@ -70,7 +68,7 @@ done
"${LZIP}" -q -o out.lz nx_file "${LZIP}" -q -o out.lz nx_file
[ $? = 1 ] || test_failed $LINENO [ $? = 1 ] || test_failed $LINENO
[ ! -e out.lz ] || test_failed $LINENO [ ! -e out.lz ] || test_failed $LINENO
"${LZIP}" -qf -S100k -o out in in "${LZIP}" -qf -S100k -o out in in # only one file with -o and -S
[ $? = 1 ] || test_failed $LINENO [ $? = 1 ] || test_failed $LINENO
{ [ ! -e out ] && [ ! -e out.lz ] ; } || test_failed $LINENO { [ ! -e out ] && [ ! -e out.lz ] ; } || test_failed $LINENO
# these are for code coverage # these are for code coverage
@ -106,37 +104,25 @@ printf "LZIP\001+.............................." | "${LZIP}" -t 2> /dev/null
printf "\ntesting decompression..." printf "\ntesting decompression..."
for i in "${in_lz}" "${in_em}" ; do "${LZIP}" -l "${in_lz}" > /dev/null || test_failed $LINENO
"${LZIP}" -lq "$i" || test_failed $LINENO "$i" "${LZIP}" -t "${in_lz}" || test_failed $LINENO
"${LZIP}" -t "$i" || test_failed $LINENO "$i" "${LZIP}" -d "${in_lz}" -o out || test_failed $LINENO
"${LZIP}" -d "$i" -o out || test_failed $LINENO "$i" cmp in out || test_failed $LINENO
cmp in out || test_failed $LINENO "$i" "${LZIP}" -cd "${in_lz}" > out || test_failed $LINENO
"${LZIP}" -cd "$i" > out || test_failed $LINENO "$i" cmp in out || test_failed $LINENO
cmp in out || test_failed $LINENO "$i" "${LZIP}" -d "${in_lz}" -o - > out || test_failed $LINENO
"${LZIP}" -d "$i" -o - > out || test_failed $LINENO "$i" cmp in out || test_failed $LINENO
cmp in out || test_failed $LINENO "$i" "${LZIP}" -d < "${in_lz}" > out || test_failed $LINENO
"${LZIP}" -d < "$i" > out || test_failed $LINENO "$i" cmp in out || test_failed $LINENO
cmp in out || test_failed $LINENO "$i" rm -f out || framework_failure
rm -f out || framework_failure
done
lines=`"${LZIP}" -tvv "${in_em}" 2>&1 | wc -l` || test_failed $LINENO cp "${in_lz}" out.lz || framework_failure
[ "${lines}" -eq 8 ] || test_failed $LINENO "${lines}"
"${LZIP}" -tq "${in_em}" --empty-error
[ $? = 2 ] || test_failed $LINENO
lines=`"${LZIP}" -lvv "${in_em}" | wc -l` || test_failed $LINENO
[ "${lines}" -eq 11 ] || test_failed $LINENO "${lines}"
"${LZIP}" -lq "${in_em}" --empty-error
[ $? = 2 ] || test_failed $LINENO
cat "${in_lz}" > out.lz || framework_failure
"${LZIP}" -dk out.lz || test_failed $LINENO "${LZIP}" -dk out.lz || test_failed $LINENO
cmp in out || test_failed $LINENO cmp in out || test_failed $LINENO
rm -f out || framework_failure rm -f out || framework_failure
"${LZIP}" -cd "${fox_lz}" > fox || test_failed $LINENO "${LZIP}" -cd "${fox_lz}" > fox || test_failed $LINENO
cat fox > copy || framework_failure cp fox copy || framework_failure
cat "${in_lz}" > copy.lz || framework_failure cp "${in_lz}" copy.lz || framework_failure
"${LZIP}" -d copy.lz out.lz 2> /dev/null # skip copy, decompress out "${LZIP}" -d copy.lz out.lz 2> /dev/null # skip copy, decompress out
[ $? = 1 ] || test_failed $LINENO [ $? = 1 ] || test_failed $LINENO
[ ! -e out.lz ] || test_failed $LINENO [ ! -e out.lz ] || test_failed $LINENO
@ -147,7 +133,7 @@ cmp in out || test_failed $LINENO
cmp in copy || test_failed $LINENO cmp in copy || test_failed $LINENO
rm -f copy out || framework_failure rm -f copy out || framework_failure
cat "${in_lz}" > out.lz || framework_failure cp "${in_lz}" out.lz || framework_failure
"${LZIP}" -d -S100k out.lz || test_failed $LINENO # ignore -S "${LZIP}" -d -S100k out.lz || test_failed $LINENO # ignore -S
[ ! -e out.lz ] || test_failed $LINENO [ ! -e out.lz ] || test_failed $LINENO
cmp in out || test_failed $LINENO cmp in out || test_failed $LINENO
@ -155,7 +141,6 @@ cmp in out || test_failed $LINENO
printf "to be overwritten" > out || framework_failure printf "to be overwritten" > out || framework_failure
"${LZIP}" -df -o out < "${in_lz}" || test_failed $LINENO "${LZIP}" -df -o out < "${in_lz}" || test_failed $LINENO
cmp in out || test_failed $LINENO cmp in out || test_failed $LINENO
rm -f out || framework_failure
"${LZIP}" -d -o ./- "${in_lz}" || test_failed $LINENO "${LZIP}" -d -o ./- "${in_lz}" || test_failed $LINENO
cmp in ./- || test_failed $LINENO cmp in ./- || test_failed $LINENO
rm -f ./- || framework_failure rm -f ./- || framework_failure
@ -163,12 +148,12 @@ rm -f ./- || framework_failure
cmp in ./- || test_failed $LINENO cmp in ./- || test_failed $LINENO
rm -f ./- || framework_failure rm -f ./- || framework_failure
cat "${in_lz}" > anyothername || framework_failure cp "${in_lz}" anyothername || framework_failure
"${LZIP}" -dv - anyothername - < "${in_lz}" > out 2> /dev/null || "${LZIP}" -dv - anyothername - < "${in_lz}" > out 2> /dev/null ||
test_failed $LINENO test_failed $LINENO
cmp in out || test_failed $LINENO cmp in out || test_failed $LINENO
cmp in anyothername.out || test_failed $LINENO cmp in anyothername.out || test_failed $LINENO
rm -f out anyothername.out || framework_failure rm -f anyothername.out || framework_failure
"${LZIP}" -lq in "${in_lz}" "${LZIP}" -lq in "${in_lz}"
[ $? = 2 ] || test_failed $LINENO [ $? = 2 ] || test_failed $LINENO
@ -185,7 +170,7 @@ cat out in | cmp in - || test_failed $LINENO # out must be empty
[ $? = 1 ] || test_failed $LINENO [ $? = 1 ] || test_failed $LINENO
cmp in out || test_failed $LINENO cmp in out || test_failed $LINENO
rm -f out || framework_failure rm -f out || framework_failure
cat "${in_lz}" > out.lz || framework_failure cp "${in_lz}" out.lz || framework_failure
for i in 1 2 3 4 5 6 7 ; do for i in 1 2 3 4 5 6 7 ; do
printf "g" >> out.lz || framework_failure printf "g" >> out.lz || framework_failure
"${LZIP}" -alvv out.lz "${in_lz}" > /dev/null 2>&1 "${LZIP}" -alvv out.lz "${in_lz}" > /dev/null 2>&1
@ -206,7 +191,7 @@ cmp in out || test_failed $LINENO
rm -f out || framework_failure rm -f out || framework_failure
cat in in > in2 || framework_failure cat in in > in2 || framework_failure
"${LZIP}" -lq "${in_lz}" "${in_lz}" || test_failed $LINENO "${LZIP}" -l "${in_lz}" "${in_lz}" > /dev/null || test_failed $LINENO
"${LZIP}" -t "${in_lz}" "${in_lz}" || test_failed $LINENO "${LZIP}" -t "${in_lz}" "${in_lz}" || test_failed $LINENO
"${LZIP}" -cd "${in_lz}" "${in_lz}" -o out > out2 || test_failed $LINENO "${LZIP}" -cd "${in_lz}" "${in_lz}" -o out > out2 || test_failed $LINENO
[ ! -e out ] || test_failed $LINENO # override -o [ ! -e out ] || test_failed $LINENO # override -o
@ -217,6 +202,11 @@ cmp in2 out2 || test_failed $LINENO
rm -f out2 || framework_failure rm -f out2 || framework_failure
cat "${in_lz}" "${in_lz}" > out2.lz || framework_failure cat "${in_lz}" "${in_lz}" > out2.lz || framework_failure
lines=`"${LZIP}" -tvv out2.lz 2>&1 | wc -l` || test_failed $LINENO
[ "${lines}" -eq 2 ] || test_failed $LINENO "${lines}"
lines=`"${LZIP}" -lvv out2.lz | wc -l` || test_failed $LINENO
[ "${lines}" -eq 5 ] || test_failed $LINENO "${lines}"
printf "\ngarbage" >> out2.lz || framework_failure printf "\ngarbage" >> out2.lz || framework_failure
"${LZIP}" -tvvvv out2.lz 2> /dev/null || test_failed $LINENO "${LZIP}" -tvvvv out2.lz 2> /dev/null || test_failed $LINENO
"${LZIP}" -alq out2.lz "${LZIP}" -alq out2.lz
@ -236,15 +226,6 @@ printf "to be overwritten" > out2 || framework_failure
cmp in2 out2 || test_failed $LINENO cmp in2 out2 || test_failed $LINENO
rm -f out2 || framework_failure rm -f out2 || framework_failure
"${LZIP}" -cd "${fox6_lz}" > out || test_failed $LINENO
"${LZIP}" -cd "${f6mk_lz}" > copy || test_failed $LINENO
cmp copy out || test_failed $LINENO
rm -f copy out || framework_failure
"${LZIP}" -lq "${f6mk_lz}" --marking-error
[ $? = 2 ] || test_failed $LINENO
"${LZIP}" -tq "${f6mk_lz}" --marking-error
[ $? = 2 ] || test_failed $LINENO
"${LZIP}" -d "${fox_lz}" -o a/b/c/fox || test_failed $LINENO "${LZIP}" -d "${fox_lz}" -o a/b/c/fox || test_failed $LINENO
cmp fox a/b/c/fox || test_failed $LINENO cmp fox a/b/c/fox || test_failed $LINENO
rm -rf a || framework_failure rm -rf a || framework_failure
@ -255,6 +236,21 @@ rm -rf a || framework_failure
[ $? = 1 ] || test_failed $LINENO [ $? = 1 ] || test_failed $LINENO
[ ! -e a ] || test_failed $LINENO [ ! -e a ] || test_failed $LINENO
touch empty em || framework_failure
"${LZIP}" -0 em || test_failed $LINENO
"${LZIP}" -l em.lz > /dev/null || test_failed $LINENO
"${LZIP}" -dk em.lz || test_failed $LINENO
cmp empty em || test_failed $LINENO
cat em.lz em.lz | "${LZIP}" -t || test_failed $LINENO
cat em.lz em.lz | "${LZIP}" -d > em || test_failed $LINENO
cmp empty em || test_failed $LINENO
cat em.lz "${in_lz}" | "${LZIP}" -t || test_failed $LINENO
cat em.lz "${in_lz}" | "${LZIP}" -d > out || test_failed $LINENO
cmp in out || test_failed $LINENO
cat "${in_lz}" em.lz | "${LZIP}" -t || test_failed $LINENO
cat "${in_lz}" em.lz | "${LZIP}" -d > out || test_failed $LINENO
cmp in out || test_failed $LINENO
printf "\ntesting compression..." printf "\ntesting compression..."
"${LZIP}" -c -0 in in in -S100k -o out3.lz > copy2.lz || test_failed $LINENO "${LZIP}" -c -0 in in in -S100k -o out3.lz > copy2.lz || test_failed $LINENO
@ -263,7 +259,7 @@ printf "\ntesting compression..."
"${LZIP}" -d copy2.lz -o out2 || test_failed $LINENO "${LZIP}" -d copy2.lz -o out2 || test_failed $LINENO
[ -e copy2.lz ] || test_failed $LINENO [ -e copy2.lz ] || test_failed $LINENO
cmp in2 out2 || test_failed $LINENO cmp in2 out2 || test_failed $LINENO
rm -f in2 out2 copy2.lz || framework_failure rm -f copy2.lz || framework_failure
"${LZIP}" -cf "${in_lz}" > lzlz 2> /dev/null # /dev/null is a tty on OS/2 "${LZIP}" -cf "${in_lz}" > lzlz 2> /dev/null # /dev/null is a tty on OS/2
[ $? = 1 ] || test_failed $LINENO [ $? = 1 ] || test_failed $LINENO
@ -331,7 +327,7 @@ rm -f in8 || framework_failure
"${LZIP}" -t out00001.lz out00002.lz || test_failed $LINENO "${LZIP}" -t out00001.lz out00002.lz || test_failed $LINENO
"${LZIP}" -cd out00001.lz out00002.lz | cmp in8.lz - || test_failed $LINENO "${LZIP}" -cd out00001.lz out00002.lz | cmp in8.lz - || test_failed $LINENO
[ ! -e out00003.lz ] || test_failed $LINENO [ ! -e out00003.lz ] || test_failed $LINENO
rm -f out00001.lz || framework_failure rm -f out00001.lz out00002.lz || framework_failure
"${LZIP}" -1 -S100k -o a/b/c/out < in8.lz || test_failed $LINENO "${LZIP}" -1 -S100k -o a/b/c/out < in8.lz || test_failed $LINENO
"${LZIP}" -t a/b/c/out00001.lz a/b/c/out00002.lz || test_failed $LINENO "${LZIP}" -t a/b/c/out00001.lz a/b/c/out00002.lz || test_failed $LINENO
"${LZIP}" -cd a/b/c/out00001.lz a/b/c/out00002.lz | cmp in8.lz - || "${LZIP}" -cd a/b/c/out00001.lz a/b/c/out00002.lz | cmp in8.lz - ||
@ -357,11 +353,44 @@ rm -rf a || framework_failure
printf "\ntesting bad input..." printf "\ntesting bad input..."
cat em.lz em.lz > ee.lz || framework_failure
"${LZIP}" -l < ee.lz > /dev/null || test_failed $LINENO
"${LZIP}" -t < ee.lz || test_failed $LINENO
"${LZIP}" -d < ee.lz > em || test_failed $LINENO
cmp empty em || test_failed $LINENO
"${LZIP}" -lq ee.lz
[ $? = 2 ] || test_failed $LINENO
"${LZIP}" -tq ee.lz
[ $? = 2 ] || test_failed $LINENO
"${LZIP}" -dq ee.lz
[ $? = 2 ] || test_failed $LINENO
[ ! -e ee ] || test_failed $LINENO
"${LZIP}" -cdq ee.lz > em
[ $? = 2 ] || test_failed $LINENO
cmp empty em || test_failed $LINENO
rm -f empty em || framework_failure
cat "${in_lz}" em.lz "${in_lz}" > inein.lz || framework_failure
"${LZIP}" -l < inein.lz > /dev/null || test_failed $LINENO
"${LZIP}" -t < inein.lz || test_failed $LINENO
"${LZIP}" -d < inein.lz > out2 || test_failed $LINENO
cmp in2 out2 || test_failed $LINENO
"${LZIP}" -lq inein.lz
[ $? = 2 ] || test_failed $LINENO
"${LZIP}" -tq inein.lz
[ $? = 2 ] || test_failed $LINENO
"${LZIP}" -dq inein.lz
[ $? = 2 ] || test_failed $LINENO
[ ! -e inein ] || test_failed $LINENO
"${LZIP}" -cdq inein.lz > out2
[ $? = 2 ] || test_failed $LINENO
cmp in2 out2 || test_failed $LINENO
rm -f in2 out2 inein.lz em.lz || framework_failure
headers='LZIp LZiP LZip LzIP LzIp LziP lZIP lZIp lZiP lzIP' headers='LZIp LZiP LZip LzIP LzIp LziP lZIP lZIp lZiP lzIP'
body='\001\014\000\203\377\373\377\377\300\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000$\000\000\000\000\000\000\000' body='\001\014\000\000\101\376\367\377\377\340\000\200\000\215\357\002\322\001\000\000\000\000\000\000\000\045\000\000\000\000\000\000\000'
cat "${in_lz}" > int.lz || framework_failure cp "${in_lz}" int.lz || framework_failure
printf "LZIP${body}" >> int.lz || framework_failure printf "LZIP${body}" >> int.lz || framework_failure
if "${LZIP}" -tq int.lz ; then if "${LZIP}" -t int.lz ; then
for header in ${headers} ; do for header in ${headers} ; do
printf "${header}${body}" > int.lz || framework_failure printf "${header}${body}" > int.lz || framework_failure
"${LZIP}" -lq int.lz # first member "${LZIP}" -lq int.lz # first member
@ -380,7 +409,7 @@ if "${LZIP}" -tq int.lz ; then
[ $? = 2 ] || test_failed $LINENO ${header} [ $? = 2 ] || test_failed $LINENO ${header}
"${LZIP}" -cdq --loose-trailing int.lz > /dev/null "${LZIP}" -cdq --loose-trailing int.lz > /dev/null
[ $? = 2 ] || test_failed $LINENO ${header} [ $? = 2 ] || test_failed $LINENO ${header}
cat "${in_lz}" > int.lz || framework_failure cp "${in_lz}" int.lz || framework_failure
printf "${header}${body}" >> int.lz || framework_failure printf "${header}${body}" >> int.lz || framework_failure
"${LZIP}" -lq int.lz # trailing data "${LZIP}" -lq int.lz # trailing data
[ $? = 2 ] || test_failed $LINENO ${header} [ $? = 2 ] || test_failed $LINENO ${header}
@ -390,7 +419,7 @@ if "${LZIP}" -tq int.lz ; then
[ $? = 2 ] || test_failed $LINENO ${header} [ $? = 2 ] || test_failed $LINENO ${header}
"${LZIP}" -cdq int.lz > /dev/null "${LZIP}" -cdq int.lz > /dev/null
[ $? = 2 ] || test_failed $LINENO ${header} [ $? = 2 ] || test_failed $LINENO ${header}
"${LZIP}" -lq --loose-trailing int.lz || "${LZIP}" -l --loose-trailing int.lz > /dev/null ||
test_failed $LINENO ${header} test_failed $LINENO ${header}
"${LZIP}" -t --loose-trailing int.lz || "${LZIP}" -t --loose-trailing int.lz ||
test_failed $LINENO ${header} test_failed $LINENO ${header}
@ -408,10 +437,14 @@ if "${LZIP}" -tq int.lz ; then
[ $? = 2 ] || test_failed $LINENO ${header} [ $? = 2 ] || test_failed $LINENO ${header}
done done
else else
printf "\nwarning: skipping header test: 'printf' does not work on your system." printf "warning: skipping header test: 'printf' does not work on your system."
fi fi
rm -f int.lz || framework_failure rm -f int.lz || framework_failure
"${LZIP}" -l "${fnz_lz}" > /dev/null || test_failed $LINENO
"${LZIP}" -tq "${fnz_lz}"
[ $? = 2 ] || test_failed $LINENO
for i in fox_v2.lz fox_s11.lz fox_de20.lz \ for i in fox_v2.lz fox_s11.lz fox_de20.lz \
fox_bcrc.lz fox_crc0.lz fox_das46.lz fox_mes81.lz ; do fox_bcrc.lz fox_crc0.lz fox_das46.lz fox_mes81.lz ; do
"${LZIP}" -tq "${testdir}"/$i "${LZIP}" -tq "${testdir}"/$i
@ -423,13 +456,13 @@ for i in fox_bcrc.lz fox_crc0.lz fox_das46.lz fox_mes81.lz ; do
[ $? = 2 ] || test_failed $LINENO $i [ $? = 2 ] || test_failed $LINENO $i
cmp fox out || test_failed $LINENO $i cmp fox out || test_failed $LINENO $i
done done
rm -f fox out || framework_failure rm -f fox || framework_failure
cat "${in_lz}" "${in_lz}" > in2.lz || framework_failure cat "${in_lz}" "${in_lz}" > in2.lz || framework_failure
cat "${in_lz}" "${in_lz}" "${in_lz}" > in3.lz || framework_failure cat "${in_lz}" "${in_lz}" "${in_lz}" > in3.lz || framework_failure
if dd if=in3.lz of=trunc.lz bs=14752 count=1 2> /dev/null && if dd if=in3.lz of=trunc.lz bs=14682 count=1 2> /dev/null &&
[ -e trunc.lz ] && cmp in2.lz trunc.lz > /dev/null 2>&1 ; then [ -e trunc.lz ] && cmp in2.lz trunc.lz ; then
for i in 6 20 14734 14753 14754 14755 14756 14757 14758 ; do for i in 6 20 14664 14683 14684 14685 14686 14687 14688 ; do
dd if=in3.lz of=trunc.lz bs=$i count=1 2> /dev/null dd if=in3.lz of=trunc.lz bs=$i count=1 2> /dev/null
"${LZIP}" -lq trunc.lz "${LZIP}" -lq trunc.lz
[ $? = 2 ] || test_failed $LINENO $i [ $? = 2 ] || test_failed $LINENO $i
@ -443,11 +476,11 @@ if dd if=in3.lz of=trunc.lz bs=14752 count=1 2> /dev/null &&
[ $? = 2 ] || test_failed $LINENO $i [ $? = 2 ] || test_failed $LINENO $i
done done
else else
printf "\nwarning: skipping truncation test: 'dd' does not work on your system." printf "warning: skipping truncation test: 'dd' does not work on your system."
fi fi
rm -f in2.lz in3.lz trunc.lz || framework_failure rm -f in2.lz in3.lz trunc.lz || framework_failure
cat "${in_lz}" > ingin.lz || framework_failure cp "${in_lz}" ingin.lz || framework_failure
printf "g" >> ingin.lz || framework_failure printf "g" >> ingin.lz || framework_failure
cat "${in_lz}" >> ingin.lz || framework_failure cat "${in_lz}" >> ingin.lz || framework_failure
"${LZIP}" -lq ingin.lz "${LZIP}" -lq ingin.lz
@ -456,17 +489,21 @@ cat "${in_lz}" >> ingin.lz || framework_failure
[ $? = 2 ] || test_failed $LINENO [ $? = 2 ] || test_failed $LINENO
"${LZIP}" -atq < ingin.lz "${LZIP}" -atq < ingin.lz
[ $? = 2 ] || test_failed $LINENO [ $? = 2 ] || test_failed $LINENO
"${LZIP}" -acdq ingin.lz > /dev/null "${LZIP}" -acdq ingin.lz > out
[ $? = 2 ] || test_failed $LINENO [ $? = 2 ] || test_failed $LINENO
"${LZIP}" -adq < ingin.lz > /dev/null cmp in out || test_failed $LINENO
"${LZIP}" -adq < ingin.lz > out
[ $? = 2 ] || test_failed $LINENO [ $? = 2 ] || test_failed $LINENO
cmp in out || test_failed $LINENO
"${LZIP}" -t ingin.lz || test_failed $LINENO "${LZIP}" -t ingin.lz || test_failed $LINENO
"${LZIP}" -t < ingin.lz || test_failed $LINENO "${LZIP}" -t < ingin.lz || test_failed $LINENO
"${LZIP}" -dk ingin.lz || test_failed $LINENO
cmp in ingin || test_failed $LINENO
"${LZIP}" -cd ingin.lz > out || test_failed $LINENO "${LZIP}" -cd ingin.lz > out || test_failed $LINENO
cmp in out || test_failed $LINENO cmp in out || test_failed $LINENO
"${LZIP}" -d < ingin.lz > out || test_failed $LINENO "${LZIP}" -d < ingin.lz > out || test_failed $LINENO
cmp in out || test_failed $LINENO cmp in out || test_failed $LINENO
rm -f out ingin.lz || framework_failure rm -f out ingin ingin.lz || framework_failure
echo echo
if [ ${fail} = 0 ] ; then if [ ${fail} = 0 ] ; then

BIN
testsuite/fox6.lz
View file

Binary file not shown.

BIN
testsuite/fox6_mark.lz
View file

Binary file not shown.

BIN
testsuite/fox_nz.lz Normal file
View file

Binary file not shown.

6
testsuite/test.txt
View file

@ -1,8 +1,7 @@
GNU GENERAL PUBLIC LICENSE GNU GENERAL PUBLIC LICENSE
Version 2, June 1991 Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc., Copyright (C) 1989, 1991 Free Software Foundation, Inc. <http://fsf.org/>
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed. of this license document, but changing it is not allowed.
@ -339,8 +338,7 @@ Public License instead of this License.
GNU GENERAL PUBLIC LICENSE GNU GENERAL PUBLIC LICENSE
Version 2, June 1991 Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc., Copyright (C) 1989, 1991 Free Software Foundation, Inc. <http://fsf.org/>
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed. of this license document, but changing it is not allowed.

BIN
testsuite/test.txt.lz
View file

Binary file not shown.

BIN
testsuite/test_em.txt.lz
View file

Binary file not shown.