lzip/clzip
1
0
Fork 0
Code Issues Activity

Merging upstream version 1.15~rc1.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-17 20:54:18 +01:00
parent f99c7a8d1c
commit fe6e695444
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
31 changed files with 1415 additions and 1373 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
COPYING
View file

@ -1,8 +1,7 @@
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Copyright (C) 1989, 1991 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.

46
ChangeLog
View file

@ -1,3 +1,13 @@
2024-11-23 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.15-rc1 released.
* Remove options '--empty-error' and '--marking-error'.
* main.c (Pp_free): New function.
* decoder.c (LZd_decode_member): Remove support for Sync Flush marker.
* clzip.texi: New chapter 'Syntax of command-line arguments'.
* check.sh: Use 'cp' instead of 'cat'.
* testsuite: Add fox_nz.lz. Remove fox6.lz,fox6_mark.lz,test_em.txt.lz.
2024-01-22 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.14 released.
@ -27,12 +37,12 @@
Make '-o' behave like '-c', but writing to file instead of stdout.
Make '-c' and '-o' check whether the output is a terminal only once.
Do not open output if input is a terminal.
Set a valid invocation_name even if argc == 0.
* Replace 'decompressed', 'compressed' with 'out', 'in' in output.
* lzip_index.c: Improve messages for corruption in last header.
* main.c: Set a valid invocation_name even if argc == 0.
* Document extraction from tar.lz in manual, '--help', and man page.
* clzip.texi (Introduction): Mention plzip and tarlz as alternatives.
* clzip.texi: Several fixes and improvements.
Several fixes and improvements.
* testsuite: Add 9 new test files.
2019-01-03 Antonio Diaz Diaz <antonio@gnu.org>
@ -42,7 +52,7 @@
* lzip.h (Lzip_trailer): New function 'Lt_verify_consistency'.
* lzip_index.c: Detect some kinds of corrupt trailers.
* main.c (main): Check return value of close( infd ).
* main.c: Compile on DOS with DJGPP.
Compile on DOS with DJGPP. (Reported by Robert Riebisch).
* clzip.texi: Improve descriptions of '-0..-9', '-m', and '-s'.
* configure: Accept appending to CFLAGS; 'CFLAGS+=OPTIONS'.
* INSTALL: Document use of CFLAGS+='-D __USE_MINGW_ANSI_STDIO'.
@ -53,13 +63,13 @@
* New option '--loose-trailing'.
* Improve corrupt header detection to HD=3.
* main.c: Show corrupt or truncated header in multimember file.
* main.c (main): Option '-S, --volume-size' now keeps input files.
(main): Make option '-S, --volume-size' keep input files.
(main): Show final diagnostic when testing multiple files.
(set_c_outname): Do not add a second '.lz' to the arg of '-o'.
* encoder_base.*: Adjust dictionary size for each member.
* Replace 'bits/byte' with inverse compression ratio in output.
* Show progress of decompression at verbosity level 2 (-vv).
* Show progress of (de)compression only if stderr is a terminal.
* main.c: Show final diagnostic when testing multiple files.
* main.c: Do not add a second .lz extension to the arg of -o.
* decoder.c (LZd_verify_trailer): Show stored sizes also in hex.
Show dictionary size at verbosity level 4 (-vvvv).
* clzip.texi: New chapter 'Meaning of clzip's output'.
@ -73,7 +83,7 @@
* Compression time of options -1 to -9 has been reduced by 1%.
* Decompression time has been reduced by 7%.
* main.c: Continue testing if any input file is a terminal.
* main.c: Show trailing data in both hexadecimal and ASCII.
Show trailing data in both hexadecimal and ASCII.
* lzip_index.c: Improve detection of bad dict and trailing data.
* lzip.h: Unify messages for bad magic, trailing data, etc.
* clzip.texi: Add missing chapters from lzip.texi.
@ -82,16 +92,14 @@
* Version 1.8 released.
* New option '-a, --trailing-error'.
* main.c (decompress): Print up to 6 bytes of trailing data when
'-vvvv' is specified.
* decoder.c (LZd_verify_trailer): Remove test of final code.
* main.c (main): Delete '--output' file if infd is a terminal.
* main.c (main): Don't use stdin more than once.
(main): Don't use stdin more than once.
(decompress): Print 6 bytes of trailing data at verbosity level 4.
* decoder.c (LZd_verify_trailer): Remove test of final code.
* clzip.texi: New chapter 'Trailing data'.
* configure: Avoid warning on some shells when testing for gcc.
* Makefile.in: Detect the existence of install-info.
* check.sh: A POSIX shell is required to run the tests.
* check.sh: Don't check error messages.
* check.sh: Require a POSIX shell. Don't check error messages.
2015-07-07 Antonio Diaz Diaz <antonio@gnu.org>
@ -123,8 +131,8 @@
* Compression time has been reduced by 10%.
* Decompression time has been reduced by 8%.
* Makefile.in: New targets 'install-as-lzip' and 'install-bin'.
* main.c: Use 'setmode' instead of '_setmode' on Windows and OS/2.
* main.c: Define 'strtoull' to 'strtoul' on Windows.
(main): Use 'setmode' instead of '_setmode' on Windows and OS/2.
2012-02-25 Antonio Diaz Diaz <ant_diaz@teleline.es>
@ -141,29 +149,29 @@
* Version 1.2 released.
* New option '-F, --recompress'.
* main.c (decompress): Print only one status line for each
multimember file when only one '-v' is specified.
* encoder.h (Lee_update_prices): Update high length symbol prices
independently of the value of 'pos_state'. This gives better
compression for large values of '--match-length' without being
slower.
* encoder.h, encoder.c: Optimize pair price calculations, reducing
compression time for large values of '--match-length' by up to 6%.
* main.c (decompress): Print only one status line for each
multimember file when only one '-v' is specified.
2011-01-11 Antonio Diaz Diaz <ant_diaz@teleline.es>
* Version 1.1 released.
* Code has been converted to 'C89 + long long' from C99.
* main.c: Fix warning about fchown return value being ignored.
* decoder.c: '-tvvvv' now shows compression ratio.
* main.c: Match length limit set by options -1 to -8 has been
reduced to extend range of use towards gzip. Lower numbers now
compress less but faster. (-1 now takes 43% less time for only 20%
larger compressed size).
Exit with status 1 if any output file exists and is skipped.
* Compression ratio of option '-9' has been slightly increased.
* main.c (open_instream): Don't show the message
Fix warning about fchown's return value being ignored.
(open_instream): Don't show the message
" and '--stdout' was not specified" for directories, etc.
* Compression ratio of option '-9' has been slightly increased.
* New examples have been added to the manual.
2010-04-05 Antonio Diaz Diaz <ant_diaz@teleline.es>

10
Makefile.in
View file

@ -2,8 +2,8 @@
DISTNAME = $(pkgname)-$(pkgversion)
INSTALL = install
INSTALL_PROGRAM = $(INSTALL) -m 755
INSTALL_DATA = $(INSTALL) -m 644
INSTALL_DIR = $(INSTALL) -d -m 755
INSTALL_DATA = $(INSTALL) -m 644
SHELL = /bin/sh
CAN_RUN_INSTALLINFO = $(SHELL) -c "install-info --version" > /dev/null 2>&1
@ -31,7 +31,8 @@ main.o : main.c
# prevent 'make' from trying to remake source files
$(VPATH)/configure $(VPATH)/Makefile.in $(VPATH)/doc/$(pkgname).texi : ;
%.h %.c : ;
MAKEFLAGS += -r
.SUFFIXES :
$(objs) : Makefile
carg_parser.o : carg_parser.h
@ -130,10 +131,7 @@ dist : doc
$(DISTNAME)/testsuite/test.txt \
$(DISTNAME)/testsuite/fox.lz \
$(DISTNAME)/testsuite/fox_*.lz \
$(DISTNAME)/testsuite/fox6.lz \
$(DISTNAME)/testsuite/fox6_mark.lz \
$(DISTNAME)/testsuite/test.txt.lz \
$(DISTNAME)/testsuite/test_em.txt.lz
$(DISTNAME)/testsuite/test.txt.lz
rm -f $(DISTNAME)
lzip -v -9 $(DISTNAME).tar

27
NEWS
View file

@ -1,24 +1,11 @@
Changes in version 1.14:
Changes in version 1.15:
The option '--empty-error', which forces exit status 2 if any empty member
is found, has been added.
clzip now exits with error status 2 if any empty member is found in a
multimember file.
The option '--marking-error', which forces exit status 2 if the first LZMA
byte is non-zero in any member, has been added.
clzip now exits with error status 2 if the first byte of the LZMA stream is
not 0.
File diagnostics have been reformatted as 'PROGRAM: FILE: MESSAGE'.
Options '--empty-error' and '--marking-error' have been removed.
Diagnostics caused by invalid arguments to command-line options now show the
argument and the name of the option.
The option '-o, --output' now preserves dates, permissions, and ownership of
the file when (de)compressing exactly one file.
The option '-o, --output' now creates missing intermediate directories when
writing to a file.
The variable MAKEINFO has been added to configure and Makefile.in.
It has been documented in INSTALL that when choosing a C standard, the POSIX
features need to be enabled explicitly:
./configure CFLAGS+='--std=c99 -D_XOPEN_SOURCE=500'
The chapter 'Syntax of command-line arguments' has been added to the manual.

41
README
View file

@ -1,20 +1,21 @@
See the file INSTALL for compilation and installation instructions.
Description
Clzip is a C language version of lzip, compatible with lzip 1.4 or newer. As
clzip is written in C, it may be easier to integrate in applications like
package managers, embedded devices, or systems lacking a C++ compiler.
Clzip is a C language version of lzip intended for systems lacking a C++
compiler.
Lzip is a lossless data compressor with a user interface similar to the one
of gzip or bzip2. Lzip uses a simplified form of the 'Lempel-Ziv-Markov
chain-Algorithm' (LZMA) stream format to maximize interoperability. The
maximum dictionary size is 512 MiB so that any lzip file can be decompressed
on 32-bit machines. Lzip provides accurate and robust 3-factor integrity
checking. Lzip can compress about as fast as gzip (lzip -0) or compress most
files more than bzip2 (lzip -9). Decompression speed is intermediate between
gzip and bzip2. Lzip is better than gzip and bzip2 from a data recovery
perspective. Lzip has been designed, written, and tested with great care to
replace gzip and bzip2 as the standard general-purpose compressed format for
Unix-like systems.
of gzip or bzip2. Lzip uses a simplified form of LZMA (Lempel-Ziv-Markov
chain-Algorithm) designed to achieve complete interoperability between
implementations. The maximum dictionary size is 512 MiB so that any lzip
file can be decompressed on 32-bit machines. Lzip provides accurate and
robust 3-factor integrity checking. 'lzip -0' compresses about as fast as
gzip, while 'lzip -9' compresses most files more than bzip2. Decompression
speed is intermediate between gzip and bzip2. Lzip provides better data
recovery capabilities than gzip and bzip2. Lzip has been designed, written,
and tested with great care to replace gzip and bzip2 as general-purpose
compressed format for Unix-like systems.
For compressing/decompressing large files on multiprocessor machines plzip
can be much faster than lzip at the cost of a slightly reduced compression
@ -95,7 +96,6 @@ also supported.
Clzip can produce multimember files, and lziprecover can safely recover the
undamaged members in case of file damage. Clzip can also split the compressed
output in volumes of a given size, even when reading from standard input.
This allows the direct creation of multivolume compressed tar archives.
Clzip is able to compress and decompress streams of unlimited size by
automatically creating multimember output. The members so created are large,
@ -105,16 +105,16 @@ In spite of its name (Lempel-Ziv-Markov chain-Algorithm), LZMA is not a
concrete algorithm; it is more like "any algorithm using the LZMA coding
scheme". For example, the option '-0' of lzip uses the scheme in almost the
simplest way possible; issuing the longest match it can find, or a literal
byte if it can't find a match. Inversely, a much more elaborated way of
finding coding sequences of minimum size than the one currently used by lzip
could be developed, and the resulting sequence could also be coded using the
LZMA coding scheme.
byte if it can't find a match. Inversely, a more elaborate way of finding
coding sequences of minimum size than the one currently used by lzip could
be developed, and the resulting sequence could also be coded using the LZMA
coding scheme.
Clzip currently implements two variants of the LZMA algorithm: fast
(used by option '-0') and normal (used by all other compression levels).
The high compression of LZMA comes from combining two basic, well-proven
compression ideas: sliding dictionaries (LZ77) and markov models (the thing
compression ideas: sliding dictionaries (LZ77) and Markov models (the thing
used by every compression algorithm that uses a range encoder or similar
order-0 entropy coder as its last stage) with segregation of contexts
according to what the bits are used for.
@ -125,6 +125,9 @@ definition of Markov chains), G.N.N. Martin (for the definition of range
encoding), Igor Pavlov (for putting all the above together in LZMA), and
Julian Seward (for bzip2's CLI).
Clzip uses Arg_parser for command-line argument parsing:
http://www.nongnu.org/arg-parser/arg_parser.html
LANGUAGE NOTE: Uncompressed = not compressed = plain data; it may never have
been compressed. Decompressed is used to refer to data which have undergone
the process of decompression.

60
carg_parser.c
View file

@ -32,15 +32,15 @@ static void * ap_resize_buffer( void * buf, const int min_size )
}
static char push_back_record( struct Arg_parser * const ap, const int code,
static char push_back_record( Arg_parser * const ap, const int code,
const char * const long_name,
const char * const argument )
{
struct ap_Record * p;
ap_Record * p;
void * tmp = ap_resize_buffer( ap->data,
( ap->data_size + 1 ) * sizeof (struct ap_Record) );
( ap->data_size + 1 ) * sizeof (ap_Record) );
if( !tmp ) return 0;
ap->data = (struct ap_Record *)tmp;
ap->data = (ap_Record *)tmp;
p = &(ap->data[ap->data_size]);
p->code = code;
if( long_name )
@ -71,7 +71,7 @@ static char push_back_record( struct Arg_parser * const ap, const int code,
}
static char add_error( struct Arg_parser * const ap, const char * const msg )
static char add_error( Arg_parser * const ap, const char * const msg )
{
const int len = strlen( msg );
void * tmp = ap_resize_buffer( ap->error, ap->error_size + len + 1 );
@ -83,7 +83,7 @@ static char add_error( struct Arg_parser * const ap, const char * const msg )
}
static void free_data( struct Arg_parser * const ap )
static void free_data( Arg_parser * const ap )
{
int i;
for( i = 0; i < ap->data_size; ++i )
@ -94,10 +94,9 @@ static void free_data( struct Arg_parser * const ap )
/* Return 0 only if out of memory. */
static char parse_long_option( struct Arg_parser * const ap,
static char parse_long_option( Arg_parser * const ap,
const char * const opt, const char * const arg,
const struct ap_Option options[],
int * const argindp )
const ap_Option options[], int * const argindp )
{
unsigned len;
int index = -1, i;
@ -148,21 +147,21 @@ static char parse_long_option( struct Arg_parser * const ap,
add_error( ap, "' requires an argument" );
return 1;
}
return push_back_record( ap, options[index].code,
options[index].long_name, &opt[len+3] );
return push_back_record( ap, options[index].code, options[index].long_name,
&opt[len+3] ); /* argument may be empty */
}
if( options[index].has_arg == ap_yes )
if( options[index].has_arg == ap_yes || options[index].has_arg == ap_yme )
{
if( !arg || !arg[0] )
if( !arg || ( options[index].has_arg == ap_yes && !arg[0] ) )
{
add_error( ap, "option '--" ); add_error( ap, options[index].long_name );
add_error( ap, "' requires an argument" );
return 1;
}
++*argindp;
return push_back_record( ap, options[index].code,
options[index].long_name, arg );
return push_back_record( ap, options[index].code, options[index].long_name,
arg ); /* argument may be empty */
}
return push_back_record( ap, options[index].code,
@ -171,10 +170,9 @@ static char parse_long_option( struct Arg_parser * const ap,
/* Return 0 only if out of memory. */
static char parse_short_option( struct Arg_parser * const ap,
static char parse_short_option( Arg_parser * const ap,
const char * const opt, const char * const arg,
const struct ap_Option options[],
int * const argindp )
const ap_Option options[], int * const argindp )
{
int cind = 1; /* character index in opt */
@ -204,15 +202,15 @@ static char parse_short_option( struct Arg_parser * const ap,
if( !push_back_record( ap, c, 0, &opt[cind] ) ) return 0;
++*argindp; cind = 0;
}
else if( options[index].has_arg == ap_yes )
else if( options[index].has_arg == ap_yes || options[index].has_arg == ap_yme )
{
if( !arg || !arg[0] )
if( !arg || ( options[index].has_arg == ap_yes && !arg[0] ) )
{
add_error( ap, "option requires an argument -- '" );
add_error( ap, code_str ); add_error( ap, "'" );
return 1;
}
++*argindp; cind = 0;
++*argindp; cind = 0; /* argument may be empty */
if( !push_back_record( ap, c, 0, arg ) ) return 0;
}
else if( !push_back_record( ap, c, 0, 0 ) ) return 0;
@ -221,9 +219,9 @@ static char parse_short_option( struct Arg_parser * const ap,
}
char ap_init( struct Arg_parser * const ap,
char ap_init( Arg_parser * const ap,
const int argc, const char * const argv[],
const struct ap_Option options[], const char in_order )
const ap_Option options[], const char in_order )
{
const char ** non_options = 0; /* skipped non-options */
int non_options_size = 0; /* number of skipped non-options */
@ -282,7 +280,7 @@ out: if( non_options ) free( non_options );
}
void ap_free( struct Arg_parser * const ap )
void ap_free( Arg_parser * const ap )
{
free_data( ap );
if( ap->error ) { free( ap->error ); ap->error = 0; }
@ -290,29 +288,25 @@ void ap_free( struct Arg_parser * const ap )
}
const char * ap_error( const struct Arg_parser * const ap )
{ return ap->error; }
const char * ap_error( const Arg_parser * const ap ) { return ap->error; }
int ap_arguments( const Arg_parser * const ap ) { return ap->data_size; }
int ap_arguments( const struct Arg_parser * const ap )
{ return ap->data_size; }
int ap_code( const struct Arg_parser * const ap, const int i )
int ap_code( const Arg_parser * const ap, const int i )
{
if( i < 0 || i >= ap_arguments( ap ) ) return 0;
return ap->data[i].code;
}
const char * ap_parsed_name( const struct Arg_parser * const ap, const int i )
const char * ap_parsed_name( const Arg_parser * const ap, const int i )
{
if( i < 0 || i >= ap_arguments( ap ) || !ap->data[i].parsed_name ) return "";
return ap->data[i].parsed_name;
}
const char * ap_argument( const struct Arg_parser * const ap, const int i )
const char * ap_argument( const Arg_parser * const ap, const int i )
{
if( i < 0 || i >= ap_arguments( ap ) || !ap->data[i].argument ) return "";
return ap->data[i].argument;

43
carg_parser.h
View file

@ -37,60 +37,65 @@
The argument '--' terminates all options; any following arguments are
treated as non-option arguments, even if they begin with a hyphen.
The syntax for optional option arguments is '-<short_option><argument>'
(without whitespace), or '--<long_option>=<argument>'.
The syntax of options with an optional argument is
'-<short_option><argument>' (without whitespace), or
'--<long_option>=<argument>'.
The syntax of options with an empty argument is '-<short_option> ""',
'--<long_option> ""', or '--<long_option>=""'.
*/
#ifdef __cplusplus
extern "C" {
#endif
enum ap_Has_arg { ap_no, ap_yes, ap_maybe };
/* ap_yme = yes but maybe empty */
typedef enum ap_Has_arg { ap_no, ap_yes, ap_maybe, ap_yme } ap_Has_arg;
struct ap_Option
typedef struct ap_Option
{
int code; /* Short option letter or code ( code != 0 ) */
const char * long_name; /* Long option name (maybe null) */
enum ap_Has_arg has_arg;
};
ap_Has_arg has_arg;
} ap_Option;
struct ap_Record
typedef struct ap_Record
{
int code;
char * parsed_name;
char * argument;
};
} ap_Record;
struct Arg_parser
typedef struct Arg_parser
{
struct ap_Record * data;
ap_Record * data;
char * error;
int data_size;
int error_size;
};
} Arg_parser;
char ap_init( struct Arg_parser * const ap,
char ap_init( Arg_parser * const ap,
const int argc, const char * const argv[],
const struct ap_Option options[], const char in_order );
const ap_Option options[], const char in_order );
void ap_free( struct Arg_parser * const ap );
void ap_free( Arg_parser * const ap );
const char * ap_error( const struct Arg_parser * const ap );
const char * ap_error( const Arg_parser * const ap );
/* The number of arguments parsed. May be different from argc. */
int ap_arguments( const struct Arg_parser * const ap );
int ap_arguments( const Arg_parser * const ap );
/* If ap_code( i ) is 0, ap_argument( i ) is a non-option.
Else ap_argument( i ) is the option's argument (or empty). */
int ap_code( const struct Arg_parser * const ap, const int i );
int ap_code( const Arg_parser * const ap, const int i );
/* Full name of the option parsed (short or long). */
const char * ap_parsed_name( const struct Arg_parser * const ap, const int i );
const char * ap_parsed_name( const Arg_parser * const ap, const int i );
const char * ap_argument( const struct Arg_parser * const ap, const int i );
const char * ap_argument( const Arg_parser * const ap, const int i );
#ifdef __cplusplus
}

4
configure vendored
View file

@ -6,7 +6,7 @@
# to copy, distribute, and modify it.
pkgname=clzip
pkgversion=1.14
pkgversion=1.15-rc1
progname=clzip
srctrigger=doc/${pkgname}.texi
@ -109,7 +109,7 @@ while [ $# != 0 ] ; do
exit 1 ;;
esac
# Check if the option took a separate argument
# Check whether the option took a separate argument
if [ "${arg2}" = yes ] ; then
if [ $# != 0 ] ; then args="${args} \"$1\"" ; shift
else echo "configure: Missing argument to '${option}'" 1>&2

63
decoder.c
View file

@ -66,14 +66,14 @@ int writeblock( const int fd, const uint8_t * const buf, const int size )
}
bool Rd_read_block( struct Range_decoder * const rdec )
bool Rd_read_block( Range_decoder * const rdec )
{
if( !rdec->at_stream_end )
{
rdec->stream_pos = readblock( rdec->infd, rdec->buffer, rd_buffer_size );
if( rdec->stream_pos != rd_buffer_size && errno )
{ show_error( "Read error", errno, false ); cleanup_and_fail( 1 ); }
rdec->at_stream_end = ( rdec->stream_pos < rd_buffer_size );
rdec->at_stream_end = rdec->stream_pos < rd_buffer_size;
rdec->partial_member_pos += rdec->pos;
rdec->pos = 0;
show_dprogress( 0, 0, 0, 0 );
@ -82,7 +82,7 @@ bool Rd_read_block( struct Range_decoder * const rdec )
}
void LZd_flush_data( struct LZ_decoder * const d )
void LZd_flush_data( LZ_decoder * const d )
{
if( d->pos > d->stream_pos )
{
@ -90,7 +90,7 @@ void LZd_flush_data( struct LZ_decoder * const d )
CRC32_update_buf( &d->crc, d->buffer + d->stream_pos, size );
if( d->outfd >= 0 &&
writeblock( d->outfd, d->buffer + d->stream_pos, size ) != size )
{ show_error( "Write error", errno, false ); cleanup_and_fail( 1 ); }
{ show_error( write_error_msg, errno, false ); cleanup_and_fail( 1 ); }
if( d->pos >= d->dictionary_size )
{ d->partial_data_pos += d->pos; d->pos = 0; d->pos_wrapped = true; }
d->stream_pos = d->pos;
@ -98,9 +98,7 @@ void LZd_flush_data( struct LZ_decoder * const d )
}
static int LZd_check_trailer( struct LZ_decoder * const d,
struct Pretty_print * const pp,
const bool ignore_empty )
static bool LZd_check_trailer( LZ_decoder * const d, Pretty_print * const pp )
{
Lzip_trailer trailer;
int size = Rd_read_data( d->rdec, trailer, Lt_size );
@ -145,8 +143,7 @@ static int LZd_check_trailer( struct LZ_decoder * const d,
fprintf( stderr, "Member size mismatch; stored %llu (0x%llX), computed %llu (0x%llX)\n",
tm_size, tm_size, member_size, member_size ); }
}
if( error ) return 3;
if( !ignore_empty && data_size == 0 ) return 5;
if( error ) return false;
if( verbosity >= 2 )
{
if( verbosity >= 4 ) show_header( d->dictionary_size );
@ -161,18 +158,16 @@ static int LZd_check_trailer( struct LZ_decoder * const d,
if( verbosity >= 3 )
fprintf( stderr, "%9llu out, %8llu in. ", data_size, member_size );
}
return 0;
return true;
}
/* Return value: 0 = OK, 1 = decoder error, 2 = unexpected EOF,
3 = trailer error, 4 = unknown marker found,
5 = empty member found, 6 = marked member found. */
int LZd_decode_member( struct LZ_decoder * const d,
const struct Cl_options * const cl_opts,
struct Pretty_print * const pp )
5 = nonzero first LZMA byte found. */
int LZd_decode_member( LZ_decoder * const d, Pretty_print * const pp )
{
struct Range_decoder * const rdec = d->rdec;
Range_decoder * const rdec = d->rdec;
Bit_model bm_literal[1<<literal_context_bits][0x300];
Bit_model bm_match[states][pos_states];
Bit_model bm_rep[states];
@ -183,8 +178,8 @@ int LZd_decode_member( struct LZ_decoder * const d,
Bit_model bm_dis_slot[len_states][1<<dis_slot_bits];
Bit_model bm_dis[modeled_distances-end_dis_model+1];
Bit_model bm_align[dis_align_size];
struct Len_model match_len_model;
struct Len_model rep_len_model;
Len_model match_len_model;
Len_model rep_len_model;
unsigned rep0 = 0; /* rep[0-3] latest four distances */
unsigned rep1 = 0; /* used for efficient coding of */
unsigned rep2 = 0; /* repeated distances */
@ -204,7 +199,7 @@ int LZd_decode_member( struct LZ_decoder * const d,
Lm_init( &match_len_model );
Lm_init( &rep_len_model );
if( !Rd_load( rdec, cl_opts->ignore_marking ) ) return 6;
if( !Rd_load( rdec ) ) return 5;
while( !Rd_finished( rdec ) )
{
const int pos_state = LZd_data_position( d ) & pos_state_mask;
@ -249,39 +244,33 @@ int LZd_decode_member( struct LZ_decoder * const d,
}
else /* match */
{
rep3 = rep2; rep2 = rep1; rep1 = rep0;
len = Rd_decode_len( rdec, &match_len_model, pos_state );
unsigned distance = Rd_decode_tree6( rdec, bm_dis_slot[get_len_state(len)] );
if( distance >= start_dis_model )
rep0 = Rd_decode_tree6( rdec, bm_dis_slot[get_len_state(len)] );
if( rep0 >= start_dis_model )
{
const unsigned dis_slot = distance;
const unsigned dis_slot = rep0;
const int direct_bits = ( dis_slot >> 1 ) - 1;
distance = ( 2 | ( dis_slot & 1 ) ) << direct_bits;
rep0 = ( 2 | ( dis_slot & 1 ) ) << direct_bits;
if( dis_slot < end_dis_model )
distance += Rd_decode_tree_reversed( rdec,
bm_dis + ( distance - dis_slot ), direct_bits );
rep0 += Rd_decode_tree_reversed( rdec, bm_dis + ( rep0 - dis_slot ),
direct_bits );
else
{
distance +=
Rd_decode( rdec, direct_bits - dis_align_bits ) << dis_align_bits;
distance += Rd_decode_tree_reversed4( rdec, bm_align );
if( distance == 0xFFFFFFFFU ) /* marker found */
rep0 += Rd_decode( rdec, direct_bits - dis_align_bits ) << dis_align_bits;
rep0 += Rd_decode_tree_reversed4( rdec, bm_align );
if( rep0 == 0xFFFFFFFFU ) /* marker found */
{
Rd_normalize( rdec );
LZd_flush_data( d );
if( len == min_match_len ) /* End Of Stream marker */
return LZd_check_trailer( d, pp, cl_opts->ignore_empty );
if( len == min_match_len + 1 ) /* Sync Flush marker */
{ Rd_load( rdec, true ); continue; }
if( verbosity >= 0 )
{
Pp_show_msg( pp, 0 );
fprintf( stderr, "Unsupported marker code '%d'\n", len );
}
{ if( LZd_check_trailer( d, pp ) ) return 0; else return 3; }
if( verbosity >= 0 ) { Pp_show_msg( pp, 0 );
fprintf( stderr, "Unsupported marker code '%d'\n", len ); }
return 4;
}
}
}
rep3 = rep2; rep2 = rep1; rep1 = rep0; rep0 = distance;
state = St_set_match( state );
if( rep0 >= d->dictionary_size || ( rep0 >= d->pos && !d->pos_wrapped ) )
{ LZd_flush_data( d ); return 1; }

93
decoder.h
View file

@ -29,9 +29,9 @@ struct Range_decoder
bool at_stream_end;
};
bool Rd_read_block( struct Range_decoder * const rdec );
bool Rd_read_block( Range_decoder * const rdec );
static inline bool Rd_init( struct Range_decoder * const rdec, const int ifd )
static inline bool Rd_init( Range_decoder * const rdec, const int ifd )
{
rdec->partial_member_pos = 0;
rdec->buffer = (uint8_t *)malloc( rd_buffer_size );
@ -45,27 +45,27 @@ static inline bool Rd_init( struct Range_decoder * const rdec, const int ifd )
return true;
}
static inline void Rd_free( struct Range_decoder * const rdec )
static inline void Rd_free( Range_decoder * const rdec )
{ free( rdec->buffer ); }
static inline bool Rd_finished( struct Range_decoder * const rdec )
static inline bool Rd_finished( Range_decoder * const rdec )
{ return rdec->pos >= rdec->stream_pos && !Rd_read_block( rdec ); }
static inline unsigned long long
Rd_member_position( const struct Range_decoder * const rdec )
Rd_member_position( const Range_decoder * const rdec )
{ return rdec->partial_member_pos + rdec->pos; }
static inline void Rd_reset_member_position( struct Range_decoder * const rdec )
static inline void Rd_reset_member_position( Range_decoder * const rdec )
{ rdec->partial_member_pos = 0; rdec->partial_member_pos -= rdec->pos; }
static inline uint8_t Rd_get_byte( struct Range_decoder * const rdec )
static inline uint8_t Rd_get_byte( Range_decoder * const rdec )
{
/* 0xFF avoids decoder error if member is truncated at EOS marker */
if( Rd_finished( rdec ) ) return 0xFF;
return rdec->buffer[rdec->pos++];
}
static inline int Rd_read_data( struct Range_decoder * const rdec,
static inline int Rd_read_data( Range_decoder * const rdec,
uint8_t * const outbuf, const int size )
{
int sz = 0;
@ -79,25 +79,24 @@ static inline int Rd_read_data( struct Range_decoder * const rdec,
return sz;
}
static inline bool Rd_load( struct Range_decoder * const rdec,
const bool ignore_marking )
static inline bool Rd_load( Range_decoder * const rdec )
{
int i;
rdec->code = 0;
rdec->range = 0xFFFFFFFFU;
/* check and discard first byte of the LZMA stream */
if( Rd_get_byte( rdec ) != 0 && !ignore_marking ) return false;
for( i = 0; i < 4; ++i ) rdec->code = (rdec->code << 8) | Rd_get_byte( rdec );
/* check first byte of the LZMA stream */
if( Rd_get_byte( rdec ) != 0 ) return false;
int i; for( i = 0; i < 4; ++i )
rdec->code = (rdec->code << 8) | Rd_get_byte( rdec );
return true;
}
static inline void Rd_normalize( struct Range_decoder * const rdec )
static inline void Rd_normalize( Range_decoder * const rdec )
{
if( rdec->range <= 0x00FFFFFFU )
{ rdec->range <<= 8; rdec->code = (rdec->code << 8) | Rd_get_byte( rdec ); }
}
static inline unsigned Rd_decode( struct Range_decoder * const rdec,
static inline unsigned Rd_decode( Range_decoder * const rdec,
const int num_bits )
{
unsigned symbol = 0;
@ -108,14 +107,14 @@ static inline unsigned Rd_decode( struct Range_decoder * const rdec,
rdec->range >>= 1;
/* symbol <<= 1; */
/* if( rdec->code >= rdec->range ) { rdec->code -= rdec->range; symbol |= 1; } */
const bool bit = ( rdec->code >= rdec->range );
const bool bit = rdec->code >= rdec->range;
symbol <<= 1; symbol += bit;
rdec->code -= rdec->range & ( 0U - bit );
}
return symbol;
}
static inline unsigned Rd_decode_bit( struct Range_decoder * const rdec,
static inline unsigned Rd_decode_bit( Range_decoder * const rdec,
Bit_model * const probability )
{
Rd_normalize( rdec );
@ -135,7 +134,7 @@ static inline unsigned Rd_decode_bit( struct Range_decoder * const rdec,
}
}
static inline void Rd_decode_symbol_bit( struct Range_decoder * const rdec,
static inline void Rd_decode_symbol_bit( Range_decoder * const rdec,
Bit_model * const probability, unsigned * symbol )
{
Rd_normalize( rdec );
@ -155,7 +154,7 @@ static inline void Rd_decode_symbol_bit( struct Range_decoder * const rdec,
}
}
static inline void Rd_decode_symbol_bit_reversed( struct Range_decoder * const rdec,
static inline void Rd_decode_symbol_bit_reversed( Range_decoder * const rdec,
Bit_model * const probability, unsigned * model,
unsigned * symbol, const int i )
{
@ -177,7 +176,7 @@ static inline void Rd_decode_symbol_bit_reversed( struct Range_decoder * const r
}
}
static inline unsigned Rd_decode_tree6( struct Range_decoder * const rdec,
static inline unsigned Rd_decode_tree6( Range_decoder * const rdec,
Bit_model bm[] )
{
unsigned symbol = 1;
@ -190,7 +189,7 @@ static inline unsigned Rd_decode_tree6( struct Range_decoder * const rdec,
return symbol & 0x3F;
}
static inline unsigned Rd_decode_tree8( struct Range_decoder * const rdec,
static inline unsigned Rd_decode_tree8( Range_decoder * const rdec,
Bit_model bm[] )
{
unsigned symbol = 1;
@ -205,8 +204,7 @@ static inline unsigned Rd_decode_tree8( struct Range_decoder * const rdec,
return symbol & 0xFF;
}
static inline unsigned
Rd_decode_tree_reversed( struct Range_decoder * const rdec,
static inline unsigned Rd_decode_tree_reversed( Range_decoder * const rdec,
Bit_model bm[], const int num_bits )
{
unsigned model = 1;
@ -218,7 +216,7 @@ Rd_decode_tree_reversed( struct Range_decoder * const rdec,
}
static inline unsigned
Rd_decode_tree_reversed4( struct Range_decoder * const rdec, Bit_model bm[] )
Rd_decode_tree_reversed4( Range_decoder * const rdec, Bit_model bm[] )
{
unsigned model = 1;
unsigned symbol = 0;
@ -229,7 +227,7 @@ Rd_decode_tree_reversed4( struct Range_decoder * const rdec, Bit_model bm[] )
return symbol;
}
static inline unsigned Rd_decode_matched( struct Range_decoder * const rdec,
static inline unsigned Rd_decode_matched( Range_decoder * const rdec,
Bit_model bm[], unsigned match_byte )
{
unsigned symbol = 1;
@ -244,9 +242,8 @@ static inline unsigned Rd_decode_matched( struct Range_decoder * const rdec,
}
}
static inline unsigned Rd_decode_len( struct Range_decoder * const rdec,
struct Len_model * const lm,
const int pos_state )
static inline unsigned Rd_decode_len( Range_decoder * const rdec,
Len_model * const lm, const int pos_state )
{
Bit_model * bm;
unsigned mask, offset, symbol = 1;
@ -269,10 +266,10 @@ len3:
}
struct LZ_decoder
typedef struct LZ_decoder
{
unsigned long long partial_data_pos;
struct Range_decoder * rdec;
Range_decoder * rdec;
unsigned dictionary_size;
uint8_t * buffer; /* output buffer */
unsigned pos; /* current pos in buffer */
@ -280,14 +277,14 @@ struct LZ_decoder
uint32_t crc;
int outfd; /* output file descriptor */
bool pos_wrapped;
};
} LZ_decoder;
void LZd_flush_data( struct LZ_decoder * const d );
void LZd_flush_data( LZ_decoder * const d );
static inline uint8_t LZd_peek_prev( const struct LZ_decoder * const d )
static inline uint8_t LZd_peek_prev( const LZ_decoder * const d )
{ return d->buffer[((d->pos > 0) ? d->pos : d->dictionary_size)-1]; }
static inline uint8_t LZd_peek( const struct LZ_decoder * const d,
static inline uint8_t LZd_peek( const LZ_decoder * const d,
const unsigned distance )
{
const unsigned i = ( ( d->pos > distance ) ? 0 : d->dictionary_size ) +
@ -295,27 +292,27 @@ static inline uint8_t LZd_peek( const struct LZ_decoder * const d,
return d->buffer[i];
}
static inline void LZd_put_byte( struct LZ_decoder * const d, const uint8_t b )
static inline void LZd_put_byte( LZ_decoder * const d, const uint8_t b )
{
d->buffer[d->pos] = b;
if( ++d->pos >= d->dictionary_size ) LZd_flush_data( d );
}
static inline void LZd_copy_block( struct LZ_decoder * const d,
static inline void LZd_copy_block( LZ_decoder * const d,
const unsigned distance, unsigned len )
{
unsigned lpos = d->pos, i = lpos - distance - 1;
bool fast, fast2;
if( lpos > distance )
{
fast = ( len < d->dictionary_size - lpos );
fast2 = ( fast && len <= lpos - i );
fast = len < d->dictionary_size - lpos;
fast2 = fast && len <= lpos - i;
}
else
{
i += d->dictionary_size;
fast = ( len < d->dictionary_size - i ); /* (i == pos) may happen */
fast2 = ( fast && len <= i - lpos );
fast = len < d->dictionary_size - i; /* (i == pos) may happen */
fast2 = fast && len <= i - lpos;
}
if( fast ) /* no wrap */
{
@ -333,8 +330,7 @@ static inline void LZd_copy_block( struct LZ_decoder * const d,
}
}
static inline bool LZd_init( struct LZ_decoder * const d,
struct Range_decoder * const rde,
static inline bool LZd_init( LZ_decoder * const d, Range_decoder * const rde,
const unsigned dict_size, const int ofd )
{
d->partial_data_pos = 0;
@ -352,16 +348,13 @@ static inline bool LZd_init( struct LZ_decoder * const d,
return true;
}
static inline void LZd_free( struct LZ_decoder * const d )
static inline void LZd_free( LZ_decoder * const d )
{ free( d->buffer ); }
static inline unsigned LZd_crc( const struct LZ_decoder * const d )
static inline unsigned LZd_crc( const LZ_decoder * const d )
{ return d->crc ^ 0xFFFFFFFFU; }
static inline unsigned long long
LZd_data_position( const struct LZ_decoder * const d )
static inline unsigned long long LZd_data_position( const LZ_decoder * const d )
{ return d->partial_data_pos + d->pos; }
int LZd_decode_member( struct LZ_decoder * const d,
const struct Cl_options * const cl_opts,
struct Pretty_print * const pp );
int LZd_decode_member( LZ_decoder * const d, Pretty_print * const pp );

35
doc/clzip.1
View file

@ -1,26 +1,25 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.2.
.TH CLZIP "1" "January 2024" "clzip 1.14" "User Commands"
.TH CLZIP "1" "November 2024" "clzip 1.15-rc1" "User Commands"
.SH NAME
clzip \- reduces the size of files
.SH SYNOPSIS
.B clzip
[\fI\,options\/\fR] [\fI\,files\/\fR]
.SH DESCRIPTION
Clzip is a C language version of lzip, compatible with lzip 1.4 or newer. As
clzip is written in C, it may be easier to integrate in applications like
package managers, embedded devices, or systems lacking a C++ compiler.
Clzip is a C language version of lzip intended for systems lacking a C++
compiler.
.PP
Lzip is a lossless data compressor with a user interface similar to the one
of gzip or bzip2. Lzip uses a simplified form of the 'Lempel\-Ziv\-Markov
chain\-Algorithm' (LZMA) stream format to maximize interoperability. The
maximum dictionary size is 512 MiB so that any lzip file can be decompressed
on 32\-bit machines. Lzip provides accurate and robust 3\-factor integrity
checking. Lzip can compress about as fast as gzip (lzip \fB\-0\fR) or compress most
files more than bzip2 (lzip \fB\-9\fR). Decompression speed is intermediate between
gzip and bzip2. Lzip is better than gzip and bzip2 from a data recovery
perspective. Lzip has been designed, written, and tested with great care to
replace gzip and bzip2 as the standard general\-purpose compressed format for
Unix\-like systems.
of gzip or bzip2. Lzip uses a simplified form of LZMA (Lempel\-Ziv\-Markov
chain\-Algorithm) designed to achieve complete interoperability between
implementations. The maximum dictionary size is 512 MiB so that any lzip
file can be decompressed on 32\-bit machines. Lzip provides accurate and
robust 3\-factor integrity checking. 'lzip \fB\-0\fR' compresses about as fast as
gzip, while 'lzip \fB\-9\fR' compresses most files more than bzip2. Decompression
speed is intermediate between gzip and bzip2. Lzip provides better data
recovery capabilities than gzip and bzip2. Lzip has been designed, written,
and tested with great care to replace gzip and bzip2 as general\-purpose
compressed format for Unix\-like systems.
.SH OPTIONS
.TP
\fB\-h\fR, \fB\-\-help\fR
@ -33,7 +32,7 @@ output version information and exit
exit with error status if trailing data
.TP
\fB\-b\fR, \fB\-\-member\-size=\fR<bytes>
set member size limit in bytes
set member size limit of multimember files
.TP
\fB\-c\fR, \fB\-\-stdout\fR
write to standard output, keep input files
@ -83,12 +82,6 @@ alias for \fB\-0\fR
\fB\-\-best\fR
alias for \fB\-9\fR
.TP
\fB\-\-empty\-error\fR
exit with error status if empty member in file
.TP
\fB\-\-marking\-error\fR
exit with error status if 1st LZMA byte not 0
.TP
\fB\-\-loose\-trailing\fR
allow trailing data seeming corrupt header
.PP

753
doc/clzip.info
View file

File diff suppressed because it is too large Load diff

751
doc/clzip.texi
View file

File diff suppressed because it is too large Load diff

23
encoder.c
View file

@ -31,7 +31,7 @@
CRC32 crc32;
int LZe_get_match_pairs( struct LZ_encoder * const e, struct Pair * pairs )
int LZe_get_match_pairs( LZ_encoder * const e, Pair * pairs )
{
int len_limit = e->match_len_limit;
if( len_limit > Mb_available_bytes( &e->eb.mb ) )
@ -133,7 +133,7 @@ int LZe_get_match_pairs( struct LZ_encoder * const e, struct Pair * pairs )
}
static void LZe_update_distance_prices( struct LZ_encoder * const e )
static void LZe_update_distance_prices( LZ_encoder * const e )
{
int dis, len_state;
for( dis = start_dis_model; dis < modeled_distances; ++dis )
@ -172,7 +172,7 @@ static void LZe_update_distance_prices( struct LZ_encoder * const e )
( trials[0].dis4 == -1 ) means literal.
A match/rep longer or equal than match_len_limit finishes the sequence.
*/
static int LZe_sequence_optimizer( struct LZ_encoder * const e,
static int LZe_sequence_optimizer( LZ_encoder * const e,
const int reps[num_rep_distances],
const State state )
{
@ -291,7 +291,7 @@ static int LZe_sequence_optimizer( struct LZ_encoder * const e,
}
/* give final values to current trial */
struct Trial * cur_trial = &e->trials[cur];
Trial * cur_trial = &e->trials[cur];
State cur_state;
{
const int dis4 = cur_trial->dis4;
@ -336,7 +336,7 @@ static int LZe_sequence_optimizer( struct LZ_encoder * const e,
next_price += LZeb_price_matched( &e->eb, prev_byte, cur_byte, match_byte );
/* try last updates to next trial */
struct Trial * next_trial = &e->trials[cur+1];
Trial * next_trial = &e->trials[cur+1];
Tr_update( next_trial, next_price, -1, cur ); /* literal */
@ -346,8 +346,7 @@ static int LZe_sequence_optimizer( struct LZ_encoder * const e,
if( match_byte == cur_byte && next_trial->dis4 != 0 &&
next_trial->prev_index2 == single_step_trial )
{
const int price = rep_match_price +
LZeb_price_shortrep( &e->eb, cur_state, pos_state );
const int price = rep_match_price + LZeb_price_shortrep( &e->eb, cur_state, pos_state );
if( price <= next_trial->price )
{
next_trial->price = price;
@ -478,12 +477,12 @@ static int LZe_sequence_optimizer( struct LZ_encoder * const e,
}
bool LZe_encode_member( struct LZ_encoder * const e,
bool LZe_encode_member( LZ_encoder * const e,
const unsigned long long member_size )
{
const unsigned long long member_size_limit =
member_size - Lt_size - max_marker_size;
const bool best = ( e->match_len_limit > 12 );
const bool best = e->match_len_limit > 12;
const int dis_price_count = best ? 1 : 512;
const int align_price_count = best ? 1 : dis_align_size;
const int price_count = ( e->match_len_limit > 36 ) ? 1013 : 4093;
@ -537,7 +536,7 @@ bool LZe_encode_member( struct LZ_encoder * const e,
const int len = e->trials[i].price;
int dis = e->trials[i].dis4;
bool bit = ( dis < 0 );
bool bit = dis < 0;
Re_encode_bit( &e->eb.renc, &e->eb.bm_match[state][pos_state], !bit );
if( bit ) /* literal byte */
{
@ -556,11 +555,11 @@ bool LZe_encode_member( struct LZ_encoder * const e,
{
CRC32_update_buf( &e->eb.crc, Mb_ptr_to_current_pos( &e->eb.mb ) - ahead, len );
mtf_reps( dis, reps );
bit = ( dis < num_rep_distances );
bit = dis < num_rep_distances;
Re_encode_bit( &e->eb.renc, &e->eb.bm_rep[state], bit );
if( bit ) /* repeated match */
{
bit = ( dis == 0 );
bit = dis == 0;
Re_encode_bit( &e->eb.renc, &e->eb.bm_rep0[state], !bit );
if( bit )
Re_encode_bit( &e->eb.renc, &e->eb.bm_len[state][pos_state], len > 1 );

81
encoder.h
View file

@ -15,16 +15,16 @@
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
struct Len_prices
typedef struct Len_prices
{
const struct Len_model * lm;
const Len_model * lm;
int len_symbols;
int count;
int prices[pos_states][max_len_symbols];
int counters[pos_states]; /* may decrement below 0 */
};
} Len_prices;
static inline void Lp_update_low_mid_prices( struct Len_prices * const lp,
static inline void Lp_update_low_mid_prices( Len_prices * const lp,
const int pos_state )
{
int * const pps = lp->prices[pos_state];
@ -39,7 +39,7 @@ static inline void Lp_update_low_mid_prices( struct Len_prices * const lp,
price_symbol3( lp->lm->bm_mid[pos_state], len - len_low_symbols );
}
static inline void Lp_update_high_prices( struct Len_prices * const lp )
static inline void Lp_update_high_prices( Len_prices * const lp )
{
const int tmp = price1( lp->lm->choice1 ) + price1( lp->lm->choice2 );
int len;
@ -50,11 +50,10 @@ static inline void Lp_update_high_prices( struct Len_prices * const lp )
price_symbol8( lp->lm->bm_high, len - len_low_symbols - len_mid_symbols );
}
static inline void Lp_reset( struct Len_prices * const lp )
static inline void Lp_reset( Len_prices * const lp )
{ int i; for( i = 0; i < pos_states; ++i ) lp->counters[i] = 0; }
static inline void Lp_init( struct Len_prices * const lp,
const struct Len_model * const lm,
static inline void Lp_init( Len_prices * const lp, const Len_model * const lm,
const int match_len_limit )
{
lp->lm = lm;
@ -63,11 +62,11 @@ static inline void Lp_init( struct Len_prices * const lp,
Lp_reset( lp );
}
static inline void Lp_decrement_counter( struct Len_prices * const lp,
static inline void Lp_decrement_counter( Len_prices * const lp,
const int pos_state )
{ --lp->counters[pos_state]; }
static inline void Lp_update_prices( struct Len_prices * const lp )
static inline void Lp_update_prices( Len_prices * const lp )
{
int pos_state;
bool high_pending = false;
@ -79,23 +78,23 @@ static inline void Lp_update_prices( struct Len_prices * const lp )
Lp_update_high_prices( lp );
}
static inline int Lp_price( const struct Len_prices * const lp,
static inline int Lp_price( const Len_prices * const lp,
const int len, const int pos_state )
{ return lp->prices[pos_state][len - min_match_len]; }
struct Pair /* distance-length pair */
typedef struct Pair /* distance-length pair */
{
int dis;
int len;
};
} Pair;
enum { infinite_price = 0x0FFFFFFF,
max_num_trials = 1 << 13,
single_step_trial = -2,
dual_step_trial = -1 };
struct Trial
typedef struct Trial
{
State state;
int price; /* dual use var; cumulative price, match length */
@ -105,9 +104,9 @@ struct Trial
/* -1 literal + rep0 */
/* >= 0 ( rep or match ) + literal + rep0 */
int reps[num_rep_distances];
};
} Trial;
static inline void Tr_update( struct Trial * const trial, const int pr,
static inline void Tr_update( Trial * const trial, const int pr,
const int distance4, const int p_i )
{
if( pr < trial->price )
@ -115,7 +114,7 @@ static inline void Tr_update( struct Trial * const trial, const int pr,
trial->prev_index2 = single_step_trial; }
}
static inline void Tr_update2( struct Trial * const trial, const int pr,
static inline void Tr_update2( Trial * const trial, const int pr,
const int p_i )
{
if( pr < trial->price )
@ -123,7 +122,7 @@ static inline void Tr_update2( struct Trial * const trial, const int pr,
trial->prev_index2 = dual_step_trial; }
}
static inline void Tr_update3( struct Trial * const trial, const int pr,
static inline void Tr_update3( Trial * const trial, const int pr,
const int distance4, const int p_i,
const int p_i2 )
{
@ -133,25 +132,24 @@ static inline void Tr_update3( struct Trial * const trial, const int pr,
}
struct LZ_encoder
typedef struct LZ_encoder
{
struct LZ_encoder_base eb;
LZ_encoder_base eb;
int cycles;
int match_len_limit;
struct Len_prices match_len_prices;
struct Len_prices rep_len_prices;
Len_prices match_len_prices;
Len_prices rep_len_prices;
int pending_num_pairs;
struct Pair pairs[max_match_len+1];
struct Trial trials[max_num_trials];
Pair pairs[max_match_len+1];
Trial trials[max_num_trials];
int dis_slot_prices[len_states][2*max_dictionary_bits];
int dis_prices[len_states][modeled_distances];
int align_prices[dis_align_size];
int num_dis_slots;
};
} LZ_encoder;
static inline bool Mb_dec_pos( struct Matchfinder_base * const mb,
const int ahead )
static inline bool Mb_dec_pos( Matchfinder_base * const mb, const int ahead )
{
if( ahead < 0 || mb->pos < ahead ) return false;
mb->pos -= ahead;
@ -160,7 +158,7 @@ static inline bool Mb_dec_pos( struct Matchfinder_base * const mb,
return true;
}
int LZe_get_match_pairs( struct LZ_encoder * const e, struct Pair * pairs );
int LZe_get_match_pairs( LZ_encoder * const e, Pair * pairs );
/* move-to-front dis in/into reps; do nothing if( dis4 <= 0 ) */
static inline void mtf_reps( const int dis4, int reps[num_rep_distances] )
@ -178,13 +176,13 @@ static inline void mtf_reps( const int dis4, int reps[num_rep_distances] )
}
}
static inline int LZeb_price_shortrep( const struct LZ_encoder_base * const eb,
static inline int LZeb_price_shortrep( const LZ_encoder_base * const eb,
const State state, const int pos_state )
{
return price0( eb->bm_rep0[state] ) + price0( eb->bm_len[state][pos_state] );
}
static inline int LZeb_price_rep( const struct LZ_encoder_base * const eb,
static inline int LZeb_price_rep( const LZ_encoder_base * const eb,
const int rep, const State state,
const int pos_state )
{
@ -201,7 +199,7 @@ static inline int LZeb_price_rep( const struct LZ_encoder_base * const eb,
return price;
}
static inline int LZe_price_rep0_len( const struct LZ_encoder * const e,
static inline int LZe_price_rep0_len( const LZ_encoder * const e,
const int len, const State state,
const int pos_state )
{
@ -209,9 +207,8 @@ static inline int LZe_price_rep0_len( const struct LZ_encoder * const e,
Lp_price( &e->rep_len_prices, len, pos_state );
}
static inline int LZe_price_pair( const struct LZ_encoder * const e,
const int dis, const int len,
const int pos_state )
static inline int LZe_price_pair( const LZ_encoder * const e, const int dis,
const int len, const int pos_state )
{
const int price = Lp_price( &e->match_len_prices, len, pos_state );
const int len_state = get_len_state( len );
@ -222,7 +219,7 @@ static inline int LZe_price_pair( const struct LZ_encoder * const e,
e->align_prices[dis & (dis_align_size - 1)];
}
static inline int LZe_read_match_distances( struct LZ_encoder * const e )
static inline int LZe_read_match_distances( LZ_encoder * const e )
{
const int num_pairs = LZe_get_match_pairs( e, e->pairs );
if( num_pairs > 0 )
@ -235,7 +232,7 @@ static inline int LZe_read_match_distances( struct LZ_encoder * const e )
return num_pairs;
}
static inline void LZe_move_and_update( struct LZ_encoder * const e, int n )
static inline void LZe_move_and_update( LZ_encoder * const e, int n )
{
while( true )
{
@ -245,13 +242,13 @@ static inline void LZe_move_and_update( struct LZ_encoder * const e, int n )
}
}
static inline void LZe_backward( struct LZ_encoder * const e, int cur )
static inline void LZe_backward( LZ_encoder * const e, int cur )
{
int dis4 = e->trials[cur].dis4;
while( cur > 0 )
{
const int prev_index = e->trials[cur].prev_index;
struct Trial * const prev_trial = &e->trials[prev_index];
Trial * const prev_trial = &e->trials[prev_index];
if( e->trials[cur].prev_index2 != single_step_trial )
{
@ -260,7 +257,7 @@ static inline void LZe_backward( struct LZ_encoder * const e, int cur )
prev_trial->prev_index2 = single_step_trial;
if( e->trials[cur].prev_index2 >= 0 )
{
struct Trial * const prev_trial2 = &e->trials[prev_index-1];
Trial * const prev_trial2 = &e->trials[prev_index-1];
prev_trial2->dis4 = dis4; dis4 = 0; /* rep0 */
prev_trial2->prev_index = e->trials[cur].prev_index2;
prev_trial2->prev_index2 = single_step_trial;
@ -275,7 +272,7 @@ static inline void LZe_backward( struct LZ_encoder * const e, int cur )
enum { num_prev_positions3 = 1 << 16,
num_prev_positions2 = 1 << 10 };
static inline bool LZe_init( struct LZ_encoder * const e,
static inline bool LZe_init( LZ_encoder * const e,
const int dict_size, const int len_limit,
const int ifd, const int outfd )
{
@ -300,7 +297,7 @@ static inline bool LZe_init( struct LZ_encoder * const e,
return true;
}
static inline void LZe_reset( struct LZ_encoder * const e )
static inline void LZe_reset( LZ_encoder * const e )
{
LZeb_reset( &e->eb );
Lp_reset( &e->match_len_prices );
@ -308,5 +305,5 @@ static inline void LZe_reset( struct LZ_encoder * const e )
e->pending_num_pairs = 0;
}
bool LZe_encode_member( struct LZ_encoder * const e,
bool LZe_encode_member( LZ_encoder * const e,
const unsigned long long member_size );

16
encoder_base.c
View file

@ -31,7 +31,7 @@ Dis_slots dis_slots;
Prob_prices prob_prices;
bool Mb_read_block( struct Matchfinder_base * const mb )
bool Mb_read_block( Matchfinder_base * const mb )
{
if( !mb->at_stream_end && mb->stream_pos < mb->buffer_size )
{
@ -46,7 +46,7 @@ bool Mb_read_block( struct Matchfinder_base * const mb )
}
void Mb_normalize_pos( struct Matchfinder_base * const mb )
void Mb_normalize_pos( Matchfinder_base * const mb )
{
if( mb->pos > mb->stream_pos )
internal_error( "pos > stream_pos in Mb_normalize_pos." );
@ -69,7 +69,7 @@ void Mb_normalize_pos( struct Matchfinder_base * const mb )
}
bool Mb_init( struct Matchfinder_base * const mb, const int before_size,
bool Mb_init( Matchfinder_base * const mb, const int before_size,
const int dict_size, const int after_size,
const int dict_factor, const int num_prev_positions23,
const int pos_array_factor, const int ifd )
@ -123,7 +123,7 @@ bool Mb_init( struct Matchfinder_base * const mb, const int before_size,
}
void Mb_reset( struct Matchfinder_base * const mb )
void Mb_reset( Matchfinder_base * const mb )
{
int i;
if( mb->stream_pos > mb->pos )
@ -147,13 +147,13 @@ void Mb_reset( struct Matchfinder_base * const mb )
}
void Re_flush_data( struct Range_encoder * const renc )
void Re_flush_data( Range_encoder * const renc )
{
if( renc->pos > 0 )
{
if( renc->outfd >= 0 &&
writeblock( renc->outfd, renc->buffer, renc->pos ) != renc->pos )
{ show_error( "Write error", errno, false ); cleanup_and_fail( 1 ); }
{ show_error( write_error_msg, errno, false ); cleanup_and_fail( 1 ); }
renc->partial_member_pos += renc->pos;
renc->pos = 0;
show_cprogress( 0, 0, 0, 0 );
@ -162,7 +162,7 @@ void Re_flush_data( struct Range_encoder * const renc )
/* End Of Stream marker => (dis == 0xFFFFFFFFU, len == min_match_len) */
void LZeb_full_flush( struct LZ_encoder_base * const eb, const State state )
void LZeb_full_flush( LZ_encoder_base * const eb, const State state )
{
const int pos_state = Mb_data_position( &eb->mb ) & pos_state_mask;
Re_encode_bit( &eb->renc, &eb->bm_match[state][pos_state], 1 );
@ -178,7 +178,7 @@ void LZeb_full_flush( struct LZ_encoder_base * const eb, const State state )
}
void LZeb_reset( struct LZ_encoder_base * const eb )
void LZeb_reset( LZ_encoder_base * const eb )
{
Mb_reset( &eb->mb );
eb->crc = 0xFFFFFFFFU;

101
encoder_base.h
View file

@ -174,36 +174,36 @@ struct Matchfinder_base
bool at_stream_end; /* stream_pos shows real end of file */
};
bool Mb_read_block( struct Matchfinder_base * const mb );
void Mb_normalize_pos( struct Matchfinder_base * const mb );
bool Mb_read_block( Matchfinder_base * const mb );
void Mb_normalize_pos( Matchfinder_base * const mb );
bool Mb_init( struct Matchfinder_base * const mb, const int before_size,
bool Mb_init( Matchfinder_base * const mb, const int before_size,
const int dict_size, const int after_size,
const int dict_factor, const int num_prev_positions23,
const int pos_array_factor, const int ifd );
static inline void Mb_free( struct Matchfinder_base * const mb )
static inline void Mb_free( Matchfinder_base * const mb )
{ free( mb->prev_positions ); free( mb->buffer ); }
static inline uint8_t Mb_peek( const struct Matchfinder_base * const mb,
static inline uint8_t Mb_peek( const Matchfinder_base * const mb,
const int distance )
{ return mb->buffer[mb->pos-distance]; }
static inline int Mb_available_bytes( const struct Matchfinder_base * const mb )
static inline int Mb_available_bytes( const Matchfinder_base * const mb )
{ return mb->stream_pos - mb->pos; }
static inline unsigned long long
Mb_data_position( const struct Matchfinder_base * const mb )
Mb_data_position( const Matchfinder_base * const mb )
{ return mb->partial_data_pos + mb->pos; }
static inline bool Mb_data_finished( const struct Matchfinder_base * const mb )
static inline bool Mb_data_finished( const Matchfinder_base * const mb )
{ return mb->at_stream_end && mb->pos >= mb->stream_pos; }
static inline const uint8_t *
Mb_ptr_to_current_pos( const struct Matchfinder_base * const mb )
Mb_ptr_to_current_pos( const Matchfinder_base * const mb )
{ return mb->buffer + mb->pos; }
static inline int Mb_true_match_len( const struct Matchfinder_base * const mb,
static inline int Mb_true_match_len( const Matchfinder_base * const mb,
const int index, const int distance )
{
const uint8_t * const data = mb->buffer + mb->pos;
@ -213,18 +213,18 @@ static inline int Mb_true_match_len( const struct Matchfinder_base * const mb,
return i;
}
static inline void Mb_move_pos( struct Matchfinder_base * const mb )
static inline void Mb_move_pos( Matchfinder_base * const mb )
{
if( ++mb->cyclic_pos > mb->dictionary_size ) mb->cyclic_pos = 0;
if( ++mb->pos >= mb->pos_limit ) Mb_normalize_pos( mb );
}
void Mb_reset( struct Matchfinder_base * const mb );
void Mb_reset( Matchfinder_base * const mb );
enum { re_buffer_size = 65536 };
struct Range_encoder
typedef struct Range_encoder
{
uint64_t low;
unsigned long long partial_member_pos;
@ -235,22 +235,21 @@ struct Range_encoder
int outfd; /* output file descriptor */
uint8_t cache;
Lzip_header header;
};
} Range_encoder;
void Re_flush_data( struct Range_encoder * const renc );
void Re_flush_data( Range_encoder * const renc );
static inline void Re_put_byte( struct Range_encoder * const renc,
const uint8_t b )
static inline void Re_put_byte( Range_encoder * const renc, const uint8_t b )
{
renc->buffer[renc->pos] = b;
if( ++renc->pos >= re_buffer_size ) Re_flush_data( renc );
}
static inline void Re_shift_low( struct Range_encoder * const renc )
static inline void Re_shift_low( Range_encoder * const renc )
{
if( renc->low >> 24 != 0xFF )
{
const bool carry = ( renc->low > 0xFFFFFFFFU );
const bool carry = renc->low > 0xFFFFFFFFU;
Re_put_byte( renc, renc->cache + carry );
for( ; renc->ff_count > 0; --renc->ff_count )
Re_put_byte( renc, 0xFF + carry );
@ -260,7 +259,7 @@ static inline void Re_shift_low( struct Range_encoder * const renc )
renc->low = ( renc->low & 0x00FFFFFFU ) << 8;
}
static inline void Re_reset( struct Range_encoder * const renc,
static inline void Re_reset( Range_encoder * const renc,
const unsigned dictionary_size )
{
renc->low = 0;
@ -273,7 +272,7 @@ static inline void Re_reset( struct Range_encoder * const renc,
int i; for( i = 0; i < Lh_size; ++i ) Re_put_byte( renc, renc->header[i] );
}
static inline bool Re_init( struct Range_encoder * const renc,
static inline bool Re_init( Range_encoder * const renc,
const unsigned dictionary_size, const int ofd )
{
renc->buffer = (uint8_t *)malloc( re_buffer_size );
@ -284,17 +283,17 @@ static inline bool Re_init( struct Range_encoder * const renc,
return true;
}
static inline void Re_free( struct Range_encoder * const renc )
static inline void Re_free( Range_encoder * const renc )
{ free( renc->buffer ); }
static inline unsigned long long
Re_member_position( const struct Range_encoder * const renc )
Re_member_position( const Range_encoder * const renc )
{ return renc->partial_member_pos + renc->pos + renc->ff_count; }
static inline void Re_flush( struct Range_encoder * const renc )
static inline void Re_flush( Range_encoder * const renc )
{ int i; for( i = 0; i < 5; ++i ) Re_shift_low( renc ); }
static inline void Re_encode( struct Range_encoder * const renc,
static inline void Re_encode( Range_encoder * const renc,
const int symbol, const int num_bits )
{
unsigned mask;
@ -306,7 +305,7 @@ static inline void Re_encode( struct Range_encoder * const renc,
}
}
static inline void Re_encode_bit( struct Range_encoder * const renc,
static inline void Re_encode_bit( Range_encoder * const renc,
Bit_model * const probability, const bool bit )
{
const uint32_t bound = ( renc->range >> bit_model_total_bits ) * *probability;
@ -324,7 +323,7 @@ static inline void Re_encode_bit( struct Range_encoder * const renc,
if( renc->range <= 0x00FFFFFFU ) { renc->range <<= 8; Re_shift_low( renc ); }
}
static inline void Re_encode_tree3( struct Range_encoder * const renc,
static inline void Re_encode_tree3( Range_encoder * const renc,
Bit_model bm[], const int symbol )
{
bool bit = ( symbol >> 2 ) & 1;
@ -335,7 +334,7 @@ static inline void Re_encode_tree3( struct Range_encoder * const renc,
Re_encode_bit( renc, &bm[model], symbol & 1 );
}
static inline void Re_encode_tree6( struct Range_encoder * const renc,
static inline void Re_encode_tree6( Range_encoder * const renc,
Bit_model bm[], const unsigned symbol )
{
bool bit = ( symbol >> 5 ) & 1;
@ -352,7 +351,7 @@ static inline void Re_encode_tree6( struct Range_encoder * const renc,
Re_encode_bit( renc, &bm[model], symbol & 1 );
}
static inline void Re_encode_tree8( struct Range_encoder * const renc,
static inline void Re_encode_tree8( Range_encoder * const renc,
Bit_model bm[], const int symbol )
{
int model = 1;
@ -365,7 +364,7 @@ static inline void Re_encode_tree8( struct Range_encoder * const renc,
}
}
static inline void Re_encode_tree_reversed( struct Range_encoder * const renc,
static inline void Re_encode_tree_reversed( Range_encoder * const renc,
Bit_model bm[], int symbol, const int num_bits )
{
int model = 1;
@ -379,7 +378,7 @@ static inline void Re_encode_tree_reversed( struct Range_encoder * const renc,
}
}
static inline void Re_encode_matched( struct Range_encoder * const renc,
static inline void Re_encode_matched( Range_encoder * const renc,
Bit_model bm[], unsigned symbol,
unsigned match_byte )
{
@ -395,17 +394,17 @@ static inline void Re_encode_matched( struct Range_encoder * const renc,
}
}
static inline void Re_encode_len( struct Range_encoder * const renc,
struct Len_model * const lm,
static inline void Re_encode_len( Range_encoder * const renc,
Len_model * const lm,
int symbol, const int pos_state )
{
bool bit = ( ( symbol -= min_match_len ) >= len_low_symbols );
bool bit = ( symbol -= min_match_len ) >= len_low_symbols;
Re_encode_bit( renc, &lm->choice1, bit );
if( !bit )
Re_encode_tree3( renc, lm->bm_low[pos_state], symbol );
else
{
bit = ( ( symbol -= len_low_symbols ) >= len_mid_symbols );
bit = ( symbol -= len_low_symbols ) >= len_mid_symbols;
Re_encode_bit( renc, &lm->choice2, bit );
if( !bit )
Re_encode_tree3( renc, lm->bm_mid[pos_state], symbol );
@ -418,9 +417,9 @@ static inline void Re_encode_len( struct Range_encoder * const renc,
enum { max_marker_size = 16,
num_rep_distances = 4 }; /* must be 4 */
struct LZ_encoder_base
typedef struct LZ_encoder_base
{
struct Matchfinder_base mb;
Matchfinder_base mb;
uint32_t crc;
Bit_model bm_literal[1<<literal_context_bits][0x300];
@ -433,14 +432,14 @@ struct LZ_encoder_base
Bit_model bm_dis_slot[len_states][1<<dis_slot_bits];
Bit_model bm_dis[modeled_distances-end_dis_model+1];
Bit_model bm_align[dis_align_size];
struct Len_model match_len_model;
struct Len_model rep_len_model;
struct Range_encoder renc;
};
Len_model match_len_model;
Len_model rep_len_model;
Range_encoder renc;
} LZ_encoder_base;
void LZeb_reset( struct LZ_encoder_base * const eb );
void LZeb_reset( LZ_encoder_base * const eb );
static inline bool LZeb_init( struct LZ_encoder_base * const eb,
static inline bool LZeb_init( LZ_encoder_base * const eb,
const int before_size, const int dict_size,
const int after_size, const int dict_factor,
const int num_prev_positions23,
@ -454,31 +453,31 @@ static inline bool LZeb_init( struct LZ_encoder_base * const eb,
return true;
}
static inline void LZeb_free( struct LZ_encoder_base * const eb )
static inline void LZeb_free( LZ_encoder_base * const eb )
{ Re_free( &eb->renc ); Mb_free( &eb->mb ); }
static inline unsigned LZeb_crc( const struct LZ_encoder_base * const eb )
static inline unsigned LZeb_crc( const LZ_encoder_base * const eb )
{ return eb->crc ^ 0xFFFFFFFFU; }
static inline int LZeb_price_literal( const struct LZ_encoder_base * const eb,
static inline int LZeb_price_literal( const LZ_encoder_base * const eb,
const uint8_t prev_byte, const uint8_t symbol )
{ return price_symbol8( eb->bm_literal[get_lit_state(prev_byte)], symbol ); }
static inline int LZeb_price_matched( const struct LZ_encoder_base * const eb,
static inline int LZeb_price_matched( const LZ_encoder_base * const eb,
const uint8_t prev_byte, const uint8_t symbol, const uint8_t match_byte )
{ return price_matched( eb->bm_literal[get_lit_state(prev_byte)], symbol,
match_byte ); }
static inline void LZeb_encode_literal( struct LZ_encoder_base * const eb,
static inline void LZeb_encode_literal( LZ_encoder_base * const eb,
const uint8_t prev_byte, const uint8_t symbol )
{ Re_encode_tree8( &eb->renc, eb->bm_literal[get_lit_state(prev_byte)], symbol ); }
static inline void LZeb_encode_matched( struct LZ_encoder_base * const eb,
static inline void LZeb_encode_matched( LZ_encoder_base * const eb,
const uint8_t prev_byte, const uint8_t symbol, const uint8_t match_byte )
{ Re_encode_matched( &eb->renc, eb->bm_literal[get_lit_state(prev_byte)],
symbol, match_byte ); }
static inline void LZeb_encode_pair( struct LZ_encoder_base * const eb,
static inline void LZeb_encode_pair( LZ_encoder_base * const eb,
const unsigned dis, const int len,
const int pos_state )
{
@ -504,4 +503,4 @@ static inline void LZeb_encode_pair( struct LZ_encoder_base * const eb,
}
}
void LZeb_full_flush( struct LZ_encoder_base * const eb, const State state );
void LZeb_full_flush( LZ_encoder_base * const eb, const State state );

4
fast_encoder.c
View file

@ -28,7 +28,7 @@
#include "fast_encoder.h"
int FLZe_longest_match_len( struct FLZ_encoder * const fe, int * const distance )
int FLZe_longest_match_len( FLZ_encoder * const fe, int * const distance )
{
enum { len_limit = 16 };
const int available = min( Mb_available_bytes( &fe->eb.mb ), max_match_len );
@ -69,7 +69,7 @@ int FLZe_longest_match_len( struct FLZ_encoder * const fe, int * const distance
}
bool FLZe_encode_member( struct FLZ_encoder * const fe,
bool FLZe_encode_member( FLZ_encoder * const fe,
const unsigned long long member_size )
{
const unsigned long long member_size_limit =

20
fast_encoder.h
View file

@ -15,13 +15,13 @@
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
struct FLZ_encoder
typedef struct FLZ_encoder
{
struct LZ_encoder_base eb;
LZ_encoder_base eb;
unsigned key4; /* key made from latest 4 bytes */
};
} FLZ_encoder;
static inline void FLZe_reset_key4( struct FLZ_encoder * const fe )
static inline void FLZe_reset_key4( FLZ_encoder * const fe )
{
int i;
fe->key4 = 0;
@ -29,11 +29,11 @@ static inline void FLZe_reset_key4( struct FLZ_encoder * const fe )
fe->key4 = ( fe->key4 << 4 ) ^ fe->eb.mb.buffer[i];
}
int FLZe_longest_match_len( struct FLZ_encoder * const fe, int * const distance );
int FLZe_longest_match_len( FLZ_encoder * const fe, int * const distance );
static inline void FLZe_update_and_move( struct FLZ_encoder * const fe, int n )
static inline void FLZe_update_and_move( FLZ_encoder * const fe, int n )
{
struct Matchfinder_base * const mb = &fe->eb.mb;
Matchfinder_base * const mb = &fe->eb.mb;
while( --n >= 0 )
{
if( Mb_available_bytes( mb ) >= 4 )
@ -46,7 +46,7 @@ static inline void FLZe_update_and_move( struct FLZ_encoder * const fe, int n )
}
}
static inline bool FLZe_init( struct FLZ_encoder * const fe,
static inline bool FLZe_init( FLZ_encoder * const fe,
const int ifd, const int outfd )
{
enum { before_size = 0,
@ -61,8 +61,8 @@ static inline bool FLZe_init( struct FLZ_encoder * const fe,
num_prev_positions23, pos_array_factor, ifd, outfd );
}
static inline void FLZe_reset( struct FLZ_encoder * const fe )
static inline void FLZe_reset( FLZ_encoder * const fe )
{ LZeb_reset( &fe->eb ); }
bool FLZe_encode_member( struct FLZ_encoder * const fe,
bool FLZe_encode_member( FLZ_encoder * const fe,
const unsigned long long member_size );

21
list.c
View file

@ -17,6 +17,7 @@
#define _FILE_OFFSET_BITS 64
#include <errno.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
@ -43,7 +44,7 @@ static void list_line( const unsigned long long uncomp_size,
int list_files( const char * const filenames[], const int num_filenames,
const struct Cl_options * const cl_opts )
const Cl_options * const cl_opts )
{
unsigned long long total_comp = 0, total_uncomp = 0;
int files = 0, retval = 0;
@ -53,7 +54,7 @@ int list_files( const char * const filenames[], const int num_filenames,
for( i = 0; i < num_filenames; ++i )
{
const bool from_stdin = ( strcmp( filenames[i], "-" ) == 0 );
const bool from_stdin = strcmp( filenames[i], "-" ) == 0;
if( from_stdin ) { if( stdin_used ) continue; else stdin_used = true; }
const char * const input_filename = from_stdin ? "(stdin)" : filenames[i];
struct stat in_stats; /* not used */
@ -61,7 +62,7 @@ int list_files( const char * const filenames[], const int num_filenames,
open_instream( input_filename, &in_stats, false, true );
if( infd < 0 ) { set_retval( &retval, 1 ); continue; }
struct Lzip_index lzip_index;
Lzip_index lzip_index;
Li_init( &lzip_index, infd, cl_opts );
close( infd );
if( lzip_index.retval != 0 )
@ -70,6 +71,8 @@ int list_files( const char * const filenames[], const int num_filenames,
set_retval( &retval, lzip_index.retval );
Li_free( &lzip_index ); continue;
}
const bool multi_empty = !from_stdin && Li_multi_empty( &lzip_index );
if( multi_empty ) set_retval( &retval, 2 );
if( verbosity < 0 ) { Li_free( &lzip_index ); continue; }
const unsigned long long udata_size = Li_udata_size( &lzip_index );
const unsigned long long cdata_size = Li_cdata_size( &lzip_index );
@ -81,6 +84,8 @@ int list_files( const char * const filenames[], const int num_filenames,
if( verbosity >= 1 ) fputs( " dict memb trail ", stdout );
fputs( " uncompressed compressed saved name\n", stdout );
}
if( multi_empty )
{ fflush( stdout ); show_file_error( input_filename, empty_msg, 0 ); }
if( verbosity >= 1 )
printf( "%s %5ld %6lld ", format_ds( lzip_index.dictionary_size ),
members, Li_file_size( &lzip_index ) - cdata_size );
@ -92,8 +97,8 @@ int list_files( const char * const filenames[], const int num_filenames,
fputs( " member data_pos data_size member_pos member_size\n", stdout );
for( i = 0; i < members; ++i )
{
const struct Block * db = Li_dblock( &lzip_index, i );
const struct Block * mb = Li_mblock( &lzip_index, i );
const Block * db = Li_dblock( &lzip_index, i );
const Block * mb = Li_mblock( &lzip_index, i );
printf( "%6ld %14llu %14llu %14llu %14llu\n",
i + 1, db->pos, db->size, mb->pos, mb->size );
}
@ -101,12 +106,16 @@ int list_files( const char * const filenames[], const int num_filenames,
}
fflush( stdout );
Li_free( &lzip_index );
if( ferror( stdout ) ) break;
}
if( verbosity >= 0 && files > 1 )
if( verbosity >= 0 && files > 1 && !ferror( stdout ) )
{
if( verbosity >= 1 ) fputs( " ", stdout );
list_line( total_uncomp, total_comp, "(totals)" );
fflush( stdout );
}
if( verbosity >= 0 && ( ferror( stdout ) || fclose( stdout ) != 0 ) )
{ show_file_error( "(stdout)", write_error_msg, errno );
set_retval( &retval, 1 ); }
return retval;
}

40
lzip.h
View file

@ -92,16 +92,16 @@ static inline void Bm_init( Bit_model * const probability )
static inline void Bm_array_init( Bit_model bm[], const int size )
{ int i; for( i = 0; i < size; ++i ) Bm_init( &bm[i] ); }
struct Len_model
typedef struct Len_model
{
Bit_model choice1;
Bit_model choice2;
Bit_model bm_low[pos_states][len_low_symbols];
Bit_model bm_mid[pos_states][len_mid_symbols];
Bit_model bm_high[len_high_symbols];
};
} Len_model;
static inline void Lm_init( struct Len_model * const lm )
static inline void Lm_init( Len_model * const lm )
{
Bm_init( &lm->choice1 );
Bm_init( &lm->choice2 );
@ -276,17 +276,14 @@ static inline bool Lt_check_consistency( const Lzip_trailer data )
}
struct Cl_options /* command-line options */
typedef struct Cl_options /* command-line options */
{
bool ignore_empty;
bool ignore_marking;
bool ignore_trailing;
bool loose_trailing;
};
} Cl_options;
static inline void Cl_options_init( struct Cl_options * cl_opts )
{ cl_opts->ignore_empty = true; cl_opts->ignore_marking = true;
cl_opts->ignore_trailing = true; cl_opts->loose_trailing = false; }
static inline void Cl_options_init( Cl_options * cl_opts )
{ cl_opts->ignore_trailing = true; cl_opts->loose_trailing = false; }
static inline void set_retval( int * retval, const int new_val )
@ -296,9 +293,10 @@ static const char * const bad_magic_msg = "Bad magic number (file not in lzip fo
static const char * const bad_dict_msg = "Invalid dictionary size in member header.";
static const char * const corrupt_mm_msg = "Corrupt header in multimember file.";
static const char * const empty_msg = "Empty member not allowed.";
static const char * const marking_msg = "Marking data not allowed.";
static const char * const trailing_msg = "Trailing data not allowed.";
static const char * const mem_msg = "Not enough memory.";
static const char * const nonzero_msg = "Nonzero first LZMA byte.";
static const char * const trailing_msg = "Trailing data not allowed.";
static const char * const write_error_msg = "Write error";
/* defined in decoder.c */
int readblock( const int fd, uint8_t * const buf, const int size );
@ -306,14 +304,16 @@ int writeblock( const int fd, const uint8_t * const buf, const int size );
/* defined in list.c */
int list_files( const char * const filenames[], const int num_filenames,
const struct Cl_options * const cl_opts );
const Cl_options * const cl_opts );
/* defined in main.c */
struct stat;
struct Pretty_print;
typedef struct Matchfinder_base Matchfinder_base;
typedef struct Pretty_print Pretty_print;
typedef struct Range_decoder Range_decoder;
extern int verbosity;
void * resize_buffer( void * buf, const unsigned min_size );
void Pp_show_msg( struct Pretty_print * const pp, const char * const msg );
void Pp_show_msg( Pretty_print * const pp, const char * const msg );
const char * bad_version( const unsigned version );
const char * format_ds( const unsigned dictionary_size );
void show_header( const unsigned dictionary_size );
@ -324,13 +324,11 @@ void show_error( const char * const msg, const int errcode, const bool help );
void show_file_error( const char * const filename, const char * const msg,
const int errcode );
void internal_error( const char * const msg );
struct Matchfinder_base;
void show_cprogress( const unsigned long long cfile_size,
const unsigned long long partial_size,
const struct Matchfinder_base * const m,
struct Pretty_print * const p );
struct Range_decoder;
const Matchfinder_base * const m,
Pretty_print * const p );
void show_dprogress( const unsigned long long cfile_size,
const unsigned long long partial_size,
const struct Range_decoder * const d,
struct Pretty_print * const p );
const Range_decoder * const d,
Pretty_print * const p );

78
lzip_index.c
View file

@ -38,7 +38,7 @@ static int seek_read( const int fd, uint8_t * const buf, const int size,
}
static bool add_error( struct Lzip_index * const li, const char * const msg )
static bool add_error( Lzip_index * const li, const char * const msg )
{
const int len = strlen( msg );
void * tmp = resize_buffer( li->error, li->error_size + len + 1 );
@ -50,16 +50,15 @@ static bool add_error( struct Lzip_index * const li, const char * const msg )
}
static bool push_back_member( struct Lzip_index * const li,
const long long dp, const long long ds,
const long long mp, const long long ms,
const unsigned dict_size )
static bool push_back_member( Lzip_index * const li, const long long dp,
const long long ds, const long long mp,
const long long ms, const unsigned dict_size )
{
struct Member * p;
Member * p;
void * tmp = resize_buffer( li->member_vector,
( li->members + 1 ) * sizeof li->member_vector[0] );
if( !tmp ) { add_error( li, mem_msg ); li->retval = 1; return false; }
li->member_vector = (struct Member *)tmp;
li->member_vector = (Member *)tmp;
p = &(li->member_vector[li->members]);
init_member( p, dp, ds, mp, ms, dict_size );
++li->members;
@ -67,7 +66,7 @@ static bool push_back_member( struct Lzip_index * const li,
}
static void Li_free_member_vector( struct Lzip_index * const li )
static void Li_free_member_vector( Lzip_index * const li )
{
if( li->member_vector )
{ free( li->member_vector ); li->member_vector = 0; }
@ -75,9 +74,9 @@ static void Li_free_member_vector( struct Lzip_index * const li )
}
static void Li_reverse_member_vector( struct Lzip_index * const li )
static void Li_reverse_member_vector( Lzip_index * const li )
{
struct Member tmp;
Member tmp;
long i;
for( i = 0; i < li->members / 2; ++i )
{
@ -88,8 +87,7 @@ static void Li_reverse_member_vector( struct Lzip_index * const li )
}
static bool Li_check_header( struct Lzip_index * const li,
const Lzip_header header )
static bool Li_check_header( Lzip_index * const li, const Lzip_header header )
{
if( !Lh_check_magic( header ) )
{ add_error( li, bad_magic_msg ); li->retval = 2; return false; }
@ -101,15 +99,14 @@ static bool Li_check_header( struct Lzip_index * const li,
return true;
}
static void Li_set_errno_error( struct Lzip_index * const li,
const char * const msg )
static void Li_set_errno_error( Lzip_index * const li, const char * const msg )
{
add_error( li, msg ); add_error( li, strerror( errno ) );
li->retval = 1;
}
static void Li_set_num_error( struct Lzip_index * const li,
const char * const msg, unsigned long long num )
static void Li_set_num_error( Lzip_index * const li, const char * const msg,
unsigned long long num )
{
char buf[80];
snprintf( buf, sizeof buf, "%s%llu", msg, num );
@ -118,22 +115,19 @@ static void Li_set_num_error( struct Lzip_index * const li,
}
static bool Li_read_header( struct Lzip_index * const li, const int fd,
Lzip_header header, const long long pos, const bool ignore_marking )
static bool Li_read_header( Lzip_index * const li, const int fd,
Lzip_header header, const long long pos )
{
if( seek_read( fd, header, Lh_size, pos ) != Lh_size )
{ Li_set_errno_error( li, "Error reading member header: " ); return false; }
uint8_t byte;
if( !ignore_marking && readblock( fd, &byte, 1 ) == 1 && byte != 0 )
{ add_error( li, marking_msg ); li->retval = 2; return false; }
return true;
}
/* If successful, push last member and set pos to member header. */
static bool Li_skip_trailing_data( struct Lzip_index * const li, const int fd,
static bool Li_skip_trailing_data( Lzip_index * const li, const int fd,
unsigned long long * const pos,
const struct Cl_options * const cl_opts )
const Cl_options * const cl_opts )
{
if( *pos < min_member_size ) return false;
enum { block_size = 16384,
@ -162,8 +156,8 @@ static bool Li_skip_trailing_data( struct Lzip_index * const li, const int fd,
if( member_size > ipos + i || !Lt_check_consistency( *trailer ) )
continue;
Lzip_header header;
if( !Li_read_header( li, fd, header, ipos + i - member_size,
cl_opts->ignore_marking ) ) return false;
if( !Li_read_header( li, fd, header, ipos + i - member_size ) )
return false;
if( !Lh_check( header ) ) continue;
const Lzip_header * header2 = (const Lzip_header *)( buffer + i );
const bool full_h2 = bsize - i >= Lh_size;
@ -178,15 +172,12 @@ static bool Li_skip_trailing_data( struct Lzip_index * const li, const int fd,
{ add_error( li, corrupt_mm_msg ); li->retval = 2; return false; }
if( !cl_opts->ignore_trailing )
{ add_error( li, trailing_msg ); li->retval = 2; return false; }
const unsigned long long data_size = Lt_get_data_size( *trailer );
if( !cl_opts->ignore_empty && data_size == 0 )
{ add_error( li, empty_msg ); li->retval = 2; return false; }
*pos = ipos + i - member_size; /* good member */
const unsigned dictionary_size = Lh_get_dictionary_size( header );
if( li->dictionary_size < dictionary_size )
li->dictionary_size = dictionary_size;
return push_back_member( li, 0, data_size, *pos, member_size,
dictionary_size );
return push_back_member( li, 0, Lt_get_data_size( *trailer ), *pos,
member_size, dictionary_size );
}
if( ipos == 0 )
{ Li_set_num_error( li, "Bad trailer at pos ", *pos - Lt_size );
@ -200,8 +191,8 @@ static bool Li_skip_trailing_data( struct Lzip_index * const li, const int fd,
}
bool Li_init( struct Lzip_index * const li, const int infd,
const struct Cl_options * const cl_opts )
bool Li_init( Lzip_index * const li, const int infd,
const Cl_options * const cl_opts )
{
li->member_vector = 0;
li->error = 0;
@ -212,6 +203,10 @@ bool Li_init( struct Lzip_index * const li, const int infd,
li->dictionary_size = 0;
if( li->insize < 0 )
{ Li_set_errno_error( li, "Input file is not seekable: " ); return false; }
Lzip_header header;
if( li->insize >= Lh_size &&
( !Li_read_header( li, infd, header, 0 ) ||
!Li_check_header( li, header ) ) ) return false;
if( li->insize < min_member_size )
{ add_error( li, "Input file is too short." ); li->retval = 2;
return false; }
@ -219,10 +214,6 @@ bool Li_init( struct Lzip_index * const li, const int infd,
{ add_error( li, "Input file is too long (2^63 bytes or more)." );
li->retval = 2; return false; }
Lzip_header header;
if( !Li_read_header( li, infd, header, 0, cl_opts->ignore_marking ) ||
!Li_check_header( li, header ) ) return false;
unsigned long long pos = li->insize; /* always points to a header or to EOF */
while( pos >= min_member_size )
{
@ -237,8 +228,7 @@ bool Li_init( struct Lzip_index * const li, const int infd,
return false; }
Li_set_num_error( li, "Bad trailer at pos ", pos - Lt_size ); break;
}
if( !Li_read_header( li, infd, header, pos - member_size,
cl_opts->ignore_marking ) ) break;
if( !Li_read_header( li, infd, header, pos - member_size ) ) break;
if( !Lh_check( header ) ) /* bad header */
{
if( li->members <= 0 )
@ -246,15 +236,12 @@ bool Li_init( struct Lzip_index * const li, const int infd,
return false; }
Li_set_num_error( li, "Bad header at pos ", pos - member_size ); break;
}
const unsigned long long data_size = Lt_get_data_size( trailer );
if( !cl_opts->ignore_empty && data_size == 0 )
{ add_error( li, empty_msg ); li->retval = 2; break; }
pos -= member_size; /* good member */
const unsigned dictionary_size = Lh_get_dictionary_size( header );
if( li->dictionary_size < dictionary_size )
li->dictionary_size = dictionary_size;
if( !push_back_member( li, 0, data_size, pos, member_size,
dictionary_size ) ) return false;
if( !push_back_member( li, 0, Lt_get_data_size( trailer ), pos,
member_size, dictionary_size ) ) return false;
}
if( pos != 0 || li->members <= 0 || li->retval != 0 )
{
@ -264,8 +251,7 @@ bool Li_init( struct Lzip_index * const li, const int infd,
return false;
}
Li_reverse_member_vector( li );
long i;
for( i = 0; ; ++i )
long i; for( i = 0; ; ++i )
{
const long long end = block_end( li->member_vector[i].dblock );
if( end < 0 || end > INT64_MAX )
@ -281,7 +267,7 @@ bool Li_init( struct Lzip_index * const li, const int infd,
}
void Li_free( struct Lzip_index * const li )
void Li_free( Lzip_index * const li )
{
Li_free_member_vector( li );
if( li->error ) { free( li->error ); li->error = 0; }

60
lzip_index.h
View file

@ -20,72 +20,80 @@
#endif
struct Block
typedef struct Block
{
long long pos, size; /* pos >= 0, size >= 0, pos + size <= INT64_MAX */
};
} Block;
static inline void init_block( struct Block * const b,
static inline void init_block( Block * const b,
const long long p, const long long s )
{ b->pos = p; b->size = s; }
static inline long long block_end( const struct Block b )
{ return b.pos + b.size; }
static inline long long block_end( const Block b ) { return b.pos + b.size; }
struct Member
typedef struct Member
{
struct Block dblock, mblock; /* data block, member block */
Block dblock, mblock; /* data block, member block */
unsigned dictionary_size;
};
} Member;
static inline void init_member( struct Member * const m,
const long long dpos, const long long dsize,
const long long mpos, const long long msize,
const unsigned dict_size )
{ init_block( &m->dblock, dpos, dsize ); init_block( &m->mblock, mpos, msize );
m->dictionary_size = dict_size; }
static inline void init_member( Member * const m, const long long dpos,
const long long dsize, const long long mpos,
const long long msize, const unsigned dict_size )
{ init_block( &m->dblock, dpos, dsize );
init_block( &m->mblock, mpos, msize ); m->dictionary_size = dict_size; }
struct Lzip_index
typedef struct Lzip_index
{
struct Member * member_vector;
Member * member_vector;
char * error;
long long insize;
long members;
int error_size;
int retval;
unsigned dictionary_size; /* largest dictionary size in the file */
};
} Lzip_index;
bool Li_init( struct Lzip_index * const li, const int infd,
const struct Cl_options * const cl_opts );
bool Li_init( Lzip_index * const li, const int infd,
const Cl_options * const cl_opts );
void Li_free( struct Lzip_index * const li );
void Li_free( Lzip_index * const li );
static inline long long Li_udata_size( const struct Lzip_index * const li )
/* multimember file with empty member(s) */
static inline bool Li_multi_empty( Lzip_index * const li )
{
long i;
if( li->members > 1 )
for( i = 0; i < li->members; ++i )
if( li->member_vector[i].dblock.size == 0 ) return true;
return false;
}
static inline long long Li_udata_size( const Lzip_index * const li )
{
if( li->members <= 0 ) return 0;
return block_end( li->member_vector[li->members-1].dblock );
}
static inline long long Li_cdata_size( const struct Lzip_index * const li )
static inline long long Li_cdata_size( const Lzip_index * const li )
{
if( li->members <= 0 ) return 0;
return block_end( li->member_vector[li->members-1].mblock );
}
/* total size including trailing data (if any) */
static inline long long Li_file_size( const struct Lzip_index * const li )
static inline long long Li_file_size( const Lzip_index * const li )
{ if( li->insize >= 0 ) return li->insize; else return 0; }
static inline const struct Block * Li_dblock( const struct Lzip_index * const li,
static inline const Block * Li_dblock( const Lzip_index * const li,
const long i )
{ return &li->member_vector[i].dblock; }
static inline const struct Block * Li_mblock( const struct Lzip_index * const li,
static inline const Block * Li_mblock( const Lzip_index * const li,
const long i )
{ return &li->member_vector[i].mblock; }
static inline unsigned Li_dictionary_size( const struct Lzip_index * const li,
static inline unsigned Li_dictionary_size( const Lzip_index * const li,
const long i )
{ return li->member_vector[i].dictionary_size; }

176
main.c
View file

@ -26,7 +26,7 @@
#include <ctype.h>
#include <errno.h>
#include <fcntl.h>
#include <limits.h> /* SSIZE_MAX */
#include <limits.h> /* CHAR_BIT, SSIZE_MAX */
#include <signal.h>
#include <stdbool.h>
#include <stdint.h> /* SIZE_MAX */
@ -39,8 +39,10 @@
#if defined __MSVCRT__ || defined __OS2__ || defined __DJGPP__
#include <io.h>
#if defined __MSVCRT__
#include <direct.h>
#define fchmod(x,y) 0
#define fchown(x,y,z) 0
#define mkdir(name,mode) _mkdir(name)
#define strtoull strtoul
#define SIGHUP SIGTERM
#define S_ISSOCK(x) 0
@ -88,13 +90,13 @@ static const struct { const char * from; const char * to; } known_extensions[] =
{ ".tlz", ".tar" },
{ 0, 0 } };
struct Lzma_options
typedef struct Lzma_options
{
int dictionary_size; /* 4 KiB .. 512 MiB */
int match_len_limit; /* 5 .. 273 */
};
} Lzma_options;
enum Mode { m_compress, m_decompress, m_list, m_test };
typedef enum Mode { m_compress, m_decompress, m_list, m_test } Mode;
/* Variables used in signal handler context.
They are not declared volatile because the handler never returns. */
@ -105,26 +107,25 @@ static bool delete_output_on_interrupt = false;
static void show_help( void )
{
printf( "Clzip is a C language version of lzip, compatible with lzip 1.4 or newer. As\n"
"clzip is written in C, it may be easier to integrate in applications like\n"
"package managers, embedded devices, or systems lacking a C++ compiler.\n"
printf( "Clzip is a C language version of lzip intended for systems lacking a C++\n"
"compiler.\n"
"\nLzip is a lossless data compressor with a user interface similar to the one\n"
"of gzip or bzip2. Lzip uses a simplified form of the 'Lempel-Ziv-Markov\n"
"chain-Algorithm' (LZMA) stream format to maximize interoperability. The\n"
"maximum dictionary size is 512 MiB so that any lzip file can be decompressed\n"
"on 32-bit machines. Lzip provides accurate and robust 3-factor integrity\n"
"checking. Lzip can compress about as fast as gzip (lzip -0) or compress most\n"
"files more than bzip2 (lzip -9). Decompression speed is intermediate between\n"
"gzip and bzip2. Lzip is better than gzip and bzip2 from a data recovery\n"
"perspective. Lzip has been designed, written, and tested with great care to\n"
"replace gzip and bzip2 as the standard general-purpose compressed format for\n"
"Unix-like systems.\n"
"of gzip or bzip2. Lzip uses a simplified form of LZMA (Lempel-Ziv-Markov\n"
"chain-Algorithm) designed to achieve complete interoperability between\n"
"implementations. The maximum dictionary size is 512 MiB so that any lzip\n"
"file can be decompressed on 32-bit machines. Lzip provides accurate and\n"
"robust 3-factor integrity checking. 'lzip -0' compresses about as fast as\n"
"gzip, while 'lzip -9' compresses most files more than bzip2. Decompression\n"
"speed is intermediate between gzip and bzip2. Lzip provides better data\n"
"recovery capabilities than gzip and bzip2. Lzip has been designed, written,\n"
"and tested with great care to replace gzip and bzip2 as general-purpose\n"
"compressed format for Unix-like systems.\n"
"\nUsage: %s [options] [files]\n", invocation_name );
printf( "\nOptions:\n"
" -h, --help display this help and exit\n"
" -V, --version output version information and exit\n"
" -a, --trailing-error exit with error status if trailing data\n"
" -b, --member-size=<bytes> set member size limit in bytes\n"
" -b, --member-size=<bytes> set member size limit of multimember files\n"
" -c, --stdout write to standard output, keep input files\n"
" -d, --decompress decompress, test compressed file integrity\n"
" -f, --force overwrite existing output files\n"
@ -141,8 +142,6 @@ static void show_help( void )
" -0 .. -9 set compression level [default 6]\n"
" --fast alias for -0\n"
" --best alias for -9\n"
" --empty-error exit with error status if empty member in file\n"
" --marking-error exit with error status if 1st LZMA byte not 0\n"
" --loose-trailing allow trailing data seeming corrupt header\n"
"\nIf no file names are given, or if a file is '-', clzip compresses or\n"
"decompresses from standard input to standard output.\n"
@ -199,7 +198,7 @@ struct Pretty_print
bool first_post;
};
static void Pp_init( struct Pretty_print * const pp,
static void Pp_init( Pretty_print * const pp,
const char * const filenames[], const int num_filenames )
{
pp->name = 0;
@ -220,8 +219,10 @@ static void Pp_init( struct Pretty_print * const pp,
if( pp->longest_name == 0 ) pp->longest_name = stdin_name_len;
}
static void Pp_set_name( struct Pretty_print * const pp,
const char * const filename )
void Pp_free( Pretty_print * const pp )
{ if( pp->padded_name ) { free( pp->padded_name ); pp->padded_name = 0; } }
static void Pp_set_name( Pretty_print * const pp, const char * const filename )
{
unsigned name_len, padded_name_len, i = 0;
@ -239,10 +240,10 @@ static void Pp_set_name( struct Pretty_print * const pp,
pp->first_post = true;
}
static void Pp_reset( struct Pretty_print * const pp )
static void Pp_reset( Pretty_print * const pp )
{ if( pp->name && pp->name[0] ) pp->first_post = true; }
void Pp_show_msg( struct Pretty_print * const pp, const char * const msg )
void Pp_show_msg( Pretty_print * const pp, const char * const msg )
{
if( verbosity < 0 ) return;
if( pp->first_post )
@ -272,7 +273,7 @@ const char * format_ds( const unsigned dictionary_size )
const char * p = "";
const char * np = " ";
unsigned num = dictionary_size;
bool exact = ( num % factor == 0 );
bool exact = num % factor == 0;
int i; for( i = 0; i < n && ( num > 9999 || ( exact && num >= factor ) ); ++i )
{ num /= factor; if( num % factor != 0 ) exact = false;
@ -288,7 +289,7 @@ void show_header( const unsigned dictionary_size )
}
/* separate numbers of 5 or more digits in groups of 3 digits using '_' */
/* separate numbers of 6 or more digits in groups of 3 digits using '_' */
static const char * format_num3( unsigned long long num )
{
enum { buffers = 8, bufsize = 4 * sizeof num, n = 10 };
@ -300,7 +301,7 @@ static const char * format_num3( unsigned long long num )
char * const buf = buffer[current++]; current %= buffers;
char * p = buf + bufsize - 1; /* fill the buffer backwards */
*p = 0; /* terminator */
if( num > 1024 )
if( num > 9999 )
{
char prefix = 0; /* try binary first, then si */
for( i = 0; i < n && num != 0 && num % 1024 == 0; ++i )
@ -311,7 +312,7 @@ static const char * format_num3( unsigned long long num )
{ num /= 1000; prefix = si_prefix[i]; }
if( prefix ) *(--p) = prefix;
}
const bool split = num >= 10000;
const bool split = num >= 100000;
for( i = 0; ; )
{
@ -346,7 +347,7 @@ static unsigned long long getnum( const char * const arg,
if( !errno && tail[0] )
{
const unsigned factor = ( tail[1] == 'i' ) ? 1024 : 1000;
const unsigned factor = (tail[1] == 'i') ? 1024 : 1000;
int exponent = 0; /* 0 = bad multiplier */
int i;
switch( tail[0] )
@ -396,7 +397,7 @@ static int get_dict_size( const char * const arg, const char * const option_name
}
static void set_mode( enum Mode * const program_modep, const enum Mode new_mode )
static void set_mode( Mode * const program_modep, const Mode new_mode )
{
if( *program_modep != m_compress && *program_modep != new_mode )
{
@ -473,9 +474,9 @@ int open_instream( const char * const name, struct stat * const in_statsp,
{
const int i = fstat( infd, in_statsp );
const mode_t mode = in_statsp->st_mode;
const bool can_read = ( i == 0 && !reg_only &&
const bool can_read = i == 0 && !reg_only &&
( S_ISBLK( mode ) || S_ISCHR( mode ) ||
S_ISFIFO( mode ) || S_ISSOCK( mode ) ) );
S_ISFIFO( mode ) || S_ISSOCK( mode ) );
if( i != 0 || ( !S_ISREG( mode ) && ( !can_read || one_to_one ) ) )
{
if( verbosity >= 0 )
@ -491,13 +492,13 @@ int open_instream( const char * const name, struct stat * const in_statsp,
static int open_instream2( const char * const name, struct stat * const in_statsp,
const enum Mode program_mode, const int eindex,
const Mode program_mode, const int eindex,
const bool one_to_one, const bool recompress )
{
if( program_mode == m_compress && !recompress && eindex >= 0 )
{
if( verbosity >= 0 )
fprintf( stderr, "%s: %s: Input file already has '%s' suffix.\n",
fprintf( stderr, "%s: %s: Input file already has '%s' suffix, ignored.\n",
program_name, name, known_extensions[eindex].from );
return -1;
}
@ -519,7 +520,7 @@ static bool make_dirs( const char * const name )
while( i < dirsize && name[i] != '/' ) ++i;
if( first < i )
{
char partial[i+1]; memcpy( partial, name, i ); partial[i] = 0;
char partial[i+1]; memcpy( partial, name, i ); partial[i] = 0; /* vla */
const mode_t mode = S_IRWXU | S_IRGRP | S_IXGRP | S_IROTH | S_IXOTH;
struct stat st;
if( stat( partial, &st ) == 0 )
@ -590,7 +591,7 @@ static void signal_handler( int sig )
static bool check_tty_in( const char * const input_filename, const int infd,
const enum Mode program_mode, int * const retval )
const Mode program_mode, int * const retval )
{
if( ( program_mode == m_decompress || program_mode == m_test ) &&
isatty( infd ) ) /* for example /dev/tty */
@ -602,7 +603,7 @@ static bool check_tty_in( const char * const input_filename, const int infd,
return true;
}
static bool check_tty_out( const enum Mode program_mode )
static bool check_tty_out( const Mode program_mode )
{
if( program_mode == m_compress && isatty( outfd ) )
{ show_file_error( output_filename[0] ?
@ -661,30 +662,26 @@ static bool next_filename( void )
}
struct Poly_encoder
{
struct LZ_encoder_base * eb;
struct LZ_encoder * e;
struct FLZ_encoder * fe;
};
static int compress( const unsigned long long cfile_size,
const unsigned long long member_size,
const unsigned long long volume_size, const int infd,
const struct Lzma_options * const encoder_options,
struct Pretty_print * const pp,
const Lzma_options * const encoder_options,
Pretty_print * const pp,
const struct stat * const in_statsp, const bool zero )
{
int retval = 0;
struct Poly_encoder encoder = { 0, 0, 0 }; /* polymorphic encoder */
struct
{
LZ_encoder_base * eb;
LZ_encoder * e;
FLZ_encoder * fe;
} encoder = { 0, 0, 0 }; /* polymorphic encoder */
if( verbosity >= 1 ) Pp_show_msg( pp, 0 );
{
bool error = false;
if( zero )
{
encoder.fe = (struct FLZ_encoder *)malloc( sizeof *encoder.fe );
encoder.fe = (FLZ_encoder *)malloc( sizeof *encoder.fe );
if( !encoder.fe || !FLZe_init( encoder.fe, infd, outfd ) ) error = true;
else encoder.eb = &encoder.fe->eb;
}
@ -694,7 +691,7 @@ static int compress( const unsigned long long cfile_size,
if( Lh_set_dictionary_size( header, encoder_options->dictionary_size ) &&
encoder_options->match_len_limit >= min_match_len_limit &&
encoder_options->match_len_limit <= max_match_len )
encoder.e = (struct LZ_encoder *)malloc( sizeof *encoder.e );
encoder.e = (LZ_encoder *)malloc( sizeof *encoder.e );
else internal_error( "invalid argument to encoder." );
if( !encoder.e || !LZe_init( encoder.e, Lh_get_dictionary_size( header ),
encoder_options->match_len_limit, infd, outfd ) )
@ -709,9 +706,10 @@ static int compress( const unsigned long long cfile_size,
}
unsigned long long in_size = 0, out_size = 0, partial_volume_size = 0;
int retval = 0;
while( true ) /* encode one member per iteration */
{
const unsigned long long size = ( volume_size > 0 ) ?
const unsigned long long size = (volume_size > 0) ?
min( member_size, volume_size - partial_volume_size ) : member_size;
show_cprogress( cfile_size, in_size, &encoder.eb->mb, pp ); /* init */
if( ( zero && !FLZe_encode_member( encoder.fe, size ) ) ||
@ -765,7 +763,7 @@ static unsigned char xdigit( const unsigned value ) /* hex digit for 'value' */
static bool show_trailing_data( const uint8_t * const data, const int size,
struct Pretty_print * const pp, const bool all,
Pretty_print * const pp, const bool all,
const int ignore_trailing ) /* -1 = show */
{
if( verbosity >= 4 || ignore_trailing <= 0 )
@ -793,15 +791,16 @@ static bool show_trailing_data( const uint8_t * const data, const int size,
static int decompress( const unsigned long long cfile_size, const int infd,
const struct Cl_options * const cl_opts,
struct Pretty_print * const pp, const bool testing )
const Cl_options * const cl_opts, Pretty_print * const pp,
const bool from_stdin, const bool testing )
{
unsigned long long partial_file_pos = 0;
struct Range_decoder rdec;
Range_decoder rdec;
int retval = 0;
bool first_member;
if( !Rd_init( &rdec, infd ) )
{ show_error( mem_msg, 0, false ); cleanup_and_fail( 1 ); }
bool empty = false, multi = false;
for( first_member = true; ; first_member = false )
{
@ -841,11 +840,11 @@ static int decompress( const unsigned long long cfile_size, const int infd,
if( verbosity >= 2 || ( verbosity == 1 && first_member ) )
Pp_show_msg( pp, 0 );
struct LZ_decoder decoder;
LZ_decoder decoder;
if( !LZd_init( &decoder, &rdec, dictionary_size, outfd ) )
{ Pp_show_msg( pp, mem_msg ); retval = 1; break; }
show_dprogress( cfile_size, partial_file_pos, &rdec, pp ); /* init */
const int result = LZd_decode_member( &decoder, cl_opts, pp );
const int result = LZd_decode_member( &decoder, pp );
partial_file_pos += Rd_member_position( &rdec );
LZd_free( &decoder );
if( result != 0 )
@ -857,16 +856,19 @@ static int decompress( const unsigned long long cfile_size, const int infd,
"File ends unexpectedly" : "Decoder error",
partial_file_pos );
}
else if( result == 5 ) Pp_show_msg( pp, empty_msg );
else if( result == 6 ) Pp_show_msg( pp, marking_msg );
else if( result == 5 ) Pp_show_msg( pp, nonzero_msg );
retval = 2; break;
}
if( !from_stdin ) { multi = !first_member;
if( LZd_data_position( &decoder ) == 0 ) empty = true; }
if( verbosity >= 2 )
{ fputs( testing ? "ok\n" : "done\n", stderr ); Pp_reset( pp ); }
}
Rd_free( &rdec );
if( verbosity == 1 && retval == 0 )
fputs( testing ? "ok\n" : "done\n", stderr );
if( empty && multi && retval == 0 )
{ show_file_error( pp->name, empty_msg, 0 ); retval = 2; }
return retval;
}
@ -904,13 +906,13 @@ void internal_error( const char * const msg )
void show_cprogress( const unsigned long long cfile_size,
const unsigned long long partial_size,
const struct Matchfinder_base * const m,
struct Pretty_print * const p )
const Matchfinder_base * const m,
Pretty_print * const p )
{
static unsigned long long csize = 0; /* file_size / 100 */
static unsigned long long psize = 0;
static const struct Matchfinder_base * mb = 0;
static struct Pretty_print * pp = 0;
static const Matchfinder_base * mb = 0;
static Pretty_print * pp = 0;
static bool enabled = true;
if( !enabled ) return;
@ -933,13 +935,13 @@ void show_cprogress( const unsigned long long cfile_size,
void show_dprogress( const unsigned long long cfile_size,
const unsigned long long partial_size,
const struct Range_decoder * const d,
struct Pretty_print * const p )
const Range_decoder * const d,
Pretty_print * const p )
{
static unsigned long long csize = 0; /* file_size / 100 */
static unsigned long long psize = 0;
static const struct Range_decoder * rdec = 0;
static struct Pretty_print * pp = 0;
static const Range_decoder * rdec = 0;
static Pretty_print * pp = 0;
static int counter = 0;
static bool enabled = true;
@ -966,7 +968,7 @@ int main( const int argc, const char * const argv[] )
{
/* Mapping from gzip/bzip2 style 0..9 compression levels to the
corresponding LZMA compression parameters. */
const struct Lzma_options option_mapping[] =
const Lzma_options option_mapping[] =
{
{ 1 << 16, 16 }, /* -0 */
{ 1 << 20, 5 }, /* -1 */
@ -978,15 +980,14 @@ int main( const int argc, const char * const argv[] )
{ 1 << 24, 68 }, /* -7 */
{ 3 << 23, 132 }, /* -8 */
{ 1 << 25, 273 } }; /* -9 */
struct Lzma_options encoder_options = option_mapping[6]; /* default = "-6" */
Lzma_options encoder_options = option_mapping[6]; /* default = "-6" */
const unsigned long long max_member_size = 0x0008000000000000ULL; /* 2 PiB */
const unsigned long long max_volume_size = 0x4000000000000000ULL; /* 4 EiB */
unsigned long long member_size = max_member_size;
unsigned long long volume_size = 0;
const char * default_output_filename = "";
enum Mode program_mode = m_compress;
int i;
struct Cl_options cl_opts; /* command-line options */
Mode program_mode = m_compress;
Cl_options cl_opts; /* command-line options */
Cl_options_init( &cl_opts );
bool force = false;
bool keep_input_files = false;
@ -995,8 +996,8 @@ int main( const int argc, const char * const argv[] )
bool zero = false;
if( argc > 0 ) invocation_name = argv[0];
enum { opt_eer = 256, opt_lt, opt_mer };
const struct ap_Option options[] =
enum { opt_lt = 256 };
const ap_Option options[] =
{
{ '0', "fast", ap_no },
{ '1', 0, ap_no },
@ -1026,15 +1027,13 @@ int main( const int argc, const char * const argv[] )
{ 't', "test", ap_no },
{ 'v', "verbose", ap_no },
{ 'V', "version", ap_no },
{ opt_eer, "empty-error", ap_no },
{ opt_lt, "loose-trailing", ap_no },
{ opt_mer, "marking-error", ap_no },
{ 0, 0, ap_no } };
CRC32_init();
/* static because valgrind complains and memory management in C sucks */
static struct Arg_parser parser;
static Arg_parser parser;
if( !ap_init( &parser, argc, argv, options, 0 ) )
{ show_error( mem_msg, 0, false ); return 1; }
if( ap_error( &parser ) ) /* bad option */
@ -1049,9 +1048,8 @@ int main( const int argc, const char * const argv[] )
const char * const arg = ap_argument( &parser, argind );
switch( code )
{
case '0': case '1': case '2': case '3': case '4':
case '5': case '6': case '7': case '8': case '9':
zero = ( code == '0' );
case '0': case '1': case '2': case '3': case '4': case '5':
case '6': case '7': case '8': case '9': zero = code == '0';
encoder_options = option_mapping[code-'0']; break;
case 'a': cl_opts.ignore_trailing = false; break;
case 'b': member_size = getnum( arg, pn, 100000, max_member_size ); break;
@ -1065,7 +1063,7 @@ int main( const int argc, const char * const argv[] )
case 'm': encoder_options.match_len_limit =
getnum( arg, pn, min_match_len_limit, max_match_len );
zero = false; break;
case 'n': break;
case 'n': break; /* ignored */
case 'o': if( strcmp( arg, "-" ) == 0 ) to_stdout = true;
else { default_output_filename = arg; } break;
case 'q': verbosity = -1; break;
@ -1075,9 +1073,7 @@ int main( const int argc, const char * const argv[] )
case 't': set_mode( &program_mode, m_test ); break;
case 'v': if( verbosity < 4 ) ++verbosity; break;
case 'V': show_version(); return 0;
case opt_eer: cl_opts.ignore_empty = false; break;
case opt_lt: cl_opts.loose_trailing = true; break;
case opt_mer: cl_opts.ignore_marking = false; break;
default: internal_error( "uncaught option." );
}
} /* end process options */
@ -1092,6 +1088,7 @@ int main( const int argc, const char * const argv[] )
filenames = resize_buffer( filenames, num_filenames * sizeof filenames[0] );
filenames[0] = "-";
int i;
bool filenames_given = false;
for( i = 0; argind + i < ap_arguments( &parser ); ++i )
{
@ -1126,7 +1123,7 @@ int main( const int argc, const char * const argv[] )
if( !to_stdout && program_mode != m_test && ( filenames_given || to_file ) )
set_signals( signal_handler );
static struct Pretty_print pp;
static Pretty_print pp;
Pp_init( &pp, filenames, num_filenames );
int failed_tests = 0;
@ -1138,9 +1135,10 @@ int main( const int argc, const char * const argv[] )
{
const char * input_filename = "";
int infd;
const bool from_stdin = strcmp( filenames[i], "-" ) == 0;
Pp_set_name( &pp, filenames[i] );
if( strcmp( filenames[i], "-" ) == 0 )
if( from_stdin )
{
if( stdin_used ) continue; else stdin_used = true;
infd = STDIN_FILENO;
@ -1189,7 +1187,8 @@ int main( const int argc, const char * const argv[] )
tmp = compress( cfile_size, member_size, volume_size, infd,
&encoder_options, &pp, in_statsp, zero );
else
tmp = decompress( cfile_size, infd, &cl_opts, &pp, program_mode == m_test );
tmp = decompress( cfile_size, infd, &cl_opts, &pp, from_stdin,
program_mode == m_test );
if( close( infd ) != 0 )
{ show_file_error( pp.name, "Error closing input file", errno );
set_retval( &tmp, 1 ); }
@ -1217,6 +1216,7 @@ int main( const int argc, const char * const argv[] )
program_name, failed_tests,
( failed_tests == 1 ) ? "file" : "files" );
free( output_filename );
Pp_free( &pp );
free( filenames );
ap_free( &parser );
return retval;

163
testsuite/check.sh
View file

@ -28,12 +28,10 @@ if [ -d tmp ] ; then rm -rf tmp ; fi
mkdir tmp
cd "${objdir}"/tmp || framework_failure
cat "${testdir}"/test.txt > in || framework_failure
cp "${testdir}"/test.txt in || framework_failure
in_lz="${testdir}"/test.txt.lz
in_em="${testdir}"/test_em.txt.lz
fox_lz="${testdir}"/fox.lz
fox6_lz="${testdir}"/fox6.lz
f6mk_lz="${testdir}"/fox6_mark.lz
fnz_lz="${testdir}"/fox_nz.lz
fail=0
test_failed() { fail=1 ; printf " $1" ; [ -z "$2" ] || printf "($2)" ; }
@ -70,7 +68,7 @@ done
"${LZIP}" -q -o out.lz nx_file
[ $? = 1 ] || test_failed $LINENO
[ ! -e out.lz ] || test_failed $LINENO
"${LZIP}" -qf -S100k -o out in in
"${LZIP}" -qf -S100k -o out in in # only one file with -o and -S
[ $? = 1 ] || test_failed $LINENO
{ [ ! -e out ] && [ ! -e out.lz ] ; } || test_failed $LINENO
# these are for code coverage
@ -106,37 +104,25 @@ printf "LZIP\001+.............................." | "${LZIP}" -t 2> /dev/null
printf "\ntesting decompression..."
for i in "${in_lz}" "${in_em}" ; do
"${LZIP}" -lq "$i" || test_failed $LINENO "$i"
"${LZIP}" -t "$i" || test_failed $LINENO "$i"
"${LZIP}" -d "$i" -o out || test_failed $LINENO "$i"
cmp in out || test_failed $LINENO "$i"
"${LZIP}" -cd "$i" > out || test_failed $LINENO "$i"
cmp in out || test_failed $LINENO "$i"
"${LZIP}" -d "$i" -o - > out || test_failed $LINENO "$i"
cmp in out || test_failed $LINENO "$i"
"${LZIP}" -d < "$i" > out || test_failed $LINENO "$i"
cmp in out || test_failed $LINENO "$i"
rm -f out || framework_failure
done
"${LZIP}" -l "${in_lz}" > /dev/null || test_failed $LINENO
"${LZIP}" -t "${in_lz}" || test_failed $LINENO
"${LZIP}" -d "${in_lz}" -o out || test_failed $LINENO
cmp in out || test_failed $LINENO
"${LZIP}" -cd "${in_lz}" > out || test_failed $LINENO
cmp in out || test_failed $LINENO
"${LZIP}" -d "${in_lz}" -o - > out || test_failed $LINENO
cmp in out || test_failed $LINENO
"${LZIP}" -d < "${in_lz}" > out || test_failed $LINENO
cmp in out || test_failed $LINENO
rm -f out || framework_failure
lines=`"${LZIP}" -tvv "${in_em}" 2>&1 | wc -l` || test_failed $LINENO
[ "${lines}" -eq 8 ] || test_failed $LINENO "${lines}"
"${LZIP}" -tq "${in_em}" --empty-error
[ $? = 2 ] || test_failed $LINENO
lines=`"${LZIP}" -lvv "${in_em}" | wc -l` || test_failed $LINENO
[ "${lines}" -eq 11 ] || test_failed $LINENO "${lines}"
"${LZIP}" -lq "${in_em}" --empty-error
[ $? = 2 ] || test_failed $LINENO
cat "${in_lz}" > out.lz || framework_failure
cp "${in_lz}" out.lz || framework_failure
"${LZIP}" -dk out.lz || test_failed $LINENO
cmp in out || test_failed $LINENO
rm -f out || framework_failure
"${LZIP}" -cd "${fox_lz}" > fox || test_failed $LINENO
cat fox > copy || framework_failure
cat "${in_lz}" > copy.lz || framework_failure
cp fox copy || framework_failure
cp "${in_lz}" copy.lz || framework_failure
"${LZIP}" -d copy.lz out.lz 2> /dev/null # skip copy, decompress out
[ $? = 1 ] || test_failed $LINENO
[ ! -e out.lz ] || test_failed $LINENO
@ -147,7 +133,7 @@ cmp in out || test_failed $LINENO
cmp in copy || test_failed $LINENO
rm -f copy out || framework_failure
cat "${in_lz}" > out.lz || framework_failure
cp "${in_lz}" out.lz || framework_failure
"${LZIP}" -d -S100k out.lz || test_failed $LINENO # ignore -S
[ ! -e out.lz ] || test_failed $LINENO
cmp in out || test_failed $LINENO
@ -155,7 +141,6 @@ cmp in out || test_failed $LINENO
printf "to be overwritten" > out || framework_failure
"${LZIP}" -df -o out < "${in_lz}" || test_failed $LINENO
cmp in out || test_failed $LINENO
rm -f out || framework_failure
"${LZIP}" -d -o ./- "${in_lz}" || test_failed $LINENO
cmp in ./- || test_failed $LINENO
rm -f ./- || framework_failure
@ -163,12 +148,12 @@ rm -f ./- || framework_failure
cmp in ./- || test_failed $LINENO
rm -f ./- || framework_failure
cat "${in_lz}" > anyothername || framework_failure
cp "${in_lz}" anyothername || framework_failure
"${LZIP}" -dv - anyothername - < "${in_lz}" > out 2> /dev/null ||
test_failed $LINENO
cmp in out || test_failed $LINENO
cmp in anyothername.out || test_failed $LINENO
rm -f out anyothername.out || framework_failure
rm -f anyothername.out || framework_failure
"${LZIP}" -lq in "${in_lz}"
[ $? = 2 ] || test_failed $LINENO
@ -185,7 +170,7 @@ cat out in | cmp in - || test_failed $LINENO # out must be empty
[ $? = 1 ] || test_failed $LINENO
cmp in out || test_failed $LINENO
rm -f out || framework_failure
cat "${in_lz}" > out.lz || framework_failure
cp "${in_lz}" out.lz || framework_failure
for i in 1 2 3 4 5 6 7 ; do
printf "g" >> out.lz || framework_failure
"${LZIP}" -alvv out.lz "${in_lz}" > /dev/null 2>&1
@ -206,7 +191,7 @@ cmp in out || test_failed $LINENO
rm -f out || framework_failure
cat in in > in2 || framework_failure
"${LZIP}" -lq "${in_lz}" "${in_lz}" || test_failed $LINENO
"${LZIP}" -l "${in_lz}" "${in_lz}" > /dev/null || test_failed $LINENO
"${LZIP}" -t "${in_lz}" "${in_lz}" || test_failed $LINENO
"${LZIP}" -cd "${in_lz}" "${in_lz}" -o out > out2 || test_failed $LINENO
[ ! -e out ] || test_failed $LINENO # override -o
@ -217,6 +202,11 @@ cmp in2 out2 || test_failed $LINENO
rm -f out2 || framework_failure
cat "${in_lz}" "${in_lz}" > out2.lz || framework_failure
lines=`"${LZIP}" -tvv out2.lz 2>&1 | wc -l` || test_failed $LINENO
[ "${lines}" -eq 2 ] || test_failed $LINENO "${lines}"
lines=`"${LZIP}" -lvv out2.lz | wc -l` || test_failed $LINENO
[ "${lines}" -eq 5 ] || test_failed $LINENO "${lines}"
printf "\ngarbage" >> out2.lz || framework_failure
"${LZIP}" -tvvvv out2.lz 2> /dev/null || test_failed $LINENO
"${LZIP}" -alq out2.lz
@ -236,15 +226,6 @@ printf "to be overwritten" > out2 || framework_failure
cmp in2 out2 || test_failed $LINENO
rm -f out2 || framework_failure
"${LZIP}" -cd "${fox6_lz}" > out || test_failed $LINENO
"${LZIP}" -cd "${f6mk_lz}" > copy || test_failed $LINENO
cmp copy out || test_failed $LINENO
rm -f copy out || framework_failure
"${LZIP}" -lq "${f6mk_lz}" --marking-error
[ $? = 2 ] || test_failed $LINENO
"${LZIP}" -tq "${f6mk_lz}" --marking-error
[ $? = 2 ] || test_failed $LINENO
"${LZIP}" -d "${fox_lz}" -o a/b/c/fox || test_failed $LINENO
cmp fox a/b/c/fox || test_failed $LINENO
rm -rf a || framework_failure
@ -255,6 +236,21 @@ rm -rf a || framework_failure
[ $? = 1 ] || test_failed $LINENO
[ ! -e a ] || test_failed $LINENO
touch empty em || framework_failure
"${LZIP}" -0 em || test_failed $LINENO
"${LZIP}" -l em.lz > /dev/null || test_failed $LINENO
"${LZIP}" -dk em.lz || test_failed $LINENO
cmp empty em || test_failed $LINENO
cat em.lz em.lz | "${LZIP}" -t || test_failed $LINENO
cat em.lz em.lz | "${LZIP}" -d > em || test_failed $LINENO
cmp empty em || test_failed $LINENO
cat em.lz "${in_lz}" | "${LZIP}" -t || test_failed $LINENO
cat em.lz "${in_lz}" | "${LZIP}" -d > out || test_failed $LINENO
cmp in out || test_failed $LINENO
cat "${in_lz}" em.lz | "${LZIP}" -t || test_failed $LINENO
cat "${in_lz}" em.lz | "${LZIP}" -d > out || test_failed $LINENO
cmp in out || test_failed $LINENO
printf "\ntesting compression..."
"${LZIP}" -c -0 in in in -S100k -o out3.lz > copy2.lz || test_failed $LINENO
@ -263,7 +259,7 @@ printf "\ntesting compression..."
"${LZIP}" -d copy2.lz -o out2 || test_failed $LINENO
[ -e copy2.lz ] || test_failed $LINENO
cmp in2 out2 || test_failed $LINENO
rm -f in2 out2 copy2.lz || framework_failure
rm -f copy2.lz || framework_failure
"${LZIP}" -cf "${in_lz}" > lzlz 2> /dev/null # /dev/null is a tty on OS/2
[ $? = 1 ] || test_failed $LINENO
@ -331,7 +327,7 @@ rm -f in8 || framework_failure
"${LZIP}" -t out00001.lz out00002.lz || test_failed $LINENO
"${LZIP}" -cd out00001.lz out00002.lz | cmp in8.lz - || test_failed $LINENO
[ ! -e out00003.lz ] || test_failed $LINENO
rm -f out00001.lz || framework_failure
rm -f out00001.lz out00002.lz || framework_failure
"${LZIP}" -1 -S100k -o a/b/c/out < in8.lz || test_failed $LINENO
"${LZIP}" -t a/b/c/out00001.lz a/b/c/out00002.lz || test_failed $LINENO
"${LZIP}" -cd a/b/c/out00001.lz a/b/c/out00002.lz | cmp in8.lz - ||
@ -357,11 +353,44 @@ rm -rf a || framework_failure
printf "\ntesting bad input..."
cat em.lz em.lz > ee.lz || framework_failure
"${LZIP}" -l < ee.lz > /dev/null || test_failed $LINENO
"${LZIP}" -t < ee.lz || test_failed $LINENO
"${LZIP}" -d < ee.lz > em || test_failed $LINENO
cmp empty em || test_failed $LINENO
"${LZIP}" -lq ee.lz
[ $? = 2 ] || test_failed $LINENO
"${LZIP}" -tq ee.lz
[ $? = 2 ] || test_failed $LINENO
"${LZIP}" -dq ee.lz
[ $? = 2 ] || test_failed $LINENO
[ ! -e ee ] || test_failed $LINENO
"${LZIP}" -cdq ee.lz > em
[ $? = 2 ] || test_failed $LINENO
cmp empty em || test_failed $LINENO
rm -f empty em || framework_failure
cat "${in_lz}" em.lz "${in_lz}" > inein.lz || framework_failure
"${LZIP}" -l < inein.lz > /dev/null || test_failed $LINENO
"${LZIP}" -t < inein.lz || test_failed $LINENO
"${LZIP}" -d < inein.lz > out2 || test_failed $LINENO
cmp in2 out2 || test_failed $LINENO
"${LZIP}" -lq inein.lz
[ $? = 2 ] || test_failed $LINENO
"${LZIP}" -tq inein.lz
[ $? = 2 ] || test_failed $LINENO
"${LZIP}" -dq inein.lz
[ $? = 2 ] || test_failed $LINENO
[ ! -e inein ] || test_failed $LINENO
"${LZIP}" -cdq inein.lz > out2
[ $? = 2 ] || test_failed $LINENO
cmp in2 out2 || test_failed $LINENO
rm -f in2 out2 inein.lz em.lz || framework_failure
headers='LZIp LZiP LZip LzIP LzIp LziP lZIP lZIp lZiP lzIP'
body='\001\014\000\203\377\373\377\377\300\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000$\000\000\000\000\000\000\000'
cat "${in_lz}" > int.lz || framework_failure
body='\001\014\000\000\101\376\367\377\377\340\000\200\000\215\357\002\322\001\000\000\000\000\000\000\000\045\000\000\000\000\000\000\000'
cp "${in_lz}" int.lz || framework_failure
printf "LZIP${body}" >> int.lz || framework_failure
if "${LZIP}" -tq int.lz ; then
if "${LZIP}" -t int.lz ; then
for header in ${headers} ; do
printf "${header}${body}" > int.lz || framework_failure
"${LZIP}" -lq int.lz # first member
@ -380,7 +409,7 @@ if "${LZIP}" -tq int.lz ; then
[ $? = 2 ] || test_failed $LINENO ${header}
"${LZIP}" -cdq --loose-trailing int.lz > /dev/null
[ $? = 2 ] || test_failed $LINENO ${header}
cat "${in_lz}" > int.lz || framework_failure
cp "${in_lz}" int.lz || framework_failure
printf "${header}${body}" >> int.lz || framework_failure
"${LZIP}" -lq int.lz # trailing data
[ $? = 2 ] || test_failed $LINENO ${header}
@ -390,7 +419,7 @@ if "${LZIP}" -tq int.lz ; then
[ $? = 2 ] || test_failed $LINENO ${header}
"${LZIP}" -cdq int.lz > /dev/null
[ $? = 2 ] || test_failed $LINENO ${header}
"${LZIP}" -lq --loose-trailing int.lz ||
"${LZIP}" -l --loose-trailing int.lz > /dev/null ||
test_failed $LINENO ${header}
"${LZIP}" -t --loose-trailing int.lz ||
test_failed $LINENO ${header}
@ -408,10 +437,14 @@ if "${LZIP}" -tq int.lz ; then
[ $? = 2 ] || test_failed $LINENO ${header}
done
else
printf "\nwarning: skipping header test: 'printf' does not work on your system."
printf "warning: skipping header test: 'printf' does not work on your system."
fi
rm -f int.lz || framework_failure
"${LZIP}" -l "${fnz_lz}" > /dev/null || test_failed $LINENO
"${LZIP}" -tq "${fnz_lz}"
[ $? = 2 ] || test_failed $LINENO
for i in fox_v2.lz fox_s11.lz fox_de20.lz \
fox_bcrc.lz fox_crc0.lz fox_das46.lz fox_mes81.lz ; do
"${LZIP}" -tq "${testdir}"/$i
@ -423,13 +456,13 @@ for i in fox_bcrc.lz fox_crc0.lz fox_das46.lz fox_mes81.lz ; do
[ $? = 2 ] || test_failed $LINENO $i
cmp fox out || test_failed $LINENO $i
done
rm -f fox out || framework_failure
rm -f fox || framework_failure
cat "${in_lz}" "${in_lz}" > in2.lz || framework_failure
cat "${in_lz}" "${in_lz}" "${in_lz}" > in3.lz || framework_failure
if dd if=in3.lz of=trunc.lz bs=14752 count=1 2> /dev/null &&
[ -e trunc.lz ] && cmp in2.lz trunc.lz > /dev/null 2>&1 ; then
for i in 6 20 14734 14753 14754 14755 14756 14757 14758 ; do
if dd if=in3.lz of=trunc.lz bs=14682 count=1 2> /dev/null &&
[ -e trunc.lz ] && cmp in2.lz trunc.lz ; then
for i in 6 20 14664 14683 14684 14685 14686 14687 14688 ; do
dd if=in3.lz of=trunc.lz bs=$i count=1 2> /dev/null
"${LZIP}" -lq trunc.lz
[ $? = 2 ] || test_failed $LINENO $i
@ -443,11 +476,11 @@ if dd if=in3.lz of=trunc.lz bs=14752 count=1 2> /dev/null &&
[ $? = 2 ] || test_failed $LINENO $i
done
else
printf "\nwarning: skipping truncation test: 'dd' does not work on your system."
printf "warning: skipping truncation test: 'dd' does not work on your system."
fi
rm -f in2.lz in3.lz trunc.lz || framework_failure
cat "${in_lz}" > ingin.lz || framework_failure
cp "${in_lz}" ingin.lz || framework_failure
printf "g" >> ingin.lz || framework_failure
cat "${in_lz}" >> ingin.lz || framework_failure
"${LZIP}" -lq ingin.lz
@ -456,17 +489,21 @@ cat "${in_lz}" >> ingin.lz || framework_failure
[ $? = 2 ] || test_failed $LINENO
"${LZIP}" -atq < ingin.lz
[ $? = 2 ] || test_failed $LINENO
"${LZIP}" -acdq ingin.lz > /dev/null
"${LZIP}" -acdq ingin.lz > out
[ $? = 2 ] || test_failed $LINENO
"${LZIP}" -adq < ingin.lz > /dev/null
cmp in out || test_failed $LINENO
"${LZIP}" -adq < ingin.lz > out
[ $? = 2 ] || test_failed $LINENO
cmp in out || test_failed $LINENO
"${LZIP}" -t ingin.lz || test_failed $LINENO
"${LZIP}" -t < ingin.lz || test_failed $LINENO
"${LZIP}" -dk ingin.lz || test_failed $LINENO
cmp in ingin || test_failed $LINENO
"${LZIP}" -cd ingin.lz > out || test_failed $LINENO
cmp in out || test_failed $LINENO
"${LZIP}" -d < ingin.lz > out || test_failed $LINENO
cmp in out || test_failed $LINENO
rm -f out ingin.lz || framework_failure
rm -f out ingin ingin.lz || framework_failure
echo
if [ ${fail} = 0 ] ; then

BIN
testsuite/fox6.lz
View file

Binary file not shown.

BIN
testsuite/fox6_mark.lz
View file

Binary file not shown.

BIN
testsuite/fox_nz.lz Normal file
View file

Binary file not shown.

6
testsuite/test.txt
View file

@ -1,8 +1,7 @@
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Copyright (C) 1989, 1991 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
@ -339,8 +338,7 @@ Public License instead of this License.
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Copyright (C) 1989, 1991 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.

BIN
testsuite/test.txt.lz
View file

Binary file not shown.

BIN
testsuite/test_em.txt.lz
View file

Binary file not shown.