lzip/xlunzip
1
0
Fork 0
Code Issues Activity

Merging upstream version 0.5.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-17 22:51:25 +01:00
parent 2365d73c24
commit caa83acf9d
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
25 changed files with 264 additions and 251 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

26
ChangeLog
View file

@ -1,11 +1,25 @@
2020-04-27 Antonio Diaz Diaz <antonio@gnu.org>
* Version 0.5 released.
* lzip_decompress module updated to version 5.4.18.
* main.c (main): Report an error if a file name is empty.
* Replace 'decompressed', 'compressed' with 'out', 'in' in output.
* Decompression speed has been slightly increased.
* Fix a compilation error with GCC 10. (Reported by Daniel Baumann).
* main.c: Set a valid invocation_name even if argc == 0.
* Document extraction from tar.lz in '--help' output and man page.
* main.c: Compile on DOS with DJGPP.
* configure: Accept appending to CFLAGS, 'CFLAGS+=OPTIONS'.
* testsuite: Add 9 new test files.
2018-09-18 Antonio Diaz Diaz <antonio@gnu.org>
* Version 0.4 released.
* lzip_decompress module updated to version 4.18.1-2.
* lzip.c: Renamed to lzip_decompress.c.
* lzip_decompress.c (LZd_init): Fixed a warning on 32 bit systems.
* lzip.c: Rename to lzip_decompress.c.
* lzip_decompress.c (LZd_init): Fix a warning on 32 bit systems.
* in_place.c (set_file_sizes): Skip trailing zeros efficiently.
* main.c: Check return value of close( infd ).
* main.c (main): Check return value of close( infd ).
* INSTALL: Document use of '-D __USE_MINGW_ANSI_STDIO'.
2018-07-10 Antonio Diaz Diaz <antonio@gnu.org>
@ -18,7 +32,7 @@
* Version 0.2 released.
* lzip_decompress module updated to version 4.14.40-2.
* Improved corrupt header detection to HD=3.
* Improve corrupt header detection to HD=3.
* --in-place now works with both --decompress and --test.
* main.c: Show final diagnostic when testing multiple files.
@ -29,8 +43,8 @@
* Tests the code shipped in linux patches before june 2018.
Copyright (C) 2016-2018 Antonio Diaz Diaz.
Copyright (C) 2016-2020 Antonio Diaz Diaz.
This file is a collection of facts, and thus it is not copyrightable,
but just in case, you have unlimited permission to copy, distribute and
but just in case, you have unlimited permission to copy, distribute, and
modify it.

30
INSTALL
View file

@ -1,10 +1,14 @@
Requirements
------------
You will need a C compiler.
I use gcc 5.3.0 and 4.1.2, but the code should compile with any
standards compliant compiler.
I use gcc 6.1.0 and 4.1.2, but the code should compile with any standards
compliant compiler.
Gcc is available at http://gcc.gnu.org.
The operating system must allow signal handlers read access to objects with
static storage duration so that the cleanup handler for Control-C can delete
the partial output file.
Procedure
---------
@ -23,6 +27,10 @@ the main archive.
cd xlunzip[version]
./configure
If you are compiling on MinGW, use:
./configure CFLAGS+='-D __USE_MINGW_ANSI_STDIO'
3. Run make.
make
@ -36,25 +44,21 @@ the main archive.
man page after installation. (Installing compressed docs may become
the default in the future).
You can install only the program or the man page by typing 'make
install-bin' or 'make install-man' respectively.
You can install only the program or the man page by typing
'make install-bin' or 'make install-man' respectively.
Instead of 'make install', you can type 'make install-as-lzip' to
install the program and any data files and documentation, and link
the program to the name 'lzip'.
If you are compiling on MinGW, replace step 3 with:
make CFLAGS='-Wall -W -O2 -D __USE_MINGW_ANSI_STDIO'
Another way
-----------
You can also compile xlunzip into a separate directory.
To do this, you must use a version of 'make' that supports the 'VPATH'
variable, such as GNU 'make'. 'cd' to the directory where you want the
To do this, you must use a version of 'make' that supports the variable
'VPATH', such as GNU 'make'. 'cd' to the directory where you want the
object files and executables to go and run the 'configure' script.
'configure' automatically checks for the source code in '.', in '..' and
'configure' automatically checks for the source code in '.', in '..', and
in the directory that 'configure' is in.
'configure' recognizes the option '--srcdir=DIR' to control where to
@ -65,7 +69,7 @@ After running 'configure', you can run 'make' and 'make install' as
explained above.
Copyright (C) 2016-2018 Antonio Diaz Diaz.
Copyright (C) 2016-2020 Antonio Diaz Diaz.
This file is free documentation: you have unlimited permission to copy,
distribute and modify it.
distribute, and modify it.

5
Makefile.in
View file

@ -119,8 +119,11 @@ dist : doc
$(DISTNAME)/*.c \
$(DISTNAME)/testsuite/check.sh \
$(DISTNAME)/testsuite/test.txt \
$(DISTNAME)/testsuite/fox.lz \
$(DISTNAME)/testsuite/fox_*.lz \
$(DISTNAME)/testsuite/zero.lz \
$(DISTNAME)/testsuite/test.txt.lz \
$(DISTNAME)/testsuite/zero.lz
$(DISTNAME)/testsuite/test_em.txt.lz
rm -f $(DISTNAME)
lzip -v -9 $(DISTNAME).tar

25
NEWS
View file

@ -1,13 +1,22 @@
Changes in version 0.4:
Changes in version 0.5:
The lzip_decompress module has been updated to version 4.18.1-2.
The lzip_decompress module has been updated to version 5.4.18.
A harmless warning on 32 bit systems has been fixed.
Xlunzip now reports an error if a file name is empty (xlunzip -t "").
Large amounts of trailing zeros are now skipped more efficiently when
decompressing or testing in place.
The words 'decompressed' and 'compressed' have been replaced with the
shorter 'out' and 'in' in the verbose output when decompressing or testing.
Errors are now also checked when closing the input file.
Decompression speed has been slightly increased.
It has been documented in INSTALL the use of '-D __USE_MINGW_ANSI_STDIO'
when compiling on MinGW.
A compilation error with GCC 10 has been fixed. (Reported by Daniel Baumann).
The commands needed to extract files from a tar.lz archive have been
documented in the output of '--help' and in the man page.
Xlunzip now compiles on DOS with DJGPP. (Patch from Robert Riebisch).
The configure script now accepts appending options to CFLAGS using the
syntax 'CFLAGS+=OPTIONS'.
9 new test files have been added to the testsuite.

11
README
View file

@ -14,7 +14,8 @@ corrupted data.
Note that the in-place decompression of concatenated files can't be
guaranteed to work because an arbitrarily low compression ratio of the
last part of the data can be achieved by appending enough empty
compressed members to a file.
compressed members to a file, masking a high compression ratio at the
beginning of the data.
The xlunzip tarball contains a copy of the lzip_decompress module and
can be compiled and tested without downloading or applying the patch to
@ -26,8 +27,8 @@ http://download.savannah.gnu.org/releases/lzip/kernel/
Lzip related components in the kernel
=====================================
The lzip_decompress module in lib/lzip.c provides a versatile lzip
decompression function able to do buffer to buffer decompression or
The lzip_decompress module in lib/lzip_decompress.c provides a versatile
lzip decompression function able to do buffer to buffer decompression or
stream decompression with fill and flush callback functions. The usage
of the function is documented in include/linux/lzip.h.
@ -37,10 +38,10 @@ interface as the other decompress_*.c files, which is defined in
include/linux/decompress/generic.h.
Copyright (C) 2016-2018 Antonio Diaz Diaz.
Copyright (C) 2016-2020 Antonio Diaz Diaz.
This file is free documentation: you have unlimited permission to copy,
distribute and modify it.
distribute, and modify it.
The file Makefile.in is a data file used by configure to produce the
Makefile. It has the same copyright owner and permissions that configure

6
carg_parser.c
View file

@ -1,15 +1,15 @@
/* Arg_parser - POSIX/GNU command line argument parser. (C version)
Copyright (C) 2006-2018 Antonio Diaz Diaz.
Copyright (C) 2006-2020 Antonio Diaz Diaz.
This library is free software. Redistribution and use in source and
binary forms, with or without modification, are permitted provided
that the following conditions are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
notice, this list of conditions, and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
notice, this list of conditions, and the following disclaimer in the
documentation and/or other materials provided with the distribution.
This library is distributed in the hope that it will be useful,

8
carg_parser.h
View file

@ -1,15 +1,15 @@
/* Arg_parser - POSIX/GNU command line argument parser. (C version)
Copyright (C) 2006-2018 Antonio Diaz Diaz.
Copyright (C) 2006-2020 Antonio Diaz Diaz.
This library is free software. Redistribution and use in source and
binary forms, with or without modification, are permitted provided
that the following conditions are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
notice, this list of conditions, and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
notice, this list of conditions, and the following disclaimer in the
documentation and/or other materials provided with the distribution.
This library is distributed in the hope that it will be useful,
@ -18,7 +18,7 @@
*/
/* Arg_parser reads the arguments in 'argv' and creates a number of
option codes, option arguments and non-option arguments.
option codes, option arguments, and non-option arguments.
In case of error, 'ap_error' returns a non-null pointer to an error
message.

26
configure vendored
View file

@ -1,12 +1,12 @@
#! /bin/sh
# configure script for Xlunzip - Test tool for the lzip_decompress linux module
# Copyright (C) 2016-2018 Antonio Diaz Diaz.
# Copyright (C) 2016-2020 Antonio Diaz Diaz.
#
# This configure script is free software: you have unlimited permission
# to copy, distribute and modify it.
# to copy, distribute, and modify it.
pkgname=xlunzip
pkgversion=0.4
pkgversion=0.5
progname=xlunzip
srctrigger=doc/${progname}.1
@ -46,7 +46,7 @@ while [ $# != 0 ] ; do
# Split out the argument for options that take them
case ${option} in
*=*) optarg=`echo ${option} | sed -e 's,^[^=]*=,,;s,/$,,'` ;;
*=*) optarg=`echo "${option}" | sed -e 's,^[^=]*=,,;s,/$,,'` ;;
esac
# Process the options
@ -70,6 +70,7 @@ while [ $# != 0 ] ; do
echo " CC=COMPILER C compiler to use [${CC}]"
echo " CPPFLAGS=OPTIONS command line options for the preprocessor [${CPPFLAGS}]"
echo " CFLAGS=OPTIONS command line options for the C compiler [${CFLAGS}]"
echo " CFLAGS+=OPTIONS append options to the current value of CFLAGS"
echo " LDFLAGS=OPTIONS command line options for the linker [${LDFLAGS}]"
echo
exit 0 ;;
@ -93,10 +94,11 @@ while [ $# != 0 ] ; do
--mandir=*) mandir=${optarg} ;;
--no-create) no_create=yes ;;
CC=*) CC=${optarg} ;;
CPPFLAGS=*) CPPFLAGS=${optarg} ;;
CFLAGS=*) CFLAGS=${optarg} ;;
LDFLAGS=*) LDFLAGS=${optarg} ;;
CC=*) CC=${optarg} ;;
CPPFLAGS=*) CPPFLAGS=${optarg} ;;
CFLAGS=*) CFLAGS=${optarg} ;;
CFLAGS+=*) CFLAGS="${CFLAGS} ${optarg}" ;;
LDFLAGS=*) LDFLAGS=${optarg} ;;
--*)
echo "configure: WARNING: unrecognized option: '${option}'" 1>&2 ;;
@ -123,7 +125,7 @@ if [ -z "${srcdir}" ] ; then
if [ ! -r "${srcdir}/${srctrigger}" ] ; then srcdir=.. ; fi
if [ ! -r "${srcdir}/${srctrigger}" ] ; then
## the sed command below emulates the dirname command
srcdir=`echo $0 | sed -e 's,[^/]*$,,;s,/$,,;s,^$,.,'`
srcdir=`echo "$0" | sed -e 's,[^/]*$,,;s,/$,,;s,^$,.,'`
fi
fi
@ -146,7 +148,7 @@ if [ -z "${no_create}" ] ; then
# Run this file to recreate the current configuration.
#
# This script is free software: you have unlimited permission
# to copy, distribute and modify it.
# to copy, distribute, and modify it.
exec /bin/sh $0 ${args} --no-create
EOF
@ -168,11 +170,11 @@ echo "LDFLAGS = ${LDFLAGS}"
rm -f Makefile
cat > Makefile << EOF
# Makefile for Xlunzip - Test tool for the lzip_decompress linux module
# Copyright (C) 2016-2018 Antonio Diaz Diaz.
# Copyright (C) 2016-2020 Antonio Diaz Diaz.
# This file was generated automatically by configure. Don't edit.
#
# This Makefile is free software: you have unlimited permission
# to copy, distribute and modify it.
# to copy, distribute, and modify it.
pkgname = ${pkgname}
pkgversion = ${pkgversion}

2
decompress_lunzip.c
View file

@ -1,7 +1,7 @@
/*
* Wrapper for decompressing LZIP-compressed kernel, initramfs, and initrd
*
* Copyright (C) 2016-2018 Antonio Diaz Diaz.
* Copyright (C) 2016-2020 Antonio Diaz Diaz.
*
* Licensed under GPLv2 or later, see file LICENSE in this source tree.
*/

10
doc/xlunzip.1
View file

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.46.1.
.TH XLUNZIP "1" "September 2018" "xlunzip 0.4" "User Commands"
.TH XLUNZIP "1" "April 2020" "xlunzip 0.5" "User Commands"
.SH NAME
xlunzip \- test tool for the lzip_decompress linux module
.SH SYNOPSIS
@ -20,7 +20,8 @@ corrupted data.
Note that the in\-place decompression of concatenated files can't be
guaranteed to work because an arbitrarily low compression ratio of the
last part of the data can be achieved by appending enough empty
compressed members to a file.
compressed members to a file, masking a high compression ratio at the
beginning of the data.
.SH OPTIONS
.TP
\fB\-h\fR, \fB\-\-help\fR
@ -74,6 +75,9 @@ from standard input to standard output.
Numbers may be followed by a multiplier: k = kB = 10^3 = 1000,
Ki = KiB = 2^10 = 1024, M = 10^6, Mi = 2^20, G = 10^9, Gi = 2^30, etc...
.PP
To extract all the files from archive 'foo.tar.lz', use the commands
\&'tar \fB\-xf\fR foo.tar.lz' or 'xlunzip \fB\-cd\fR foo.tar.lz | tar \fB\-xf\fR \-'.
.PP
Exit status: 0 for a normal exit, 1 for environmental problems (file
not found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or
invalid input file, 3 for an internal consistency error (eg, bug) which
@ -83,7 +87,7 @@ Report bugs to lzip\-bug@nongnu.org
.br
Xlunzip home page: http://www.nongnu.org/lzip/xlunzip.html
.SH COPYRIGHT
Copyright \(co 2018 Antonio Diaz Diaz.
Copyright \(co 2020 Antonio Diaz Diaz.
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>
.br
This is free software: you are free to change and redistribute it.

42
in_place.c
View file

@ -1,5 +1,5 @@
/* Xlunzip - Test tool for the lzip_decompress linux module
Copyright (C) 2016-2018 Antonio Diaz Diaz.
Copyright (C) 2016-2020 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -33,7 +33,7 @@
/* Returns the number of bytes really read.
If (returned value < size) and (errno == 0), means EOF was reached.
*/
long readblock( const int fd, uint8_t * const buf, const long size )
static long readblock( const int fd, uint8_t * const buf, const long size )
{
long sz = 0;
errno = 0;
@ -53,30 +53,30 @@ long readblock( const int fd, uint8_t * const buf, const long size )
the buffer and file sizes in '*buffer_sizep' and '*file_sizep'.
In case of error, returns 0 and does not modify '*size'.
*/
uint8_t * read_file( const int infd, long * const buffer_sizep,
long * const file_sizep, struct Pretty_print * const pp )
static uint8_t * read_file( const int infd, long * const buffer_sizep,
long * const file_sizep, const char * const filename )
{
long buffer_size = 1 << 20;
uint8_t * buffer = (uint8_t *)malloc( buffer_size );
if( !buffer )
{ show_file_error( pp->name, "Not enough memory.", 0 ); return 0; }
{ show_file_error( filename, "Not enough memory.", 0 ); return 0; }
long file_size = readblock( infd, buffer, buffer_size );
while( file_size >= buffer_size && !errno )
{
if( buffer_size >= LONG_MAX )
{ show_file_error( pp->name, "File is too large.", 0 ); free( buffer );
{ show_file_error( filename, "File is too large.", 0 ); free( buffer );
return 0; }
buffer_size = ( buffer_size <= LONG_MAX / 2 ) ? 2 * buffer_size : LONG_MAX;
uint8_t * const tmp = (uint8_t *)realloc( buffer, buffer_size );
if( !tmp )
{ show_file_error( pp->name, "Not enough memory.", 0 ); free( buffer );
{ show_file_error( filename, "Not enough memory.", 0 ); free( buffer );
return 0; }
buffer = tmp;
file_size += readblock( infd, buffer + file_size, buffer_size - file_size );
}
if( errno )
{ show_file_error( pp->name, "Error reading file", errno ); free( buffer );
{ show_file_error( filename, "Error reading file", errno ); free( buffer );
return 0; }
*buffer_sizep = buffer_size;
*file_sizep = file_size;
@ -91,9 +91,10 @@ struct File_sizes
long trailing;
};
const char * set_file_sizes( struct File_sizes * const file_sizes,
const uint8_t * const buffer, const long file_size )
static const char * set_file_sizes( struct File_sizes * const file_sizes,
const uint8_t * const buffer, const long file_size )
{
if( file_size <= Lh_size ) return "File ends unexpectedly at member header.";
if( file_size < min_member_size ) return "Input file is too short.";
const Lzip_header * header = (const Lzip_header *)buffer;
if( !Lh_verify_magic( *header ) )
@ -146,7 +147,7 @@ const char * set_file_sizes( struct File_sizes * const file_sizes,
}
const char * global_name;
static const char * global_name; /* copy of filename for 'error' */
static void error(char *x) { show_file_error( global_name, x, 0 ); }
@ -166,7 +167,7 @@ int decompress_in_place( const int infd, struct Pretty_print * const pp,
const bool testing )
{
long buffer_size = 0, file_size = 0;
uint8_t * buffer = read_file( infd, &buffer_size, &file_size, pp );
uint8_t * buffer = read_file( infd, &buffer_size, &file_size, pp->name );
if( !buffer ) return 1;
struct File_sizes file_sizes;
const char * emsg = set_file_sizes( &file_sizes, buffer, file_size );
@ -201,21 +202,6 @@ int decompress_in_place( const int infd, struct Pretty_print * const pp,
}
free( buffer );
if( retval ) return retval;
if( verbosity >= 1 ) Pp_show_msg( pp, 0 );
if( verbosity >= 2 )
{
if( out_pos <= 0 || in_pos <= 0 )
fputs( "no data compressed. ", stderr );
else
fprintf( stderr, "%6.3f:1, %5.2f%% ratio, %5.2f%% saved. ",
(double)out_pos / in_pos,
( 100.0 * in_pos ) / out_pos,
100.0 - ( ( 100.0 * in_pos ) / out_pos ) );
if( verbosity >= 3 )
fprintf( stderr, "decompressed %9lu, compressed %8lu. ",
out_pos, in_pos );
}
if( verbosity >= 1 )
fputs( testing ? "ok\n" : "done\n", stderr );
show_results( pp, in_pos, out_pos, testing );
return 0;
}

2
linux_lzip.h
View file

@ -4,7 +4,7 @@
/*
* LZIP decompressor
*
* Copyright (C) 2016-2018 Antonio Diaz Diaz.
* Copyright (C) 2016-2020 Antonio Diaz Diaz.
*/
/* Return values (< 0 = Error) */

95
lzip.h
View file

@ -1,5 +1,5 @@
/* Xlunzip - Test tool for the lzip_decompress linux module
Copyright (C) 2016-2018 Antonio Diaz Diaz.
Copyright (C) 2016-2020 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -22,19 +22,14 @@
#define min(x,y) ((x) <= (y) ? (x) : (y))
#endif
void * resize_buffer( void * buf, const unsigned min_size );
enum {
min_dictionary_bits = 12,
min_dictionary_size = 1 << min_dictionary_bits,
min_dictionary_size = 1 << min_dictionary_bits, /* >= modeled_distances */
max_dictionary_bits = 29,
max_dictionary_size = 1 << max_dictionary_bits,
min_member_size = 36 };
/* defined in main.c */
extern int verbosity;
struct Pretty_print
{
const char * name;
@ -44,94 +39,20 @@ struct Pretty_print
bool first_post;
};
static inline void Pp_init( struct Pretty_print * const pp,
const char * const filenames[],
const int num_filenames )
{
unsigned stdin_name_len;
int i;
pp->name = 0;
pp->padded_name = 0;
pp->stdin_name = "(stdin)";
pp->longest_name = 0;
pp->first_post = false;
if( verbosity <= 0 ) return;
stdin_name_len = strlen( pp->stdin_name );
for( i = 0; i < num_filenames; ++i )
{
const char * const s = filenames[i];
const unsigned len = (strcmp( s, "-" ) == 0) ? stdin_name_len : strlen( s );
if( len > pp->longest_name ) pp->longest_name = len;
}
if( pp->longest_name == 0 ) pp->longest_name = stdin_name_len;
}
static inline void Pp_set_name( struct Pretty_print * const pp,
const char * const filename )
{
unsigned name_len, padded_name_len, i = 0;
if( filename && filename[0] && strcmp( filename, "-" ) != 0 )
pp->name = filename;
else pp->name = pp->stdin_name;
name_len = strlen( pp->name );
padded_name_len = max( name_len, pp->longest_name ) + 4;
pp->padded_name = resize_buffer( pp->padded_name, padded_name_len + 1 );
while( i < 2 ) pp->padded_name[i++] = ' ';
while( i < name_len + 2 ) { pp->padded_name[i] = pp->name[i-2]; ++i; }
pp->padded_name[i++] = ':';
while( i < padded_name_len ) pp->padded_name[i++] = ' ';
pp->padded_name[i] = 0;
pp->first_post = true;
}
static inline void Pp_reset( struct Pretty_print * const pp )
{ if( pp->name && pp->name[0] ) pp->first_post = true; }
void Pp_show_msg( struct Pretty_print * const pp, const char * const msg );
static const uint8_t lzip_magic[4] = { 0x4C, 0x5A, 0x49, 0x50 }; /* "LZIP" */
typedef uint8_t Lzip_header[6]; /* 0-3 magic bytes */
/* 4 version */
/* 5 coded_dict_size */
/* 5 coded dictionary size */
enum { Lh_size = 6 };
static inline bool Lh_verify_magic( const Lzip_header data )
{ return ( memcmp( data, lzip_magic, 4 ) == 0 ); }
/* detect (truncated) header */
static inline bool Lh_verify_prefix( const Lzip_header data, const int sz )
{
int i; for( i = 0; i < sz && i < 4; ++i )
if( data[i] != lzip_magic[i] ) return false;
return ( sz > 0 );
}
/* detect corrupt header */
static inline bool Lh_verify_corrupt( const Lzip_header data )
{
int matches = 0;
int i; for( i = 0; i < 4; ++i )
if( data[i] == lzip_magic[i] ) ++matches;
return ( matches > 1 && matches < 4 );
}
static inline uint8_t Lh_version( const Lzip_header data )
{ return data[4]; }
static inline bool Lh_verify_version( const Lzip_header data )
{ return ( data[4] == 1 ); }
static inline unsigned Lh_get_dictionary_size( const Lzip_header data )
{
unsigned sz = ( 1 << ( data[5] & 0x1F ) );
if( sz > min_dictionary_size )
sz -= ( sz / 16 ) * ( ( data[5] >> 5 ) & 7 );
return sz;
}
typedef uint8_t Lzip_trailer[20];
/* 0-3 CRC32 of the uncompressed data */
@ -139,13 +60,6 @@ typedef uint8_t Lzip_trailer[20];
/* 12-19 member size including header and trailer */
enum { Lt_size = 20 };
static inline unsigned Lt_get_data_crc( const Lzip_trailer data )
{
unsigned tmp = 0;
int i; for( i = 3; i >= 0; --i ) { tmp <<= 8; tmp += data[i]; }
return tmp;
}
static inline unsigned long long Lt_get_data_size( const Lzip_trailer data )
{
unsigned long long tmp = 0;
@ -168,6 +82,7 @@ int decompress_in_place( const int infd, struct Pretty_print * const pp,
/* defined in main.c */
int convert_retval( const int retval );
long flush( void * buf, unsigned long size );
void show_error( const char * const msg, const int errcode, const bool help );
void show_results( struct Pretty_print * const pp, const long in_pos,
const long out_pos, const bool testing );
void show_file_error( const char * const filename, const char * const msg,
const int errcode );

29
lzip_decompress.c
View file

@ -1,7 +1,7 @@
/*
* LZIP decompressor
*
* Copyright (C) 2016-2018 Antonio Diaz Diaz.
* Copyright (C) 2016-2020 Antonio Diaz Diaz.
*
* Licensed under GPLv2 or later, see file LICENSE in this source tree.
*/
@ -48,7 +48,7 @@ static inline State St_set_short_rep(const State st)
enum {
min_dictionary_bits = 12,
min_dictionary_size = 1 << min_dictionary_bits,
min_dictionary_size = 1 << min_dictionary_bits, /* >= modeled_distances */
max_dictionary_bits = 29,
max_dictionary_size = 1 << max_dictionary_bits,
literal_context_bits = 3,
@ -84,7 +84,7 @@ static inline int get_len_state(const int len)
static inline int get_lit_state(const uint8_t prev_byte)
{
return (prev_byte >> (8 - literal_context_bits));
return prev_byte >> (8 - literal_context_bits);
}
@ -191,7 +191,7 @@ STATIC_RW_DATA const uint8_t lzip_magic[4] = { 0x4C, 0x5A, 0x49, 0x50 }; /* "LZI
typedef uint8_t Lzip_header[6]; /* 0-3 magic bytes */
/* 4 version */
/* 5 coded_dict_size */
/* 5 coded dictionary size */
enum { Lh_size = 6 };
static inline bool Lh_verify_magic(const Lzip_header data)
@ -391,7 +391,7 @@ static inline unsigned Rd_decode(struct Range_decoder * const rdec,
/* symbol <<= 1; */
/* if(rdec->code >= rdec->range) { rdec->code -= rdec->range; symbol |= 1; } */
bit = (rdec->code >= rdec->range);
symbol = (symbol << 1) + bit;
symbol <<= 1; symbol += bit;
rdec->code -= rdec->range & (0U - bit);
}
return symbol;
@ -419,9 +419,8 @@ static inline unsigned Rd_decode_bit(struct Range_decoder * const rdec,
static inline unsigned Rd_decode_tree3(struct Range_decoder * const rdec,
Bit_model bm[])
{
unsigned symbol = 1;
unsigned symbol = 2 | Rd_decode_bit(rdec, &bm[1]);
symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]);
symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]);
symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]);
return symbol & 7;
@ -430,14 +429,13 @@ static inline unsigned Rd_decode_tree3(struct Range_decoder * const rdec,
static inline unsigned Rd_decode_tree6(struct Range_decoder * const rdec,
Bit_model bm[])
{
unsigned symbol = 1;
unsigned symbol = 2 | Rd_decode_bit(rdec, &bm[1]);
symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]);
symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]);
symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]);
symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]);
symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]);
symbol = (symbol << 1) | Rd_decode_bit(rdec, &bm[symbol]);
return symbol & 0x3F;
}
@ -463,7 +461,7 @@ Rd_decode_tree_reversed(struct Range_decoder * const rdec,
for (i = 0; i < num_bits; ++i) {
const unsigned bit = Rd_decode_bit(rdec, &bm[model]);
model = (model << 1) + bit;
model <<= 1; model += bit;
symbol |= (bit << i);
}
return symbol;
@ -473,13 +471,10 @@ static inline unsigned
Rd_decode_tree_reversed4(struct Range_decoder * const rdec, Bit_model bm[])
{
unsigned symbol = Rd_decode_bit(rdec, &bm[1]);
unsigned model = 2 + symbol;
unsigned bit = Rd_decode_bit(rdec, &bm[model]);
model = (model << 1) + bit; symbol |= (bit << 1);
bit = Rd_decode_bit(rdec, &bm[model]);
model = (model << 1) + bit; symbol |= (bit << 2);
symbol |= (Rd_decode_bit(rdec, &bm[model]) << 3);
symbol += Rd_decode_bit(rdec, &bm[2+symbol]) << 1;
symbol += Rd_decode_bit(rdec, &bm[4+symbol]) << 2;
symbol += Rd_decode_bit(rdec, &bm[8+symbol]) << 3;
return symbol;
}
@ -493,7 +488,7 @@ static inline unsigned Rd_decode_matched(struct Range_decoder * const rdec,
const unsigned match_bit = (match_byte <<= 1) & mask;
const unsigned bit = Rd_decode_bit(rdec, &bm[symbol+match_bit+mask]);
symbol = (symbol << 1) + bit;
symbol <<= 1; symbol += bit;
if (symbol > 0xFF)
return symbol & 0xFF;
mask &= ~(match_bit ^ (bit << 8)); /* if( match_bit != bit ) mask = 0; */

160
main.c
View file

@ -1,5 +1,5 @@
/* Xlunzip - Test tool for the lzip_decompress linux module
Copyright (C) 2016-2018 Antonio Diaz Diaz.
Copyright (C) 2016-2020 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -36,8 +36,9 @@
#include <unistd.h>
#include <utime.h>
#include <sys/stat.h>
#if defined(__MSVCRT__)
#if defined(__MSVCRT__) || defined(__OS2__) || defined(__DJGPP__)
#include <io.h>
#if defined(__MSVCRT__)
#define fchmod(x,y) 0
#define fchown(x,y,z) 0
#define SIGHUP SIGTERM
@ -49,8 +50,10 @@
#define S_IWOTH 0
#endif
#endif
#if defined(__OS2__)
#include <io.h>
#if defined(__DJGPP__)
#define S_ISSOCK(x) 0
#define S_ISVTX 0
#endif
#endif
#include "carg_parser.h"
@ -66,25 +69,27 @@
#error "Environments where CHAR_BIT != 8 are not supported."
#endif
int verbosity = 0;
void cleanup_and_fail( const int retval );
static int verbosity = 0;
static void cleanup_and_fail( const int retval );
static void show_error( const char * const msg, const int errcode,
const bool help );
const char * const Program_name = "Xlunzip";
const char * const program_name = "xlunzip";
const char * const program_year = "2018";
const char * invocation_name = 0;
static const char * const program_name = "xlunzip";
static const char * const program_year = "2020";
static const char * invocation_name = "xlunzip"; /* default value */
const struct { const char * from; const char * to; } known_extensions[] = {
static const struct { const char * from; const char * to; } known_extensions[] = {
{ ".lz", "" },
{ ".tlz", ".tar" },
{ 0, 0 } };
int infd = -1; /* needed by the fill function */
static int infd = -1; /* needed by the fill function */
/* Variables used in signal handler context.
They are not declared volatile because the handler never returns. */
char * output_filename = 0;
int outfd = -1;
bool delete_output_on_interrupt = false;
static char * output_filename = 0;
static int outfd = -1;
static bool delete_output_on_interrupt = false;
static void show_help( void )
@ -102,7 +107,8 @@ static void show_help( void )
"\nNote that the in-place decompression of concatenated files can't be\n"
"guaranteed to work because an arbitrarily low compression ratio of the\n"
"last part of the data can be achieved by appending enough empty\n"
"compressed members to a file.\n"
"compressed members to a file, masking a high compression ratio at the\n"
"beginning of the data.\n"
"\nUsage: %s [options] [files]\n", invocation_name );
printf( "\nOptions:\n"
" -h, --help display this help and exit\n"
@ -121,10 +127,12 @@ static void show_help( void )
" --outsize[=<size>] pre-allocate outbuf [default 512 MiB]\n"
" --nofill do not pass a fill function; requires --insize\n"
" --noflush do not pass a flush function; requires --outsize\n"
"If no file names are given, or if a file is '-', xlunzip decompresses\n"
"\nIf no file names are given, or if a file is '-', xlunzip decompresses\n"
"from standard input to standard output.\n"
"Numbers may be followed by a multiplier: k = kB = 10^3 = 1000,\n"
"Ki = KiB = 2^10 = 1024, M = 10^6, Mi = 2^20, G = 10^9, Gi = 2^30, etc...\n"
"\nTo extract all the files from archive 'foo.tar.lz', use the commands\n"
"'tar -xf foo.tar.lz' or 'xlunzip -cd foo.tar.lz | tar -xf -'.\n"
"\nExit status: 0 for a normal exit, 1 for environmental problems (file\n"
"not found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or\n"
"invalid input file, 3 for an internal consistency error (eg, bug) which\n"
@ -145,7 +153,7 @@ static void show_version( void )
/* assure at least a minimum size for buffer 'buf' */
void * resize_buffer( void * buf, const unsigned min_size )
static void * resize_buffer( void * buf, const unsigned min_size )
{
if( buf ) buf = realloc( buf, min_size );
else buf = malloc( min_size );
@ -158,7 +166,48 @@ void * resize_buffer( void * buf, const unsigned min_size )
}
void Pp_show_msg( struct Pretty_print * const pp, const char * const msg )
static void Pp_init( struct Pretty_print * const pp,
const char * const filenames[], const int num_filenames )
{
unsigned stdin_name_len;
int i;
pp->name = 0;
pp->padded_name = 0;
pp->stdin_name = "(stdin)";
pp->longest_name = 0;
pp->first_post = false;
if( verbosity <= 0 ) return;
stdin_name_len = strlen( pp->stdin_name );
for( i = 0; i < num_filenames; ++i )
{
const char * const s = filenames[i];
const unsigned len = (strcmp( s, "-" ) == 0) ? stdin_name_len : strlen( s );
if( pp->longest_name < len ) pp->longest_name = len;
}
if( pp->longest_name == 0 ) pp->longest_name = stdin_name_len;
}
static void Pp_set_name( struct Pretty_print * const pp,
const char * const filename )
{
unsigned name_len, padded_name_len, i = 0;
if( filename && filename[0] && strcmp( filename, "-" ) != 0 )
pp->name = filename;
else pp->name = pp->stdin_name;
name_len = strlen( pp->name );
padded_name_len = max( name_len, pp->longest_name ) + 4;
pp->padded_name = resize_buffer( pp->padded_name, padded_name_len + 1 );
while( i < 2 ) pp->padded_name[i++] = ' ';
while( i < name_len + 2 ) { pp->padded_name[i] = pp->name[i-2]; ++i; }
pp->padded_name[i++] = ':';
while( i < padded_name_len ) pp->padded_name[i++] = ' ';
pp->padded_name[i] = 0;
pp->first_post = true;
}
static void Pp_show_msg( struct Pretty_print * const pp, const char * const msg )
{
if( verbosity >= 0 )
{
@ -325,7 +374,7 @@ static void set_signals( void (*action)(int) )
}
void cleanup_and_fail( const int retval )
static void cleanup_and_fail( const int retval )
{
set_signals( SIG_IGN ); /* ignore signals */
if( delete_output_on_interrupt )
@ -342,7 +391,7 @@ void cleanup_and_fail( const int retval )
}
void signal_handler( int sig )
static void signal_handler( int sig )
{
if( sig ) {} /* keep compiler happy */
show_error( "Control-C or similar caught, quitting.", 0, false );
@ -350,7 +399,7 @@ void signal_handler( int sig )
}
/* Set permissions, owner and times. */
/* Set permissions, owner, and times. */
static void close_and_set_permissions( const struct stat * const in_statsp )
{
bool warning = false;
@ -435,7 +484,7 @@ long flush( void * buf, unsigned long size )
return sz;
}
const char * global_name; /* copy of filename for 'error' */
static const char * global_name; /* copy of filename for 'error' */
static void error(char *x) { show_file_error( global_name, x, 0 ); }
@ -470,27 +519,33 @@ static int decompress( struct Pretty_print * const pp, const long cl_insize,
if( len < out_pos )
{ show_file_error( pp->name, "Write error", errno ); return 1; }
}
if( verbosity >= 1 ) Pp_show_msg( pp, 0 );
if( verbosity >= 2 )
{
if( out_pos <= 0 || in_pos <= 0 )
fputs( "no data compressed. ", stderr );
else
fprintf( stderr, "%6.3f:1, %5.2f%% ratio, %5.2f%% saved. ",
(double)out_pos / in_pos,
( 100.0 * in_pos ) / out_pos,
100.0 - ( ( 100.0 * in_pos ) / out_pos ) );
if( verbosity >= 3 )
fprintf( stderr, "decompressed %9lu, compressed %8lu. ",
out_pos, in_pos );
}
if( verbosity >= 1 )
fputs( testing ? "ok\n" : "done\n", stderr );
show_results( pp, in_pos, out_pos, testing );
return 0;
}
void show_error( const char * const msg, const int errcode, const bool help )
void show_results( struct Pretty_print * const pp, const long in_pos,
const long out_pos, const bool testing )
{
if( verbosity >= 1 ) Pp_show_msg( pp, 0 );
if( verbosity >= 2 )
{
if( out_pos <= 0 || in_pos <= 0 )
fputs( "no data compressed. ", stderr );
else
fprintf( stderr, "%6.3f:1, %5.2f%% ratio, %5.2f%% saved. ",
(double)out_pos / in_pos,
( 100.0 * in_pos ) / out_pos,
100.0 - ( ( 100.0 * in_pos ) / out_pos ) );
if( verbosity >= 3 )
fprintf( stderr, "%9lu out, %8lu in. ", out_pos, in_pos );
}
if( verbosity >= 1 ) fputs( testing ? "ok\n" : "done\n", stderr );
}
static void show_error( const char * const msg, const int errcode,
const bool help )
{
if( verbosity < 0 ) return;
if( msg && msg[0] )
@ -513,7 +568,7 @@ void show_file_error( const char * const filename, const char * const msg,
}
void internal_error( const char * const msg )
static void internal_error( const char * const msg )
{
if( verbosity >= 0 )
fprintf( stderr, "%s: internal error: %s\n", program_name, msg );
@ -524,7 +579,9 @@ void internal_error( const char * const msg )
int main( const int argc, const char * const argv[] )
{
const char * default_output_filename = "";
const char ** filenames = 0;
static struct Arg_parser parser; /* static because valgrind complains */
static struct Pretty_print pp; /* and memory management in C sucks */
static const char ** filenames = 0;
long cl_insize = 0;
long cl_outsize = 0;
int num_filenames = 0;
@ -541,7 +598,6 @@ int main( const int argc, const char * const argv[] )
bool stdin_used = false;
bool testing = false;
bool to_stdout = false;
struct Pretty_print pp;
enum { opt_insize = 256, opt_outsize, opt_nofill, opt_noflush };
const struct ap_Option options[] =
@ -562,10 +618,9 @@ int main( const int argc, const char * const argv[] )
{ opt_outsize, "outsize", ap_maybe },
{ opt_nofill, "nofill", ap_no },
{ opt_noflush, "noflush", ap_no },
{ 0 , 0, ap_no } };
{ 0 , 0, ap_no } };
struct Arg_parser parser;
invocation_name = argv[0];
if( argc > 0 ) invocation_name = argv[0];
if( !ap_init( &parser, argc, argv, options, 0 ) )
{ show_error( "Not enough memory.", 0, false ); return 1; }
@ -602,7 +657,7 @@ int main( const int argc, const char * const argv[] )
}
} /* end process options */
#if defined(__MSVCRT__) || defined(__OS2__)
#if defined(__MSVCRT__) || defined(__OS2__) || defined(__DJGPP__)
setmode( STDIN_FILENO, O_BINARY );
setmode( STDOUT_FILENO, O_BINARY );
#endif
@ -635,7 +690,7 @@ int main( const int argc, const char * const argv[] )
const struct stat * in_statsp;
output_filename[0] = 0;
if( !filenames[i][0] || strcmp( filenames[i], "-" ) == 0 )
if( strcmp( filenames[i], "-" ) == 0 )
{
if( stdin_used ) continue; else stdin_used = true;
infd = STDIN_FILENO;
@ -679,7 +734,7 @@ int main( const int argc, const char * const argv[] )
}
Pp_set_name( &pp, input_filename );
if( isatty( infd ) )
if( isatty( infd ) ) /* for example /dev/tty */
{
show_file_error( pp.name,
"I won't read compressed data from a terminal.", 0 );
@ -707,11 +762,8 @@ int main( const int argc, const char * const argv[] )
if( delete_output_on_interrupt )
close_and_set_permissions( in_statsp );
if( input_filename[0] )
{
if( !keep_input_files && !to_stdout && !testing )
remove( input_filename );
}
if( input_filename[0] && !keep_input_files && !to_stdout && !testing )
remove( input_filename );
}
if( outfd >= 0 && close( outfd ) != 0 )
{

38
testsuite/check.sh
View file

@ -1,9 +1,9 @@
#! /bin/sh
# check script for Xlunzip - Test tool for the lzip_decompress linux module
# Copyright (C) 2016-2018 Antonio Diaz Diaz.
# Copyright (C) 2016-2020 Antonio Diaz Diaz.
#
# This script is free software: you have unlimited permission
# to copy, distribute and modify it.
# to copy, distribute, and modify it.
LC_ALL=C
export LC_ALL
@ -30,6 +30,8 @@ cd "${objdir}"/tmp || framework_failure
cat "${testdir}"/test.txt > in || framework_failure
in_lz="${testdir}"/test.txt.lz
in_em="${testdir}"/test_em.txt.lz
fox_lz="${testdir}"/fox.lz
zero_lz="${testdir}"/zero.lz
fail=0
test_failed() { fail=1 ; printf " $1" ; [ -z "$2" ] || printf "($2)" ; }
@ -49,6 +51,8 @@ printf "testing xlunzip-%s..." "$2"
# these are for code coverage
"${LZIP}" -t -- nx_file 2> /dev/null
[ $? = 1 ] || test_failed $LINENO
"${LZIP}" -t "" < /dev/null 2> /dev/null
[ $? = 1 ] || test_failed $LINENO
"${LZIP}" --help > /dev/null || test_failed $LINENO
"${LZIP}" -n1 -V > /dev/null || test_failed $LINENO
"${LZIP}" -m 2> /dev/null
@ -71,9 +75,11 @@ printf "LZIP\001+.............................." | "${LZIP}" -t 2> /dev/null
printf "\ntesting decompression..."
"${LZIP}" -t "${in_lz}" || test_failed $LINENO
"${LZIP}" -cd "${in_lz}" > copy || test_failed $LINENO
cmp in copy || test_failed $LINENO
for i in "${in_lz}" "${in_em}" ; do
"${LZIP}" -t "$i" || test_failed $LINENO "$i"
"${LZIP}" -cd "$i" > copy || test_failed $LINENO "$i"
cmp in copy || test_failed $LINENO "$i"
done
rm -f copy || framework_failure
cat "${in_lz}" > copy.lz || framework_failure
@ -145,16 +151,38 @@ if "${LZIP}" -tq int.lz ; then
printf "${header}${body}" > int.lz # first member
"${LZIP}" -tq int.lz
[ $? = 2 ] || test_failed $LINENO ${header}
"${LZIP}" -tq < int.lz
[ $? = 2 ] || test_failed $LINENO ${header}
"${LZIP}" -cdq int.lz > /dev/null
[ $? = 2 ] || test_failed $LINENO ${header}
cat "${in_lz}" > int.lz
printf "${header}${body}" >> int.lz # trailing data
"${LZIP}" -tq int.lz
[ $? = 2 ] || test_failed $LINENO ${header}
"${LZIP}" -tq < int.lz
[ $? = 2 ] || test_failed $LINENO ${header}
"${LZIP}" -cdq int.lz > /dev/null
[ $? = 2 ] || test_failed $LINENO ${header}
done
else
printf "\nwarning: skipping header test: 'printf' does not work on your system."
fi
rm -f int.lz || framework_failure
for i in fox_v2.lz fox_s11.lz fox_de20.lz \
fox_bcrc.lz fox_crc0.lz fox_das46.lz fox_mes81.lz ; do
"${LZIP}" -tq "${testdir}"/$i
[ $? = 2 ] || test_failed $LINENO $i
done
"${LZIP}" -cd "${fox_lz}" > fox || test_failed $LINENO
for i in fox_bcrc.lz fox_crc0.lz fox_das46.lz fox_mes81.lz ; do
"${LZIP}" -cdq "${testdir}"/$i > out
[ $? = 2 ] || test_failed $LINENO $i
cmp fox out || test_failed $LINENO $i
done
rm -f fox out || framework_failure
cat "${in_lz}" "${in_lz}" "${in_lz}" > in3.lz || framework_failure
if dd if=in3.lz of=trunc.lz bs=14752 count=1 2> /dev/null &&
[ -e trunc.lz ] && cmp in2.lz trunc.lz > /dev/null 2>&1 ; then

BIN
testsuite/fox.lz Normal file
View file

Binary file not shown.

BIN
testsuite/fox_bcrc.lz Normal file
View file

Binary file not shown.

BIN
testsuite/fox_crc0.lz Normal file
View file

Binary file not shown.

BIN
testsuite/fox_das46.lz Normal file
View file

Binary file not shown.

BIN
testsuite/fox_de20.lz Normal file
View file

Binary file not shown.

BIN
testsuite/fox_mes81.lz Normal file
View file

Binary file not shown.

BIN
testsuite/fox_s11.lz Normal file
View file

Binary file not shown.

BIN
testsuite/fox_v2.lz Normal file
View file

Binary file not shown.

BIN
testsuite/test_em.txt.lz Normal file
View file

Binary file not shown.