lzip/zutils
1
0
Fork 0
Code Issues Activity

Merging upstream version 1.8.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-24 05:58:06 +01:00
parent 482a36ad54
commit 0b238fdfef
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
34 changed files with 858 additions and 455 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

19
ChangeLog
View file

@ -1,3 +1,20 @@
2019-01-01 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.8 released.
* zcat.cc: Fixed a buffer overflow on outbuf when '-v' is used.
* zcat.cc (cat): A canary byte has been added to outbuf.
* Added new option '-R, --dereference-recursive'.
* Option '-r, --recursive' now skips symlinks.
* If no files and recursive, examine current working directory.
* recursive.cc (test_full_name): Detect directory loops.
* recursive.cc: Ignore directories if not --recursive.
* recursive.cc: Remove extra trailing slashes from directory args.
* zcatgrep.cc (open_instream): Show correct errno.
* zutils.cc (good_status): Wait for killed child.
* Test and document continuation or exit of zcat, zgrep, ztest
and zupdate in case of error.
* configure: Accept appending to CXXFLAGS, 'CXXFLAGS+=OPTIONS'.
2018-02-13 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.7 released.
@ -140,7 +157,7 @@
* Version 0.1 released.
Copyright (C) 2009-2018 Antonio Diaz Diaz.
Copyright (C) 2009-2019 Antonio Diaz Diaz.
This file is a collection of facts, and thus it is not copyrightable,
but just in case, you have unlimited permission to copy, distribute and

9
INSTALL
View file

@ -1,12 +1,11 @@
Requirements
------------
You will need a C++ compiler.
I use gcc 5.3.0 and 4.1.2, but the code should compile with any
standards compliant compiler.
I use gcc 5.3.0 and 4.1.2, but the code should compile with any standards
compliant compiler.
Gcc is available at http://gcc.gnu.org.
Compressors for bzip2, gzip and lzip formats are required to run the
tests.
Compressors for bzip2, gzip and lzip formats are required to run the tests.
If you are installing zutils along with GNU gzip and want to keep the
gzip scripts, the recommended method is to configure gzip as follows:
@ -70,7 +69,7 @@ After running 'configure', you can run 'make' and 'make install' as
explained above.
Copyright (C) 2009-2018 Antonio Diaz Diaz.
Copyright (C) 2009-2019 Antonio Diaz Diaz.
This file is free documentation: you have unlimited permission to copy,
distribute and modify it.

6
Makefile.in
View file

@ -223,7 +223,11 @@ dist : doc
$(DISTNAME)/z*.in \
$(DISTNAME)/testsuite/check.sh \
$(DISTNAME)/testsuite/test.txt \
$(DISTNAME)/testsuite/test.txt.tar
$(DISTNAME)/testsuite/test.txt.tar \
$(DISTNAME)/testsuite/zcat_vs.dat \
$(DISTNAME)/testsuite/test_bad_crc.lz \
$(DISTNAME)/testsuite/zero_bad_crc.lz \
$(DISTNAME)/testsuite/zero_bad_crc.gz
rm -f $(DISTNAME)
lzip -v -9 $(DISTNAME).tar

40
NEWS
View file

@ -1,4 +1,38 @@
Changes in version 1.7:
Changes in version 1.8:
zgrep now passes the '--color' option to grep. (But it only works if the
grep program used supports it).
A buffer overflow has been fixed in zcat which happened sometimes when
the '-v, --show-nonprinting' option was used (or indirectly enabled).
A canary byte has been added to the output buffer to prevent the buffer
overflow from happening again.
The option '-R, --dereference-recursive', which recursively follows
symbolic links, has been added to zcat, zgrep, ztest and zupdate.
The option '-r, --recursive' now skips symlinks that are encountered
recursively.
If no files are given to zcat, zgrep, ztest and zupdate, a recursive
search will now examine the current working directory.
Recursive directory loops are now detected.
zcat and zgrep now ignore directories given in the command line if
'--recursive' is not specified, instead of reporting an error.
Extra trailing slashes are now removed from directories given in the
command line before recursing into them.
zcat and zgrep now show the right error when they can't open an input
file instead of showing "No such file or directory".
Killed decompressors are now waited for, preventing failure caused by
too many open pipes.
Test and document that if a file fails to decompress, zcat, zgrep and
ztest continue processing the rest of the files.
Test and document that if an error happens while recompressing a file,
zupdate exits immediately without recompressing the rest of the files.
The configure script now accepts appending options to CXXFLAGS using the
syntax 'CXXFLAGS+=OPTIONS'.

6
README
View file

@ -10,8 +10,8 @@ These utilities are not wrapper scripts but safer and more efficient C++
programs. In particular the '--recursive' option is very efficient in
those utilities supporting it.
The provided utilities are zcat, zcmp, zdiff, zgrep, ztest and zupdate.
The supported formats are bzip2, gzip, lzip and xz.
The utilities provided are zcat, zcmp, zdiff, zgrep, ztest and zupdate.
The formats supported are bzip2, gzip, lzip and xz.
Zutils uses external compressors. The compressor to be used for each
format is configurable at runtime.
@ -37,7 +37,7 @@ have been compressed. Decompressed is used to refer to data which have
undergone the process of decompression.
Copyright (C) 2009-2018 Antonio Diaz Diaz.
Copyright (C) 2009-2019 Antonio Diaz Diaz.
This file is free documentation: you have unlimited permission to copy,
distribute and modify it.

2
arg_parser.cc
View file

@ -1,5 +1,5 @@
/* Arg_parser - POSIX/GNU command line argument parser. (C++ version)
Copyright (C) 2006-2018 Antonio Diaz Diaz.
Copyright (C) 2006-2019 Antonio Diaz Diaz.
This library is free software. Redistribution and use in source and
binary forms, with or without modification, are permitted provided

2
arg_parser.h
View file

@ -1,5 +1,5 @@
/* Arg_parser - POSIX/GNU command line argument parser. (C++ version)
Copyright (C) 2006-2018 Antonio Diaz Diaz.
Copyright (C) 2006-2019 Antonio Diaz Diaz.
This library is free software. Redistribution and use in source and
binary forms, with or without modification, are permitted provided

8
configure vendored
View file

@ -1,12 +1,12 @@
#! /bin/sh
# configure script for Zutils - Utilities dealing with compressed files
# Copyright (C) 2009-2018 Antonio Diaz Diaz.
# Copyright (C) 2009-2019 Antonio Diaz Diaz.
#
# This configure script is free software: you have unlimited permission
# to copy, distribute and modify it.
pkgname=zutils
pkgversion=1.7
pkgversion=1.8
srctrigger=doc/${pkgname}.texi
# clear some things potentially inherited from environment.
@ -73,6 +73,7 @@ while [ $# != 0 ] ; do
echo " CXX=COMPILER C++ compiler to use [${CXX}]"
echo " CPPFLAGS=OPTIONS command line options for the preprocessor [${CPPFLAGS}]"
echo " CXXFLAGS=OPTIONS command line options for the C++ compiler [${CXXFLAGS}]"
echo " CXXFLAGS+=OPTIONS append options to the current value of CXXFLAGS"
echo " LDFLAGS=OPTIONS command line options for the linker [${LDFLAGS}]"
echo " DIFF=NAME diff program to use with zdiff [${DIFF}]"
echo " GREP=NAME grep program to use with zgrep [${GREP}]"
@ -103,6 +104,7 @@ while [ $# != 0 ] ; do
CXX=*) CXX=${optarg} ;;
CPPFLAGS=*) CPPFLAGS=${optarg} ;;
CXXFLAGS=*) CXXFLAGS=${optarg} ;;
CXXFLAGS+=*) CXXFLAGS="${CXXFLAGS} ${optarg}" ;;
LDFLAGS=*) LDFLAGS=${optarg} ;;
DIFF=*) DIFF=${optarg} ;;
GREP=*) GREP=${optarg} ;;
@ -180,7 +182,7 @@ echo "GREP = ${GREP}"
rm -f Makefile
cat > Makefile << EOF
# Makefile for Zutils - Utilities dealing with compressed files
# Copyright (C) 2009-2018 Antonio Diaz Diaz.
# Copyright (C) 2009-2019 Antonio Diaz Diaz.
# This file was generated automatically by configure. Don't edit.
#
# This Makefile is free software: you have unlimited permission

22
doc/zcat.1
View file

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.46.1.
.TH ZCAT "1" "February 2018" "zcat (zutils) 1.7" "User Commands"
.TH ZCAT "1" "January 2019" "zcat (zutils) 1.8" "User Commands"
.SH NAME
zcat \- decompress and concatenate files to standard output
.SH SYNOPSIS
@ -9,14 +9,17 @@ zcat \- decompress and concatenate files to standard output
Zcat copies each given file to standard output. If any given file is
compressed, its decompressed content is used. If a given file does not
exist, and its name does not end with one of the known extensions, zcat
tries the compressed file names corresponding to the supported formats.
tries the compressed file names corresponding to the formats supported.
.PP
If no files are specified, or if a file is specified as '\-', data are
read from standard input, decompressed if needed, and sent to standard
output. Data read from standard input must be of the same type; all
uncompressed or all in the same compression format.
If a file is specified as '\-', data are read from standard input,
decompressed if needed, and sent to standard output. Data read from
standard input must be of the same type; all uncompressed or all in the
same compression format.
.PP
The supported formats are bzip2, gzip, lzip and xz.
If no files are specified, recursive searches examine the current
working directory, and nonrecursive searches read standard input.
.PP
The formats supported are bzip2, gzip, lzip and xz.
.PP
Exit status is 0 if no errors occurred, non\-zero otherwise.
.SH OPTIONS
@ -57,6 +60,9 @@ suppress all messages
\fB\-r\fR, \fB\-\-recursive\fR
operate recursively on directories
.TP
\fB\-R\fR, \fB\-\-dereference\-recursive\fR
recursively follow symbolic links
.TP
\fB\-s\fR, \fB\-\-squeeze\-blank\fR
never more than one single blank line
.TP
@ -88,7 +94,7 @@ Report bugs to zutils\-bug@nongnu.org
.br
Zutils home page: http://www.nongnu.org/zutils/zutils.html
.SH COPYRIGHT
Copyright \(co 2018 Antonio Diaz Diaz.
Copyright \(co 2019 Antonio Diaz Diaz.
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>
.br
This is free software: you are free to change and redistribute it.

6
doc/zcmp.1
View file

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.46.1.
.TH ZCMP "1" "February 2018" "zcmp (zutils) 1.7" "User Commands"
.TH ZCMP "1" "January 2019" "zcmp (zutils) 1.8" "User Commands"
.SH NAME
zcmp \- decompress and compare two files byte by byte
.SH SYNOPSIS
@ -12,7 +12,7 @@ are numbered starting with 1. If any given file is compressed, its
decompressed content is used. Compressed files are decompressed on the
fly; no temporary files are created.
.PP
The supported formats are bzip2, gzip, lzip and xz.
The formats supported are bzip2, gzip, lzip and xz.
.PP
Zcmp compares file1 to file2. If file2 is omitted zcmp tries the
following:
@ -85,7 +85,7 @@ Report bugs to zutils\-bug@nongnu.org
.br
Zutils home page: http://www.nongnu.org/zutils/zutils.html
.SH COPYRIGHT
Copyright \(co 2018 Antonio Diaz Diaz.
Copyright \(co 2019 Antonio Diaz Diaz.
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>
.br
This is free software: you are free to change and redistribute it.

6
doc/zdiff.1
View file

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.46.1.
.TH ZDIFF "1" "February 2018" "zdiff (zutils) 1.7" "User Commands"
.TH ZDIFF "1" "January 2019" "zdiff (zutils) 1.8" "User Commands"
.SH NAME
zdiff \- decompress and compare two files line by line
.SH SYNOPSIS
@ -12,7 +12,7 @@ compressed, its decompressed content is used. Zdiff is a front end to
the diff program and has the limitation that messages from diff refer to
temporary filenames instead of those specified.
.PP
The supported formats are bzip2, gzip, lzip and xz.
The formats supported are bzip2, gzip, lzip and xz.
.PP
Zdiff compares file1 to file2. If file2 is omitted zdiff tries the
following:
@ -109,7 +109,7 @@ Report bugs to zutils\-bug@nongnu.org
.br
Zutils home page: http://www.nongnu.org/zutils/zutils.html
.SH COPYRIGHT
Copyright \(co 2018 Antonio Diaz Diaz.
Copyright \(co 2019 Antonio Diaz Diaz.
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>
.br
This is free software: you are free to change and redistribute it.

22
doc/zgrep.1
View file

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.46.1.
.TH ZGREP "1" "February 2018" "zgrep (zutils) 1.7" "User Commands"
.TH ZGREP "1" "January 2019" "zgrep (zutils) 1.8" "User Commands"
.SH NAME
zgrep \- search compressed files for a regular expression
.SH SYNOPSIS
@ -11,14 +11,17 @@ on any combination of compressed and uncompressed files. If any given
file is compressed, its decompressed content is used. If a given file
does not exist, and its name does not end with one of the known
extensions, zgrep tries the compressed file names corresponding to the
supported formats.
formats supported.
.PP
If no files are specified, or if a file is specified as '\-', data are
read from standard input, decompressed if needed, and fed to grep. Data
read from standard input must be of the same type; all uncompressed or
all in the same compression format.
If a file is specified as '\-', data are read from standard input,
decompressed if needed, and fed to grep. Data read from standard input
must be of the same type; all uncompressed or all in the same
compression format.
.PP
The supported formats are bzip2, gzip, lzip and xz.
If no files are specified, recursive searches examine the current
working directory, and nonrecursive searches read standard input.
.PP
The formats supported are bzip2, gzip, lzip and xz.
.PP
Exit status is 0 if match, 1 if no match, 2 if trouble.
.SH OPTIONS
@ -104,6 +107,9 @@ suppress all messages
\fB\-r\fR, \fB\-\-recursive\fR
operate recursively on directories
.TP
\fB\-R\fR, \fB\-\-dereference\-recursive\fR
recursively follow symbolic links
.TP
\fB\-s\fR, \fB\-\-no\-messages\fR
suppress error messages
.TP
@ -138,7 +144,7 @@ Report bugs to zutils\-bug@nongnu.org
.br
Zutils home page: http://www.nongnu.org/zutils/zutils.html
.SH COPYRIGHT
Copyright \(co 2018 Antonio Diaz Diaz.
Copyright \(co 2019 Antonio Diaz Diaz.
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>
.br
This is free software: you are free to change and redistribute it.

19
doc/ztest.1
View file

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.46.1.
.TH ZTEST "1" "February 2018" "ztest (zutils) 1.7" "User Commands"
.TH ZTEST "1" "January 2019" "ztest (zutils) 1.8" "User Commands"
.SH NAME
ztest \- verify the integrity of compressed files
.SH SYNOPSIS
@ -7,12 +7,14 @@ ztest \- verify the integrity of compressed files
[\fI\,options\/\fR] [\fI\,files\/\fR]
.SH DESCRIPTION
Ztest verifies the integrity of the specified compressed files.
Uncompressed files are ignored. If no files are specified, or if a file
is specified as '\-', the integrity of compressed data read from standard
input is verified. Data read from standard input must be all in the same
compression format.
Uncompressed files are ignored. If a file is specified as '\-', the
integrity of compressed data read from standard input is verified. Data
read from standard input must be all in the same compression format.
.PP
The supported formats are bzip2, gzip, lzip and xz.
If no files are specified, recursive searches examine the current
working directory, and nonrecursive searches read standard input.
.PP
The formats supported are bzip2, gzip, lzip and xz.
.PP
Note that error detection in the xz format is broken. First, some xz
files lack integrity information. Second, not all xz decompressors can
@ -47,6 +49,9 @@ suppress all messages
\fB\-r\fR, \fB\-\-recursive\fR
operate recursively on directories
.TP
\fB\-R\fR, \fB\-\-dereference\-recursive\fR
recursively follow symbolic links
.TP
\fB\-v\fR, \fB\-\-verbose\fR
be verbose (a 2nd \fB\-v\fR gives more)
.TP
@ -66,7 +71,7 @@ Report bugs to zutils\-bug@nongnu.org
.br
Zutils home page: http://www.nongnu.org/zutils/zutils.html
.SH COPYRIGHT
Copyright \(co 2018 Antonio Diaz Diaz.
Copyright \(co 2019 Antonio Diaz Diaz.
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>
.br
This is free software: you are free to change and redistribute it.

23
doc/zupdate.1
View file

@ -1,19 +1,21 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.46.1.
.TH ZUPDATE "1" "February 2018" "zupdate (zutils) 1.7" "User Commands"
.TH ZUPDATE "1" "January 2019" "zupdate (zutils) 1.8" "User Commands"
.SH NAME
zupdate \- recompress bzip2, gzip, xz files to lzip format
.SH SYNOPSIS
.B zupdate
[\fI\,options\/\fR] [\fI\,files\/\fR]
.SH DESCRIPTION
Zupdate recompresses files from bzip2, gzip, and xz formats to lzip format.
The originals are compared with the new files and then deleted.
Zupdate recompresses files from bzip2, gzip, and xz formats to lzip
format. Each original is compared with the new file and then deleted.
Only regular files with standard file name extensions are recompressed,
other files are ignored.
Compressed files are decompressed and then recompressed on the fly; no
temporary files are created.
The lzip format is chosen as destination because it is by far the most
appropriate for long\-term data archiving.
other files are ignored. Compressed files are decompressed and then
recompressed on the fly; no temporary files are created. The lzip format
is chosen as destination because it is the most appropriate for
long\-term data archiving.
.PP
If no files are specified, recursive searches examine the current
working directory, and nonrecursive searches do nothing.
.PP
If the lzip compressed version of a file already exists, the file is
skipped unless the '\-\-force' option is given. In this case, if the
@ -54,6 +56,9 @@ suppress all messages
\fB\-r\fR, \fB\-\-recursive\fR
operate recursively on directories
.TP
\fB\-R\fR, \fB\-\-dereference\-recursive\fR
recursively follow symbolic links
.TP
\fB\-v\fR, \fB\-\-verbose\fR
be verbose (a 2nd \fB\-v\fR gives more)
.TP
@ -76,7 +81,7 @@ Report bugs to zutils\-bug@nongnu.org
.br
Zutils home page: http://www.nongnu.org/zutils/zutils.html
.SH COPYRIGHT
Copyright \(co 2018 Antonio Diaz Diaz.
Copyright \(co 2019 Antonio Diaz Diaz.
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>
.br
This is free software: you are free to change and redistribute it.

125
doc/zutils.info
View file

@ -12,7 +12,7 @@ File: zutils.info, Node: Top, Next: Introduction, Up: (dir)
Zutils Manual
*************
This manual is for Zutils (version 1.7, 13 February 2018).
This manual is for Zutils (version 1.8, 1 January 2019).
* Menu:
@ -29,7 +29,7 @@ This manual is for Zutils (version 1.7, 13 February 2018).
* Concept index:: Index of concepts
Copyright (C) 2009-2018 Antonio Diaz Diaz.
Copyright (C) 2009-2019 Antonio Diaz Diaz.
This manual is free documentation: you have unlimited permission to
copy, distribute and modify it.
@ -50,8 +50,8 @@ are created.
C++ programs. In particular the '--recursive' option is very efficient
in those utilities supporting it.
The provided utilities are zcat, zcmp, zdiff, zgrep, ztest and zupdate.
The supported formats are bzip2, gzip, lzip and xz.
The utilities provided are zcat, zcmp, zdiff, zgrep, ztest and zupdate.
The formats supported are bzip2, gzip, lzip and xz.
Zutils uses external compressors. The compressor to be used for each
format is configurable at runtime.
@ -110,7 +110,8 @@ described here.
'-V'
'--version'
Print the version number on the standard output and exit.
Print the version number on the standard output and exit. This
version number should be included in all bug reports.
'-M FORMAT_LIST'
'--format=FORMAT_LIST'
@ -190,12 +191,17 @@ File: zutils.info, Node: Zcat, Next: Zcmp, Prev: The zutilsrc file, Up: Top
zcat copies each given file to standard output. If any given file is
compressed, its decompressed content is used. If a given file does not
exist, and its name does not end with one of the known extensions, zcat
tries the compressed file names corresponding to the supported formats.
tries the compressed file names corresponding to the formats supported.
If a file fails to decompress, zcat continues copying the rest of the
files.
If no files are specified, or if a file is specified as '-', data
are read from standard input, decompressed if needed, and sent to
standard output. Data read from standard input must be of the same type;
all uncompressed or all in the same compression format.
If a file is specified as '-', data are read from standard input,
decompressed if needed, and sent to standard output. Data read from
standard input must be of the same type; all uncompressed or all in the
same compression format.
If no files are specified, recursive searches examine the current
working directory, and nonrecursive searches read standard input.
The format for running zcat is:
@ -240,7 +246,14 @@ Exit status is 0 if no errors occurred, non-zero otherwise.
'-r'
'--recursive'
Operate recursively on directories.
For each directory operand, read and process all files in that
directory, recursively. Follow symbolic links in the command line,
but skip symlinks that are encountered recursively.
'-R'
'--dereference-recursive'
For each directory operand, read and process all files in that
directory, recursively, following all symbolic links.
'-s'
'--squeeze-blank'
@ -459,12 +472,16 @@ on any combination of compressed and uncompressed files. If any given
file is compressed, its decompressed content is used. If a given file
does not exist, and its name does not end with one of the known
extensions, zgrep tries the compressed file names corresponding to the
supported formats.
formats supported. If a file fails to decompress, zgrep continues
searching the rest of the files.
If no files are specified, or if a file is specified as '-', data
are read from standard input, decompressed if needed, and fed to grep.
Data read from standard input must be of the same type; all uncompressed
or all in the same compression format.
If a file is specified as '-', data are read from standard input,
decompressed if needed, and fed to grep. Data read from standard input
must be of the same type; all uncompressed or all in the same
compression format.
If no files are specified, recursive searches examine the current
working directory, and nonrecursive searches read standard input.
The format for running zgrep is:
@ -573,7 +590,14 @@ grep program used supports them):
'-r'
'--recursive'
Operate recursively on directories.
For each directory operand, read and process all files in that
directory, recursively. Follow symbolic links in the command line,
but skip symlinks that are encountered recursively.
'-R'
'--dereference-recursive'
For each directory operand, read and process all files in that
directory, recursively, following all symbolic links.
'-s'
'--no-messages'
@ -602,10 +626,14 @@ File: zutils.info, Node: Ztest, Next: Zupdate, Prev: Zgrep, Up: Top
*******
ztest verifies the integrity of the specified compressed files.
Uncompressed files are ignored. If no files are specified, or if a file
is specified as '-', the integrity of compressed data read from
standard input is verified. Data read from standard input must be all in
the same compression format.
Uncompressed files are ignored. If a file is specified as '-', the
integrity of compressed data read from standard input is verified. Data
read from standard input must be all in the same compression format. If
a file fails to decompress, ztest continues verifying the rest of the
files.
If no files are specified, recursive searches examine the current
working directory, and nonrecursive searches read standard input.
Note that error detection in the xz format is broken. First, some xz
files lack integrity information. Second, not all xz decompressors can
@ -640,7 +668,14 @@ environmental problems (file not found, invalid flags, I/O errors, etc),
'-r'
'--recursive'
Operate recursively on directories.
For each directory operand, read and process all files in that
directory, recursively. Follow symbolic links in the command line,
but skip symlinks that are encountered recursively.
'-R'
'--dereference-recursive'
For each directory operand, read and process all files in that
directory, recursively, following all symbolic links.
'-v'
'--verbose'
@ -658,9 +693,14 @@ zupdate recompresses files from bzip2, gzip, and xz formats to lzip
format. Each original is compared with the new file and then deleted.
Only regular files with standard file name extensions are recompressed,
other files are ignored. Compressed files are decompressed and then
recompressed on the fly; no temporary files are created. The lzip format
is chosen as destination because it is by far the most appropriate for
long-term data archiving.
recompressed on the fly; no temporary files are created. If an error
happens while recompressing a file, zupdate exits immediately without
recompressing the rest of the files. The lzip format is chosen as
destination because it is the most appropriate for long-term data
archiving.
If no files are specified, recursive searches examine the current
working directory, and nonrecursive searches do nothing.
If the lzip compressed version of a file already exists, the file is
skipped unless the '--force' option is given. In this case, if the
@ -679,7 +719,7 @@ files produced have the extensions '.lz' or '.tar.lz'.
Recompressing a file is much like copying or moving it; therefore
zupdate preserves the access and modification dates, permissions, and,
when possible, ownership of the file just as "cp -p" does. (If the user
when possible, ownership of the file just as 'cp -p' does. (If the user
ID or the group ID can't be duplicated, the file permission bits S_ISUID
and S_ISGID are cleared).
@ -718,7 +758,14 @@ otherwise.
'-r'
'--recursive'
Operate recursively on directories.
For each directory operand, read and process all files in that
directory, recursively. Follow symbolic links in the command line,
but skip symlinks that are encountered recursively.
'-R'
'--dereference-recursive'
For each directory operand, read and process all files in that
directory, recursively, following all symbolic links.
'-v'
'--verbose'
@ -770,18 +817,18 @@ Concept index

Tag Table:
Node: Top222
Node: Introduction1151
Node: Common options3775
Ref: compressor-requirements5533
Node: The zutilsrc file5905
Node: Zcat6830
Node: Zcmp8884
Node: Zdiff11343
Node: Zgrep14047
Node: Ztest17541
Node: Zupdate19375
Node: Problems22247
Node: Concept index22781
Node: Introduction1149
Node: Common options3773
Ref: compressor-requirements5596
Node: The zutilsrc file5968
Node: Zcat6893
Node: Zcmp9445
Node: Zdiff11904
Node: Zgrep14608
Node: Ztest18603
Node: Zupdate20938
Node: Problems24364
Node: Concept index24898

End Tag Table

101
doc/zutils.texi
View file

@ -6,8 +6,8 @@
@finalout
@c %**end of header
@set UPDATED 13 February 2018
@set VERSION 1.7
@set UPDATED 1 January 2019
@set VERSION 1.8
@dircategory Data Compression
@direntry
@ -49,7 +49,7 @@ This manual is for Zutils (version @value{VERSION}, @value{UPDATED}).
@end menu
@sp 1
Copyright @copyright{} 2009-2018 Antonio Diaz Diaz.
Copyright @copyright{} 2009-2019 Antonio Diaz Diaz.
This manual is free documentation: you have unlimited permission
to copy, distribute and modify it.
@ -70,8 +70,8 @@ programs. In particular the @samp{--recursive} option is very efficient
in those utilities supporting it.
@noindent
The provided utilities are zcat, zcmp, zdiff, zgrep, ztest and zupdate.@*
The supported formats are bzip2, gzip, lzip and xz.@*
The utilities provided are zcat, zcmp, zdiff, zgrep, ztest and zupdate.@*
The formats supported are bzip2, gzip, lzip and xz.@*
Zutils uses external compressors. The compressor to be used for each
format is configurable at runtime.
@ -133,6 +133,7 @@ only supports the @samp{--help} form of this option.
@item -V
@itemx --version
Print the version number on the standard output and exit.
This version number should be included in all bug reports.
@item -M @var{format_list}
@itemx --format=@var{format_list}
@ -221,12 +222,17 @@ where <format> is one of @samp{bz2}, @samp{gz}, @samp{lz} or @samp{xz}.
zcat copies each given file to standard output. If any given file is
compressed, its decompressed content is used. If a given file does not
exist, and its name does not end with one of the known extensions, zcat
tries the compressed file names corresponding to the supported formats.
tries the compressed file names corresponding to the formats supported.
If a file fails to decompress, zcat continues copying the rest of the
files.
If no files are specified, or if a file is specified as @samp{-}, data
are read from standard input, decompressed if needed, and sent to
standard output. Data read from standard input must be of the same type;
all uncompressed or all in the same compression format.
If a file is specified as @samp{-}, data are read from standard input,
decompressed if needed, and sent to standard output. Data read from
standard input must be of the same type; all uncompressed or all in the
same compression format.
If no files are specified, recursive searches examine the current
working directory, and nonrecursive searches read standard input.
The format for running zcat is:
@ -274,7 +280,14 @@ Quiet operation. Suppress all messages.
@item -r
@itemx --recursive
Operate recursively on directories.
For each directory operand, read and process all files in that
directory, recursively. Follow symbolic links in the command line, but
skip symlinks that are encountered recursively.
@item -R
@itemx --dereference-recursive
For each directory operand, read and process all files in that
directory, recursively, following all symbolic links.
@item -s
@itemx --squeeze-blank
@ -509,12 +522,16 @@ on any combination of compressed and uncompressed files. If any given
file is compressed, its decompressed content is used. If a given file
does not exist, and its name does not end with one of the known
extensions, zgrep tries the compressed file names corresponding to the
supported formats.
formats supported. If a file fails to decompress, zgrep continues
searching the rest of the files.
If no files are specified, or if a file is specified as @samp{-}, data
are read from standard input, decompressed if needed, and fed to grep.
Data read from standard input must be of the same type; all uncompressed
or all in the same compression format.
If a file is specified as @samp{-}, data are read from standard input,
decompressed if needed, and fed to grep. Data read from standard input
must be of the same type; all uncompressed or all in the same
compression format.
If no files are specified, recursive searches examine the current
working directory, and nonrecursive searches read standard input.
The format for running zgrep is:
@ -629,7 +646,14 @@ found, even if an error was detected.
@item -r
@itemx --recursive
Operate recursively on directories.
For each directory operand, read and process all files in that
directory, recursively. Follow symbolic links in the command line, but
skip symlinks that are encountered recursively.
@item -R
@itemx --dereference-recursive
For each directory operand, read and process all files in that
directory, recursively, following all symbolic links.
@item -s
@itemx --no-messages
@ -658,10 +682,14 @@ Match only whole lines.
@cindex ztest
ztest verifies the integrity of the specified compressed files.
Uncompressed files are ignored. If no files are specified, or if a file
is specified as @samp{-}, the integrity of compressed data read from
standard input is verified. Data read from standard input must be all in
the same compression format.
Uncompressed files are ignored. If a file is specified as @samp{-}, the
integrity of compressed data read from standard input is verified. Data
read from standard input must be all in the same compression format. If
a file fails to decompress, ztest continues verifying the rest of the
files.
If no files are specified, recursive searches examine the current
working directory, and nonrecursive searches read standard input.
Note that error detection in the xz format is broken. First, some xz
files lack integrity information. Second, not all xz decompressors can
@ -703,7 +731,14 @@ Quiet operation. Suppress all messages.
@item -r
@itemx --recursive
Operate recursively on directories.
For each directory operand, read and process all files in that
directory, recursively. Follow symbolic links in the command line, but
skip symlinks that are encountered recursively.
@item -R
@itemx --dereference-recursive
For each directory operand, read and process all files in that
directory, recursively, following all symbolic links.
@item -v
@itemx --verbose
@ -721,9 +756,14 @@ zupdate recompresses files from bzip2, gzip, and xz formats to lzip
format. Each original is compared with the new file and then deleted.
Only regular files with standard file name extensions are recompressed,
other files are ignored. Compressed files are decompressed and then
recompressed on the fly; no temporary files are created. The lzip format
is chosen as destination because it is by far the most appropriate for
long-term data archiving.
recompressed on the fly; no temporary files are created. If an error
happens while recompressing a file, zupdate exits immediately without
recompressing the rest of the files. The lzip format is chosen as
destination because it is the most appropriate for long-term data
archiving.
If no files are specified, recursive searches examine the current
working directory, and nonrecursive searches do nothing.
If the lzip compressed version of a file already exists, the file is
skipped unless the @samp{--force} option is given. In this case, if the
@ -743,7 +783,7 @@ extensions @samp{.lz} or @samp{.tar.lz}.
Recompressing a file is much like copying or moving it; therefore
zupdate preserves the access and modification dates, permissions, and,
when possible, ownership of the file just as "cp -p" does. (If the user
when possible, ownership of the file just as @samp{cp -p} does. (If the user
ID or the group ID can't be duplicated, the file permission bits S_ISUID
and S_ISGID are cleared).
@ -785,7 +825,14 @@ Quiet operation. Suppress all messages.
@item -r
@itemx --recursive
Operate recursively on directories.
For each directory operand, read and process all files in that
directory, recursively. Follow symbolic links in the command line, but
skip symlinks that are encountered recursively.
@item -R
@itemx --dereference-recursive
For each directory operand, read and process all files in that
directory, recursively, following all symbolic links.
@item -v
@itemx --verbose

32
rc.cc
View file

@ -1,5 +1,5 @@
/* Zutils - Utilities dealing with compressed files
Copyright (C) 2009-2018 Antonio Diaz Diaz.
Copyright (C) 2009-2019 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -37,7 +37,7 @@ int verbosity = 0;
namespace {
const char * const config_file_name = "zutilsrc";
const char * const program_year = "2018";
const char * const program_year = "2019";
std::string compressor_names[num_formats] =
{ "bzip2", "gzip", "lzip", "xz" }; // default compressor names
@ -211,8 +211,8 @@ int process_rcfile( const std::string & name )
bool enabled_format( const int format_index )
{
if( enabled_formats.size() <= num_formats ) return true;
if( format_index < 0 ) return enabled_formats[num_formats];
if( enabled_formats.size() <= num_formats ) return true; // all enabled
if( format_index < 0 ) return enabled_formats[num_formats]; // uncompressed
return enabled_formats[format_index];
}
@ -336,11 +336,9 @@ void show_error( const char * const msg, const int errcode, const bool help )
{
if( verbosity < 0 ) return;
if( msg && msg[0] )
{
std::fprintf( stderr, "%s: %s", program_name, msg );
if( errcode > 0 ) std::fprintf( stderr, ": %s", std::strerror( errcode ) );
std::fputc( '\n', stderr );
}
std::fprintf( stderr, "%s: %s%s%s\n", program_name, msg,
( errcode > 0 ) ? ": " : "",
( errcode > 0 ) ? std::strerror( errcode ) : "" );
if( help )
std::fprintf( stderr, "Try '%s --help' for more information.\n",
invocation_name );
@ -350,18 +348,10 @@ void show_error( const char * const msg, const int errcode, const bool help )
void show_file_error( const char * const filename, const char * const msg,
const int errcode )
{
if( verbosity < 0 ) return;
std::fprintf( stderr, "%s: %s: %s", program_name, filename, msg );
if( errcode > 0 ) std::fprintf( stderr, ": %s", std::strerror( errcode ) );
std::fputc( '\n', stderr );
}
void show_error2( const char * const msg, const char * const name )
{
if( verbosity >= 0 )
std::fprintf( stderr, "%s: %s '%s': %s\n",
program_name, msg, name, std::strerror( errno ) );
std::fprintf( stderr, "%s: %s: %s%s%s\n", program_name, filename, msg,
( errcode > 0 ) ? ": " : "",
( errcode > 0 ) ? std::strerror( errcode ) : "" );
}
@ -376,7 +366,7 @@ void internal_error( const char * const msg )
void show_close_error( const char * const prog_name )
{
if( verbosity >= 0 )
std::fprintf( stderr, "%s: Can't close output of %s: %s\n",
std::fprintf( stderr, "%s: Error closing output of %s: %s\n",
program_name, prog_name, std::strerror( errno ) );
}

3
rc.h
View file

@ -1,5 +1,5 @@
/* Zutils - Utilities dealing with compressed files
Copyright (C) 2009-2018 Antonio Diaz Diaz.
Copyright (C) 2009-2019 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -51,7 +51,6 @@ void show_error( const char * const msg, const int errcode = 0,
const bool help = false );
void show_file_error( const char * const filename, const char * const msg,
const int errcode = 0 );
void show_error2( const char * const msg, const char * const name );
void internal_error( const char * const msg );
void show_close_error( const char * const prog_name = "data feeder" );
void show_exec_error( const char * const prog_name );

61
recursive.cc
View file

@ -1,5 +1,5 @@
/* Zutils - Utilities dealing with compressed files
Copyright (C) 2009-2018 Antonio Diaz Diaz.
Copyright (C) 2009-2019 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -15,9 +15,41 @@
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
// Returns true if full_name is a regular file with an enabled extension
// or (a link to) a directory.
bool test_full_name( const std::string & full_name, const struct stat * stp,
const bool follow )
{
struct stat st, st2;
if( follow && stat( full_name.c_str(), &st ) != 0 ) return false;
if( !follow && lstat( full_name.c_str(), &st ) != 0 ) return false;
if( S_ISREG( st.st_mode ) ) // regular file
return enabled_format( extension_format( extension_index( full_name ) ) );
if( !S_ISDIR( st.st_mode ) ) return false;
std::string prev_dir( full_name );
bool loop = ( stp && st.st_ino == stp->st_ino && st.st_dev == stp->st_dev );
if( !loop )
for( unsigned i = prev_dir.size(); i > 1; )
{
while( i > 0 && prev_dir[i-1] != '/' ) --i;
if( i == 0 ) break;
if( i > 1 ) --i; // remove trailing slash except at root dir
prev_dir.resize( i );
if( stat( prev_dir.c_str(), &st2 ) != 0 || !S_ISDIR( st2.st_mode ) ||
( st.st_ino == st2.st_ino && st.st_dev == st2.st_dev ) )
{ loop = true; break; }
}
if( loop ) // full_name already visited or above tree
show_file_error( full_name.c_str(), "warning: Recursive directory loop" );
return !loop; // (link to) directory
}
// Returns in input_filename the next filename.
bool next_filename( std::list< std::string > & filenames,
std::string & input_filename, bool & error,
const bool recursive, const bool ignore_stdin = false,
const int recursive, const bool ignore_stdin = false,
const bool no_messages = false )
{
while( !filenames.empty() )
@ -29,18 +61,30 @@ bool next_filename( std::list< std::string > & filenames,
if( ignore_stdin ) continue;
input_filename.clear(); return true;
}
if( recursive )
{
struct stat st;
if( stat( input_filename.c_str(), &st ) == 0 && S_ISDIR( st.st_mode ) )
{
if( recursive )
{
DIR * const dirp = opendir( input_filename.c_str() );
if( !dirp )
{
if( !no_messages )
show_error2( "Can't open directory", input_filename.c_str() );
show_file_error( input_filename.c_str(), "Can't open directory", errno );
error = true; continue;
}
for( unsigned i = input_filename.size();
i > 1 && input_filename[i-1] == '/'; --i )
input_filename.resize( i - 1 ); // remove trailing slashes
struct stat stdot, *stdotp = 0;
if( input_filename[0] != '/' ) // relative path
{
if( input_filename == "." ) input_filename.clear();
if( stat( ".", &stdot ) == 0 && S_ISDIR( stdot.st_mode ) )
stdotp = &stdot;
}
if( input_filename.size() && input_filename != "/" )
input_filename += '/';
std::list< std::string > tmp_list;
while( true )
{
@ -48,14 +92,13 @@ bool next_filename( std::list< std::string > & filenames,
if( !entryp ) { closedir( dirp ); break; }
const std::string tmp_name( entryp->d_name );
if( tmp_name == "." || tmp_name == ".." ) continue;
const std::string full_name( input_filename + "/" + tmp_name );
if( enabled_format( extension_format( extension_index( tmp_name ) ) ) ||
( stat( full_name.c_str(), &st ) == 0 && S_ISDIR( st.st_mode ) ) )
const std::string full_name( input_filename + tmp_name );
if( test_full_name( full_name, stdotp, recursive == 2 ) )
tmp_list.push_back( full_name );
}
filenames.splice( filenames.begin(), tmp_list );
continue;
}
continue;
}
return true;
}

150
testsuite/check.sh
View file

@ -1,6 +1,6 @@
#! /bin/sh
# check script for Zutils - Utilities dealing with compressed files
# Copyright (C) 2009-2018 Antonio Diaz Diaz.
# Copyright (C) 2009-2019 Antonio Diaz Diaz.
#
# This script is free software: you have unlimited permission
# to copy, distribute and modify it.
@ -53,6 +53,9 @@ cat in > -in- || framework_failure
cat in.lz > -in-.lz || framework_failure
cat in.lz > lz_only.lz || framework_failure
cat in in in in in in > in6 || framework_failure
bad0_lz="${testdir}"/zero_bad_crc.lz
bad0_gz="${testdir}"/zero_bad_crc.gz
bad1_lz="${testdir}"/test_bad_crc.lz
fail=0
test_failed() { fail=1 ; printf " $1" ; [ -z "$2" ] || printf "($2)" ; }
@ -71,6 +74,7 @@ for i in ${extensions}; do
test_failed $LINENO $i
done
"${ZCAT}" -N -v -s "${testdir}"/zcat_vs.dat > /dev/null || test_failed $LINENO
"${ZCAT}" -N < in > copy || test_failed $LINENO
cmp in copy || test_failed $LINENO
"${ZCAT}" -N < in.gz > copy || test_failed $LINENO
@ -89,6 +93,18 @@ cmp in copy || test_failed $LINENO
cmp in copy || test_failed $LINENO
"${ZCAT}" -N in in.gz in.bz2 in.lz -- -in- -in-.lz > copy || test_failed $LINENO
cmp in6 copy || test_failed $LINENO
"${ZCAT}" -Nq in in.gz in.bz2 in.lz "${bad0_lz}" -- -in- -in-.lz > copy
[ $? = 1 ] || test_failed $LINENO
cmp in6 copy || test_failed $LINENO
"${ZCAT}" -Nq "${bad1_lz}" -- -in-.lz in in.gz in.bz2 in.lz > copy
[ $? = 1 ] || test_failed $LINENO
cmp in6 copy || test_failed $LINENO
"${ZCAT}" -N . || test_failed $LINENO
"${ZCAT}" -N -r . > /dev/null || test_failed $LINENO
"${ZCAT}" -N -r > /dev/null || test_failed $LINENO
"${ZCAT}" -N -R . > /dev/null || test_failed $LINENO
"${ZCAT}" -N -R > /dev/null || test_failed $LINENO
"${ZCAT}" -Nq --format=, in.lz
[ $? = 1 ] || test_failed $LINENO
"${ZCAT}" -Nq --format=,lz in.lz
@ -258,6 +274,9 @@ done
"${ZGREP}" -N pin.tar -e "GNU" > /dev/null || test_failed $LINENO
"${ZGREP}" -N "GNU" < pin.tar > /dev/null || test_failed $LINENO
"${ZGREP}" -N -r "GNU" . > /dev/null || test_failed $LINENO
"${ZGREP}" -N -r "GNU" > /dev/null || test_failed $LINENO
"${ZGREP}" -N -R "GNU" . > /dev/null || test_failed $LINENO
"${ZGREP}" -N -R "GNU" > /dev/null || test_failed $LINENO
"${ZGREP}" -N "nx_pattern" -r . in > /dev/null && test_failed $LINENO
"${ZGREP}" -N -e "GNU" in > /dev/null || test_failed $LINENO
"${ZGREP}" -N "GNU" < in > /dev/null || test_failed $LINENO
@ -275,6 +294,12 @@ done
test_failed $LINENO
"${ZGREP}" -N -L "nx_pattern" in in.gz in.bz2 in.lz -- -in- > /dev/null &&
test_failed $LINENO
"${ZGREP}" -Nq -l "01234567890" in "${bad1_lz}" in.lz && test_failed $LINENO
"${ZGREP}" -Nq -l "01234567890" in "${bad1_lz}" in.lz pin.tar > /dev/null ||
test_failed $LINENO
"${ZGREP}" -N "GNU" .
[ $? = 1 ] || test_failed $LINENO
"${ZGREP}" -N --bad-option 2> /dev/null
[ $? = 2 ] || test_failed $LINENO
"${ZGREP}" -N "GNU" -s nx_file
@ -300,9 +325,20 @@ done
"${ZTEST}" -N < in.gz || test_failed $LINENO
"${ZTEST}" -N < in.bz2 || test_failed $LINENO
"${ZTEST}" -N < in.lz || test_failed $LINENO
"${ZTEST}" -N - < in.lz || test_failed $LINENO
"${ZTEST}" -N - in.gz - < in.lz || test_failed $LINENO
"${ZTEST}" -N --lz='lzip -q' < in.lz || test_failed $LINENO
"${ZTEST}" -N -r . || test_failed $LINENO
"${ZTEST}" -N -r || test_failed $LINENO
"${ZTEST}" -N -R . || test_failed $LINENO
"${ZTEST}" -N -R || test_failed $LINENO
"${ZTEST}" -Nq in.gz "${bad0_lz}" in.bz2 "${bad1_lz}" in.lz
[ $? = 2 ] || test_failed $LINENO
lines=`"${ZTEST}" -N in.gz "${bad0_lz}" in.bz2 "${bad1_lz}" in.lz 2>&1 | wc -l`
[ "${lines}" -eq 2 ] || test_failed $LINENO
lines=`"${ZTEST}" -Nv in.gz "${bad0_lz}" in.bz2 "${bad1_lz}" in.lz 2>&1 | wc -l`
[ "${lines}" -eq 5 ] || test_failed $LINENO
"${ZTEST}" -Nq < in
[ $? = 2 ] || test_failed $LINENO
dd if=in.lz bs=1000 count=1 2> /dev/null | "${ZTEST}" -N -q
@ -336,58 +372,122 @@ cat in.gz > a.gz || framework_failure
cat in.lz in.lz > a.lz || framework_failure
"${ZUPDATE}" -Nq -f a.bz2 a.gz
{ [ $? = 1 ] && [ -e a.bz2 ] && [ -e a.gz ] && [ -e a.lz ] ; } ||
test_failed $LINENO
[ $? = 1 ] || test_failed $LINENO
[ -e a.bz2 ] || test_failed $LINENO
[ -e a.gz ] || test_failed $LINENO
[ -e a.lz ] || test_failed $LINENO
rm -f a.lz || framework_failure
"${ZUPDATE}" -N a.bz2
{ [ $? = 0 ] && [ ! -e a.bz2 ] && [ -e a.gz ] && [ -e a.lz ] ; } ||
test_failed $LINENO
"${ZUPDATE}" -N a.bz2 || test_failed $LINENO
[ ! -e a.bz2 ] || test_failed $LINENO
[ -e a.gz ] || test_failed $LINENO
[ -e a.lz ] || test_failed $LINENO
rm -f a.lz || framework_failure
"${ZUPDATE}" -N a.gz
{ [ $? = 0 ] && [ ! -e a.bz2 ] && [ ! -e a.gz ] && [ -e a.lz ] ; } ||
test_failed $LINENO
"${ZUPDATE}" -N a.gz || test_failed $LINENO
[ ! -e a.bz2 ] || test_failed $LINENO
[ ! -e a.gz ] || test_failed $LINENO
[ -e a.lz ] || test_failed $LINENO
rm -f a.lz || framework_failure
cat in.bz2 > a.bz2 || framework_failure
cat in.gz > a.gz || framework_failure
"${ZUPDATE}" -Nq a.bz2 a.gz
{ [ $? = 1 ] && [ ! -e a.bz2 ] && [ -e a.gz ] && [ -e a.lz ] ; } ||
test_failed $LINENO
[ $? = 1 ] || test_failed $LINENO
[ ! -e a.bz2 ] || test_failed $LINENO
[ -e a.gz ] || test_failed $LINENO
[ -e a.lz ] || test_failed $LINENO
rm -f a.lz || framework_failure
cat in.bz2 > a.bz2 || framework_failure
cat in.gz > a.gz || framework_failure
"${ZUPDATE}" -N -f -k a.bz2 a.gz
{ [ $? = 0 ] && [ -e a.bz2 ] && [ -e a.gz ] && [ -e a.lz ] ; } ||
test_failed $LINENO
"${ZUPDATE}" -N -f -k a.bz2 a.gz || test_failed $LINENO
[ -e a.bz2 ] || test_failed $LINENO
[ -e a.gz ] || test_failed $LINENO
[ -e a.lz ] || test_failed $LINENO
rm -f a.lz || framework_failure
cat in.bz2 > a.bz2 || framework_failure
cat in.gz > a.gz || framework_failure
"${ZUPDATE}" -N -f a.bz2 a.gz
{ [ $? = 0 ] && [ ! -e a.bz2 ] && [ ! -e a.gz ] && [ ! -e a ] &&
[ -e a.lz ] ; } || test_failed $LINENO
"${ZUPDATE}" -N -f a.bz2 a.gz || test_failed $LINENO
[ ! -e a.bz2 ] || test_failed $LINENO
[ ! -e a.gz ] || test_failed $LINENO
[ ! -e a ] || test_failed $LINENO
[ -e a.lz ] || test_failed $LINENO
rm -f a.lz || framework_failure
cat in.bz2 > a.bz2 || framework_failure
"${ZUPDATE}" -N -1 -q a.bz2
{ [ $? = 0 ] && [ ! -e a.bz2 ] && [ -e a.lz ] ; } || test_failed $LINENO
cat "${bad0_gz}" > b.gz || framework_failure
cat in.gz > c.gz || framework_failure
"${ZUPDATE}" -N -f a.bz2 b.gz c.gz 2> /dev/null && test_failed $LINENO
[ ! -e a.bz2 ] || test_failed $LINENO
[ -e b.gz ] || test_failed $LINENO
[ -e c.gz ] || test_failed $LINENO
[ ! -e a ] || test_failed $LINENO
[ ! -e b ] || test_failed $LINENO
[ ! -e c ] || test_failed $LINENO
[ -e a.lz ] || test_failed $LINENO
rm -f a.lz b.gz c.gz || framework_failure
cat in.bz2 > a.bz2 || framework_failure
"${ZUPDATE}" -N -1 -q a.bz2 || test_failed $LINENO
[ ! -e a.bz2 ] || test_failed $LINENO
[ -e a.lz ] || test_failed $LINENO
rm -f a.lz || framework_failure
mkdir tmp2
mkdir tmp2/tmp3
cat in.bz2 > tmp2/tmp3/a.bz2 || framework_failure
cat in.gz > tmp2/tmp3/a.gz || framework_failure
"${ZUPDATE}" -N -r --format=gz tmp2
{ [ $? = 0 ] && [ -e tmp2/tmp3/a.bz2 ] && [ ! -e tmp2/tmp3/a.gz ] &&
[ -e tmp2/tmp3/a.lz ] ; } || test_failed $LINENO
"${ZUPDATE}" -N -r --format=gz tmp2 || test_failed $LINENO
[ -e tmp2/tmp3/a.bz2 ] || test_failed $LINENO
[ ! -e tmp2/tmp3/a.gz ] || test_failed $LINENO
[ -e tmp2/tmp3/a.lz ] || test_failed $LINENO
rm -f tmp2/tmp3/a.lz || framework_failure
"${ZUPDATE}" -N -r --format=bz2 tmp2
{ [ $? = 0 ] && [ ! -e tmp2/tmp3/a.bz2 ] && [ ! -e tmp2/tmp3/a.gz ] &&
[ -e tmp2/tmp3/a.lz ] ; } || test_failed $LINENO
"${ZUPDATE}" -N -r --format=bz2 tmp2 || test_failed $LINENO
[ ! -e tmp2/tmp3/a.bz2 ] || test_failed $LINENO
[ ! -e tmp2/tmp3/a.gz ] || test_failed $LINENO
[ -e tmp2/tmp3/a.lz ] || test_failed $LINENO
rm -f tmp2/tmp3/a.lz || framework_failure
cat in.bz2 > tmp2/tmp3/a.bz2 || framework_failure
cat in.gz > tmp2/tmp3/a.gz || framework_failure
cd tmp2 || framework_failure
"${ZUPDATE}" -N -r -k -f . || test_failed $LINENO
[ -e tmp3/a.bz2 ] || test_failed $LINENO
[ -e tmp3/a.gz ] || test_failed $LINENO
[ -e tmp3/a.lz ] || test_failed $LINENO
rm -f tmp3/a.lz || framework_failure
"${ZUPDATE}" -N -r -k -f || test_failed $LINENO
[ -e tmp3/a.bz2 ] || test_failed $LINENO
[ -e tmp3/a.gz ] || test_failed $LINENO
[ -e tmp3/a.lz ] || test_failed $LINENO
rm -f tmp3/a.lz || framework_failure
"${ZUPDATE}" -N -R -k -f . || test_failed $LINENO
[ -e tmp3/a.bz2 ] || test_failed $LINENO
[ -e tmp3/a.gz ] || test_failed $LINENO
[ -e tmp3/a.lz ] || test_failed $LINENO
rm -f tmp3/a.lz || framework_failure
"${ZUPDATE}" -N -R -k -f || test_failed $LINENO
[ -e tmp3/a.bz2 ] || test_failed $LINENO
[ -e tmp3/a.gz ] || test_failed $LINENO
[ -e tmp3/a.lz ] || test_failed $LINENO
rm -f tmp3/a.lz || framework_failure
"${ZUPDATE}" -N -r -f . || test_failed $LINENO
[ ! -e tmp3/a.bz2 ] || test_failed $LINENO
[ ! -e tmp3/a.gz ] || test_failed $LINENO
[ -e tmp3/a.lz ] || test_failed $LINENO
cd .. || framework_failure
rm -r tmp2 || framework_failure
if ln -s '.' slink 2> /dev/null ; then
"${ZCAT}" -N -r slink > /dev/null || test_failed $LINENO
"${ZGREP}" -N -r "GNU" slink > /dev/null || test_failed $LINENO
"${ZTEST}" -N -r slink || test_failed $LINENO
"${ZUPDATE}" -N -r -f slink || test_failed $LINENO
else
printf "\nwarning: skipping link test: 'ln' does not work on your system."
fi
rm -f slink || framework_failure
echo
if [ ${fail} = 0 ] ; then
echo "tests completed successfully."

BIN
testsuite/test_bad_crc.lz Normal file
View file

Binary file not shown.

68
testsuite/zcat_vs.dat Normal file
View file

@ -0,0 +1,68 @@
Worst case test file for zcat -vs.
First 4096 input bytes produce 4095 output bytes because of -s.
Next 4096 input bytes produce 16384 output bytes, accumulating a total
of 20479 bytes in the output buffer.
----------------------------------------------
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
...............................................................
€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€

BIN
testsuite/zero_bad_crc.gz Normal file
View file

Binary file not shown.

BIN
testsuite/zero_bad_crc.lz Normal file
View file

Binary file not shown.

45
zcat.cc
View file

@ -1,5 +1,5 @@
/* Zcat - decompress and concatenate files to standard output
Copyright (C) 2010-2018 Antonio Diaz Diaz.
Copyright (C) 2010-2019 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -93,12 +93,14 @@ void show_help()
std::printf( "Zcat copies each given file to standard output. If any given file is\n"
"compressed, its decompressed content is used. If a given file does not\n"
"exist, and its name does not end with one of the known extensions, zcat\n"
"tries the compressed file names corresponding to the supported formats.\n"
"\nIf no files are specified, or if a file is specified as '-', data are\n"
"read from standard input, decompressed if needed, and sent to standard\n"
"output. Data read from standard input must be of the same type; all\n"
"uncompressed or all in the same compression format.\n"
"\nThe supported formats are bzip2, gzip, lzip and xz.\n"
"tries the compressed file names corresponding to the formats supported.\n"
"\nIf a file is specified as '-', data are read from standard input,\n"
"decompressed if needed, and sent to standard output. Data read from\n"
"standard input must be of the same type; all uncompressed or all in the\n"
"same compression format.\n"
"\nIf no files are specified, recursive searches examine the current\n"
"working directory, and nonrecursive searches read standard input.\n"
"\nThe formats supported are bzip2, gzip, lzip and xz.\n"
"\nUsage: zcat [options] [files]\n"
"\nExit status is 0 if no errors occurred, non-zero otherwise.\n"
"\nOptions:\n"
@ -114,6 +116,7 @@ void show_help()
" -O, --force-format=<fmt> force given format (bz2, gz, lz, xz)\n"
" -q, --quiet suppress all messages\n"
" -r, --recursive operate recursively on directories\n"
" -R, --dereference-recursive recursively follow symbolic links\n"
" -s, --squeeze-blank never more than one single blank line\n"
" -t equivalent to '-vT'\n"
" -T, --show-tabs display TAB characters as '^I'\n"
@ -153,7 +156,7 @@ int do_cat( const int infd, const int buffer_size,
rd = readblock( infd, inbuf, buffer_size );
if( rd != buffer_size && errno )
{
show_error2( "Error reading file", input_filename.c_str() );
show_file_error( input_filename.c_str(), "Read error", errno );
return 1;
}
if( rd == 0 )
@ -226,11 +229,13 @@ int do_cat( const int infd, const int buffer_size,
int cat( int infd, const int format_index, const std::string & input_filename,
const Cat_options & cat_options )
{
enum { buffer_size = 4096 };
enum { buffer_size = 4096, outbuf_size = (5 * buffer_size) + 256 + 1 };
// buffer with space for sentinel newline at the end
uint8_t * const inbuf = new uint8_t[buffer_size+1];
// buffer with space for character quoting and 255-digit line number
uint8_t * const outbuf = new uint8_t[(4*buffer_size)+256];
// buffer with space for character quoting, 255-digit line number,
// worst case flushing respect to inbuf, and a canary byte.
uint8_t * const outbuf = new uint8_t[outbuf_size];
outbuf[outbuf_size-1] = 0;
int retval = 0;
Children children;
if( !set_data_feeder( input_filename, &infd, children, format_index ) )
@ -243,6 +248,8 @@ int cat( int infd, const int format_index, const std::string & input_filename,
if( retval == 0 && close( infd ) != 0 )
{ show_close_error(); retval = 1; }
if( outbuf[outbuf_size-1] != 0 )
internal_error( "buffer overflow." );
delete[] outbuf; delete[] inbuf;
return retval;
}
@ -253,10 +260,8 @@ int cat( int infd, const int format_index, const std::string & input_filename,
int main( const int argc, const char * const argv[] )
{
enum { verbose_opt = 256, bz2_opt, gz_opt, lz_opt, xz_opt };
int infd = -1;
int format_index = -1;
bool recursive = false;
std::string input_filename;
int recursive = 0; // 1 = '-r', 2 = '-R'
std::list< std::string > filenames;
Cat_options cat_options;
invocation_name = argv[0];
@ -280,6 +285,7 @@ int main( const int argc, const char * const argv[] )
{ 'O', "force-format", Arg_parser::yes },
{ 'q', "quiet", Arg_parser::no },
{ 'r', "recursive", Arg_parser::no },
{ 'R', "dereference-recursive", Arg_parser::no },
{ 's', "squeeze-blank", Arg_parser::no }, // cat
{ 't', 0, Arg_parser::no }, // cat
{ 'T', "show-tabs", Arg_parser::no }, // cat
@ -324,7 +330,8 @@ int main( const int argc, const char * const argv[] )
case 'N': break;
case 'O': format_index = parse_format_type( arg ); break;
case 'q': verbosity = -1; break;
case 'r': recursive = true; break;
case 'r': recursive = 1; break;
case 'R': recursive = 2; break;
case 's': cat_options.squeeze_blank = true; break;
case 't': cat_options.show_nonprinting = true; // fall through
case 'T': cat_options.show_tabs = true; break;
@ -347,13 +354,15 @@ int main( const int argc, const char * const argv[] )
for( ; argind < parser.arguments(); ++argind )
filenames.push_back( parser.argument( argind ) );
if( filenames.empty() ) filenames.push_back( "-" );
if( filenames.empty() ) filenames.push_back( recursive ? "." : "-" );
std::string input_filename;
int retval = 0;
bool error = false;
bool stdin_used = false;
while( next_filename( filenames, input_filename, error, recursive ) )
{
int infd;
if( input_filename.empty() )
{
if( stdin_used ) continue; else stdin_used = true;
@ -368,12 +377,12 @@ int main( const int argc, const char * const argv[] )
const int tmp = cat( infd, format_index, input_filename, cat_options );
if( tmp > retval ) retval = tmp;
if( input_filename.size() ) { close( infd ); infd = -1; }
if( input_filename.size() ) close( infd );
}
if( std::fclose( stdout ) != 0 )
{
show_error( "Can't close stdout", errno );
show_error( "Error closing stdout", errno );
error = true;
}
if( error && retval == 0 ) retval = 1;

6
zcatgrep.cc
View file

@ -1,5 +1,5 @@
/* Common code for zcat and zgrep
Copyright (C) 2010-2018 Antonio Diaz Diaz.
Copyright (C) 2010-2019 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -39,6 +39,7 @@ int open_instream( std::string & input_filename, const bool search,
int infd = open( input_filename.c_str(), O_RDONLY | O_BINARY );
if( infd < 0 )
{
const int saved_errno = errno;
if( search && simple_extension_index( input_filename ) < 0 )
{
for( int i = 0; i < num_formats; ++i )
@ -51,7 +52,8 @@ int open_instream( std::string & input_filename, const bool search,
}
}
if( infd < 0 && !no_messages )
show_error2( "Can't open input file", input_filename.c_str() );
show_file_error( input_filename.c_str(), "Can't open input file",
saved_errno );
}
return infd;
}

15
zcmp.cc
View file

@ -1,5 +1,5 @@
/* Zcmp - decompress and compare two files byte by byte
Copyright (C) 2010-2018 Antonio Diaz Diaz.
Copyright (C) 2010-2019 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -55,7 +55,7 @@ void show_help()
"are numbered starting with 1. If any given file is compressed, its\n"
"decompressed content is used. Compressed files are decompressed on the\n"
"fly; no temporary files are created.\n"
"\nThe supported formats are bzip2, gzip, lzip and xz.\n"
"\nThe formats supported are bzip2, gzip, lzip and xz.\n"
"\nUsage: zcmp [options] file1 [file2]\n"
"\nZcmp compares file1 to file2. If file2 is omitted zcmp tries the\n"
"following:\n"
@ -243,7 +243,7 @@ int cmp( const long long max_size, const int infd[2],
rd[i] = readblock( infd[i], buffer[i], size );
if( rd[i] != size && errno )
{
show_error2( "Error reading file", filenames[i].c_str() );
show_file_error( filenames[i].c_str(), "Read error", errno );
return 2;
}
}
@ -274,6 +274,7 @@ int cmp( const long long max_size, const int infd[2],
filenames[0].c_str(), filenames[1].c_str(),
byte_number, line_number, c0, buf0, c1, buf1 );
}
std::fflush( stdout );
return 1;
}
else // verbosity > 0 ; show all differences
@ -296,6 +297,7 @@ int cmp( const long long max_size, const int infd[2],
}
}
}
std::fflush( stdout );
}
}
@ -435,7 +437,8 @@ int main( const int argc, const char * const argv[] )
for( int i = 0; i < 2; ++i )
if( !skip_ignore_initial( ignore_initial[i], infd[i] ) )
{
show_error2( "Can't skip initial bytes from file", filenames[i].c_str() );
show_file_error( filenames[i].c_str(),
"Read error skipping initial bytes", errno );
return 2;
}
@ -450,13 +453,13 @@ int main( const int argc, const char * const argv[] )
{ show_close_error(); retval = 2; }
if( filenames[i] != "-" && close( old_infd[i] ) != 0 )
{
show_error2( "Can't close input file", filenames[i].c_str() );
show_file_error( filenames[i].c_str(), "Error closing input file", errno );
retval = 2;
}
}
if( std::fclose( stdout ) != 0 )
{
show_error( "Can't close stdout", errno );
show_error( "Error closing stdout", errno );
retval = 2;
}

4
zcmpdiff.cc
View file

@ -1,5 +1,5 @@
/* Common code for zcmp and zdiff
Copyright (C) 2010-2018 Antonio Diaz Diaz.
Copyright (C) 2010-2019 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -24,7 +24,7 @@ int open_instream( const std::string & input_filename )
{
const int infd = open( input_filename.c_str(), O_RDONLY | O_BINARY );
if( infd < 0 )
show_error2( "Can't open input file", input_filename.c_str() );
show_file_error( input_filename.c_str(), "Can't open input file", errno );
return infd;
}

8
zdiff.cc
View file

@ -1,5 +1,5 @@
/* Zdiff - decompress and compare two files line by line
Copyright (C) 2010-2018 Antonio Diaz Diaz.
Copyright (C) 2010-2019 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -53,7 +53,7 @@ void show_help()
"compressed, its decompressed content is used. Zdiff is a front end to\n"
"the diff program and has the limitation that messages from diff refer to\n"
"temporary filenames instead of those specified.\n"
"\nThe supported formats are bzip2, gzip, lzip and xz.\n"
"\nThe formats supported are bzip2, gzip, lzip and xz.\n"
"\nUsage: zdiff [options] file1 [file2]\n"
"\nZdiff compares file1 to file2. If file2 is omitted zdiff tries the\n"
"following:\n"
@ -141,7 +141,7 @@ bool set_fifonames( const std::string filenames[2] )
if( mkfifo( fifonames[i].c_str(), S_IRUSR | S_IWUSR ) == 0 )
continue;
}
show_error2( "Can't create FIFO", fifonames[i].c_str() );
show_file_error( fifonames[i].c_str(), "Can't create FIFO", errno );
return false;
}
return true;
@ -419,7 +419,7 @@ int main( const int argc, const char * const argv[] )
for( int i = 0; i < 2; ++i )
if( filenames[i] != "-" && close( infd[i] ) != 0 )
{
show_error2( "Can't close input file", filenames[i].c_str() );
show_file_error( filenames[i].c_str(), "Error closing input file", errno );
retval = 2;
}

38
zgrep.cc
View file

@ -1,5 +1,5 @@
/* Zgrep - search compressed files for a regular expression
Copyright (C) 2010-2018 Antonio Diaz Diaz.
Copyright (C) 2010-2019 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -52,12 +52,14 @@ void show_help()
"file is compressed, its decompressed content is used. If a given file\n"
"does not exist, and its name does not end with one of the known\n"
"extensions, zgrep tries the compressed file names corresponding to the\n"
"supported formats.\n"
"\nIf no files are specified, or if a file is specified as '-', data are\n"
"read from standard input, decompressed if needed, and fed to grep. Data\n"
"read from standard input must be of the same type; all uncompressed or\n"
"all in the same compression format.\n"
"\nThe supported formats are bzip2, gzip, lzip and xz.\n"
"formats supported.\n"
"\nIf a file is specified as '-', data are read from standard input,\n"
"decompressed if needed, and fed to grep. Data read from standard input\n"
"must be of the same type; all uncompressed or all in the same\n"
"compression format.\n"
"\nIf no files are specified, recursive searches examine the current\n"
"working directory, and nonrecursive searches read standard input.\n"
"\nThe formats supported are bzip2, gzip, lzip and xz.\n"
"\nUsage: zgrep [options] <pattern> [files]\n"
"\nExit status is 0 if match, 1 if no match, 2 if trouble.\n"
"\nOptions:\n"
@ -88,6 +90,7 @@ void show_help()
" -O, --force-format=<fmt> force given format (bz2, gz, lz, xz)\n"
" -q, --quiet suppress all messages\n"
" -r, --recursive operate recursively on directories\n"
" -R, --dereference-recursive recursively follow symbolic links\n"
" -s, --no-messages suppress error messages\n"
" -v, --invert-match select non-matching lines\n"
" --verbose verbose mode (show error messages)\n"
@ -187,7 +190,8 @@ int zgrep_file( int infd, const int format_index,
putchar( buffer[i] );
}
else if( std::fwrite( buffer, 1, size, stdout ) != (unsigned)size )
{ show_error( "Write error", errno ); return 2; }
{ std::fflush( stdout ); show_error( "Write error", errno ); return 2; }
std::fflush( stdout );
}
if( size < buffer_size ) break;
}
@ -197,7 +201,7 @@ int zgrep_file( int infd, const int format_index,
if( !good_status( children, retval == 1 ) ) retval = 2;
if( list_mode && (retval == 0) == (list_mode == 1) )
std::printf( "%s\n", input_filename.c_str() );
{ std::printf( "%s\n", input_filename.c_str() ); std::fflush( stdout ); }
if( close( infd ) != 0 )
{ show_close_error(); return 2; }
if( close( fda[0] ) != 0 )
@ -213,12 +217,10 @@ int main( const int argc, const char * const argv[] )
enum { help_opt = 256, verbose_opt, color_opt,
bz2_opt, gz_opt, lz_opt, xz_opt };
int format_index = -1;
int infd = -1;
int list_mode = 0; // 1 = list matches, -1 = list non matches
int recursive = 0; // 1 = '-r', 2 = '-R'
int show_name = -1; // tri-state bool
bool no_messages = false;
bool recursive = false;
std::string input_filename;
std::list< std::string > filenames;
std::vector< const char * > grep_args; // args to grep, maybe empty
std::string color_option; // needed because of optional arg
@ -251,6 +253,7 @@ int main( const int argc, const char * const argv[] )
{ 'O', "force-format", Arg_parser::yes },
{ 'q', "quiet", Arg_parser::no },
{ 'r', "recursive", Arg_parser::no },
{ 'R', "dereference-recursive", Arg_parser::no },
{ 's', "no-messages", Arg_parser::no }, // grep
{ 'v', "invert-match", Arg_parser::no }, // grep
{ 'V', "version", Arg_parser::no },
@ -309,7 +312,8 @@ int main( const int argc, const char * const argv[] )
case 'o': grep_args.push_back( "-o" ); break;
case 'O': format_index = parse_format_type( arg ); break;
case 'q': grep_args.push_back( "-q" ); verbosity = -1; break;
case 'r': recursive = true; break;
case 'r': recursive = 1; break;
case 'R': recursive = 2; break;
case 's': grep_args.push_back( "-s" ); no_messages = true; break;
case 'v': grep_args.push_back( "-v" ); break;
case 'V': show_version(); return 0;
@ -349,16 +353,18 @@ int main( const int argc, const char * const argv[] )
for( ; argind < parser.arguments(); ++argind )
filenames.push_back( parser.argument( argind ) );
if( filenames.empty() ) filenames.push_back( "-" );
if( filenames.empty() ) filenames.push_back( recursive ? "." : "-" );
if( show_name < 0 ) show_name = ( filenames.size() != 1 || recursive );
std::string input_filename;
int retval = 1;
bool error = false;
bool stdin_used = false;
while( next_filename( filenames, input_filename, error, recursive,
false, no_messages ) )
{
int infd;
if( input_filename.empty() )
{
if( stdin_used ) continue; else stdin_used = true;
@ -377,13 +383,13 @@ int main( const int argc, const char * const argv[] )
list_mode, show_name );
if( tmp == 0 || ( tmp == 2 && retval == 1 ) ) retval = tmp;
if( input_filename.size() ) { close( infd ); infd = -1; }
if( input_filename.size() ) close( infd );
if( retval == 0 && verbosity < 0 ) break;
}
if( std::fclose( stdout ) != 0 )
{
show_error( "Can't close stdout", errno );
show_error( "Error closing stdout", errno );
error = true;
}
if( error && ( retval != 0 || verbosity >= 0 ) ) retval = 2;

34
ztest.cc
View file

@ -1,5 +1,5 @@
/* Ztest - verify the integrity of compressed files
Copyright (C) 2010-2018 Antonio Diaz Diaz.
Copyright (C) 2010-2019 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -51,11 +51,12 @@ namespace {
void show_help()
{
std::printf( "Ztest verifies the integrity of the specified compressed files.\n"
"Uncompressed files are ignored. If no files are specified, or if a file\n"
"is specified as '-', the integrity of compressed data read from standard\n"
"input is verified. Data read from standard input must be all in the same\n"
"compression format.\n"
"\nThe supported formats are bzip2, gzip, lzip and xz.\n"
"Uncompressed files are ignored. If a file is specified as '-', the\n"
"integrity of compressed data read from standard input is verified. Data\n"
"read from standard input must be all in the same compression format.\n"
"\nIf no files are specified, recursive searches examine the current\n"
"working directory, and nonrecursive searches read standard input.\n"
"\nThe formats supported are bzip2, gzip, lzip and xz.\n"
"\nNote that error detection in the xz format is broken. First, some xz\n"
"files lack integrity information. Second, not all xz decompressors can\n"
"verify the integrity of all xz files. Third, section 2.1.1.2 'Stream\n"
@ -74,6 +75,7 @@ void show_help()
" -O, --force-format=<fmt> force given format (bz2, gz, lz, xz)\n"
" -q, --quiet suppress all messages\n"
" -r, --recursive operate recursively on directories\n"
" -R, --dereference-recursive recursively follow symbolic links\n"
" -v, --verbose be verbose (a 2nd -v gives more)\n"
" --bz2=<command> set compressor and options for bzip2 format\n"
" --gz=<command> set compressor and options for gzip format\n"
@ -87,7 +89,7 @@ int open_instream( const std::string & input_filename )
{
const int infd = open( input_filename.c_str(), O_RDONLY | O_BINARY );
if( infd < 0 )
show_error2( "Can't open input file", input_filename.c_str() );
show_file_error( input_filename.c_str(), "Can't open input file", errno );
return infd;
}
@ -211,10 +213,8 @@ int ztest_file( const int infd, int format_index,
int main( const int argc, const char * const argv[] )
{
enum { bz2_opt = 256, gz_opt, lz_opt, xz_opt };
int infd = -1;
int format_index = -1;
bool recursive = false;
std::string input_filename;
int recursive = 0; // 1 = '-r', 2 = '-R'
std::list< std::string > filenames;
std::vector< const char * > ztest_args; // args to ztest, maybe empty
invocation_name = argv[0];
@ -228,6 +228,7 @@ int main( const int argc, const char * const argv[] )
{ 'O', "force-format", Arg_parser::yes },
{ 'q', "quiet", Arg_parser::no },
{ 'r', "recursive", Arg_parser::no },
{ 'R', "dereference-recursive", Arg_parser::no },
{ 'v', "verbose", Arg_parser::no },
{ 'V', "version", Arg_parser::no },
{ bz2_opt, "bz2", Arg_parser::yes },
@ -255,7 +256,8 @@ int main( const int argc, const char * const argv[] )
case 'N': break;
case 'O': format_index = parse_format_type( arg ); break;
case 'q': verbosity = -1; ztest_args.push_back( "-q" ); break;
case 'r': recursive = true; break;
case 'r': recursive = 1; break;
case 'R': recursive = 2; break;
case 'v': if( verbosity < 4 ) ++verbosity;
ztest_args.push_back( "-v" ); break;
case 'V': show_version(); return 0;
@ -275,13 +277,15 @@ int main( const int argc, const char * const argv[] )
for( ; argind < parser.arguments(); ++argind )
filenames.push_back( parser.argument( argind ) );
if( filenames.empty() ) filenames.push_back( "-" );
if( filenames.empty() ) filenames.push_back( recursive ? "." : "-" );
std::string input_filename;
int retval = 0;
bool error = false;
bool stdin_used = false;
while( next_filename( filenames, input_filename, error, recursive ) )
{
int infd;
if( input_filename.empty() )
{
if( stdin_used ) continue; else stdin_used = true;
@ -299,12 +303,12 @@ int main( const int argc, const char * const argv[] )
else tmp = ztest_file( infd, format_index, input_filename, ztest_args );
if( tmp > retval ) retval = tmp;
if( input_filename.size() ) { close( infd ); infd = -1; }
if( input_filename.size() ) close( infd );
}
if( std::fclose( stdout ) != 0 )
if( std::fclose( stdout ) != 0 ) // in case decompressor writes to stdout
{
show_error( "Can't close stdout", errno );
show_error( "Error closing stdout", errno );
error = true;
}
if( error && retval == 0 ) retval = 1;

28
zupdate.cc
View file

@ -1,5 +1,5 @@
/* Zupdate - recompress bzip2, gzip, xz files to lzip format
Copyright (C) 2013-2018 Antonio Diaz Diaz.
Copyright (C) 2013-2019 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -51,14 +51,15 @@ namespace {
void show_help()
{
std::printf( "Zupdate recompresses files from bzip2, gzip, and xz formats to lzip format.\n"
"The originals are compared with the new files and then deleted.\n"
std::printf( "Zupdate recompresses files from bzip2, gzip, and xz formats to lzip\n"
"format. Each original is compared with the new file and then deleted.\n"
"Only regular files with standard file name extensions are recompressed,\n"
"other files are ignored.\n"
"Compressed files are decompressed and then recompressed on the fly; no\n"
"temporary files are created.\n"
"The lzip format is chosen as destination because it is by far the most\n"
"appropriate for long-term data archiving.\n"
"other files are ignored. Compressed files are decompressed and then\n"
"recompressed on the fly; no temporary files are created. The lzip format\n"
"is chosen as destination because it is the most appropriate for\n"
"long-term data archiving.\n"
"\nIf no files are specified, recursive searches examine the current\n"
"working directory, and nonrecursive searches do nothing.\n"
"\nIf the lzip compressed version of a file already exists, the file is\n"
"skipped unless the '--force' option is given. In this case, if the\n"
"comparison with the existing lzip version fails, an error is returned\n"
@ -79,6 +80,7 @@ void show_help()
" -N, --no-rcfile don't read runtime configuration file\n"
" -q, --quiet suppress all messages\n"
" -r, --recursive operate recursively on directories\n"
" -R, --dereference-recursive recursively follow symbolic links\n"
" -v, --verbose be verbose (a 2nd -v gives more)\n"
" -0 .. -9 set compression level [default 9]\n"
" --bz2=<command> set compressor and options for bzip2 format\n"
@ -293,13 +295,12 @@ int zupdate_file( const std::string & name, const char * const lzip_name,
int main( const int argc, const char * const argv[] )
{
enum { bz2_opt = 256, gz_opt, lz_opt, xz_opt };
std::string input_filename;
int recursive = 0; // 1 = '-r', 2 = '-R'
std::list< std::string > filenames;
std::vector< std::string > lzip_args2; // args to lzip, maybe empty
bool force = false;
bool keep_input_files = false;
bool no_rcfile = false;
bool recursive = false;
invocation_name = argv[0];
program_name = "zupdate";
@ -323,6 +324,7 @@ int main( const int argc, const char * const argv[] )
{ 'N', "no-rcfile", Arg_parser::no },
{ 'q', "quiet", Arg_parser::no },
{ 'r', "recursive", Arg_parser::no },
{ 'R', "dereference-recursive", Arg_parser::no },
{ 'v', "verbose", Arg_parser::no },
{ 'V', "version", Arg_parser::no },
{ bz2_opt, "bz2", Arg_parser::yes },
@ -355,7 +357,8 @@ int main( const int argc, const char * const argv[] )
case 'M': parse_format_list( arg ); break;
case 'N': no_rcfile = true; break;
case 'q': verbosity = -1; lzip_args2.push_back( "-q" ); break;
case 'r': recursive = true; break;
case 'r': recursive = 1; break;
case 'R': recursive = 2; break;
case 'v': if( verbosity < 4 ) ++verbosity; break;
case 'V': show_version(); return 0;
case bz2_opt: parse_compressor( arg, fmt_bz2, 1 ); break;
@ -378,6 +381,9 @@ int main( const int argc, const char * const argv[] )
for( ; argind < parser.arguments(); ++argind )
filenames.push_back( parser.argument( argind ) );
if( filenames.empty() && recursive ) filenames.push_back( "." );
std::string input_filename;
int retval = 0;
bool error = false;
while( next_filename( filenames, input_filename, error, recursive, true ) )

11
zutils.cc
View file

@ -1,5 +1,5 @@
/* Zutils - Utilities dealing with compressed files
Copyright (C) 2009-2018 Antonio Diaz Diaz.
Copyright (C) 2009-2019 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@ -143,16 +143,17 @@ bool good_status( const Children & children, const bool finished )
const pid_t pid = children.pid[i];
if( pid )
{
const char * const msg =
const char * const name =
( i & 1 ) ? children.compressor_name : "data feeder";
if( !finished )
{
const int tmp = child_status( pid, msg );
if( tmp < 0 ) kill( pid, SIGTERM ); // child not terminated
const int tmp = child_status( pid, name );
if( tmp < 0 ) // child not terminated
{ kill( pid, SIGTERM ); wait_for_child( pid, name ); }
else if( tmp != 0 ) error = true; // child status != 0
}
else
if( wait_for_child( pid, msg ) != 0 ) error = true;
if( wait_for_child( pid, name ) != 0 ) error = true;
}
}
return !error;

2
zutils.h
View file

@ -1,5 +1,5 @@
/* Zutils - Utilities dealing with compressed files
Copyright (C) 2009-2018 Antonio Diaz Diaz.
Copyright (C) 2009-2019 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by