1
0
Fork 0

Merging upstream version 1.13~rc1.

Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-24 06:03:46 +01:00
parent f40403d840
commit 95e3ee3bd3
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
29 changed files with 472 additions and 517 deletions

View file

@ -1,3 +1,14 @@
2023-12-31 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.13-rc1 released.
* zutils.cc (test_format): Fix detection of bzip2 with no blocks.
* rc.h (format_order): Put fmt_gz before fmt_bz2.
* zcmpdiff.cc (open_other_instream): Try also other compressed formats.
* zcmp.cc (cmp): Report EOF on empty file like GNU cmp.
* zupdate.cc: Reformat file diagnostics as 'PROGRAM: FILE: MESSAGE'.
* Replace 'verify' with 'check'.
* configure, Makefile.in: New variable 'MAKEINFO'.
2023-01-07 Antonio Diaz Diaz <antonio@gnu.org> 2023-01-07 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.12 released. * Version 1.12 released.
@ -113,7 +124,7 @@
* Version 1.3 released. * Version 1.3 released.
* check.sh: Fix two values of expected exit status. * check.sh: Fix two values of expected exit status.
* zutils.texi: Document that '--format' does not verify format. * zutils.texi: Document that '--format' does not check format.
* Add two missing #includes. * Add two missing #includes.
* Change license to GPL version 2 or later. * Change license to GPL version 2 or later.
@ -175,7 +186,7 @@
* ztest.cc: New file implementing ztest functionality in C++. * ztest.cc: New file implementing ztest functionality in C++.
* Makefile.in: Add quotes to directory names. * Makefile.in: Add quotes to directory names.
* check.sh: Use 'test.txt' instead of 'COPYING' for testing. * check.sh: Use 'test.txt' instead of 'COPYING' for testing.
* Remove environment safeguards from configure as requested by * configure: Remove environment safeguards as requested by
Richard Stallman. Now environment variables affect configure. Richard Stallman. Now environment variables affect configure.
2009-10-21 Antonio Diaz Diaz <ant_diaz@teleline.es> 2009-10-21 Antonio Diaz Diaz <ant_diaz@teleline.es>

View file

@ -20,8 +20,8 @@ gzip scripts, the recommended method is to configure gzip as follows:
./configure --program-transform-name='s/^z/gz/' ./configure --program-transform-name='s/^z/gz/'
This renames, at installation time, the gzip scripts and man pages to This renames, at installation time, the gzip scripts and man pages to
'gzcat', 'gzcat.1', etc, avoiding the name clashing with the programs 'gzcat', 'gzcat.1', etc, avoiding the name clashing with the programs and
and man pages from zutils. man pages from zutils.
Procedure Procedure
@ -48,7 +48,8 @@ extracted from the archive.
4. Optionally, type 'make check' to run the tests that come with zutils. 4. Optionally, type 'make check' to run the tests that come with zutils.
5. Type 'make install' to install the programs and any data files and 5. Type 'make install' to install the programs and any data files and
documentation. documentation. You need root privileges to install into a prefix owned
by root.
Or type 'make install-compress', which additionally compresses the Or type 'make install-compress', which additionally compresses the
info manual and the man pages after installation. info manual and the man pages after installation.

View file

@ -66,6 +66,10 @@ zgrep.o : zgrep.cc
%.o : %.cc %.o : %.cc
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -c -o $@ $< $(CXX) $(CPPFLAGS) $(CXXFLAGS) -c -o $@ $<
# prevent 'make' from trying to remake source files
$(VPATH)/configure $(VPATH)/Makefile.in $(VPATH)/doc/$(pkgname).texi : ;
%.h %.cc : ;
$(objs) : Makefile $(objs) : Makefile
$(scripts) : Makefile $(scripts) : Makefile
arg_parser.o : arg_parser.h arg_parser.o : arg_parser.h
@ -78,7 +82,6 @@ ztest.o : arg_parser.h rc.h zutils.h recursive.cc
zupdate.o : arg_parser.h rc.h recursive.cc zupdate.o : arg_parser.h rc.h recursive.cc
zutils.o : rc.h zutils.h zutils.o : rc.h zutils.h
doc : info man doc : info man
info : $(VPATH)/doc/$(pkgname).info info : $(VPATH)/doc/$(pkgname).info
@ -106,7 +109,7 @@ $(VPATH)/doc/zgrep.1 : zgrep
-o $@ --info-page=$(pkgname) ./zgrep -o $@ --info-page=$(pkgname) ./zgrep
$(VPATH)/doc/ztest.1 : ztest $(VPATH)/doc/ztest.1 : ztest
help2man -n 'verify the integrity of compressed files' \ help2man -n 'check the integrity of compressed files' \
-o $@ --info-page=$(pkgname) ./ztest -o $@ --info-page=$(pkgname) ./ztest
$(VPATH)/doc/zupdate.1 : zupdate $(VPATH)/doc/zupdate.1 : zupdate

70
NEWS
View file

@ -1,64 +1,18 @@
Changes in version 1.12: Changes in version 1.13:
The zutils configuration file 'zutilsrc' has been renamed to 'zutils.conf'. The detection of bzip2 files with no compressed blocks has been fixed.
Zutils now looks for the configuration file in $XDG_CONFIG_HOME/zutils.conf (Error introduced in version 1.9).
instead of $HOME/.zutilsrc. (XDG_CONFIG_HOME defaults to $HOME/.config).
(Suggested by Adam Tuja).
In zcat, zcmp, zdiff, and zgrep, the option '-O, --force-format' now can When zcat, zcmp, zdiff, or zgrep need to try compressed file names, gzip
force also "uncompressed" format. (.gz) is now tried before bzip2 (.bz2).
zcmp now accepts the option '-H, --hexadecimal' to print byte values in When only one compressed file is passed to zcmp or zdiff, they now try to
hexadecimal instead of octal. compare it with a compressed file of any of the remaining formats if the
corresponding uncompressed file does not exist.
In zcmp: zcmp now reports EOF on empty file like GNU cmp:
The long name of option '-s' has been changed to '--script' following a "zcmp: EOF on FILE which is empty".
similar change made to GNU ed.
The short name '-q' has been assigned to options '--quiet' and '--silent'. File diagnostics in zupdate have been reformatted as 'PROGRAM: FILE: MESSAGE'.
Option '-q' now only suppresses diagnostic messages written to stderr. The variable MAKEINFO has been added to configure and Makefile.in.
Option '-s' now only suppresses messages about file differences written to
stdout or stderr.
Option '-l, --list' is now different from option '-v, --verbose', which
now undoes the effect of '--quiet'.
zcmp now prints byte and line in EOF message like GNU cmp:
"zcmp: EOF on FILE after byte B, in line L".
zgrep now also accepts the following options: '-G, --basic-regexp',
'--label=<label>', '--line-buffered', '-P, --perl-regexp', '--silent',
'-T, --initial-tab', '-U, --binary', and '-Z, --null'.
(Reported by Chris Jamboretz and Leah Neukirchen).
ztest now exits with status 2 if an uncompressed file has a compressed file
name extension, or if a compressed file has a wrong compressed extension.
zupdate now accepts option '-d, --destdir' to write recompressed files to
another directory. This allows, for example, recompressing files from a
read-only file system to another place without needing to copy or link them
to the destination directory first.
zupdate now accepts option '-e, --expand-extensions', which makes it expand
combined file name extensions; tgz --> tar.lz.
zupdate now also accepts option '-i, --ignore-errors', which makes it ignore
non-fatal errors. (Suggested by Antoni Sawicki).
All utilities now support compress'd (.Z) files through gzip. For this to
work, the gzip program used (for example GNU gzip) must be able to
decompress .Z files.
At verbosity level 1 (2 for zdiff and zgrep) or higher, '-V, --version' now
prints the versions of the compressors used (limited by option '-M, --format').
(The compressors used must support option '-V' for this to work).
Diagnostics caused by invalid arguments to command line options now show the
argument and the name of the option.
It has been documented in the manual that the data format is detected by its
magic bytes, not by the file name extension.
The testsuite now tests tarlz (if available) as compressor for zupdate.

9
README
View file

@ -4,8 +4,8 @@ Zutils is a collection of utilities able to process any combination of
compressed and uncompressed files transparently. If any file given, compressed and uncompressed files transparently. If any file given,
including standard input, is compressed, its decompressed content is used. including standard input, is compressed, its decompressed content is used.
Compressed files are decompressed on the fly; no temporary files are Compressed files are decompressed on the fly; no temporary files are
created. Data format is detected by its magic bytes, not by the file name created. Data format is detected by its identifier string (magic bytes), not
extension. by the file name extension. Empty files are considered uncompressed.
These utilities are not wrapper scripts but safer and more efficient C++ These utilities are not wrapper scripts but safer and more efficient C++
programs. In particular the option '--recursive' is very efficient in programs. In particular the option '--recursive' is very efficient in
@ -45,6 +45,5 @@ Copyright (C) 2009-2023 Antonio Diaz Diaz.
This file is free documentation: you have unlimited permission to copy, This file is free documentation: you have unlimited permission to copy,
distribute, and modify it. distribute, and modify it.
The file Makefile.in is a data file used by configure to produce the The file Makefile.in is a data file used by configure to produce the Makefile.
Makefile. It has the same copyright owner and permissions that configure It has the same copyright owner and permissions that configure itself.
itself.

View file

@ -1,4 +1,4 @@
/* Arg_parser - POSIX/GNU command line argument parser. (C++ version) /* Arg_parser - POSIX/GNU command-line argument parser. (C++ version)
Copyright (C) 2006-2023 Antonio Diaz Diaz. Copyright (C) 2006-2023 Antonio Diaz Diaz.
This library is free software. Redistribution and use in source and This library is free software. Redistribution and use in source and

View file

@ -1,4 +1,4 @@
/* Arg_parser - POSIX/GNU command line argument parser. (C++ version) /* Arg_parser - POSIX/GNU command-line argument parser. (C++ version)
Copyright (C) 2006-2023 Antonio Diaz Diaz. Copyright (C) 2006-2023 Antonio Diaz Diaz.
This library is free software. Redistribution and use in source and This library is free software. Redistribution and use in source and

10
configure vendored
View file

@ -6,7 +6,7 @@
# to copy, distribute, and modify it. # to copy, distribute, and modify it.
pkgname=zutils pkgname=zutils
pkgversion=1.12 pkgversion=1.13-rc1
srctrigger=doc/${pkgname}.texi srctrigger=doc/${pkgname}.texi
# clear some things potentially inherited from environment. # clear some things potentially inherited from environment.
@ -69,10 +69,10 @@ while [ $# != 0 ] ; do
echo " --mandir=DIR man pages directory [${mandir}]" echo " --mandir=DIR man pages directory [${mandir}]"
echo " --sysconfdir=DIR read-only single-machine data directory [${sysconfdir}]" echo " --sysconfdir=DIR read-only single-machine data directory [${sysconfdir}]"
echo " CXX=COMPILER C++ compiler to use [${CXX}]" echo " CXX=COMPILER C++ compiler to use [${CXX}]"
echo " CPPFLAGS=OPTIONS command line options for the preprocessor [${CPPFLAGS}]" echo " CPPFLAGS=OPTIONS command-line options for the preprocessor [${CPPFLAGS}]"
echo " CXXFLAGS=OPTIONS command line options for the C++ compiler [${CXXFLAGS}]" echo " CXXFLAGS=OPTIONS command-line options for the C++ compiler [${CXXFLAGS}]"
echo " CXXFLAGS+=OPTIONS append options to the current value of CXXFLAGS" echo " CXXFLAGS+=OPTIONS append options to the current value of CXXFLAGS"
echo " LDFLAGS=OPTIONS command line options for the linker [${LDFLAGS}]" echo " LDFLAGS=OPTIONS command-line options for the linker [${LDFLAGS}]"
echo " MAKEINFO=NAME makeinfo program to use [${MAKEINFO}]" echo " MAKEINFO=NAME makeinfo program to use [${MAKEINFO}]"
echo " DIFF=NAME diff program to use with zdiff [${DIFF}]" echo " DIFF=NAME diff program to use with zdiff [${DIFF}]"
echo " GREP=NAME grep program to use with zgrep [${GREP}]" echo " GREP=NAME grep program to use with zgrep [${GREP}]"
@ -159,7 +159,7 @@ if [ -z "${no_create}" ] ; then
# This script is free software: you have unlimited permission # This script is free software: you have unlimited permission
# to copy, distribute, and modify it. # to copy, distribute, and modify it.
exec /bin/sh $0 ${args} --no-create exec /bin/sh "$0" ${args} --no-create
EOF EOF
chmod +x config.status chmod +x config.status
fi fi

View file

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.16. .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.2.
.TH ZCAT "1" "January 2023" "zutils 1.12" "User Commands" .TH ZCAT "1" "December 2023" "zutils 1.13-rc1" "User Commands"
.SH NAME .SH NAME
zcat \- decompress and concatenate files to standard output zcat \- decompress and concatenate files to standard output
.SH SYNOPSIS .SH SYNOPSIS
@ -10,8 +10,8 @@ zcat copies each file argument to standard output in sequence. If any
file given is compressed, its decompressed content is copied. If a file file given is compressed, its decompressed content is copied. If a file
given does not exist, and its name does not end with one of the known given does not exist, and its name does not end with one of the known
extensions, zcat tries the compressed file names corresponding to the extensions, zcat tries the compressed file names corresponding to the
formats supported. If a file fails to decompress, zcat continues copying the formats supported until one is found. If a file fails to decompress, zcat
rest of the files. continues copying the rest of the files.
.PP .PP
If a file is specified as '\-', data are read from standard input, If a file is specified as '\-', data are read from standard input,
decompressed if needed, and sent to standard output. Data read from decompressed if needed, and sent to standard output. Data read from

View file

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.16. .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.2.
.TH ZCMP "1" "January 2023" "zutils 1.12" "User Commands" .TH ZCMP "1" "December 2023" "zutils 1.13-rc1" "User Commands"
.SH NAME .SH NAME
zcmp \- decompress and compare two files byte by byte zcmp \- decompress and compare two files byte by byte
.SH SYNOPSIS .SH SYNOPSIS
@ -15,15 +15,10 @@ files are decompressed on the fly; no temporary files are created.
The formats supported are bzip2, gzip, lzip, xz, and zstd. The formats supported are bzip2, gzip, lzip, xz, and zstd.
.PP .PP
zcmp compares file1 to file2. The standard input is used only if file1 or zcmp compares file1 to file2. The standard input is used only if file1 or
file2 refers to standard input. If file2 is omitted zcmp tries the file2 refers to standard input. If file2 is omitted zcmp tries to compare
following: file1 with the corresponding uncompressed file (if file1 is compressed), and
.IP then with the corresponding compressed files of the remaining formats until
\- If file1 is compressed, compares its decompressed contents with one is found.
the corresponding uncompressed file (the name of file1 with the
extension removed).
.IP
\- If file1 is uncompressed, compares it with the decompressed
contents of file1.[lz|bz2|gz|zst|xz] (the first one that is found).
.PP .PP
Exit status is 0 if inputs are identical, 1 if different, 2 if trouble. Exit status is 0 if inputs are identical, 1 if different, 2 if trouble.
.SH OPTIONS .SH OPTIONS

View file

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.16. .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.2.
.TH ZDIFF "1" "January 2023" "zutils 1.12" "User Commands" .TH ZDIFF "1" "December 2023" "zutils 1.13-rc1" "User Commands"
.SH NAME .SH NAME
zdiff \- decompress and compare two files line by line zdiff \- decompress and compare two files line by line
.SH SYNOPSIS .SH SYNOPSIS
@ -17,15 +17,10 @@ from diff refer to temporary file names instead of those specified.
The formats supported are bzip2, gzip, lzip, xz, and zstd. The formats supported are bzip2, gzip, lzip, xz, and zstd.
.PP .PP
zdiff compares file1 to file2. The standard input is used only if file1 or zdiff compares file1 to file2. The standard input is used only if file1 or
file2 refers to standard input. If file2 is omitted zdiff tries the file2 refers to standard input. If file2 is omitted zdiff tries to compare
following: file1 with the corresponding uncompressed file (if file1 is compressed), and
.IP then with the corresponding compressed files of the remaining formats until
\- If file1 is compressed, compares its decompressed contents with one is found.
the corresponding uncompressed file (the name of file1 with the
extension removed).
.IP
\- If file1 is uncompressed, compares it with the decompressed
contents of file1.[lz|bz2|gz|zst|xz] (the first one that is found).
.PP .PP
Exit status is 0 if inputs are identical, 1 if different, 2 if trouble. Exit status is 0 if inputs are identical, 1 if different, 2 if trouble.
Some options only work if the diff program used supports them. Some options only work if the diff program used supports them.
@ -59,7 +54,7 @@ try hard to find a smaller set of changes
ignore changes due to tab expansion ignore changes due to tab expansion
.TP .TP
\fB\-i\fR, \fB\-\-ignore\-case\fR \fB\-i\fR, \fB\-\-ignore\-case\fR
ignore case differences in file contents ignore case differences
.TP .TP
\fB\-M\fR, \fB\-\-format=\fR<list> \fB\-M\fR, \fB\-\-format=\fR<list>
process only the formats in <list> process only the formats in <list>

View file

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.16. .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.2.
.TH ZGREP "1" "January 2023" "zutils 1.12" "User Commands" .TH ZGREP "1" "December 2023" "zutils 1.13-rc1" "User Commands"
.SH NAME .SH NAME
zgrep \- search compressed files for a regular expression zgrep \- search compressed files for a regular expression
.SH SYNOPSIS .SH SYNOPSIS
@ -11,8 +11,8 @@ on any combination of compressed and uncompressed files. If any file
given is compressed, its decompressed content is used. If a file given given is compressed, its decompressed content is used. If a file given
does not exist, and its name does not end with one of the known does not exist, and its name does not end with one of the known
extensions, zgrep tries the compressed file names corresponding to the extensions, zgrep tries the compressed file names corresponding to the
formats supported. If a file fails to decompress, zgrep continues formats supported until one is found. If a file fails to decompress, zgrep
searching the rest of the files. continues searching the rest of the files.
.PP .PP
If a file is specified as '\-', data are read from standard input, If a file is specified as '\-', data are read from standard input,
decompressed if needed, and fed to grep. Data read from standard input decompressed if needed, and fed to grep. Data read from standard input

View file

@ -1,19 +1,19 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.16. .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.2.
.TH ZTEST "1" "January 2023" "zutils 1.12" "User Commands" .TH ZTEST "1" "December 2023" "zutils 1.13-rc1" "User Commands"
.SH NAME .SH NAME
ztest \- verify the integrity of compressed files ztest \- check the integrity of compressed files
.SH SYNOPSIS .SH SYNOPSIS
.B ztest .B ztest
[\fI\,options\/\fR] [\fI\,files\/\fR] [\fI\,options\/\fR] [\fI\,files\/\fR]
.SH DESCRIPTION .SH DESCRIPTION
ztest verifies the integrity of the compressed files specified. It ztest checks the integrity of the compressed files specified. It
also warns if an uncompressed file has a compressed file name extension, or also warns if an uncompressed file has a compressed file name extension, or
if a compressed file has a wrong compressed extension. Uncompressed files if a compressed file has a wrong compressed extension. Uncompressed files
are otherwise ignored. If a file is specified as '\-', the integrity of are otherwise ignored. If a file is specified as '\-', the integrity of
compressed data read from standard input is verified. Data read from compressed data read from standard input is checked. Data read from
standard input must be all in the same compressed format. If a file fails to standard input must be all in the same compressed format. If a file fails to
decompress, does not exist, can't be opened, or is a terminal, ztest decompress, does not exist, can't be opened, or is a terminal, ztest
continues verifying the rest of the files. A final diagnostic is shown at continues testing the rest of the files. A final diagnostic is shown at
verbosity level 1 or higher if any file fails the test when testing multiple verbosity level 1 or higher if any file fails the test when testing multiple
files. files.
.PP .PP
@ -22,15 +22,15 @@ working directory, and nonrecursive searches read standard input.
.PP .PP
The formats supported are bzip2, gzip, lzip, xz, and zstd. The formats supported are bzip2, gzip, lzip, xz, and zstd.
.PP .PP
Note that error detection in the xz format is broken. First, some xz Note that error detection in the xz format is broken. First, some xz files
files lack integrity information. Second, not all xz decompressors can lack integrity information. Second, not all xz decompressors can check the
verify the integrity of all xz files. Third, section 2.1.1.2 'Stream integrity of all xz files. Third, section 2.1.1.2 'Stream Flags' of the
Flags' of the xz format specification allows xz decompressors to produce xz format specification allows xz decompressors to produce garbage output
garbage output without issuing any warning. Therefore, xz files can't without issuing any warning. Therefore, xz files can't always be checked as
always be verified as reliably as files in the other formats can. reliably as files in the other formats can.
.PP .PP
Exit status is 0 if all compressed files verify OK, 1 if environmental Exit status is 0 if all compressed files check OK, 1 if environmental
problems (file not found, invalid command line options, I/O errors, etc), problems (file not found, invalid command\-line options, I/O errors, etc),
2 if any compressed file is corrupt or invalid, or if any file has an 2 if any compressed file is corrupt or invalid, or if any file has an
incorrect file name extension. incorrect file name extension.
.SH OPTIONS .SH OPTIONS

View file

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.16. .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.2.
.TH ZUPDATE "1" "January 2023" "zutils 1.12" "User Commands" .TH ZUPDATE "1" "December 2023" "zutils 1.13-rc1" "User Commands"
.SH NAME .SH NAME
zupdate \- recompress bzip2, gzip, xz, zstd files to lzip format zupdate \- recompress bzip2, gzip, xz, zstd files to lzip format
.SH SYNOPSIS .SH SYNOPSIS
@ -12,17 +12,17 @@ Only regular files with standard file name extensions are recompressed,
other files are ignored. Compressed files are decompressed and then other files are ignored. Compressed files are decompressed and then
recompressed on the fly; no temporary files are created. The lzip format recompressed on the fly; no temporary files are created. The lzip format
is chosen as destination because it is the most appropriate for is chosen as destination because it is the most appropriate for
long\-term data archiving. long\-term archiving.
.PP .PP
If no files are specified, recursive searches examine the current If no files are specified, recursive searches examine the current
working directory, and nonrecursive searches do nothing. working directory, and nonrecursive searches do nothing.
.PP .PP
If the lzip compressed version of a file already exists, the file is If the lzip\-compressed version of a file already exists, the file is skipped
skipped unless the option '\-\-force' is given. In this case, if the unless the option '\-\-force' is given. In this case, if the comparison with
comparison with the existing lzip version fails, an error is returned the existing lzip version fails, an error is returned and the original file
and the original file is not deleted. The operation of zupdate is meant is not deleted. The operation of zupdate is meant to be safe and not cause
to be safe and not cause any data loss. Therefore, existing lzip any data loss. Therefore, existing lzip\-compressed files are never
compressed files are never overwritten nor deleted. overwritten nor deleted.
.PP .PP
The names of the original files must have one of the following extensions: The names of the original files must have one of the following extensions:
.PP .PP
@ -33,7 +33,7 @@ The names of the original files must have one of the following extensions:
Exit status is 0 if all the compressed files were successfully recompressed Exit status is 0 if all the compressed files were successfully recompressed
(if needed), compared, and deleted (if requested). 1 if a non\-fatal error (if needed), compared, and deleted (if requested). 1 if a non\-fatal error
occurred (file not found or not regular, or has invalid format, or can't be occurred (file not found or not regular, or has invalid format, or can't be
deleted). 2 if a fatal error occurred (invalid command line options, deleted). 2 if a fatal error occurred (invalid command\-line options,
compressor can't be run, or comparison fails). compressor can't be run, or comparison fails).
.SH OPTIONS .SH OPTIONS
.TP .TP

View file

@ -11,7 +11,7 @@ File: zutils.info, Node: Top, Next: Introduction, Up: (dir)
Zutils Manual Zutils Manual
************* *************
This manual is for Zutils (version 1.12, 7 January 2023). This manual is for Zutils (version 1.13-rc1, 31 December 2023).
* Menu: * Menu:
@ -43,8 +43,8 @@ Zutils is a collection of utilities able to process any combination of
compressed and uncompressed files transparently. If any file given, compressed and uncompressed files transparently. If any file given,
including standard input, is compressed, its decompressed content is used. including standard input, is compressed, its decompressed content is used.
Compressed files are decompressed on the fly; no temporary files are Compressed files are decompressed on the fly; no temporary files are
created. Data format is detected by its magic bytes, not by the file name created. Data format is detected by its identifier string (magic bytes), not
extension. by the file name extension. Empty files are considered uncompressed.
These utilities are not wrapper scripts but safer and more efficient C++ These utilities are not wrapper scripts but safer and more efficient C++
programs. In particular the option '--recursive' is very efficient in those programs. In particular the option '--recursive' is very efficient in those
@ -60,6 +60,10 @@ is configurable at runtime.
shell scripts provided by GNU gzip. 'ztest' is unique to zutils. 'zupdate' shell scripts provided by GNU gzip. 'ztest' is unique to zutils. 'zupdate'
is similar to gzip's znew. is similar to gzip's znew.
When 'zcat', 'zcmp', 'zdiff', or 'zgrep' need to try compressed file
names, the search order is: lzip, gzip, bzip2, zstd, xz.
(FILE.[lz|gz|bz2|zst|xz]).
NOTE: Bzip2 and lzip provide well-defined values of exit status, which NOTE: Bzip2 and lzip provide well-defined values of exit status, which
makes them safe to use with zutils. Gzip and xz may return ambiguous warning makes them safe to use with zutils. Gzip and xz may return ambiguous warning
values, making them less reliable back ends for zutils. Zstd currently does values, making them less reliable back ends for zutils. Zstd currently does
@ -79,22 +83,6 @@ example GNU gzip) must be able to decompress .Z files.
have been compressed. Decompressed is used to refer to data which have have been compressed. Decompressed is used to refer to data which have
undergone the process of decompression. undergone the process of decompression.
Numbers given as arguments to options (positions, sizes) may be followed
by a multiplier and an optional 'B' for "byte".
Table of SI and binary prefixes (unit multipliers):
Prefix Value | Prefix Value
k kilobyte (10^3 = 1000) | Ki kibibyte (2^10 = 1024)
M megabyte (10^6) | Mi mebibyte (2^20)
G gigabyte (10^9) | Gi gibibyte (2^30)
T terabyte (10^12) | Ti tebibyte (2^40)
P petabyte (10^15) | Pi pebibyte (2^50)
E exabyte (10^18) | Ei exbibyte (2^60)
Z zettabyte (10^21) | Zi zebibyte (2^70)
Y yottabyte (10^24) | Yi yobibyte (2^80)
 
File: zutils.info, Node: Common options, Next: Configuration, Prev: Introduction, Up: Top File: zutils.info, Node: Common options, Next: Configuration, Prev: Introduction, Up: Top
@ -103,7 +91,8 @@ File: zutils.info, Node: Common options, Next: Configuration, Prev: Introduct
The following options: are available in all the utilities. Rather than The following options: are available in all the utilities. Rather than
writing identical descriptions for each of the programs, they are described writing identical descriptions for each of the programs, they are described
here. *Note Argument syntax: (arg_parser)Argument syntax. here. Remember to prepend './' to any file name beginning with a hyphen, or
use '--'. *Note Argument syntax: (arg_parser)Argument syntax.
'-h' '-h'
'--help' '--help'
@ -168,6 +157,24 @@ here. *Note Argument syntax: (arg_parser)Argument syntax.
otherwise. otherwise.
Numbers given as arguments to options may be expressed in decimal,
hexadecimal, or octal (using the same syntax as integer constants in C++),
and may be followed by a multiplier and an optional 'B' for "byte".
Table of SI and binary prefixes (unit multipliers):
Prefix Value | Prefix Value
k kilobyte (10^3 = 1000) | Ki kibibyte (2^10 = 1024)
M megabyte (10^6) | Mi mebibyte (2^20)
G gigabyte (10^9) | Gi gibibyte (2^30)
T terabyte (10^12) | Ti tebibyte (2^40)
P petabyte (10^15) | Pi pebibyte (2^50)
E exabyte (10^18) | Ei exbibyte (2^60)
Z zettabyte (10^21) | Zi zebibyte (2^70)
Y yottabyte (10^24) | Yi yobibyte (2^80)
R ronnabyte (10^27) | Ri robibyte (2^90)
Q quettabyte (10^30) | Qi quebibyte (2^100)
 
File: zutils.info, Node: Configuration, Next: Zcat, Prev: Common options, Up: Top File: zutils.info, Node: Configuration, Next: Zcat, Prev: Common options, Up: Top
@ -204,8 +211,8 @@ File: zutils.info, Node: Zcat, Next: Zcmp, Prev: Configuration, Up: Top
file given is compressed, its decompressed content is copied. If a file file given is compressed, its decompressed content is copied. If a file
given does not exist, and its name does not end with one of the known given does not exist, and its name does not end with one of the known
extensions, 'zcat' tries the compressed file names corresponding to the extensions, 'zcat' tries the compressed file names corresponding to the
formats supported. If a file fails to decompress, 'zcat' continues copying formats supported until one is found. *Note search-order::. If a file fails
the rest of the files. to decompress, 'zcat' continues copying the rest of the files.
If a file is specified as '-', data are read from standard input, If a file is specified as '-', data are read from standard input,
decompressed if needed, and sent to standard output. Data read from decompressed if needed, and sent to standard output. Data read from
@ -248,8 +255,8 @@ Exit status is 0 if no errors occurred, 1 otherwise.
Force the compressed format given. Valid values for FORMAT are 'bz2', Force the compressed format given. Valid values for FORMAT are 'bz2',
'gz', 'lz', 'xz', 'zst', and 'un' for 'uncompressed'. If this option 'gz', 'lz', 'xz', 'zst', and 'un' for 'uncompressed'. If this option
is used, the files are passed to the corresponding decompressor (or is used, the files are passed to the corresponding decompressor (or
transmitted unmodified) without verifying their format, and the exact transmitted unmodified) without checking their format, and the exact
file name must be given. Other names won't be tried. file name must be given. Other names are not tried.
'-q' '-q'
'--quiet' '--quiet'
@ -306,15 +313,10 @@ are created.
zcmp [OPTIONS] FILE1 [FILE2] zcmp [OPTIONS] FILE1 [FILE2]
This compares FILE1 to FILE2. The standard input is used only if FILE1 or This compares FILE1 to FILE2. The standard input is used only if FILE1 or
FILE2 refers to standard input. If FILE2 is omitted 'zcmp' tries the FILE2 refers to standard input. If FILE2 is omitted 'zcmp' tries to compare
following: FILE1 with the corresponding uncompressed file (if FILE1 is compressed),
and then with the corresponding compressed files of the remaining formats
- If FILE1 is compressed, compares its decompressed contents with the until one is found. *Note search-order::.
corresponding uncompressed file (the name of FILE1 with the extension
removed).
- If FILE1 is uncompressed, compares it with the decompressed contents
of FILE1.[lz|bz2|gz|zst|xz] (the first one that is found).
An exit status of 0 means no differences were found, 1 means some An exit status of 0 means no differences were found, 1 means some
differences were found, and 2 means trouble. differences were found, and 2 means trouble.
@ -351,14 +353,13 @@ differences were found, and 2 means trouble.
'-O [FORMAT1][,FORMAT2]' '-O [FORMAT1][,FORMAT2]'
'--force-format=[FORMAT1][,FORMAT2]' '--force-format=[FORMAT1][,FORMAT2]'
Force the compressed formats given. Any of FORMAT1 or FORMAT2 may be Force the compressed formats given. If FORMAT1 or FORMAT2 is omitted,
omitted and the corresponding format will be automatically detected. the corresponding format is automatically detected. Valid values for
Valid values for FORMAT are 'bz2', 'gz', 'lz', 'xz', 'zst', and 'un' FORMAT are 'bz2', 'gz', 'lz', 'xz', 'zst', and 'un' for
for 'uncompressed'. If at least one format is specified with this 'uncompressed'. If at least one format is specified with this option,
option, the file is passed to the corresponding decompressor (or the file is passed to the corresponding decompressor (or transmitted
transmitted unmodified) without verifying its format, and the exact unmodified) without checking its format, and the exact file names of
file names of both FILE1 and FILE2 must be given. Other names won't be both FILE1 and FILE2 must be given. Other names are not tried.
tried.
'-q' '-q'
'--quiet' '--quiet'
@ -382,22 +383,6 @@ differences were found, and 2 means trouble.
the verbosity level. *Note version::. the verbosity level. *Note version::.
Byte counts given as arguments to options may be expressed in decimal,
hexadecimal, or octal (using the same syntax as integer constants in C++),
and may be followed by a multiplier and an optional 'B' for "byte".
Table of SI and binary prefixes (unit multipliers):
Prefix Value | Prefix Value
k kilobyte (10^3 = 1000) | Ki kibibyte (2^10 = 1024)
M megabyte (10^6) | Mi mebibyte (2^20)
G gigabyte (10^9) | Gi gibibyte (2^30)
T terabyte (10^12) | Ti tebibyte (2^40)
P petabyte (10^15) | Pi pebibyte (2^50)
E exabyte (10^18) | Ei exbibyte (2^60)
Z zettabyte (10^21) | Zi zebibyte (2^70)
Y yottabyte (10^24) | Yi yobibyte (2^80)
 
File: zutils.info, Node: Zdiff, Next: Zgrep, Prev: Zcmp, Up: Top File: zutils.info, Node: Zdiff, Next: Zgrep, Prev: Zcmp, Up: Top
@ -416,15 +401,10 @@ specified.
zdiff [OPTIONS] FILE1 [FILE2] zdiff [OPTIONS] FILE1 [FILE2]
This compares FILE1 to FILE2. The standard input is used only if FILE1 or This compares FILE1 to FILE2. The standard input is used only if FILE1 or
FILE2 refers to standard input. If FILE2 is omitted 'zdiff' tries the FILE2 refers to standard input. If FILE2 is omitted 'zdiff' tries to
following: compare FILE1 with the corresponding uncompressed file (if FILE1 is
compressed), and then with the corresponding compressed files of the
- If FILE1 is compressed, compares its decompressed contents with the remaining formats until one is found. *Note search-order::.
corresponding uncompressed file (the name of FILE1 with the extension
removed).
- If FILE1 is uncompressed, compares it with the decompressed contents
of FILE1.[lz|bz2|gz|zst|xz] (the first one that is found).
An exit status of 0 means no differences were found, 1 means some An exit status of 0 means no differences were found, 1 means some
differences were found, and 2 means trouble. differences were found, and 2 means trouble.
@ -461,18 +441,18 @@ diff program used supports them):
'-i' '-i'
'--ignore-case' '--ignore-case'
Ignore case differences in file contents. Ignore case differences. Consider uppercase and lowercase letters
equivalent.
'-O [FORMAT1][,FORMAT2]' '-O [FORMAT1][,FORMAT2]'
'--force-format=[FORMAT1][,FORMAT2]' '--force-format=[FORMAT1][,FORMAT2]'
Force the compressed formats given. Any of FORMAT1 or FORMAT2 may be Force the compressed formats given. If FORMAT1 or FORMAT2 is omitted,
omitted and the corresponding format will be automatically detected. the corresponding format is automatically detected. Valid values for
Valid values for FORMAT are 'bz2', 'gz', 'lz', 'xz', 'zst', and 'un' FORMAT are 'bz2', 'gz', 'lz', 'xz', 'zst', and 'un' for
for 'uncompressed'. If at least one format is specified with this 'uncompressed'. If at least one format is specified with this option,
option, the file is passed to the corresponding decompressor (or the file is passed to the corresponding decompressor (or transmitted
transmitted unmodified) without verifying its format, and the exact unmodified) without checking its format, and the exact file names of
file names of both FILE1 and FILE2 must be given. Other names won't be both FILE1 and FILE2 must be given. Other names are not tried.
tried.
'-p' '-p'
'--show-c-function' '--show-c-function'
@ -531,9 +511,9 @@ File: zutils.info, Node: Zgrep, Next: Ztest, Prev: Zdiff, Up: Top
on any combination of compressed and uncompressed files. If any file given on any combination of compressed and uncompressed files. If any file given
is compressed, its decompressed content is used. If a file given does not is compressed, its decompressed content is used. If a file given does not
exist, and its name does not end with one of the known extensions, 'zgrep' exist, and its name does not end with one of the known extensions, 'zgrep'
tries the compressed file names corresponding to the formats supported. If tries the compressed file names corresponding to the formats supported
a file fails to decompress, 'zgrep' continues searching the rest of the until one is found. *Note search-order::. If a file fails to decompress,
files. 'zgrep' continues searching the rest of the files.
If a file is specified as '-', data are read from standard input, If a file is specified as '-', data are read from standard input,
decompressed if needed, and fed to grep. Data read from standard input must decompressed if needed, and fed to grep. Data read from standard input must
@ -665,8 +645,8 @@ by 'zgrep' and not passed to grep):
Force the compressed format given. Valid values for FORMAT are 'bz2', Force the compressed format given. Valid values for FORMAT are 'bz2',
'gz', 'lz', 'xz', 'zst', and 'un' for 'uncompressed'. If this option 'gz', 'lz', 'xz', 'zst', and 'un' for 'uncompressed'. If this option
is used, the files are passed to the corresponding decompressor (or is used, the files are passed to the corresponding decompressor (or
transmitted unmodified) without verifying their format, and the exact transmitted unmodified) without checking their format, and the exact
file name must be given. Other names won't be tried. file name must be given. Other names are not tried.
'-P' '-P'
'--perl-regexp' '--perl-regexp'
@ -735,14 +715,14 @@ File: zutils.info, Node: Ztest, Next: Zupdate, Prev: Zgrep, Up: Top
8 Ztest 8 Ztest
******* *******
'ztest' verifies the integrity of the compressed files specified. It also 'ztest' checks the integrity of the compressed files specified. It also
warns if an uncompressed file has a compressed file name extension, or if a warns if an uncompressed file has a compressed file name extension, or if a
compressed file has a wrong compressed extension. Uncompressed files are compressed file has a wrong compressed extension. Uncompressed files are
otherwise ignored. If a file is specified as '-', the integrity of otherwise ignored. If a file is specified as '-', the integrity of
compressed data read from standard input is verified. Data read from compressed data read from standard input is checked. Data read from
standard input must be all in the same compressed format. If a file fails to standard input must be all in the same compressed format. If a file fails to
decompress, does not exist, can't be opened, or is a terminal, 'ztest' decompress, does not exist, can't be opened, or is a terminal, 'ztest'
continues verifying the rest of the files. A final diagnostic is shown at continues testing the rest of the files. A final diagnostic is shown at
verbosity level 1 or higher if any file fails the test when testing multiple verbosity level 1 or higher if any file fails the test when testing multiple
files. files.
@ -755,17 +735,17 @@ corresponding files are ignored.
Note that error detection in the xz format is broken. First, some xz Note that error detection in the xz format is broken. First, some xz
files lack integrity information. Second, not all xz decompressors can files lack integrity information. Second, not all xz decompressors can
verify the integrity of all xz files. Third, section 2.1.1.2 'Stream Flags' check the integrity of all xz files. Third, section 2.1.1.2 'Stream Flags'
of the xz format specification allows xz decompressors to produce garbage of the xz format specification allows xz decompressors to produce garbage
output without issuing any warning. Therefore, xz files can't always be output without issuing any warning. Therefore, xz files can't always be
verified as reliably as files in the other formats can. checked as reliably as files in the other formats can.
The format for running 'ztest' is: The format for running 'ztest' is:
ztest [OPTIONS] [FILES] ztest [OPTIONS] [FILES]
Exit status is 0 if all compressed files verify OK, 1 if environmental Exit status is 0 if all compressed files check OK, 1 if environmental
problems (file not found, invalid command line options, I/O errors, etc), 2 problems (file not found, invalid command-line options, I/O errors, etc), 2
if any compressed file is corrupt or invalid, or if any file has an if any compressed file is corrupt or invalid, or if any file has an
incorrect file name extension. incorrect file name extension.
@ -775,9 +755,9 @@ incorrect file name extension.
'--force-format=FORMAT' '--force-format=FORMAT'
Force the compressed format given. Valid values for FORMAT are 'bz2', Force the compressed format given. Valid values for FORMAT are 'bz2',
'gz', 'lz', 'xz', and 'zst'. If this option is used, the files are 'gz', 'lz', 'xz', and 'zst'. If this option is used, the files are
passed to the corresponding decompressor without verifying their passed to the corresponding decompressor without checking their
format, and any files in a format that the decompressor can't format, and any files in a format that the decompressor can't
understand will fail. understand fail the test.
'-q' '-q'
'--quiet' '--quiet'
@ -796,7 +776,7 @@ incorrect file name extension.
'-v' '-v'
'--verbose' '--verbose'
Verbose mode. Show the verify status for each file processed. Further Verbose mode. Show the check status for each file processed. Further
-v's increase the verbosity level. *Note version::. -v's increase the verbosity level. *Note version::.
@ -813,20 +793,20 @@ files are ignored. Compressed files are decompressed and then recompressed
on the fly; no temporary files are created. If an error happens while on the fly; no temporary files are created. If an error happens while
recompressing a file, 'zupdate' exits immediately without recompressing the recompressing a file, 'zupdate' exits immediately without recompressing the
rest of the files. The lzip format is chosen as destination because it is rest of the files. The lzip format is chosen as destination because it is
the most appropriate for long-term data archiving. the most appropriate for long-term archiving.
If no files are specified, recursive searches examine the current working If no files are specified, recursive searches examine the current working
directory, and nonrecursive searches do nothing. directory, and nonrecursive searches do nothing.
If the lzip compressed version of a file already exists, the file is If the lzip-compressed version of a file already exists, the file is
skipped unless the option '--force' is given. In this case, if the skipped unless the option '--force' is given. In this case, if the
comparison with the existing lzip version fails, an error is returned and comparison with the existing lzip version fails, an error is returned and
the original file is not deleted. The operation of 'zupdate' is meant to be the original file is not deleted. The operation of 'zupdate' is meant to be
safe and not cause any data loss. Therefore, existing lzip compressed files safe and not cause any data loss. Therefore, existing lzip-compressed files
are never overwritten nor deleted. are never overwritten nor deleted.
Combining the options '--force' and '--keep', as in Combining the options '--force' and '--keep', as in
'zupdate -f -k *.gz', verifies that there are no differences between each 'zupdate -f -k *.gz', checks that there are no differences between each
pair of files in a multiformat set of files. pair of files in a multiformat set of files.
The names of the original files must have one of the following The names of the original files must have one of the following
@ -854,7 +834,7 @@ permission bits S_ISUID and S_ISGID are cleared).
Exit status is 0 if all the compressed files were successfully recompressed Exit status is 0 if all the compressed files were successfully recompressed
(if needed), compared, and deleted (if requested). 1 if a non-fatal error (if needed), compared, and deleted (if requested). 1 if a non-fatal error
occurred (file not found or not regular, or has invalid format, or can't be occurred (file not found or not regular, or has invalid format, or can't be
deleted). 2 if a fatal error occurred (invalid command line options, deleted). 2 if a fatal error occurred (invalid command-line options,
compressor can't be run, or comparison fails). compressor can't be run, or comparison fails).
'zupdate' supports the following options: 'zupdate' supports the following options:
@ -883,7 +863,7 @@ compressor can't be run, or comparison fails).
'-f' '-f'
'--force' '--force'
Don't skip a file for which a lzip compressed version already exists. Don't skip a file for which a lzip-compressed version already exists.
'--force' compares the content of the input file with the content of '--force' compares the content of the input file with the content of
the existing lzip file and deletes the input file if both contents are the existing lzip file and deletes the input file if both contents are
identical. identical.
@ -984,20 +964,21 @@ Concept index
 
Tag Table: Tag Table:
Node: Top217 Node: Top217
Node: Introduction1151 Node: Introduction1157
Node: Common options3998 Ref: search-order2309
Ref: version4484 Node: Common options3466
Ref: compressor-requirements6435 Ref: version4032
Node: Configuration6830 Ref: compressor-requirements5983
Node: Zcat7863 Node: Configuration7372
Node: Zcmp10563 Node: Zcat8405
Node: Zdiff14820 Node: Zcmp11144
Node: Zgrep18003 Node: Zdiff14412
Node: Ztest24111 Node: Zgrep17495
Node: Zupdate26910 Node: Ztest23642
Ref: lz-compressor32325 Node: Zupdate26435
Node: Problems33026 Ref: lz-compressor31843
Node: Concept index33560 Node: Problems32544
Node: Concept index33078
 
End Tag Table End Tag Table

View file

@ -6,8 +6,8 @@
@finalout @finalout
@c %**end of header @c %**end of header
@set UPDATED 7 January 2023 @set UPDATED 31 December 2023
@set VERSION 1.12 @set VERSION 1.13-rc1
@dircategory Compression @dircategory Compression
@direntry @direntry
@ -66,8 +66,8 @@ is a collection of utilities able to process any combination of
compressed and uncompressed files transparently. If any file given, compressed and uncompressed files transparently. If any file given,
including standard input, is compressed, its decompressed content is used. including standard input, is compressed, its decompressed content is used.
Compressed files are decompressed on the fly; no temporary files are Compressed files are decompressed on the fly; no temporary files are
created. Data format is detected by its magic bytes, not by the file name created. Data format is detected by its identifier string (magic bytes), not
extension. by the file name extension. Empty files are considered uncompressed.
These utilities are not wrapper scripts but safer and more efficient C++ These utilities are not wrapper scripts but safer and more efficient C++
programs. In particular the option @option{--recursive} is very efficient in programs. In particular the option @option{--recursive} is very efficient in
@ -86,6 +86,11 @@ improved replacements for the shell scripts provided by GNU gzip.
@command{ztest} is unique to zutils. @command{zupdate} is similar to gzip's @command{ztest} is unique to zutils. @command{zupdate} is similar to gzip's
znew. znew.
@anchor{search-order}
When @command{zcat}, @command{zcmp}, @command{zdiff}, or @command{zgrep}
need to try compressed file names, the search order is: lzip, gzip, bzip2,
zstd, xz. (@var{file}.[lz|gz|bz2|zst|xz]).
NOTE: Bzip2 and lzip provide well-defined values of exit status, which makes NOTE: Bzip2 and lzip provide well-defined values of exit status, which makes
them safe to use with zutils. Gzip and xz may return ambiguous warning them safe to use with zutils. Gzip and xz may return ambiguous warning
values, making them less reliable back ends for zutils. Zstd currently does values, making them less reliable back ends for zutils. Zstd currently does
@ -106,24 +111,6 @@ LANGUAGE NOTE: Uncompressed = not compressed = plain data; it may never have
been compressed. Decompressed is used to refer to data which have undergone been compressed. Decompressed is used to refer to data which have undergone
the process of decompression. the process of decompression.
@sp 1
Numbers given as arguments to options (positions, sizes) may be followed
by a multiplier and an optional @samp{B} for "byte".
Table of SI and binary prefixes (unit multipliers):
@multitable {Prefix} {kilobyte (10^3 = 1000)} {|} {Prefix} {kibibyte (2^10 = 1024)}
@item Prefix @tab Value @tab | @tab Prefix @tab Value
@item k @tab kilobyte (10^3 = 1000) @tab | @tab Ki @tab kibibyte (2^10 = 1024)
@item M @tab megabyte (10^6) @tab | @tab Mi @tab mebibyte (2^20)
@item G @tab gigabyte (10^9) @tab | @tab Gi @tab gibibyte (2^30)
@item T @tab terabyte (10^12) @tab | @tab Ti @tab tebibyte (2^40)
@item P @tab petabyte (10^15) @tab | @tab Pi @tab pebibyte (2^50)
@item E @tab exabyte (10^18) @tab | @tab Ei @tab exbibyte (2^60)
@item Z @tab zettabyte (10^21) @tab | @tab Zi @tab zebibyte (2^70)
@item Y @tab yottabyte (10^24) @tab | @tab Yi @tab yobibyte (2^80)
@end multitable
@node Common options @node Common options
@chapter Common options @chapter Common options
@ -132,7 +119,8 @@ Table of SI and binary prefixes (unit multipliers):
The following The following
@uref{http://www.nongnu.org/arg-parser/manual/arg_parser_manual.html#Argument-syntax,,options}: @uref{http://www.nongnu.org/arg-parser/manual/arg_parser_manual.html#Argument-syntax,,options}:
are available in all the utilities. Rather than writing identical are available in all the utilities. Rather than writing identical
descriptions for each of the programs, they are described here. descriptions for each of the programs, they are described here. Remember to
prepend @file{./} to any file name beginning with a hyphen, or use @samp{--}.
@ifnothtml @ifnothtml
@xref{Argument syntax,,,arg_parser}. @xref{Argument syntax,,,arg_parser}.
@end ifnothtml @end ifnothtml
@ -209,6 +197,26 @@ It must return 0 if no errors occurred, and a non-zero value otherwise.
@end table @end table
Numbers given as arguments to options may be expressed in decimal,
hexadecimal, or octal (using the same syntax as integer constants in C++),
and may be followed by a multiplier and an optional @samp{B} for "byte".
Table of SI and binary prefixes (unit multipliers):
@multitable {Prefix} {kilobyte (10^3 = 1000)} {|} {Prefix} {kibibyte (2^10 = 1024)}
@item Prefix @tab Value @tab | @tab Prefix @tab Value
@item k @tab kilobyte (10^3 = 1000) @tab | @tab Ki @tab kibibyte (2^10 = 1024)
@item M @tab megabyte (10^6) @tab | @tab Mi @tab mebibyte (2^20)
@item G @tab gigabyte (10^9) @tab | @tab Gi @tab gibibyte (2^30)
@item T @tab terabyte (10^12) @tab | @tab Ti @tab tebibyte (2^40)
@item P @tab petabyte (10^15) @tab | @tab Pi @tab pebibyte (2^50)
@item E @tab exabyte (10^18) @tab | @tab Ei @tab exbibyte (2^60)
@item Z @tab zettabyte (10^21) @tab | @tab Zi @tab zebibyte (2^70)
@item Y @tab yottabyte (10^24) @tab | @tab Yi @tab yobibyte (2^80)
@item R @tab ronnabyte (10^27) @tab | @tab Ri @tab robibyte (2^90)
@item Q @tab quettabyte (10^30) @tab | @tab Qi @tab quebibyte (2^100)
@end multitable
@node Configuration @node Configuration
@chapter The configuration file 'zutils.conf' @chapter The configuration file 'zutils.conf'
@ -249,8 +257,9 @@ where <format> is one of @samp{bz2}, @samp{gz}, @samp{lz}, @samp{xz}, or
sequence. If any file given is compressed, its decompressed content is sequence. If any file given is compressed, its decompressed content is
copied. If a file given does not exist, and its name does not end with one copied. If a file given does not exist, and its name does not end with one
of the known extensions, @command{zcat} tries the compressed file names of the known extensions, @command{zcat} tries the compressed file names
corresponding to the formats supported. If a file fails to decompress, corresponding to the formats supported until one is found.
@command{zcat} continues copying the rest of the files. @xref{search-order}. If a file fails to decompress, @command{zcat} continues
copying the rest of the files.
If a file is specified as @samp{-}, data are read from standard input, If a file is specified as @samp{-}, data are read from standard input,
decompressed if needed, and sent to standard output. Data read from decompressed if needed, and sent to standard output. Data read from
@ -297,8 +306,8 @@ Number all output lines, starting with 1. The line count is unlimited.
Force the compressed format given. Valid values for @var{format} are Force the compressed format given. Valid values for @var{format} are
@samp{bz2}, @samp{gz}, @samp{lz}, @samp{xz}, @samp{zst}, and @samp{un} for @samp{bz2}, @samp{gz}, @samp{lz}, @samp{xz}, @samp{zst}, and @samp{un} for
@samp{uncompressed}. If this option is used, the files are passed to the @samp{uncompressed}. If this option is used, the files are passed to the
corresponding decompressor (or transmitted unmodified) without verifying corresponding decompressor (or transmitted unmodified) without checking
their format, and the exact file name must be given. Other names won't be their format, and the exact file name must be given. Other names are not
tried. tried.
@item -q @item -q
@ -360,17 +369,10 @@ zcmp [@var{options}] @var{file1} [@var{file2}]
@noindent @noindent
This compares @var{file1} to @var{file2}. The standard input is used only if This compares @var{file1} to @var{file2}. The standard input is used only if
@var{file1} or @var{file2} refers to standard input. If @var{file2} is @var{file1} or @var{file2} refers to standard input. If @var{file2} is
omitted @command{zcmp} tries the following: omitted @command{zcmp} tries to compare @var{file1} with the corresponding
uncompressed file (if @var{file1} is compressed), and then with the
@itemize - corresponding compressed files of the remaining formats until one is found.
@item @xref{search-order}.
If @var{file1} is compressed, compares its decompressed contents with
the corresponding uncompressed file (the name of @var{file1} with the
extension removed).
@item
If @var{file1} is uncompressed, compares it with the decompressed
contents of @var{file1}.[lz|bz2|gz|zst|xz] (the first one that is found).
@end itemize
@noindent @noindent
An exit status of 0 means no differences were found, 1 means some An exit status of 0 means no differences were found, 1 means some
@ -409,14 +411,14 @@ Compare at most @var{count} input bytes.
@item -O [@var{format1}][,@var{format2}] @item -O [@var{format1}][,@var{format2}]
@itemx --force-format=[@var{format1}][,@var{format2}] @itemx --force-format=[@var{format1}][,@var{format2}]
Force the compressed formats given. Any of @var{format1} or @var{format2} Force the compressed formats given. If @var{format1} or @var{format2} is
may be omitted and the corresponding format will be automatically detected. omitted, the corresponding format is automatically detected. Valid values
Valid values for @var{format} are @samp{bz2}, @samp{gz}, @samp{lz}, for @var{format} are @samp{bz2}, @samp{gz}, @samp{lz}, @samp{xz},
@samp{xz}, @samp{zst}, and @samp{un} for @samp{uncompressed}. If at least @samp{zst}, and @samp{un} for @samp{uncompressed}. If at least one format is
one format is specified with this option, the file is passed to the specified with this option, the file is passed to the corresponding
corresponding decompressor (or transmitted unmodified) without verifying its decompressor (or transmitted unmodified) without checking its format, and
format, and the exact file names of both @var{file1} and @var{file2} must be the exact file names of both @var{file1} and @var{file2} must be given.
given. Other names won't be tried. Other names are not tried.
@item -q @item -q
@itemx --quiet @itemx --quiet
@ -441,24 +443,6 @@ the verbosity level. @xref{version}.
@end table @end table
Byte counts given as arguments to options may be expressed in decimal,
hexadecimal, or octal (using the same syntax as integer constants in C++),
and may be followed by a multiplier and an optional @samp{B} for "byte".
Table of SI and binary prefixes (unit multipliers):
@multitable {Prefix} {kilobyte (10^3 = 1000)} {|} {Prefix} {kibibyte (2^10 = 1024)}
@item Prefix @tab Value @tab | @tab Prefix @tab Value
@item k @tab kilobyte (10^3 = 1000) @tab | @tab Ki @tab kibibyte (2^10 = 1024)
@item M @tab megabyte (10^6) @tab | @tab Mi @tab mebibyte (2^20)
@item G @tab gigabyte (10^9) @tab | @tab Gi @tab gibibyte (2^30)
@item T @tab terabyte (10^12) @tab | @tab Ti @tab tebibyte (2^40)
@item P @tab petabyte (10^15) @tab | @tab Pi @tab pebibyte (2^50)
@item E @tab exabyte (10^18) @tab | @tab Ei @tab exbibyte (2^60)
@item Z @tab zettabyte (10^21) @tab | @tab Zi @tab zebibyte (2^70)
@item Y @tab yottabyte (10^24) @tab | @tab Yi @tab yobibyte (2^80)
@end multitable
@node Zdiff @node Zdiff
@chapter Zdiff @chapter Zdiff
@ -480,17 +464,10 @@ zdiff [@var{options}] @var{file1} [@var{file2}]
@noindent @noindent
This compares @var{file1} to @var{file2}. The standard input is used only if This compares @var{file1} to @var{file2}. The standard input is used only if
@var{file1} or @var{file2} refers to standard input. If @var{file2} is @var{file1} or @var{file2} refers to standard input. If @var{file2} is
omitted @command{zdiff} tries the following: omitted @command{zdiff} tries to compare @var{file1} with the corresponding
uncompressed file (if @var{file1} is compressed), and then with the
@itemize - corresponding compressed files of the remaining formats until one is found.
@item @xref{search-order}.
If @var{file1} is compressed, compares its decompressed contents with
the corresponding uncompressed file (the name of @var{file1} with the
extension removed).
@item
If @var{file1} is uncompressed, compares it with the decompressed
contents of @var{file1}.[lz|bz2|gz|zst|xz] (the first one that is found).
@end itemize
@noindent @noindent
An exit status of 0 means no differences were found, 1 means some An exit status of 0 means no differences were found, 1 means some
@ -529,18 +506,18 @@ Ignore changes due to tab expansion.
@item -i @item -i
@itemx --ignore-case @itemx --ignore-case
Ignore case differences in file contents. Ignore case differences. Consider uppercase and lowercase letters equivalent.
@item -O [@var{format1}][,@var{format2}] @item -O [@var{format1}][,@var{format2}]
@itemx --force-format=[@var{format1}][,@var{format2}] @itemx --force-format=[@var{format1}][,@var{format2}]
Force the compressed formats given. Any of @var{format1} or @var{format2} Force the compressed formats given. If @var{format1} or @var{format2} is
may be omitted and the corresponding format will be automatically detected. omitted, the corresponding format is automatically detected. Valid values
Valid values for @var{format} are @samp{bz2}, @samp{gz}, @samp{lz}, for @var{format} are @samp{bz2}, @samp{gz}, @samp{lz}, @samp{xz},
@samp{xz}, @samp{zst}, and @samp{un} for @samp{uncompressed}. If at least @samp{zst}, and @samp{un} for @samp{uncompressed}. If at least one format is
one format is specified with this option, the file is passed to the specified with this option, the file is passed to the corresponding
corresponding decompressor (or transmitted unmodified) without verifying its decompressor (or transmitted unmodified) without checking its format, and
format, and the exact file names of both @var{file1} and @var{file2} must be the exact file names of both @var{file1} and @var{file2} must be given.
given. Other names won't be tried. Other names are not tried.
@item -p @item -p
@itemx --show-c-function @itemx --show-c-function
@ -599,13 +576,12 @@ search on any combination of compressed and uncompressed files. If any file
given is compressed, its decompressed content is used. If a file given does given is compressed, its decompressed content is used. If a file given does
not exist, and its name does not end with one of the known extensions, not exist, and its name does not end with one of the known extensions,
@command{zgrep} tries the compressed file names corresponding to the formats @command{zgrep} tries the compressed file names corresponding to the formats
supported. If a file fails to decompress, @command{zgrep} continues supported until one is found. @xref{search-order}. If a file fails to
searching the rest of the files. decompress, @command{zgrep} continues searching the rest of the files.
If a file is specified as @samp{-}, data are read from standard input, If a file is specified as @samp{-}, data are read from standard input,
decompressed if needed, and fed to grep. Data read from standard input decompressed if needed, and fed to grep. Data read from standard input must
must be of the same type; all uncompressed or all in the same be of the same type; all uncompressed or all in the same compressed format.
compressed format.
If no files are specified, recursive searches examine the current working If no files are specified, recursive searches examine the current working
directory, and nonrecursive searches read standard input. directory, and nonrecursive searches read standard input.
@ -738,8 +714,8 @@ Show only the part of matching lines that actually matches @var{pattern}.
Force the compressed format given. Valid values for @var{format} are Force the compressed format given. Valid values for @var{format} are
@samp{bz2}, @samp{gz}, @samp{lz}, @samp{xz}, @samp{zst}, and @samp{un} for @samp{bz2}, @samp{gz}, @samp{lz}, @samp{xz}, @samp{zst}, and @samp{un} for
@samp{uncompressed}. If this option is used, the files are passed to the @samp{uncompressed}. If this option is used, the files are passed to the
corresponding decompressor (or transmitted unmodified) without verifying corresponding decompressor (or transmitted unmodified) without checking
their format, and the exact file name must be given. Other names won't be their format, and the exact file name must be given. Other names are not
tried. tried.
@item -P @item -P
@ -809,14 +785,14 @@ unusual characters like newlines.
@chapter Ztest @chapter Ztest
@cindex ztest @cindex ztest
@command{ztest} verifies the integrity of the compressed files specified. It @command{ztest} checks the integrity of the compressed files specified. It
also warns if an uncompressed file has a compressed file name extension, or also warns if an uncompressed file has a compressed file name extension, or
if a compressed file has a wrong compressed extension. Uncompressed files if a compressed file has a wrong compressed extension. Uncompressed files
are otherwise ignored. If a file is specified as @samp{-}, the integrity of are otherwise ignored. If a file is specified as @samp{-}, the integrity of
compressed data read from standard input is verified. Data read from compressed data read from standard input is checked. Data read from
standard input must be all in the same compressed format. If a file fails to standard input must be all in the same compressed format. If a file fails to
decompress, does not exist, can't be opened, or is a terminal, @command{ztest} decompress, does not exist, can't be opened, or is a terminal, @command{ztest}
continues verifying the rest of the files. A final diagnostic is shown at continues testing the rest of the files. A final diagnostic is shown at
verbosity level 1 or higher if any file fails the test when testing multiple verbosity level 1 or higher if any file fails the test when testing multiple
files. files.
@ -827,14 +803,14 @@ Bzip2, gzip, and lzip are the primary formats. Xz and zstd are optional. If
the decompressor for the xz or zstd formats is not found, the corresponding the decompressor for the xz or zstd formats is not found, the corresponding
files are ignored. files are ignored.
Note that error detection in the xz format is broken. First, some xz Note that error detection in the xz format is broken. First, some xz files
files lack integrity information. Second, not all xz decompressors can lack integrity information. Second, not all xz decompressors can
@uref{http://www.nongnu.org/lzip/xz_inadequate.html#fragmented,,verify the integrity} @uref{http://www.nongnu.org/lzip/xz_inadequate.html#fragmented,,check the integrity}
of all xz files. Third, section 2.1.1.2 'Stream Flags' of the of all xz files. Third, section 2.1.1.2 'Stream Flags' of the
@uref{http://tukaani.org/xz/xz-file-format.txt,,xz format specification} @uref{http://tukaani.org/xz/xz-file-format.txt,,xz format specification}
allows xz decompressors to produce garbage output without issuing any allows xz decompressors to produce garbage output without issuing any
warning. Therefore, xz files can't always be verified as reliably as warning. Therefore, xz files can't always be checked as reliably as files in
files in the other formats can. the other formats can.
@c We can only hope that xz is soon abandoned. @c We can only hope that xz is soon abandoned.
The format for running @command{ztest} is: The format for running @command{ztest} is:
@ -844,8 +820,8 @@ ztest [@var{options}] [@var{files}]
@end example @end example
@noindent @noindent
Exit status is 0 if all compressed files verify OK, 1 if environmental Exit status is 0 if all compressed files check OK, 1 if environmental
problems (file not found, invalid command line options, I/O errors, etc), problems (file not found, invalid command-line options, I/O errors, etc),
2 if any compressed file is corrupt or invalid, or if any file has an 2 if any compressed file is corrupt or invalid, or if any file has an
incorrect file name extension. incorrect file name extension.
@ -857,8 +833,8 @@ incorrect file name extension.
Force the compressed format given. Valid values for @var{format} are Force the compressed format given. Valid values for @var{format} are
@samp{bz2}, @samp{gz}, @samp{lz}, @samp{xz}, and @samp{zst}. If this option @samp{bz2}, @samp{gz}, @samp{lz}, @samp{xz}, and @samp{zst}. If this option
is used, the files are passed to the corresponding decompressor without is used, the files are passed to the corresponding decompressor without
verifying their format, and any files in a format that the decompressor checking their format, and any files in a format that the decompressor can't
can't understand will fail. understand fail the test.
@item -q @item -q
@itemx --quiet @itemx --quiet
@ -877,7 +853,7 @@ recursively, following all symbolic links.
@item -v @item -v
@itemx --verbose @itemx --verbose
Verbose mode. Show the verify status for each file processed. Further -v's Verbose mode. Show the check status for each file processed. Further -v's
increase the verbosity level. @xref{version}. increase the verbosity level. @xref{version}.
@end table @end table
@ -894,21 +870,21 @@ recompressed, other files are ignored. Compressed files are decompressed and
then recompressed on the fly; no temporary files are created. If an error then recompressed on the fly; no temporary files are created. If an error
happens while recompressing a file, @command{zupdate} exits immediately happens while recompressing a file, @command{zupdate} exits immediately
without recompressing the rest of the files. The lzip format is chosen as without recompressing the rest of the files. The lzip format is chosen as
destination because it is the most appropriate for long-term data archiving. destination because it is the most appropriate for long-term archiving.
If no files are specified, recursive searches examine the current working If no files are specified, recursive searches examine the current working
directory, and nonrecursive searches do nothing. directory, and nonrecursive searches do nothing.
If the lzip compressed version of a file already exists, the file is skipped If the lzip-compressed version of a file already exists, the file is skipped
unless the option @option{--force} is given. In this case, if the comparison unless the option @option{--force} is given. In this case, if the comparison
with the existing lzip version fails, an error is returned and the original with the existing lzip version fails, an error is returned and the original
file is not deleted. The operation of @command{zupdate} is meant to be safe file is not deleted. The operation of @command{zupdate} is meant to be safe
and not cause any data loss. Therefore, existing lzip compressed files are and not cause any data loss. Therefore, existing lzip-compressed files are
never overwritten nor deleted. never overwritten nor deleted.
Combining the options @option{--force} and @option{--keep}, as in Combining the options @option{--force} and @option{--keep}, as in
@w{@samp{zupdate -f -k *.gz}}, verifies that there are no differences @w{@samp{zupdate -f -k *.gz}}, checks that there are no differences between
between each pair of files in a multiformat set of files. each pair of files in a multiformat set of files.
The names of the original files must have one of the following extensions:@* The names of the original files must have one of the following extensions:@*
@samp{.bz2}, @samp{.gz}, @samp{.xz}, @samp{.zst}, or @samp{.Z}, which are @samp{.bz2}, @samp{.gz}, @samp{.xz}, @samp{.zst}, or @samp{.Z}, which are
@ -938,7 +914,7 @@ zupdate [@var{options}] [@var{files}]
Exit status is 0 if all the compressed files were successfully recompressed Exit status is 0 if all the compressed files were successfully recompressed
(if needed), compared, and deleted (if requested). 1 if a non-fatal error (if needed), compared, and deleted (if requested). 1 if a non-fatal error
occurred (file not found or not regular, or has invalid format, or can't be occurred (file not found or not regular, or has invalid format, or can't be
deleted). 2 if a fatal error occurred (invalid command line options, deleted). 2 if a fatal error occurred (invalid command-line options,
compressor can't be run, or comparison fails). compressor can't be run, or comparison fails).
@command{zupdate} supports the following options: @command{zupdate} supports the following options:
@ -968,10 +944,10 @@ Expand combined file name extensions; recompress @samp{.tbz}, @samp{.tbz2},
@item -f @item -f
@itemx --force @itemx --force
Don't skip a file for which a lzip compressed version already exists. Don't skip a file for which a lzip-compressed version already exists.
@option{--force} compares the content of the input file with the content @option{--force} compares the content of the input file with the content of
of the existing lzip file and deletes the input file if both contents the existing lzip file and deletes the input file if both contents are
are identical. identical.
@item -i @item -i
@itemx --ignore-errors @itemx --ignore-errors

4
rc.cc
View file

@ -46,7 +46,7 @@ std::string compressor_names[num_formats] =
std::vector< std::string > compressor_args[num_formats]; std::vector< std::string > compressor_args[num_formats];
// vector of enabled formats plus [num_formats] for uncompressed. // vector of enabled formats plus [num_formats] for uncompressed.
// empty means all enabled. // empty or incomplete (size <= num_formats) means all enabled.
std::vector< bool > enabled_formats; std::vector< bool > enabled_formats;
const struct { const char * from; const char * to; int format_index; } const struct { const char * from; const char * to; int format_index; }
@ -292,7 +292,7 @@ int extension_format( const int eindex )
{ return ( eindex >= 0 ) ? known_extensions[eindex].format_index : -1; } { return ( eindex >= 0 ) ? known_extensions[eindex].format_index : -1; }
const char * extension_from( const int eindex ) const char * extension_from( const int eindex )
{ return known_extensions[eindex].from; } { return ( eindex >= 0 ) ? known_extensions[eindex].from : ""; }
const char * extension_to( const int eindex ) const char * extension_to( const int eindex )
{ return known_extensions[eindex].to; } { return known_extensions[eindex].to; }

4
rc.h
View file

@ -23,7 +23,7 @@ const char * const format_names[num_formats] =
const char * const simple_extensions[num_formats] = const char * const simple_extensions[num_formats] =
{ ".bz2", ".gz", ".lz", ".xz", ".zst" }; { ".bz2", ".gz", ".lz", ".xz", ".zst" };
const int format_order[num_formats] = const int format_order[num_formats] =
{ fmt_lz, fmt_bz2, fmt_gz, fmt_zst, fmt_xz }; // search order { fmt_lz, fmt_gz, fmt_bz2, fmt_zst, fmt_xz }; // search order
bool enabled_format( const int format_index ); // -1 == uncompressed bool enabled_format( const int format_index ); // -1 == uncompressed
void parse_format_list( const std::string & arg, const char * const pn ); void parse_format_list( const std::string & arg, const char * const pn );
@ -33,7 +33,7 @@ int parse_format_type( const std::string & arg, const char * const pn,
int extension_index( const std::string & name ); // -1 if unknown int extension_index( const std::string & name ); // -1 if unknown
int extension_format( const int eindex ); // -1 if uncompressed int extension_format( const int eindex ); // -1 if uncompressed
const char * extension_from( const int eindex ); const char * extension_from( const int eindex ); // -1 if uncompressed
const char * extension_to( const int eindex ); const char * extension_to( const int eindex );
// Return format_index, or -1 if uncompressed. // Return format_index, or -1 if uncompressed.

View file

@ -41,7 +41,7 @@ bool test_full_name( const std::string & full_name, const struct stat * stp,
{ loop = true; break; } { loop = true; break; }
} }
if( loop ) // full_name already visited or above tree if( loop ) // full_name already visited or above tree
show_file_error( full_name.c_str(), "warning: Recursive directory loop." ); show_file_error( full_name.c_str(), "warning: recursive directory loop." );
return !loop; // (link to) directory return !loop; // (link to) directory
} }

View file

@ -43,6 +43,8 @@ for i in ${compressors}; do
$i in || compressor_needed $i in || compressor_needed
printf "Hello World!\n" > hello || framework_failure printf "Hello World!\n" > hello || framework_failure
$i hello || compressor_needed $i hello || compressor_needed
touch zero || framework_failure
$i zero || compressor_needed
done done
cat "${testdir}"/test.txt > in || framework_failure cat "${testdir}"/test.txt > in || framework_failure
@ -56,24 +58,34 @@ cat in in in in in in > in6 || framework_failure
bad0_lz="${testdir}"/zero_bad_crc.lz bad0_lz="${testdir}"/zero_bad_crc.lz
bad0_gz="${testdir}"/zero_bad_crc.gz bad0_gz="${testdir}"/zero_bad_crc.gz
bad1_lz="${testdir}"/test_bad_crc.lz bad1_lz="${testdir}"/test_bad_crc.lz
touch empty empty.bz2 empty.gz empty.lz touch empty empty.bz2 empty.gz empty.lz || framework_failure
fail=0 fail=0
test_failed() { fail=1 ; printf " $1" ; [ -z "$2" ] || printf "($2)" ; } test_failed() { fail=1 ; printf " $1" ; [ -z "$2" ] || printf "($2)" ; }
printf "testing zcat-%s..." "$2" printf "testing zcat-%s..." "$2"
for i in ${extensions}; do for i in ${extensions}; do
"${ZCAT}" -N in.$i > copy || test_failed $LINENO $i "${ZCAT}" -N in.$i > out || test_failed $LINENO $i
cmp in copy || test_failed $LINENO $i cmp in out || test_failed $LINENO $i
"${ZCAT}" -N empty.$i in.$i > copy || test_failed $LINENO $i "${ZCAT}" -N empty in.$i > out || test_failed $LINENO $i
cmp in copy || test_failed $LINENO $i cmp in out || test_failed $LINENO $i
"${ZCAT}" -N --format=un in.$i > copy || test_failed $LINENO $i "${ZCAT}" -N empty.$i in.$i > out || test_failed $LINENO $i
cmp in copy || test_failed $LINENO $i cmp in out || test_failed $LINENO $i
"${ZCAT}" -N --force-format=$i in.$i > copy || test_failed $LINENO $i "${ZCAT}" -N in.$i empty > out || test_failed $LINENO $i
cmp in copy || test_failed $LINENO $i cmp in out || test_failed $LINENO $i
"${ZCAT}" -N in.$i | dd bs=1000 count=1 > copy 2> /dev/null || "${ZCAT}" -N in.$i empty.$i > out || test_failed $LINENO $i
cmp in out || test_failed $LINENO $i
"${ZCAT}" -N zero.$i in.$i > out || test_failed $LINENO $i
cmp in out || test_failed $LINENO $i
"${ZCAT}" -N in.$i zero.$i > out || test_failed $LINENO $i
cmp in out || test_failed $LINENO $i
"${ZCAT}" -N --format=un in.$i > out || test_failed $LINENO $i
cmp in out || test_failed $LINENO $i
"${ZCAT}" -N --force-format=$i in.$i > out || test_failed $LINENO $i
cmp in out || test_failed $LINENO $i
"${ZCAT}" -N in.$i | dd bs=1000 count=1 > out 2> /dev/null ||
test_failed $LINENO $i test_failed $LINENO $i
dd if=in bs=1000 count=1 2> /dev/null | cmp - copy || dd if=in bs=1000 count=1 2> /dev/null | cmp - out ||
test_failed $LINENO $i test_failed $LINENO $i
done done
@ -82,34 +94,34 @@ printf "LZIP\001-.............................." | "${ZCAT}" -N > /dev/null 2>&1
printf "LZIPxxxxxx" | "${ZCAT}" -N > /dev/null || test_failed $LINENO printf "LZIPxxxxxx" | "${ZCAT}" -N > /dev/null || test_failed $LINENO
printf "BZh9xxxxxx" | "${ZCAT}" -N > /dev/null || test_failed $LINENO printf "BZh9xxxxxx" | "${ZCAT}" -N > /dev/null || test_failed $LINENO
"${ZCAT}" -N -v -s "${testdir}"/zcat_vs.dat > /dev/null || test_failed $LINENO "${ZCAT}" -N -v -s "${testdir}"/zcat_vs.dat > /dev/null || test_failed $LINENO
"${ZCAT}" -N < in > copy || test_failed $LINENO "${ZCAT}" -N < in > out || test_failed $LINENO
cmp in copy || test_failed $LINENO cmp in out || test_failed $LINENO
"${ZCAT}" -N < in.gz > copy || test_failed $LINENO "${ZCAT}" -N < in.gz > out || test_failed $LINENO
cmp in copy || test_failed $LINENO cmp in out || test_failed $LINENO
"${ZCAT}" -N < in.bz2 > copy || test_failed $LINENO "${ZCAT}" -N < in.bz2 > out || test_failed $LINENO
cmp in copy || test_failed $LINENO cmp in out || test_failed $LINENO
"${ZCAT}" -N < in.lz > copy || test_failed $LINENO "${ZCAT}" -N < in.lz > out || test_failed $LINENO
cmp in copy || test_failed $LINENO cmp in out || test_failed $LINENO
"${ZCAT}" -N -O lz - - < in.lz > copy || test_failed $LINENO "${ZCAT}" -N -O lz - - < in.lz > out || test_failed $LINENO
cmp in copy || test_failed $LINENO cmp in out || test_failed $LINENO
"${ZCAT}" -N -O un in.lz | lzip -d > copy || test_failed $LINENO "${ZCAT}" -N -O un in.lz | lzip -d > out || test_failed $LINENO
cmp in copy || test_failed $LINENO cmp in out || test_failed $LINENO
"${ZCAT}" -N --lz='lzip -q' < in.lz > copy || test_failed $LINENO "${ZCAT}" -N --lz='lzip -q' < in.lz > out || test_failed $LINENO
cmp in copy || test_failed $LINENO cmp in out || test_failed $LINENO
"${ZCAT}" -N in > copy || test_failed $LINENO "${ZCAT}" -N in > out || test_failed $LINENO
cmp in copy || test_failed $LINENO cmp in out || test_failed $LINENO
"${ZCAT}" -N lz_only > copy || test_failed $LINENO "${ZCAT}" -N lz_only > out || test_failed $LINENO
cmp in copy || test_failed $LINENO cmp in out || test_failed $LINENO
cat in.lz in in in in | "${ZCAT}" -N > copy || test_failed $LINENO # tdata cat in.lz in in in in | "${ZCAT}" -N > out || test_failed $LINENO # tdata
cmp in copy || test_failed $LINENO cmp in out || test_failed $LINENO
"${ZCAT}" -N in in.gz in.bz2 in.lz -- -in- -in-.lz > copy || test_failed $LINENO "${ZCAT}" -N in in.gz in.bz2 in.lz -- -in- -in-.lz > out || test_failed $LINENO
cmp in6 copy || test_failed $LINENO cmp in6 out || test_failed $LINENO
"${ZCAT}" -Nq in in.gz in.bz2 in.lz "${bad0_lz}" -- -in- -in-.lz > copy "${ZCAT}" -Nq in in.gz in.bz2 in.lz "${bad0_lz}" -- -in- -in-.lz > out
[ $? = 1 ] || test_failed $LINENO [ $? = 1 ] || test_failed $LINENO
cmp in6 copy || test_failed $LINENO cmp in6 out || test_failed $LINENO
"${ZCAT}" -Nq "${bad1_lz}" -- -in-.lz in in.gz in.bz2 in.lz > copy "${ZCAT}" -Nq "${bad1_lz}" -- -in-.lz in in.gz in.bz2 in.lz > out
[ $? = 1 ] || test_failed $LINENO [ $? = 1 ] || test_failed $LINENO
cmp in6 copy || test_failed $LINENO cmp in6 out || test_failed $LINENO
"${ZCAT}" -N . || test_failed $LINENO "${ZCAT}" -N . || test_failed $LINENO
"${ZCAT}" -N -r . > /dev/null || test_failed $LINENO "${ZCAT}" -N -r . > /dev/null || test_failed $LINENO
"${ZCAT}" -N -r > /dev/null || test_failed $LINENO "${ZCAT}" -N -r > /dev/null || test_failed $LINENO
@ -151,6 +163,7 @@ for i in ${extensions}; do
"${ZCMP}" -N -i 1kB:1000 -n 500 in6 in.$i || test_failed $LINENO $i "${ZCMP}" -N -i 1kB:1000 -n 500 in6 in.$i || test_failed $LINENO $i
"${ZCMP}" -N -i 1KiB:1024 -n 50 in.$i in6 || test_failed $LINENO $i "${ZCMP}" -N -i 1KiB:1024 -n 50 in.$i in6 || test_failed $LINENO $i
"${ZCMP}" -N empty empty.$i || test_failed $LINENO $i "${ZCMP}" -N empty empty.$i || test_failed $LINENO $i
"${ZCMP}" -N empty zero.$i || test_failed $LINENO $i
done done
"${ZCMP}" -N -q in in6 "${ZCMP}" -N -q in in6
@ -214,10 +227,18 @@ cat in.lz | "${ZCMP}" -N -O un,un in.lz - || test_failed $LINENO
[ $? = 2 ] || test_failed $LINENO [ $? = 2 ] || test_failed $LINENO
"${ZCMP}" -N --bad-option in in 2> /dev/null "${ZCMP}" -N --bad-option in in 2> /dev/null
[ $? = 2 ] || test_failed $LINENO [ $? = 2 ] || test_failed $LINENO
cat in.gz > a.gz || framework_failure
cat in.lz > a.lz || framework_failure
"${ZCMP}" -N a.gz || test_failed $LINENO
"${ZCMP}" -N a.lz || test_failed $LINENO
printf "\ntesting zdiff-%s..." "$2" printf "\ntesting zdiff-%s..." "$2"
"${ZDIFF}" -N a.gz || test_failed $LINENO
"${ZDIFF}" -N a.lz || test_failed $LINENO
rm -f a.gz a.lz || framework_failure
for i in ${extensions}; do for i in ${extensions}; do
"${ZDIFF}" -N in.$i > /dev/null || test_failed $LINENO $i "${ZDIFF}" -N in.$i > /dev/null || test_failed $LINENO $i
"${ZDIFF}" -N in in.$i > /dev/null || test_failed $LINENO $i "${ZDIFF}" -N in in.$i > /dev/null || test_failed $LINENO $i
@ -227,11 +248,12 @@ for i in ${extensions}; do
"${ZDIFF}" -N in.$i in --force-format=$i, > /dev/null || "${ZDIFF}" -N in.$i in --force-format=$i, > /dev/null ||
test_failed $LINENO $i test_failed $LINENO $i
"${ZDIFF}" -N empty empty.$i > /dev/null || test_failed $LINENO $i "${ZDIFF}" -N empty empty.$i > /dev/null || test_failed $LINENO $i
"${ZDIFF}" -N empty zero.$i > /dev/null || test_failed $LINENO $i
done done
"${ZDIFF}" -N in in6 > /dev/null "${ZDIFF}" -N in in6 > /dev/null
[ $? = 1 ] || test_failed $LINENO [ $? = 1 ] || test_failed $LINENO
# GNU diff 3.0 returns 2 when binary files differ # GNU diff 3.0 returns 2 (instead of 1) when binary files differ
"${ZDIFF}" -N in.tar pin.tar4 > /dev/null && test_failed $LINENO "${ZDIFF}" -N in.tar pin.tar4 > /dev/null && test_failed $LINENO
"${ZDIFF}" -N - - || test_failed $LINENO "${ZDIFF}" -N - - || test_failed $LINENO
"${ZDIFF}" -N - 2> /dev/null "${ZDIFF}" -N - 2> /dev/null
@ -307,8 +329,10 @@ for i in ${extensions}; do
"${ZGREP}" -N --force-format=$i "GNU" in 2> /dev/null "${ZGREP}" -N --force-format=$i "GNU" in 2> /dev/null
[ $? = 2 ] || test_failed $LINENO $i [ $? = 2 ] || test_failed $LINENO $i
"${ZGREP}" -N "nx_pattern" empty.$i && test_failed $LINENO $i "${ZGREP}" -N "nx_pattern" empty.$i && test_failed $LINENO $i
"${ZGREP}" -N "nx_pattern" zero.$i && test_failed $LINENO $i
done done
"${ZGREP}" -N "nx_pattern" empty && test_failed $LINENO
"${ZGREP}" -N pin.tar4 -e "GNU" > /dev/null || test_failed $LINENO "${ZGREP}" -N pin.tar4 -e "GNU" > /dev/null || test_failed $LINENO
"${ZGREP}" -N "GNU" < pin.tar4 > /dev/null || test_failed $LINENO "${ZGREP}" -N "GNU" < pin.tar4 > /dev/null || test_failed $LINENO
"${ZGREP}" -N -r "GNU" . > /dev/null || test_failed $LINENO "${ZGREP}" -N -r "GNU" . > /dev/null || test_failed $LINENO
@ -379,7 +403,6 @@ rm -f empty.bz2 empty.gz empty.lz || framework_failure
"${ZTEST}" -N -R . || test_failed $LINENO "${ZTEST}" -N -R . || test_failed $LINENO
"${ZTEST}" -N -R || test_failed $LINENO "${ZTEST}" -N -R || test_failed $LINENO
"${ZTEST}" -N empty || test_failed $LINENO "${ZTEST}" -N empty || test_failed $LINENO
rm -f empty || framework_failure
# test wrong compressed extensions # test wrong compressed extensions
cat in.bz2 > in_bz2.gz || framework_failure cat in.bz2 > in_bz2.gz || framework_failure
@ -569,6 +592,13 @@ cat in.gz > 'name with spaces.gz' || framework_failure
"${ZCMP}" -N in 'name with spaces.lz' || test_failed $LINENO "${ZCMP}" -N in 'name with spaces.lz' || test_failed $LINENO
rm -f 'name with spaces.lz' || framework_failure rm -f 'name with spaces.lz' || framework_failure
cat zero.gz > z.gz || framework_failure
"${ZUPDATE}" -N -0 -q z.gz || test_failed $LINENO
[ ! -e z.gz ] || test_failed $LINENO
[ -e z.lz ] || test_failed $LINENO
"${ZCMP}" -N empty z.lz || test_failed $LINENO
rm -f empty z.lz || framework_failure
mkdir tmp2 mkdir tmp2
mkdir tmp2/tmp3 mkdir tmp2/tmp3
cat in.bz2 > tmp2/tmp3/a.bz2 || framework_failure cat in.bz2 > tmp2/tmp3/a.bz2 || framework_failure
@ -584,7 +614,7 @@ cat in.gz > tmp2/tmp3/a.gz || framework_failure
[ -e tmp2/tmp3/a.gz ] || test_failed $LINENO [ -e tmp2/tmp3/a.gz ] || test_failed $LINENO
[ -e ddir2/tmp3/a.lz ] || test_failed $LINENO [ -e ddir2/tmp3/a.lz ] || test_failed $LINENO
# test non-recursive to destdir # test non-recursive to destdir
"${ZUPDATE}" -N -0 -k --destdir=ddir3 tmp2/tmp3/a.gz || test_failed $LINENO "${ZUPDATE}" -N -0 -k --destdir=ddir3/// tmp2/tmp3/a.gz || test_failed $LINENO
[ -e tmp2/tmp3/a.bz2 ] || test_failed $LINENO [ -e tmp2/tmp3/a.bz2 ] || test_failed $LINENO
[ -e tmp2/tmp3/a.gz ] || test_failed $LINENO [ -e tmp2/tmp3/a.gz ] || test_failed $LINENO
[ -e ddir3/a.lz ] || test_failed $LINENO [ -e ddir3/a.lz ] || test_failed $LINENO

View file

@ -94,8 +94,8 @@ void show_help()
"file given is compressed, its decompressed content is copied. If a file\n" "file given is compressed, its decompressed content is copied. If a file\n"
"given does not exist, and its name does not end with one of the known\n" "given does not exist, and its name does not end with one of the known\n"
"extensions, zcat tries the compressed file names corresponding to the\n" "extensions, zcat tries the compressed file names corresponding to the\n"
"formats supported. If a file fails to decompress, zcat continues copying the\n" "formats supported until one is found. If a file fails to decompress, zcat\n"
"rest of the files.\n" "continues copying the rest of the files.\n"
"\nIf a file is specified as '-', data are read from standard input,\n" "\nIf a file is specified as '-', data are read from standard input,\n"
"decompressed if needed, and sent to standard output. Data read from\n" "decompressed if needed, and sent to standard output. Data read from\n"
"standard input must be of the same type; all uncompressed or all in the\n" "standard input must be of the same type; all uncompressed or all in the\n"
@ -345,7 +345,7 @@ int main( const int argc, const char * const argv[] )
case lz_opt: parse_compressor( arg, pn, fmt_lz, 1 ); break; case lz_opt: parse_compressor( arg, pn, fmt_lz, 1 ); break;
case xz_opt: parse_compressor( arg, pn, fmt_xz, 1 ); break; case xz_opt: parse_compressor( arg, pn, fmt_xz, 1 ); break;
case zst_opt: parse_compressor( arg, pn, fmt_zst, 1 ); break; case zst_opt: parse_compressor( arg, pn, fmt_zst, 1 ); break;
default : internal_error( "uncaught option." ); default: internal_error( "uncaught option." );
} }
} // end process options } // end process options

53
zcmp.cc
View file

@ -58,13 +58,10 @@ void show_help()
"\nThe formats supported are bzip2, gzip, lzip, xz, and zstd.\n" "\nThe formats supported are bzip2, gzip, lzip, xz, and zstd.\n"
"\nUsage: zcmp [options] file1 [file2]\n" "\nUsage: zcmp [options] file1 [file2]\n"
"\nzcmp compares file1 to file2. The standard input is used only if file1 or\n" "\nzcmp compares file1 to file2. The standard input is used only if file1 or\n"
"file2 refers to standard input. If file2 is omitted zcmp tries the\n" "file2 refers to standard input. If file2 is omitted zcmp tries to compare\n"
"following:\n" "file1 with the corresponding uncompressed file (if file1 is compressed), and\n"
"\n - If file1 is compressed, compares its decompressed contents with\n" "then with the corresponding compressed files of the remaining formats until\n"
" the corresponding uncompressed file (the name of file1 with the\n" "one is found.\n"
" extension removed).\n"
"\n - If file1 is uncompressed, compares it with the decompressed\n"
" contents of file1.[lz|bz2|gz|zst|xz] (the first one that is found).\n"
"\nExit status is 0 if inputs are identical, 1 if different, 2 if trouble.\n" "\nExit status is 0 if inputs are identical, 1 if different, 2 if trouble.\n"
"\nOptions:\n" "\nOptions:\n"
" -h, --help display this help and exit\n" " -h, --help display this help and exit\n"
@ -98,9 +95,9 @@ void show_help()
// separate numbers of 5 or more digits in groups of 3 digits using '_' // separate numbers of 5 or more digits in groups of 3 digits using '_'
const char * format_num3( long long num ) const char * format_num3( long long num )
{ {
const char * const si_prefix = "kMGTPEZY"; enum { buffers = 8, bufsize = 4 * sizeof num, n = 10 };
const char * const binary_prefix = "KMGTPEZY"; const char * const si_prefix = "kMGTPEZYRQ";
enum { buffers = 8, bufsize = 4 * sizeof num }; const char * const binary_prefix = "KMGTPEZYRQ";
static char buffer[buffers][bufsize]; // circle of static buffers for printf static char buffer[buffers][bufsize]; // circle of static buffers for printf
static int current = 0; static int current = 0;
@ -108,19 +105,22 @@ const char * format_num3( long long num )
char * p = buf + bufsize - 1; // fill the buffer backwards char * p = buf + bufsize - 1; // fill the buffer backwards
*p = 0; // terminator *p = 0; // terminator
const bool negative = num < 0; const bool negative = num < 0;
char prefix = 0; // try binary first, then si if( num > 1024 || num < -1024 )
for( int i = 0; i < 8 && num != 0 && ( num / 1024 ) * 1024 == num; ++i ) {
{ num /= 1024; prefix = binary_prefix[i]; } char prefix = 0; // try binary first, then si
if( prefix ) *(--p) = 'i'; for( int i = 0; i < n && num != 0 && num % 1024 == 0; ++i )
else { num /= 1024; prefix = binary_prefix[i]; }
for( int i = 0; i < 8 && num != 0 && ( num / 1000 ) * 1000 == num; ++i ) if( prefix ) *(--p) = 'i';
{ num /= 1000; prefix = si_prefix[i]; } else
if( prefix ) *(--p) = prefix; for( int i = 0; i < n && num != 0 && num % 1000 == 0; ++i )
{ num /= 1000; prefix = si_prefix[i]; }
if( prefix ) *(--p) = prefix;
}
const bool split = num >= 10000 || num <= -10000; const bool split = num >= 10000 || num <= -10000;
for( int i = 0; ; ) for( int i = 0; ; )
{ {
long long onum = num; num /= 10; const long long onum = num; num /= 10;
*(--p) = llabs( onum - ( 10 * num ) ) + '0'; if( num == 0 ) break; *(--p) = llabs( onum - ( 10 * num ) ) + '0'; if( num == 0 ) break;
if( split && ++i >= 3 ) { i = 0; *(--p) = '_'; } if( split && ++i >= 3 ) { i = 0; *(--p) = '_'; }
} }
@ -129,6 +129,7 @@ const char * format_num3( long long num )
} }
// Recognized formats: <num>k[B], <num>Ki[B], <num>[MGTPEZYRQ][i][B]
long long getnum( const char * const arg, const char * const option_name, long long getnum( const char * const arg, const char * const option_name,
const char ** const tailp = 0, const char ** const tailp = 0,
const long long llimit = 0, const long long llimit = 0,
@ -152,6 +153,8 @@ long long getnum( const char * const arg, const char * const option_name,
int exponent = -1; // -1 = bad multiplier int exponent = -1; // -1 = bad multiplier
switch( ch ) switch( ch )
{ {
case 'Q': exponent = 10; break;
case 'R': exponent = 9; break;
case 'Y': exponent = 8; break; case 'Y': exponent = 8; break;
case 'Z': exponent = 7; break; case 'Z': exponent = 7; break;
case 'E': exponent = 6; break; case 'E': exponent = 6; break;
@ -210,7 +213,7 @@ bool skip_ignore_initial( const long long ignore_initial, const int infd )
const int size = std::min( rest, (long long)buffer_size ); const int size = std::min( rest, (long long)buffer_size );
const int rd = readblock( infd, buffer, size ); const int rd = readblock( infd, buffer, size );
if( rd != size && errno ) return false; if( rd != size && errno ) return false;
if( rd < size ) break; if( rd < size ) break; // EOF
rest -= rd; rest -= rd;
} }
} }
@ -270,6 +273,7 @@ int cmp( const long long max_size, const int infd[2],
uint8_t * buffer[2]; uint8_t * buffer[2];
buffer[0] = buffer0; buffer[1] = buffer1; buffer[0] = buffer0; buffer[1] = buffer1;
int retval = 0; int retval = 0;
bool empty[2] = { true, true };
while( rest > 0 ) while( rest > 0 )
{ {
@ -282,6 +286,7 @@ int cmp( const long long max_size, const int infd[2],
if( rd[i] != size && errno ) if( rd[i] != size && errno )
{ show_file_error( filenames[i].c_str(), "Read error", errno ); { show_file_error( filenames[i].c_str(), "Read error", errno );
retval = 2; goto done; } retval = 2; goto done; }
if( rd[i] > 0 ) empty[i] = false;
} }
for( int i = 0; i < 2; ++i ) for( int i = 0; i < 2; ++i )
if( rd[i] < size ) finished[i] = true; if( rd[i] < size ) finished[i] = true;
@ -345,11 +350,13 @@ int cmp( const long long max_size, const int infd[2],
if( rd[0] != rd[1] ) if( rd[0] != rd[1] )
{ {
const int i = rd[1] < rd[0];
if( verbosity >= 0 ) if( verbosity >= 0 )
std::fprintf( stderr, list ? std::fprintf( stderr, empty[i] ?
"%s: EOF on %s which is empty\n" : list ?
"%s: EOF on %s after byte %llu\n" : "%s: EOF on %s after byte %llu\n" :
"%s: EOF on %s after byte %llu, in line %llu\n", "%s: EOF on %s after byte %llu, in line %llu\n",
program_name, filenames[rd[1]<rd[0]].c_str(), program_name, filenames[i].c_str(),
byte_number - 1, line_number ); byte_number - 1, line_number );
retval = 1; break; retval = 1; break;
} }
@ -434,7 +441,7 @@ int main( const int argc, const char * const argv[] )
case lz_opt: parse_compressor( sarg, pn, fmt_lz ); break; case lz_opt: parse_compressor( sarg, pn, fmt_lz ); break;
case xz_opt: parse_compressor( sarg, pn, fmt_xz ); break; case xz_opt: parse_compressor( sarg, pn, fmt_xz ); break;
case zst_opt: parse_compressor( sarg, pn, fmt_zst ); break; case zst_opt: parse_compressor( sarg, pn, fmt_zst ); break;
default : internal_error( "uncaught option." ); default: internal_error( "uncaught option." );
} }
} // end process options } // end process options

View file

@ -31,21 +31,26 @@ int open_instream( const std::string & input_filename )
int open_other_instream( std::string & name ) int open_other_instream( std::string & name )
{ {
const int eindex = extension_index( name ); const int eindex = extension_index( name ); // search extension
if( eindex >= 0 && enabled_format( -1 ) ) if( eindex >= 0 && enabled_format( -1 ) ) // open uncompressed version
{ // open uncompressed version {
name.resize( name.size() - std::strlen( extension_from( eindex ) ) ); std::string s( name, 0, name.size() - std::strlen( extension_from( eindex ) ) );
name += extension_to( eindex ); s += extension_to( eindex );
return open( name.c_str(), O_RDONLY | O_BINARY ); const int infd = open( s.c_str(), O_RDONLY | O_BINARY );
if( infd >= 0 ) { name = s; return infd; }
}
const int eformat = extension_format( eindex );
for( int i = 0; i < num_formats; ++i ) // search compressed version
{
const int format_index = format_order[i];
if( eformat != format_index && enabled_format( format_index ) )
{
std::string s( name, 0, name.size() - std::strlen( extension_from( eindex ) ) );
s += simple_extensions[format_index];
const int infd = open( s.c_str(), O_RDONLY | O_BINARY );
if( infd >= 0 ) { name = s; return infd; }
}
} }
if( eindex < 0 ) // search compressed version
for( int i = 0; i < num_formats; ++i )
if( enabled_format( format_order[i] ) )
{
const std::string s( name + simple_extensions[format_order[i]] );
const int infd = open( s.c_str(), O_RDONLY | O_BINARY );
if( infd >= 0 ) { name = s; return infd; }
}
return -1; return -1;
} }

View file

@ -57,13 +57,10 @@ void show_help()
"\nThe formats supported are bzip2, gzip, lzip, xz, and zstd.\n" "\nThe formats supported are bzip2, gzip, lzip, xz, and zstd.\n"
"\nUsage: zdiff [options] file1 [file2]\n" "\nUsage: zdiff [options] file1 [file2]\n"
"\nzdiff compares file1 to file2. The standard input is used only if file1 or\n" "\nzdiff compares file1 to file2. The standard input is used only if file1 or\n"
"file2 refers to standard input. If file2 is omitted zdiff tries the\n" "file2 refers to standard input. If file2 is omitted zdiff tries to compare\n"
"following:\n" "file1 with the corresponding uncompressed file (if file1 is compressed), and\n"
"\n - If file1 is compressed, compares its decompressed contents with\n" "then with the corresponding compressed files of the remaining formats until\n"
" the corresponding uncompressed file (the name of file1 with the\n" "one is found.\n"
" extension removed).\n"
"\n - If file1 is uncompressed, compares it with the decompressed\n"
" contents of file1.[lz|bz2|gz|zst|xz] (the first one that is found).\n"
"\nExit status is 0 if inputs are identical, 1 if different, 2 if trouble.\n" "\nExit status is 0 if inputs are identical, 1 if different, 2 if trouble.\n"
"Some options only work if the diff program used supports them.\n" "Some options only work if the diff program used supports them.\n"
"\nOptions:\n" "\nOptions:\n"
@ -76,7 +73,7 @@ void show_help()
" -C, --context=<n> same as -c but use <n> lines of context\n" " -C, --context=<n> same as -c but use <n> lines of context\n"
" -d, --minimal try hard to find a smaller set of changes\n" " -d, --minimal try hard to find a smaller set of changes\n"
" -E, --ignore-tab-expansion ignore changes due to tab expansion\n" " -E, --ignore-tab-expansion ignore changes due to tab expansion\n"
" -i, --ignore-case ignore case differences in file contents\n" " -i, --ignore-case ignore case differences\n"
" -M, --format=<list> process only the formats in <list>\n" " -M, --format=<list> process only the formats in <list>\n"
" -N, --no-rcfile don't read runtime configuration file\n" " -N, --no-rcfile don't read runtime configuration file\n"
" -O, --force-format=[<f1>][,<f2>] force one or both input formats\n" " -O, --force-format=[<f1>][,<f2>] force one or both input formats\n"
@ -344,7 +341,7 @@ int main( const int argc, const char * const argv[] )
case lz_opt: parse_compressor( sarg, pn, fmt_lz ); break; case lz_opt: parse_compressor( sarg, pn, fmt_lz ); break;
case xz_opt: parse_compressor( sarg, pn, fmt_xz ); break; case xz_opt: parse_compressor( sarg, pn, fmt_xz ); break;
case zst_opt: parse_compressor( sarg, pn, fmt_zst ); break; case zst_opt: parse_compressor( sarg, pn, fmt_zst ); break;
default : internal_error( "uncaught option." ); default: internal_error( "uncaught option." );
} }
} // end process options } // end process options

View file

@ -52,8 +52,8 @@ void show_help()
"given is compressed, its decompressed content is used. If a file given\n" "given is compressed, its decompressed content is used. If a file given\n"
"does not exist, and its name does not end with one of the known\n" "does not exist, and its name does not end with one of the known\n"
"extensions, zgrep tries the compressed file names corresponding to the\n" "extensions, zgrep tries the compressed file names corresponding to the\n"
"formats supported. If a file fails to decompress, zgrep continues\n" "formats supported until one is found. If a file fails to decompress, zgrep\n"
"searching the rest of the files.\n" "continues searching the rest of the files.\n"
"\nIf a file is specified as '-', data are read from standard input,\n" "\nIf a file is specified as '-', data are read from standard input,\n"
"decompressed if needed, and fed to grep. Data read from standard input\n" "decompressed if needed, and fed to grep. Data read from standard input\n"
"must be of the same type; all uncompressed or all in the same\n" "must be of the same type; all uncompressed or all in the same\n"
@ -338,7 +338,8 @@ int main( const int argc, const char * const argv[] )
case color_opt: color_option = "--color"; case color_opt: color_option = "--color";
if( !sarg.empty() ) { color_option += '='; color_option += sarg; } if( !sarg.empty() ) { color_option += '='; color_option += sarg; }
break; break;
case label_opt: label_option = sarg; label = arg; break; case label_opt: label_option = "--label="; label_option += sarg;
label = arg; break;
case linebuf_opt: grep_args.push_back( "--line-buffered" ); case linebuf_opt: grep_args.push_back( "--line-buffered" );
line_buffered = true; break; line_buffered = true; break;
case bz2_opt: parse_compressor( sarg, pn, fmt_bz2 ); break; case bz2_opt: parse_compressor( sarg, pn, fmt_bz2 ); break;
@ -346,14 +347,14 @@ int main( const int argc, const char * const argv[] )
case lz_opt: parse_compressor( sarg, pn, fmt_lz ); break; case lz_opt: parse_compressor( sarg, pn, fmt_lz ); break;
case xz_opt: parse_compressor( sarg, pn, fmt_xz ); break; case xz_opt: parse_compressor( sarg, pn, fmt_xz ); break;
case zst_opt: parse_compressor( sarg, pn, fmt_zst ); break; case zst_opt: parse_compressor( sarg, pn, fmt_zst ); break;
default : internal_error( "uncaught option." ); default: internal_error( "uncaught option." );
} }
} // end process options } // end process options
if( !color_option.empty() ) // push the last value set if( !color_option.empty() ) // push the last value set
grep_args.push_back( color_option.c_str() ); grep_args.push_back( color_option.c_str() );
if( !label_option.empty() ) // for "Binary file <label> matches" if( !label_option.empty() ) // for "Binary file <label> matches"
grep_args.push_back( label_option.insert( 0, "--label=" ).c_str() ); grep_args.push_back( label_option.c_str() );
#if defined __MSVCRT__ || defined __OS2__ #if defined __MSVCRT__ || defined __OS2__
setmode( STDIN_FILENO, O_BINARY ); setmode( STDIN_FILENO, O_BINARY );

View file

@ -1,4 +1,4 @@
/* Ztest - verify the integrity of compressed files /* Ztest - check the integrity of compressed files
Copyright (C) 2010-2023 Antonio Diaz Diaz. Copyright (C) 2010-2023 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
@ -50,28 +50,28 @@ namespace {
void show_help() void show_help()
{ {
std::printf( "ztest verifies the integrity of the compressed files specified. It\n" std::printf( "ztest checks the integrity of the compressed files specified. It\n"
"also warns if an uncompressed file has a compressed file name extension, or\n" "also warns if an uncompressed file has a compressed file name extension, or\n"
"if a compressed file has a wrong compressed extension. Uncompressed files\n" "if a compressed file has a wrong compressed extension. Uncompressed files\n"
"are otherwise ignored. If a file is specified as '-', the integrity of\n" "are otherwise ignored. If a file is specified as '-', the integrity of\n"
"compressed data read from standard input is verified. Data read from\n" "compressed data read from standard input is checked. Data read from\n"
"standard input must be all in the same compressed format. If a file fails to\n" "standard input must be all in the same compressed format. If a file fails to\n"
"decompress, does not exist, can't be opened, or is a terminal, ztest\n" "decompress, does not exist, can't be opened, or is a terminal, ztest\n"
"continues verifying the rest of the files. A final diagnostic is shown at\n" "continues testing the rest of the files. A final diagnostic is shown at\n"
"verbosity level 1 or higher if any file fails the test when testing multiple\n" "verbosity level 1 or higher if any file fails the test when testing multiple\n"
"files.\n" "files.\n"
"\nIf no files are specified, recursive searches examine the current\n" "\nIf no files are specified, recursive searches examine the current\n"
"working directory, and nonrecursive searches read standard input.\n" "working directory, and nonrecursive searches read standard input.\n"
"\nThe formats supported are bzip2, gzip, lzip, xz, and zstd.\n" "\nThe formats supported are bzip2, gzip, lzip, xz, and zstd.\n"
"\nNote that error detection in the xz format is broken. First, some xz\n" "\nNote that error detection in the xz format is broken. First, some xz files\n"
"files lack integrity information. Second, not all xz decompressors can\n" "lack integrity information. Second, not all xz decompressors can check the\n"
"verify the integrity of all xz files. Third, section 2.1.1.2 'Stream\n" "integrity of all xz files. Third, section 2.1.1.2 'Stream Flags' of the\n"
"Flags' of the xz format specification allows xz decompressors to produce\n" "xz format specification allows xz decompressors to produce garbage output\n"
"garbage output without issuing any warning. Therefore, xz files can't\n" "without issuing any warning. Therefore, xz files can't always be checked as\n"
"always be verified as reliably as files in the other formats can.\n" "reliably as files in the other formats can.\n"
"\nUsage: ztest [options] [files]\n" "\nUsage: ztest [options] [files]\n"
"\nExit status is 0 if all compressed files verify OK, 1 if environmental\n" "\nExit status is 0 if all compressed files check OK, 1 if environmental\n"
"problems (file not found, invalid command line options, I/O errors, etc),\n" "problems (file not found, invalid command-line options, I/O errors, etc),\n"
"2 if any compressed file is corrupt or invalid, or if any file has an\n" "2 if any compressed file is corrupt or invalid, or if any file has an\n"
"incorrect file name extension.\n" "incorrect file name extension.\n"
"\nOptions:\n" "\nOptions:\n"
@ -303,7 +303,7 @@ int main( const int argc, const char * const argv[] )
case lz_opt: parse_compressor( arg, pn, fmt_lz, 1 ); break; case lz_opt: parse_compressor( arg, pn, fmt_lz, 1 ); break;
case xz_opt: parse_compressor( arg, pn, fmt_xz, 1 ); break; case xz_opt: parse_compressor( arg, pn, fmt_xz, 1 ); break;
case zst_opt: parse_compressor( arg, pn, fmt_zst, 1 ); break; case zst_opt: parse_compressor( arg, pn, fmt_zst, 1 ); break;
default : internal_error( "uncaught option." ); default: internal_error( "uncaught option." );
} }
} // end process options } // end process options

View file

@ -57,15 +57,15 @@ void show_help()
"other files are ignored. Compressed files are decompressed and then\n" "other files are ignored. Compressed files are decompressed and then\n"
"recompressed on the fly; no temporary files are created. The lzip format\n" "recompressed on the fly; no temporary files are created. The lzip format\n"
"is chosen as destination because it is the most appropriate for\n" "is chosen as destination because it is the most appropriate for\n"
"long-term data archiving.\n" "long-term archiving.\n"
"\nIf no files are specified, recursive searches examine the current\n" "\nIf no files are specified, recursive searches examine the current\n"
"working directory, and nonrecursive searches do nothing.\n" "working directory, and nonrecursive searches do nothing.\n"
"\nIf the lzip compressed version of a file already exists, the file is\n" "\nIf the lzip-compressed version of a file already exists, the file is skipped\n"
"skipped unless the option '--force' is given. In this case, if the\n" "unless the option '--force' is given. In this case, if the comparison with\n"
"comparison with the existing lzip version fails, an error is returned\n" "the existing lzip version fails, an error is returned and the original file\n"
"and the original file is not deleted. The operation of zupdate is meant\n" "is not deleted. The operation of zupdate is meant to be safe and not cause\n"
"to be safe and not cause any data loss. Therefore, existing lzip\n" "any data loss. Therefore, existing lzip-compressed files are never\n"
"compressed files are never overwritten nor deleted.\n" "overwritten nor deleted.\n"
"\nThe names of the original files must have one of the following extensions:\n" "\nThe names of the original files must have one of the following extensions:\n"
"\n'.bz2', '.gz', '.xz', '.zst', or '.Z', which are recompressed to '.lz'.\n" "\n'.bz2', '.gz', '.xz', '.zst', or '.Z', which are recompressed to '.lz'.\n"
"\n'.tbz', '.tbz2', '.tgz', '.txz', or '.tzst', which are recompressed to '.tlz'.\n" "\n'.tbz', '.tbz2', '.tgz', '.txz', or '.tzst', which are recompressed to '.tlz'.\n"
@ -73,7 +73,7 @@ void show_help()
"\nExit status is 0 if all the compressed files were successfully recompressed\n" "\nExit status is 0 if all the compressed files were successfully recompressed\n"
"(if needed), compared, and deleted (if requested). 1 if a non-fatal error\n" "(if needed), compared, and deleted (if requested). 1 if a non-fatal error\n"
"occurred (file not found or not regular, or has invalid format, or can't be\n" "occurred (file not found or not regular, or has invalid format, or can't be\n"
"deleted). 2 if a fatal error occurred (invalid command line options,\n" "deleted). 2 if a fatal error occurred (invalid command-line options,\n"
"compressor can't be run, or comparison fails).\n" "compressor can't be run, or comparison fails).\n"
"\nOptions:\n" "\nOptions:\n"
" -h, --help display this help and exit\n" " -h, --help display this help and exit\n"
@ -120,28 +120,28 @@ void extract_srcdir_name( const std::string & name, std::string & srcdir )
bool make_dirs( const std::string & name ) bool make_dirs( const std::string & name )
{ {
static std::string cached_dirname; static std::string cached_dirname;
unsigned dirsize = name.size(); // size of dirname without last slash unsigned i = name.size();
while( i > 0 && name[i-1] != '/' ) --i; // remove last component
for( unsigned i = name.size(); i > 0; --i ) while( i > 0 && name[i-1] == '/' ) --i; // remove slash(es)
if( name[i-1] == '/' ) { dirsize = i - 1; break; } if( i == 0 ) return true; // dirname is '/' or empty
if( dirsize >= name.size() ) return true; // no dirname const unsigned dirsize = i; // size of dirname without trailing slash(es)
if( dirsize == 0 ) return true; // dirname is '/'
if( cached_dirname.size() == dirsize && if( cached_dirname.size() == dirsize &&
cached_dirname.compare( 0, dirsize, name ) == 0 ) return true; cached_dirname.compare( 0, dirsize, name ) == 0 ) return true;
for( unsigned i = 0; i < dirsize; ) for( i = 0; i < dirsize; )
{ {
while( i < dirsize && name[i] == '/' ) ++i; while( i < dirsize && name[i] == '/' ) ++i;
const unsigned first = i; const unsigned first = i;
while( i < dirsize && name[i] != '/' ) ++i; while( i < dirsize && name[i] != '/' ) ++i;
if( first < i ) if( first < i )
{ {
std::string partial( name, 0, i ); const std::string partial( name, 0, i );
const mode_t mode = S_IRWXU | S_IRGRP | S_IXGRP | S_IROTH | S_IXOTH;
struct stat st; struct stat st;
if( stat( partial.c_str(), &st ) == 0 ) if( stat( partial.c_str(), &st ) == 0 )
{ if( !S_ISDIR( st.st_mode ) ) return false; } { if( !S_ISDIR( st.st_mode ) ) { errno = ENOTDIR; return false; } }
else if( mkdir( partial.c_str(), S_IRWXU | S_IRGRP | S_IXGRP | S_IROTH | else if( mkdir( partial.c_str(), mode ) != 0 && errno != EEXIST )
S_IXOTH ) != 0 && errno != EEXIST ) return false; return false; // if EEXIST, another process created the dir
} }
} }
cached_dirname.assign( name, 0, dirsize ); cached_dirname.assign( name, 0, dirsize );
@ -168,7 +168,7 @@ void set_permissions( const char * const rname, const struct stat & in_stats )
{ {
bool warning = false; bool warning = false;
const mode_t mode = in_stats.st_mode; const mode_t mode = in_stats.st_mode;
// chown will in many cases return with EPERM, which can be safely ignored. // chown in many cases returns with EPERM, which can be safely ignored.
if( chown( rname, in_stats.st_uid, in_stats.st_gid ) == 0 ) if( chown( rname, in_stats.st_uid, in_stats.st_gid ) == 0 )
{ if( chmod( rname, mode ) != 0 ) warning = true; } { if( chmod( rname, mode ) != 0 ) warning = true; }
else else
@ -180,7 +180,8 @@ void set_permissions( const char * const rname, const struct stat & in_stats )
t.modtime = in_stats.st_mtime; t.modtime = in_stats.st_mtime;
if( utime( rname, &t ) != 0 ) warning = true; if( utime( rname, &t ) != 0 ) warning = true;
if( warning && verbosity >= 2 ) if( warning && verbosity >= 2 )
show_file_error( rname, "Can't change output file attributes.", errno ); show_file_error( rname,
"warning: can't change output file attributes", errno );
} }
@ -204,7 +205,7 @@ int zupdate_file( const std::string & name, const char * const lzip_name,
if( format_index == fmt_lz ) if( format_index == fmt_lz )
{ {
if( verbosity >= 2 ) if( verbosity >= 2 )
std::fprintf( stderr, "%s: Input file '%s' already has '%s' suffix.\n", std::fprintf( stderr, "%s: %s: Input file already has '%s' suffix.\n",
program_name, name.c_str(), extension_from( eindex ) ); program_name, name.c_str(), extension_from( eindex ) );
return 0; // ignore this file return 0; // ignore this file
} }
@ -227,8 +228,7 @@ int zupdate_file( const std::string & name, const char * const lzip_name,
if( !compressor_name ) if( !compressor_name )
{ {
if( verbosity >= 2 ) if( verbosity >= 2 )
std::fprintf( stderr, "%s: Unknown extension in file name '%s' -- ignored.\n", show_file_error( name.c_str(), "Unknown extension in file name -- ignored." );
program_name, name.c_str() );
return 0; // ignore this file return 0; // ignore this file
} }
@ -247,9 +247,7 @@ int zupdate_file( const std::string & name, const char * const lzip_name,
const bool lz_lz_exists = ( stat( rname2.c_str(), &st ) == 0 ); const bool lz_lz_exists = ( stat( rname2.c_str(), &st ) == 0 );
if( lz_exists && !force ) if( lz_exists && !force )
{ {
if( verbosity >= 0 ) show_file_error( rname.c_str(), "Output file already exists, skipping." );
std::fprintf( stderr, "%s: Output file '%s' already exists, skipping.\n",
program_name, rname.c_str() );
return -1; return -1;
} }
@ -283,8 +281,8 @@ int zupdate_file( const std::string & name, const char * const lzip_name,
if( verbosity >= 1 ) if( verbosity >= 1 )
std::fprintf( stderr, "Recompressing file '%s'\n", name.c_str() ); std::fprintf( stderr, "Recompressing file '%s'\n", name.c_str() );
if( destdir.size() && !make_dirs( rname ) ) if( destdir.size() && !make_dirs( rname ) )
{ show_file_error( rname.c_str(), "Error creating intermediate directory." ); { show_file_error( rname.c_str(),
return 2; } "Error creating intermediate directory", errno ); return 2; }
int fda[2]; // pipe between decompressor and compressor int fda[2]; // pipe between decompressor and compressor
if( pipe( fda ) < 0 ) if( pipe( fda ) < 0 )
{ show_error( "Can't create pipe", errno ); return 2; } { show_error( "Can't create pipe", errno ); return 2; }
@ -465,7 +463,7 @@ int main( const int argc, const char * const argv[] )
case lz_opt: parse_compressor( arg, pn, fmt_lz, 1 ); break; case lz_opt: parse_compressor( arg, pn, fmt_lz, 1 ); break;
case xz_opt: parse_compressor( arg, pn, fmt_xz, 1 ); break; case xz_opt: parse_compressor( arg, pn, fmt_xz, 1 ); break;
case zst_opt: parse_compressor( arg, pn, fmt_zst, 1 ); break; case zst_opt: parse_compressor( arg, pn, fmt_zst, 1 ); break;
default : internal_error( "uncaught option." ); default: internal_error( "uncaught option." );
} }
} // end process options } // end process options

View file

@ -244,7 +244,7 @@ bool set_data_feeder( const std::string & filename, int * const infdp,
} }
// Return format_index, or -1 if uncompressed. // Return format_index, or -1 if uncompressed or shorter than magic_buf_size.
// //
int test_format( const int infd, uint8_t magic_data[], int test_format( const int infd, uint8_t magic_data[],
int * const magic_sizep ) int * const magic_sizep )
@ -269,22 +269,24 @@ int test_format( const int infd, uint8_t magic_data[],
{ 0x28, 0xB5, 0x2F, 0xFD }; // 0xFD2FB528 LE { 0x28, 0xB5, 0x2F, 0xFD }; // 0xFD2FB528 LE
*magic_sizep = readblock( infd, magic_data, magic_buf_size ); *magic_sizep = readblock( infd, magic_data, magic_buf_size );
if( *magic_sizep == magic_buf_size ) // test formats in search order if( *magic_sizep < magic_buf_size )
{ { if( errno ) return -1; // read error
if( std::memcmp( magic_data, lzip_magic, lzip_magic_size ) == 0 && for( int i = *magic_sizep; i < magic_buf_size; ++i ) magic_data[i] = 0; }
isvalid_ds( magic_data[lzip_magic_size] ) ) // test formats in search order
return fmt_lz; if( std::memcmp( magic_data, lzip_magic, lzip_magic_size ) == 0 &&
if( std::memcmp( magic_data, bzip2_magic, bzip2_magic_size ) == 0 && isvalid_ds( magic_data[lzip_magic_size] ) )
magic_data[3] >= '1' && magic_data[3] <= '9' && return fmt_lz;
std::memcmp( magic_data + 4, "1AY&SY", 6 ) == 0 ) if( std::memcmp( magic_data, bzip2_magic, bzip2_magic_size ) == 0 &&
return fmt_bz2; magic_data[3] >= '1' && magic_data[3] <= '9' &&
if( std::memcmp( magic_data, gzip_magic, gzip_magic_size ) == 0 || ( std::memcmp( magic_data + 4, "1AY&SY", 6 ) == 0 ||
std::memcmp( magic_data, compress_magic, compress_magic_size ) == 0 ) std::memcmp( magic_data + 4, "\x17rE8P\x90", 6 ) == 0 ) )
return fmt_gz; return fmt_bz2;
if( std::memcmp( magic_data, zstd_magic, zstd_magic_size ) == 0 ) if( std::memcmp( magic_data, gzip_magic, gzip_magic_size ) == 0 ||
return fmt_zst; std::memcmp( magic_data, compress_magic, compress_magic_size ) == 0 )
if( std::memcmp( magic_data, xz_magic, xz_magic_size ) == 0 ) return fmt_gz;
return fmt_xz; if( std::memcmp( magic_data, zstd_magic, zstd_magic_size ) == 0 )
} return fmt_zst;
if( std::memcmp( magic_data, xz_magic, xz_magic_size ) == 0 )
return fmt_xz;
return -1; return -1;
} }

View file

@ -7,7 +7,7 @@
# XDG_CONFIG_HOME defaults to $HOME/.config # XDG_CONFIG_HOME defaults to $HOME/.config
# This file sets the compressor and options to be used for each format. # This file sets the compressor and options to be used for each format.
# The command line options override compressors specified in this file. # The command-line options override compressors specified in this file.
# Syntax: <format> = <compressor> [options] # Syntax: <format> = <compressor> [options]
# Uncomment each line you want to take effect. # Uncomment each line you want to take effect.