lzip/lziprecover
1
0
Fork 0
Code Issues Activity

Merging upstream version 1.25.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-21 11:33:36 +01:00
parent 09c9ea500b
commit ccc1759b5f
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
40 changed files with 207 additions and 245 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

26
ChangeLog
View file

@ -1,26 +1,18 @@
2024-11-18 Antonio Diaz Diaz <antonio@gnu.org> 2025-01-08 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.25-rc1 released. * Version 1.25 released.
* byte_repair.cc: Repair a nonzero first LZMA byte.
* Integrate options '--ignore-empty' and '--ignore-nonzero' into
'-i, --ignore-errors'.
* merge.cc (copy_file): Add name arguments, use 'show_file_error'.
* lziprecover.texi: New chapter 'Syntax of command-line arguments'.
* check.sh: Use 'cp' instead of 'cat'.
* testsuite: Add fox_nz.lz, fox6_b1nz.lz.
Remove fox6.lz, fox6_nz.lz, test_em.txt.lz, test_3m.txt.lz.md5.
2024-10-01 Antonio Diaz Diaz <antonio@gnu.org>
* Version 1.25-pre1 released.
* New options '-F, --fec', '-0' to '-9', '-b, --block-size', * New options '-F, --fec', '-0' to '-9', '-b, --block-size',
'--fec-file', '-r, --recursive', and '-R, --dereference-recursive'. '--fec-file', '-r, --recursive', and '-R, --dereference-recursive'.
* Change short name of option '--byte-repair' to '-B'. * Change short name of option '--byte-repair' to '-B'.
* New options '--ignore-empty' and '--ignore-nonzero'. * byte_repair.cc: Repair a nonzero first LZMA byte.
* Make '-i' ignore empty members and nonzero first LZMA byte.
* Rename option '--clear-marking' to '--nonzero-repair'. * Rename option '--clear-marking' to '--nonzero-repair'.
* Remove options '--empty-error' and '--marking-error'. * Remove options '--empty-error' and '--marking-error'.
* decoder.cc (decode_member): Remove support for Sync Flush marker. * decoder.cc (decode_member): Remove support for Sync Flush marker.
* testsuite: Require lzip/clzip. Add fox6_nz.lz. Remove fox6_mark.lz. * merge.cc (copy_file): Add name arguments, use 'show_file_error'.
* lziprecover.texi: New chapter 'Syntax of command-line arguments'.
* check.sh: Use 'cp' instead of 'cat'.
* testsuite: Require lzip/clzip. Change several test files.
2024-01-20 Antonio Diaz Diaz <antonio@gnu.org> 2024-01-20 Antonio Diaz Diaz <antonio@gnu.org>
@ -263,7 +255,7 @@
* unzcrash.cc: Test all 1-byte errors. * unzcrash.cc: Test all 1-byte errors.
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This file is a collection of facts, and thus it is not copyrightable, but just This file is a collection of facts, and thus it is not copyrightable, but just
in case, you have unlimited permission to copy, distribute, and modify it. in case, you have unlimited permission to copy, distribute, and modify it.

5
INSTALL
View file

@ -4,9 +4,10 @@ You will need a C++98 compiler with support for 'long long'.
(gcc 3.3.6 or newer is recommended). (gcc 3.3.6 or newer is recommended).
I use gcc 6.1.0 and 3.3.6, but the code should compile with any standards I use gcc 6.1.0 and 3.3.6, but the code should compile with any standards
compliant compiler. compliant compiler.
Gcc is available at http://gcc.gnu.org. Gcc is available at http://gcc.gnu.org
Lzip 1.16 (or clzip 1.6) or newer is required to run the tests. Lzip 1.16 (or clzip 1.6) or newer is required to run the tests.
Lzip is available at http://www.nongnu.org/lzip/lzip.html
Unzcrash needs a 'zcmp' program able to understand the format being tested. Unzcrash needs a 'zcmp' program able to understand the format being tested.
For example the zcmp provided by zutils. For example the zcmp provided by zutils.
@ -78,7 +79,7 @@ After running 'configure', you can run 'make' and 'make install' as
explained above. explained above.
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This file is free documentation: you have unlimited permission to copy, This file is free documentation: you have unlimited permission to copy,
distribute, and modify it. distribute, and modify it.

2
Makefile.in
View file

@ -76,7 +76,7 @@ $(VPATH)/doc/$(pkgname).info : $(VPATH)/doc/$(pkgname).texi
man : $(VPATH)/doc/$(progname).1 man : $(VPATH)/doc/$(progname).1
$(VPATH)/doc/$(progname).1 : $(progname) $(VPATH)/doc/$(progname).1 : $(progname)
help2man -n 'recovers data from damaged lzip files' -o $@ ./$(progname) help2man -n 'recovers data from damaged files' -o $@ ./$(progname)
Makefile : $(VPATH)/configure $(VPATH)/Makefile.in Makefile : $(VPATH)/configure $(VPATH)/Makefile.in
./config.status ./config.status

33
README
View file

@ -1,3 +1,5 @@
See the file INSTALL for compilation and installation instructions.
Description Description
Lziprecover is a data recovery tool and decompressor for files in the lzip Lziprecover is a data recovery tool and decompressor for files in the lzip
@ -13,25 +15,6 @@ decompresses the members containing the desired data.
Lziprecover is not a replacement for regular backups, but a last line of Lziprecover is not a replacement for regular backups, but a last line of
defense for the case where the backups are also damaged. defense for the case where the backups are also damaged.
The lzip file format is designed for data sharing and long-term archiving,
taking into account both data integrity and decoder availability:
* The lzip format provides very safe integrity checking and some data
recovery means. The program lziprecover can repair bit flip errors
(one of the most common forms of data corruption) in lzip files, and
provides data recovery capabilities, including error-checked merging
of damaged copies of a file.
* The lzip format is as simple as possible (but not simpler). The lzip
manual provides the source code of a simple decompressor along with a
detailed explanation of how it works, so that with the only help of the
lzip manual it would be possible for a digital archaeologist to extract
the data from a lzip file long after quantum computers eventually
render LZMA obsolete.
* Additionally the lzip reference implementation is copylefted, which
guarantees that it will remain free forever.
A nice feature of the lzip format is that a corrupt byte is easier to repair A nice feature of the lzip format is that a corrupt byte is easier to repair
the nearer it is from the beginning of the file. Therefore, with the help of the nearer it is from the beginning of the file. Therefore, with the help of
lziprecover, losing an entire archive just because of a corrupt byte near lziprecover, losing an entire archive just because of a corrupt byte near
@ -48,9 +31,10 @@ Lziprecover is able to recover or decompress files produced by any of the
compressors in the lzip family: lzip, plzip, minilzip/lzlib, clzip, and compressors in the lzip family: lzip, plzip, minilzip/lzlib, clzip, and
pdlzip. pdlzip.
If the cause of file corruption is a damaged medium, the combination GNU ddrescue provides data recovery capabilities which nicely complement
GNU ddrescue + lziprecover is the recommended option for recovering data those of lziprecover. If the cause of file corruption is a damaged medium,
from damaged lzip files. the combination GNU ddrescue + lziprecover is the recommended option for
recovering data from damaged files.
If a file is too damaged for lziprecover to repair it, all the recoverable If a file is too damaged for lziprecover to repair it, all the recoverable
data in all members of the file can be extracted with the command data in all members of the file can be extracted with the command
@ -69,8 +53,11 @@ robustness to decompression of corrupted data, inspired by unzcrash.c from
Julian Seward's bzip2. Type 'make unzcrash' in the lziprecover source Julian Seward's bzip2. Type 'make unzcrash' in the lziprecover source
directory to build it. Then try 'unzcrash --help'. directory to build it. Then try 'unzcrash --help'.
Lziprecover uses Arg_parser for command-line argument parsing:
http://www.nongnu.org/arg-parser/arg_parser.html
Copyright (C) 2009-2024 Antonio Diaz Diaz.
Copyright (C) 2009-2025 Antonio Diaz Diaz.
This file is free documentation: you have unlimited permission to copy, This file is free documentation: you have unlimited permission to copy,
distribute, and modify it. distribute, and modify it.

4
alone_to_lz.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by
@ -143,7 +143,7 @@ int alone_to_lz( const int infd, const Pretty_print & pp )
{ pp( "conversion failed" ); std::free( buffer ); return 2; } { pp( "conversion failed" ); std::free( buffer ); return 2; }
if( writeblock( outfd, buffer + offset, lzip_size ) != lzip_size ) if( writeblock( outfd, buffer + offset, lzip_size ) != lzip_size )
{ {
show_file_error( printable_name( output_filename, false ), write_error_msg, show_file_error( printable_name( output_filename, false ), wr_err_msg,
errno ); std::free( buffer ); return 1; errno ); std::free( buffer ); return 1;
} }
std::free( buffer ); std::free( buffer );

2
arg_parser.cc
View file

@ -1,5 +1,5 @@
/* Arg_parser - POSIX/GNU command-line argument parser. (C++ version) /* Arg_parser - POSIX/GNU command-line argument parser. (C++ version)
Copyright (C) 2006-2024 Antonio Diaz Diaz. Copyright (C) 2006-2025 Antonio Diaz Diaz.
This library is free software. Redistribution and use in source and This library is free software. Redistribution and use in source and
binary forms, with or without modification, are permitted provided binary forms, with or without modification, are permitted provided

2
arg_parser.h
View file

@ -1,5 +1,5 @@
/* Arg_parser - POSIX/GNU command-line argument parser. (C++ version) /* Arg_parser - POSIX/GNU command-line argument parser. (C++ version)
Copyright (C) 2006-2024 Antonio Diaz Diaz. Copyright (C) 2006-2025 Antonio Diaz Diaz.
This library is free software. Redistribution and use in source and This library is free software. Redistribution and use in source and
binary forms, with or without modification, are permitted provided binary forms, with or without modification, are permitted provided

4
byte_repair.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by
@ -277,7 +277,7 @@ int byte_repair( const std::string & input_filename,
seek_write( outfd, mbuffer + 6, 1, mpos + 6 ) != 1 ) || seek_write( outfd, mbuffer + 6, 1, mpos + 6 ) != 1 ) ||
seek_write( outfd, mbuffer + pos, 1, mpos + pos ) != 1 ) seek_write( outfd, mbuffer + pos, 1, mpos + pos ) != 1 )
{ show_file_error( printable_name( output_filename, false ), { show_file_error( printable_name( output_filename, false ),
write_error_msg, errno ); cleanup_and_fail( 1 ); } wr_err_msg, errno ); cleanup_and_fail( 1 ); }
} }
delete[] mbuffer; delete[] mbuffer;
if( pos == 0 ) if( pos == 0 )

2
common.h
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by

6
configure vendored
View file

@ -1,12 +1,12 @@
#! /bin/sh #! /bin/sh
# configure script for Lziprecover - Data recovery tool for the lzip format # configure script for Lziprecover - Data recovery tool for the lzip format
# Copyright (C) 2009-2024 Antonio Diaz Diaz. # Copyright (C) 2009-2025 Antonio Diaz Diaz.
# #
# This configure script is free software: you have unlimited permission # This configure script is free software: you have unlimited permission
# to copy, distribute, and modify it. # to copy, distribute, and modify it.
pkgname=lziprecover pkgname=lziprecover
pkgversion=1.25-rc1 pkgversion=1.25
progname=lziprecover progname=lziprecover
srctrigger=doc/${pkgname}.texi srctrigger=doc/${pkgname}.texi
@ -175,7 +175,7 @@ echo "MAKEINFO = ${MAKEINFO}"
rm -f Makefile rm -f Makefile
cat > Makefile << EOF cat > Makefile << EOF
# Makefile for Lziprecover - Data recovery tool for the lzip format # Makefile for Lziprecover - Data recovery tool for the lzip format
# Copyright (C) 2009-2024 Antonio Diaz Diaz. # Copyright (C) 2009-2025 Antonio Diaz Diaz.
# This file was generated automatically by configure. Don't edit. # This file was generated automatically by configure. Don't edit.
# #
# This Makefile is free software: you have unlimited permission # This Makefile is free software: you have unlimited permission

6
decoder.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by
@ -99,7 +99,7 @@ void LZ_decoder::flush_data()
const long long s = const long long s =
std::min( positive_diff( outend, sp ), (unsigned long long)size ) - i; std::min( positive_diff( outend, sp ), (unsigned long long)size ) - i;
if( s > 0 && writeblock( outfd, buffer + stream_pos + i, s ) != s ) if( s > 0 && writeblock( outfd, buffer + stream_pos + i, s ) != s )
throw Error( write_error_msg ); throw Error( wr_err_msg );
} }
if( pos >= dictionary_size ) if( pos >= dictionary_size )
{ partial_data_pos += pos; pos = 0; pos_wrapped = true; } { partial_data_pos += pos; pos = 0; pos_wrapped = true; }
@ -222,7 +222,7 @@ int LZ_decoder::decode_member( const Pretty_print & pp,
if( rdec.decode_bit( bm_rep0[state()] ) == 0 ) // 3rd bit if( rdec.decode_bit( bm_rep0[state()] ) == 0 ) // 3rd bit
{ {
if( rdec.decode_bit( bm_len[state()][pos_state] ) == 0 ) // 4th bit if( rdec.decode_bit( bm_len[state()][pos_state] ) == 0 ) // 4th bit
{ state.set_short_rep(); put_byte( peek( rep0 ) ); continue; } { state.set_shortrep(); put_byte( peek( rep0 ) ); continue; }
} }
else else
{ {

2
decoder.h
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by

6
doc/lziprecover.1
View file

@ -1,7 +1,7 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.2. .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.2.
.TH LZIPRECOVER "1" "November 2024" "lziprecover 1.25-rc1" "User Commands" .TH LZIPRECOVER "1" "January 2025" "lziprecover 1.25" "User Commands"
.SH NAME .SH NAME
lziprecover \- recovers data from damaged lzip files lziprecover \- recovers data from damaged files
.SH SYNOPSIS .SH SYNOPSIS
.B lziprecover .B lziprecover
[\fI\,options\/\fR] [\fI\,files\/\fR] [\fI\,options\/\fR] [\fI\,files\/\fR]
@ -144,7 +144,7 @@ Report bugs to lzip\-bug@nongnu.org
.br .br
Lziprecover home page: http://www.nongnu.org/lzip/lziprecover.html Lziprecover home page: http://www.nongnu.org/lzip/lziprecover.html
.SH COPYRIGHT .SH COPYRIGHT
Copyright \(co 2024 Antonio Diaz Diaz. Copyright \(co 2025 Antonio Diaz Diaz.
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html> License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>
.br .br
This is free software: you are free to change and redistribute it. This is free software: you are free to change and redistribute it.

145
doc/lziprecover.info
View file

@ -12,7 +12,7 @@ File: lziprecover.info, Node: Top, Next: Introduction, Up: (dir)
Lziprecover Manual Lziprecover Manual
****************** ******************
This manual is for Lziprecover (version 1.25-rc1, 18 November 2024). This manual is for Lziprecover (version 1.25, 8 January 2025).
* Menu: * Menu:
@ -34,7 +34,7 @@ This manual is for Lziprecover (version 1.25-rc1, 18 November 2024).
* Concept index:: Index of concepts * Concept index:: Index of concepts
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This manual is free documentation: you have unlimited permission to copy, This manual is free documentation: you have unlimited permission to copy,
distribute, and modify it. distribute, and modify it.
@ -69,31 +69,12 @@ the lzip format is extraordinarily safe. The simple and safe design of the
file format complements the embedded error detection provided by the LZMA file format complements the embedded error detection provided by the LZMA
data stream. Any distance larger than the dictionary size acts as a data stream. Any distance larger than the dictionary size acts as a
forbidden symbol, allowing the decompressor to detect the approximate forbidden symbol, allowing the decompressor to detect the approximate
position of errors, and leaving very little work for the check sequence position of errors, and leaving little work for the check sequence (CRC and
(CRC and data sizes) in the detection of errors. Lzip is usually able to data sizes) in the detection of errors. Lzip is usually able to detect all
detect all possible bit flips in the compressed data without resorting to possible bit flips in the compressed data without resorting to the check
the check sequence. It would be difficult to write an automatic recovery sequence. It would be difficult to write an automatic recovery tool like
tool like lziprecover for the gzip format. And, as far as I know, it has lziprecover for the gzip format. And, as far as I know, it has never been
never been written. written.
The lzip file format is designed for data sharing and long-term
archiving, taking into account both data integrity and decoder availability:
* The lzip format provides very safe integrity checking and some data
recovery means. The program lziprecover can repair bit flip errors
(one of the most common forms of data corruption) in lzip files, and
provides data recovery capabilities, including error-checked merging
of damaged copies of a file. *Note Data safety::.
* The lzip format is as simple as possible (but not simpler). The lzip
manual provides the source code of a simple decompressor along with a
detailed explanation of how it works, so that with the only help of the
lzip manual it would be possible for a digital archaeologist to extract
the data from a lzip file long after quantum computers eventually
render LZMA obsolete.
* Additionally the lzip reference implementation is copylefted, which
guarantees that it will remain free forever.
A nice feature of the lzip format is that a corrupt byte is easier to A nice feature of the lzip format is that a corrupt byte is easier to
repair the nearer it is from the beginning of the file. Therefore, with the repair the nearer it is from the beginning of the file. Therefore, with the
@ -111,11 +92,12 @@ able to find and combine the good parts of several damaged copies.
compressors in the lzip family: lzip, plzip, minilzip/lzlib, clzip, and compressors in the lzip family: lzip, plzip, minilzip/lzlib, clzip, and
pdlzip. pdlzip.
If the cause of file corruption is a damaged medium, the combination GNU ddrescue provides data recovery capabilities which nicely complement
GNU ddrescue + lziprecover is the recommended option for recovering data those of lziprecover. If the cause of file corruption is a damaged medium,
from damaged files. *Note ddrescue-example::, *note ddrescue-example2::, and the combination GNU ddrescue + lziprecover is the recommended option for
*note ddrescue-example3::, for examples. *Note GNU ddrescue manual: recovering data from damaged files. *Note ddrescue-example::, *note
(ddrescue)Top, for details about ddrescue. ddrescue-example2::, and *note ddrescue-example3::, for examples. *Note GNU
ddrescue manual: (ddrescue)Top, for details about ddrescue.
If a file is too damaged for lziprecover to repair it, all the If a file is too damaged for lziprecover to repair it, all the
recoverable data in all members of the file can be extracted with the recoverable data in all members of the file can be extracted with the
@ -397,15 +379,15 @@ lziprecover supports the following options: *Note Argument syntax::.
When creating or reading fec files (but not when listing), for each When creating or reading fec files (but not when listing), for each
directory operand, read and process all files in that directory, directory operand, read and process all files in that directory,
recursively. Follow symbolic links given in the command line, but skip recursively. Follow symbolic links given in the command line, but skip
symbolic links that are encountered recursively. Ignore files with symbolic links that are encountered recursively. Ignore files and
extension '.fec', and files and directories named 'fec'. directories named 'fec' or '*[-._]fec'.
'-R' '-R'
'--dereference-recursive' '--dereference-recursive'
When creating or reading fec files (but not when listing), for each When creating or reading fec files (but not when listing), for each
directory operand, read and process all files in that directory, directory operand, read and process all files in that directory,
recursively, following all symbolic links. Ignore files with extension recursively, following all symbolic links. Ignore files and
'.fec', and files and directories named 'fec'. directories named 'fec' or '*[-._]fec'.
'-s' '-s'
'--split' '--split'
@ -691,9 +673,8 @@ POSIX recommends these conventions for command-line arguments.
* Certain options require an argument. * Certain options require an argument.
* An option and its argument may or may not appear as separate tokens. * An option and its argument may or may not appear as separate tokens.
(In other words, the whitespace separating them is optional, unless the (In other words, the whitespace separating them is optional). Thus,
argument is the empty string). Thus, '-o foo' and '-ofoo' are '-o foo' and '-ofoo' are equivalent.
equivalent.
* One or more options without arguments, followed by at most one option * One or more options without arguments, followed by at most one option
that takes an argument, may follow a hyphen in a single token. Thus, that takes an argument, may follow a hyphen in a single token. Thus,
@ -836,8 +817,8 @@ total device failure is storing backup copies in separate media.
The extraordinary safety of the lzip format allows lziprecover to use the The extraordinary safety of the lzip format allows lziprecover to use the
redundance that occurs naturally when making compressed backups. Lziprecover redundance that occurs naturally when making compressed backups. Lziprecover
can recover data that would not be recoverable from files compressed in can recover data that would not be recoverable from files compressed in
other formats. Let's see two examples of how much better is lzip compared other formats. See these two examples of the data recovery capabilities
with gzip and bzip2 with respect to data safety: offered by lziprecover:
* Menu: * Menu:
@ -1225,7 +1206,7 @@ payload_crc 36 + 4N 4
'fbs (coded fec_block_size)' 'fbs (coded fec_block_size)'
Number of FEC bytes per block. It is a multiple of 512 bytes between Number of FEC bytes per block. It is a multiple of 512 bytes between
512 bytes and 128 TiB. 512 bytes and 128 TiB. *Note fbs::.
'prodata_size' 'prodata_size'
Size of the protected file. 1 byte to 4 EiB. Size of the protected file. 1 byte to 4 EiB.
@ -1271,10 +1252,12 @@ payload_crc 12 + fbs 4
(0xB3, 0x46, 0x45, 0x43). (0xB3, 0x46, 0x45, 0x43).
'fbn (fec_block_number)' 'fbn (fec_block_number)'
Number of this FEC block. Required to compute the decode matrix. Number of this FEC block (0 to 32767). Required to compute the decode
matrix.
'fbs (coded fec_block_size)' 'fbs (coded fec_block_size)'
*Note fbs::. Number of FEC bytes per block. It is a multiple of 512 bytes between
512 bytes and 128 TiB. *Note fbs::.
'header_crc' 'header_crc'
CRC32 of the previous fields, including magic. CRC32 of the previous fields, including magic.
@ -2033,43 +2016,43 @@ Concept index
 
Tag Table: Tag Table:
Node: Top226 Node: Top226
Node: Introduction1535 Node: Introduction1529
Node: Invoking lziprecover6387 Node: Invoking lziprecover5451
Ref: --trailing-error7308 Ref: --trailing-error6372
Ref: --byte-repair8402 Ref: --byte-repair7466
Ref: range-format10483 Ref: range-format9547
Ref: --reproduce10728 Ref: --reproduce9792
Ref: --unzcrash28455 Ref: --unzcrash27483
Node: Argument syntax33048 Node: Argument syntax32076
Node: File format35005 Node: File format33987
Node: Data safety37759 Node: Data safety36741
Node: Merging with a backup40005 Node: Merging with a backup38960
Node: Reproducing a mailbox41268 Node: Reproducing a mailbox40223
Node: Fec files43722 Node: Fec files42677
Node: How Reed-Solomon works45988 Node: How Reed-Solomon works44943
Node: Implementation details48159 Node: Implementation details47114
Node: Creating fec files50224 Node: Creating fec files49179
Node: Testing with fec files51068 Node: Testing with fec files50023
Node: Repairing with fec files52023 Node: Repairing with fec files50978
Ref: ddrescue-example52841 Ref: ddrescue-example51796
Node: Fec file format53351 Node: Fec file format52306
Ref: fbs56222 Ref: fbs53073
Node: Repairing one byte58011 Node: Repairing one byte57094
Node: Merging files60103 Node: Merging files59186
Ref: performance-of-merge61282 Ref: performance-of-merge60365
Ref: ddrescue-example262890 Ref: ddrescue-example261973
Node: Reproducing one sector64106 Node: Reproducing one sector63189
Ref: performance-of-reproduce68043 Ref: performance-of-reproduce67126
Ref: ddrescue-example370716 Ref: ddrescue-example369799
Node: Tarlz73135 Node: Tarlz72218
Node: File names76808 Node: File names75891
Node: Trailing data77541 Node: Trailing data76624
Node: Examples80854 Node: Examples79937
Ref: concat-example81426 Ref: concat-example80509
Node: Unzcrash82825 Node: Unzcrash81908
Ref: --set-byte87437 Ref: --set-byte86520
Node: Problems89295 Node: Problems88378
Node: Concept index89847 Node: Concept index88930
 
End Tag Table End Tag Table

74
doc/lziprecover.texi
View file

@ -6,8 +6,8 @@
@finalout @finalout
@c %**end of header @c %**end of header
@set UPDATED 18 November 2024 @set UPDATED 8 January 2025
@set VERSION 1.25-rc1 @set VERSION 1.25
@dircategory Compression @dircategory Compression
@direntry @direntry
@ -55,7 +55,7 @@ This manual is for Lziprecover (version @value{VERSION}, @value{UPDATED}).
@end menu @end menu
@sp 1 @sp 1
Copyright @copyright{} 2009-2024 Antonio Diaz Diaz. Copyright @copyright{} 2009-2025 Antonio Diaz Diaz.
This manual is free documentation: you have unlimited permission to copy, This manual is free documentation: you have unlimited permission to copy,
distribute, and modify it. distribute, and modify it.
@ -91,36 +91,11 @@ lzip format is extraordinarily safe. The simple and safe design of the file
format complements the embedded error detection provided by the LZMA data format complements the embedded error detection provided by the LZMA data
stream. Any distance larger than the dictionary size acts as a forbidden stream. Any distance larger than the dictionary size acts as a forbidden
symbol, allowing the decompressor to detect the approximate position of symbol, allowing the decompressor to detect the approximate position of
errors, and leaving very little work for the check sequence (CRC and data errors, and leaving little work for the check sequence (CRC and data sizes)
sizes) in the detection of errors. Lzip is usually able to detect all in the detection of errors. Lzip is usually able to detect all possible bit
possible bit flips in the compressed data without resorting to the check flips in the compressed data without resorting to the check sequence. It
sequence. It would be difficult to write an automatic recovery tool like would be difficult to write an automatic recovery tool like lziprecover for
lziprecover for the gzip format. And, as far as I know, it has never been the gzip format. And, as far as I know, it has never been written.
written.
The lzip file format is designed for data sharing and long-term archiving,
taking into account both data integrity and decoder availability:
@itemize @bullet
@item
The lzip format provides very safe integrity checking and some data
recovery means. The program lziprecover can repair bit flip errors
(one of the most common forms of data corruption) in lzip files, and
provides data recovery capabilities, including error-checked merging
of damaged copies of a file. @xref{Data safety}.
@item
The lzip format is as simple as possible (but not simpler). The lzip
manual provides the source code of a simple decompressor along with a
detailed explanation of how it works, so that with the only help of the
lzip manual it would be possible for a digital archaeologist to extract
the data from a lzip file long after quantum computers eventually
render LZMA obsolete.
@item
Additionally the lzip reference implementation is copylefted, which
guarantees that it will remain free forever.
@end itemize
A nice feature of the lzip format is that a corrupt byte is easier to repair A nice feature of the lzip format is that a corrupt byte is easier to repair
the nearer it is from the beginning of the file. Therefore, with the help of the nearer it is from the beginning of the file. Therefore, with the help of
@ -138,10 +113,11 @@ Lziprecover is able to recover or decompress files produced by any of the
compressors in the lzip family: lzip, plzip, minilzip/lzlib, clzip, and compressors in the lzip family: lzip, plzip, minilzip/lzlib, clzip, and
pdlzip. pdlzip.
If the cause of file corruption is a damaged medium, the combination GNU ddrescue provides data recovery capabilities which nicely complement
@w{GNU ddrescue + lziprecover} is the recommended option for recovering data those of lziprecover. If the cause of file corruption is a damaged medium,
from damaged files. @xref{ddrescue-example}, @ref{ddrescue-example2}, and the combination @w{GNU ddrescue + lziprecover} is the recommended option for
@ref{ddrescue-example3}, for examples. recovering data from damaged files. @xref{ddrescue-example},
@ref{ddrescue-example2}, and @ref{ddrescue-example3}, for examples.
@ifnothtml @ifnothtml
@xref{Top,GNU ddrescue manual,,ddrescue}, @xref{Top,GNU ddrescue manual,,ddrescue},
@end ifnothtml @end ifnothtml
@ -447,15 +423,15 @@ Quiet operation. Suppress all messages.
When creating or reading fec files (but not when listing), for each directory When creating or reading fec files (but not when listing), for each directory
operand, read and process all files in that directory, recursively. Follow operand, read and process all files in that directory, recursively. Follow
symbolic links given in the command line, but skip symbolic links that are symbolic links given in the command line, but skip symbolic links that are
encountered recursively. Ignore files with extension @file{.fec}, and files encountered recursively. Ignore files and directories named @file{fec} or
and directories named @file{fec}. @file{*[-._]fec}.
@item -R @item -R
@itemx --dereference-recursive @itemx --dereference-recursive
When creating or reading fec files (but not when listing), for each directory When creating or reading fec files (but not when listing), for each directory
operand, read and process all files in that directory, recursively, operand, read and process all files in that directory, recursively,
following all symbolic links. Ignore files with extension @file{.fec}, and following all symbolic links. Ignore files and directories named @file{fec}
files and directories named @file{fec}. or @file{*[-._]fec}.
@item -s @item -s
@itemx --split @itemx --split
@ -741,8 +717,7 @@ POSIX recommends these conventions for command-line arguments.
@item Certain options require an argument. @item Certain options require an argument.
@item An option and its argument may or may not appear as separate tokens. @item An option and its argument may or may not appear as separate tokens.
(In other words, the whitespace separating them is optional, unless the (In other words, the whitespace separating them is optional).
argument is the empty string).
Thus, @w{@option{-o foo}} and @option{-ofoo} are equivalent. Thus, @w{@option{-o foo}} and @option{-ofoo} are equivalent.
@item One or more options without arguments, followed by at most one option @item One or more options without arguments, followed by at most one option
@ -903,8 +878,8 @@ device failure is storing backup copies in separate media.
The extraordinary safety of the lzip format allows lziprecover to use the The extraordinary safety of the lzip format allows lziprecover to use the
redundance that occurs naturally when making compressed backups. Lziprecover redundance that occurs naturally when making compressed backups. Lziprecover
can recover data that would not be recoverable from files compressed in can recover data that would not be recoverable from files compressed in
other formats. Let's see two examples of how much better is lzip compared other formats. See these two examples of the data recovery capabilities
with gzip and bzip2 with respect to data safety: offered by lziprecover:
@menu @menu
* Merging with a backup:: Recovering a file using a damaged backup * Merging with a backup:: Recovering a file using a damaged backup
@ -1279,6 +1254,7 @@ possible:
All multibyte values are stored in little endian order except All multibyte values are stored in little endian order except
@samp{prodata_md5}. @samp{prodata_md5}.
@anchor{fbs}
The @samp{fbs} (fec_block_size) field is coded as a little endian 16-bit The @samp{fbs} (fec_block_size) field is coded as a little endian 16-bit
floating point unsigned integer with an 11-bit mantissa at bits 0-10 and a floating point unsigned integer with an 11-bit mantissa at bits 0-10 and a
5-bit exponent at bits 11-15. The mantissa is an integer between 0 and 2047. 5-bit exponent at bits 11-15. The mantissa is an integer between 0 and 2047.
@ -1337,10 +1313,9 @@ Bit 0 (is_crc_c): crc_array contains CRC32 (0) or CRC32-C (1).@*
Bit 1 (gf16): Galois field is GF(2^8) (0) or GF(2^16) (1).@* Bit 1 (gf16): Galois field is GF(2^8) (0) or GF(2^16) (1).@*
Bits 2-7: zero. Bits 2-7: zero.
@anchor{fbs}
@item fbs (coded fec_block_size) @item fbs (coded fec_block_size)
Number of FEC bytes per block. It is a multiple of 512 bytes between 512 Number of FEC bytes per block. It is a multiple of 512 bytes between 512
bytes and @w{128 TiB}. bytes and @w{128 TiB}. @xref{fbs}.
@item prodata_size @item prodata_size
Size of the protected file. 1 byte to @w{4 EiB}. Size of the protected file. 1 byte to @w{4 EiB}.
@ -1388,10 +1363,11 @@ A four byte string identifying the fec packet, with the value "\xB3FEC"
(0xB3, 0x46, 0x45, 0x43). (0xB3, 0x46, 0x45, 0x43).
@item fbn (fec_block_number) @item fbn (fec_block_number)
Number of this FEC block. Required to compute the decode matrix. Number of this FEC block (0 to 32767). Required to compute the decode matrix.
@item fbs (coded fec_block_size) @item fbs (coded fec_block_size)
@xref{fbs}. Number of FEC bytes per block. It is a multiple of 512 bytes between 512
bytes and @w{128 TiB}. @xref{fbs}.
@item header_crc @item header_crc
CRC32 of the previous fields, including magic. CRC32 of the previous fields, including magic.

2
dump_remove.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by

2
fec.h
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2023-2024 Antonio Diaz Diaz. Copyright (C) 2023-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by

11
fec_create.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2023-2024 Antonio Diaz Diaz. Copyright (C) 2023-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by
@ -279,11 +279,10 @@ unsigned compute_fec_blocks( const unsigned long prodata_size,
} }
// return random number between 0 and 32767
unsigned my_rand( unsigned long & state ) unsigned my_rand( unsigned long & state )
{ {
state = state * 1103515245 + 12345; state = state * 1103515245 + 12345;
return ( state / 65536 ) % 32768; return ( state / 65536 ) % 32768; // random number from 0 to 32767
} }
void random_fbn_vector( const unsigned fec_blocks, const bool gf16, void random_fbn_vector( const unsigned fec_blocks, const bool gf16,
@ -297,7 +296,7 @@ void random_fbn_vector( const unsigned fec_blocks, const bool gf16,
for( unsigned i = 0; i < fec_blocks; ++i ) for( unsigned i = 0; i < fec_blocks; ++i )
{ {
again: const unsigned fbn = again: const unsigned fbn =
gf16 ? my_rand( state ) : my_rand( state ) % 128; gf16 ? my_rand( state ) % max_k16 : my_rand( state ) % max_k8;
for( unsigned j = 0; j < fbn_vector.size(); ++j ) for( unsigned j = 0; j < fbn_vector.size(); ++j )
if( fbn == fbn_vector[j] ) goto again; if( fbn == fbn_vector[j] ) goto again;
fbn_vector.push_back( fbn ); fbn_vector.push_back( fbn );
@ -387,8 +386,8 @@ bool write_fec( const char * const input_filename,
return true; return true;
} }
fail: fail:
show_file_error( printable_name( output_filename, false ), write_error_msg, show_file_error( printable_name( output_filename, false ), wr_err_msg, errno );
errno ); return false; return false;
} }

8
fec_repair.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2023-2024 Antonio Diaz Diaz. Copyright (C) 2023-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by
@ -338,12 +338,12 @@ void Fec_index::show_fec_data( const std::string & input_filename,
const double fpercent = ( 100.0 * fec_bytes_ ) / prodata_size_; const double fpercent = ( 100.0 * fec_bytes_ ) / prodata_size_;
if( input_filename.size() ) if( input_filename.size() )
std::fprintf( f, "Protected file: '%s'\n", input_filename.c_str() ); std::fprintf( f, "Protected file: '%s'\n", input_filename.c_str() );
std::fprintf( f, "Protected size: %11s Block size: %5s Data blocks: %u\n" std::fprintf( f, "Protected size: %11s Block size: %5s Data blocks: %s\n"
" Fec file: '%s'\n" " Fec file: '%s'\n"
" Fec size: %11s %6.2f%% Fec blocks: %u\n" " Fec size: %11s %6.2f%% Fec blocks: %u\n"
" Fec bytes: %11s %6.2f%% Fec numbers:", " Fec bytes: %11s %6.2f%% Fec numbers:",
format_num3( prodata_size_ ), format_num3( fec_block_size_ ), format_num3( prodata_size_ ), format_num3( fec_block_size_ ),
prodata_blocks(), printable_name( fec_filename ), format_num3( prodata_blocks() ), printable_name( fec_filename ),
format_num3( fec_net_size_ ), spercent, fec_blocks(), format_num3( fec_net_size_ ), spercent, fec_blocks(),
format_num3( fec_bytes_ ), fpercent ); format_num3( fec_bytes_ ), fpercent );
for( unsigned i = 0; i < fec_blocks(); ++i ) // print ranges of fbn's for( unsigned i = 0; i < fec_blocks(); ++i ) // print ranges of fbn's
@ -797,7 +797,7 @@ int fec_test( const std::vector< std::string > & filenames,
// write repaired prodata // write repaired prodata
if( writeblock( outfd, prodata, prodata_size ) != prodata_size ) if( writeblock( outfd, prodata, prodata_size ) != prodata_size )
{ show_file_error( printable_name( output_filename, false ), { show_file_error( printable_name( output_filename, false ),
write_error_msg, errno ); set_retval( retval, 1 ); } wr_err_msg, errno ); set_retval( retval, 1 ); }
else if( !close_outstream( &in_stats ) ) set_retval( retval, 1 ); else if( !close_outstream( &in_stats ) ) set_retval( retval, 1 );
if( retval ) cleanup_and_fail( retval ); if( retval ) cleanup_and_fail( retval );
if( verbosity >= 1 ) if( verbosity >= 1 )

2
gf16.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2023-2024 Antonio Diaz Diaz. Copyright (C) 2023-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by

2
gf8.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2023-2024 Antonio Diaz Diaz. Copyright (C) 2023-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by

4
list.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by
@ -128,7 +128,7 @@ int list_files( const std::vector< std::string > & filenames,
std::fflush( stdout ); std::fflush( stdout );
} }
if( verbosity >= 0 && ( std::ferror( stdout ) || std::fclose( stdout ) != 0 ) ) if( verbosity >= 0 && ( std::ferror( stdout ) || std::fclose( stdout ) != 0 ) )
{ show_file_error( "(stdout)", write_error_msg, errno ); { show_file_error( "(stdout)", wr_err_msg, errno );
set_retval( retval, 1 ); } set_retval( retval, 1 ); }
return retval; return retval;
} }

4
lunzcrash.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by
@ -378,7 +378,7 @@ int md5sum_files( const std::vector< std::string > & filenames )
if( std::ferror( stdout ) ) break; if( std::ferror( stdout ) ) break;
} }
if( verbosity >= 0 && ( std::ferror( stdout ) || std::fclose( stdout ) != 0 ) ) if( verbosity >= 0 && ( std::ferror( stdout ) || std::fclose( stdout ) != 0 ) )
{ show_file_error( "(stdout)", write_error_msg, errno ); { show_file_error( "(stdout)", wr_err_msg, errno );
set_retval( retval, 1 ); } set_retval( retval, 1 ); }
return retval; return retval;
} }

8
lzip.h
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by
@ -35,7 +35,7 @@ public:
bool is_char_set_char() { set_char(); return st < 4; } bool is_char_set_char() { set_char(); return st < 4; }
void set_match() { st = ( st < 7 ) ? 7 : 10; } void set_match() { st = ( st < 7 ) ? 7 : 10; }
void set_rep() { st = ( st < 7 ) ? 8 : 11; } void set_rep() { st = ( st < 7 ) ? 8 : 11; }
void set_short_rep() { st = ( st < 7 ) ? 9 : 11; } void set_shortrep() { st = ( st < 7 ) ? 9 : 11; }
}; };
@ -430,9 +430,9 @@ const char * const corrupt_mm_msg = "Corrupt header in multimember file.";
const char * const empty_msg = "Empty member not allowed."; const char * const empty_msg = "Empty member not allowed.";
const char * const mmap_msg = "Can't mmap"; const char * const mmap_msg = "Can't mmap";
const char * const nonzero_msg = "Nonzero first LZMA byte."; const char * const nonzero_msg = "Nonzero first LZMA byte.";
const char * const short_file_msg = "Input file is too short."; const char * const short_file_msg = "Input file is truncated.";
const char * const trailing_msg = "Trailing data not allowed."; const char * const trailing_msg = "Trailing data not allowed.";
const char * const write_error_msg = "Write error"; const char * const wr_err_msg = "Write error";
// defined in alone_to_lz.cc // defined in alone_to_lz.cc
int alone_to_lz( const int infd, const Pretty_print & pp ); int alone_to_lz( const int infd, const Pretty_print & pp );

4
lzip_index.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by
@ -190,7 +190,7 @@ Lzip_index::Lzip_index( const int infd, const Cl_options & cl_opts,
( !read_header( infd, header, 0 ) || ( !read_header( infd, header, 0 ) ||
!check_header( header, ignore_bad_ds ) ) ) return; !check_header( header, ignore_bad_ds ) ) ) return;
if( insize < min_member_size ) if( insize < min_member_size )
{ error_ = short_file_msg; retval_ = 2; return; } { error_ = "Input file is truncated."; retval_ = 2; return; }
if( insize > INT64_MAX ) if( insize > INT64_MAX )
{ error_ = "Input file is too long (2^63 bytes or more)."; { error_ = "Input file is too long (2^63 bytes or more).";
retval_ = 2; return; } retval_ = 2; return; }

2
lzip_index.h
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by

2
main.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by

8
main_common.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by
@ -20,7 +20,7 @@ int verbosity = 0;
namespace { namespace {
const char * const program_year = "2024"; const char * const program_year = "2025";
void show_version() void show_version()
{ {
@ -32,7 +32,7 @@ void show_version()
} }
// separate numbers of 6 or more digits in groups of 3 digits using '_' // separate numbers of 5 or more digits in groups of 3 digits using '_'
const char * format_num3p( long long num, const bool raw = false ) const char * format_num3p( long long num, const bool raw = false )
{ {
enum { buffers = 8, bufsize = 4 * sizeof num, n = 10 }; enum { buffers = 8, bufsize = 4 * sizeof num, n = 10 };
@ -56,7 +56,7 @@ const char * format_num3p( long long num, const bool raw = false )
{ num /= 1000; prefix = si_prefix[i]; } { num /= 1000; prefix = si_prefix[i]; }
if( prefix ) *(--p) = prefix; if( prefix ) *(--p) = prefix;
} }
const bool split = num >= 100000 || num <= -100000; const bool split = num >= 10000 || num <= -10000;
for( int i = 0; ; ) for( int i = 0; ; )
{ {

2
md5.cc
View file

@ -1,6 +1,6 @@
/* Functions to compute MD5 message digest of memory blocks according to the /* Functions to compute MD5 message digest of memory blocks according to the
definition of MD5 in RFC 1321 from April 1992. definition of MD5 in RFC 1321 from April 1992.
Copyright (C) 2020-2024 Antonio Diaz Diaz. Copyright (C) 2020-2025 Antonio Diaz Diaz.
This library is free software. Redistribution and use in source and This library is free software. Redistribution and use in source and
binary forms, with or without modification, are permitted provided binary forms, with or without modification, are permitted provided

2
md5.h
View file

@ -1,6 +1,6 @@
/* Functions to compute MD5 message digest of memory blocks according to the /* Functions to compute MD5 message digest of memory blocks according to the
definition of MD5 in RFC 1321 from April 1992. definition of MD5 in RFC 1321 from April 1992.
Copyright (C) 2020-2024 Antonio Diaz Diaz. Copyright (C) 2020-2025 Antonio Diaz Diaz.
This library is free software. Redistribution and use in source and This library is free software. Redistribution and use in source and
binary forms, with or without modification, are permitted provided binary forms, with or without modification, are permitted provided

6
merge.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by
@ -526,8 +526,8 @@ bool copy_file( const int infd, const int outfd, const std::string & iname,
{ {
const int wr = writeblock( outfd, buffer, rd ); const int wr = writeblock( outfd, buffer, rd );
if( wr != rd ) if( wr != rd )
{ show_file_error( printable_name( oname, false ), write_error_msg, { show_file_error( printable_name( oname, false ), wr_err_msg, errno );
errno ); error = true; break; } error = true; break; }
copied_size += rd; copied_size += rd;
} }
if( rd < size ) break; // EOF if( rd < size ) break; // EOF

8
mtester.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by
@ -83,7 +83,7 @@ void LZ_mtester::flush_data()
crc32.update_buf( crc_, buffer + stream_pos, size ); crc32.update_buf( crc_, buffer + stream_pos, size );
if( md5sum ) md5sum->md5_update( buffer + stream_pos, size ); if( md5sum ) md5sum->md5_update( buffer + stream_pos, size );
if( outfd >= 0 && writeblock( outfd, buffer + stream_pos, size ) != size ) if( outfd >= 0 && writeblock( outfd, buffer + stream_pos, size ) != size )
throw Error( write_error_msg ); throw Error( wr_err_msg );
if( pos >= dictionary_size ) if( pos >= dictionary_size )
{ partial_data_pos += pos; pos = 0; pos_wrapped = true; } { partial_data_pos += pos; pos = 0; pos_wrapped = true; }
stream_pos = pos; stream_pos = pos;
@ -171,7 +171,7 @@ int LZ_mtester::test_member( const unsigned long mpos_limit,
if( rdec.decode_bit( bm_rep0[state()] ) == 0 ) // 3rd bit if( rdec.decode_bit( bm_rep0[state()] ) == 0 ) // 3rd bit
{ {
if( rdec.decode_bit( bm_len[state()][pos_state] ) == 0 ) // 4th bit if( rdec.decode_bit( bm_len[state()][pos_state] ) == 0 ) // 4th bit
{ state.set_short_rep(); put_byte( peek( rep0 ) ); continue; } { state.set_shortrep(); put_byte( peek( rep0 ) ); continue; }
} }
else else
{ {
@ -289,7 +289,7 @@ int LZ_mtester::debug_decode_member( const long long dpos, const long long mpos,
std::printf( "%6llu %6llu shortrep %s %6u (%6llu)\n", std::printf( "%6llu %6llu shortrep %s %6u (%6llu)\n",
mp, dp, format_byte( peek( rep0 ) ), mp, dp, format_byte( peek( rep0 ) ),
rep0 + 1, dp - rep0 - 1 ); rep0 + 1, dp - rep0 - 1 );
state.set_short_rep(); put_byte( peek( rep0 ) ); continue; state.set_shortrep(); put_byte( peek( rep0 ) ); continue;
} }
} }
else else

2
mtester.h
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by

2
nrep_stats.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by

2
range_dec.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by

22
recursive.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2023-2024 Antonio Diaz Diaz. Copyright (C) 2023-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by
@ -33,15 +33,14 @@
namespace { namespace {
/* Return true if full_name is a regular file without extension .fec // Return true if full_name is a regular file or (a link to) a directory.
or (a link to) a directory. */
bool test_full_name( const std::string & full_name, const struct stat * stp, bool test_full_name( const std::string & full_name, const struct stat * stp,
const bool follow ) const bool follow )
{ {
struct stat st, st2; struct stat st, st2;
if( ( follow && stat( full_name.c_str(), &st ) != 0 ) || if( ( follow && stat( full_name.c_str(), &st ) != 0 ) ||
( !follow && lstat( full_name.c_str(), &st ) != 0 ) ) return false; ( !follow && lstat( full_name.c_str(), &st ) != 0 ) ) return false;
if( S_ISREG( st.st_mode ) ) return !has_fec_extension( full_name ); if( S_ISREG( st.st_mode ) ) return true;
if( !S_ISDIR( st.st_mode ) ) return false; if( !S_ISDIR( st.st_mode ) ) return false;
std::string prev_dir( full_name ); std::string prev_dir( full_name );
@ -62,11 +61,21 @@ bool test_full_name( const std::string & full_name, const struct stat * stp,
return !loop; // (link to) directory return !loop; // (link to) directory
} }
bool ignore_name( const std::string & name )
{
if( name == "." || name == ".." || name == "fec" || name == "FEC" ||
has_fec_extension( name ) ) return true;
return name.size() > 3 && name.compare( name.size() - 3, 3, "fec" ) == 0 &&
( name.end()[-4] == '-' || name.end()[-4] == '.' ||
name.end()[-4] == '_' );
}
} // end namespace } // end namespace
/* Return in input_filename the next file name. ('-' is a valid file name). /* Return in input_filename the next file name. ('-' is a valid file name).
Recursively found files and directories named "fec" are ignored. Ignore recursively found files and directories named "fec" or "*[-._]fec".
Set 'retval' to 1 if a directory fails to open. */ Set 'retval' to 1 if a directory fails to open. */
bool next_filename( std::list< std::string > & filelist, bool next_filename( std::list< std::string > & filelist,
std::string & input_filename, int & retval, std::string & input_filename, int & retval,
@ -105,8 +114,7 @@ bool next_filename( std::list< std::string > & filelist,
const struct dirent * const entryp = readdir( dirp ); const struct dirent * const entryp = readdir( dirp );
if( !entryp ) { closedir( dirp ); break; } if( !entryp ) { closedir( dirp ); break; }
const std::string tmp_name( entryp->d_name ); const std::string tmp_name( entryp->d_name );
if( tmp_name == "." || tmp_name == ".." || tmp_name == "fec" || if( ignore_name( tmp_name ) ) continue;
tmp_name == "FEC" ) continue;
const std::string full_name( input_filename + tmp_name ); const std::string full_name( input_filename + tmp_name );
if( test_full_name( full_name, stdotp, recursive == 2 ) ) if( test_full_name( full_name, stdotp, recursive == 2 ) )
tmp_list.push_back( full_name ); tmp_list.push_back( full_name );

2
reproduce.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by

2
split.cc
View file

@ -1,5 +1,5 @@
/* Lziprecover - Data recovery tool for the lzip format /* Lziprecover - Data recovery tool for the lzip format
Copyright (C) 2009-2024 Antonio Diaz Diaz. Copyright (C) 2009-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by

18
testsuite/check.sh
View file

@ -1,6 +1,6 @@
#! /bin/sh #! /bin/sh
# check script for Lziprecover - Data recovery tool for the lzip format # check script for Lziprecover - Data recovery tool for the lzip format
# Copyright (C) 2009-2024 Antonio Diaz Diaz. # Copyright (C) 2009-2025 Antonio Diaz Diaz.
# #
# This script is free software: you have unlimited permission # This script is free software: you have unlimited permission
# to copy, distribute, and modify it. # to copy, distribute, and modify it.
@ -719,12 +719,28 @@ cmp fec/test.txt.lz.fec fecfile.fec || test_failed $LINENO
"${LZIPRECOVER}" -Ft "${in_lz}" --fec-file=fec/ || test_failed $LINENO "${LZIPRECOVER}" -Ft "${in_lz}" --fec-file=fec/ || test_failed $LINENO
rm -rf fec || framework_failure rm -rf fec || framework_failure
mkdir a mkdir a
mkdir a/bfec
mkdir a/b-fec
mkdir a/b.fec
mkdir a/b_fec
cp fox6.lz "${in_lz}" a || framework_failure cp fox6.lz "${in_lz}" a || framework_failure
cp "${fox_lz}" a/bfec || framework_failure
cp "${fox_lz}" a/b-fec || framework_failure
cp "${fox_lz}" a/b.fec || framework_failure
cp "${fox_lz}" a/b_fec || framework_failure
"${LZIPRECOVER}" -r -Fc a/ -o fec/ || test_failed $LINENO "${LZIPRECOVER}" -r -Fc a/ -o fec/ || test_failed $LINENO
[ -e fec/fox6.lz.fec ] || test_failed $LINENO [ -e fec/fox6.lz.fec ] || test_failed $LINENO
[ -e fec/bfec/fox.lz.fec ] || test_failed $LINENO
[ ! -e fec/b-fec ] || test_failed $LINENO
[ ! -e fec/b.fec ] || test_failed $LINENO
[ ! -e fec/b_fec ] || test_failed $LINENO
cmp fec/test.txt.lz.fec fecfile.fec || test_failed $LINENO cmp fec/test.txt.lz.fec fecfile.fec || test_failed $LINENO
"${LZIPRECOVER}" -r -Fc a -o fec/ || test_failed $LINENO "${LZIPRECOVER}" -r -Fc a -o fec/ || test_failed $LINENO
[ -e fec/a/fox6.lz.fec ] || test_failed $LINENO [ -e fec/a/fox6.lz.fec ] || test_failed $LINENO
[ -e fec/a/bfec/fox.lz.fec ] || test_failed $LINENO
[ ! -e fec/a/b-fec ] || test_failed $LINENO
[ ! -e fec/a/b.fec ] || test_failed $LINENO
[ ! -e fec/a/b_fec ] || test_failed $LINENO
cmp fec/a/test.txt.lz.fec fecfile.fec || test_failed $LINENO cmp fec/a/test.txt.lz.fec fecfile.fec || test_failed $LINENO
rm -rf a fec fecfile.fec || framework_failure rm -rf a fec fecfile.fec || framework_failure

2
unzcrash.cc
View file

@ -1,6 +1,6 @@
/* Unzcrash - Tests robustness of decompressors to corrupted data. /* Unzcrash - Tests robustness of decompressors to corrupted data.
Inspired by unzcrash.c from Julian Seward's bzip2. Inspired by unzcrash.c from Julian Seward's bzip2.
Copyright (C) 2008-2024 Antonio Diaz Diaz. Copyright (C) 2008-2025 Antonio Diaz Diaz.
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by