diff --git a/ChangeLog b/ChangeLog index 2249712..541a369 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,20 +1,3 @@ -2025-02-28 Antonio Diaz Diaz - - * Version 0.27 released. - * common_decode.cc (format_member_name): Print seconds since epoch - if date is out of range. Use at least 4 digits to print years. - Print typeflag after the member name if unknown file type. - (make_dirs): stat last dir before trying to create directories. - * decode.cc (skip_warn): Diagnose a corrupt tar header. - * extended.cc (Extended::parse): Diagnose a CRC mismatch. - New argument 'msg_vecp' for multi-threaded diagnostics. - * Many small fixes and improvements to the code and the manual. - * tarlz.texi: New chapter 'Creating backups safely'. - (Suggested by Aren Tyr). - * check.sh: Require lzip. Create .tar files from .tar.lz files. - Limit '--mtime' test to safe dates. (Reported by Aren Tyr). - * testsuite: Add 5 new test files. - 2024-12-07 Antonio Diaz Diaz * Version 0.26 released. @@ -90,7 +73,8 @@ * Lzlib 1.12 or newer is now required. * decode.cc (decode): Skip members without name except when listing. decode_lz.cc (dworker): Likewise. (Reported by Florian Schmaus). - * New options '-z, --compress', '-o, --output', and '--warn-newer'. + * New options '-z, --compress' and '-o, --output'. + * New option '--warn-newer'. * tarlz.texi (Invoking tarlz): Document concatenation to stdout. * check.sh: Fix the '--diff' test on OS/2. (Reported by Elbert Pol). @@ -113,10 +97,10 @@ 2020-07-30 Antonio Diaz Diaz * Version 0.17 released. - * New options '--mtime' and '-p, --preserve-permissions'. + * New option '--mtime'. + * New option '-p, --preserve-permissions'. * Implement multi-threaded '-d, --diff'. * list_lz.cc: Rename to decode_lz.cc. - (decode_lz): Limit num_workers to number of members. * main.cc (main): Report an error if a file name is empty or if the archive is specified more than once. * lzip_index.cc: Improve messages for corruption in last header. @@ -141,7 +125,8 @@ 2019-03-12 Antonio Diaz Diaz * Version 0.14 released. - * New options '--exclude' and '-h, --dereference'. + * New option '--exclude'. + * New option '-h, --dereference'. * Short option name '-h' no longer means '--help'. * create.cc: Implement '-A, --concatenate' and '-r, --append' to uncompressed archives and to standard output. @@ -160,7 +145,8 @@ * create.cc (fill_headers): Fix use of st_rdev instead of st_dev. * Save just numerical uid/gid if user or group not in database. * extract.cc (format_member_name): Print devmajor and devminor. - * New options '-d, --diff' and '--ignore-ids'. + * New option '-d, --diff'. + * New option '--ignore-ids'. * extract.cc: Fast '-t, --list' on seekable uncompressed archives. 2019-02-13 Antonio Diaz Diaz @@ -175,7 +161,8 @@ 2019-01-31 Antonio Diaz Diaz * Version 0.10 released. - * New options '--bsolid' and '-B, --data-size'. + * New option '--bsolid'. + * New option '-B, --data-size'. * create.cc: Set ustar name to zero if extended header is used. 2019-01-22 Antonio Diaz Diaz @@ -198,7 +185,8 @@ 2018-11-23 Antonio Diaz Diaz * Version 0.7 released. - * New options '--keep-damaged' and '--no-solid'. + * New option '--keep-damaged'. + * New option '--no-solid'. * create.cc (archive_write): Minimize dictionary size. Detect and skip archive in '-A', '-c', and '-r'. * main.cc (show_version): Show the version of lzlib being used. @@ -207,7 +195,7 @@ * Version 0.6 released. * New option '-A, --concatenate'. - * Replace option '--ignore-crc' with '--missing-crc'. + * Option '--ignore-crc' replaced with '--missing-crc'. * create.cc (add_member): Check that uid, gid, mtime, devmajor, and devminor are in ustar range. * configure: Accept appending to CXXFLAGS; 'CXXFLAGS+=OPTIONS'. @@ -232,10 +220,11 @@ * Version 0.3 released. * Rename project to 'tarlz' from 'pmtar' (Poor Man's Tar). - * New options '-C, --directory' and '-r, --append'. + * New option '-C, --directory'. + * Implement lzip compression of members at archive creation. + * New option '-r, --append'. * New options '--owner' and '--group'. * New options '--asolid', '--dsolid', and '--solid'. - * Implement lzip compression of members at archive creation. * Implement file appending to compressed archive. * Implement transparent decompression of the archive. * Implement skipping over damaged (un)compressed members. @@ -253,7 +242,7 @@ * Version 0.1 released. -Copyright (C) 2013-2025 Antonio Diaz Diaz. +Copyright (C) 2013-2024 Antonio Diaz Diaz. This file is a collection of facts, and thus it is not copyrightable, but just in case, you have unlimited permission to copy, distribute, and modify it. diff --git a/INSTALL b/INSTALL index 24f9e51..4de87a7 100644 --- a/INSTALL +++ b/INSTALL @@ -4,12 +4,10 @@ You will need a C++98 compiler with support for 'long long', and the compression library lzlib installed. (gcc 3.3.6 or newer is recommended). I use gcc 6.1.0 and 3.3.6, but the code should compile with any standards compliant compiler. -Lzlib must be version 1.12 or newer. -Gcc is available at http://gcc.gnu.org -Lzlib is available at http://www.nongnu.org/lzip/lzlib.html +Gcc is available at http://gcc.gnu.org. +Lzlib is available at http://www.nongnu.org/lzip/lzlib.html. -Lzip is required to run the tests. -Lzip is available at http://www.nongnu.org/lzip/lzip.html +Lzlib must be version 1.12 or newer. The operating system must allow signal handlers read access to objects with static storage duration so that the cleanup handler for Control-C can delete @@ -20,8 +18,6 @@ Procedure --------- 1. Unpack the archive if you have not done so already: - tarlz -xf tarlz[version].tar.lz -or tar -xf tarlz[version].tar.lz or lzip -cd tarlz[version].tar.lz | tar -xf - @@ -78,7 +74,7 @@ After running 'configure', you can run 'make' and 'make install' as explained above. -Copyright (C) 2013-2025 Antonio Diaz Diaz. +Copyright (C) 2013-2024 Antonio Diaz Diaz. This file is free documentation: you have unlimited permission to copy, distribute, and modify it. diff --git a/Makefile.in b/Makefile.in index 75183d5..a9f27c4 100644 --- a/Makefile.in +++ b/Makefile.in @@ -135,34 +135,42 @@ dist : doc $(DISTNAME)/*.cc \ $(DISTNAME)/testsuite/check.sh \ $(DISTNAME)/testsuite/test.txt \ + $(DISTNAME)/testsuite/test.txt.tar \ $(DISTNAME)/testsuite/test_bad1.txt.tar \ $(DISTNAME)/testsuite/test_bad[12].txt \ $(DISTNAME)/testsuite/rfoo \ $(DISTNAME)/testsuite/rbar \ $(DISTNAME)/testsuite/rbaz \ + $(DISTNAME)/testsuite/test3.tar \ + $(DISTNAME)/testsuite/test3_nn.tar \ + $(DISTNAME)/testsuite/test3_eoa[1-4].tar \ + $(DISTNAME)/testsuite/test3_gh[1-4].tar \ $(DISTNAME)/testsuite/test3_bad[1-5].tar \ + $(DISTNAME)/testsuite/test3_dir.tar \ + $(DISTNAME)/testsuite/t155.tar \ + $(DISTNAME)/testsuite/t155_fv[1-3].tar \ + $(DISTNAME)/testsuite/eoa_blocks.tar \ $(DISTNAME)/testsuite/em.lz \ + $(DISTNAME)/testsuite/test.txt.lz \ $(DISTNAME)/testsuite/test.txt.tar.lz \ $(DISTNAME)/testsuite/test_bad[12].txt.tar.lz \ $(DISTNAME)/testsuite/test3.tar.lz \ $(DISTNAME)/testsuite/test3_eoa[1-5].tar.lz \ - $(DISTNAME)/testsuite/test3_gh[1-8].tar.lz \ + $(DISTNAME)/testsuite/test3_gh[1-6].tar.lz \ $(DISTNAME)/testsuite/test3_nn.tar.lz \ $(DISTNAME)/testsuite/test3_sm[1-4].tar.lz \ $(DISTNAME)/testsuite/test3_bad[1-6].tar.lz \ - $(DISTNAME)/testsuite/test3_crc.tar.lz \ - $(DISTNAME)/testsuite/test3_uk.tar.lz \ $(DISTNAME)/testsuite/test3_dir.tar.lz \ $(DISTNAME)/testsuite/test3_dot.tar.lz \ $(DISTNAME)/testsuite/tar_in_tlz[12].tar.lz \ $(DISTNAME)/testsuite/tlz_in_tar[12].tar \ $(DISTNAME)/testsuite/ts_in_link.tar.lz \ $(DISTNAME)/testsuite/t155.tar.lz \ - $(DISTNAME)/testsuite/t155_fv[1-7].tar.lz \ + $(DISTNAME)/testsuite/t155_fv[1-6].tar.lz \ $(DISTNAME)/testsuite/dotdot[1-5].tar.lz \ $(DISTNAME)/testsuite/ug32767.tar.lz \ $(DISTNAME)/testsuite/ug32chars.tar.lz \ - $(DISTNAME)/testsuite/eoa_blocks.lz + $(DISTNAME)/testsuite/eoa_blocks.tar.lz rm -f $(DISTNAME) clean : diff --git a/NEWS b/NEWS index de882ae..e2ff75a 100644 --- a/NEWS +++ b/NEWS @@ -1,33 +1,13 @@ -Changes in version 0.27: +Changes in version 0.26: -tarlz now prints seconds since epoch if a file date is out of range. +tarlz now exits with error status 2 if any empty lzip member is found in a +multimember compressed archive. -tarlz now uses at least 4 digits to print years. +Scalability of parallel compressed creation and decoding has been increased. -'tarlz -tv' now prints the value of typeflag after the member name for -unknown file types. +A diagnostic message for read error has been improved. -tarlz now prints a diagnostic when it finds a corrupt tar header (or random -data where a tar header is expected). +The chapter 'Syntax of command-line arguments' has been added to the manual. -tarlz now diagnoses CRC mismatches in extended records separately. - -Multi-threaded decoding now prints diagnostics about CRC mismatches and -unknown keywords in extended records in the correct order. - -Many small fixes and improvements have been made to the code and the manual. - -The chapter 'Creating backups safely' has been added to the manual. -(Suggested by Aren Tyr). - -Lzip is now required to run the tests because I have not found any other -portable and reliable way to tell compressed archives from non-compressed. - -Where possible, .tar archives for the testsuite are now decompressed from -their .tar.lz versions instead of distributed. - -'make check' no longer tests '--mtime' with extreme dates to avoid test -failures caused by differences with the system tool 'touch'. -(Reported by Aren Tyr). - -5 new test files have been added to the testsuite. +'make check' now skips time stamps out of range or not recognized by system +tools. (Reported by J Dean). diff --git a/README b/README index 08f135c..8d2701c 100644 --- a/README +++ b/README @@ -15,7 +15,7 @@ compressed archives. Keeping the alignment between tar members and lzip members has two advantages. It adds an indexed lzip layer on top of the tar archive, making -it possible to decode the archive safely in parallel. It also reduces the +it possible to decode the archive safely in parallel. It also minimizes the amount of data lost in case of corruption. Compressing a tar archive with plzip may even double the amount of files lost for each lzip member damaged because it does not keep the members aligned. @@ -93,7 +93,7 @@ Tarlz uses Arg_parser for command-line argument parsing: http://www.nongnu.org/arg-parser/arg_parser.html -Copyright (C) 2013-2025 Antonio Diaz Diaz. +Copyright (C) 2013-2024 Antonio Diaz Diaz. This file is free documentation: you have unlimited permission to copy, distribute, and modify it. diff --git a/archive_reader.cc b/archive_reader.cc index fbdaba7..2da8f56 100644 --- a/archive_reader.cc +++ b/archive_reader.cc @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -82,9 +82,10 @@ Archive_descriptor::Archive_descriptor( const std::string & archive_name ) int Archive_reader_base::parse_records( Extended & extended, - const Tar_header header, Resizable_buffer & rbuf, - const char * const default_msg, const bool permissive, - std::vector< std::string > * const msg_vecp ) + const Tar_header header, + Resizable_buffer & rbuf, + const char * const default_msg, + const bool permissive ) { const long long edsize = parse_octal( header + size_o, size_l ); const long long bufsize = round_up( edsize ); @@ -94,7 +95,7 @@ int Archive_reader_base::parse_records( Extended & extended, if( !rbuf.resize( bufsize ) ) return err( -1, mem_msg ); e_msg_ = ""; e_code_ = 0; int retval = read( rbuf.u8(), bufsize ); // extended records buffer - if( retval == 0 && !extended.parse( rbuf(), edsize, permissive, msg_vecp ) ) + if( retval == 0 && !extended.parse( rbuf(), edsize, permissive ) ) retval = 2; if( retval && !*e_msg_ ) e_msg_ = default_msg; return retval; @@ -155,9 +156,16 @@ int Archive_reader::read( uint8_t * const buf, const int size ) const int rd = LZ_decompress_read( decoder, buf + sz, size - sz ); if( rd < 0 ) { + const unsigned long long old_pos = LZ_decompress_total_in_size( decoder ); if( LZ_decompress_sync_to_member( decoder ) < 0 ) internal_error( "library error (LZ_decompress_sync_to_member)." ); - e_skip_ = true; set_error_status( 2 ); return err( 2, "", 0, sz, true ); + e_skip_ = true; set_error_status( 2 ); + const unsigned long long new_pos = LZ_decompress_total_in_size( decoder ); + // lzlib < 1.8 does not update total_in_size when syncing to member + if( new_pos >= old_pos && new_pos < LLONG_MAX ) + return err( 2, "", 0, sz, true ); + return err( -1, "Skipping to next header failed. " + "Lzlib 1.8 or newer required.", 0, sz ); } if( rd == 0 && LZ_decompress_finished( decoder ) == 1 ) { return err( -2, end_msg, 0, sz ); } diff --git a/archive_reader.h b/archive_reader.h index d313e2e..a68372a 100644 --- a/archive_reader.h +++ b/archive_reader.h @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -65,10 +65,9 @@ public: If !OK, fills all the e_* variables. */ virtual int read( uint8_t * const buf, const int size ) = 0; - int parse_records( Extended & extended, - const Tar_header header, Resizable_buffer & rbuf, - const char * const default_msg, const bool permissive, - std::vector< std::string > * const msg_vecp = 0 ); + int parse_records( Extended & extended, const Tar_header header, + Resizable_buffer & rbuf, const char * const default_msg, + const bool permissive ); }; @@ -113,7 +112,7 @@ public: long long mdata_end() const { return mdata_end_; } bool at_member_end() const { return data_pos_ == mdata_end_; } - // Reset decoder and set position to the start of the member. + // Resets decoder and sets position to the start of the member. void set_member( const long i ); int read( uint8_t * const buf, const int size ); diff --git a/arg_parser.cc b/arg_parser.cc index 9275846..0c528b2 100644 --- a/arg_parser.cc +++ b/arg_parser.cc @@ -1,5 +1,5 @@ /* Arg_parser - POSIX/GNU command-line argument parser. (C++ version) - Copyright (C) 2006-2025 Antonio Diaz Diaz. + Copyright (C) 2006-2024 Antonio Diaz Diaz. This library is free software. Redistribution and use in source and binary forms, with or without modification, are permitted provided diff --git a/arg_parser.h b/arg_parser.h index 2fe5a61..ab77fc5 100644 --- a/arg_parser.h +++ b/arg_parser.h @@ -1,5 +1,5 @@ /* Arg_parser - POSIX/GNU command-line argument parser. (C++ version) - Copyright (C) 2006-2025 Antonio Diaz Diaz. + Copyright (C) 2006-2024 Antonio Diaz Diaz. This library is free software. Redistribution and use in source and binary forms, with or without modification, are permitted provided diff --git a/common.cc b/common.cc index 834d421..b653e01 100644 --- a/common.cc +++ b/common.cc @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by diff --git a/common_decode.cc b/common_decode.cc index 36f6a43..632ad3f 100644 --- a/common_decode.cc +++ b/common_decode.cc @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -117,56 +117,53 @@ bool block_is_zero( const uint8_t * const buf, const int size ) bool format_member_name( const Extended & extended, const Tar_header header, Resizable_buffer & rbuf, const bool long_format ) { - if( !long_format ) + if( long_format ) { - if( !rbuf.resize( extended.path().size() + 2 ) ) return false; - snprintf( rbuf(), rbuf.size(), "%s\n", extended.path().c_str() ); - return true; - } - format_mode_string( header, rbuf() ); - const int group_string_len = - format_user_group_string( extended, header, rbuf() + mode_string_size ); - int offset = mode_string_size + group_string_len; - const time_t mtime = extended.mtime().sec(); - struct tm t; - char buf[32]; // if local time and UTC fail, use seconds since epoch - if( localtime_r( &mtime, &t ) || gmtime_r( &mtime, &t ) ) - snprintf( buf, sizeof buf, "%04d-%02u-%02u %02u:%02u", 1900 + t.tm_year, - 1 + t.tm_mon, t.tm_mday, t.tm_hour, t.tm_min ); - else snprintf( buf, sizeof buf, "%lld", extended.mtime().sec() ); - const Typeflag typeflag = (Typeflag)header[typeflag_o]; - const bool islink = typeflag == tf_link || typeflag == tf_symlink; - const char * const link_string = !islink ? "" : - ( ( typeflag == tf_link ) ? " link to " : " -> " ); - // print "user/group size" in a field of width 19 with 8 or more for size - if( typeflag == tf_chardev || typeflag == tf_blockdev ) - { - const unsigned devmajor = parse_octal( header + devmajor_o, devmajor_l ); - const unsigned devminor = parse_octal( header + devminor_o, devminor_l ); - const int width = std::max( 1, - std::max( 8, 19 - group_string_len ) - 1 - decimal_digits( devminor ) ); - offset += snprintf( rbuf() + offset, rbuf.size() - offset, " %*u,%u", - width, devmajor, devminor ); + format_mode_string( header, rbuf() ); + const int group_string_len = + format_user_group_string( extended, header, rbuf() + mode_string_size ); + int offset = mode_string_size + group_string_len; + const time_t mtime = extended.mtime().sec(); + struct tm t; + if( !localtime_r( &mtime, &t ) ) // if local time fails + { time_t z = 0; if( !gmtime_r( &z, &t ) ) // use UTC, the epoch + { t.tm_year = 70; t.tm_mon = t.tm_hour = t.tm_min = 0; t.tm_mday = 1; } } + const Typeflag typeflag = (Typeflag)header[typeflag_o]; + const bool islink = typeflag == tf_link || typeflag == tf_symlink; + const char * const link_string = !islink ? "" : + ( ( typeflag == tf_link ) ? " link to " : " -> " ); + // print "user/group size" in a field of width 19 with 8 or more for size + if( typeflag == tf_chardev || typeflag == tf_blockdev ) + { + const unsigned devmajor = parse_octal( header + devmajor_o, devmajor_l ); + const unsigned devminor = parse_octal( header + devminor_o, devminor_l ); + const int width = std::max( 1, + std::max( 8, 19 - group_string_len ) - 1 - decimal_digits( devminor ) ); + offset += snprintf( rbuf() + offset, rbuf.size() - offset, " %*u,%u", + width, devmajor, devminor ); + } + else + { + const int width = std::max( 8, 19 - group_string_len ); + offset += snprintf( rbuf() + offset, rbuf.size() - offset, " %*llu", + width, extended.file_size() ); + } + for( int i = 0; i < 2; ++i ) // resize rbuf if not large enough + { + const int len = snprintf( rbuf() + offset, rbuf.size() - offset, + " %4d-%02u-%02u %02u:%02u %s%s%s\n", + 1900 + t.tm_year, 1 + t.tm_mon, t.tm_mday, t.tm_hour, + t.tm_min, extended.path().c_str(), link_string, + islink ? extended.linkpath().c_str() : "" ); + if( len + offset < (int)rbuf.size() ) break; + if( !rbuf.resize( len + offset + 1 ) ) return false; + } } else { - const int width = std::max( 8, 19 - group_string_len ); - offset += snprintf( rbuf() + offset, rbuf.size() - offset, " %*llu", - width, extended.file_size() ); - } - for( int i = 0; i < 2; ++i ) // resize rbuf if not large enough - { - const int len = snprintf( rbuf() + offset, rbuf.size() - offset, - " %s %s%s%s\n", buf, extended.path().c_str(), link_string, - islink ? extended.linkpath().c_str() : "" ); - if( len + offset < (int)rbuf.size() ) { offset += len; break; } - if( !rbuf.resize( len + offset + 1 ) ) return false; - } - if( rbuf()[0] == '?' ) - { - if( !rbuf.resize( offset + 25 + 1 ) ) return false; - snprintf( rbuf() + offset - 1, rbuf.size() - offset, - ": Unknown file type 0x%02X\n", typeflag ); + if( rbuf.size() < extended.path().size() + 2 && + !rbuf.resize( extended.path().size() + 2 ) ) return false; + snprintf( rbuf(), rbuf.size(), "%s\n", extended.path().c_str() ); } return true; } @@ -186,12 +183,9 @@ bool show_member_name( const Extended & extended, const Tar_header header, } -/* Return true if file must be skipped. - Execute -C options if cwd_fd >= 0 (diff or extract). */ bool check_skip_filename( const Cl_options & cl_opts, std::vector< char > & name_pending, - const char * const filename, const int cwd_fd, - std::string * const msgp ) + const char * const filename, const int chdir_fd ) { static int c_idx = -1; // parser index of last -C executed if( Exclude::excluded( filename ) ) return true; // skip excluded files @@ -203,19 +197,18 @@ bool check_skip_filename( const Cl_options & cl_opts, { if( cl_opts.parser.code( i ) == 'C' ) { chdir_pending = true; continue; } if( !nonempty_arg( cl_opts.parser, i ) ) continue; // skip opts, empty names - std::string removed_prefix; // prefix of cl argument + std::string removed_prefix; const char * const name = remove_leading_dotslash( cl_opts.parser.argument( i ).c_str(), &removed_prefix ); if( compare_prefix_dir( name, filename ) || compare_tslash( name, filename ) ) { - print_removed_prefix( removed_prefix, msgp ); + print_removed_prefix( removed_prefix ); skip = false; name_pending[i] = false; - // only serial decoder sets cwd_fd >= 0 to process -C options - if( chdir_pending && cwd_fd >= 0 ) + if( chdir_pending && chdir_fd >= 0 ) { if( c_idx > i ) - { if( fchdir( cwd_fd ) != 0 ) + { if( fchdir( chdir_fd ) != 0 ) { show_error( "Error changing to initial working directory", errno ); throw Chdir_error(); } c_idx = -1; } for( int j = c_idx + 1; j < i; ++j ) @@ -241,11 +234,7 @@ bool make_dirs( const std::string & name ) while( i > 0 && name[i-1] != '/' ) --i; // remove last component while( i > 0 && name[i-1] == '/' ) --i; // remove more slashes const int dirsize = i; // first slash before last component - struct stat st; - if( dirsize > 0 && lstat( std::string( name, 0, dirsize ).c_str(), &st ) == 0 ) - { if( !S_ISDIR( st.st_mode ) ) { errno = ENOTDIR; return false; } - return true; } for( i = 0; i < dirsize; ) // if dirsize == 0, dirname is '/' or empty { while( i < dirsize && name[i] == '/' ) ++i; @@ -255,6 +244,7 @@ bool make_dirs( const std::string & name ) { const std::string partial( name, 0, i ); const mode_t mode = S_IRWXU | S_IRGRP | S_IXGRP | S_IROTH | S_IXOTH; + struct stat st; if( lstat( partial.c_str(), &st ) == 0 ) { if( !S_ISDIR( st.st_mode ) ) { errno = ENOTDIR; return false; } } else if( mkdir( partial.c_str(), mode ) != 0 && errno != EEXIST ) diff --git a/common_mutex.cc b/common_mutex.cc index f7770d7..fb253ed 100644 --- a/common_mutex.cc +++ b/common_mutex.cc @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -131,10 +131,10 @@ bool print_removed_prefix( const std::string & prefix, if( prefixes[i] == prefix ) { xunlock( &mutex ); if( msgp ) msgp->clear(); return false; } prefixes.push_back( prefix ); - xunlock( &mutex ); std::string msg( "Removing leading '" ); msg += prefix; - msg += "' from member names."; // from archive or command line + msg += "' from member names."; if( msgp ) *msgp = msg; else show_error( msg.c_str() ); + xunlock( &mutex ); // put here to prevent mixing calls to show_error return true; } diff --git a/common_mutex.h b/common_mutex.h index 158cb7b..ed3999c 100644 --- a/common_mutex.h +++ b/common_mutex.h @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by diff --git a/compress.cc b/compress.cc index 78ec227..e421fa4 100644 --- a/compress.cc +++ b/compress.cc @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -212,6 +212,7 @@ int compress_archive( const Cl_options & cl_opts, Extended extended; // metadata from extended records Resizable_buffer rbuf; // headers and extended records buffer if( !rbuf.size() ) { show_error( mem_msg ); return 1; } + const char * const rderr_msg = "Read error"; bool first_header = true; while( true ) // process one tar member per iteration @@ -223,7 +224,7 @@ int compress_archive( const Cl_options & cl_opts, show_file_error( filename, "Archive is empty." ); close( infd ); return 2; } if( rd != header_size ) - { show_file_error( filename, rd_err_msg, errno ); close( infd ); return 1; } + { show_file_error( filename, rderr_msg, errno ); close( infd ); return 1; } first_header = false; const bool is_header = check_ustar_chksum( rbuf.u8() ); @@ -258,8 +259,7 @@ int compress_archive( const Cl_options & cl_opts, if( !rbuf.resize( total_header_size + bufsize ) ) { show_file_error( filename, mem_msg ); close( infd ); return 1; } if( readblock( infd, rbuf.u8() + total_header_size, bufsize ) != bufsize ) - { show_file_error( filename, rd_err_msg, errno ); - close( infd ); return 1; } + { show_file_error( filename, rderr_msg, errno ); close( infd ); return 1; } total_header_size += bufsize; if( typeflag == tf_extended ) // do not parse global headers { @@ -269,7 +269,7 @@ int compress_archive( const Cl_options & cl_opts, if( !rbuf.resize( total_header_size + header_size ) ) { show_file_error( filename, mem_msg ); close( infd ); return 1; } if( readblock( infd, rbuf.u8() + total_header_size, header_size ) != header_size ) - { show_file_error( filename, errno ? rd_err_msg : end_msg, errno ); + { show_file_error( filename, errno ? rderr_msg : end_msg, errno ); close( infd ); return errno ? 1 : 2; } if( !check_ustar_chksum( rbuf.u8() ) ) { show_file_error( filename, bad_hdr_msg ); close( infd ); return 2; } diff --git a/configure b/configure index fc65649..c8ebc5b 100755 --- a/configure +++ b/configure @@ -1,12 +1,12 @@ #! /bin/sh # configure script for Tarlz - Archiver with multimember lzip compression -# Copyright (C) 2013-2025 Antonio Diaz Diaz. +# Copyright (C) 2013-2024 Antonio Diaz Diaz. # # This configure script is free software: you have unlimited permission # to copy, distribute, and modify it. pkgname=tarlz -pkgversion=0.27 +pkgversion=0.26 progname=tarlz srctrigger=doc/${pkgname}.texi @@ -175,7 +175,7 @@ echo "MAKEINFO = ${MAKEINFO}" rm -f Makefile cat > Makefile << EOF # Makefile for Tarlz - Archiver with multimember lzip compression -# Copyright (C) 2013-2025 Antonio Diaz Diaz. +# Copyright (C) 2013-2024 Antonio Diaz Diaz. # This file was generated automatically by configure. Don't edit. # # This Makefile is free software: you have unlimited permission diff --git a/create.cc b/create.cc index aeb1b23..09f5797 100644 --- a/create.cc +++ b/create.cc @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -231,7 +231,7 @@ bool store_name( const char * const filename, Extended & extended, } -// add one tar member to the archive and print filename +// add one tar member to the archive int add_member( const char * const filename, const struct stat *, const int flag, struct FTW * ) { @@ -323,7 +323,7 @@ bool copy_file( const int infd, const int outfd, const char * const filename, if( max_size >= 0 ) rest -= size; const int rd = readblock( infd, buffer, size ); if( rd != size && errno ) - { show_file_error( filename, rd_err_msg, errno ); error = true; break; } + { show_file_error( filename, "Read error", errno ); error = true; break; } if( rd > 0 ) { if( !writeblock_wrapper( outfd, buffer, rd ) ) { error = true; break; } @@ -455,7 +455,7 @@ bool fill_headers( const char * const filename, Extended & extended, show_file_error( filename, "Wrong size reading symbolic link.\n" "Please, send a bug report to the maintainers of your filesystem, " "mentioning\n'wrong st_size of symbolic link'.\nSee " - "http://pubs.opengroup.org/onlinepubs/9799919799/basedefs/sys_stat.h.html" ); + "http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_stat.h.html" ); set_error_status( 1 ); return false; } } @@ -607,7 +607,7 @@ int concatenate( const Cl_options & cl_opts ) "Not an appendable tar archive." ); close( infd ); retval = 2; break; } if( !copy_file( infd, outfd, filename, size ) || close( infd ) != 0 ) - { show_file_error( filename, "Error concatenating archive", errno ); + { show_file_error( filename, "Error copying archive", errno ); eoa_pending = false; retval = 1; break; } eoa_pending = true; if( verbosity >= 1 ) std::fprintf( stderr, "%s\n", filename ); @@ -621,34 +621,6 @@ int concatenate( const Cl_options & cl_opts ) } -// Return value: 0 = skip arg, 1 = error, 2 = arg done -int parse_cl_arg( const Cl_options & cl_opts, const int i, - int (* add_memberp)( const char * const filename, - const struct stat *, const int flag, struct FTW * ) ) - { - const int code = cl_opts.parser.code( i ); - const std::string & arg = cl_opts.parser.argument( i ); - const char * filename = arg.c_str(); // filename from command line - if( code == 'C' && chdir( filename ) != 0 ) - { show_file_error( filename, chdir_msg, errno ); return 1; } - if( code ) return 0; // skip options - if( cl_opts.parser.argument( i ).empty() ) return 0; // skip empty names - std::string deslashed; // filename without trailing slashes - unsigned len = arg.size(); - while( len > 1 && arg[len-1] == '/' ) --len; - if( len < arg.size() ) - { deslashed.assign( arg, 0, len ); filename = deslashed.c_str(); } - if( Exclude::excluded( filename ) ) return 0; // skip excluded files - struct stat st; - if( lstat( filename, &st ) != 0 ) - { show_file_error( filename, cant_stat, errno ); - set_error_status( 1 ); return 0; } - if( nftw( filename, add_memberp, 16, cl_opts.dereference ? 0 : FTW_PHYS ) ) - return 1; // write error or OOM - return 2; - } - - int encode( const Cl_options & cl_opts ) { if( !grbuf.size() ) { show_error( mem_msg ); return 1; } @@ -726,11 +698,27 @@ int encode( const Cl_options & cl_opts ) int retval = 0; for( int i = 0; i < cl_opts.parser.arguments(); ++i ) // parse command line { - const int ret = parse_cl_arg( cl_opts, i, add_member ); - if( ret == 0 ) continue; // skip arg - if( ret == 1 ) { retval = 1; break; } // error - if( encoder && cl_opts.solidity == dsolid && // end of group - !archive_write( 0, 0 ) ) { retval = 1; break; } + const int code = cl_opts.parser.code( i ); + const std::string & arg = cl_opts.parser.argument( i ); + const char * filename = arg.c_str(); + if( code == 'C' && chdir( filename ) != 0 ) + { show_file_error( filename, chdir_msg, errno ); retval = 1; break; } + if( code ) continue; // skip options + if( cl_opts.parser.argument( i ).empty() ) continue; // skip empty names + std::string deslashed; // arg without trailing slashes + unsigned len = arg.size(); + while( len > 1 && arg[len-1] == '/' ) --len; + if( len < arg.size() ) + { deslashed.assign( arg, 0, len ); filename = deslashed.c_str(); } + if( Exclude::excluded( filename ) ) continue; // skip excluded files + struct stat st; + if( lstat( filename, &st ) != 0 ) // filename from command line + { show_file_error( filename, cant_stat, errno ); set_error_status( 1 ); } + else if( ( retval = nftw( filename, add_member, 16, + cl_opts.dereference ? 0 : FTW_PHYS ) ) != 0 ) + break; // write error + else if( encoder && cl_opts.solidity == dsolid && !archive_write( 0, 0 ) ) + { retval = 1; break; } } if( retval == 0 ) // write End-Of-Archive records diff --git a/create.h b/create.h index bbbf4c6..d5ef7bc 100644 --- a/create.h +++ b/create.h @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -45,8 +45,3 @@ public: extern Archive_attrs archive_attrs; const char * const cant_stat = "Can't stat input file"; - -// defined in create.cc -int parse_cl_arg( const Cl_options & cl_opts, const int i, - int (* add_memberp)( const char * const filename, - const struct stat *, const int flag, struct FTW * ) ); diff --git a/create_lz.cc b/create_lz.cc index 7afff16..b390290 100644 --- a/create_lz.cc +++ b/create_lz.cc @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -250,7 +250,7 @@ public: }; -// send one ipacket with tar member metadata to courier and print filename +// send one ipacket with tar member metadata to courier int add_member_lz( const char * const filename, const struct stat *, const int flag, struct FTW * ) { @@ -300,10 +300,26 @@ extern "C" void * grouper( void * arg ) for( int i = 0; i < cl_opts.parser.arguments(); ++i ) // parse command line { - const int ret = parse_cl_arg( cl_opts, i, add_member_lz ); - if( ret == 0 ) continue; // skip arg - if( ret == 1 ) exit_fail_mt(); // error - if( cl_opts.solidity == dsolid ) // end of group + const int code = cl_opts.parser.code( i ); + const std::string & arg = cl_opts.parser.argument( i ); + const char * filename = arg.c_str(); + if( code == 'C' && chdir( filename ) != 0 ) + { show_file_error( filename, chdir_msg, errno ); exit_fail_mt(); } + if( code ) continue; // skip options + if( cl_opts.parser.argument( i ).empty() ) continue; // skip empty names + std::string deslashed; // arg without trailing slashes + unsigned len = arg.size(); + while( len > 1 && arg[len-1] == '/' ) --len; + if( len < arg.size() ) + { deslashed.assign( arg, 0, len ); filename = deslashed.c_str(); } + if( Exclude::excluded( filename ) ) continue; // skip excluded files + struct stat st; + if( lstat( filename, &st ) != 0 ) // filename from command line + { show_file_error( filename, cant_stat, errno ); set_error_status( 1 ); } + else if( nftw( filename, add_member_lz, 16, + cl_opts.dereference ? 0 : FTW_PHYS ) != 0 ) + exit_fail_mt(); // write error or OOM + else if( cl_opts.solidity == dsolid ) // end of group courier.receive_packet( new Ipacket ); } diff --git a/debian/changelog b/debian/changelog index afdd4e5..3fa71d6 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,13 +1,3 @@ -tarlz (0.27-1) sid; urgency=medium - - * Updating to standards version 4.7.1. - * Merging upstream version 0.27. - * Updating years in upstream copyright for 2025. - * Fixing FTBFS about undeclared PTHREAD_MUTEX_INITIALIZER by including - missing pthread header. - - -- Daniel Baumann Tue, 04 Mar 2025 07:46:25 +0100 - tarlz (0.26-2) sid; urgency=medium * Updating vcs fields. diff --git a/debian/copyright b/debian/copyright index f29acd9..07bf502 100644 --- a/debian/copyright +++ b/debian/copyright @@ -4,11 +4,11 @@ Upstream-Contact: lzip-bug@nongnu.org Source: https://download.savannah.gnu.org/releases/lzip/tarlz Files: * -Copyright: 2013-2025 Antonio Diaz Diaz +Copyright: 2013-2024 Antonio Diaz Diaz License: GPL-2+ Files: arg_parser.* -Copyright: 2006-2025 Antonio Diaz Diaz +Copyright: 2006-2024 Antonio Diaz Diaz License: BSD-2-clause Files: debian/* diff --git a/debian/patches/debian/0002-pthread.patch b/debian/patches/debian/0002-pthread.patch deleted file mode 100644 index ea5f32e..0000000 --- a/debian/patches/debian/0002-pthread.patch +++ /dev/null @@ -1,15 +0,0 @@ -Author: Daniel Baumann -Description: Fixing FTBFS about undeclared PTHREAD_MUTEX_INITIALIZER by including missing pthread header. - -diff -Naurp tarlz.orig/extended.cc tarlz/extended.cc ---- tarlz.orig/extended.cc -+++ tarlz/extended.cc -@@ -21,6 +21,8 @@ - #include - #include - -+#include -+ - #include "tarlz.h" - #include "common_mutex.h" - diff --git a/debian/patches/series b/debian/patches/series index abf2679..655df63 100644 --- a/debian/patches/series +++ b/debian/patches/series @@ -1,2 +1 @@ debian/0001-build.patch -debian/0002-pthread.patch diff --git a/decode.cc b/decode.cc index 98f6c1d..f2ffa52 100644 --- a/decode.cc +++ b/decode.cc @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -48,18 +48,14 @@ namespace { Resizable_buffer grbuf; -bool skip_warn( const bool reset = false, const unsigned chksum = 0 ) +bool skip_warn( const bool reset = false ) // avoid duplicate warnings { - static bool skipping = false; // avoid duplicate warnings + static bool skipping = false; - if( reset ) { skipping = false; return false; } - if( skipping ) return false; - skipping = true; - if( chksum != 0 ) - { if( verbosity < 1 ) show_error( "Corrupt header." ); - else std::fprintf( stderr, "%s: Corrupt header: ustar chksum = %06o\n", - program_name, chksum ); } - show_error( "Skipping to next header." ); return true; + if( reset ) skipping = false; + else if( !skipping ) + { skipping = true; show_error( "Skipping to next header." ); return true; } + return false; } @@ -131,20 +127,20 @@ int extract_member( const Cl_options & cl_opts, Archive_reader & ar, int outfd = -1; if( !show_member_name( extended, header, 1, grbuf ) ) return 1; + // remove file (or empty dir) before extraction to prevent following links + std::remove( filename ); if( !make_dirs( filename ) ) { show_file_error( filename, intdir_msg, errno ); set_error_status( 1 ); return skip_member( ar, extended, typeflag ); } - // remove file or empty dir before extraction to prevent following links - std::remove( filename ); switch( typeflag ) { case tf_regular: case tf_hiperf: - outfd = open_outstream( filename, true, 0, false ); + outfd = open_outstream( filename ); if( outfd < 0 ) { set_error_status( 1 ); return skip_member( ar, extended, typeflag ); } break; @@ -227,7 +223,7 @@ int extract_member( const Cl_options & cl_opts, Archive_reader & ar, if( cl_opts.keep_damaged ) { writeblock( outfd, buf, std::min( rest, (long long)ar.e_size() ) ); close( outfd ); } - else { close( outfd ); unlink( filename ); } + else { close( outfd ); std::remove( filename ); } } if( ar.fatal() ) return ret; else return 0; } @@ -390,7 +386,7 @@ bool compare_file_contents( std::string & estr, std::string & ostr, const int rd = readblock( infd2, buf2, rsize2 ); if( rd != rsize2 ) { - if( errno ) format_file_error( estr, filename, rd_err_msg, errno ); + if( errno ) format_file_error( estr, filename, "Read error", errno ); else format_file_diff( ostr, filename, "EOF found in file" ); diff = true; } @@ -424,8 +420,8 @@ int decode( const Cl_options & cl_opts ) const bool c_after_name = c_present && option_C_after_filename( cl_opts.parser ); // save current working directory for sequential decoding - const int cwd_fd = c_after_name ? open( ".", O_RDONLY | O_DIRECTORY ) : -1; - if( c_after_name && cwd_fd < 0 ) + const int chdir_fd = c_after_name ? open( ".", O_RDONLY | O_DIRECTORY ) : -1; + if( c_after_name && chdir_fd < 0 ) { show_error( "Can't save current working directory", errno ); return 1; } if( c_present && !c_after_name ) // execute all -C options for( int i = 0; i < cl_opts.parser.arguments(); ++i ) @@ -468,7 +464,8 @@ int decode( const Cl_options & cl_opts ) show_file_error( ad.namep, fv_msg1 ); retval = 2; break; } - skip_warn( false, ustar_chksum( header ) ); + if( skip_warn() && verbosity >= 2 ) + std::fprintf( stderr, "ustar chksum = %07o\n", ustar_chksum( header ) ); set_error_status( 2 ); continue; } skip_warn( true ); // reset warning @@ -483,7 +480,7 @@ int decode( const Cl_options & cl_opts ) if( ret != 0 ) { show_file_error( ad.namep, ar.e_msg(), ar.e_code() ); if( ar.fatal() ) { retval = ret; break; } - set_error_status( ret ); } + skip_warn(); set_error_status( ret ); } continue; } if( typeflag == tf_extended ) @@ -495,7 +492,7 @@ int decode( const Cl_options & cl_opts ) if( ret != 0 ) { show_file_error( ad.namep, ar.e_msg(), ar.e_code() ); if( ar.fatal() ) { retval = ret; break; } - extended.reset(); set_error_status( ret ); } + skip_warn(); extended.reset(); set_error_status( ret ); } else if( !extended.crc_present() && cl_opts.missing_crc ) { show_file_error( ad.namep, miscrc_msg ); retval = 2; break; } prev_extended = true; continue; @@ -507,7 +504,7 @@ int decode( const Cl_options & cl_opts ) try { // members without name are skipped except when listing if( check_skip_filename( cl_opts, name_pending, extended.path().c_str(), - cwd_fd ) ) retval = skip_member( ar, extended, typeflag ); + chdir_fd ) ) retval = skip_member( ar, extended, typeflag ); else { print_removed_prefix( extended.removed_prefix ); diff --git a/decode.h b/decode.h index 867e3e9..05d3072 100644 --- a/decode.h +++ b/decode.h @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -16,7 +16,7 @@ */ inline bool data_may_follow( const Typeflag typeflag ) - { return typeflag == tf_regular || typeflag == tf_hiperf; } + { return typeflag <= 0 || typeflag >= 7; } inline bool uid_gid_in_range( const long long uid, const long long gid ) { return uid == (long long)( (uid_t)uid ) && @@ -27,7 +27,7 @@ const char * const cantln_msg = "Can't %slink '%s' to '%s'"; const char * const mkdir_msg = "Can't create directory"; const char * const mknod_msg = "Can't create device node"; const char * const mkfifo_msg = "Can't create FIFO file"; -const char * const uftype_msg = "%s: Unknown file type 0x%02X, skipping."; +const char * const uftype_msg = "%s: Unknown file type '%c', skipping."; const char * const chown_msg = "Can't change file owner"; mode_t get_umask(); diff --git a/decode_lz.cc b/decode_lz.cc index 8190186..b8763f3 100644 --- a/decode_lz.cc +++ b/decode_lz.cc @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -41,8 +41,7 @@ #include "common_mutex.h" #include "decode.h" -/* Parallel decode does not skip; it exits at the first error. - When a problem is detected by any worker: +/* When a problem is detected by any worker: - the worker requests mastership and returns. - the courier discards new packets received or collected. - the other workers return. @@ -50,9 +49,9 @@ namespace { -const char * const other_msg = "Another worker found an error."; +const char * const other_msg = "Other worker found an error."; -/* line is preformatted and newline terminated except for prefix and errors. +/* line is preformatted and newline terminated except for prefix, error. ok with an empty line is a no-op. */ struct Packet // member name and metadata or error message { @@ -61,7 +60,7 @@ struct Packet // member name and metadata or error message long member_id; // lzip member containing the header of this tar member std::string line; // member name and metadata ready to print, if any Status status; // diagnostics and errors go to stderr - int errcode; // for errors + int errcode; // for error Packet( const long i, const char * const msg, const Status s, const int e ) : member_id( i ), line( msg ), status( s ), errcode( e ) {} }; @@ -120,12 +119,11 @@ public: { xunlock( &omutex ); return master_id == worker_id; } if( error_member_id < 0 || error_member_id > member_id ) error_member_id = member_id; - while( !mastership_granted() && - ( worker_id != deliver_id || !opacket_queues[deliver_id].empty() ) ) + while( !mastership_granted() && ( worker_id != deliver_id || + !opacket_queues[deliver_id].empty() ) ) xwait( &check_master, &omutex ); - if( !mastership_granted() && - // redundant conditions useful for the compiler - worker_id == deliver_id && opacket_queues[deliver_id].empty() ) + if( !mastership_granted() && worker_id == deliver_id && + opacket_queues[deliver_id].empty() ) { master_id = worker_id; // grant mastership for( int i = 0; i < num_workers; ++i ) // delete all packets @@ -358,9 +356,10 @@ Trival extract_member_lz( const Cl_options & cl_opts, if( !courier.collect_packet( member_id, worker_id, rbuf(), Packet::ok ) ) return Trival( other_msg, 0, 1 ); } - struct stat st; - bool exists = lstat( filename, &st ) == 0; - if( !exists && !make_dirs( filename ) ) + /* Remove file before extraction to prevent following links. + Don't remove an empty dir because other thread may need it. */ + if( typeflag != tf_directory ) std::remove( filename ); + if( !make_dirs( filename ) ) { if( format_file_error( rbuf, filename, intdir_msg, errno ) && !courier.collect_packet( member_id, worker_id, rbuf(), Packet::diag ) ) @@ -368,16 +367,12 @@ Trival extract_member_lz( const Cl_options & cl_opts, set_error_status( 1 ); return skip_member_lz( ar, courier, extended, member_id, worker_id, typeflag ); } - /* Remove file before extraction to prevent following links. - Don't remove an empty dir; another thread may need it. */ - if( exists && ( typeflag != tf_directory || !S_ISDIR( st.st_mode ) ) ) - { exists = false; std::remove( filename ); } switch( typeflag ) { case tf_regular: case tf_hiperf: - outfd = open_outstream( filename, true, &rbuf, false ); + outfd = open_outstream( filename, true, &rbuf ); if( outfd < 0 ) { if( verbosity >= 0 && @@ -404,6 +399,11 @@ Trival extract_member_lz( const Cl_options & cl_opts, } } break; case tf_directory: + { + struct stat st; + bool exists = stat( filename, &st ) == 0; + if( exists && !S_ISDIR( st.st_mode ) ) + { exists = false; std::remove( filename ); } if( !exists && mkdir( filename, mode ) != 0 && errno != EEXIST ) { if( format_file_error( rbuf, filename, mkdir_msg, errno ) && @@ -411,7 +411,7 @@ Trival extract_member_lz( const Cl_options & cl_opts, return Trival( other_msg, 0, 1 ); set_error_status( 1 ); } - break; + } break; case tf_chardev: case tf_blockdev: { @@ -483,7 +483,7 @@ Trival extract_member_lz( const Cl_options & cl_opts, if( cl_opts.keep_damaged ) { writeblock( outfd, buf, std::min( rest, (long long)ar.e_size() ) ); close( outfd ); } - else { close( outfd ); unlink( filename ); } + else { close( outfd ); std::remove( filename ); } } return Trival( ar.e_msg(), ar.e_code(), ret ); } @@ -614,18 +614,13 @@ extern "C" void * dworker( void * arg ) } if( typeflag == tf_extended ) { - std::vector< std::string > msg_vec; - const char * msg = 0; int ret = 2; bool good = false; + const char * msg = 0; int ret = 2; if( prev_extended && !cl_opts.permissive ) msg = fv_msg3; else if( ( ret = ar.parse_records( extended, header, rbuf, extrec_msg, - cl_opts.permissive, &msg_vec ) ) != 0 ) msg = ar.e_msg(); + cl_opts.permissive ) ) != 0 ) msg = ar.e_msg(); else if( !extended.crc_present() && cl_opts.missing_crc ) { msg = miscrc_msg; ret = 2; } - else { prev_extended = true; good = true; } - for( unsigned j = 0; j < msg_vec.size(); ++j ) - if( !courier.collect_packet( i, worker_id, msg_vec[j].c_str(), - Packet::diag ) ) { good = false; break; } - if( good ) continue; + else { prev_extended = true; continue; } if( courier.request_mastership( i, worker_id ) ) courier.collect_packet( i, worker_id, msg, ( ret == 1 ) ? Packet::error1 : Packet::error2 ); @@ -637,18 +632,13 @@ extern "C" void * dworker( void * arg ) /* Skip members with an empty name in the ustar header. If there is an extended header in a previous lzip member, its worker will request - mastership and the skip may fail here. Else the ustar-only unnamed - member will be ignored. */ - std::string rpmsg; // removed prefix + mastership. Else the ustar-only unnamed member will be ignored. */ Trival trival; - if( check_skip_filename( cl_opts, name_pending, extended.path().c_str(), - -1, &rpmsg ) ) + if( check_skip_filename( cl_opts, name_pending, extended.path().c_str() ) ) trival = skip_member_lz( ar, courier, extended, i, worker_id, typeflag ); else { - if( verbosity >= 0 && rpmsg.size() && - !courier.collect_packet( i, worker_id, rpmsg.c_str(), Packet::prefix ) ) - { trival = Trival( other_msg, 0, 1 ); goto fatal; } + std::string rpmsg; if( print_removed_prefix( extended.removed_prefix, &rpmsg ) && !courier.collect_packet( i, worker_id, rpmsg.c_str(), Packet::prefix ) ) { trival = Trival( other_msg, 0, 1 ); goto fatal; } @@ -677,14 +667,14 @@ done: } -/* Get from courier the processed and sorted packets. - Print the member lines on stdout and the diagnostics and errors on stderr. +/* Get from courier the processed and sorted packets, and print + the member lines on stdout or the diagnostics and errors on stderr. */ -int muxer( const char * const archive_namep, Packet_courier & courier ) +void muxer( const char * const archive_namep, Packet_courier & courier ) { std::vector< const Packet * > opacket_vector; int retval = 0; - while( retval == 0 ) // exit loop at first error packet + while( retval == 0 ) { courier.deliver_packets( opacket_vector ); if( opacket_vector.empty() ) break; // queue is empty. all workers exited @@ -708,7 +698,7 @@ int muxer( const char * const archive_namep, Packet_courier & courier ) } if( retval == 0 && !courier.eoa_found() ) // no worker found EOA blocks { show_file_error( archive_namep, end_msg ); retval = 2; } - return retval; + if( retval ) exit_fail_mt( retval ); } } // end namespace @@ -725,6 +715,8 @@ int decode_lz( const Cl_options & cl_opts, const Archive_descriptor & ad, Name_monitor name_monitor( ( cl_opts.program_mode == m_extract ) ? num_workers : 0 ); + /* If an error happens after any threads have been started, exit must be + called before courier goes out of scope. */ Packet_courier courier( num_workers, out_slots ); Worker_arg * worker_args = new( std::nothrow ) Worker_arg[num_workers]; @@ -745,7 +737,7 @@ int decode_lz( const Cl_options & cl_opts, const Archive_descriptor & ad, { show_error( "Can't create worker threads", errcode ); exit_fail_mt(); } } - int retval = muxer( ad.namep, courier ); + muxer( ad.namep, courier ); for( int i = num_workers - 1; i >= 0; --i ) { @@ -756,8 +748,9 @@ int decode_lz( const Cl_options & cl_opts, const Archive_descriptor & ad, delete[] worker_threads; delete[] worker_args; + int retval = 0; if( close( ad.infd ) != 0 ) - { show_file_error( ad.namep, eclosa_msg, errno ); set_retval( retval, 1 ); } + { show_file_error( ad.namep, eclosa_msg, errno ); retval = 1; } if( retval == 0 ) for( int i = 0; i < cl_opts.parser.arguments(); ++i ) diff --git a/delete.cc b/delete.cc index de9fe1e..8a4a40f 100644 --- a/delete.cc +++ b/delete.cc @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by diff --git a/delete_lz.cc b/delete_lz.cc index b32350e..a9c31ad 100644 --- a/delete_lz.cc +++ b/delete_lz.cc @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by diff --git a/doc/tarlz.1 b/doc/tarlz.1 index 45bcff9..c1e6dd6 100644 --- a/doc/tarlz.1 +++ b/doc/tarlz.1 @@ -1,5 +1,5 @@ .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.2. -.TH TARLZ "1" "March 2025" "tarlz 0.27" "User Commands" +.TH TARLZ "1" "December 2024" "tarlz 0.26" "User Commands" .SH NAME tarlz \- creates tar archives with multimember lzip compression .SH SYNOPSIS @@ -19,7 +19,7 @@ compressed archives. .PP Keeping the alignment between tar members and lzip members has two advantages. It adds an indexed lzip layer on top of the tar archive, making -it possible to decode the archive safely in parallel. It also reduces the +it possible to decode the archive safely in parallel. It also minimizes the amount of data lost in case of corruption. .PP The tarlz file format is a safe POSIX\-style backup format. In case of @@ -160,12 +160,12 @@ Report bugs to lzip\-bug@nongnu.org .br Tarlz home page: http://www.nongnu.org/lzip/tarlz.html .SH COPYRIGHT -Copyright \(co 2025 Antonio Diaz Diaz. +Copyright \(co 2024 Antonio Diaz Diaz. License GPLv2+: GNU GPL version 2 or later .br This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. -Using lzlib 1.15 +Using lzlib 1.15\-rc1 Using LZ_API_VERSION = 1015 .SH "SEE ALSO" The full documentation for diff --git a/doc/tarlz.info b/doc/tarlz.info index 78b7729..46d5ab4 100644 --- a/doc/tarlz.info +++ b/doc/tarlz.info @@ -11,14 +11,13 @@ File: tarlz.info, Node: Top, Next: Introduction, Up: (dir) Tarlz Manual ************ -This manual is for Tarlz (version 0.27, 28 February 2025). +This manual is for Tarlz (version 0.26, 7 December 2024). * Menu: * Introduction:: Purpose and features of tarlz * Invoking tarlz:: Command-line interface * Argument syntax:: By convention, options start with a hyphen -* Creating backups safely:: Checking integrity and accuracy of archives * Portable character set:: POSIX portable filename character set * File format:: Detailed format of the compressed archive * Amendments to pax format:: The reasons for the differences with pax @@ -30,7 +29,7 @@ This manual is for Tarlz (version 0.27, 28 February 2025). * Concept index:: Index of concepts - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This manual is free documentation: you have unlimited permission to copy, distribute, and modify it. @@ -54,7 +53,7 @@ compressed archives. Keeping the alignment between tar members and lzip members has two advantages. It adds an indexed lzip layer on top of the tar archive, making -it possible to decode the archive safely in parallel. It also reduces the +it possible to decode the archive safely in parallel. It also minimizes the amount of data lost in case of corruption. Compressing a tar archive with plzip may even double the amount of files lost for each lzip member damaged because it does not keep the members aligned. @@ -217,7 +216,7 @@ tarlz supports the following operations: '-t' '--list' List the contents of an archive. If FILES are given, list only the - FILES given. *Note mt-listing::. + FILES given. '-x' '--extract' @@ -228,23 +227,20 @@ tarlz supports the following operations: directories unconditionally before extracting over them. Other than that, it does not make any special effort to extract a file over an incompatible type of file. For example, extracting a file over a - non-empty directory usually fails. *Note mt-extraction::. + non-empty directory usually fails. '-z' '--compress' Compress existing POSIX tar archives aligning the lzip members to the tar members with choice of granularity ('--bsolid' by default, - '--dsolid' works like '--asolid'). Each input archive is compressed to - a file with the extension '.lz' added unless the option '--output' is - used. If no archives are specified, or if a hyphen '-' is used as the - name of an archive, tarlz reads from standard input and writes to - standard output (unless the option '--output' is used). When - '--output' is used, only one input archive can be specified. Exit with - error status 2 if any input archive is an empty file. The input - archives are kept unchanged. Existing compressed archives are not - overwritten. Tarlz can be used as compressor for GNU tar by using a - command like 'tar -c -Hustar foo | tarlz -z -o foo.tar.lz'. Tarlz can - be used as compressor for zupdate (zutils) by using a command like + '--dsolid' works like '--asolid'). Exit with error status 2 if any + input archive is an empty file. The input archives are kept unchanged. + Existing compressed archives are not overwritten. A hyphen '-' used as + the name of an input archive reads from standard input and writes to + standard output (unless the option '--output' is used). Tarlz can be + used as compressor for GNU tar by using a command like + 'tar -c -Hustar foo | tarlz -z -o foo.tar.lz'. Tarlz can be used as + compressor for zupdate (zutils) by using a command like 'zupdate --lz='tarlz -z' foo.tar.gz'. Note that tarlz only works reliably on archives without global headers, or with global headers whose content can be ignored. @@ -255,8 +251,11 @@ tarlz supports the following operations: archive. Unless solid compression is requested, the end-of-archive blocks are compressed in a lzip member separated from the preceding members and from any nonzero garbage following the end-of-archive - blocks. '--compress' implies plzip argument style, not tar style. '-f' - can't be used with '--compress'. + blocks. '--compress' implies plzip argument style, not tar style. Each + input archive is compressed to a file with the extension '.lz' added + unless the option '--output' is used. When '--output' is used, only + one input archive can be specified. '-f' can't be used with + '--compress'. '--check-lib' Compare the version of lzlib used to compile tarlz with the version @@ -277,9 +276,7 @@ tarlz supports the following options: *Note Argument syntax::. Set target size of input data blocks for the option '--bsolid'. *Note --bsolid::. Valid values range from 8 KiB to 1 GiB. Default value is two times the dictionary size, except for option '-0' where it - defaults to 1 MiB. *Note Minimum archive sizes::. Tarlz does not split - tar members. If a file is larger than BYTES, tarlz will create a lzip - member large enough to contain the file. + defaults to 1 MiB. *Note Minimum archive sizes::. '-C DIR' '--directory=DIR' @@ -427,12 +424,12 @@ tarlz supports the following options: *Note Argument syntax::. group ID. '--exclude=PATTERN' - Exclude files matching a shell pattern like '*.o', even if the files - are specified in the command line. A file is considered to match if any - component of the file name matches. For example, '*.o' matches - 'foo.o', 'foo.o/bar' and 'foo/bar.o'. If PATTERN contains a '/', it - matches a corresponding '/' in the file name. For example, 'foo/*.o' - matches 'foo/bar.o'. Multiple '--exclude' options can be specified. + Exclude files matching a shell pattern like '*.o'. A file is considered + to match if any component of the file name matches. For example, '*.o' + matches 'foo.o', 'foo.o/bar' and 'foo/bar.o'. If PATTERN contains a + '/', it matches a corresponding '/' in the file name. For example, + 'foo/*.o' matches 'foo/bar.o'. Multiple '--exclude' options can be + specified. '--ignore-ids' Make '--diff' ignore differences in owner and group IDs. This option is @@ -489,10 +486,10 @@ tarlz supports the following options: *Note Argument syntax::. During archive creation, warn if any file being archived has a modification time newer than the archive creation time. This option may slow archive creation somewhat because it makes an extra call to - 'stat' after archiving each file, but it nearly guarantees that file - contents were not modified during the creation of the archive. Note - that the file must be at least one second newer than the archive for - it to be detected as newer. + 'stat' after archiving each file, but it guarantees that file contents + were not modified during the creation of the archive. Note that the + file must be at least one second newer than the archive for it to be + detected as newer. Exit status: 0 for a normal exit, 1 for environmental problems (file not @@ -501,7 +498,7 @@ indicate a corrupt or invalid input file, 3 for an internal consistency error (e.g., bug) which caused tarlz to panic.  -File: tarlz.info, Node: Argument syntax, Next: Creating backups safely, Prev: Invoking tarlz, Up: Top +File: tarlz.info, Node: Argument syntax, Next: Portable character set, Prev: Invoking tarlz, Up: Top 3 Syntax of command-line arguments ********************************** @@ -544,55 +541,9 @@ GNU adds "long options" to these conventions: Thus, '--foo bar' and '--foo=bar' are equivalent.  -File: tarlz.info, Node: Creating backups safely, Next: Portable character set, Prev: Argument syntax, Up: Top +File: tarlz.info, Node: Portable character set, Next: File format, Prev: Argument syntax, Up: Top -4 Checking the integrity and accuracy of tar.lz archives -******************************************************** - -Uncompressed tar archives do not offer any integrity checking for the files -they store. The pax format even fails to offer integrity checking for some -of the metadata. *Note crc32::. The integrity checking of tar archives is -usually provided by a compression layer or by an external hash. - - Lzip compression provides safe integrity checking to tar archives. But it -does not matter how safe is the archiving format if the archive is created -corrupt because of a concurrent modification of the files being archived, a -faulty RAM, or a bug in the archiving tool. The only way of guaranteeing -that a backup archive is correct is to check its integrity and accuracy -after creating it. - - Testing the integrity of the archive with 'lzip -tv' guarantees that the -compression layer of the archive is valid, but it does not guarantee that -the tar layer is valid nor that the files in the archive match the files in -the file system. For example, if the RAM is faulty and a bit flip happens -in the input buffer before tarlz compresses it, the archive will not match -the files. It is safer to check the archive with 'tarlz -d' just after -creation because it checks the compression layer and the tar layer, and it -compares the files in the archive with the files in the file system: - - tarlz -cf archive.tar.lz somedir # create the archive - tarlz -df archive.tar.lz # check the archive - - Once the integrity and accuracy of an archive have been verified as in -the example above, they can be verified again anywhere at any time with -'tarlz -t -n0'. It is important to disable multi-threading with '-n0' -because multi-threaded listing does not detect corruption in the tar member -data of multimember archives: *Note mt-listing::. - - tarlz -t -n0 -f archive.tar.lz > /dev/null - - 'lzip -tv' checks the integrity of the compression layer, and therefore -the integrity and accuracy of any archive created and verified as explained -above. This test is reliable for solidly compressed archives, but it does -not detect a truncated multimember archive if the truncation happens just -at a member boundary: - - lzip -tv archive.tar.lz - - -File: tarlz.info, Node: Portable character set, Next: File format, Prev: Creating backups safely, Up: Top - -5 POSIX portable filename character set +4 POSIX portable filename character set *************************************** The set of characters from which portable file names are constructed. @@ -610,7 +561,7 @@ names use only the portable character set without spaces added.  File: tarlz.info, Node: File format, Next: Amendments to pax format, Prev: Portable character set, Up: Top -6 File format +5 File format ************* In the diagram below, a box like this: @@ -681,7 +632,7 @@ tar.lz | member | member | member | +===============+=================================================+========+ -6.1 Pax header block +5.1 Pax header block ==================== The pax header block is identical to the ustar header block described below @@ -725,7 +676,7 @@ space, equal-sign, and newline. previously archived. This record overrides the field 'linkname' in the following ustar header block. The following ustar header block determines the type of link created. If typeflag of the following - header block is '1', a hard link is created. If typeflag is '2', a + header block is 1, a hard link is created. If typeflag is 2, a symbolic link is created and the linkpath value is used as the contents of the symbolic link. The linkpath record is created only for links with a link name that does not fit in the space provided by the @@ -765,17 +716,17 @@ space, equal-sign, and newline. CRC32-C (Castagnoli) of the extended header data excluding the 8 bytes representing the CRC itself. The is represented as 8 hexadecimal digits in big endian order, '22 GNU.crc32=00000000\n'. The - option '--missing-crc' guarantees that corruption is always detected - (except in case of CRC collision). A CRC was chosen because a checksum - is too weak for a potentially large list of variable sized records. A + keyword of the CRC record is protected by the CRC to guarantee that + corruption is always detected when using '--missing-crc' (except in + case of CRC collision). A CRC was chosen because a checksum is too + weak for a potentially large list of variable sized records. A checksum can't detect simple errors like the swapping of two bytes. - *Note --missing-crc::. At verbosity level 1 or higher tarlz prints a diagnostic for each unknown extended header keyword found in an archive, once per keyword. -6.2 Ustar header block +5.2 Ustar header block ====================== The ustar header block has a length of 512 bytes and is structured as shown @@ -799,7 +750,6 @@ gname 297 32 devmajor 329 8 devminor 337 8 prefix 345 155 -padding 500 12 All characters in the header block are coded using the ISO/IEC 646:1991 (ASCII) standard, except in fields storing names for files, users, and @@ -889,7 +839,7 @@ file archived: ''7'' Reserved to represent a file to which an implementation has associated some high-performance attribute (contiguous file). Tarlz treats this - type of file as a regular file (type '0'). + type of file as a regular file (type 0). The field 'magic' contains the ASCII null-terminated string "ustar". The @@ -898,13 +848,13 @@ field 'version' contains the characters "00" (0x30,0x30). The fields characters in the array contain non-null characters including the last character. Each numeric field contains a leading space- or zero-filled, optionally null-terminated octal number using digits from the ISO/IEC -646:1991 (ASCII) standard. Tarlz is able to decode numeric fields one byte +646:1991 (ASCII) standard. Tarlz is able to decode numeric fields 1 byte longer than standard ustar by not requiring a terminating null character.  File: tarlz.info, Node: Amendments to pax format, Next: Program design, Prev: File format, Up: Top -7 The reasons for the differences with pax +6 The reasons for the differences with pax ****************************************** Tarlz creates safe archives that allow the reliable detection of invalid or @@ -915,7 +865,7 @@ achieve this goal and avoid some other flaws in the pax format, tarlz makes some changes to the variant of the pax format that it uses. This chapter describes these changes and the concrete reasons to implement them. -7.1 Add a CRC of the extended records +6.1 Add a CRC of the extended records ===================================== The POSIX pax format has a serious flaw. The metadata stored in pax extended @@ -942,7 +892,7 @@ place. Redundancy Check (CRC) in a way compatible with standard tar tools. *Note key_crc32::. -7.2 Remove flawed backward compatibility +6.2 Remove flawed backward compatibility ======================================== In order to allow the extraction of pax archives by a tar utility conforming @@ -975,7 +925,7 @@ trying to extract the file or link. This also makes easier during parallel decoding the detection of a tar member split between two lzip members at the boundary between the extended header and the ustar header. -7.3 As simple as possible (but not simpler) +6.3 As simple as possible (but not simpler) =========================================== The tarlz format is mainly ustar. Extended pax headers are used only when @@ -990,7 +940,7 @@ corruption. ignored. Some operations may not behave as expected if the archive contains global headers. -7.4 Improve reproducibility +6.4 Improve reproducibility =========================== Pax includes by default the process ID of the pax process in the ustar name @@ -1002,7 +952,7 @@ extended records, making it easier to produce reproducible archives. ten; '99<97_bytes>' or '100<97_bytes>'. Tarlz minimizes the length of the record and always produces a length of x-1 in these cases. -7.5 No data in hard links +6.5 No data in hard links ========================= Tarlz does not allow data in hard link members. The data (if any) must be in @@ -1011,26 +961,27 @@ the names of a file are stored as hard links, the type of the file is lost. Not allowing data in hard links also prevents invalid actions like extracting file data for a hard link to a symbolic link or to a directory. -7.6 Avoid misconversions to/from UTF-8 +6.6 Avoid misconversions to/from UTF-8 ====================================== There is no portable way to tell what charset a text string is coded into. Therefore, tarlz stores all fields representing text strings unmodified, without conversion to UTF-8 nor any other transformation. This prevents -accidental double UTF-8 conversions. +accidental double UTF-8 conversions. If the need arises this behavior will +be adjusted with a command-line option in the future.  File: tarlz.info, Node: Program design, Next: Multi-threaded decoding, Prev: Amendments to pax format, Up: Top -8 Internal structure of tarlz +7 Internal structure of tarlz ***************************** The parts of tarlz related to sequential processing of the archive are more or less similar to any other tar and won't be described here. The -interesting parts described here are those related to multi-threaded +interesting parts described here are those related to Multi-threaded processing. - The structure of the part of tarlz performing multi-threaded archive + The structure of the part of tarlz performing Multi-threaded archive creation is somewhat similar to that of plzip with the added complication of the solidity levels. *Note Program design: (plzip)Program design. A grouper thread and several worker threads are created, acting the main @@ -1102,7 +1053,7 @@ error be avoided.  File: tarlz.info, Node: Multi-threaded decoding, Next: Minimum archive sizes, Prev: Program design, Up: Top -9 Limitations of parallel tar decoding +8 Limitations of parallel tar decoding ************************************** Safely decoding a tar archive in parallel is only possible if one decodes @@ -1142,14 +1093,11 @@ tar.lz archives, keeping backwards compatibility. If tarlz finds a member misalignment during multi-threaded decoding, it switches to single-threaded mode and continues decoding the archive. -9.1 Multi-threaded listing -========================== - -If the files in the archive are large, multi-threaded '--list' on a regular -(seekable) tar.lz archive can be hundreds of times faster than sequential -'--list' because, in addition to using several processors, it only needs to -decompress part of each lzip member. See the following example listing the -Silesia corpus on a dual core machine: + If the files in the archive are large, multi-threaded '--list' on a +regular (seekable) tar.lz archive can be hundreds of times faster than +sequential '--list' because, in addition to using several processors, it +only needs to decompress part of each lzip member. See the following +example listing the Silesia corpus on a dual core machine: tarlz -9 --no-solid -cf silesia.tar.lz silesia time lzip -cd silesia.tar.lz | tar -tf - (5.032s) @@ -1158,12 +1106,10 @@ Silesia corpus on a dual core machine: On the other hand, multi-threaded '--list' won't detect corruption in the tar member data because it only decodes the part of each lzip member -corresponding to the tar member header. Partial decoding of a lzip member -can't guarantee the integrity of the data decoded. This is another reason -why the tar headers (including the extended records) must provide their own -integrity checking. +corresponding to the tar member header. This is another reason why the tar +headers must provide their own integrity checking. -9.2 Limitations of multi-threaded extraction +8.1 Limitations of multi-threaded extraction ============================================ Multi-threaded extraction may produce different output than single-threaded @@ -1193,8 +1139,8 @@ links to.  File: tarlz.info, Node: Minimum archive sizes, Next: Examples, Prev: Multi-threaded decoding, Up: Top -10 Minimum archive sizes required for multi-threaded block compression -********************************************************************** +9 Minimum archive sizes required for multi-threaded block compression +********************************************************************* When creating or appending to a compressed archive using multi-threaded block compression, tarlz puts tar members together in blocks and compresses @@ -1231,7 +1177,7 @@ Level  File: tarlz.info, Node: Examples, Next: Problems, Prev: Minimum archive sizes, Up: Top -11 A small tutorial with examples +10 A small tutorial with examples ********************************* Example 1: Create a multimember compressed archive 'archive.tar.lz' @@ -1287,12 +1233,10 @@ other members can still be extracted). tarlz -z --no-solid archive.tar -Example 10: Recompress the archive 'archive.tar.lz' with different -solidity, write the output to 'archive-ns.tar.lz', and compare both -archives. +Example 10: Compress the archive 'archive.tar' and write the output to +'foo.tar.lz'. - lzip -cd archive.tar.lz | tarlz -9z --no-solid -o archive-ns.tar.lz - zcmp archive.tar.lz archive-ns.tar.lz + tarlz -z -o foo.tar.lz archive.tar Example 11: Concatenate and compress two archives 'archive1.tar' and 'archive2.tar', and write the output to 'foo.tar.lz'. @@ -1302,7 +1246,7 @@ Example 11: Concatenate and compress two archives 'archive1.tar' and  File: tarlz.info, Node: Problems, Next: Concept index, Prev: Examples, Up: Top -12 Reporting bugs +11 Reporting bugs ***************** There are probably bugs in tarlz. There are certainly errors and omissions @@ -1326,7 +1270,6 @@ Concept index * Amendments to pax format: Amendments to pax format. (line 6) * argument syntax: Argument syntax. (line 6) * bugs: Problems. (line 6) -* creating backups: Creating backups safely. (line 6) * examples: Examples. (line 6) * file format: File format. (line 6) * getting help: Problems. (line 6) @@ -1344,29 +1287,26 @@ Concept index  Tag Table: Node: Top216 -Node: Introduction1356 -Node: Invoking tarlz4179 -Ref: --data-size13265 -Ref: --bsolid17924 -Ref: --missing-crc21532 -Node: Argument syntax23897 -Node: Creating backups safely25673 -Node: Portable character set28057 -Node: File format28709 -Ref: key_crc3235756 -Ref: ustar-uid-gid39052 -Ref: ustar-mtime39859 -Node: Amendments to pax format41866 -Ref: crc3242574 -Ref: flawed-compat43885 -Node: Program design47870 -Node: Multi-threaded decoding51797 -Ref: mt-listing54198 -Ref: mt-extraction55236 -Node: Minimum archive sizes56542 -Node: Examples58671 -Node: Problems61166 -Node: Concept index61721 +Node: Introduction1281 +Node: Invoking tarlz4106 +Ref: --data-size13109 +Ref: --bsolid17626 +Node: Argument syntax23539 +Node: Portable character set25314 +Node: File format25958 +Ref: key_crc3233001 +Ref: ustar-uid-gid36305 +Ref: ustar-mtime37112 +Node: Amendments to pax format39115 +Ref: crc3239823 +Ref: flawed-compat41134 +Node: Program design45211 +Node: Multi-threaded decoding49138 +Ref: mt-extraction52407 +Node: Minimum archive sizes53713 +Node: Examples55840 +Node: Problems58199 +Node: Concept index58754  End Tag Table diff --git a/doc/tarlz.texi b/doc/tarlz.texi index 40ee137..79c145d 100644 --- a/doc/tarlz.texi +++ b/doc/tarlz.texi @@ -6,8 +6,8 @@ @finalout @c %**end of header -@set UPDATED 28 February 2025 -@set VERSION 0.27 +@set UPDATED 7 December 2024 +@set VERSION 0.26 @dircategory Archiving @direntry @@ -39,7 +39,6 @@ This manual is for Tarlz (version @value{VERSION}, @value{UPDATED}). * Introduction:: Purpose and features of tarlz * Invoking tarlz:: Command-line interface * Argument syntax:: By convention, options start with a hyphen -* Creating backups safely:: Checking integrity and accuracy of archives * Portable character set:: POSIX portable filename character set * File format:: Detailed format of the compressed archive * Amendments to pax format:: The reasons for the differences with pax @@ -52,7 +51,7 @@ This manual is for Tarlz (version @value{VERSION}, @value{UPDATED}). @end menu @sp 1 -Copyright @copyright{} 2013-2025 Antonio Diaz Diaz. +Copyright @copyright{} 2013-2024 Antonio Diaz Diaz. This manual is free documentation: you have unlimited permission to copy, distribute, and modify it. @@ -77,7 +76,7 @@ compressed archives. Keeping the alignment between tar members and lzip members has two advantages. It adds an indexed lzip layer on top of the tar archive, making -it possible to decode the archive safely in parallel. It also reduces the +it possible to decode the archive safely in parallel. It also minimizes the amount of data lost in case of corruption. Compressing a tar archive with plzip may even double the amount of files lost for each lzip member damaged because it does not keep the members aligned. @@ -255,7 +254,7 @@ during multi-threaded extraction. @xref{mt-extraction}. @item -t @itemx --list List the contents of an archive. If @var{files} are given, list only the -@var{files} given. @xref{mt-listing}. +@var{files} given. @item -x @itemx --extract @@ -266,23 +265,20 @@ directory without extracting the files under it, use empty directories unconditionally before extracting over them. Other than that, it does not make any special effort to extract a file over an incompatible type of file. For example, extracting a file over a non-empty -directory usually fails. @xref{mt-extraction}. +directory usually fails. @item -z @itemx --compress Compress existing POSIX tar archives aligning the lzip members to the tar members with choice of granularity (@option{--bsolid} by default, -@option{--dsolid} works like @option{--asolid}). Each input archive is -compressed to a file with the extension @file{.lz} added unless the option -@option{--output} is used. If no archives are specified, or if a hyphen -@samp{-} is used as the name of an archive, tarlz reads from standard input -and writes to standard output (unless the option @option{--output} is used). -When @option{--output} is used, only one input archive can be specified. -Exit with error status 2 if any input archive is an empty file. The input -archives are kept unchanged. Existing compressed archives are not -overwritten. Tarlz can be used as compressor for GNU tar by using a command -like @w{@samp{tar -c -Hustar foo | tarlz -z -o foo.tar.lz}}. Tarlz can be -used as compressor for zupdate (zutils) by using a command like +@option{--dsolid} works like @option{--asolid}). Exit with error status 2 if +any input archive is an empty file. The input archives are kept unchanged. +Existing compressed archives are not overwritten. A hyphen @samp{-} used as +the name of an input archive reads from standard input and writes to +standard output (unless the option @option{--output} is used). Tarlz can be +used as compressor for GNU tar by using a command like +@w{@samp{tar -c -Hustar foo | tarlz -z -o foo.tar.lz}}. Tarlz can be used as +compressor for zupdate (zutils) by using a command like @w{@samp{zupdate --lz='tarlz -z' foo.tar.gz}}. Note that tarlz only works reliably on archives without global headers, or with global headers whose content can be ignored. @@ -293,8 +289,10 @@ block is found, and then compresses the rest of the archive. Unless solid compression is requested, the end-of-archive blocks are compressed in a lzip member separated from the preceding members and from any nonzero garbage following the end-of-archive blocks. @option{--compress} implies plzip -argument style, not tar style. @option{-f} can't be used with -@option{--compress}. +argument style, not tar style. Each input archive is compressed to a file +with the extension @file{.lz} added unless the option @option{--output} is +used. When @option{--output} is used, only one input archive can be specified. +@option{-f} can't be used with @option{--compress}. @item --check-lib Compare the @@ -321,10 +319,8 @@ tarlz supports the following options: @xref{Argument syntax}. @itemx --data-size=@var{bytes} Set target size of input data blocks for the option @option{--bsolid}. @xref{--bsolid}. Valid values range from @w{8 KiB} to @w{1 GiB}. Default -value is two times the dictionary size, except for option @option{-0} where -it defaults to @w{1 MiB}. @xref{Minimum archive sizes}. Tarlz does not split -tar members. If a file is larger than @var{bytes}, tarlz will create a lzip -member large enough to contain the file. +value is two times the dictionary size, except for option @option{-0} where it +defaults to @w{1 MiB}. @xref{Minimum archive sizes}. @item -C @var{dir} @itemx --directory=@var{dir} @@ -469,13 +465,12 @@ If @var{group} is not a valid group name, it is decoded as a decimal numeric group ID. @item --exclude=@var{pattern} -Exclude files matching a shell pattern like @file{*.o}, even if the files -are specified in the command line. A file is considered to match if any -component of the file name matches. For example, @file{*.o} matches -@file{foo.o}, @file{foo.o/bar} and @file{foo/bar.o}. If @var{pattern} -contains a @samp{/}, it matches a corresponding @samp{/} in the file name. -For example, @file{foo/*.o} matches @file{foo/bar.o}. Multiple -@option{--exclude} options can be specified. +Exclude files matching a shell pattern like @file{*.o}. A file is considered +to match if any component of the file name matches. For example, @file{*.o} +matches @file{foo.o}, @file{foo.o/bar} and @file{foo/bar.o}. If +@var{pattern} contains a @samp{/}, it matches a corresponding @samp{/} in +the file name. For example, @file{foo/*.o} matches @file{foo/bar.o}. +Multiple @option{--exclude} options can be specified. @item --ignore-ids Make @option{--diff} ignore differences in owner and group IDs. This option is @@ -498,7 +493,6 @@ recover as much data as possible from each damaged member. It is recommended to run tarlz in single-threaded mode (@option{--threads=0}) when using this option. -@anchor{--missing-crc} @item --missing-crc Exit with error status 2 if the CRC of the extended records is missing. When this option is used, tarlz detects any corruption in the extended records @@ -531,9 +525,9 @@ values range from 1 to 1024. The default value is 64. During archive creation, warn if any file being archived has a modification time newer than the archive creation time. This option may slow archive creation somewhat because it makes an extra call to @samp{stat} after -archiving each file, but it nearly guarantees that file contents were not -modified during the creation of the archive. Note that the file must be at -least one second newer than the archive for it to be detected as newer. +archiving each file, but it guarantees that file contents were not modified +during the creation of the archive. Note that the file must be at least one +second newer than the archive for it to be detected as newer. @ignore @item --permissive @@ -597,58 +591,6 @@ Thus, @w{@option{--foo bar}} and @option{--foo=bar} are equivalent. @end itemize -@node Creating backups safely -@chapter Checking the integrity and accuracy of tar.lz archives -@cindex creating backups - -Uncompressed tar archives do not offer any integrity checking for the files -they store. The pax format even fails to offer integrity checking for some -of the metadata. @xref{crc32}. The integrity checking of tar archives is -usually provided by a compression layer or by an external hash. - -Lzip compression provides safe integrity checking to tar archives. But it -does not matter how safe is the archiving format if the archive is created -corrupt because of a concurrent modification of the files being archived, a -faulty RAM, or a bug in the archiving tool. The only way of guaranteeing -that a backup archive is correct is to check its integrity and accuracy -after creating it. - -Testing the integrity of the archive with @w{@samp{lzip -tv}} guarantees -that the compression layer of the archive is valid, but it does not -guarantee that the tar layer is valid nor that the files in the archive -match the files in the file system. For example, if the RAM is faulty and a -bit flip happens in the input buffer before tarlz compresses it, the archive -will not match the files. It is safer to check the archive with -@w{@samp{tarlz -d}} just after creation because it checks the compression -layer and the tar layer, and it compares the files in the archive with the -files in the file system: - -@example -tarlz -cf archive.tar.lz somedir # create the archive -tarlz -df archive.tar.lz # check the archive -@end example - -Once the integrity and accuracy of an archive have been verified as in the -example above, they can be verified again anywhere at any time with -@w{@samp{tarlz -t -n0}}. It is important to disable multi-threading with -@option{-n0} because multi-threaded listing does not detect corruption in -the tar member data of multimember archives: @xref{mt-listing}. - -@example -tarlz -t -n0 -f archive.tar.lz > /dev/null -@end example - -@w{@samp{lzip -tv}} checks the integrity of the compression layer, and -therefore the integrity and accuracy of any archive created and verified as -explained above. This test is reliable for solidly compressed archives, but -it does not detect a truncated multimember archive if the truncation happens -just at a member boundary: - -@example -lzip -tv archive.tar.lz -@end example - - @node Portable character set @chapter POSIX portable filename character set @cindex portable character set @@ -699,7 +641,7 @@ are not allowed in multimember files. Each lzip member contains one or more tar members in a simplified POSIX pax interchange format. The only pax typeflag value supported by tarlz (in -addition to the typeflag values defined by the ustar format) is 'x'. +addition to the typeflag values defined by the ustar format) is @samp{x}. The pax format is an extension on top of the ustar format that removes the size limitations of the ustar format. @@ -712,7 +654,7 @@ An optional extended header block followed by one or more blocks that contain the extended header records as if they were the contents of a file; i.e., the extended header records are included as the data for this header block. This header block is of the form described in pax header block, with -a typeflag value of 'x'. +a typeflag value of @samp{x}. @item A header block in ustar format that describes the file. Any fields defined @@ -771,7 +713,7 @@ An extended header just before the end-of-archive blocks. @section Pax header block The pax header block is identical to the ustar header block described below -except that the typeflag has the value 'x' (extended). The field +except that the typeflag has the value @samp{x} (extended). The field @samp{size} is the size of the extended header data in bytes. Most other fields in the pax header block are zeroed on archive creation to prevent trouble if the archive is read by a ustar tool, and are ignored by tarlz on @@ -810,8 +752,8 @@ greater than 2_097_151 @w{(octal 7_777_777)}. @xref{ustar-uid-gid}. The file name of a link being created to another file, of any type, previously archived. This record overrides the field @samp{linkname} in the following ustar header block. The following ustar header block determines -the type of link created. If typeflag of the following header block is '1', a -hard link is created. If typeflag is '2', a symbolic link is created and the +the type of link created. If typeflag of the following header block is 1, a +hard link is created. If typeflag is 2, a symbolic link is created and the linkpath value is used as the contents of the symbolic link. The linkpath record is created only for links with a link name that does not fit in the space provided by the ustar header. @@ -847,12 +789,13 @@ greater than 2_097_151 @w{(octal 7_777_777)}. @xref{ustar-uid-gid}. @item GNU.crc32 CRC32-C (Castagnoli) of the extended header data excluding the 8 bytes representing the CRC itself. The is represented as 8 -hexadecimal digits in big endian order, @w{@samp{22 GNU.crc32=00000000\n}}. -The option @option{--missing-crc} guarantees that corruption is always -detected (except in case of CRC collision). A CRC was chosen because a -checksum is too weak for a potentially large list of variable sized records. -A checksum can't detect simple errors like the swapping of two bytes. -@xref{--missing-crc}. +hexadecimal digits in big endian order, +@w{@samp{22 GNU.crc32=00000000\n}}. The keyword of the CRC record is +protected by the CRC to guarantee that corruption is always detected when +using @option{--missing-crc} (except in case of CRC collision). A CRC was +chosen because a checksum is too weak for a potentially large list of +variable sized records. A checksum can't detect simple errors like the +swapping of two bytes. @end table @@ -882,7 +825,6 @@ shown in the following table. All lengths and offsets are in decimal: @item devmajor @tab 329 @tab 8 @item devminor @tab 337 @tab 8 @item prefix @tab 345 @tab 155 -@item padding @tab 500 @tab 12 @end multitable All characters in the header block are coded using the ISO/IEC 646:1991 @@ -977,7 +919,7 @@ FIFO special file. @item '7' Reserved to represent a file to which an implementation has associated some high-performance attribute (contiguous file). Tarlz treats this type of file -as a regular file (type '0'). +as a regular file (type 0). @end table @@ -988,8 +930,8 @@ except when all characters in the array contain non-null characters including the last character. Each numeric field contains a leading space- or zero-filled, optionally null-terminated octal number using digits from the ISO/IEC 646:1991 (ASCII) standard. Tarlz is able to decode numeric -fields one byte longer than standard ustar by not requiring a terminating -null character. +fields 1 byte longer than standard ustar by not requiring a terminating null +character. @node Amendments to pax format @@ -1102,7 +1044,8 @@ extracting file data for a hard link to a symbolic link or to a directory. There is no portable way to tell what charset a text string is coded into. Therefore, tarlz stores all fields representing text strings unmodified, without conversion to UTF-8 nor any other transformation. This prevents -accidental double UTF-8 conversions. +accidental double UTF-8 conversions. If the need arises this behavior will +be adjusted with a command-line option in the future. @node Program design @@ -1111,12 +1054,12 @@ accidental double UTF-8 conversions. The parts of tarlz related to sequential processing of the archive are more or less similar to any other tar and won't be described here. The interesting -parts described here are those related to multi-threaded processing. +parts described here are those related to Multi-threaded processing. -The structure of the part of tarlz performing multi-threaded archive +The structure of the part of tarlz performing Multi-threaded archive creation is somewhat similar to that of -@uref{http://www.nongnu.org/lzip/manual/plzip_manual.html#Program-design,,plzip} -with the added complication of the solidity levels. +@uref{http://www.nongnu.org/lzip/plzip.html#Program-design,,plzip} with the +added complication of the solidity levels. @ifnothtml @xref{Program design,,,plzip}. @end ifnothtml @@ -1231,9 +1174,6 @@ tar.lz archives, keeping backwards compatibility. If tarlz finds a member misalignment during multi-threaded decoding, it switches to single-threaded mode and continues decoding the archive. -@anchor{mt-listing} -@section Multi-threaded listing - If the files in the archive are large, multi-threaded @option{--list} on a regular (seekable) tar.lz archive can be hundreds of times faster than sequential @option{--list} because, in addition to using several processors, @@ -1249,10 +1189,8 @@ time tarlz -tf silesia.tar.lz (0.020s) On the other hand, multi-threaded @option{--list} won't detect corruption in the tar member data because it only decodes the part of each lzip member -corresponding to the tar member header. Partial decoding of a lzip member -can't guarantee the integrity of the data decoded. This is another reason -why the tar headers (including the extended records) must provide their own -integrity checking. +corresponding to the tar member header. This is another reason why the tar +headers must provide their own integrity checking. @anchor{mt-extraction} @section Limitations of multi-threaded extraction @@ -1406,13 +1344,11 @@ tarlz -z --no-solid archive.tar @end example @noindent -Example 10: Recompress the archive @file{archive.tar.lz} with different -solidity, write the output to @file{archive-ns.tar.lz}, and compare both -archives. +Example 10: Compress the archive @file{archive.tar} and write the output to +@file{foo.tar.lz}. @example -lzip -cd archive.tar.lz | tarlz -9z --no-solid -o archive-ns.tar.lz -zcmp archive.tar.lz archive-ns.tar.lz +tarlz -z -o foo.tar.lz archive.tar @end example @noindent diff --git a/exclude.cc b/exclude.cc index 8854e0e..44a53a5 100644 --- a/exclude.cc +++ b/exclude.cc @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by diff --git a/extended.cc b/extended.cc index daccbbf..0dfba9b 100644 --- a/extended.cc +++ b/extended.cc @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -22,7 +22,6 @@ #include #include "tarlz.h" -#include "common_mutex.h" const CRC32 crc32c( true ); @@ -59,9 +58,9 @@ long long parse_decimal( const char * const ptr, const char ** const tailp, } -unsigned parse_record_crc( const char * const ptr ) +uint32_t parse_record_crc( const char * const ptr ) { - unsigned crc = 0; + uint32_t crc = 0; for( int i = 0; i < 8; ++i ) { crc <<= 4; @@ -202,25 +201,16 @@ void Extended::calculate_sizes() const // print a diagnostic for each unknown keyword once per keyword -void Extended::unknown_keyword( const char * const buf, const int size, - std::vector< std::string > * const msg_vecp ) const +void Extended::unknown_keyword( const char * const buf, const int size ) const { - // prevent two threads from modifying the list of keywords at the same time - static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; int eq_pos = 0; // position of '=' in buf - while( eq_pos < size && buf[eq_pos] != '=' ) ++eq_pos; const std::string keyword( buf, eq_pos ); - xlock( &mutex ); for( unsigned i = 0; i < unknown_keywords.size(); ++i ) - if( keyword == unknown_keywords[i] ) { xunlock( &mutex ); return; } + if( keyword == unknown_keywords[i] ) return; unknown_keywords.push_back( keyword ); - xunlock( &mutex ); - const char * str = "Ignoring unknown extended header keyword '%s'"; - if( !msg_vecp ) print_error( 0, str, keyword.c_str() ); - else - { msg_vecp->push_back( std::string() ); - format_error( msg_vecp->back(), 0, str, keyword.c_str() ); } + print_error( 0, "Ignoring unknown extended header keyword '%s'", + keyword.c_str() ); } @@ -291,8 +281,7 @@ const char * Extended::full_size_error() const bool Extended::parse( const char * const buf, const int edsize, - const bool permissive, - std::vector< std::string > * const msg_vecp ) + const bool permissive ) { reset(); full_size_ = -4; // invalidate cached sizes for( int pos = 0; pos < edsize; ) // parse records @@ -359,22 +348,18 @@ bool Extended::parse( const char * const buf, const int edsize, if( crc_present_ && !permissive ) return false; if( rsize != (int)crc_record.size() ) return false; crc_present_ = true; - const unsigned stored_crc = parse_record_crc( tail + 10 ); - const unsigned computed_crc = + const uint32_t stored_crc = parse_record_crc( tail + 10 ); + const uint32_t computed_crc = crc32c.windowed_crc( (const uint8_t *)buf, pos + rsize - 9, edsize ); if( stored_crc != computed_crc ) { - if( verbosity < 1 ) return false; - const char * str = "CRC mismatch in extended records; stored %08X, computed %08X"; - if( !msg_vecp ) print_error( 0, str, stored_crc, computed_crc ); - else - { msg_vecp->push_back( std::string() ); - format_error( msg_vecp->back(), 0, str, stored_crc, computed_crc ); } + if( verbosity >= 2 ) + std::fprintf( stderr, "CRC32-C = %08X\n", (unsigned)computed_crc ); return false; } } else if( ( rest < 8 || std::memcmp( tail, "comment=", 8 ) != 0 ) && - verbosity >= 1 ) unknown_keyword( tail, rest, msg_vecp ); + verbosity >= 1 ) unknown_keyword( tail, rest ); pos += rsize; } return true; diff --git a/lzip_index.cc b/lzip_index.cc index 9d9c43a..a8e7333 100644 --- a/lzip_index.cc +++ b/lzip_index.cc @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by diff --git a/lzip_index.h b/lzip_index.h index 81ea3fd..de8fe19 100644 --- a/lzip_index.h +++ b/lzip_index.h @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by diff --git a/main.cc b/main.cc index 013c43f..b77ad4d 100644 --- a/main.cc +++ b/main.cc @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -57,7 +57,7 @@ const char * const program_name = "tarlz"; namespace { -const char * const program_year = "2025"; +const char * const program_year = "2024"; const char * invocation_name = program_name; // default value @@ -74,7 +74,7 @@ void show_help( const long num_online ) "compressed archives.\n" "\nKeeping the alignment between tar members and lzip members has two\n" "advantages. It adds an indexed lzip layer on top of the tar archive, making\n" - "it possible to decode the archive safely in parallel. It also reduces the\n" + "it possible to decode the archive safely in parallel. It also minimizes the\n" "amount of data lost in case of corruption.\n" "\nThe tarlz file format is a safe POSIX-style backup format. In case of\n" "corruption, tarlz can extract all the undamaged members from the tar.lz\n" @@ -457,43 +457,17 @@ bool format_error( Resizable_buffer & rbuf, const int errcode, for( int i = 0; i < 2; ++i ) // resize rbuf if not large enough { int len = snprintf( rbuf(), rbuf.size(), "%s: ", program_name ); - if( !rbuf.resize( len + 1 ) ) break; + if( len >= (int)rbuf.size() && !rbuf.resize( len + 1 ) ) break; va_start( args, format ); len += vsnprintf( rbuf() + len, rbuf.size() - len, format, args ); va_end( args ); - if( !rbuf.resize( len + 2 ) ) break; - if( errcode <= 0 ) { rbuf()[len++] = '\n'; rbuf()[len] = 0; } + if( len >= (int)rbuf.size() && !rbuf.resize( len + 1 ) ) break; + if( errcode <= 0 ) rbuf()[len++] = '\n'; else len += snprintf( rbuf() + len, rbuf.size() - len, ": %s\n", std::strerror( errcode ) ); - if( len < (int)rbuf.size() ) return true; - if( i > 0 || !rbuf.resize( len + 1 ) ) break; + if( len < (int)rbuf.size() || !rbuf.resize( len + 1 ) ) break; } - return false; - } - - -bool format_error( std::string & msg, const int errcode, - const char * const format, ... ) - { - if( verbosity < 0 ) return false; - Resizable_buffer rbuf; - if( !rbuf.size() ) return false; - va_list args; - for( int i = 0; i < 2; ++i ) // resize rbuf if not large enough - { - int len = snprintf( rbuf(), rbuf.size(), "%s: ", program_name ); - if( !rbuf.resize( len + 1 ) ) break; - va_start( args, format ); - len += vsnprintf( rbuf() + len, rbuf.size() - len, format, args ); - va_end( args ); - if( !rbuf.resize( len + 2 ) ) break; - if( errcode <= 0 ) { rbuf()[len++] = '\n'; rbuf()[len] = 0; } - else len += snprintf( rbuf() + len, rbuf.size() - len, ": %s\n", - std::strerror( errcode ) ); - if( len < (int)rbuf.size() ) { msg.assign( rbuf(), len ); return true; } - if( i > 0 || !rbuf.resize( len + 1 ) ) break; - } - return false; + return true; } diff --git a/tarlz.h b/tarlz.h index c0c5007..5c91cfa 100644 --- a/tarlz.h +++ b/tarlz.h @@ -1,5 +1,5 @@ /* Tarlz - Archiver with multimember lzip compression - Copyright (C) 2013-2025 Antonio Diaz Diaz. + Copyright (C) 2013-2024 Antonio Diaz Diaz. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -120,7 +120,7 @@ public: } return true; } - char * operator()() { return p; } // no need for operator[] + char * operator()() { return p; } const char * operator()() const { return p; } uint8_t * u8() { return (uint8_t *)p; } const uint8_t * u8() const { return (const uint8_t *)p; } @@ -184,8 +184,7 @@ class Extended // stores metadata from/for extended records mutable bool crc_present_; void calculate_sizes() const; - void unknown_keyword( const char * const buf, const int size, - std::vector< std::string > * const msg_vecp = 0 ) const; + void unknown_keyword( const char * const buf, const int size ) const; public: static const std::string crc_record; @@ -235,8 +234,7 @@ public: bool crc_present() const { return crc_present_; } bool parse( const char * const buf, const int edsize, - const bool permissive, - std::vector< std::string > * const msg_vecp = 0 ); + const bool permissive ); void fill_from_ustar( const Tar_header header ); }; @@ -281,7 +279,7 @@ public: return crc ^ 0xFFFFFFFFU; } - // compute the crc of size bytes except a window of 8 bytes at pos + // Calculates the crc of size bytes except a window of 8 bytes at pos uint32_t windowed_crc( const uint8_t * const buffer, const int pos, const int size ) const { @@ -487,9 +485,8 @@ const char * const posix_lz_msg = "This does not look like a POSIX tar.lz archiv const char * const eclosa_msg = "Error closing archive"; const char * const eclosf_msg = "Error closing file"; const char * const nfound_msg = "Not found in archive."; -const char * const rd_err_msg = "Read error"; -const char * const wr_err_msg = "Write error"; const char * const seek_msg = "Seek error"; +const char * const wr_err_msg = "Write error"; const char * const chdir_msg = "Error changing working directory"; const char * const intdir_msg = "Failed to create intermediate directory"; @@ -506,8 +503,7 @@ bool show_member_name( const Extended & extended, const Tar_header header, const int vlevel, Resizable_buffer & rbuf ); bool check_skip_filename( const Cl_options & cl_opts, std::vector< char > & name_pending, - const char * const filename, const int cwd_fd = -1, - std::string * const msgp = 0 ); + const char * const filename, const int chdir_fd = -1 ); bool make_dirs( const std::string & name ); // defined in common_mutex.cc @@ -601,8 +597,6 @@ void show_error( const char * const msg, const int errcode = 0, const bool help = false ); bool format_error( Resizable_buffer & rbuf, const int errcode, const char * const format, ... ); -bool format_error( std::string & msg, const int errcode, - const char * const format, ... ); void print_error( const int errcode, const char * const format, ... ); void format_file_error( std::string & estr, const char * const filename, const char * const msg, const int errcode = 0 ); diff --git a/testsuite/check.sh b/testsuite/check.sh index c3a8aa1..3a7fef4 100755 --- a/testsuite/check.sh +++ b/testsuite/check.sh @@ -1,6 +1,6 @@ #! /bin/sh # check script for Tarlz - Archiver with multimember lzip compression -# Copyright (C) 2013-2025 Antonio Diaz Diaz. +# Copyright (C) 2013-2024 Antonio Diaz Diaz. # # This script is free software: you have unlimited permission # to copy, distribute, and modify it. @@ -23,91 +23,97 @@ fi echo "Try bash -c \"$0 $1 $2\"" exit 1 } -/bin/sh -c "lzip --version" > /dev/null 2>&1 || - { echo "$0: lzip is needed to run the tests" ; exit 1 ; } if [ -d tmp ] ; then rm -rf tmp ; fi mkdir tmp cd "${objdir}"/tmp || framework_failure in="${testdir}"/test.txt -lzip -cs18KiB "${in}" > in.lz || framework_failure -cp "${testdir}"/test.txt.tar.lz in.tar.lz || framework_failure -lzip -cd in.tar.lz > in.tar || framework_failure -test_bad1="${testdir}"/test_bad1.txt -test_bad2="${testdir}"/test_bad2.txt +in_lz="${testdir}"/test.txt.lz +in_tar="${testdir}"/test.txt.tar +in_tar_lz="${testdir}"/test.txt.tar.lz +inbad1="${testdir}"/test_bad1.txt +inbad2="${testdir}"/test_bad2.txt em_lz="${testdir}"/em.lz -cp "${testdir}"/test3.tar.lz t3.tar.lz || framework_failure -lzip -cd t3.tar.lz > t3.tar || framework_failure -cp "${testdir}"/test3_dir.tar.lz t3_dir.tar.lz || framework_failure -lzip -cd t3_dir.tar.lz > t3_dir.tar || framework_failure +test3="${testdir}"/test3.tar +test3_lz="${testdir}"/test3.tar.lz +test3dir="${testdir}"/test3_dir.tar +test3dir_lz="${testdir}"/test3_dir.tar.lz test3dot_lz="${testdir}"/test3_dot.tar.lz -cp "${testdir}"/t155.tar.lz t155.tar.lz || framework_failure -lzip -cd t155.tar.lz > t155.tar || framework_failure -cp "${testdir}"/eoa_blocks.lz eoa.lz || framework_failure -lzip -cd eoa.lz > eoa || framework_failure -t3_bad1_lz="${testdir}"/test3_bad1.tar.lz -t3_bad2_lz="${testdir}"/test3_bad2.tar.lz -t3_bad3_lz="${testdir}"/test3_bad3.tar.lz -t3_bad4_lz="${testdir}"/test3_bad4.tar.lz -t3_bad5_lz="${testdir}"/test3_bad5.tar.lz +t155="${testdir}"/t155.tar +t155_lz="${testdir}"/t155.tar.lz +tlzit1="${testdir}"/tlz_in_tar1.tar +tlzit2="${testdir}"/tlz_in_tar2.tar +bad1="${testdir}"/test3_bad1.tar +bad2="${testdir}"/test3_bad2.tar +bad3="${testdir}"/test3_bad3.tar +bad4="${testdir}"/test3_bad4.tar +bad5="${testdir}"/test3_bad5.tar +bad1_lz="${testdir}"/test3_bad1.tar.lz +bad2_lz="${testdir}"/test3_bad2.tar.lz +bad3_lz="${testdir}"/test3_bad3.tar.lz +bad4_lz="${testdir}"/test3_bad4.tar.lz +bad5_lz="${testdir}"/test3_bad5.tar.lz +bad6_lz="${testdir}"/test3_bad6.tar.lz +eoa="${testdir}"/eoa_blocks.tar +eoa_lz="${testdir}"/eoa_blocks.tar.lz fail=0 lwarnc=0 test_failed() { fail=1 ; printf " $1" ; [ -z "$2" ] || printf "($2)" ; } -is_compressed() { lzip -lq "$1" ; } -is_uncompressed() { lzip -lq "$1" ; [ $? = 2 ] ; } +is_compressed() { [ "`dd if="$1" bs=4 count=1 2> /dev/null`" = LZIP ] ; } +is_uncompressed() { [ "`dd if="$1" bs=4 count=1 2> /dev/null`" != LZIP ] ; } cyg_symlink() { [ ${lwarnc} = 0 ] && printf "\nwarning: your OS follows symbolic links to directories even when tarlz asks it not to\n$1" lwarnc=1 ; } -# Description of test files for tarlz (use level -6 or -9): -# test.txt.tar.lz: 1 member (test.txt) + EOA. -# test_bad1.txt.tar: truncated at offset 17082 (of 37888) -# test_bad1.txt.tar.lz: truncated at offset 6000 (of 7459) -# test_bad2.txt.tar.lz: byte at offset 6000 changed from 0x53 to 0x43 -# t155.tar.lz: directory + 3 links + file + EOA, all with 155 char names -# symlinks: long linkpath, long both, long path. -# t155_fv?.tar.lz: like t155.tar but with 4 kinds of format violations -# t155_fv1.tar.lz: extra extended header before EOA blocks -# t155_fv2.tar.lz: first extended header followed by global header -# t155_fv3.tar.lz: consecutive extended headers in last member +# Description of test files for tarlz: +# test.txt.tar.lz: 1 member (test.txt). +# t155.tar[.lz]: directory + 3 links + file + EOA, all with 155 char names +# t155_fv?.tar[.lz]: like t155.tar but with 3 kinds of format violations +# t155_fv1.tar[.lz]: extra extended header before EOA blocks +# t155_fv2.tar[.lz]: first extended header followed by global header +# t155_fv3.tar[.lz]: consecutive extended headers in last member # t155_fv[456].tar.lz: like t155_fv[123].tar.lz but violation alone in member -# t155_fv7.tar.lz: 2 linkpath extended records in first symlink -# ts_in_link.tar.lz: 4 symbolic links (link[1-4]) to / /dir/ dir/ dir(107/) -# test3.tar.lz: 3 members (foo bar baz) + 2 zeroed 512-byte blocks -# test3_bad1.tar: byte at offset 259 changed from 't' to '0' (tar magic) -# test3_bad2.tar: byte at offset 1283 changed from 't' to '0' (tar magic) -# test3_bad3.tar: byte at offset 2559 changed from 0x00 to 0x20 (baz header padding) -# test3_bad4.tar: byte at offset 1283 changed from 't' to '0' (tar magic) -# byte at offset 2307 changed from 't' to '0' (tar magic) -# test3_bad5.tar: 510 zeros + "LZ" prepended to test3.tar (bogus lz header) -# test3_bad1.tar.lz: byte at offset 2 changed from 'I' to 'i' (lzip magic) -# test3_bad2.tar.lz: byte at offset 49 changed from 0x29 to 0x69 (foo header) -# test3_bad3.tar.lz: byte at offset 166 changed from 0x9A to 0x7A (bar data padding) -# test3_bad4.tar.lz: combined damage of test3_bad2.tar.lz and test3_bad3.tar.lz -# test3_bad5.tar.lz: [68-131] --> zeroed (foo data padding + bar) -# test3_bad6.tar.lz: 510 zeros prepended to test3.tar.lz (header in two blocks) -# test3_crc.tar.lz: test3.tar with extended headers and bad crc32 records -# test3_uk.tar.lz: test3.tar with extended headers and 15 unknownNN keywords -# test3_dir.tar.lz: like test3.tar but members /dir/foo /dir/bar /dir/baz -# test3_dot.tar.lz: 3 times 3 members ./foo ././bar ./././baz -# the 3 central members with filename in extended header -# test3_eoa1.tar.lz: test3.tar without EOA blocks -# test3_eoa2.tar.lz: test3.tar with only one EOA block -# test3_eoa3.tar.lz: test3.tar with one zeroed block between foo and bar -# test3_eoa4.tar.lz: test3.tar ended by extended header without EOA blocks -# test3_eoa5.tar.lz: test3.tar split extended bar member, without EOA blocks -# test3_gh[1-4].tar.lz: test3.tar with global header in own member at each pos -# test3_gh5.tar.lz: test3.tar with global before foo in same member -# test3_gh[678].tar.lz: test3.tar with global before (bar split in 3 ways) -# test3_nn.tar.lz: test3.tar with zeroed name (no name) in bar member -# test3_sm?.tar.lz: test3.tar with extended bar member split in 4 ways # tar_in_tlz1.tar.lz: 2 members (test.txt.tar test3.tar) 3 lzip members # tar_in_tlz2.tar.lz: 2 members (test.txt.tar test3.tar) 5 lzip members -# tlz_in_tar1.tar: 1 member (test3.tar.lz) first tar magic damaged -# tlz_in_tar2.tar: 2 members (foo test3.tar.lz) first tar magic damaged -# ug32chars.tar.lz: 1 member (foo) with 32-character owner and group names -# ug32767.tar.lz: 1 member (foo) with numerical-only owner and group +# ts_in_link.tar.lz: 4 symbolic links (link[1-4]) to / /dir/ dir/ dir(107/) +# test_bad1.txt.tar.lz: truncated at offset 6000 (of 7459) +# test_bad2.txt.tar.lz: byte at offset 6000 changed from 0x53 to 0x43 +# test3.tar[.lz]: 3 members (foo bar baz) + 2 zeroed 512-byte blocks +# test3_dir.tar[.lz] like test3.tar but members /dir/foo /dir/bar /dir/baz +# test3_dot.tar.lz: 3 times 3 members ./foo ././bar ./././baz +# the 3 central members with filename in extended header +# test3_bad1.tar: byte at offset 259 changed from 't' to '0' (magic) +# test3_bad2.tar: byte at offset 1283 changed from 't' to '0' (magic) +# test3_bad3.tar: byte at offset 2559 changed from 0x00 to 0x20 (padding) +# test3_bad4.tar: byte at offset 1283 changed from 't' to '0' (magic) +# byte at offset 2307 changed from 't' to '0' (magic) +# test3_bad5.tar: 510 zeros + "LZ" prepended to test3.tar (bogus lz header) +# test3_bad1.tar.lz: byte at offset 2 changed from 'I' to 'i' (magic) +# test3_bad2.tar.lz: byte at offset 49 changed from 0x49 to 0x69 (mid stream) +# test3_bad3.tar.lz: byte at offset 176 changed from 0x7D to 0x6D (mid stream) +# test3_bad4.tar.lz: combined damage of test3_bad2.tar.lz and test3_bad3.tar.lz +# test3_bad5.tar.lz: [71-134] --> zeroed (first trailer + second header) +# test3_bad6.tar.lz: 510 zeros prepended to test3.tar.lz (header in two blocks) +# test3_eoa?.tar: like test3_eoa?.tar.lz but uncompressed +# test3_eoa1.tar.lz: test3.tar.lz without EOA blocks +# test3_eoa2.tar.lz: test3.tar.lz with only one EOA block +# test3_eoa3.tar.lz: test3.tar.lz with one zeroed block between foo and bar +# test3_eoa4.tar.lz: test3.tar.lz ended by extended header without EOA blocks +# test3_eoa5.tar.lz: test3.tar.lz split extended bar member, without EOA blocks +# test3_gh?.tar: test3.tar with global header at each position +# test3_gh?.tar.lz: test3.tar.lz with global before bar split in 4 ways +# test3_gh5.tar.lz: test3.tar.lz with global in lzip member before foo +# test3_gh6.tar.lz: test3.tar.lz with global before foo in same member +# test3_nn.tar[.lz]: test3.tar[.lz] with zeroed name (no name) in bar member +# test3_sm?.tar.lz: test3.tar.lz with extended bar member split in 4 ways +# tlz_in_tar1.tar: 1 member (test3.tar.lz) first magic damaged +# tlz_in_tar2.tar: 2 members (foo test3.tar.lz) first magic damaged +# ug32chars.tar.lz: 1 member (foo) with 32-character owner and group names +# ug32767.tar.lz: 1 member (foo) with numerical-only owner and group + +# Note that multi-threaded --list succeeds with test_bad2.txt.tar.lz and +# test3_bad3.tar.lz because their headers are intact. "${TARLZ}" --check-lib # just print warning [ $? != 2 ] || test_failed $LINENO # unless bad lzlib.h @@ -116,9 +122,9 @@ printf "testing tarlz-%s..." "$2" "${TARLZ}" -q -tf "${in}" [ $? = 2 ] || test_failed $LINENO -"${TARLZ}" -q -tf in.lz +"${TARLZ}" -q -tf "${in_lz}" [ $? = 2 ] || test_failed $LINENO -"${TARLZ}" -q -tf in.tar.lz -f in.tar.lz +"${TARLZ}" -q -tf "${in_tar_lz}" -f "${in_tar_lz}" [ $? = 1 ] || test_failed $LINENO "${TARLZ}" -q -tf nx_file [ $? = 1 ] || test_failed $LINENO @@ -143,12 +149,9 @@ printf "testing tarlz-%s..." "$2" [ ! -e out.tar ] || test_failed $LINENO "${TARLZ}" -q -c "${in}" nx_file > /dev/null [ $? = 1 ] || test_failed $LINENO -touch empty.tar || framework_failure -"${TARLZ}" -q -c -C nx_dir -f out.tar "${in}" +"${TARLZ}" -q -c -C nx_dir "${in}" [ $? = 1 ] || test_failed $LINENO -cmp empty.tar out.tar || test_failed $LINENO -rm -f out.tar || framework_failure -"${TARLZ}" -q -x -C nx_dir -f t3.tar.lz +"${TARLZ}" -q -x -C nx_dir "${test3_lz}" [ $? = 1 ] || test_failed $LINENO touch empty.tar.lz empty.tlz || framework_failure # list an empty lz file "${TARLZ}" -q -tf empty.tar.lz @@ -156,7 +159,8 @@ touch empty.tar.lz empty.tlz || framework_failure # list an empty lz file "${TARLZ}" -q -tf empty.tlz [ $? = 2 ] || test_failed $LINENO rm -f empty.tar.lz empty.tlz || framework_failure -"${TARLZ}" -q -z empty.tar # compress an empty archive +touch empty.tar || framework_failure # compress an empty archive +"${TARLZ}" -q -z empty.tar [ $? = 2 ] || test_failed $LINENO [ ! -e empty.tar.lz ] || test_failed $LINENO rm -f empty.tar empty.tar.lz || framework_failure @@ -180,13 +184,15 @@ done [ $? = 1 ] || test_failed $LINENO "${TARLZ}" -q -z . [ $? = 1 ] || test_failed $LINENO -"${TARLZ}" -z -o - --uncompressed t3.tar > /dev/null 2>&1 +"${TARLZ}" -z -o - --uncompressed "${test3}" > /dev/null 2>&1 [ $? = 1 ] || test_failed $LINENO -"${TARLZ}" -q -tf in.tar.lz "" # empty non-option argument +"${TARLZ}" -q -tf "${in_tar_lz}" "" # empty non-option argument [ $? = 1 ] || test_failed $LINENO "${TARLZ}" --help > /dev/null || test_failed $LINENO "${TARLZ}" -V > /dev/null || test_failed $LINENO -"${TARLZ}" --bad_option -tf t3.tar.lz 2> /dev/null +"${TARLZ}" --bad_option -tf "${test3_lz}" 2> /dev/null +[ $? = 1 ] || test_failed $LINENO +"${TARLZ}" -tf 2> /dev/null [ $? = 1 ] || test_failed $LINENO bad_dates='@-9223372036854775809 @9223372036854775808 -2147481749-01-01T00:00:00 2147483648-01-01T00:00:00 @@ -195,34 +201,34 @@ for i in ${bad_dates} ; do "${TARLZ}" -c --mtime="$i" "${in}" > /dev/null 2>&1 [ $? = 1 ] || test_failed $LINENO "$i" done -"${TARLZ}" --owner=invalid_owner_name -tf t3.tar.lz 2> /dev/null +"${TARLZ}" --owner=invalid_owner_name -tf "${test3_lz}" 2> /dev/null [ $? = 1 ] || test_failed $LINENO -"${TARLZ}" --group=invalid_group_name -tf t3.tar.lz 2> /dev/null +"${TARLZ}" --group=invalid_group_name -tf "${test3_lz}" 2> /dev/null [ $? = 1 ] || test_failed $LINENO printf "\ntesting --list and --extract..." # test --list and --extract -"${TARLZ}" -tf eoa.lz --missing-crc || test_failed $LINENO -"${TARLZ}" -xf eoa.lz --missing-crc || test_failed $LINENO -"${TARLZ}" -C nx_dir -tf in.tar > /dev/null || test_failed $LINENO -"${TARLZ}" -xf in.tar --missing-crc || test_failed $LINENO +"${TARLZ}" -tf "${eoa_lz}" --missing-crc || test_failed $LINENO +"${TARLZ}" -xf "${eoa_lz}" --missing-crc || test_failed $LINENO +"${TARLZ}" -C nx_dir -tf "${in_tar}" > /dev/null || test_failed $LINENO +"${TARLZ}" -xf "${in_tar}" --missing-crc || test_failed $LINENO cmp "${in}" test.txt || test_failed $LINENO rm -f test.txt || framework_failure -"${TARLZ}" -tf in.tar.lz --missing-crc > /dev/null || test_failed $LINENO +"${TARLZ}" -tf "${in_tar_lz}" --missing-crc > /dev/null || test_failed $LINENO for i in 0 2 6 ; do - "${TARLZ}" -n$i -xf in.tar.lz --missing-crc || test_failed $LINENO $i + "${TARLZ}" -n$i -xf "${in_tar_lz}" --missing-crc || test_failed $LINENO $i cmp "${in}" test.txt || test_failed $LINENO $i rm -f test.txt || framework_failure done # test3 reference files for -t and -tv (list3, vlist3) -"${TARLZ}" -tf t3.tar > list3 || test_failed $LINENO -"${TARLZ}" -tvf t3.tar > vlist3 || test_failed $LINENO +"${TARLZ}" -tf "${test3}" > list3 || test_failed $LINENO +"${TARLZ}" -tvf "${test3}" > vlist3 || test_failed $LINENO for i in 0 2 6 ; do - "${TARLZ}" -n$i -tf t3.tar.lz > out || test_failed $LINENO $i + "${TARLZ}" -n$i -tf "${test3_lz}" > out || test_failed $LINENO $i diff -u list3 out || test_failed $LINENO $i - "${TARLZ}" -n$i -tvf t3.tar.lz > out || test_failed $LINENO $i + "${TARLZ}" -n$i -tvf "${test3_lz}" > out || test_failed $LINENO $i diff -u vlist3 out || test_failed $LINENO $i done rm -f out || framework_failure @@ -234,31 +240,31 @@ cp "${testdir}"/rbaz cbaz || framework_failure # test --list and --extract test3 rm -f foo bar baz || framework_failure -"${TARLZ}" -xf t3.tar --missing-crc || test_failed $LINENO +"${TARLZ}" -xf "${test3}" --missing-crc || test_failed $LINENO cmp cfoo foo || test_failed $LINENO cmp cbar bar || test_failed $LINENO cmp cbaz baz || test_failed $LINENO # time and mode comparison always fails on OS/2 -if "${TARLZ}" -q -df t3.tar --ignore-ids ; then d_works=yes -else printf "\nwarning: some '--diff' tests will be skipped." +if "${TARLZ}" -df "${test3}" --ignore-ids ; then d_works=yes +else printf "warning: some '--diff' tests will be skipped.\n" fi rm -f foo bar baz || framework_failure for i in 0 2 6 ; do - "${TARLZ}" -n$i -xf t3.tar.lz --missing-crc || test_failed $LINENO $i + "${TARLZ}" -n$i -xf "${test3_lz}" --missing-crc || test_failed $LINENO $i cmp cfoo foo || test_failed $LINENO $i cmp cbar bar || test_failed $LINENO $i cmp cbaz baz || test_failed $LINENO $i rm -f foo bar baz || framework_failure - "${TARLZ}" -n$i -tvf t3.tar.lz ./foo ./bar ./baz > out 2> /dev/null || + "${TARLZ}" -n$i -tvf "${test3_lz}" ./foo ./bar ./baz > out 2> /dev/null || test_failed $LINENO $i diff -u vlist3 out || test_failed $LINENO $i rm -f out || framework_failure - "${TARLZ}" -q -n$i -xf t3.tar.lz ./foo ./bar ./baz || test_failed $LINENO $i + "${TARLZ}" -q -n$i -xf "${test3_lz}" ./foo ./bar ./baz || test_failed $LINENO $i cmp cfoo foo || test_failed $LINENO $i cmp cbar bar || test_failed $LINENO $i cmp cbaz baz || test_failed $LINENO $i rm -f foo bar baz || framework_failure - "${TARLZ}" -n$i -xf t3.tar.lz foo/ bar// baz/// || test_failed $LINENO $i + "${TARLZ}" -n$i -xf "${test3_lz}" foo/ bar// baz/// || test_failed $LINENO $i cmp cfoo foo || test_failed $LINENO $i cmp cbar bar || test_failed $LINENO $i cmp cbaz baz || test_failed $LINENO $i @@ -274,23 +280,10 @@ for i in 0 2 6 ; do cmp cbar bar || test_failed $LINENO $i cmp cbaz baz || test_failed $LINENO $i rm -f foo bar baz || framework_failure - "${TARLZ}" -q -n$i -tf "${testdir}"/test3_uk.tar.lz || test_failed $LINENO $i - "${TARLZ}" -q -n$i -xf "${testdir}"/test3_uk.tar.lz || test_failed $LINENO $i - cmp cfoo foo || test_failed $LINENO $i - cmp cbar bar || test_failed $LINENO $i - cmp cbaz baz || test_failed $LINENO $i - rm -f foo bar baz || framework_failure done -lzip -cd "${testdir}"/test3_uk.tar.lz | "${TARLZ}" -q -t || test_failed $LINENO -lzip -cd "${testdir}"/test3_uk.tar.lz | "${TARLZ}" -q -x foo bar baz || - test_failed $LINENO -cmp cfoo foo || test_failed $LINENO -cmp cbar bar || test_failed $LINENO -cmp cbaz baz || test_failed $LINENO -rm -f foo bar baz || framework_failure # test -C in --diff and --extract -for i in t3.tar t3.tar.lz ; do +for i in "${test3}" "${test3_lz}" ; do mkdir dir1 dir2 dir3 || framework_failure "${TARLZ}" -q -xf "$i" -C dir1 foo -C ../dir2 bar -C ../dir3 baz || test_failed $LINENO "$i" @@ -305,7 +298,7 @@ for i in t3.tar t3.tar.lz ; do fi rm -rf dir1 dir2 dir3 || framework_failure done -for i in t3_dir.tar t3_dir.tar.lz ; do +for i in "${test3dir}" "${test3dir_lz}" ; do mkdir dir1 dir2 dir3 || framework_failure "${TARLZ}" -q -xf "$i" -C dir2 dir/bar -C ../dir1 dir/foo \ -C ../dir3 dir/baz || test_failed $LINENO "$i" @@ -315,13 +308,13 @@ for i in t3_dir.tar t3_dir.tar.lz ; do if [ "${d_works}" = yes ] ; then "${TARLZ}" -q -df "$i" --ignore-ids -C dir1 dir/foo -C ../dir2 dir/bar \ -C ../dir3 dir/baz || test_failed $LINENO "$i" - "${TARLZ}" -q -df t3.tar -C dir1/dir foo -C ../../dir2/dir bar \ + "${TARLZ}" -q -df "${test3}" -C dir1/dir foo -C ../../dir2/dir bar \ --ignore-ids -C ../../dir3/dir baz || test_failed $LINENO "$i" fi rm -rf dir1 dir2 dir3 || framework_failure done -for i in t3_dir.tar t3_dir.tar.lz ; do +for i in "${test3dir}" "${test3dir_lz}" ; do "${TARLZ}" -q -tf "$i" --missing-crc || test_failed $LINENO "$i" "${TARLZ}" -q -xf "$i" --missing-crc || test_failed $LINENO "$i" cmp cfoo dir/foo || test_failed $LINENO "$i" @@ -343,55 +336,51 @@ for i in t3_dir.tar t3_dir.tar.lz ; do done # test --extract --exclude -"${TARLZ}" -xf t3.tar --exclude='f*o' --exclude=baz || test_failed $LINENO +"${TARLZ}" -xf "${test3}" --exclude='f*o' --exclude=baz || test_failed $LINENO [ ! -e foo ] || test_failed $LINENO cmp cbar bar || test_failed $LINENO [ ! -e baz ] || test_failed $LINENO rm -f foo bar baz || framework_failure for i in 0 2 6 ; do - "${TARLZ}" -n$i -xf t3.tar.lz --exclude=bar || test_failed $LINENO $i + "${TARLZ}" -n$i -xf "${test3_lz}" --exclude=bar || test_failed $LINENO $i cmp cfoo foo || test_failed $LINENO $i [ ! -e bar ] || test_failed $LINENO $i cmp cbaz baz || test_failed $LINENO $i rm -f foo bar baz || framework_failure - "${TARLZ}" -q -n$i -xf t3_dir.tar.lz --exclude='?ar' || test_failed $LINENO $i + "${TARLZ}" -q -n$i -xf "${test3dir_lz}" --exclude='?ar' || test_failed $LINENO $i cmp cfoo dir/foo || test_failed $LINENO $i [ ! -e dir/bar ] || test_failed $LINENO $i cmp cbaz dir/baz || test_failed $LINENO $i rm -rf dir || framework_failure - "${TARLZ}" -q -n$i -xf t3_dir.tar.lz --exclude=dir/bar || test_failed $LINENO $i + "${TARLZ}" -q -n$i -xf "${test3dir_lz}" --exclude=dir/bar || test_failed $LINENO $i cmp cfoo dir/foo || test_failed $LINENO $i [ ! -e dir/bar ] || test_failed $LINENO $i cmp cbaz dir/baz || test_failed $LINENO $i rm -rf dir || framework_failure - "${TARLZ}" -q -n$i -xf t3_dir.tar.lz --exclude=dir || test_failed $LINENO $i + "${TARLZ}" -q -n$i -xf "${test3dir_lz}" --exclude=dir || test_failed $LINENO $i [ ! -e dir ] || test_failed $LINENO $i rm -rf dir || framework_failure - "${TARLZ}" -q -n$i -xf t3_dir.tar.lz --exclude='dir/*' || test_failed $LINENO $i + "${TARLZ}" -q -n$i -xf "${test3dir_lz}" --exclude='dir/*' || test_failed $LINENO $i [ ! -e dir ] || test_failed $LINENO $i rm -rf dir || framework_failure - "${TARLZ}" -q -n$i -xf t3_dir.tar.lz --exclude='[bf][ao][orz]' || + "${TARLZ}" -q -n$i -xf "${test3dir_lz}" --exclude='[bf][ao][orz]' || test_failed $LINENO $i [ ! -e dir ] || test_failed $LINENO $i rm -rf dir || framework_failure - "${TARLZ}" -q -n$i -xf t3_dir.tar.lz --exclude='*o' dir/foo || + "${TARLZ}" -q -n$i -xf "${test3dir_lz}" --exclude='*o' dir/foo || test_failed $LINENO $i [ ! -e dir ] || test_failed $LINENO $i rm -rf dir || framework_failure done # test --list and --extract EOA -lzip -cd "${testdir}"/test3_eoa1.tar.lz > test3_eoa1.tar || framework_failure -lzip -cd "${testdir}"/test3_eoa2.tar.lz > test3_eoa2.tar || framework_failure -lzip -cd "${testdir}"/test3_eoa3.tar.lz > test3_eoa3.tar || framework_failure -lzip -cd "${testdir}"/test3_eoa4.tar.lz > test3_eoa4.tar || framework_failure -"${TARLZ}" -tvf test3_eoa1.tar > out 2> /dev/null +"${TARLZ}" -tvf "${testdir}"/test3_eoa1.tar > out 2> /dev/null [ $? = 2 ] || test_failed $LINENO diff -u vlist3 out || test_failed $LINENO -"${TARLZ}" -tvf test3_eoa2.tar > out || test_failed $LINENO +"${TARLZ}" -tvf "${testdir}"/test3_eoa2.tar > out || test_failed $LINENO diff -u vlist3 out || test_failed $LINENO -"${TARLZ}" -q -tf test3_eoa3.tar || test_failed $LINENO -"${TARLZ}" -tvf test3_eoa4.tar > out 2> /dev/null +"${TARLZ}" -q -tf "${testdir}"/test3_eoa3.tar || test_failed $LINENO +"${TARLZ}" -tvf "${testdir}"/test3_eoa4.tar > out 2> /dev/null [ $? = 2 ] || test_failed $LINENO diff -u vlist3 out || test_failed $LINENO for i in 0 2 6 ; do @@ -412,41 +401,32 @@ for i in 0 2 6 ; do done rm -f out || framework_failure # -"${TARLZ}" -q -xf test3_eoa1.tar +"${TARLZ}" -q -xf "${testdir}"/test3_eoa1.tar [ $? = 2 ] || test_failed $LINENO -if [ "${d_works}" = yes ] ; then - "${TARLZ}" -q -df test3_eoa1.tar --ignore-ids - [ $? = 2 ] || test_failed $LINENO -fi cmp cfoo foo || test_failed $LINENO cmp cbar bar || test_failed $LINENO cmp cbaz baz || test_failed $LINENO rm -f foo bar baz || framework_failure -"${TARLZ}" -xf test3_eoa2.tar || test_failed $LINENO +"${TARLZ}" -xf "${testdir}"/test3_eoa2.tar || test_failed $LINENO cmp cfoo foo || test_failed $LINENO cmp cbar bar || test_failed $LINENO cmp cbaz baz || test_failed $LINENO rm -f foo bar baz || framework_failure -"${TARLZ}" -xf test3_eoa3.tar || test_failed $LINENO +"${TARLZ}" -xf "${testdir}"/test3_eoa3.tar || test_failed $LINENO cmp cfoo foo || test_failed $LINENO [ ! -e bar ] || test_failed $LINENO [ ! -e baz ] || test_failed $LINENO rm -f foo bar baz || framework_failure -"${TARLZ}" -q -xf test3_eoa4.tar +"${TARLZ}" -q -xf "${testdir}"/test3_eoa4.tar [ $? = 2 ] || test_failed $LINENO cmp cfoo foo || test_failed $LINENO cmp cbar bar || test_failed $LINENO cmp cbaz baz || test_failed $LINENO -rm -f foo bar baz test3_eoa1.tar || framework_failure -rm -f test3_eoa2.tar test3_eoa3.tar test3_eoa4.tar || framework_failure +rm -f foo bar baz || framework_failure # for i in 0 2 6 ; do "${TARLZ}" -q -n$i -xf "${testdir}"/test3_eoa1.tar.lz [ $? = 2 ] || test_failed $LINENO $i - if [ "${d_works}" = yes ] ; then - "${TARLZ}" -q -n$i -df "${testdir}"/test3_eoa1.tar.lz --ignore-ids - [ $? = 2 ] || test_failed $LINENO $i - fi cmp cfoo foo || test_failed $LINENO $i cmp cbar bar || test_failed $LINENO $i cmp cbaz baz || test_failed $LINENO $i @@ -492,28 +472,27 @@ for i in "${testdir}"/tar_in_tlz1.tar.lz "${testdir}"/tar_in_tlz2.tar.lz ; do rm -f out0 out2 out6 outv0 outv2 outv6 || framework_failure for j in 0 2 6 ; do "${TARLZ}" -xf "$i" -n$j || test_failed $LINENO "$i $j" - cmp in.tar test.txt.tar || test_failed $LINENO "$i $j" - cmp t3.tar test3.tar || test_failed $LINENO "$i $j" + cmp "${in_tar}" test.txt.tar || test_failed $LINENO "$i $j" + cmp "${test3}" test3.tar || test_failed $LINENO "$i $j" rm -f test.txt.tar test3.tar || framework_failure done done # test --list and --extract with global headers uncompressed for i in gh1 gh2 gh3 gh4 ; do - lzip -cd "${testdir}"/test3_${i}.tar.lz > test3_${i}.tar || framework_failure - "${TARLZ}" -tf test3_${i}.tar > out || test_failed $LINENO $i + "${TARLZ}" -tf "${testdir}"/test3_${i}.tar > out || test_failed $LINENO $i diff -u list3 out || test_failed $LINENO $i - "${TARLZ}" -tvf test3_${i}.tar > out || test_failed $LINENO $i + "${TARLZ}" -tvf "${testdir}"/test3_${i}.tar > out || test_failed $LINENO $i diff -u vlist3 out || test_failed $LINENO $i - "${TARLZ}" -xf test3_${i}.tar || test_failed $LINENO $i + "${TARLZ}" -xf "${testdir}"/test3_${i}.tar || test_failed $LINENO $i cmp cfoo foo || test_failed $LINENO $i cmp cbar bar || test_failed $LINENO $i cmp cbaz baz || test_failed $LINENO $i - rm -f foo bar baz out test3_${i}.tar || framework_failure + rm -f foo bar baz out || framework_failure done # test --list and --extract with empty lzip member -cat "${em_lz}" t3.tar.lz > test3_em.tar.lz || framework_failure +cat "${em_lz}" "${test3_lz}" > test3_em.tar.lz || framework_failure "${TARLZ}" -q -tf test3_em.tar.lz > out [ $? = 2 ] || test_failed $LINENO "${TARLZ}" -tvf - < test3_em.tar.lz > out || test_failed $LINENO @@ -531,7 +510,7 @@ rm -f foo bar baz || framework_failure # test --list and --extract with global headers and extended tar members # split among lzip members -for i in gh1 gh2 gh3 gh4 gh5 gh6 gh7 gh8 sm1 sm2 sm3 sm4 ; do +for i in gh1 gh2 gh3 gh4 gh5 gh6 sm1 sm2 sm3 sm4 ; do for j in 0 2 6 ; do "${TARLZ}" -n$j -tf "${testdir}"/test3_${i}.tar.lz > out || test_failed $LINENO "$i $j" @@ -552,23 +531,18 @@ for i in gh1 gh2 gh3 gh4 gh5 gh6 gh7 gh8 sm1 sm2 sm3 sm4 ; do done rm -f list3 vlist3 || framework_failure -# multi-threaded --list succeeds with test_bad2.txt.tar.lz and -# test3_bad3.tar.lz because their headers are intact. -"${TARLZ}" -tf "${test_bad2}.tar.lz" > /dev/null || test_failed $LINENO -"${TARLZ}" -tf "${t3_bad3_lz}" > /dev/null || test_failed $LINENO - printf "\ntesting --concatenate..." # test --concatenate compressed cp "${in}" out.tar.lz || framework_failure # invalid tar.lz -"${TARLZ}" -Aqf out.tar.lz t3.tar.lz +"${TARLZ}" -Aqf out.tar.lz "${test3_lz}" [ $? = 2 ] || test_failed $LINENO -cp in.tar.lz out.tar.lz || framework_failure -"${TARLZ}" -q --un -Af out.tar.lz t3.tar.lz # contradictory ext +cp "${in_tar_lz}" out.tar.lz || framework_failure +"${TARLZ}" -q --un -Af out.tar.lz "${test3_lz}" # contradictory ext [ $? = 1 ] || test_failed $LINENO -cmp in.tar.lz out.tar.lz || test_failed $LINENO -cp in.tar.lz out.tar.lz || framework_failure -"${TARLZ}" -Af out.tar.lz t3.tar.lz || test_failed $LINENO +cmp "${in_tar_lz}" out.tar.lz || test_failed $LINENO +cp "${in_tar_lz}" out.tar.lz || framework_failure +"${TARLZ}" -Af out.tar.lz "${test3_lz}" || test_failed $LINENO "${TARLZ}" -xf out.tar.lz || test_failed $LINENO cmp "${in}" test.txt || test_failed $LINENO cmp cfoo foo || test_failed $LINENO @@ -576,47 +550,47 @@ cmp cbar bar || test_failed $LINENO cmp cbaz baz || test_failed $LINENO rm -f test.txt foo bar baz || framework_failure touch aout.tar.lz || framework_failure # concatenate to empty file -"${TARLZ}" -Aqf aout.tar.lz in.tar +"${TARLZ}" -Aqf aout.tar.lz "${in_tar}" [ $? = 2 ] || test_failed $LINENO -"${TARLZ}" -Af aout.tar.lz in.tar.lz t3.tar.lz || test_failed $LINENO +"${TARLZ}" -Af aout.tar.lz "${in_tar_lz}" "${test3_lz}" || test_failed $LINENO cmp out.tar.lz aout.tar.lz || test_failed $LINENO "${TARLZ}" -Af aout.tar.lz || test_failed $LINENO # concatenate nothing cmp out.tar.lz aout.tar.lz || test_failed $LINENO "${TARLZ}" -Aqf aout.tar.lz aout.tar.lz || test_failed $LINENO cmp out.tar.lz aout.tar.lz || test_failed $LINENO -"${TARLZ}" -Aq in.tar.lz t3.tar > aout.tar.lz # to stdout +"${TARLZ}" -Aq "${in_tar_lz}" "${test3}" > aout.tar.lz # to stdout [ $? = 2 ] || test_failed $LINENO -cmp in.tar.lz aout.tar.lz || test_failed $LINENO -"${TARLZ}" -A in.tar.lz t3.tar.lz > aout.tar.lz || test_failed $LINENO +cmp "${in_tar_lz}" aout.tar.lz || test_failed $LINENO +"${TARLZ}" -A "${in_tar_lz}" "${test3_lz}" > aout.tar.lz || test_failed $LINENO cmp out.tar.lz aout.tar.lz || test_failed $LINENO -cp eoa.lz aout.tar.lz || framework_failure -"${TARLZ}" -Aqf aout.tar.lz in.tar # concatenate to empty archive +cp "${eoa_lz}" aout.tar.lz || framework_failure +"${TARLZ}" -Aqf aout.tar.lz "${in_tar}" # concatenate to empty archive [ $? = 2 ] || test_failed $LINENO -"${TARLZ}" -Af aout.tar.lz in.tar.lz t3.tar.lz || test_failed $LINENO +"${TARLZ}" -Af aout.tar.lz "${in_tar_lz}" "${test3_lz}" || test_failed $LINENO cmp out.tar.lz aout.tar.lz || test_failed $LINENO -cp in.tar.lz aout.tar.lz || framework_failure -"${TARLZ}" -Aqf aout.tar.lz t3.tar.lz t3.tar +cp "${in_tar_lz}" aout.tar.lz || framework_failure +"${TARLZ}" -Aqf aout.tar.lz "${test3_lz}" "${test3}" [ $? = 2 ] || test_failed $LINENO cmp out.tar.lz aout.tar.lz || test_failed $LINENO rm -f aout.tar.lz || framework_failure touch aout.tar.lz || framework_failure # --exclude -"${TARLZ}" -Af aout.tar.lz in.tar.lz t3.tar.lz --exclude 't3*' || +"${TARLZ}" -Af aout.tar.lz "${in_tar_lz}" "${test3_lz}" --exclude 'test3*' || test_failed $LINENO -"${TARLZ}" -Af aout.tar.lz in.tar.lz t3.tar.lz --exclude 'in*' || +"${TARLZ}" -Af aout.tar.lz "${in_tar_lz}" "${test3_lz}" --exclude '*txt*' || test_failed $LINENO cmp out.tar.lz aout.tar.lz || test_failed $LINENO rm -f out.tar.lz aout.tar.lz || framework_failure # test --concatenate uncompressed -cp "${in}" out.tar || framework_failure # invalid tar -"${TARLZ}" -Aqf out.tar t3.tar +cp "${in}" out.tar || framework_failure # invalid tar +"${TARLZ}" -Aqf out.tar "${test3}" [ $? = 2 ] || test_failed $LINENO -cp in.tar out.tar || framework_failure -"${TARLZ}" -q -0 -Af out.tar t3.tar # contradictory ext +cp "${in_tar}" out.tar || framework_failure +"${TARLZ}" -q -0 -Af out.tar "${test3}" # contradictory ext [ $? = 1 ] || test_failed $LINENO -cmp in.tar out.tar || test_failed $LINENO -cp in.tar out.tar || framework_failure -"${TARLZ}" -Af out.tar t3.tar || test_failed $LINENO +cmp "${in_tar}" out.tar || test_failed $LINENO +cp "${in_tar}" out.tar || framework_failure +"${TARLZ}" -Af out.tar "${test3}" || test_failed $LINENO "${TARLZ}" -xf out.tar || test_failed $LINENO cmp "${in}" test.txt || test_failed $LINENO cmp cfoo foo || test_failed $LINENO @@ -624,32 +598,34 @@ cmp cbar bar || test_failed $LINENO cmp cbaz baz || test_failed $LINENO rm -f test.txt foo bar baz || framework_failure touch aout.tar || framework_failure # concatenate to empty file -"${TARLZ}" -Aqf aout.tar in.tar.lz +"${TARLZ}" -Aqf aout.tar "${in_tar_lz}" [ $? = 2 ] || test_failed $LINENO -"${TARLZ}" -Af aout.tar in.tar t3.tar || test_failed $LINENO +"${TARLZ}" -Af aout.tar "${in_tar}" "${test3}" || test_failed $LINENO cmp out.tar aout.tar || test_failed $LINENO "${TARLZ}" -Af aout.tar || test_failed $LINENO # concatenate nothing cmp out.tar aout.tar || test_failed $LINENO "${TARLZ}" -Aqf aout.tar aout.tar || test_failed $LINENO cmp out.tar aout.tar || test_failed $LINENO -"${TARLZ}" -Aq in.tar t3.tar.lz > aout.tar # to stdout +"${TARLZ}" -Aq "${in_tar}" "${test3_lz}" > aout.tar # to stdout [ $? = 2 ] || test_failed $LINENO -cmp in.tar aout.tar || test_failed $LINENO -"${TARLZ}" -A in.tar t3.tar > aout.tar || test_failed $LINENO +cmp "${in_tar}" aout.tar || test_failed $LINENO +"${TARLZ}" -A "${in_tar}" "${test3}" > aout.tar || test_failed $LINENO cmp out.tar aout.tar || test_failed $LINENO -cp eoa aout.tar || framework_failure # concatenate to empty archive -"${TARLZ}" -Aqf aout.tar in.tar.lz +cp "${eoa}" aout.tar || framework_failure # concatenate to empty archive +"${TARLZ}" -Aqf aout.tar "${in_tar_lz}" [ $? = 2 ] || test_failed $LINENO -"${TARLZ}" -Af aout.tar in.tar t3.tar || test_failed $LINENO +"${TARLZ}" -Af aout.tar "${in_tar}" "${test3}" || test_failed $LINENO cmp out.tar aout.tar || test_failed $LINENO -cp in.tar aout.tar || framework_failure -"${TARLZ}" -Aqf aout.tar t3.tar t3.tar.lz +cp "${in_tar}" aout.tar || framework_failure +"${TARLZ}" -Aqf aout.tar "${test3}" "${test3_lz}" [ $? = 2 ] || test_failed $LINENO cmp out.tar aout.tar || test_failed $LINENO rm -f aout.tar || framework_failure touch aout.tar || framework_failure # --exclude -"${TARLZ}" -Af aout.tar t3.tar in.tar --exclude 't3*' || test_failed $LINENO -"${TARLZ}" -Af aout.tar t3.tar in.tar --exclude 'in*' || test_failed $LINENO +"${TARLZ}" -Af aout.tar "${test3}" "${in_tar}" --exclude 'test3*' || + test_failed $LINENO +"${TARLZ}" -Af aout.tar "${test3}" "${in_tar}" --exclude '*txt*' || + test_failed $LINENO cmp out.tar aout.tar || test_failed $LINENO rm -f out.tar aout.tar || framework_failure @@ -772,23 +748,24 @@ cmp cfoo foo || test_failed $LINENO rm -f out.tar foo bar baz || framework_failure # test --create --mtime -safe_dates='1969-12-31T23:59:58 1969-12-31T23:59:59 - 1970-01-01T00:00:00 1970-01-01T00:00:01 @0 - 2038-01-18T03:14:07 2038-01-19T03:14:07 2038-01-19T03:14:08 - 2106-02-07T06:28:15 2106-02-07T06:28:16 - 2242-03-16T12:56:31 2242-03-16T12:56:32 @8589934591 @8589934592' dates='@-9223372036854775808 @-9223372036854775807 -2147481748-12-31T23:59:59 -1970-01-01T00:00:00 0000-01-01T00:00:00 0000-01-01T00:00:01 0000-01-02T00:00:00 1697-10-17T11:03:27 1697-10-17T11:03:28 1697-10-17T11:03:29 1833-11-24T17:31:43 1833-11-24T17:31:44 1833-11-24T17:31:45 1901-12-13T20:45:51 1901-12-13T20:45:52 1901-12-13T20:45:53 - 1901-12-14T20:45:51 9999-12-31T23:59:58 9999-12-31T23:59:59 + 1901-12-14T20:45:51 + 1969-12-31T23:59:58 1969-12-31T23:59:59 + 1970-01-01T00:00:00 1970-01-01T00:00:01 @0 + 2038-01-18T03:14:07 2038-01-19T03:14:07 2038-01-19T03:14:08 + 2106-02-07T06:28:15 2106-02-07T06:28:16 + 2242-03-16T12:56:31 2242-03-16T12:56:32 @8589934591 @8589934592 + 9999-12-31T23:59:58 9999-12-31T23:59:59 2147483647-12-31T23:59:59 @9223372036854775807' touch -d 2022-01-05T12:22:13 bar >/dev/null 2>&1 || touch -d '2022-01-05 12:22:13' bar >/dev/null 2>&1 || touch bar || framework_failure -for i in ${safe_dates} '2017-10-01 09:00:00' '2017-10-1 9:0:0' \ +for i in ${dates} @-8Ei '2017-10-01 09:00:00' '2017-10-1 9:0:0' \ '2017-10-01 09:00' '2017-10-01 09' 2017-10-01 ./bar ; do # Skip a time stamp $i if it's out of range for this platform, # or if it uses a notation that this platform does not recognize. @@ -799,15 +776,13 @@ for i in ${safe_dates} '2017-10-01 09:00:00' '2017-10-1 9:0:0' \ "${TARLZ}" -q -df out.tar && test_failed $LINENO "$i" "${TARLZ}" -xf out.tar || test_failed $LINENO "$i" if [ "${d_works}" = yes ] ; then - "${TARLZ}" -df out.tar --ignore-overflow || - { echo ; "${TARLZ}" -tvf out.tar ; ls -l foo ; test_failed $LINENO "$i" ; } + "${TARLZ}" -df out.tar --ignore-overflow || test_failed $LINENO "$i" fi done rm -f out.tar foo bar || framework_failure mkdir dir || framework_failure -for i in ${safe_dates} ${dates} ; do touch -d "$i" "dir/f$i" >/dev/null 2>&1 -done +for i in ${dates} ; do touch -d "$i" "dir/f$i" >/dev/null 2>&1 ; done "${TARLZ}" -cf out.tar dir || test_failed $LINENO is_uncompressed out.tar || test_failed $LINENO "${TARLZ}" -df out.tar || test_failed $LINENO @@ -815,7 +790,7 @@ rm -rf out.tar dir || framework_failure printf "\ntesting --diff..." -"${TARLZ}" -xf t3.tar.lz || test_failed $LINENO +"${TARLZ}" -xf "${test3_lz}" || test_failed $LINENO "${TARLZ}" -cf out.tar foo || test_failed $LINENO "${TARLZ}" -cf aout.tar foo --anonymous || test_failed $LINENO is_uncompressed out.tar || test_failed $LINENO @@ -824,25 +799,27 @@ if cmp out.tar aout.tar > /dev/null ; then printf "\nwarning: '--diff' test can't be run as root.\n" else for i in 0 2 6 ; do - "${TARLZ}" -n$i -xf t3.tar.lz || test_failed $LINENO $i - "${TARLZ}" -n$i -df t3.tar.lz > out$i + "${TARLZ}" -n$i -xf "${test3_lz}" || test_failed $LINENO $i + "${TARLZ}" -n$i -df "${test3_lz}" > out$i [ $? = 1 ] || test_failed $LINENO $i - "${TARLZ}" -n$i -df t3.tar.lz --ignore-ids || test_failed $LINENO $i - "${TARLZ}" -n$i -df t3.tar.lz --exclude '*' || test_failed $LINENO $i - "${TARLZ}" -n$i -df in.tar.lz --exclude '*' || test_failed $LINENO $i + "${TARLZ}" -n$i -df "${test3_lz}" --ignore-ids || test_failed $LINENO $i + "${TARLZ}" -n$i -df "${test3_lz}" --exclude '*' || test_failed $LINENO $i + "${TARLZ}" -n$i -df "${in_tar_lz}" --exclude '*' || test_failed $LINENO $i rm -f bar || framework_failure - "${TARLZ}" -n$i -df t3.tar.lz --ignore-ids foo baz || + "${TARLZ}" -n$i -df "${test3_lz}" --ignore-ids foo baz || test_failed $LINENO $i - "${TARLZ}" -n$i -df t3.tar.lz --ignore-metadata foo baz || + "${TARLZ}" -n$i -df "${test3_lz}" --ignore-metadata foo baz || test_failed $LINENO $i - "${TARLZ}" -n$i -df t3.tar.lz --exclude bar --ignore-ids || + "${TARLZ}" -n$i -df "${test3_lz}" --exclude bar --ignore-ids || test_failed $LINENO $i rm -f foo baz || framework_failure - "${TARLZ}" -q -n$i -xf t3_dir.tar.lz || test_failed $LINENO $i - "${TARLZ}" -q -n$i -df t3_dir.tar.lz --ignore-ids || test_failed $LINENO $i - "${TARLZ}" -q -n$i -df t3_dir.tar.lz dir --ignore-ids || + "${TARLZ}" -q -n$i -xf "${test3dir_lz}" || test_failed $LINENO $i + "${TARLZ}" -q -n$i -df "${test3dir_lz}" --ignore-ids || + test_failed $LINENO $i + "${TARLZ}" -q -n$i -df "${test3dir_lz}" dir --ignore-ids || + test_failed $LINENO $i + "${TARLZ}" -n$i -df "${test3_lz}" --ignore-ids -C dir || test_failed $LINENO $i - "${TARLZ}" -n$i -df t3.tar.lz --ignore-ids -C dir || test_failed $LINENO $i rm -rf dir || framework_failure done cmp out0 out2 || test_failed $LINENO @@ -857,72 +834,68 @@ printf "\ntesting --delete..." cp "${in}" out.tar || framework_failure # invalid tar "${TARLZ}" -q -f out.tar --delete foo [ $? = 2 ] || test_failed $LINENO -cmp "${in}" out.tar || test_failed $LINENO rm -f out.tar || framework_failure cp "${in}" out.tar.lz || framework_failure # invalid tar.lz "${TARLZ}" -q -f out.tar.lz --delete foo [ $? = 2 ] || test_failed $LINENO -cmp "${in}" out.tar.lz || test_failed $LINENO -cp in.lz out.tar.lz || framework_failure # invalid tar.lz +cp "${in_lz}" out.tar.lz || framework_failure # invalid tar.lz "${TARLZ}" -q -f out.tar.lz --delete foo [ $? = 2 ] || test_failed $LINENO -cmp in.lz out.tar.lz || test_failed $LINENO -rm -f in.lz out.tar.lz || framework_failure +rm -f out.tar.lz || framework_failure for e in "" .lz ; do - "${TARLZ}" -A in.tar$e t3.tar$e > out.tar$e || test_failed $LINENO $e + "${TARLZ}" -A "${in_tar}"$e "${test3}"$e > out.tar$e || test_failed $LINENO $e "${TARLZ}" -f out.tar$e --delete test.txt || test_failed $LINENO $e - cmp t3.tar$e out.tar$e || test_failed $LINENO $e + cmp "${test3}"$e out.tar$e || test_failed $LINENO $e "${TARLZ}" -f out.tar$e --delete || test_failed $LINENO $e # delete nothing - cmp t3.tar$e out.tar$e || test_failed $LINENO $e + cmp "${test3}"$e out.tar$e || test_failed $LINENO $e "${TARLZ}" -q -f out.tar$e --delete nx_file [ $? = 1 ] || test_failed $LINENO $e - cmp t3.tar$e out.tar$e || test_failed $LINENO $e - "${TARLZ}" -A in.tar$e t3_dir.tar$e > out.tar$e || test_failed $LINENO $e + cmp "${test3}"$e out.tar$e || test_failed $LINENO $e + "${TARLZ}" -A "${in_tar}"$e "${test3dir}"$e > out.tar$e || test_failed $LINENO $e "${TARLZ}" -q -f out.tar$e --delete test.txt || test_failed $LINENO $e - cmp t3_dir.tar$e out.tar$e || test_failed $LINENO $e - "${TARLZ}" -A in.tar$e t3_dir.tar$e > out.tar$e || test_failed $LINENO $e + cmp "${test3dir}"$e out.tar$e || test_failed $LINENO $e + "${TARLZ}" -A "${in_tar}"$e "${test3dir}"$e > out.tar$e || test_failed $LINENO $e "${TARLZ}" -q -f out.tar$e --delete dir || test_failed $LINENO $e - cmp in.tar$e out.tar$e || test_failed $LINENO $e - "${TARLZ}" -A in.tar$e t3_dir.tar$e > out.tar$e || test_failed $LINENO $e + cmp "${in_tar}"$e out.tar$e || test_failed $LINENO $e + "${TARLZ}" -A "${in_tar}"$e "${test3dir}"$e > out.tar$e || test_failed $LINENO $e "${TARLZ}" -q -f out.tar$e --del dir/foo dir/bar dir/baz || test_failed $LINENO $e - cmp in.tar$e out.tar$e || test_failed $LINENO $e - "${TARLZ}" -A in.tar$e t3_dir.tar$e > out.tar$e || test_failed $LINENO $e + cmp "${in_tar}"$e out.tar$e || test_failed $LINENO $e + "${TARLZ}" -A "${in_tar}"$e "${test3dir}"$e > out.tar$e || test_failed $LINENO $e "${TARLZ}" -q -f out.tar$e --del dir/foo dir/baz || test_failed $LINENO $e - cmp in.tar$e out.tar$e > /dev/null && test_failed $LINENO $e + cmp "${in_tar}"$e out.tar$e > /dev/null && test_failed $LINENO $e "${TARLZ}" -q -f out.tar$e --del dir/bar || test_failed $LINENO $e - cmp in.tar$e out.tar$e || test_failed $LINENO $e - "${TARLZ}" -A in.tar$e t3.tar$e > out.tar$e || test_failed $LINENO $e + cmp "${in_tar}"$e out.tar$e || test_failed $LINENO $e + "${TARLZ}" -A "${in_tar}"$e "${test3}"$e > out.tar$e || test_failed $LINENO $e "${TARLZ}" -f out.tar$e --delete foo bar baz || test_failed $LINENO $e - cmp in.tar$e out.tar$e || test_failed $LINENO $e - "${TARLZ}" -A in.tar$e t3.tar$e > out.tar$e || test_failed $LINENO $e + cmp "${in_tar}"$e out.tar$e || test_failed $LINENO $e + "${TARLZ}" -A "${in_tar}"$e "${test3}"$e > out.tar$e || test_failed $LINENO $e "${TARLZ}" -f out.tar$e --del test.txt foo bar baz || test_failed $LINENO $e - cmp eoa$e out.tar$e || test_failed $LINENO $e - "${TARLZ}" -A in.tar$e t3.tar$e > out.tar$e || test_failed $LINENO $e + cmp "${eoa}"$e out.tar$e || test_failed $LINENO $e + "${TARLZ}" -A "${in_tar}"$e "${test3}"$e > out.tar$e || test_failed $LINENO $e for i in test.txt foo bar baz ; do "${TARLZ}" -f out.tar$e --delete $i || test_failed $LINENO "$e $i" done - cmp eoa$e out.tar$e || test_failed $LINENO $e - "${TARLZ}" -A in.tar$e t3.tar$e > out.tar$e || test_failed $LINENO $e + cmp "${eoa}"$e out.tar$e || test_failed $LINENO $e + "${TARLZ}" -A "${in_tar}"$e "${test3}"$e > out.tar$e || test_failed $LINENO $e for i in baz bar foo test.txt ; do "${TARLZ}" -f out.tar$e --delete $i || test_failed $LINENO "$e $i" done - cmp eoa$e out.tar$e || test_failed $LINENO $e - "${TARLZ}" -A in.tar$e t3.tar$e > out.tar$e || test_failed $LINENO $e + cmp "${eoa}"$e out.tar$e || test_failed $LINENO $e + "${TARLZ}" -A "${in_tar}"$e "${test3}"$e > out.tar$e || test_failed $LINENO $e for i in foo bar test.txt baz ; do "${TARLZ}" -f out.tar$e --delete $i || test_failed $LINENO "$e $i" done - cmp eoa$e out.tar$e || test_failed $LINENO $e - "${TARLZ}" -A in.tar$e t155.tar$e t3.tar$e > out.tar$e || + cmp "${eoa}"$e out.tar$e || test_failed $LINENO $e + "${TARLZ}" -A "${in_tar}"$e "${t155}"$e "${test3}"$e > out.tar$e || test_failed $LINENO $e "${TARLZ}" -f out.tar$e --del baz foo test.txt bar || test_failed $LINENO $e - cmp t155.tar$e out.tar$e || test_failed $LINENO $e + cmp "${t155}"$e out.tar$e || test_failed $LINENO $e "${TARLZ}" -f out.tar$e --delete link || test_failed $LINENO $e "${TARLZ}" -q -tf out.tar$e || test_failed $LINENO $e - cmp t155.tar$e out.tar$e > /dev/null && test_failed $LINENO $e + cmp "${t155}"$e out.tar$e > /dev/null && test_failed $LINENO $e rm -f out.tar$e || framework_failure done -rm -f in.tar in.tar.lz t3.tar t3_dir.tar t3_dir.tar.lz || framework_failure # test --delete individual member after collective member cp cfoo foo || framework_failure @@ -948,17 +921,17 @@ rm -f out.tar.lz foo bar baz test.txt || framework_failure "${TARLZ}" -q -f test3_em.tar.lz --delete foo [ $? = 2 ] || test_failed $LINENO rm -f test3_em.tar.lz || framework_failure -cp "${testdir}"/test3_gh1.tar.lz out.tar.lz || framework_failure +cp "${testdir}"/test3_gh5.tar.lz out.tar.lz || framework_failure for i in foo bar baz ; do "${TARLZ}" -f out.tar.lz --delete $i || test_failed $LINENO $i done rm -f out.tar.lz || framework_failure -for i in gh1 gh2 gh3 gh4 ; do - lzip -cd "${testdir}"/test3_${i}.tar.lz > out.tar || framework_failure - for j in foo bar baz ; do - "${TARLZ}" -f out.tar --delete $j || test_failed $LINENO "$i $j" - done - rm -f out.tar || framework_failure +for i in 1 2 3 4 ; do + cp "${testdir}"/test3_gh${i}.tar out.tar || framework_failure + for j in foo bar baz ; do + "${TARLZ}" -f out.tar --delete $j || test_failed $LINENO "$i $j" + done + rm -f out.tar || framework_failure done printf "\ntesting --dereference..." @@ -1034,10 +1007,10 @@ cmp out.tar.lz aout.tar.lz || test_failed $LINENO "${TARLZ}" --un -q -rf aout.tar.lz foo bar baz # contradictory ext [ $? = 1 ] || test_failed $LINENO cmp out.tar.lz aout.tar.lz || test_failed $LINENO -cp eoa.lz aout.tar.lz || framework_failure # append to empty archive +cp "${eoa_lz}" aout.tar.lz || framework_failure # append to empty archive "${TARLZ}" -0 -rf aout.tar.lz foo bar baz || test_failed $LINENO cmp out.tar.lz aout.tar.lz || test_failed $LINENO -rm -f eoa.lz out.tar.lz aout.tar.lz || framework_failure +rm -f out.tar.lz aout.tar.lz || framework_failure # test --append --uncompressed "${TARLZ}" -cf out.tar foo bar baz || test_failed $LINENO @@ -1068,10 +1041,10 @@ cmp out.tar aout.tar || test_failed $LINENO "${TARLZ}" -0 -q -rf aout.tar foo bar baz # contradictory ext [ $? = 1 ] || test_failed $LINENO cmp out.tar aout.tar || test_failed $LINENO -cp eoa aout.tar || framework_failure # append to empty archive +cp "${eoa}" aout.tar || framework_failure # append to empty archive "${TARLZ}" -rf aout.tar foo bar baz || test_failed $LINENO cmp out.tar aout.tar || test_failed $LINENO -rm -f eoa out.tar aout.tar || framework_failure +rm -f out.tar aout.tar || framework_failure # test --append to solid archive "${TARLZ}" --solid -q -0 -cf out.tar.lz "${in}" foo bar || test_failed $LINENO @@ -1163,13 +1136,13 @@ fi printf "\ntesting long names..." -"${TARLZ}" -q -tf t155.tar || test_failed $LINENO -"${TARLZ}" -q -tf t155.tar.lz || test_failed $LINENO +"${TARLZ}" -q -tf "${t155}" || test_failed $LINENO +"${TARLZ}" -q -tf "${t155_lz}" || test_failed $LINENO if [ "${ln_works}" = yes ] ; then mkdir dir1 || framework_failure - "${TARLZ}" -C dir1 -xf t155.tar || test_failed $LINENO + "${TARLZ}" -C dir1 -xf "${t155}" || test_failed $LINENO mkdir dir2 || framework_failure - "${TARLZ}" -C dir2 -xf t155.tar.lz || test_failed $LINENO + "${TARLZ}" -C dir2 -xf "${t155_lz}" || test_failed $LINENO diff -ru dir1 dir2 || test_failed $LINENO "${TARLZ}" -cf out.tar.lz dir2 || test_failed $LINENO rm -rf dir2 || framework_failure @@ -1179,7 +1152,6 @@ if [ "${ln_works}" = yes ] ; then rmdir dir1 2> /dev/null && test_failed $LINENO rm -rf out.tar.lz dir2 dir1 || framework_failure fi -rm -f t155.tar.lz || framework_failure "${TARLZ}" -tvf "${testdir}"/ug32chars.tar.lz | grep -q \ -e very_long_owner_name_of_32_chars/very_long_group_name_of_32_chars || @@ -1283,19 +1255,6 @@ rm -f foo bar baz test.txt out.tar.lz out.tar outz.tar foobarbaz.tar.lz \ printf "\ntesting bad input..." -"${TARLZ}" -q -tf "${testdir}"/test3_crc.tar.lz -[ $? = 2 ] || test_failed $LINENO -"${TARLZ}" -q -n0 -tf "${testdir}"/test3_crc.tar.lz -[ $? = 2 ] || test_failed $LINENO -lzip -cd "${testdir}"/test3_crc.tar.lz | "${TARLZ}" -q -tf - -[ $? = 2 ] || test_failed $LINENO -"${TARLZ}" -q -tf "${testdir}"/test3_crc.tar.lz --missing-crc -[ $? = 2 ] || test_failed $LINENO -"${TARLZ}" -q -n0 -tf "${testdir}"/test3_crc.tar.lz --missing-crc -[ $? = 2 ] || test_failed $LINENO -lzip -cd "${testdir}"/test3_crc.tar.lz | "${TARLZ}" -q -tf - --missing-crc -[ $? = 2 ] || test_failed $LINENO - # test --extract ".." mkdir dir1 || framework_failure cd dir1 || framework_failure @@ -1314,6 +1273,15 @@ done cd .. || framework_failure rm -rf dir1 || framework_failure +# test --list and --extract truncated tar +dd if="${in_tar}" of=truncated.tar bs=1000 count=1 2> /dev/null +"${TARLZ}" -q -tf truncated.tar > /dev/null +[ $? = 2 ] || test_failed $LINENO +"${TARLZ}" -q -xf truncated.tar +[ $? = 2 ] || test_failed $LINENO +[ ! -e test.txt ] || test_failed $LINENO +rm -f truncated.tar || framework_failure + # test --delete with split 'bar' tar member for i in 1 2 3 4 ; do cp "${testdir}"/test3_sm${i}.tar.lz out.tar.lz || framework_failure @@ -1334,25 +1302,22 @@ done # test --list and --extract format violations if [ "${ln_works}" = yes ] ; then mkdir dir1 || framework_failure - "${TARLZ}" -C dir1 -xf t155.tar || test_failed $LINENO + "${TARLZ}" -C dir1 -xf "${t155}" || test_failed $LINENO fi -for i in 1 2 3 7 ; do - lzip -cd "${testdir}"/t155_fv${i}.tar.lz > t155_fv${i}.tar || - framework_failure - "${TARLZ}" -q -tf t155_fv${i}.tar +for i in 1 2 3 ; do + "${TARLZ}" -q -tf "${testdir}"/t155_fv${i}.tar [ $? = 2 ] || test_failed $LINENO $i - "${TARLZ}" -q -tf t155_fv${i}.tar --permissive || test_failed $LINENO $i + "${TARLZ}" -q -tf "${testdir}"/t155_fv${i}.tar --permissive || + test_failed $LINENO $i if [ "${ln_works}" = yes ] ; then mkdir dir2 || framework_failure - "${TARLZ}" -C dir2 -xf t155_fv${i}.tar --permissive || + "${TARLZ}" -C dir2 -xf "${testdir}"/t155_fv${i}.tar --permissive || test_failed $LINENO $i diff -ru dir1 dir2 || test_failed $LINENO $i rm -rf dir2 || framework_failure fi - rm -f t155_fv${i}.tar || framework_failure done -rm -f t155.tar || framework_failure -for i in 1 2 3 4 5 6 7 ; do +for i in 1 2 3 4 5 6 ; do "${TARLZ}" -q -tf "${testdir}"/t155_fv${i}.tar.lz [ $? = 2 ] || test_failed $LINENO $i "${TARLZ}" -q -tf "${testdir}"/t155_fv${i}.tar.lz --permissive || @@ -1367,8 +1332,7 @@ for i in 1 2 3 4 5 6 7 ; do done if [ "${ln_works}" = yes ] ; then rm -rf dir1 || framework_failure ; fi -lzip -cd "${testdir}"/test3_nn.tar.lz > test3_nn.tar || framework_failure -for i in test3_nn.tar "${testdir}"/test3_nn.tar.lz ; do +for i in "${testdir}"/test3_nn.tar "${testdir}"/test3_nn.tar.lz ; do "${TARLZ}" -q -n0 -tf "$i" || test_failed $LINENO "$i" "${TARLZ}" -q -n4 -tf "$i" || test_failed $LINENO "$i" "${TARLZ}" -q -n0 -xf "$i" || test_failed $LINENO "$i" @@ -1388,15 +1352,12 @@ for i in test3_nn.tar "${testdir}"/test3_nn.tar.lz ; do cmp cbaz baz || test_failed $LINENO "$i" rm -f foo bar baz || framework_failure done -rm -f test3_nn.tar || framework_failure printf "\ntesting --keep-damaged..." -# test --list, --extract, and --keep-damaged truncated/damaged compressed +# test --extract and --keep-damaged compressed rm -f test.txt || framework_failure -for i in "${test_bad1}" "${test_bad2}" ; do - "${TARLZ}" -q -n0 -tf "${i}.tar.lz" - [ $? = 2 ] || test_failed $LINENO "$i" +for i in "${inbad1}" "${inbad2}" ; do "${TARLZ}" -q -xf "${i}.tar.lz" [ $? = 2 ] || test_failed $LINENO "$i" [ ! -e test.txt ] || test_failed $LINENO "$i" @@ -1404,106 +1365,103 @@ for i in "${test_bad1}" "${test_bad2}" ; do "${TARLZ}" -q -n0 -xf "${i}.tar.lz" --keep-damaged [ $? = 2 ] || test_failed $LINENO "$i" [ -e test.txt ] || test_failed $LINENO "$i" - cmp "$i" test.txt || test_failed $LINENO "$i" + cmp "$i" test.txt 2> /dev/null || test_failed $LINENO "$i" rm -f test.txt || framework_failure done # rm -f foo bar baz || framework_failure -"${TARLZ}" -q -n0 -xf "${t3_bad1_lz}" +"${TARLZ}" -q -n0 -xf "${bad1_lz}" [ $? = 2 ] || test_failed $LINENO [ ! -e foo ] || test_failed $LINENO cmp cbar bar || test_failed $LINENO cmp cbaz baz || test_failed $LINENO rm -f foo bar baz || framework_failure -"${TARLZ}" -q -n0 -xf "${t3_bad2_lz}" +"${TARLZ}" -q -n0 -xf "${bad2_lz}" [ $? = 2 ] || test_failed $LINENO [ ! -e foo ] || test_failed $LINENO cmp cbar bar || test_failed $LINENO cmp cbaz baz || test_failed $LINENO rm -f foo bar baz || framework_failure -"${TARLZ}" -q -n0 -xf "${t3_bad3_lz}" +"${TARLZ}" -q -n0 -xf "${bad3_lz}" [ $? = 2 ] || test_failed $LINENO cmp cfoo foo || test_failed $LINENO [ ! -e bar ] || test_failed $LINENO cmp cbaz baz || test_failed $LINENO rm -f foo bar baz || framework_failure -"${TARLZ}" -q -n0 -xf "${t3_bad3_lz}" --keep-damaged +"${TARLZ}" -q -n0 -xf "${bad3_lz}" --keep-damaged [ $? = 2 ] || test_failed $LINENO cmp cfoo foo || test_failed $LINENO -cmp cbar bar || test_failed $LINENO +cmp cbar bar 2> /dev/null || test_failed $LINENO cmp cbaz baz || test_failed $LINENO rm -f foo bar baz || framework_failure -"${TARLZ}" -q -n0 -xf "${t3_bad4_lz}" +"${TARLZ}" -q -n0 -xf "${bad4_lz}" [ $? = 2 ] || test_failed $LINENO [ ! -e foo ] || test_failed $LINENO [ ! -e bar ] || test_failed $LINENO cmp cbaz baz || test_failed $LINENO rm -f foo bar baz || framework_failure -"${TARLZ}" -q -n0 -xf "${t3_bad4_lz}" --keep-damaged +"${TARLZ}" -q -n0 -xf "${bad4_lz}" --keep-damaged [ $? = 2 ] || test_failed $LINENO [ ! -e foo ] || test_failed $LINENO -cmp cbar bar || test_failed $LINENO +cmp cbar bar 2> /dev/null || test_failed $LINENO cmp cbaz baz || test_failed $LINENO rm -f foo bar baz || framework_failure -"${TARLZ}" -q -n0 -xf "${t3_bad5_lz}" +"${TARLZ}" -q -n0 -xf "${bad5_lz}" [ $? = 2 ] || test_failed $LINENO [ ! -e foo ] || test_failed $LINENO [ ! -e bar ] || test_failed $LINENO cmp cbaz baz || test_failed $LINENO rm -f foo bar baz || framework_failure -"${TARLZ}" -q -n0 -xf "${t3_bad5_lz}" --keep-damaged +"${TARLZ}" -q -n0 -xf "${bad5_lz}" --keep-damaged [ $? = 2 ] || test_failed $LINENO -cmp cfoo foo || test_failed $LINENO +cmp cfoo foo 2> /dev/null || test_failed $LINENO [ ! -e bar ] || test_failed $LINENO cmp cbaz baz || test_failed $LINENO rm -f foo bar baz || framework_failure -"${TARLZ}" -q -n0 -xf "${testdir}"/test3_bad6.tar.lz +"${TARLZ}" -q -n0 -xf "${bad6_lz}" [ $? = 2 ] || test_failed $LINENO cmp cfoo foo || test_failed $LINENO cmp cbar bar || test_failed $LINENO cmp cbaz baz || test_failed $LINENO -rm -f foo bar baz || framework_failure -# test --list, --extract, and --keep-damaged truncated uncompressed -"${TARLZ}" -q -tf "${test_bad1}.tar" -[ $? = 2 ] || test_failed $LINENO +# test --extract and --keep-damaged uncompressed rm -f test.txt || framework_failure -"${TARLZ}" -q -xf "${test_bad1}.tar" +"${TARLZ}" -q -xf "${inbad1}.tar" [ $? = 2 ] || test_failed $LINENO [ ! -e test.txt ] || test_failed $LINENO rm -f test.txt || framework_failure -"${TARLZ}" -q -xf "${test_bad1}.tar" --keep-damaged +"${TARLZ}" -q -xf "${inbad1}.tar" --keep-damaged [ $? = 2 ] || test_failed $LINENO [ -e test.txt ] || test_failed $LINENO -cmp "${test_bad1}" test.txt || test_failed $LINENO +cmp "${inbad1}" test.txt 2> /dev/null || test_failed $LINENO rm -f test.txt || framework_failure - -# test --extract and --keep-damaged uncompressed -"${TARLZ}" -q -xf "${testdir}"/test3_bad1.tar +# +rm -f foo bar baz || framework_failure +"${TARLZ}" -q -xf "${bad1}" [ $? = 2 ] || test_failed $LINENO [ ! -e foo ] || test_failed $LINENO cmp cbar bar || test_failed $LINENO cmp cbaz baz || test_failed $LINENO rm -f foo bar baz || framework_failure -"${TARLZ}" -q -xf "${testdir}"/test3_bad2.tar +"${TARLZ}" -q -xf "${bad2}" [ $? = 2 ] || test_failed $LINENO cmp cfoo foo || test_failed $LINENO [ ! -e bar ] || test_failed $LINENO cmp cbaz baz || test_failed $LINENO rm -f foo bar baz || framework_failure -"${TARLZ}" -q -xf "${testdir}"/test3_bad3.tar +"${TARLZ}" -q -xf "${bad3}" [ $? = 2 ] || test_failed $LINENO cmp cfoo foo || test_failed $LINENO cmp cbar bar || test_failed $LINENO [ ! -e baz ] || test_failed $LINENO rm -f foo bar baz || framework_failure -"${TARLZ}" -q -xf "${testdir}"/test3_bad4.tar +"${TARLZ}" -q -xf "${bad4}" [ $? = 2 ] || test_failed $LINENO cmp cfoo foo || test_failed $LINENO [ ! -e bar ] || test_failed $LINENO [ ! -e baz ] || test_failed $LINENO rm -f foo bar baz || framework_failure -"${TARLZ}" -q -xf "${testdir}"/test3_bad5.tar +"${TARLZ}" -q -xf "${bad5}" [ $? = 2 ] || test_failed $LINENO cmp cfoo foo || test_failed $LINENO cmp cbar bar || test_failed $LINENO @@ -1511,20 +1469,20 @@ cmp cbaz baz || test_failed $LINENO rm -f cfoo cbar cbaz foo bar baz || framework_failure # rm -f test3.tar.lz || framework_failure -"${TARLZ}" -q -xf "${testdir}"/tlz_in_tar1.tar +"${TARLZ}" -q -xf "${tlzit1}" [ $? = 2 ] || test_failed $LINENO [ ! -e foo ] || test_failed $LINENO [ ! -e bar ] || test_failed $LINENO [ ! -e baz ] || test_failed $LINENO [ ! -e test3.tar.lz ] || test_failed $LINENO rm -f foo bar baz test3.tar.lz || framework_failure -"${TARLZ}" -q -xf "${testdir}"/tlz_in_tar2.tar +"${TARLZ}" -q -xf "${tlzit2}" [ $? = 2 ] || test_failed $LINENO [ ! -e foo ] || test_failed $LINENO [ ! -e bar ] || test_failed $LINENO [ ! -e baz ] || test_failed $LINENO -cmp t3.tar.lz test3.tar.lz || test_failed $LINENO -rm -f foo bar baz t3.tar.lz test3.tar.lz || framework_failure +cmp "${test3_lz}" test3.tar.lz || test_failed $LINENO +rm -f foo bar baz test3.tar.lz || framework_failure echo if [ ${fail} = 0 ] ; then diff --git a/testsuite/eoa_blocks.tar b/testsuite/eoa_blocks.tar new file mode 100644 index 0000000..06d7405 Binary files /dev/null and b/testsuite/eoa_blocks.tar differ diff --git a/testsuite/eoa_blocks.lz b/testsuite/eoa_blocks.tar.lz similarity index 100% rename from testsuite/eoa_blocks.lz rename to testsuite/eoa_blocks.tar.lz diff --git a/testsuite/t155.tar b/testsuite/t155.tar new file mode 100644 index 0000000..f2b8a4e Binary files /dev/null and b/testsuite/t155.tar differ diff --git a/testsuite/t155_fv1.tar b/testsuite/t155_fv1.tar new file mode 100644 index 0000000..1ef64c3 Binary files /dev/null and b/testsuite/t155_fv1.tar differ diff --git a/testsuite/t155_fv2.tar b/testsuite/t155_fv2.tar new file mode 100644 index 0000000..f732b30 Binary files /dev/null and b/testsuite/t155_fv2.tar differ diff --git a/testsuite/t155_fv3.tar b/testsuite/t155_fv3.tar new file mode 100644 index 0000000..fe5db13 Binary files /dev/null and b/testsuite/t155_fv3.tar differ diff --git a/testsuite/t155_fv7.tar.lz b/testsuite/t155_fv7.tar.lz deleted file mode 100644 index 0e87d6e..0000000 Binary files a/testsuite/t155_fv7.tar.lz and /dev/null differ diff --git a/testsuite/test.txt.lz b/testsuite/test.txt.lz new file mode 100644 index 0000000..5dc169f Binary files /dev/null and b/testsuite/test.txt.lz differ diff --git a/testsuite/test.txt.tar b/testsuite/test.txt.tar new file mode 100644 index 0000000..151ab1e Binary files /dev/null and b/testsuite/test.txt.tar differ diff --git a/testsuite/test.txt.tar.lz b/testsuite/test.txt.tar.lz index d2f5aa1..654b2b0 100644 Binary files a/testsuite/test.txt.tar.lz and b/testsuite/test.txt.tar.lz differ diff --git a/testsuite/test3.tar b/testsuite/test3.tar new file mode 100644 index 0000000..d58fb45 Binary files /dev/null and b/testsuite/test3.tar differ diff --git a/testsuite/test3.tar.lz b/testsuite/test3.tar.lz index 549f5ac..779ace4 100644 Binary files a/testsuite/test3.tar.lz and b/testsuite/test3.tar.lz differ diff --git a/testsuite/test3_bad1.tar.lz b/testsuite/test3_bad1.tar.lz index dc5927a..9f5d40f 100644 Binary files a/testsuite/test3_bad1.tar.lz and b/testsuite/test3_bad1.tar.lz differ diff --git a/testsuite/test3_bad2.tar.lz b/testsuite/test3_bad2.tar.lz index f9203dc..182c048 100644 Binary files a/testsuite/test3_bad2.tar.lz and b/testsuite/test3_bad2.tar.lz differ diff --git a/testsuite/test3_bad3.tar.lz b/testsuite/test3_bad3.tar.lz index a1b9348..3b46163 100644 Binary files a/testsuite/test3_bad3.tar.lz and b/testsuite/test3_bad3.tar.lz differ diff --git a/testsuite/test3_bad4.tar.lz b/testsuite/test3_bad4.tar.lz index 9473744..7ac6d98 100644 Binary files a/testsuite/test3_bad4.tar.lz and b/testsuite/test3_bad4.tar.lz differ diff --git a/testsuite/test3_bad5.tar.lz b/testsuite/test3_bad5.tar.lz index 660757e..5b4feb3 100644 Binary files a/testsuite/test3_bad5.tar.lz and b/testsuite/test3_bad5.tar.lz differ diff --git a/testsuite/test3_bad6.tar.lz b/testsuite/test3_bad6.tar.lz index 7809a85..42b3888 100644 Binary files a/testsuite/test3_bad6.tar.lz and b/testsuite/test3_bad6.tar.lz differ diff --git a/testsuite/test3_crc.tar.lz b/testsuite/test3_crc.tar.lz deleted file mode 100644 index fe66267..0000000 Binary files a/testsuite/test3_crc.tar.lz and /dev/null differ diff --git a/testsuite/test3_dir.tar b/testsuite/test3_dir.tar new file mode 100644 index 0000000..e0c2b29 Binary files /dev/null and b/testsuite/test3_dir.tar differ diff --git a/testsuite/test3_eoa1.tar b/testsuite/test3_eoa1.tar new file mode 100644 index 0000000..175b807 Binary files /dev/null and b/testsuite/test3_eoa1.tar differ diff --git a/testsuite/test3_eoa1.tar.lz b/testsuite/test3_eoa1.tar.lz index 3491bec..0eb86e4 100644 Binary files a/testsuite/test3_eoa1.tar.lz and b/testsuite/test3_eoa1.tar.lz differ diff --git a/testsuite/test3_eoa2.tar b/testsuite/test3_eoa2.tar new file mode 100644 index 0000000..458be1e Binary files /dev/null and b/testsuite/test3_eoa2.tar differ diff --git a/testsuite/test3_eoa2.tar.lz b/testsuite/test3_eoa2.tar.lz index 9e90248..1f47953 100644 Binary files a/testsuite/test3_eoa2.tar.lz and b/testsuite/test3_eoa2.tar.lz differ diff --git a/testsuite/test3_eoa3.tar b/testsuite/test3_eoa3.tar new file mode 100644 index 0000000..3003a93 Binary files /dev/null and b/testsuite/test3_eoa3.tar differ diff --git a/testsuite/test3_eoa3.tar.lz b/testsuite/test3_eoa3.tar.lz index 2f7bea5..20ba9f8 100644 Binary files a/testsuite/test3_eoa3.tar.lz and b/testsuite/test3_eoa3.tar.lz differ diff --git a/testsuite/test3_eoa4.tar b/testsuite/test3_eoa4.tar new file mode 100644 index 0000000..4012fea Binary files /dev/null and b/testsuite/test3_eoa4.tar differ diff --git a/testsuite/test3_eoa4.tar.lz b/testsuite/test3_eoa4.tar.lz index 42a629c..1593feb 100644 Binary files a/testsuite/test3_eoa4.tar.lz and b/testsuite/test3_eoa4.tar.lz differ diff --git a/testsuite/test3_eoa5.tar.lz b/testsuite/test3_eoa5.tar.lz index a633a0a..156bd3a 100644 Binary files a/testsuite/test3_eoa5.tar.lz and b/testsuite/test3_eoa5.tar.lz differ diff --git a/testsuite/test3_gh1.tar b/testsuite/test3_gh1.tar new file mode 100644 index 0000000..f969561 Binary files /dev/null and b/testsuite/test3_gh1.tar differ diff --git a/testsuite/test3_gh1.tar.lz b/testsuite/test3_gh1.tar.lz index d7cb082..d38f46b 100644 Binary files a/testsuite/test3_gh1.tar.lz and b/testsuite/test3_gh1.tar.lz differ diff --git a/testsuite/test3_gh2.tar b/testsuite/test3_gh2.tar new file mode 100644 index 0000000..f5f0c31 Binary files /dev/null and b/testsuite/test3_gh2.tar differ diff --git a/testsuite/test3_gh2.tar.lz b/testsuite/test3_gh2.tar.lz index 92c93a7..48f18dd 100644 Binary files a/testsuite/test3_gh2.tar.lz and b/testsuite/test3_gh2.tar.lz differ diff --git a/testsuite/test3_gh3.tar b/testsuite/test3_gh3.tar new file mode 100644 index 0000000..e0d3a9d Binary files /dev/null and b/testsuite/test3_gh3.tar differ diff --git a/testsuite/test3_gh3.tar.lz b/testsuite/test3_gh3.tar.lz index 3af21ef..89a31a6 100644 Binary files a/testsuite/test3_gh3.tar.lz and b/testsuite/test3_gh3.tar.lz differ diff --git a/testsuite/test3_gh4.tar b/testsuite/test3_gh4.tar new file mode 100644 index 0000000..0655c31 Binary files /dev/null and b/testsuite/test3_gh4.tar differ diff --git a/testsuite/test3_gh4.tar.lz b/testsuite/test3_gh4.tar.lz index 5e291e3..5b9f605 100644 Binary files a/testsuite/test3_gh4.tar.lz and b/testsuite/test3_gh4.tar.lz differ diff --git a/testsuite/test3_gh5.tar.lz b/testsuite/test3_gh5.tar.lz index 41c48f3..b8f4abe 100644 Binary files a/testsuite/test3_gh5.tar.lz and b/testsuite/test3_gh5.tar.lz differ diff --git a/testsuite/test3_gh6.tar.lz b/testsuite/test3_gh6.tar.lz index 4b27b11..7be9aca 100644 Binary files a/testsuite/test3_gh6.tar.lz and b/testsuite/test3_gh6.tar.lz differ diff --git a/testsuite/test3_gh7.tar.lz b/testsuite/test3_gh7.tar.lz deleted file mode 100644 index 8f7f629..0000000 Binary files a/testsuite/test3_gh7.tar.lz and /dev/null differ diff --git a/testsuite/test3_gh8.tar.lz b/testsuite/test3_gh8.tar.lz deleted file mode 100644 index ca78e35..0000000 Binary files a/testsuite/test3_gh8.tar.lz and /dev/null differ diff --git a/testsuite/test3_nn.tar b/testsuite/test3_nn.tar new file mode 100644 index 0000000..c738dee Binary files /dev/null and b/testsuite/test3_nn.tar differ diff --git a/testsuite/test3_nn.tar.lz b/testsuite/test3_nn.tar.lz index ea1343d..8f78c1b 100644 Binary files a/testsuite/test3_nn.tar.lz and b/testsuite/test3_nn.tar.lz differ diff --git a/testsuite/test3_sm1.tar.lz b/testsuite/test3_sm1.tar.lz index 177c2a0..6eb3947 100644 Binary files a/testsuite/test3_sm1.tar.lz and b/testsuite/test3_sm1.tar.lz differ diff --git a/testsuite/test3_sm2.tar.lz b/testsuite/test3_sm2.tar.lz index 6fbc8a7..f312fcb 100644 Binary files a/testsuite/test3_sm2.tar.lz and b/testsuite/test3_sm2.tar.lz differ diff --git a/testsuite/test3_sm3.tar.lz b/testsuite/test3_sm3.tar.lz index 60fcd9f..82ceb18 100644 Binary files a/testsuite/test3_sm3.tar.lz and b/testsuite/test3_sm3.tar.lz differ diff --git a/testsuite/test3_sm4.tar.lz b/testsuite/test3_sm4.tar.lz index 31c6055..601a640 100644 Binary files a/testsuite/test3_sm4.tar.lz and b/testsuite/test3_sm4.tar.lz differ diff --git a/testsuite/test3_uk.tar.lz b/testsuite/test3_uk.tar.lz deleted file mode 100644 index 4f22a5f..0000000 Binary files a/testsuite/test3_uk.tar.lz and /dev/null differ diff --git a/testsuite/tlz_in_tar1.tar b/testsuite/tlz_in_tar1.tar index a29026d..f2dfd6c 100644 Binary files a/testsuite/tlz_in_tar1.tar and b/testsuite/tlz_in_tar1.tar differ diff --git a/testsuite/tlz_in_tar2.tar b/testsuite/tlz_in_tar2.tar index f077358..be860c6 100644 Binary files a/testsuite/tlz_in_tar2.tar and b/testsuite/tlz_in_tar2.tar differ