From owner-svn-src-all@FreeBSD.ORG Tue Mar 18 00:59:53 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 07E0AAE3; Tue, 18 Mar 2014 00:59:53 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E53A9154; Tue, 18 Mar 2014 00:59:52 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s2I0xqa4049978; Tue, 18 Mar 2014 00:59:52 GMT (envelope-from delphij@svn.freebsd.org) Received: (from delphij@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s2I0xnS8049957; Tue, 18 Mar 2014 00:59:49 GMT (envelope-from delphij@svn.freebsd.org) Message-Id: <201403180059.s2I0xnS8049957@svn.freebsd.org> From: Xin LI Date: Tue, 18 Mar 2014 00:59:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r263285 - in stable/10/contrib/xz: . src/liblzma/api/lzma src/liblzma/check src/liblzma/common src/lzmainfo src/xz src/xzdec X-SVN-Group: stable-10 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 00:59:53 -0000 Author: delphij Date: Tue Mar 18 00:59:49 2014 New Revision: 263285 URL: http://svnweb.freebsd.org/changeset/base/263285 Log: MFC r262754: MFV r253848 (mm): Update vendor/xz from v5.0 branch to post-5.0.5 Modified: stable/10/contrib/xz/ChangeLog stable/10/contrib/xz/README stable/10/contrib/xz/THANKS stable/10/contrib/xz/src/liblzma/api/lzma/version.h stable/10/contrib/xz/src/liblzma/check/sha256.c stable/10/contrib/xz/src/liblzma/common/alone_decoder.c stable/10/contrib/xz/src/liblzma/common/alone_decoder.h stable/10/contrib/xz/src/liblzma/common/auto_decoder.c stable/10/contrib/xz/src/lzmainfo/lzmainfo.1 stable/10/contrib/xz/src/xz/coder.c stable/10/contrib/xz/src/xz/coder.h stable/10/contrib/xz/src/xz/file_io.c stable/10/contrib/xz/src/xz/list.c stable/10/contrib/xz/src/xz/util.c stable/10/contrib/xz/src/xz/xz.1 stable/10/contrib/xz/src/xzdec/xzdec.1 stable/10/contrib/xz/src/xzdec/xzdec.c Directory Properties: stable/10/ (props changed) Modified: stable/10/contrib/xz/ChangeLog ============================================================================== --- stable/10/contrib/xz/ChangeLog Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/ChangeLog Tue Mar 18 00:59:49 2014 (r263285) @@ -1,3 +1,348 @@ +commit b69900ed0b2f914fc6c0a180dcb522dbe5b80ea7 +Author: Lasse Collin +Date: Sun Jun 30 18:02:27 2013 +0300 + + Man pages: Use similar syntax for synopsis as in xz. + + The man pages of lzmainfo, xzmore, and xzdec had similar + constructs as the man page of xz had before the commit + eb6ca9854b8eb9fbf72497c1cf608d6b19d2d494. Eric S. Raymond + didn't mention these man pages in his bug report, but + it's nice to be consistent. + +commit cf4a1e1879d89be314ef3c064bd2656ea452f87e +Author: Lasse Collin +Date: Sun Jun 30 15:55:09 2013 +0300 + + Update NEWS for 5.0.5. + +commit cb94bb6d1f34e1e93c2d634ea9c3b7dfb3981d05 +Author: Lasse Collin +Date: Sun Jun 30 15:54:38 2013 +0300 + + Bump version and soname for 5.0.5. + +commit b7dee202d5b041ccae028d0c5433b83cecbe9e5d +Author: Lasse Collin +Date: Fri Jun 28 23:56:17 2013 +0300 + + xz: Fix return value type in io_write_buf(). + + It didn't affect the behavior of the code since -1 + becomes true anyway. + +commit 265e7b44d804b47373f10b7da28350db7611cea6 +Author: Lasse Collin +Date: Fri Jun 28 18:46:13 2013 +0300 + + xz: Remove an outdated NetBSD-specific comment. + + Nowadays errno == EFTYPE is documented in open(2). + +commit 78c2f8db902195468b8249c432252a6b281db836 +Author: Lasse Collin +Date: Fri Jun 28 18:09:47 2013 +0300 + + xz: Fix error detection of fcntl(fd, F_SETFL, flags) calls. + + POSIX says that fcntl(fd, F_SETFL, flags) returns -1 on + error and "other than -1" on success. This is how it is + documented e.g. on OpenBSD too. On Linux, success with + F_SETFL is always 0 (at least accorinding to fcntl(2) + from man-pages 3.51). + +commit 91750dff8f2c654ff636f12a2acdffe5492374b3 +Author: Lasse Collin +Date: Fri Jun 28 17:36:47 2013 +0300 + + xz: Fix use of wrong variable in a fcntl() call. + + Due to a wrong variable name, when writing a sparse file + to standard output, *all* file status flags were cleared + (to the extent the operating system allowed it) instead of + only clearing the O_APPEND flag. In practice this worked + fine in the common situations on GNU/Linux, but I didn't + check how it behaved elsewhere. + + The original flags were still restored correctly. I still + changed the code to use a separate boolean variable to + indicate when the flags should be restored instead of + relying on a special value in stdout_flags. + +commit e11888a79a4a77a69afde60445880d44f63d01aa +Author: Lasse Collin +Date: Wed Jun 26 13:30:57 2013 +0300 + + xz: Check the value of lzma_stream_flags.version in --list. + + It is a no-op for now, but if an old xz version is used + together with a newer liblzma that supports something new, + then this check becomes important and will stop the old xz + from trying to parse files that it won't understand. + +commit f39ddd88f3222219ada88998cf30abfdd3e0e96c +Author: Lasse Collin +Date: Wed Jun 26 12:17:00 2013 +0300 + + Build: Require Automake 1.12 and use serial-tests option. + + It should actually still work with Automake 1.10 if + the serial-tests option is removed. Automake 1.13 started + using parallel tests by default and the option to get + the old behavior isn't supported before 1.12. + + At least for now, parallel tests don't improve anything + in XZ Utils but they hide the progress output from + test_compress.sh. + +commit cb84e278027a90e9827a6f4d3bb0b4d4744a2fbb +Author: Lasse Collin +Date: Sun Jun 23 17:36:47 2013 +0300 + + xz: Validate Uncompressed Size from Block Header in list.c. + + This affects only "xz -lvv". Normal decompression with xz + already detected if Block Header and Index had mismatched + Uncompressed Size fields. So this just makes "xz -lvv" + show such files as corrupt instead of showing the + Uncompressed Size from Index. + +commit f01780fce454c7489f7dcbf806299b50da5f51b7 +Author: Lasse Collin +Date: Wed Jun 26 10:58:58 2013 +0300 + + Update THANKS. + +commit d98ede7d700b892e32d9c2f46563b6ebc566786d +Author: Lasse Collin +Date: Fri Jun 21 22:04:45 2013 +0300 + + xz: Make the man page more friendly to doclifter. + + Thanks to Eric S. Raymond. + +commit 19b447b64b3f520cd5b11429000b092f7c76709b +Author: Lasse Collin +Date: Fri Jun 21 21:54:59 2013 +0300 + + xz: A couple of man page fixes. + + Now the interaction of presets and custom filter chains + is described correctly. Earlier it contradicted itself. + + Thanks to DevHC who reported these issues on IRC to me + on 2012-12-14. + +commit 45edf2966fc9a4d2eae8f84b2fa027fb4fa1df8b +Author: Lasse Collin +Date: Fri Jun 21 21:50:26 2013 +0300 + + xz: Fix interaction between preset and custom filter chains. + + There was somewhat illogical behavior when --extreme was + specified and mixed with custom filter chains. + + Before this commit, "xz -9 --lzma2 -e" was equivalent + to "xz --lzma2". After it is equivalent to "xz -6e" + (all earlier preset options get forgotten when a custom + filter chain is specified and the default preset is 6 + to which -e is applied). I find this less illogical. + + This also affects the meaning of "xz -9e --lzma2 -7". + Earlier it was equivalent to "xz -7e" (the -e specified + before a custom filter chain wasn't forgotten). Now it + is "xz -7". Note that "xz -7e" still is the same as "xz -e7". + + Hopefully very few cared about this in the first place, + so pretty much no one should even notice this change. + + Thanks to Conley Moorhous. + +commit b065984e5a9272eb50bc0c6d3731e6199c0ae8a8 +Author: Lasse Collin +Date: Fri Apr 8 17:53:05 2011 +0300 + + xz: Change size_t to uint32_t in a few places. + +commit 32be621f52f2e1686db88baa7b01dc1ae338f426 +Author: Lasse Collin +Date: Sat Apr 27 22:07:46 2013 +0300 + + Build: Use -Wvla with GCC if supported. + + Variable-length arrays are mandatory in C99 but optional in C11. + The code doesn't currently use any VLAs and it shouldn't in the + future either to stay compatible with C11 without requiring any + optional C11 features. + +commit efb07cfba65e9e05984c02cd796c1b0338ce04dc +Author: Lasse Collin +Date: Mon Apr 15 19:29:09 2013 +0300 + + xzdec: Improve the --help message. + + The options are now ordered in the same order as in xz's help + message. + + Descriptions were added to the options that are ignored. + I left them in parenthesis even if it looks a bit weird + because I find it easier to spot the ignored vs. non-ignored + options from the list that way. + +commit e3c8be13699e2813f5e2879d8187444b46d82d89 +Author: Lasse Collin +Date: Fri Apr 5 19:25:40 2013 +0300 + + Update THANKS. + +commit ad8282efe483612f6b5544f9a0d2e4914fb2532a +Author: Jeff Bastian +Date: Wed Apr 3 13:59:17 2013 +0200 + + xzgrep: make the '-h' option to be --no-filename equivalent + + * src/scripts/xzgrep.in: Accept the '-h' option in argument parsing. + +commit 9271a3eb0e022b23e8712154be851d0afe4c02e4 +Author: Lasse Collin +Date: Fri Apr 5 19:34:09 2013 +0300 + + liblzma: Be less picky in lzma_alone_decoder(). + + To avoid false positives when detecting .lzma files, + rare values in dictionary size and uncompressed size fields + were rejected. They will still be rejected if .lzma files + are decoded with lzma_auto_decoder(), but when using + lzma_alone_decoder() directly, such files will now be accepted. + Hopefully this is an OK compromise. + + This doesn't affect xz because xz still has its own file + format detection code. This does affect lzmadec though. + So after this commit lzmadec will accept files that xz or + xz-emulating-lzma doesn't. + + NOTE: lzma_alone_decoder() still won't decode all .lzma files + because liblzma's LZMA decoder doesn't support lc + lp > 4. + + Reported here: + http://sourceforge.net/projects/lzmautils/forums/forum/708858/topic/7068827 + + Conflicts: + src/liblzma/common/alone_decoder.c + src/liblzma/common/alone_decoder.h + +commit 211b931cee58626c1d2e021810cb108cb5cbc10f +Author: Lasse Collin +Date: Tue Mar 5 19:14:50 2013 +0200 + + Avoid unneeded use of awk in xzless. + + Use "read" instead of "awk" in xzless to get the version + number of "less". The need for awk was introduced in + the commit db5c1817fabf7cbb9e4087b1576eb26f0747338e. + + Thanks to Ariel P for the patch. + +commit 9f62fd9605eade23b62b07a235d1f02156f7a5c6 +Author: Jonathan Nieder +Date: Mon Nov 19 00:10:10 2012 -0800 + + xzless: Make "less -V" parsing more robust + + In v4.999.9beta~30 (xzless: Support compressed standard input, + 2009-08-09), xzless learned to parse ‘less -V’ output to figure out + whether less is new enough to handle $LESSOPEN settings starting + with “|-”. That worked well for a while, but the version string from + ‘less’ versions 448 (June, 2012) is misparsed, producing a warning: + + $ xzless /tmp/test.xz; echo $? + /usr/bin/xzless: line 49: test: 456 (GNU regular expressions): \ + integer expression expected + 0 + + More precisely, modern ‘less’ lists the regexp implementation along + with its version number, and xzless passes the entire version number + with attached parenthetical phrase as a number to "test $a -gt $b", + producing the above confusing message. + + $ less-444 -V | head -1 + less 444 + $ less -V | head -1 + less 456 (no regular expressions) + + So relax the pattern matched --- instead of expecting "less ", + look for a line of the form "less [ (extra parenthetical)]". + While at it, improve the behavior when no matching line is found --- + instead of producing a cryptic message, we can fall back on a LESSPIPE + setting that is supported by all versions of ‘less’. + + The implementation uses "awk" for simplicity. Hopefully that’s + portable enough. + + Reported-by: Jörg-Volker Peetz + Signed-off-by: Jonathan Nieder + +commit 1d05980f5b5c2c94d833001daccacce4a466876e +Author: Lasse Collin +Date: Wed Oct 3 15:54:24 2012 +0300 + + xz: Fix the note about --rsyncable on the man page. + +commit fb68497333598688d309a92838d91fd560f7e9f0 +Author: Lasse Collin +Date: Fri Sep 28 20:11:09 2012 +0300 + + xz: Improve handling of failed realloc in xrealloc. + + Thanks to Jim Meyering. + +commit 75013db6d4d63c195bd8b8d45729b4be0665a812 +Author: Lasse Collin +Date: Sat Dec 15 20:01:02 2012 +0200 + + A few typo fixes to comments and the xz man page. + + Thanks to Jim Meyering. + +commit e44b21839b1dcbac5097be39b87dd2ddb6e114fd +Author: Lasse Collin +Date: Thu Aug 2 17:13:30 2012 +0300 + + Build: Bump gettext version requirement to 0.18. + + Otherwise too old version of m4/lib-link.m4 gets included + when autoreconf -fi is run. + +commit fd3dbb23ca7e75a7a888d7e897c381dc06308307 +Author: Lasse Collin +Date: Thu Jul 5 07:36:28 2012 +0300 + + Tests: Remove tests/test_block.c that had gotten committed accidentally. + +commit 05a735d279d74af437c31f25f69aded4713c1a3d +Author: Lasse Collin +Date: Thu Jul 5 07:33:35 2012 +0300 + + Build: Include macosx/build.sh in the distribution. + + It has been in the Git repository since 2010 but probably + few people have seen it since it hasn't been included in + the release tarballs. :-( + +commit 4e6d62793b5e7b87edcc93c7ded072c1ecd94173 +Author: Lasse Collin +Date: Thu Jul 5 07:24:45 2012 +0300 + + Docs: Fix the name LZMA Utils -> XZ Utils in debug/README. + +commit dd95b5e7614baf1f07a1316b5106bd616a9efa79 +Author: Lasse Collin +Date: Thu Jul 5 07:23:17 2012 +0300 + + Include debug/translation.bash in the distribution. + + Also fix the script name mentioned in README. + commit 20778053a07eb90c159c1377ca8dc05a90fd530b Author: Lasse Collin Date: Fri Jun 22 14:36:16 2012 +0300 Modified: stable/10/contrib/xz/README ============================================================================== --- stable/10/contrib/xz/README Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/README Tue Mar 18 00:59:49 2014 (r263285) @@ -210,8 +210,8 @@ XZ Utils # make -C po update-po make install - bash debug/translations.bash | less - bash debug/translations.bash | less -S # For --list outputs + bash debug/translation.bash | less + bash debug/translation.bash | less -S # For --list outputs Repeat the above as needed (no need to re-run configure though). Modified: stable/10/contrib/xz/THANKS ============================================================================== --- stable/10/contrib/xz/THANKS Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/THANKS Tue Mar 18 00:59:49 2014 (r263285) @@ -6,6 +6,7 @@ Some people have helped more, some less, has been important. :-) In alphabetical order: - Mark Adler - H. Peter Anvin + - Jeff Bastian - Nelson H. F. Beebe - Karl Berry - Anders F. Björklund @@ -47,6 +48,7 @@ has been important. :-) In alphabetical - Bela Lubkin - Gregory Margo - Jim Meyering + - Conley Moorhous - Rafał Mużyło - Adrien Nader - Hongbo Ni @@ -58,8 +60,10 @@ has been important. :-) In alphabetical - Diego Elio Pettenò - Elbert Pol - Mikko Pouru + - Pavel Raiskup - Robert Readman - Bernhard Reutner-Fischer + - Eric S. Raymond - Cristian Rodríguez - Christian von Roques - Jukka Salmi Modified: stable/10/contrib/xz/src/liblzma/api/lzma/version.h ============================================================================== --- stable/10/contrib/xz/src/liblzma/api/lzma/version.h Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/src/liblzma/api/lzma/version.h Tue Mar 18 00:59:49 2014 (r263285) @@ -22,7 +22,7 @@ */ #define LZMA_VERSION_MAJOR 5 #define LZMA_VERSION_MINOR 0 -#define LZMA_VERSION_PATCH 4 +#define LZMA_VERSION_PATCH 5 #define LZMA_VERSION_STABILITY LZMA_VERSION_STABILITY_STABLE #ifndef LZMA_VERSION_COMMIT Modified: stable/10/contrib/xz/src/liblzma/check/sha256.c ============================================================================== --- stable/10/contrib/xz/src/liblzma/check/sha256.c Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/src/liblzma/check/sha256.c Tue Mar 18 00:59:49 2014 (r263285) @@ -6,7 +6,6 @@ /// \todo Crypto++ has x86 ASM optimizations. They use SSE so if they /// are imported to liblzma, SSE instructions need to be used /// conditionally to keep the code working on older boxes. -/// We could also support using some external libary for SHA-256. // // This code is based on the code found from 7-Zip, which has a modified // version of the SHA-256 found from Crypto++ . Modified: stable/10/contrib/xz/src/liblzma/common/alone_decoder.c ============================================================================== --- stable/10/contrib/xz/src/liblzma/common/alone_decoder.c Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/src/liblzma/common/alone_decoder.c Tue Mar 18 00:59:49 2014 (r263285) @@ -26,6 +26,11 @@ struct lzma_coder_s { SEQ_CODE, } sequence; + /// If true, reject files that are unlikely to be .lzma files. + /// If false, more non-.lzma files get accepted and will give + /// LZMA_DATA_ERROR either immediately or after a few output bytes. + bool picky; + /// Position in the header fields size_t pos; @@ -68,13 +73,13 @@ alone_decode(lzma_coder *coder, |= (size_t)(in[*in_pos]) << (coder->pos * 8); if (++coder->pos == 4) { - if (coder->options.dict_size != UINT32_MAX) { + if (coder->picky && coder->options.dict_size + != UINT32_MAX) { // A hack to ditch tons of false positives: // We allow only dictionary sizes that are // 2^n or 2^n + 2^(n-1). LZMA_Alone created // only files with 2^n, but accepts any - // dictionary size. If someone complains, this - // will be reconsidered. + // dictionary size. uint32_t d = coder->options.dict_size - 1; d |= d >> 2; d |= d >> 3; @@ -103,9 +108,9 @@ alone_decode(lzma_coder *coder, // Another hack to ditch false positives: Assume that // if the uncompressed size is known, it must be less - // than 256 GiB. Again, if someone complains, this - // will be reconsidered. - if (coder->uncompressed_size != LZMA_VLI_UNKNOWN + // than 256 GiB. + if (coder->picky + && coder->uncompressed_size != LZMA_VLI_UNKNOWN && coder->uncompressed_size >= (LZMA_VLI_C(1) << 38)) return LZMA_FORMAT_ERROR; @@ -189,7 +194,7 @@ alone_decoder_memconfig(lzma_coder *code extern lzma_ret lzma_alone_decoder_init(lzma_next_coder *next, lzma_allocator *allocator, - uint64_t memlimit) + uint64_t memlimit, bool picky) { lzma_next_coder_init(&lzma_alone_decoder_init, next, allocator); @@ -208,6 +213,7 @@ lzma_alone_decoder_init(lzma_next_coder } next->coder->sequence = SEQ_PROPERTIES; + next->coder->picky = picky; next->coder->pos = 0; next->coder->options.dict_size = 0; next->coder->options.preset_dict = NULL; @@ -223,7 +229,7 @@ lzma_alone_decoder_init(lzma_next_coder extern LZMA_API(lzma_ret) lzma_alone_decoder(lzma_stream *strm, uint64_t memlimit) { - lzma_next_strm_init(lzma_alone_decoder_init, strm, memlimit); + lzma_next_strm_init(lzma_alone_decoder_init, strm, memlimit, false); strm->internal->supported_actions[LZMA_RUN] = true; strm->internal->supported_actions[LZMA_FINISH] = true; Modified: stable/10/contrib/xz/src/liblzma/common/alone_decoder.h ============================================================================== --- stable/10/contrib/xz/src/liblzma/common/alone_decoder.h Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/src/liblzma/common/alone_decoder.h Tue Mar 18 00:59:49 2014 (r263285) @@ -16,7 +16,8 @@ #include "common.h" -extern lzma_ret lzma_alone_decoder_init(lzma_next_coder *next, - lzma_allocator *allocator, uint64_t memlimit); +extern lzma_ret lzma_alone_decoder_init( + lzma_next_coder *next, lzma_allocator *allocator, + uint64_t memlimit, bool picky); #endif Modified: stable/10/contrib/xz/src/liblzma/common/auto_decoder.c ============================================================================== --- stable/10/contrib/xz/src/liblzma/common/auto_decoder.c Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/src/liblzma/common/auto_decoder.c Tue Mar 18 00:59:49 2014 (r263285) @@ -54,7 +54,7 @@ auto_decode(lzma_coder *coder, lzma_allo coder->memlimit, coder->flags)); } else { return_if_error(lzma_alone_decoder_init(&coder->next, - allocator, coder->memlimit)); + allocator, coder->memlimit, true)); // If the application wants to know about missing // integrity check or about the check in general, we Modified: stable/10/contrib/xz/src/lzmainfo/lzmainfo.1 ============================================================================== --- stable/10/contrib/xz/src/lzmainfo/lzmainfo.1 Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/src/lzmainfo/lzmainfo.1 Tue Mar 18 00:59:49 2014 (r263285) @@ -4,14 +4,14 @@ .\" This file has been put into the public domain. .\" You can do whatever you want with this file. .\" -.TH LZMAINFO 1 "2010-09-27" "Tukaani" "XZ Utils" +.TH LZMAINFO 1 "2013-06-30" "Tukaani" "XZ Utils" .SH NAME lzmainfo \- show information stored in the .lzma file header .SH SYNOPSIS .B lzmainfo .RB [ \-\-help ] .RB [ \-\-version ] -.RI [ file ]... +.RI [ file... ] .SH DESCRIPTION .B lzmainfo shows information stored in the Modified: stable/10/contrib/xz/src/xz/coder.c ============================================================================== --- stable/10/contrib/xz/src/xz/coder.c Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/src/xz/coder.c Tue Mar 18 00:59:49 2014 (r263285) @@ -37,15 +37,10 @@ static io_buf in_buf; static io_buf out_buf; /// Number of filters. Zero indicates that we are using a preset. -static size_t filters_count = 0; +static uint32_t filters_count = 0; /// Number of the preset (0-9) -static size_t preset_number = 6; - -/// If a preset is used (no custom filter chain) and preset_extreme is true, -/// a significantly slower compression is used to achieve slightly better -/// compression ratio. -static bool preset_extreme = false; +static uint32_t preset_number = LZMA_PRESET_DEFAULT; /// Integrity check type static lzma_check check; @@ -63,11 +58,9 @@ coder_set_check(lzma_check new_check) } -extern void -coder_set_preset(size_t new_preset) +static void +forget_filter_chain(void) { - preset_number = new_preset; - // Setting a preset makes us forget a possibly defined custom // filter chain. while (filters_count > 0) { @@ -81,9 +74,20 @@ coder_set_preset(size_t new_preset) extern void +coder_set_preset(uint32_t new_preset) +{ + preset_number &= ~LZMA_PRESET_LEVEL_MASK; + preset_number |= new_preset; + forget_filter_chain(); + return; +} + + +extern void coder_set_extreme(void) { - preset_extreme = true; + preset_number |= LZMA_PRESET_EXTREME; + forget_filter_chain(); return; } @@ -98,6 +102,12 @@ coder_add_filter(lzma_vli id, void *opti filters[filters_count].options = options; ++filters_count; + // Setting a custom filter chain makes us forget the preset options. + // This makes a difference if one specifies e.g. "xz -9 --lzma2 -e" + // where the custom filter chain resets the preset level back to + // the default 6, making the example equivalent to "xz -6e". + preset_number = LZMA_PRESET_DEFAULT; + return; } @@ -134,9 +144,6 @@ coder_set_compression_settings(void) } // Get the preset for LZMA1 or LZMA2. - if (preset_extreme) - preset_number |= LZMA_PRESET_EXTREME; - if (lzma_lzma_preset(&opt_lzma, preset_number)) message_bug(); Modified: stable/10/contrib/xz/src/xz/coder.h ============================================================================== --- stable/10/contrib/xz/src/xz/coder.h Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/src/xz/coder.h Tue Mar 18 00:59:49 2014 (r263285) @@ -46,7 +46,7 @@ extern bool opt_auto_adjust; extern void coder_set_check(lzma_check check); /// Set preset number -extern void coder_set_preset(size_t new_preset); +extern void coder_set_preset(uint32_t new_preset); /// Enable extreme mode extern void coder_set_extreme(void); Modified: stable/10/contrib/xz/src/xz/file_io.c ============================================================================== --- stable/10/contrib/xz/src/xz/file_io.c Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/src/xz/file_io.c Tue Mar 18 00:59:49 2014 (r263285) @@ -41,9 +41,10 @@ static bool warn_fchown; static bool try_sparse = true; #ifndef TUKLIB_DOSLIKE -/// File status flags of standard output. This is used by io_open_dest() -/// and io_close_dest(). -static int stdout_flags = 0; +/// Original file status flags of standard output. This is used by +/// io_open_dest() and io_close_dest() to save and restore the flags. +static int stdout_flags; +static bool restore_stdout_flags = false; #endif @@ -397,10 +398,6 @@ io_open_src_real(file_pair *pair) was_symlink = true; # elif defined(__NetBSD__) - // As of 2010-09-05, NetBSD doesn't document what errno is - // used with O_NOFOLLOW. It is EFTYPE though, and I - // understood that is very unlikely to change even though - // it is undocumented. if (errno == EFTYPE) was_symlink = true; @@ -441,7 +438,7 @@ io_open_src_real(file_pair *pair) flags &= ~O_NONBLOCK; - if (fcntl(pair->src_fd, F_SETFL, flags)) + if (fcntl(pair->src_fd, F_SETFL, flags) == -1) goto error_msg; } #endif @@ -634,11 +631,11 @@ io_open_dest_real(file_pair *pair) if (!S_ISREG(pair->dest_st.st_mode)) return false; - const int flags = fcntl(STDOUT_FILENO, F_GETFL); - if (flags == -1) + stdout_flags = fcntl(STDOUT_FILENO, F_GETFL); + if (stdout_flags == -1) return false; - if (flags & O_APPEND) { + if (stdout_flags & O_APPEND) { // Creating a sparse file is not possible // when O_APPEND is active (it's used by // shell's >> redirection). As I understand @@ -657,12 +654,14 @@ io_open_dest_real(file_pair *pair) return false; if (fcntl(STDOUT_FILENO, F_SETFL, - stdout_flags & ~O_APPEND)) + stdout_flags & ~O_APPEND) + == -1) return false; - // Remember the flags so that io_close_dest() - // can restore them. - stdout_flags = flags; + // Disabling O_APPEND succeeded. Mark + // that the flags should be restored + // in io_close_dest(). + restore_stdout_flags = true; } else if (lseek(STDOUT_FILENO, 0, SEEK_CUR) != pair->dest_st.st_size) { @@ -703,13 +702,12 @@ io_close_dest(file_pair *pair, bool succ { #ifndef TUKLIB_DOSLIKE // If io_open_dest() has disabled O_APPEND, restore it here. - if (stdout_flags != 0) { + if (restore_stdout_flags) { assert(pair->dest_fd == STDOUT_FILENO); - const int fail = fcntl(STDOUT_FILENO, F_SETFL, stdout_flags); - stdout_flags = 0; + restore_stdout_flags = false; - if (fail) { + if (fcntl(STDOUT_FILENO, F_SETFL, stdout_flags) == -1) { message_error(_("Error restoring the O_APPEND flag " "to standard output: %s"), strerror(errno)); @@ -882,7 +880,7 @@ io_write_buf(file_pair *pair, const uint if (amount == -1) { if (errno == EINTR) { if (user_abort) - return -1; + return true; continue; } Modified: stable/10/contrib/xz/src/xz/list.c ============================================================================== --- stable/10/contrib/xz/src/xz/list.c Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/src/xz/list.c Tue Mar 18 00:59:49 2014 (r263285) @@ -203,6 +203,20 @@ parse_indexes(xz_file_info *xfi, file_pa goto error; } + // Check that the Stream Footer doesn't specify something + // that we don't support. This can only happen if the xz + // version is older than liblzma and liblzma supports + // something new. + // + // It is enough to check Stream Footer. Stream Header must + // match when it is compared against Stream Footer with + // lzma_stream_flags_compare(). + if (footer_flags.version != 0) { + message_error("%s: %s", pair->src_name, + message_strm(LZMA_OPTIONS_ERROR)); + goto error; + } + // Check that the size of the Index field looks sane. lzma_vli index_size = footer_flags.backward_size; if ((lzma_vli)(pos) < index_size + LZMA_STREAM_HEADER_SIZE) { @@ -429,7 +443,19 @@ parse_block_header(file_pair *pair, cons switch (lzma_block_compressed_size(&block, iter->block.unpadded_size)) { case LZMA_OK: - break; + // Validate also block.uncompressed_size if it is present. + // If it isn't present, there's no need to set it since + // we aren't going to actually decompress the Block; if + // we were decompressing, then we should set it so that + // the Block decoder could validate the Uncompressed Size + // that was stored in the Index. + if (block.uncompressed_size == LZMA_VLI_UNKNOWN + || block.uncompressed_size + == iter->block.uncompressed_size) + break; + + // If the above fails, the file is corrupt so + // LZMA_DATA_ERROR is a good error code. case LZMA_DATA_ERROR: // Free the memory allocated by lzma_block_header_decode(). Modified: stable/10/contrib/xz/src/xz/util.c ============================================================================== --- stable/10/contrib/xz/src/xz/util.c Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/src/xz/util.c Tue Mar 18 00:59:49 2014 (r263285) @@ -26,9 +26,19 @@ xrealloc(void *ptr, size_t size) { assert(size > 0); + // Save ptr so that we can free it if realloc fails. + // The point is that message_fatal ends up calling stdio functions + // which in some libc implementations might allocate memory from + // the heap. Freeing ptr improves the chances that there's free + // memory for stdio functions if they need it. + void *p = ptr; ptr = realloc(ptr, size); - if (ptr == NULL) - message_fatal("%s", strerror(errno)); + + if (ptr == NULL) { + const int saved_errno = errno; + free(p); + message_fatal("%s", strerror(saved_errno)); + } return ptr; } Modified: stable/10/contrib/xz/src/xz/xz.1 ============================================================================== --- stable/10/contrib/xz/src/xz/xz.1 Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/src/xz/xz.1 Tue Mar 18 00:59:49 2014 (r263285) @@ -5,16 +5,17 @@ .\" This file has been put into the public domain. .\" You can do whatever you want with this file. .\" -.TH XZ 1 "2012-05-27" "Tukaani" "XZ Utils" +.TH XZ 1 "2013-06-21" "Tukaani" "XZ Utils" . .SH NAME xz, unxz, xzcat, lzma, unlzma, lzcat \- Compress or decompress .xz and .lzma files . .SH SYNOPSIS .B xz -.RI [ option ]... -.RI [ file ]... -.PP +.RI [ option... ] +.RI [ file... ] +. +.SH COMMAND ALIASES .B unxz is equivalent to .BR "xz \-\-decompress" . @@ -708,7 +709,7 @@ be dramatically higher than that of the DecMem contains the decompressor memory requirements. That is, the compression settings determine the memory requirements of the decompressor. -The exact decompressor memory usage is slighly more than +The exact decompressor memory usage is slightly more than the LZMA2 dictionary size, but the values in the table have been rounded up to the next full MiB. .RE @@ -897,11 +898,14 @@ if threading will be enabled by default. .SS "Custom compressor filter chains" A custom filter chain allows specifying the compression settings in detail instead of relying on -the settings associated to the preset levels. +the settings associated to the presets. When a custom filter chain is specified, -the compression preset level options -(\fB\-0\fR ... \fB\-9\fR and \fB\-\-extreme\fR) are -silently ignored. +preset options (\fB\-0\fR ... \fB\-9\fR and \fB\-\-extreme\fR) +earlier on the command line are forgotten. +If a preset option is specified +after one or more custom filter chain options, +the new preset takes effect and +the custom filter chain options specified earlier are forgotten. .PP A filter chain is comparable to piping on the command line. When compressing, the uncompressed input goes to the first filter, @@ -934,6 +938,15 @@ Extra commas in are ignored. Every option has a default value, so you need to specify only those you want to change. +.PP +To see the whole filter chain and +.IR options , +use +.B "xz \-vv" +(that is, use +.B \-\-verbose +twice). +This works also for viewing the filter chain options used by presets. .TP \fB\-\-lzma1\fR[\fB=\fIoptions\fR] .PD 0 @@ -976,13 +989,12 @@ The only supported modifier is currently .BR e , which matches .BR \-\-extreme . -The default -.I preset -is -.BR 6 , -from which the default values for the rest of the LZMA1 or LZMA2 +If no +.B preset +is specified, the default values of LZMA1 or LZMA2 .I options -are taken. +are taken from the preset +.BR 6 . .TP .BI dict= size Dictionary (history buffer) @@ -1578,7 +1590,7 @@ is supported only together with .BR \-\-info\-memory , and .BR \-\-list . -It will be supported for normal compression and +It will be supported for compression and decompression in the future. . .SS Version @@ -2179,14 +2191,15 @@ The output can vary even between differe builds of the same XZ Utils version, if different build options are used. .PP -The above means that implementing +The above means that once .B \-\-rsyncable -to create rsyncable -.B .xz -files is not going to happen without -freezing a part of the encoder -implementation, which can then be used with -.BR \-\-rsyncable . +has been implemented, +the resulting files won't necessarily be rsyncable +unless both old and new files have been compressed +with the same xz version. +This problem can be fixed if a part of the encoder +implementation is frozen to keep rsyncable output +stable across xz versions. . .SS "Embedded .xz decompressors" Embedded Modified: stable/10/contrib/xz/src/xzdec/xzdec.1 ============================================================================== --- stable/10/contrib/xz/src/xzdec/xzdec.1 Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/src/xzdec/xzdec.1 Tue Mar 18 00:59:49 2014 (r263285) @@ -4,17 +4,17 @@ .\" This file has been put into the public domain. .\" You can do whatever you want with this file. .\" -.TH XZDEC 1 "2010-09-27" "Tukaani" "XZ Utils" +.TH XZDEC 1 "2013-06-30" "Tukaani" "XZ Utils" .SH NAME xzdec, lzmadec \- Small .xz and .lzma decompressors .SH SYNOPSIS .B xzdec -.RI [ option ]... -.RI [ file ]... +.RI [ option... ] +.RI [ file... ] .br .B lzmadec -.RI [ option ]... -.RI [ file ]... +.RI [ option... ] +.RI [ file... ] .SH DESCRIPTION .B xzdec is a liblzma-based decompression-only tool for Modified: stable/10/contrib/xz/src/xzdec/xzdec.c ============================================================================== --- stable/10/contrib/xz/src/xzdec/xzdec.c Tue Mar 18 00:55:40 2014 (r263284) +++ stable/10/contrib/xz/src/xzdec/xzdec.c Tue Mar 18 00:59:49 2014 (r263285) @@ -62,13 +62,13 @@ help(void) { printf( "Usage: %s [OPTION]... [FILE]...\n" -"Uncompress files in the ." TOOL_FORMAT " format to the standard output.\n" +"Decompress files in the ." TOOL_FORMAT " format to standard output.\n" "\n" -" -c, --stdout (ignored)\n" -" -d, --decompress (ignored)\n" -" -k, --keep (ignored)\n" +" -d, --decompress (ignored, only decompression is supported)\n" +" -k, --keep (ignored, files are never deleted)\n" +" -c, --stdout (ignored, output is always written to standard output)\n" " -q, --quiet specify *twice* to suppress errors\n" -" -Q, --no-warn (ignored)\n" +" -Q, --no-warn (ignored, the exit status 2 is never used)\n" " -h, --help display this help and exit\n" " -V, --version display the version number and exit\n" "\n"