From owner-svn-src-vendor@freebsd.org Sun May 8 09:27:30 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FEE9B301F2; Sun, 8 May 2016 09:27:30 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2CF13111A; Sun, 8 May 2016 09:27:30 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u489RTKO071436; Sun, 8 May 2016 09:27:29 GMT (envelope-from delphij@FreeBSD.org) Received: (from delphij@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u489RTSK071435; Sun, 8 May 2016 09:27:29 GMT (envelope-from delphij@FreeBSD.org) Message-Id: <201605080927.u489RTSK071435@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: delphij set sender to delphij@FreeBSD.org using -f From: Xin LI Date: Sun, 8 May 2016 09:27:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299233 - vendor/file/dist/src X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 May 2016 09:27:30 -0000 Author: delphij Date: Sun May 8 09:27:29 2016 New Revision: 299233 URL: https://svnweb.freebsd.org/changeset/base/299233 Log: Apply upstream fix 11edb37a71851b5bcbd4e51ca6ad3dcbf57e9761: Revert: Don't close stdin... Modified: vendor/file/dist/src/magic.c Modified: vendor/file/dist/src/magic.c ============================================================================== --- vendor/file/dist/src/magic.c Sun May 8 09:08:05 2016 (r299232) +++ vendor/file/dist/src/magic.c Sun May 8 09:27:29 2016 (r299233) @@ -346,7 +346,7 @@ private void close_and_restore(const struct magic_set *ms, const char *name, int fd, const struct stat *sb) { - if (name == NULL) + if (fd == STDIN_FILENO || name == NULL) return; (void) close(fd); From owner-svn-src-vendor@freebsd.org Sun May 8 18:15:59 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6188DB3378F; Sun, 8 May 2016 18:15:59 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3DFC11BCC; Sun, 8 May 2016 18:15:59 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u48IFwpS032755; Sun, 8 May 2016 18:15:58 GMT (envelope-from delphij@FreeBSD.org) Received: (from delphij@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u48IFwJ8032754; Sun, 8 May 2016 18:15:58 GMT (envelope-from delphij@FreeBSD.org) Message-Id: <201605081815.u48IFwJ8032754@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: delphij set sender to delphij@FreeBSD.org using -f From: Xin LI Date: Sun, 8 May 2016 18:15:58 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299237 - vendor/file/dist/src X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 May 2016 18:15:59 -0000 Author: delphij Date: Sun May 8 18:15:58 2016 New Revision: 299237 URL: https://svnweb.freebsd.org/changeset/base/299237 Log: Apply upstream changes: 3ca1d7b07cf7b7c7433e8dabbcbc566c64c6e07f Don't make exceeding magic offsets an error; just fail this magic entry test. Consider truncated files... 20c59ad54afc7427ea680f84c8ee5a576ba54b08: Downgrade DER comparison and offset lookup failures to be handled as match failures. Upstream bug: http://bugs.gw.com/view.php?id=540 Modified: vendor/file/dist/src/softmagic.c Modified: vendor/file/dist/src/softmagic.c ============================================================================== --- vendor/file/dist/src/softmagic.c Sun May 8 18:09:30 2016 (r299236) +++ vendor/file/dist/src/softmagic.c Sun May 8 18:15:58 2016 (r299237) @@ -32,7 +32,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: softmagic.c,v 1.229 2016/03/21 23:04:40 christos Exp $") +FILE_RCSID("@(#)$File: softmagic.c,v 1.230 2016/04/18 15:10:34 christos Exp $") #endif /* lint */ #include "magic.h" @@ -186,11 +186,11 @@ match(struct magic_set *ms, struct magic ((text && (m->str_flags & FLT) == STRING_BINTEST) || (!text && (m->str_flags & FLT) == STRING_TEXTTEST))) || (m->flag & mode) != mode) { +flush: /* Skip sub-tests */ - while (magindex + 1 < nmagic && - magic[magindex + 1].cont_level != 0 && - ++magindex) - continue; + while (magindex < nmagic - 1 && + magic[magindex + 1].cont_level != 0) + magindex++; continue; /* Skip to next top-level test*/ } @@ -227,10 +227,7 @@ match(struct magic_set *ms, struct magic * main entry didn't match, * flush its continuations */ - while (magindex < nmagic - 1 && - magic[magindex + 1].cont_level != 0) - magindex++; - continue; + goto flush; } if ((e = handle_annotation(ms, m)) != 0) { @@ -255,8 +252,13 @@ match(struct magic_set *ms, struct magic if (print && mprint(ms, m) == -1) return -1; - if (moffset(ms, m, nbytes, &ms->c.li[cont_level].off) == -1) - return -1; + switch (moffset(ms, m, nbytes, &ms->c.li[cont_level].off)) { + case -1: + case 0: + goto flush; + default: + break; + } /* and any continuations that match */ if (file_check_mem(ms, ++cont_level) == -1) @@ -362,9 +364,15 @@ match(struct magic_set *ms, struct magic if (print && mprint(ms, m) == -1) return -1; - if (moffset(ms, m, nbytes, - &ms->c.li[cont_level].off) == -1) - return -1; + switch (moffset(ms, m, nbytes, + &ms->c.li[cont_level].off)) { + case -1: + case 0: + flush = 1; + break; + default: + break; + } if (*m->desc) *need_separator = 1; @@ -813,9 +821,13 @@ moffset(struct magic_set *ms, struct mag case FILE_DER: { o = der_offs(ms, m, nbytes); - if (o == -1) { - file_error(ms, 0, "EOF computing DER offset"); - return -1; + if (o == -1 || (size_t)o > nbytes) { + if ((ms->flags & MAGIC_DEBUG) != 0) { + (void)fprintf(stderr, + "Bad DER offset %d nbytes=%zu", + o, nbytes); + } + return 0; } break; } @@ -825,12 +837,15 @@ moffset(struct magic_set *ms, struct mag break; } - if ((size_t)o >= nbytes) { - file_error(ms, 0, "Offset out of range"); + if ((size_t)o > nbytes) { +#if 0 + file_error(ms, 0, "Offset out of range %zu > %zu", + (size_t)o, nbytes); +#endif return -1; } *op = o; - return 0; + return 1; } private uint32_t @@ -2107,8 +2122,13 @@ magiccheck(struct magic_set *ms, struct return 1; case FILE_DER: matched = der_cmp(ms, m); - if (matched == -1) - file_error(ms, 0, "EOF comparing DER entries"); + if (matched == -1) { + if ((ms->flags & MAGIC_DEBUG) != 0) { + (void) fprintf(stderr, + "EOF comparing DER entries"); + } + return 0; + } return matched; default: file_magerror(ms, "invalid type %d in magiccheck()", m->type); From owner-svn-src-vendor@freebsd.org Tue May 10 21:12:34 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0AC97B36F5A; Tue, 10 May 2016 21:12:34 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CB88C1DE1; Tue, 10 May 2016 21:12:33 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4ALCWuw075359; Tue, 10 May 2016 21:12:32 GMT (envelope-from mm@FreeBSD.org) Received: (from mm@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4ALCWM6075358; Tue, 10 May 2016 21:12:32 GMT (envelope-from mm@FreeBSD.org) Message-Id: <201605102112.u4ALCWM6075358@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mm set sender to mm@FreeBSD.org using -f From: Martin Matuska Date: Tue, 10 May 2016 21:12:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299390 - in vendor/libarchive/dist: . build contrib cpio cpio/test doc examples libarchive libarchive/test tar tar/test X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 May 2016 21:12:34 -0000 Author: mm Date: Tue May 10 21:12:32 2016 New Revision: 299390 URL: https://svnweb.freebsd.org/changeset/base/299390 Log: Trim libarchive/dist using FREEBSD-Xlist Deleted: vendor/libarchive/dist/CMakeLists.txt vendor/libarchive/dist/CTestConfig.cmake vendor/libarchive/dist/INSTALL vendor/libarchive/dist/Makefile.am vendor/libarchive/dist/build/ vendor/libarchive/dist/configure.ac vendor/libarchive/dist/contrib/ vendor/libarchive/dist/cpio/CMakeLists.txt vendor/libarchive/dist/cpio/config_freebsd.h vendor/libarchive/dist/cpio/cpio_windows.c vendor/libarchive/dist/cpio/cpio_windows.h vendor/libarchive/dist/cpio/test/CMakeLists.txt vendor/libarchive/dist/doc/ vendor/libarchive/dist/examples/ vendor/libarchive/dist/libarchive/CMakeLists.txt vendor/libarchive/dist/libarchive/archive_entry_copy_bhfi.c vendor/libarchive/dist/libarchive/archive_read_disk_windows.c vendor/libarchive/dist/libarchive/archive_windows.c vendor/libarchive/dist/libarchive/archive_windows.h vendor/libarchive/dist/libarchive/archive_write_disk_windows.c vendor/libarchive/dist/libarchive/config_freebsd.h vendor/libarchive/dist/libarchive/filter_fork_windows.c vendor/libarchive/dist/libarchive/mtree.5 vendor/libarchive/dist/libarchive/test/.cvsignore vendor/libarchive/dist/libarchive/test/CMakeLists.txt vendor/libarchive/dist/tar/CMakeLists.txt vendor/libarchive/dist/tar/bsdtar_windows.c vendor/libarchive/dist/tar/bsdtar_windows.h vendor/libarchive/dist/tar/config_freebsd.h vendor/libarchive/dist/tar/test/CMakeLists.txt vendor/libarchive/dist/tar/test/test_windows.c From owner-svn-src-vendor@freebsd.org Wed May 11 10:10:12 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B51D7B34CF3; Wed, 11 May 2016 10:10:12 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6FE0C199B; Wed, 11 May 2016 10:10:12 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4BAABBL008788; Wed, 11 May 2016 10:10:11 GMT (envelope-from mm@FreeBSD.org) Received: (from mm@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4BAABAg008784; Wed, 11 May 2016 10:10:11 GMT (envelope-from mm@FreeBSD.org) Message-Id: <201605111010.u4BAABAg008784@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mm set sender to mm@FreeBSD.org using -f From: Martin Matuska Date: Wed, 11 May 2016 10:10:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299422 - in vendor/libarchive/dist: . build build/autoconf build/cmake build/pkgconfig build/utils contrib contrib/psota-benchmark contrib/shar cpio cpio/test doc doc/html doc/man doc/... X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 10:10:12 -0000 Author: mm Date: Wed May 11 10:10:11 2016 New Revision: 299422 URL: https://svnweb.freebsd.org/changeset/base/299422 Log: Keep full libarchive distribution in vendor branch (prep for 3.2.0 update) Added: vendor/libarchive/dist/.gitattributes vendor/libarchive/dist/.gitignore vendor/libarchive/dist/CMakeLists.txt (contents, props changed) vendor/libarchive/dist/CTestConfig.cmake vendor/libarchive/dist/INSTALL vendor/libarchive/dist/Makefile.am (contents, props changed) vendor/libarchive/dist/build/ vendor/libarchive/dist/build/README.txt (contents, props changed) vendor/libarchive/dist/build/autoconf/ vendor/libarchive/dist/build/autoconf/check_stdcall_func.m4 vendor/libarchive/dist/build/autoconf/config.rpath (contents, props changed) vendor/libarchive/dist/build/autoconf/iconv.m4 vendor/libarchive/dist/build/autoconf/la_uid_t.m4 vendor/libarchive/dist/build/autoconf/lib-ld.m4 vendor/libarchive/dist/build/autoconf/lib-link.m4 vendor/libarchive/dist/build/autoconf/lib-prefix.m4 vendor/libarchive/dist/build/autogen.sh (contents, props changed) vendor/libarchive/dist/build/bump-version.sh (contents, props changed) vendor/libarchive/dist/build/clean.sh (contents, props changed) vendor/libarchive/dist/build/cmake/ vendor/libarchive/dist/build/cmake/CheckFileOffsetBits.c (contents, props changed) vendor/libarchive/dist/build/cmake/CheckFileOffsetBits.cmake vendor/libarchive/dist/build/cmake/CheckFuncs.cmake vendor/libarchive/dist/build/cmake/CheckFuncs_stub.c.in (contents, props changed) vendor/libarchive/dist/build/cmake/CheckHeaderDirent.cmake vendor/libarchive/dist/build/cmake/CheckStructMember.cmake vendor/libarchive/dist/build/cmake/CheckTypeExists.cmake vendor/libarchive/dist/build/cmake/FindLZMA.cmake vendor/libarchive/dist/build/cmake/FindLibGCC.cmake vendor/libarchive/dist/build/cmake/FindNettle.cmake vendor/libarchive/dist/build/cmake/FindPCREPOSIX.cmake vendor/libarchive/dist/build/cmake/LibarchiveCheckCSourceCompiles.cmake vendor/libarchive/dist/build/cmake/LibarchiveCheckCSourceRuns.cmake vendor/libarchive/dist/build/cmake/config.h.in (contents, props changed) vendor/libarchive/dist/build/makerelease.sh (contents, props changed) vendor/libarchive/dist/build/pkgconfig/ vendor/libarchive/dist/build/pkgconfig/libarchive.pc.in (contents, props changed) vendor/libarchive/dist/build/utils/ vendor/libarchive/dist/build/utils/gen_archive_string_composition_h.sh (contents, props changed) vendor/libarchive/dist/build/version vendor/libarchive/dist/configure.ac vendor/libarchive/dist/contrib/ vendor/libarchive/dist/contrib/README vendor/libarchive/dist/contrib/libarchive.1aix53.spec vendor/libarchive/dist/contrib/libarchive.spec vendor/libarchive/dist/contrib/libarchive_autodetect-st_lib_archive.m4 vendor/libarchive/dist/contrib/psota-benchmark/ vendor/libarchive/dist/contrib/psota-benchmark/results.txt (contents, props changed) vendor/libarchive/dist/contrib/psota-benchmark/tcp.sh (contents, props changed) vendor/libarchive/dist/contrib/shar/ vendor/libarchive/dist/contrib/shar/Makefile (contents, props changed) vendor/libarchive/dist/contrib/shar/shar.1 (contents, props changed) vendor/libarchive/dist/contrib/shar/shar.c (contents, props changed) vendor/libarchive/dist/contrib/shar/tree.c (contents, props changed) vendor/libarchive/dist/contrib/shar/tree.h (contents, props changed) vendor/libarchive/dist/contrib/shar/tree_config.h (contents, props changed) vendor/libarchive/dist/contrib/untar.c (contents, props changed) vendor/libarchive/dist/cpio/CMakeLists.txt (contents, props changed) vendor/libarchive/dist/cpio/config_freebsd.h (contents, props changed) vendor/libarchive/dist/cpio/cpio_windows.c (contents, props changed) vendor/libarchive/dist/cpio/cpio_windows.h (contents, props changed) vendor/libarchive/dist/cpio/test/CMakeLists.txt (contents, props changed) vendor/libarchive/dist/doc/ vendor/libarchive/dist/doc/html/ vendor/libarchive/dist/doc/html/.ignore_me vendor/libarchive/dist/doc/man/ vendor/libarchive/dist/doc/man/.ignore_me vendor/libarchive/dist/doc/mdoc2man.awk (contents, props changed) vendor/libarchive/dist/doc/mdoc2wiki.awk (contents, props changed) vendor/libarchive/dist/doc/pdf/ vendor/libarchive/dist/doc/pdf/.ignore_me vendor/libarchive/dist/doc/text/ vendor/libarchive/dist/doc/text/.ignore_me vendor/libarchive/dist/doc/update.sh (contents, props changed) vendor/libarchive/dist/doc/wiki/ vendor/libarchive/dist/doc/wiki/.ignore_me vendor/libarchive/dist/examples/ vendor/libarchive/dist/examples/minitar/ vendor/libarchive/dist/examples/minitar/Makefile (contents, props changed) vendor/libarchive/dist/examples/minitar/README vendor/libarchive/dist/examples/minitar/minitar.c (contents, props changed) vendor/libarchive/dist/examples/tarfilter.c (contents, props changed) vendor/libarchive/dist/examples/untar.c (contents, props changed) vendor/libarchive/dist/libarchive/CMakeLists.txt (contents, props changed) vendor/libarchive/dist/libarchive/archive_entry_copy_bhfi.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_read_disk_windows.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_windows.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_windows.h (contents, props changed) vendor/libarchive/dist/libarchive/archive_write_disk_windows.c (contents, props changed) vendor/libarchive/dist/libarchive/config_freebsd.h (contents, props changed) vendor/libarchive/dist/libarchive/filter_fork_windows.c (contents, props changed) vendor/libarchive/dist/libarchive/mtree.5 (contents, props changed) vendor/libarchive/dist/libarchive/test/.cvsignore vendor/libarchive/dist/libarchive/test/CMakeLists.txt (contents, props changed) vendor/libarchive/dist/tar/CMakeLists.txt (contents, props changed) vendor/libarchive/dist/tar/bsdtar_windows.c (contents, props changed) vendor/libarchive/dist/tar/bsdtar_windows.h (contents, props changed) vendor/libarchive/dist/tar/config_freebsd.h (contents, props changed) vendor/libarchive/dist/tar/test/CMakeLists.txt (contents, props changed) vendor/libarchive/dist/tar/test/test_windows.c (contents, props changed) Added: vendor/libarchive/dist/.gitattributes ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libarchive/dist/.gitattributes Wed May 11 10:10:11 2016 (r299422) @@ -0,0 +1,4 @@ +.git* export-ignore + +*.sh crlf=input +config.rpath crlf=input Added: vendor/libarchive/dist/.gitignore ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libarchive/dist/.gitignore Wed May 11 10:10:11 2016 (r299422) @@ -0,0 +1,84 @@ +*.o +*.exe +*.lo +*.cmake +.libs/ +Makefile +Makefile.in +aclocal.m4 +autom4te.cache/ +bsdcpio +bsdcpio_test +bsdtar +bsdtar_test +build/autoconf/compile +build/autoconf/config.guess +build/autoconf/config.sub +build/autoconf/depcomp +build/autoconf/install-sh +build/autoconf/libtool.m4 +build/autoconf/ltmain.sh +build/autoconf/ltoptions.m4 +build/autoconf/ltsugar.m4 +build/autoconf/ltversion.m4 +build/autoconf/lt~obsolete.m4 +build/autoconf/missing +build/pkgconfig/libarchive.pc +config.h +config.h.in +config.h.in~ +config.log +config.status +configure +cpio/.deps/ +cpio/.dirstamp +cpio/test/.deps/ +cpio/test/.dirstamp +cpio/test/list.h +libarchive.la +libarchive/.deps/ +libarchive/.dirstamp +libarchive/test/.deps/ +libarchive/test/.dirstamp +libarchive/test/list.h +libarchive_fe.la +libarchive_fe/.deps/ +libarchive_fe/.dirstamp +libarchive_test +libtool +stamp-h1 +tar/.deps/ +tar/.dirstamp +tar/test/.deps/ +tar/test/.dirstamp +tar/test/list.h +CMakeCache.txt +CMakeFiles/ +DartConfiguration.tcl +cmake.tmp/ +cpio/CMakeFiles/ +cpio/test/CMakeFiles/ +libarchive/CMakeFiles/ +libarchive/test/CMakeFiles/ +tar/CMakeFiles/ +tar/test/CMakeFiles/ +test_utils/.deps/ +test_utils/.dirstamp + +doc/html/*.html +doc/man/*.1 +doc/man/*.3 +doc/man/*.5 +doc/pdf/*.pdf +doc/text/*.txt +doc/wiki/*.wiki + +libarchive-*.tar.gz +libarchive-*.zip + +Testing/ +libarchive/libarchive.a +libarchive/libarchive.so +libarchive/libarchive.so.* + +.DS_Store Added: vendor/libarchive/dist/CMakeLists.txt ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libarchive/dist/CMakeLists.txt Wed May 11 10:10:11 2016 (r299422) @@ -0,0 +1,1525 @@ +# +CMAKE_MINIMUM_REQUIRED(VERSION 2.8.6 FATAL_ERROR) +# +PROJECT(libarchive C) +# +SET(CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/build/cmake") +if(NOT CMAKE_RUNTIME_OUTPUT_DIRECTORY) + set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${libarchive_BINARY_DIR}/bin) +endif() +# +# Set the Build type for make based generators. +# You can choose following types: +# Debug : Debug build +# Release : Release build +# RelWithDebInfo : Release build with Debug Info +# MinSizeRel : Release Min Size build +IF(NOT CMAKE_BUILD_TYPE) + SET(CMAKE_BUILD_TYPE "Release" CACHE STRING "Build Type" FORCE) +ENDIF(NOT CMAKE_BUILD_TYPE) +# Set a value type to properly display CMAKE_BUILD_TYPE on GUI if the +# value type is "UNINITIALIZED". +GET_PROPERTY(cached_type CACHE CMAKE_BUILD_TYPE PROPERTY TYPE) +IF("${cached_type}" STREQUAL "UNINITIALIZED") + SET(CMAKE_BUILD_TYPE "${CMAKE_BUILD_TYPE}" CACHE STRING "Build Type" FORCE) +ENDIF("${cached_type}" STREQUAL "UNINITIALIZED") +# Check the Build Type. +IF(NOT "${CMAKE_BUILD_TYPE}" + MATCHES "^(Debug|Release|RelWithDebInfo|MinSizeRel)\$") + MESSAGE(FATAL_ERROR + "Unknown keyword for CMAKE_BUILD_TYPE: ${CMAKE_BUILD_TYPE}\n" + "Acceptable keywords: Debug,Release,RelWithDebInfo,MinSizeRel") +ENDIF(NOT "${CMAKE_BUILD_TYPE}" + MATCHES "^(Debug|Release|RelWithDebInfo|MinSizeRel)\$") + +# On MacOS, prefer MacPorts libraries to system libraries. +# I haven't come up with a compelling argument for this to be conditional. +list(APPEND CMAKE_PREFIX_PATH /opt/local) + +# +# Version - read from 'version' file. +# +FILE(STRINGS ${CMAKE_CURRENT_SOURCE_DIR}/build/version _version) +STRING(REGEX REPLACE + "^([0-9])[0-9][0-9][0-9][0-9][0-9][0-9][a-z]?$" "\\1" _major ${_version}) +STRING(REGEX REPLACE + "^[0-9]([0-9][0-9][0-9])[0-9][0-9][0-9][a-z]?$" "\\1" _minor ${_version}) +STRING(REGEX REPLACE + "^[0-9][0-9][0-9][0-9]([0-9][0-9][0-9])[a-z]?$" "\\1" _revision ${_version}) +STRING(REGEX REPLACE + "^[0-9][0-9][0-9][0-9][0-9][0-9][0-9]([a-z]?)$" "\\1" _quality ${_version}) +SET(_version_number ${_major}${_minor}${_revision}) +STRING(REGEX REPLACE "[0]*([^0]*[0-9])$" "\\1" _trimmed_minor ${_minor}) +STRING(REGEX REPLACE "[0]*([^0]*[0-9])$" "\\1" _trimmed_revision ${_revision}) +# +SET(VERSION "${_major}.${_trimmed_minor}.${_trimmed_revision}${_quality}") +SET(BSDCPIO_VERSION_STRING "${VERSION}") +SET(BSDTAR_VERSION_STRING "${VERSION}") +SET(LIBARCHIVE_VERSION_NUMBER "${_version_number}") +SET(LIBARCHIVE_VERSION_STRING "${VERSION}") + +# INTERFACE_VERSION increments with every release +# libarchive 2.7 == interface version 9 = 2 + 7 +# libarchive 2.8 == interface version 10 = 2 + 8 +# libarchive 2.9 == interface version 11 = 2 + 9 +# libarchive 3.0 == interface version 12 +# libarchive 3.1 == interface version 13 +math(EXPR INTERFACE_VERSION "13 + ${_minor}") + +# Set SOVERSION == Interface version +# ?? Should there be more here ?? +SET(SOVERSION "${INTERFACE_VERSION}") + +# Enalbe CMAKE_PUSH_CHECK_STATE() and CMAKE_POP_CHECK_STATE() macros +# saving and restoring the state of the variables. +INCLUDE(CMakePushCheckState) + +# Initialize the state of the variables. This initialization is not +# necessary but this shows you what value the variables initially have. +SET(CMAKE_REQUIRED_DEFINITIONS) +SET(CMAKE_REQUIRED_INCLUDES) +SET(CMAKE_REQUIRED_LIBRARIES) +SET(CMAKE_REQUIRED_FLAGS) + +# Especially for early development, we want to be a little +# aggressive about diagnosing build problems; this can get +# relaxed somewhat in final shipping versions. +IF ("CMAKE_C_COMPILER_ID" MATCHES "^GNU$") + SET(CMAKE_REQUIRED_FLAGS "-Wall -Wformat -Wformat-security") + ################################################################# + # Set compile flags for all build types. + SET(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wall -Wformat -Wformat-security") + ################################################################# + # Set compile flags for debug build. + # This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug" + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Werror -Wextra -Wunused") + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Wshadow") + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Wmissing-prototypes") + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Wcast-qual") +ENDIF ("CMAKE_C_COMPILER_ID" MATCHES "^GNU$") +IF (MSVC) + ################################################################# + # Set compile flags for debug build. + # This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug" + # Enable level 4 C4061: The enumerate has no associated handler in a switch + # statement. + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061") + # Enable level 4 C4254: A larger bit field was assigned to a smaller bit + # field. + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254") + # Enable level 4 C4295: An array was initialized but the last character in + # the array is not a null; accessing the array may + # produce unexpected results. + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4295") + # Enable level 4 C4296: An unsigned variable was used in a comparison + # operation with zero. + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4296") + # Enable level 4 C4389: An operation involved signed and unsigned variables. + # This could result in a loss of data. + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4389") + # Enable level 4 C4505: The given function is local and not referenced in + # the body of the module; therefore, the function is + # dead code. + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4505") + # Enable level 4 C4514: The optimizer removed an inline function that is not + # called. + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4514") + # Enable level 4 C4702: Unreachable code. + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4702") + # Enable level 4 C4706: The test value in a conditional expression was the + # result of an assignment. + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4706") + # /WX option is the same as gcc's -Werror option. + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /WX") + # /Oi option enables built-in functions. + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /Oi") + ################################################################# + # Set compile flags for release build. + SET(CMAKE_C_FLAGS_RELEASE "${CMAKE_C_FLAGS_RELEASE} /Oi") +ENDIF (MSVC) + +# Enable CTest/CDash support +include(CTest) + +OPTION(ENABLE_NETTLE "Enable use of Nettle" ON) +OPTION(ENABLE_OPENSSL "Enable use of OpenSSL" ON) +OPTION(ENABLE_TAR "Enable tar building" ON) +OPTION(ENABLE_TAR_SHARED "Enable dynamic build of tar" FALSE) +OPTION(ENABLE_CPIO "Enable cpio building" ON) +OPTION(ENABLE_CPIO_SHARED "Enable dynamic build of cpio" FALSE) +OPTION(ENABLE_XATTR "Enable extended attribute support" ON) +OPTION(ENABLE_ACL "Enable ACL support" ON) +OPTION(ENABLE_ICONV "Enable iconv support" ON) +OPTION(ENABLE_TEST "Enable unit and regression tests" ON) +SET(POSIX_REGEX_LIB "AUTO" CACHE STRING "Choose what library should provide POSIX regular expression support") +SET(ENABLE_SAFESEH "AUTO" CACHE STRING "Enable use of /SAFESEH linker flag (MSVC only)") +SET(WINDOWS_VERSION "" CACHE STRING "Set Windows version to use (Windows only)") + +IF(ENABLE_TEST) + ENABLE_TESTING() +ENDIF(ENABLE_TEST) + +IF(WIN32) + IF(WINDOWS_VERSION STREQUAL "WIN8") + SET(WINVER 0x0602) + ELSEIF(WINDOWS_VERSION STREQUAL "WIN7") + SET(WINVER 0x0601) + ELSEIF(WINDOWS_VERSION STREQUAL "WS08") + SET(WINVER 0x0600) + ELSEIF(WINDOWS_VERSION STREQUAL "VISTA") + SET(WINVER 0x0600) + ELSEIF(WINDOWS_VERSION STREQUAL "WS03") + SET(WINVER 0x0502) + ELSEIF(WINDOWS_VERSION STREQUAL "WINXP") + SET(WINVER 0x0501) + ELSE(WINDOWS_VERSION STREQUAL "WIN8") + # The default is to use Windows 2000 API. + SET(WINVER 0x0500) + ENDIF(WINDOWS_VERSION STREQUAL "WIN8") + SET(_WIN32_WINNT ${WINVER}) +ENDIF(WIN32) + +IF(MSVC) + IF(ENABLE_SAFESEH STREQUAL "YES") + SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /SAFESEH") + SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} /SAFESEH") + SET(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} /SAFESEH") + SET(CMAKE_REQUIRED_LINKER_FLAGS "/SAFESEH") + ELSEIF(ENABLE_SAFESEH STREQUAL "NO") + SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /SAFESEH:NO") + SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} /SAFESEH:NO") + SET(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} /SAFESEH:NO") + SET(CMAKE_REQUIRED_LINKER_FLAGS "/SAFESEH:NO") + ENDIF(ENABLE_SAFESEH STREQUAL "YES") +ENDIF(MSVC) + +IF("${CMAKE_C_PLATFORM_ID}" MATCHES "^(HP-UX)$") + ADD_DEFINITIONS(-D_XOPEN_SOURCE=500) # Ask wchar.h for mbstate_t +ENDIF() + +# +INCLUDE(LibarchiveCheckCSourceCompiles) +INCLUDE(LibarchiveCheckCSourceRuns) +INCLUDE(CheckFileOffsetBits) +INCLUDE(CheckFuncs) +INCLUDE(CheckHeaderDirent) +INCLUDE(CheckIncludeFile) +INCLUDE(CheckIncludeFiles) +INCLUDE(CheckLibraryExists) +INCLUDE(CheckStructMember) +INCLUDE(CheckSymbolExists) +INCLUDE(CheckTypeExists) +INCLUDE(CheckTypeSize) + +# +# Generate list.h +# +MACRO (GENERATE_LIST_H _listfile _cmlist __list_sources) + SET(_argv ${ARGV}) + # Remove _listfile and _cmlist from _argv + LIST(REMOVE_AT _argv 0 1) + IF (NOT EXISTS "${_listfile}" OR + ${_cmlist} IS_NEWER_THAN "${_listfile}") + + MESSAGE(STATUS "Generating ${_listfile}") + FILE(WRITE ${_listfile} "") + FOREACH (testfile ${_argv}) + IF (testfile MATCHES "^test_[^/]+[.]c$") + FILE(STRINGS ${testfile} testvar REGEX "^DEFINE_TEST") + FOREACH (deftest ${testvar}) + FILE(APPEND ${_listfile} "${deftest}\n") + ENDFOREACH (deftest) + ENDIF (testfile MATCHES "^test_[^/]+[.]c$") + ENDFOREACH (testfile) + + ENDIF (NOT EXISTS "${_listfile}" OR + ${_cmlist} IS_NEWER_THAN "${_listfile}") +ENDMACRO (GENERATE_LIST_H) +# +# Generate installation rules for man pages. +# +MACRO (INSTALL_MAN __mans) + FOREACH (_man ${ARGV}) + STRING(REGEX REPLACE "^.+[.]([1-9])" "\\1" _mansect ${_man}) + INSTALL(FILES ${_man} DESTINATION "share/man/man${_mansect}") + ENDFOREACH (_man) +ENDMACRO (INSTALL_MAN __mans) +# +# Find out what macro is needed to use libraries on Windows. +# +MACRO (TRY_MACRO_FOR_LIBRARY INCLUDES LIBRARIES + TRY_TYPE SAMPLE_SOURCE MACRO_LIST) + IF(WIN32 AND NOT CYGWIN) + CMAKE_PUSH_CHECK_STATE() # Save the state of the variables + SET(CMAKE_REQUIRED_INCLUDES ${INCLUDES}) + SET(CMAKE_REQUIRED_LIBRARIES ${LIBRARIES}) + FOREACH(VAR ${MACRO_LIST}) + # Clear ${VAR} from CACHE If the libraries which ${VAR} was + # checked with are changed. + SET(VAR_WITH_LIB "${VAR}_WITH_LIB") + GET_PROPERTY(PREV_VAR_WITH_LIB VARIABLE PROPERTY ${VAR_WITH_LIB}) + IF(NOT "${PREV_VAR_WITH_LIB}" STREQUAL "${LIBRARIES}") + UNSET(${VAR} CACHE) + ENDIF(NOT "${PREV_VAR_WITH_LIB}" STREQUAL "${LIBRARIES}") + # Check if the library can be used with the macro. + IF("${TRY_TYPE}" MATCHES "COMPILES") + LIBARCHIVE_CHECK_C_SOURCE_COMPILES("${SAMPLE_SOURCE}" ${VAR}) + ELSEIF("${TRY_TYPE}" MATCHES "RUNS") + LIBARCHIVE_CHECK_C_SOURCE_RUNS("${SAMPLE_SOURCE}" ${VAR}) + ELSE("${TRY_TYPE}" MATCHES "COMPILES") + MESSAGE(FATAL_ERROR "UNKNOWN KEYWORD \"${TRY_TYPE}\" FOR TRY_TYPE") + ENDIF("${TRY_TYPE}" MATCHES "COMPILES") + # Save the libraries which ${VAR} is checked with. + SET(${VAR_WITH_LIB} "${LIBRARIES}" CACHE INTERNAL + "Macro ${VAR} is checked with") + ENDFOREACH(VAR) + CMAKE_POP_CHECK_STATE() # Restore the state of the variables + ENDIF(WIN32 AND NOT CYGWIN) +ENDMACRO (TRY_MACRO_FOR_LIBRARY) +# +# Check compress/decompress libraries +# +IF(WIN32 AND NOT CMAKE_CL_64 AND NOT CYGWIN) + # GnuWin32 is only for Win32, not Win64. + SET(__GNUWIN32PATH "C:/Program Files/GnuWin32") +ENDIF(WIN32 AND NOT CMAKE_CL_64 AND NOT CYGWIN) +IF(DEFINED __GNUWIN32PATH AND EXISTS "${__GNUWIN32PATH}") + # You have to add a path availabel DLL file into PATH environment variable. + # Maybe DLL path is "C:/Program Files/GnuWin32/bin". + # The zlib and the bzip2 Setup program have installed programs and DLLs into + # "C:/Program Files/GnuWin32" by default. + # This is convenience setting for Windows. + SET(CMAKE_PREFIX_PATH ${__GNUWIN32PATH} $(CMAKE_PREFIX_PATH)) + # + # If you didn't use Setup program or installed into nonstandard path, + # cmake cannot find out your zlib or bzip2 libraries and include files, + # you should execute cmake with -DCMAKE_PREFIX_PATH option. + # e.g. + # cmake -DCMAKE_PREFIX_PATH= + # + # If compiling error occured in zconf.h, You may need patch to zconf.h. + #--- zconf.h.orig 2005-07-21 00:40:26.000000000 + #+++ zconf.h 2009-01-19 11:39:10.093750000 + #@@ -286,7 +286,7 @@ + # + # #if 1 /* HAVE_UNISTD_H -- this line is updated by ./configure */ + # # include /* for off_t */ + #-# include /* for SEEK_* and off_t */ + #+# include /* for SEEK_* and off_t */ + # # ifdef VMS + # # include /* for off_t */ + # # endif +ENDIF(DEFINED __GNUWIN32PATH AND EXISTS "${__GNUWIN32PATH}") + +SET(ADDITIONAL_LIBS "") +# +# Find ZLIB +# +FIND_PACKAGE(ZLIB) +IF(ZLIB_FOUND) + SET(HAVE_LIBZ 1) + SET(HAVE_ZLIB_H 1) + INCLUDE_DIRECTORIES(${ZLIB_INCLUDE_DIR}) + LIST(APPEND ADDITIONAL_LIBS ${ZLIB_LIBRARIES}) + IF(WIN32 AND NOT CYGWIN) + # + # Test if ZLIB_WINAPI macro is needed to use. + # + TRY_MACRO_FOR_LIBRARY( + "${ZLIB_INCLUDE_DIR}" "${ZLIB_LIBRARIES}" + RUNS + "#include \nint main() {uLong f = zlibCompileFlags(); return (f&(1U<<10))?0:-1; }" + ZLIB_WINAPI) + IF(ZLIB_WINAPI) + ADD_DEFINITIONS(-DZLIB_WINAPI) + ELSE(ZLIB_WINAPI) + # Test if a macro is needed for the library. + TRY_MACRO_FOR_LIBRARY( + "${ZLIB_INCLUDE_DIR}" "${ZLIB_LIBRARIES}" + COMPILES + "#include \nint main() {return zlibVersion()?1:0; }" + "ZLIB_DLL;WITHOUT_ZLIB_DLL") + IF(ZLIB_DLL) + ADD_DEFINITIONS(-DZLIB_DLL) + ENDIF(ZLIB_DLL) + ENDIF(ZLIB_WINAPI) + ENDIF(WIN32 AND NOT CYGWIN) +ENDIF(ZLIB_FOUND) +MARK_AS_ADVANCED(CLEAR ZLIB_INCLUDE_DIR) +MARK_AS_ADVANCED(CLEAR ZLIB_LIBRARY) +# +# Find BZip2 +# +FIND_PACKAGE(BZip2) +IF(BZIP2_FOUND) + SET(HAVE_LIBBZ2 1) + SET(HAVE_BZLIB_H 1) + INCLUDE_DIRECTORIES(${BZIP2_INCLUDE_DIR}) + LIST(APPEND ADDITIONAL_LIBS ${BZIP2_LIBRARIES}) + # Test if a macro is needed for the library. + TRY_MACRO_FOR_LIBRARY( + "${BZIP2_INCLUDE_DIR}" "${BZIP2_LIBRARIES}" + COMPILES + "#include \nint main() {return BZ2_bzlibVersion()?1:0; }" + "USE_BZIP2_DLL;USE_BZIP2_STATIC") + IF(USE_BZIP2_DLL) + ADD_DEFINITIONS(-DUSE_BZIP2_DLL) + ELSEIF(USE_BZIP2_STATIC) + ADD_DEFINITIONS(-DUSE_BZIP2_STATIC) + ENDIF(USE_BZIP2_DLL) +ENDIF(BZIP2_FOUND) +MARK_AS_ADVANCED(CLEAR BZIP2_INCLUDE_DIR) +MARK_AS_ADVANCED(CLEAR BZIP2_LIBRARIES) +# +# Find LZMA +# +FIND_PACKAGE(LZMA) +IF(LZMA_FOUND) + SET(HAVE_LIBLZMA 1) + SET(HAVE_LZMA_H 1) + INCLUDE_DIRECTORIES(${LZMA_INCLUDE_DIR}) + LIST(APPEND ADDITIONAL_LIBS ${LZMA_LIBRARIES}) + # Test if a macro is needed for the library. + TRY_MACRO_FOR_LIBRARY( + "${LZMA_INCLUDE_DIR}" "${LZMA_LIBRARIES}" + COMPILES + "#include \nint main() {return (int)lzma_version_number(); }" + "WITHOUT_LZMA_API_STATIC;LZMA_API_STATIC") + IF(NOT WITHOUT_LZMA_API_STATIC AND LZMA_API_STATIC) + ADD_DEFINITIONS(-DLZMA_API_STATIC) + ENDIF(NOT WITHOUT_LZMA_API_STATIC AND LZMA_API_STATIC) +ELSEIF(LZMADEC_FOUND) + SET(HAVE_LIBLZMADEC 1) + SET(HAVE_LZMADEC_H 1) + INCLUDE_DIRECTORIES(${LZMADEC_INCLUDE_DIR}) + LIST(APPEND ADDITIONAL_LIBS ${LZMADEC_LIBRARIES}) +ENDIF(LZMA_FOUND) +# +# Find LZO2 +# +IF (LZO2_INCLUDE_DIR) + # Already in cache, be silent + SET(LZO2_FIND_QUIETLY TRUE) +ENDIF (LZO2_INCLUDE_DIR) + +FIND_PATH(LZO2_INCLUDE_DIR lzo/lzoconf.h) +FIND_LIBRARY(LZO2_LIBRARY NAMES lzo2 liblzo2) +INCLUDE(FindPackageHandleStandardArgs) +FIND_PACKAGE_HANDLE_STANDARD_ARGS(LZO2 DEFAULT_MSG LZO2_LIBRARY LZO2_INCLUDE_DIR) +IF(LZO2_FOUND) + SET(HAVE_LIBLZO2 1) + SET(HAVE_LZO_LZOCONF_H 1) + SET(HAVE_LZO_LZO1X_H 1) + INCLUDE_DIRECTORIES(${LZO2_INCLUDE_DIR}) + LIST(APPEND ADDITIONAL_LIBS ${LZO2_LIBRARY}) + # + # TODO: test for static library. + # +ENDIF(LZO2_FOUND) +MARK_AS_ADVANCED(CLEAR LZO2_INCLUDE_DIR) +MARK_AS_ADVANCED(CLEAR LZO2_LIBRARY) + +# +# Check headers +# +CHECK_HEADER_DIRENT() + +SET(INCLUDES "") +MACRO (LA_CHECK_INCLUDE_FILE header var) + CHECK_INCLUDE_FILES("${INCLUDES};${header}" ${var}) + IF (${var}) + SET(INCLUDES ${INCLUDES} ${header}) + ENDIF (${var}) +ENDMACRO (LA_CHECK_INCLUDE_FILE) + +# Some FreeBSD headers assume sys/types.h was already included. +LA_CHECK_INCLUDE_FILE("sys/types.h" HAVE_SYS_TYPES_H) + +# Alphabetize the rest unless there's a compelling reason +LA_CHECK_INCLUDE_FILE("acl/libacl.h" HAVE_ACL_LIBACL_H) +LA_CHECK_INCLUDE_FILE("ctype.h" HAVE_CTYPE_H) +LA_CHECK_INCLUDE_FILE("copyfile.h" HAVE_COPYFILE_H) +LA_CHECK_INCLUDE_FILE("direct.h" HAVE_DIRECT_H) +LA_CHECK_INCLUDE_FILE("dlfcn.h" HAVE_DLFCN_H) +LA_CHECK_INCLUDE_FILE("errno.h" HAVE_ERRNO_H) +LA_CHECK_INCLUDE_FILE("ext2fs/ext2_fs.h" HAVE_EXT2FS_EXT2_FS_H) + +LIBARCHIVE_CHECK_C_SOURCE_COMPILES("#include +#include +int main(void) { return EXT2_IOC_GETFLAGS; }" HAVE_WORKING_EXT2_IOC_GETFLAGS) + +LA_CHECK_INCLUDE_FILE("fcntl.h" HAVE_FCNTL_H) +LA_CHECK_INCLUDE_FILE("grp.h" HAVE_GRP_H) +LA_CHECK_INCLUDE_FILE("inttypes.h" HAVE_INTTYPES_H) +LA_CHECK_INCLUDE_FILE("io.h" HAVE_IO_H) +LA_CHECK_INCLUDE_FILE("langinfo.h" HAVE_LANGINFO_H) +LA_CHECK_INCLUDE_FILE("limits.h" HAVE_LIMITS_H) +LA_CHECK_INCLUDE_FILE("linux/types.h" HAVE_LINUX_TYPES_H) +LA_CHECK_INCLUDE_FILE("linux/fiemap.h" HAVE_LINUX_FIEMAP_H) +LA_CHECK_INCLUDE_FILE("linux/fs.h" HAVE_LINUX_FS_H) +LA_CHECK_INCLUDE_FILE("linux/magic.h" HAVE_LINUX_MAGIC_H) +LA_CHECK_INCLUDE_FILE("locale.h" HAVE_LOCALE_H) +LA_CHECK_INCLUDE_FILE("memory.h" HAVE_MEMORY_H) +LA_CHECK_INCLUDE_FILE("paths.h" HAVE_PATHS_H) +LA_CHECK_INCLUDE_FILE("poll.h" HAVE_POLL_H) +LA_CHECK_INCLUDE_FILE("process.h" HAVE_PROCESS_H) +LA_CHECK_INCLUDE_FILE("pwd.h" HAVE_PWD_H) +LA_CHECK_INCLUDE_FILE("regex.h" HAVE_REGEX_H) +LA_CHECK_INCLUDE_FILE("signal.h" HAVE_SIGNAL_H) +LA_CHECK_INCLUDE_FILE("spawn.h" HAVE_SPAWN_H) +LA_CHECK_INCLUDE_FILE("stdarg.h" HAVE_STDARG_H) +LA_CHECK_INCLUDE_FILE("stdint.h" HAVE_STDINT_H) +LA_CHECK_INCLUDE_FILE("stdlib.h" HAVE_STDLIB_H) +LA_CHECK_INCLUDE_FILE("string.h" HAVE_STRING_H) +LA_CHECK_INCLUDE_FILE("strings.h" HAVE_STRINGS_H) +LA_CHECK_INCLUDE_FILE("sys/acl.h" HAVE_SYS_ACL_H) +LA_CHECK_INCLUDE_FILE("sys/cdefs.h" HAVE_SYS_CDEFS_H) +LA_CHECK_INCLUDE_FILE("sys/ioctl.h" HAVE_SYS_IOCTL_H) +LA_CHECK_INCLUDE_FILE("sys/mkdev.h" HAVE_SYS_MKDEV_H) +LA_CHECK_INCLUDE_FILE("sys/mount.h" HAVE_SYS_MOUNT_H) +LA_CHECK_INCLUDE_FILE("sys/param.h" HAVE_SYS_PARAM_H) +LA_CHECK_INCLUDE_FILE("sys/poll.h" HAVE_SYS_POLL_H) +LA_CHECK_INCLUDE_FILE("sys/select.h" HAVE_SYS_SELECT_H) +LA_CHECK_INCLUDE_FILE("sys/stat.h" HAVE_SYS_STAT_H) +LA_CHECK_INCLUDE_FILE("sys/statfs.h" HAVE_SYS_STATFS_H) +LA_CHECK_INCLUDE_FILE("sys/statvfs.h" HAVE_SYS_STATVFS_H) +LA_CHECK_INCLUDE_FILE("sys/time.h" HAVE_SYS_TIME_H) +LA_CHECK_INCLUDE_FILE("sys/utime.h" HAVE_SYS_UTIME_H) +LA_CHECK_INCLUDE_FILE("sys/utsname.h" HAVE_SYS_UTSNAME_H) +LA_CHECK_INCLUDE_FILE("sys/vfs.h" HAVE_SYS_VFS_H) +LA_CHECK_INCLUDE_FILE("sys/wait.h" HAVE_SYS_WAIT_H) +LA_CHECK_INCLUDE_FILE("time.h" HAVE_TIME_H) +LA_CHECK_INCLUDE_FILE("unistd.h" HAVE_UNISTD_H) +LA_CHECK_INCLUDE_FILE("utime.h" HAVE_UTIME_H) +LA_CHECK_INCLUDE_FILE("wchar.h" HAVE_WCHAR_H) +LA_CHECK_INCLUDE_FILE("wctype.h" HAVE_WCTYPE_H) +LA_CHECK_INCLUDE_FILE("windows.h" HAVE_WINDOWS_H) +# Following files need windwos.h, so we should test it after windows.h test. +LA_CHECK_INCLUDE_FILE("wincrypt.h" HAVE_WINCRYPT_H) +LA_CHECK_INCLUDE_FILE("winioctl.h" HAVE_WINIOCTL_H) + +# +# Check whether use of __EXTENSIONS__ is safe. +# We need some macro such as _GNU_SOURCE to use extension functions. +# +SET(_INCLUDE_FILES) +FOREACH (it ${_HEADER}) + SET(_INCLUDE_FILES "${_INCLUDE_FILES}#include <${it}>\n") +ENDFOREACH (it) + +LIBARCHIVE_CHECK_C_SOURCE_COMPILES( + "#define __EXTENSIONS__ 1 + ${_INCLUDE_FILES} + int main() { return 0;}" + SAFE_TO_DEFINE_EXTENSIONS) + +# +# Find Nettle +# +IF(ENABLE_NETTLE) + FIND_PACKAGE(Nettle) + IF(NETTLE_FOUND) + SET(HAVE_LIBNETTLE 1) + SET(HAVE_NETTLE_MD5_H 1) + SET(HAVE_NETTLE_RIPEMD160_H 1) + SET(HAVE_NETTLE_SHA_H 1) + INCLUDE_DIRECTORIES(${NETTLE_INCLUDE_DIR}) + LIST(APPEND ADDITIONAL_LIBS ${NETTLE_LIBRARIES}) + ENDIF(NETTLE_FOUND) + MARK_AS_ADVANCED(CLEAR NETTLE_INCLUDE_DIR) + MARK_AS_ADVANCED(CLEAR NETTLE_LIBRARIES) +ENDIF(ENABLE_NETTLE) + +# +# Find OpenSSL +# (Except on Mac, where OpenSSL is deprecated.) +# +IF(ENABLE_OPENSSL AND NOT CMAKE_SYSTEM_NAME MATCHES "Darwin") + FIND_PACKAGE(OpenSSL) +ELSE() + SET(OPENSSL_FOUND FALSE) # Override cached value +ENDIF() + +# FreeBSD libmd +IF(NOT OPENSSL_FOUND) + CHECK_LIBRARY_EXISTS(md "MD5Init" "" LIBMD_FOUND) + IF(LIBMD_FOUND) + CMAKE_PUSH_CHECK_STATE() # Save the state of the variables + SET(CMAKE_REQUIRED_LIBRARIES "md") + FIND_LIBRARY(LIBMD_LIBRARY NAMES md) + LIST(APPEND ADDITIONAL_LIBS ${LIBMD_LIBRARY}) + CMAKE_POP_CHECK_STATE() # Restore the state of the variables + ENDIF(LIBMD_FOUND) +ENDIF(NOT OPENSSL_FOUND) + +# +# How to prove that CRYPTO functions, which have several names on various +# platforms, just see if archive_crypto.c can compile and link against +# required libraries. +# +MACRO(CHECK_CRYPTO ALGORITHMS IMPLEMENTATION) + FOREACH(ALGORITHM ${ALGORITHMS}) + IF(NOT ARCHIVE_CRYPTO_${ALGORITHM}) + STRING(TOLOWER "${ALGORITHM}" lower_algorithm) + STRING(TOUPPER "${ALGORITHM}" algorithm) + IF ("${IMPLEMENTATION}" MATCHES "^OPENSSL$" AND NOT OPENSSL_FOUND) + SET(ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION} FALSE) + ELSEIF("${IMPLEMENTATION}" MATCHES "^NETTLE$" AND NOT NETTLE_FOUND) + SET(ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION} FALSE) + ENDIF("${IMPLEMENTATION}" MATCHES "^OPENSSL$" AND NOT OPENSSL_FOUND) + + IF(NOT DEFINED ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION}) + # Probe the local implementation for whether this + # crypto implementation is available on this platform. + SET(TRY_CRYPTO_REQUIRED_INCLUDES + "-DINCLUDE_DIRECTORIES:STRING=${CMAKE_BINARY_DIR};${CMAKE_CURRENT_SOURCE_DIR}/libarchive;${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeTmp") + SET(TRY_CRYPTO_REQUIRED_LIBS) + IF ("${IMPLEMENTATION}" MATCHES "^OPENSSL$" AND OPENSSL_FOUND) + SET(TRY_CRYPTO_REQUIRED_INCLUDES + "${TRY_CRYPTO_REQUIRED_INCLUDES};${OPENSSL_INCLUDE_DIR}") + SET(TRY_CRYPTO_REQUIRED_LIBS + "-DLINK_LIBRARIES:STRING=${OPENSSL_LIBRARIES}") + ELSEIF("${IMPLEMENTATION}" MATCHES "^NETTLE$" AND NETTLE_FOUND) + SET(TRY_CRYPTO_REQUIRED_INCLUDES + "${TRY_CRYPTO_REQUIRED_INCLUDES};${NETTLE_INCLUDE_DIR}") + SET(TRY_CRYPTO_REQUIRED_LIBS + "-DLINK_LIBRARIES:STRING=${NETTLE_LIBRARY}") + ELSEIF("${IMPLEMENTATION}" MATCHES "^LIBMD$" AND LIBMD_FOUND) + SET(TRY_CRYPTO_REQUIRED_LIBS + "-DLINK_LIBRARIES:STRING=${LIBMD_LIBRARY}") + ENDIF("${IMPLEMENTATION}" MATCHES "^OPENSSL$" AND OPENSSL_FOUND) + + CONFIGURE_FILE(${CMAKE_CURRENT_SOURCE_DIR}/build/cmake/config.h.in + ${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeTmp/confdefs.h) + FILE(READ "${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeTmp/confdefs.h" + CONFDEFS_H) + FILE(READ "${CMAKE_CURRENT_SOURCE_DIR}/libarchive/archive_crypto.c" + ARCHIVE_CRYPTO_C) + + SET(SOURCE "${CONFDEFS_H} + +#define ARCHIVE_${algorithm}_COMPILE_TEST +#define ARCHIVE_CRYPTO_${algorithm}_${IMPLEMENTATION} +#define PLATFORM_CONFIG_H \"check_crypto_md.h\" + +${ARCHIVE_CRYPTO_C} + +int +main(int argc, char **argv) +{ + archive_${lower_algorithm}_ctx ctx; + archive_${lower_algorithm}_init(&ctx); + archive_${lower_algorithm}_update(&ctx, *argv, argc); + archive_${lower_algorithm}_final(&ctx, NULL); + return 0; +} +") + + FILE(WRITE "${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeTmp/check_crypto_md.h" "") + FILE(WRITE "${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeTmp/check_crypto_md.c" "${SOURCE}") + MESSAGE(STATUS "Checking support for ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION}") + + IF(CMAKE_REQUIRED_LINKER_FLAGS) + SET(CHECK_CRYPTO_ADD_LINKER_FLAGS + "-DCMAKE_EXE_LINKER_FLAGS:STRING=${CMAKE_REQUIRED_LINKER_FLAGS} -DCMAKE_SHARED_LINKER_FLAGS:STRING=${CMAKE_REQUIRED_LINKER_FLAGS} -DCMAKE_MODULE_LINKER_FLAGS:STRING=${CMAKE_REQUIRED_LINKER_FLAGS}") + ELSE(CMAKE_REQUIRED_LINKER_FLAGS) + SET(CHECK_CRYPTO_ADD_LINKER_FLAGS) + ENDIF(CMAKE_REQUIRED_LINKER_FLAGS) + TRY_COMPILE(ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION} + ${CMAKE_BINARY_DIR} + ${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeTmp/check_crypto_md.c + CMAKE_FLAGS ${CHECK_CRYPTO_ADD_LINKER_FLAGS} + "${TRY_CRYPTO_REQUIRED_LIBS}" + "${TRY_CRYPTO_REQUIRED_INCLUDES}" + OUTPUT_VARIABLE OUTPUT) + + # Inform user whether or not we found it; if not, log why we didn't. + IF (ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION}) + MESSAGE(STATUS "Checking support for ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION} -- found") + SET(ARCHIVE_CRYPTO_${ALGORITHM} 1) + ELSE (ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION}) + MESSAGE(STATUS "Checking support for ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION} -- not found") + FILE(APPEND ${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeOutput.log + "Checking support for ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION} failed with the following output:\n" + "${OUTPUT}\n" + "Source file was:\n${SOURCE}\n") + ENDIF (ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION}) + ENDIF(NOT DEFINED ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION}) + + # Add appropriate libs/includes depending on whether the implementation + # was found on this platform. + IF (ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION}) + IF ("${IMPLEMENTATION}" MATCHES "^OPENSSL$" AND OPENSSL_FOUND) + INCLUDE_DIRECTORIES(${OPENSSL_INCLUDE_DIR}) + LIST(APPEND ADDITIONAL_LIBS ${OPENSSL_LIBRARIES}) + LIST(REMOVE_DUPLICATES ADDITIONAL_LIBS) + ENDIF ("${IMPLEMENTATION}" MATCHES "^OPENSSL$" AND OPENSSL_FOUND) + ENDIF (ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION}) + ENDIF(NOT ARCHIVE_CRYPTO_${ALGORITHM}) + ENDFOREACH(ALGORITHM ${ALGORITHMS}) +ENDMACRO(CHECK_CRYPTO ALGORITHMS IMPLEMENTATION) + +# +# CRYPTO functions on Windows is defined at archive_windows.c, thus we do not +# need the test what the functions can be mapped to archive_{crypto name}_init, +# archive_{crypto name}_update and archive_{crypto name}_final. +# The functions on Windows use CALG_{crypto name} macro to create a crypt object +# and then we need to know what CALG_{crypto name} macros is available to show +# ARCHIVE_CRYPTO_{crypto name}_WIN macros because Windows 2000 and earlier version +# of Windows XP do not support SHA256, SHA384 and SHA512. +# +MACRO(CHECK_CRYPTO_WIN CRYPTO_LIST) + IF(WIN32 AND NOT CYGWIN) + FOREACH(CRYPTO ${CRYPTO_LIST}) + IF(NOT ARCHIVE_CRYPTO_${CRYPTO}) + IF(NOT DEFINED ARCHIVE_CRYPTO_${CRYPTO}_WIN) + STRING(TOUPPER "${CRYPTO}" crypto) + SET(ALGID "") + IF ("${CRYPTO}" MATCHES "^MD5$") + SET(ALGID "CALG_MD5") + ENDIF ("${CRYPTO}" MATCHES "^MD5$") + IF ("${CRYPTO}" MATCHES "^SHA1$") + SET(ALGID "CALG_SHA1") + ENDIF ("${CRYPTO}" MATCHES "^SHA1$") + IF ("${CRYPTO}" MATCHES "^SHA256$") + SET(ALGID "CALG_SHA_256") + ENDIF ("${CRYPTO}" MATCHES "^SHA256$") + IF ("${CRYPTO}" MATCHES "^SHA384$") + SET(ALGID "CALG_SHA_384") + ENDIF ("${CRYPTO}" MATCHES "^SHA384$") + IF ("${CRYPTO}" MATCHES "^SHA512$") + SET(ALGID "CALG_SHA_512") + ENDIF ("${CRYPTO}" MATCHES "^SHA512$") + + CONFIGURE_FILE(${CMAKE_CURRENT_SOURCE_DIR}/build/cmake/config.h.in + ${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeTmp/confdefs.h) + FILE(READ "${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeTmp/confdefs.h" + CONFDEFS_H) + + SET(SOURCE "${CONFDEFS_H} + +#define ${crypto}_COMPILE_TEST +#include +#include + +int +main(int argc, char **argv) +{ + return ${ALGID}; +} +") + SET(SOURCE_FILE "${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeTmp/check_crypto_win.c") + + FILE(WRITE "${SOURCE_FILE}" "${SOURCE}") + MESSAGE(STATUS "Checking support for ARCHIVE_CRYPTO_${CRYPTO}_WIN") + + IF(CMAKE_REQUIRED_LINKER_FLAGS) + SET(CHECK_CRYPTO_WIN_ADD_LINKER_FLAGS + "-DCMAKE_EXE_LINKER_FLAGS:STRING=${CMAKE_REQUIRED_LINKER_FLAGS} -DCMAKE_SHARED_LINKER_FLAGS:STRING=${CMAKE_REQUIRED_LINKER_FLAGS} -DCMAKE_MODULE_LINKER_FLAGS:STRING=${CMAKE_REQUIRED_LINKER_FLAGS}") + ELSE(CMAKE_REQUIRED_LINKER_FLAGS) + SET(CHECK_CRYPTO_WIN_ADD_LINKER_FLAGS) + ENDIF(CMAKE_REQUIRED_LINKER_FLAGS) + TRY_COMPILE(ARCHIVE_CRYPTO_${CRYPTO}_WIN + ${CMAKE_BINARY_DIR} + ${SOURCE_FILE} + CMAKE_FLAGS "-DINCLUDE_DIRECTORIES:STRING=${CMAKE_BINARY_DIR};${CMAKE_CURRENT_SOURCE_DIR}/libarchive" ${CHECK_CRYPTO_WIN_ADD_LINKER_FLAGS} + OUTPUT_VARIABLE OUTPUT) + + IF (ARCHIVE_CRYPTO_${CRYPTO}_WIN) + MESSAGE(STATUS + "Checking support for ARCHIVE_CRYPTO_${CRYPTO}_WIN -- found") + SET(ARCHIVE_CRYPTO_${CRYPTO} 1) + ELSE (ARCHIVE_CRYPTO_${CRYPTO}_WIN) + MESSAGE(STATUS + "Checking support for ARCHIVE_CRYPTO_${CRYPTO}_WIN -- not found") + FILE(APPEND + ${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeOutput.log + "Checking support for ARCHIVE_CRYPTO_${CRYPTO}_WIN failed with the following output:\n" + "${OUTPUT}\n" + "Source file was:\n${SOURCE}\n") + ENDIF (ARCHIVE_CRYPTO_${CRYPTO}_WIN) + + ENDIF(NOT DEFINED ARCHIVE_CRYPTO_${CRYPTO}_WIN) + ENDIF(NOT ARCHIVE_CRYPTO_${CRYPTO}) + ENDFOREACH(CRYPTO) + ENDIF(WIN32 AND NOT CYGWIN) +ENDMACRO(CHECK_CRYPTO_WIN CRYPTO_LIST) + +# +# Find iconv +# POSIX defines the second arg as const char ** +# and requires it to be in libc. But we can accept +# a non-const argument here and can support iconv() +# being in libiconv. +# +MACRO(CHECK_ICONV LIB TRY_ICONV_CONST) + IF(NOT HAVE_ICONV) + CMAKE_PUSH_CHECK_STATE() # Save the state of the variables + IF ("CMAKE_C_COMPILER_ID" MATCHES "^GNU$") + # + # During checking iconv proto type, we should use -Werror to avoid the + # success of iconv detection with a warnig which success is a miss + # detection. So this needs for all build mode(even it's a release mode). + # + SET(CMAKE_REQUIRED_FLAGS "${CMAKE_REQUIRED_FLAGS} -Werror") + ENDIF ("CMAKE_C_COMPILER_ID" MATCHES "^GNU$") + IF (MSVC) + # NOTE: /WX option is the same as gcc's -Werror option. + SET(CMAKE_REQUIRED_FLAGS "${CMAKE_REQUIRED_FLAGS} /WX") + ENDIF (MSVC) + # + LIBARCHIVE_CHECK_C_SOURCE_COMPILES( + "#include + #include + int main() { + ${TRY_ICONV_CONST} char *ccp; + iconv_t cd = iconv_open(\"\", \"\"); + iconv(cd, &ccp, (size_t *)0, (char **)0, (size_t *)0); + iconv_close(cd); + return 0; + }" + HAVE_ICONV_${LIB}_${TRY_ICONV_CONST}) + IF(HAVE_ICONV_${LIB}_${TRY_ICONV_CONST}) + SET(HAVE_ICONV true) + SET(ICONV_CONST ${TRY_ICONV_CONST}) + ENDIF(HAVE_ICONV_${LIB}_${TRY_ICONV_CONST}) + CMAKE_POP_CHECK_STATE() # Restore the state of the variables + ENDIF(NOT HAVE_ICONV) +ENDMACRO(CHECK_ICONV TRY_ICONV_CONST) + +IF(ENABLE_ICONV) + CMAKE_PUSH_CHECK_STATE() # Save the state of the variables + FIND_PATH(ICONV_INCLUDE_DIR iconv.h) + IF(ICONV_INCLUDE_DIR) + #SET(INCLUDES ${INCLUDES} "iconv.h") + SET(HAVE_ICONV_H 1) + INCLUDE_DIRECTORIES(${ICONV_INCLUDE_DIR}) + SET(CMAKE_REQUIRED_INCLUDES ${ICONV_INCLUDE_DIR}) + CHECK_ICONV("libc" "const") + CHECK_ICONV("libc" "") + + # If iconv isn't in libc and we have a libiconv, try that. + FIND_LIBRARY(LIBICONV_PATH NAMES iconv libiconv) + IF(NOT HAVE_ICONV AND LIBICONV_PATH) + LIST(APPEND CMAKE_REQUIRED_LIBRARIES ${LIBICONV_PATH}) + # Test if a macro is needed for the library. + TRY_MACRO_FOR_LIBRARY( + "${ICONV_INCLUDE_DIR}" "${LIBICONV_PATH}" + COMPILES + "#include \nint main() {return iconv_close((iconv_t)0);}" + "WITHOUT_LIBICONV_STATIC;LIBICONV_STATIC") + IF(NOT WITHOUT_LIBICONV_STATIC AND LIBICONV_STATIC) + ADD_DEFINITIONS(-DLIBICONV_STATIC) + ENDIF(NOT WITHOUT_LIBICONV_STATIC AND LIBICONV_STATIC) + # + # Set up CMAKE_REQUIRED_* for CHECK_ICONV + # + SET(CMAKE_REQUIRED_INCLUDES ${ICONV_INCLUDE_DIR}) + SET(CMAKE_REQUIRED_LIBRARIES ${LIBICONV_PATH}) + IF(LIBICONV_STATIC) + # LIBICONV_STATIC is necessary for the success of CHECK_ICONV + # on Windows. + SET(CMAKE_REQUIRED_DEFINITIONS "-DLIBICONV_STATIC") + ELSE(LIBICONV_STATIC) + SET(CMAKE_REQUIRED_DEFINITIONS) + ENDIF(LIBICONV_STATIC) + CHECK_ICONV("libiconv" "const") + CHECK_ICONV("libiconv" "") + IF (HAVE_ICONV) + LIST(APPEND ADDITIONAL_LIBS ${LIBICONV_PATH}) + ENDIF(HAVE_ICONV) + ENDIF(NOT HAVE_ICONV AND LIBICONV_PATH) + ENDIF(ICONV_INCLUDE_DIR) + # + # Find locale_charset() for libiconv. + # + IF(LIBICONV_PATH) + SET(CMAKE_REQUIRED_DEFINITIONS) + SET(CMAKE_REQUIRED_INCLUDES ${ICONV_INCLUDE_DIR}) + SET(CMAKE_REQUIRED_LIBRARIES) + CHECK_INCLUDE_FILES("localcharset.h" HAVE_LOCALCHARSET_H) + FIND_LIBRARY(LIBCHARSET_PATH NAMES charset libcharset) + IF(LIBCHARSET_PATH) + SET(CMAKE_REQUIRED_LIBRARIES ${LIBCHARSET_PATH}) + IF(WIN32 AND NOT CYGWIN) + # Test if a macro is needed for the library. + TRY_MACRO_FOR_LIBRARY( + "${ICONV_INCLUDE_DIR}" "${LIBCHARSET_PATH}" + COMPILES + "#include \nint main() {return locale_charset()?1:0;}" + "WITHOUT_LIBCHARSET_STATIC;LIBCHARSET_STATIC") + IF(NOT WITHOUT_LIBCHARSET_STATIC AND LIBCHARSET_STATIC) + ADD_DEFINITIONS(-DLIBCHARSET_STATIC) + ENDIF(NOT WITHOUT_LIBCHARSET_STATIC AND LIBCHARSET_STATIC) + IF(WITHOUT_LIBCHARSET_STATIC OR LIBCHARSET_STATIC) + SET(HAVE_LOCALE_CHARSET ON CACHE INTERNAL + "Have function locale_charset") + ENDIF(WITHOUT_LIBCHARSET_STATIC OR LIBCHARSET_STATIC) + ELSE(WIN32 AND NOT CYGWIN) + CHECK_FUNCTION_EXISTS_GLIBC(locale_charset HAVE_LOCALE_CHARSET) + ENDIF(WIN32 AND NOT CYGWIN) + IF(HAVE_LOCALE_CHARSET) + LIST(APPEND ADDITIONAL_LIBS ${LIBCHARSET_PATH}) + ENDIF(HAVE_LOCALE_CHARSET) + ENDIF(LIBCHARSET_PATH) + ENDIF(LIBICONV_PATH) + CMAKE_POP_CHECK_STATE() # Restore the state of the variables +ELSE(ENABLE_ICONV) + # Make sure ICONV variables are not in CACHE after ENABLE_ICONV disabled + # (once enabled). + UNSET(HAVE_LOCALE_CHARSET CACHE) + UNSET(HAVE_ICONV CACHE) + UNSET(HAVE_ICONV_libc_ CACHE) + UNSET(HAVE_ICONV_libc_const CACHE) + UNSET(HAVE_ICONV_libiconv_ CACHE) + UNSET(HAVE_ICONV_libiconv_const CACHE) + UNSET(ICONV_INCLUDE_DIR CACHE) + UNSET(LIBICONV_PATH CACHE) + UNSET(LIBICONV_DLL CACHE) + UNSET(LIBICONV_STATIC CACHE) + UNSET(LIBCHARSET_DLL CACHE) + UNSET(LIBCHARSET_STATIC CACHE) +ENDIF(ENABLE_ICONV) + +# +# Find Libxml2 +# +FIND_PACKAGE(LibXml2) +IF(LIBXML2_FOUND) + CMAKE_PUSH_CHECK_STATE() # Save the state of the variables + INCLUDE_DIRECTORIES(${LIBXML2_INCLUDE_DIR}) + LIST(APPEND ADDITIONAL_LIBS ${LIBXML2_LIBRARIES}) + SET(HAVE_LIBXML2 1) + # libxml2's include files use iconv.h + SET(CMAKE_REQUIRED_INCLUDES ${ICONV_INCLUDE_DIR} ${LIBXML2_INCLUDE_DIR}) + CHECK_INCLUDE_FILES("libxml/xmlreader.h" HAVE_LIBXML_XMLREADER_H) + CHECK_INCLUDE_FILES("libxml/xmlwriter.h" HAVE_LIBXML_XMLWRITER_H) + # Test if a macro is needed for the library. + TRY_MACRO_FOR_LIBRARY( + "${ICONV_INCLUDE_DIR};${LIBXML2_INCLUDE_DIR}" + "ws2_32.lib;${ZLIB_LIBRARIES};${LIBICONV_PATH};${LIBXML2_LIBRARIES}" + COMPILES + "#include \n#include \nint main() {return xmlTextReaderRead((xmlTextReaderPtr)(void *)0);}" + "WITHOUT_LIBXML_STATIC;LIBXML_STATIC") *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Wed May 11 10:19:46 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C775B37052; Wed, 11 May 2016 10:19:46 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2A027111E; Wed, 11 May 2016 10:19:46 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4BAJjc3011851; Wed, 11 May 2016 10:19:45 GMT (envelope-from mm@FreeBSD.org) Received: (from mm@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4BAJiJP011843; Wed, 11 May 2016 10:19:44 GMT (envelope-from mm@FreeBSD.org) Message-Id: <201605111019.u4BAJiJP011843@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mm set sender to mm@FreeBSD.org using -f From: Martin Matuska Date: Wed, 11 May 2016 10:19:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299425 - in vendor/libarchive/dist: . build build/autoconf build/cmake cat cat/test contrib contrib/android contrib/android/config contrib/android/include cpio cpio/test doc examples e... X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 10:19:46 -0000 Author: mm Date: Wed May 11 10:19:44 2016 New Revision: 299425 URL: https://svnweb.freebsd.org/changeset/base/299425 Log: Update vendor/libarchive/dist to git commit 61c56e5 (post 3.2.0) Added: vendor/libarchive/dist/.travis.yml vendor/libarchive/dist/CONTRIBUTING.md vendor/libarchive/dist/build/autoconf/ax_append_compile_flags.m4 vendor/libarchive/dist/build/autoconf/ax_append_flag.m4 vendor/libarchive/dist/build/autoconf/ax_check_compile_flag.m4 vendor/libarchive/dist/build/autoconf/ax_require_defined.m4 vendor/libarchive/dist/build/cmake/CreatePkgConfigFile.cmake vendor/libarchive/dist/build/cmake/LibarchiveCodeCoverage.cmake vendor/libarchive/dist/cat/ vendor/libarchive/dist/cat/CMakeLists.txt (contents, props changed) vendor/libarchive/dist/cat/bsdcat.1 (contents, props changed) vendor/libarchive/dist/cat/bsdcat.c (contents, props changed) vendor/libarchive/dist/cat/bsdcat.h (contents, props changed) vendor/libarchive/dist/cat/bsdcat_platform.h (contents, props changed) vendor/libarchive/dist/cat/cmdline.c (contents, props changed) vendor/libarchive/dist/cat/test/ vendor/libarchive/dist/cat/test/CMakeLists.txt (contents, props changed) vendor/libarchive/dist/cat/test/main.c (contents, props changed) vendor/libarchive/dist/cat/test/test.h (contents, props changed) vendor/libarchive/dist/cat/test/test_0.c (contents, props changed) vendor/libarchive/dist/cat/test/test_empty.gz.uu vendor/libarchive/dist/cat/test/test_empty.lz4.uu vendor/libarchive/dist/cat/test/test_empty.xz.uu vendor/libarchive/dist/cat/test/test_empty_gz.c (contents, props changed) vendor/libarchive/dist/cat/test/test_empty_lz4.c (contents, props changed) vendor/libarchive/dist/cat/test/test_empty_xz.c (contents, props changed) vendor/libarchive/dist/cat/test/test_error.c (contents, props changed) vendor/libarchive/dist/cat/test/test_error_mixed.c (contents, props changed) vendor/libarchive/dist/cat/test/test_expand.Z.uu vendor/libarchive/dist/cat/test/test_expand.bz2.uu vendor/libarchive/dist/cat/test/test_expand.gz.uu vendor/libarchive/dist/cat/test/test_expand.lz4.uu vendor/libarchive/dist/cat/test/test_expand.plain.uu vendor/libarchive/dist/cat/test/test_expand.xz.uu vendor/libarchive/dist/cat/test/test_expand_Z.c (contents, props changed) vendor/libarchive/dist/cat/test/test_expand_bz2.c (contents, props changed) vendor/libarchive/dist/cat/test/test_expand_gz.c (contents, props changed) vendor/libarchive/dist/cat/test/test_expand_lz4.c (contents, props changed) vendor/libarchive/dist/cat/test/test_expand_mixed.c (contents, props changed) vendor/libarchive/dist/cat/test/test_expand_plain.c (contents, props changed) vendor/libarchive/dist/cat/test/test_expand_xz.c (contents, props changed) vendor/libarchive/dist/cat/test/test_help.c (contents, props changed) vendor/libarchive/dist/cat/test/test_version.c (contents, props changed) vendor/libarchive/dist/contrib/android/ vendor/libarchive/dist/contrib/android/Android.mk (contents, props changed) vendor/libarchive/dist/contrib/android/config/ vendor/libarchive/dist/contrib/android/config/android.h (contents, props changed) vendor/libarchive/dist/contrib/android/config/linux_host.h (contents, props changed) vendor/libarchive/dist/contrib/android/config/windows_host.h (contents, props changed) vendor/libarchive/dist/contrib/android/include/ vendor/libarchive/dist/contrib/android/include/Bcrypt.h (contents, props changed) vendor/libarchive/dist/contrib/android/include/Windows.h (contents, props changed) vendor/libarchive/dist/contrib/android/include/android_lf.h (contents, props changed) vendor/libarchive/dist/cpio/test/test_extract.cpio.lz4.uu vendor/libarchive/dist/cpio/test/test_extract_cpio_lz4.c (contents, props changed) vendor/libarchive/dist/cpio/test/test_option_lz4.c (contents, props changed) vendor/libarchive/dist/cpio/test/test_option_passphrase.c (contents, props changed) vendor/libarchive/dist/cpio/test/test_option_passphrase.zip.uu vendor/libarchive/dist/libarchive/archive_cryptor.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_cryptor_private.h (contents, props changed) vendor/libarchive/dist/libarchive/archive_digest.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_digest_private.h (contents, props changed) vendor/libarchive/dist/libarchive/archive_getdate.h (contents, props changed) vendor/libarchive/dist/libarchive/archive_hmac.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_hmac_private.h (contents, props changed) vendor/libarchive/dist/libarchive/archive_pack_dev.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_pack_dev.h (contents, props changed) vendor/libarchive/dist/libarchive/archive_random.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_random_private.h (contents, props changed) vendor/libarchive/dist/libarchive/archive_read_add_passphrase.3 (contents, props changed) vendor/libarchive/dist/libarchive/archive_read_add_passphrase.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_read_extract2.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_read_support_filter_lz4.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_read_support_format_warc.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_write_add_filter_lz4.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_write_set_format_filter_by_ext.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_write_set_format_raw.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_write_set_format_warc.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_write_set_passphrase.3 (contents, props changed) vendor/libarchive/dist/libarchive/archive_write_set_passphrase.c (contents, props changed) vendor/libarchive/dist/libarchive/archive_xxhash.h (contents, props changed) vendor/libarchive/dist/libarchive/test/test_archive_digest.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_archive_read_add_passphrase.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_archive_write_set_format_filter_by_ext.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_archive_write_set_passphrase.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_compat_lz4.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_compat_lz4_1.tar.lz4.uu vendor/libarchive/dist/libarchive/test/test_compat_lz4_2.tar.lz4.uu vendor/libarchive/dist/libarchive/test/test_compat_lz4_3.tar.lz4.uu vendor/libarchive/dist/libarchive/test/test_compat_lz4_B4.tar.lz4.uu vendor/libarchive/dist/libarchive/test/test_compat_lz4_B4BD.tar.lz4.uu vendor/libarchive/dist/libarchive/test/test_compat_lz4_B4BDBX.tar.lz4.uu vendor/libarchive/dist/libarchive/test/test_compat_lz4_B5.tar.lz4.uu vendor/libarchive/dist/libarchive/test/test_compat_lz4_B5BD.tar.lz4.uu vendor/libarchive/dist/libarchive/test/test_compat_lz4_B6.tar.lz4.uu vendor/libarchive/dist/libarchive/test/test_compat_lz4_B6BD.tar.lz4.uu vendor/libarchive/dist/libarchive/test/test_compat_lz4_B7.tar.lz4.uu vendor/libarchive/dist/libarchive/test/test_compat_lz4_B7BD.tar.lz4.uu vendor/libarchive/dist/libarchive/test/test_compat_uudecode_large.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_compat_uudecode_large.tar.Z.uu vendor/libarchive/dist/libarchive/test/test_read_filter_compress.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_7zip_encryption.7z.uu vendor/libarchive/dist/libarchive/test/test_read_format_7zip_encryption_data.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_7zip_encryption_header.7z.uu vendor/libarchive/dist/libarchive/test/test_read_format_7zip_encryption_header.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_7zip_encryption_partially.7z.uu vendor/libarchive/dist/libarchive/test/test_read_format_7zip_encryption_partially.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_7zip_malformed.7z.uu vendor/libarchive/dist/libarchive/test/test_read_format_7zip_malformed.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_7zip_malformed2.7z.uu vendor/libarchive/dist/libarchive/test/test_read_format_cpio_bin_le.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_cpio_bin_le.cpio.uu vendor/libarchive/dist/libarchive/test/test_read_format_lha_bugfix_0.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_lha_bugfix_0.lzh.uu vendor/libarchive/dist/libarchive/test/test_read_format_rar_encryption_data.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_rar_encryption_data.rar.uu vendor/libarchive/dist/libarchive/test/test_read_format_rar_encryption_header.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_rar_encryption_header.rar.uu vendor/libarchive/dist/libarchive/test/test_read_format_rar_encryption_partially.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_rar_encryption_partially.rar.uu vendor/libarchive/dist/libarchive/test/test_read_format_tar_concatenated.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_tar_concatenated.tar.uu vendor/libarchive/dist/libarchive/test/test_read_format_tar_empty_pax.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_tar_empty_pax.tar.Z.uu vendor/libarchive/dist/libarchive/test/test_read_format_warc.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_warc.warc.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_encryption_data.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_zip_encryption_data.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_encryption_header.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_zip_encryption_header.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_encryption_partially.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_zip_encryption_partially.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_malformed.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_zip_malformed1.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_msdos.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_zip_msdos.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_nested.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_zip_nested.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_nofiletype.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_zip_nofiletype.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_padded.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_zip_padded1.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_padded2.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_padded3.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_traditional_encryption_data.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_zip_traditional_encryption_data.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_winzip_aes.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_zip_winzip_aes128.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_winzip_aes256.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_winzip_aes256_large.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_winzip_aes256_stored.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_winzip_aes_large.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_zip_zip64.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_format_zip_zip64a.zip.uu vendor/libarchive/dist/libarchive/test/test_read_format_zip_zip64b.zip.uu vendor/libarchive/dist/libarchive/test/test_read_too_many_filters.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_read_too_many_filters.gz.uu vendor/libarchive/dist/libarchive/test/test_warn_missing_hardlink_target.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_write_filter_lz4.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_write_format_raw.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_write_format_raw_b64.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_write_format_warc.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_write_format_warc_empty.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_write_format_zip_compression_store.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_write_format_zip_empty_zip64.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_write_format_zip_file.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_write_format_zip_file_zip64.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_write_format_zip_large.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_write_format_zip_zip64.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_write_read_format_zip.c (contents, props changed) vendor/libarchive/dist/libarchive/xxhash.c (contents, props changed) vendor/libarchive/dist/libarchive_fe/passphrase.c (contents, props changed) vendor/libarchive/dist/libarchive_fe/passphrase.h (contents, props changed) vendor/libarchive/dist/tar/test/test_extract.tar.lz4.uu vendor/libarchive/dist/tar/test/test_extract_tar_lz4.c (contents, props changed) vendor/libarchive/dist/tar/test/test_leading_slash.c (contents, props changed) vendor/libarchive/dist/tar/test/test_leading_slash.tar.uu vendor/libarchive/dist/tar/test/test_option_lz4.c (contents, props changed) vendor/libarchive/dist/tar/test/test_option_passphrase.c (contents, props changed) vendor/libarchive/dist/tar/test/test_option_passphrase.zip.uu Deleted: vendor/libarchive/dist/build/cmake/LibarchiveCheckCSourceCompiles.cmake vendor/libarchive/dist/build/cmake/LibarchiveCheckCSourceRuns.cmake vendor/libarchive/dist/libarchive/archive_crypto.c vendor/libarchive/dist/libarchive/archive_crypto_private.h vendor/libarchive/dist/libarchive/test/test_archive_crypto.c vendor/libarchive/dist/libarchive/test/test_write_format_zip_no_compression.c vendor/libarchive/dist/libarchive/test/test_write_zip_set_compression_store.c Modified: vendor/libarchive/dist/.gitignore vendor/libarchive/dist/CMakeLists.txt vendor/libarchive/dist/COPYING vendor/libarchive/dist/INSTALL vendor/libarchive/dist/Makefile.am vendor/libarchive/dist/NEWS vendor/libarchive/dist/README vendor/libarchive/dist/build/autoconf/iconv.m4 vendor/libarchive/dist/build/autogen.sh vendor/libarchive/dist/build/clean.sh vendor/libarchive/dist/build/cmake/config.h.in vendor/libarchive/dist/build/version vendor/libarchive/dist/configure.ac vendor/libarchive/dist/contrib/libarchive.spec vendor/libarchive/dist/contrib/untar.c vendor/libarchive/dist/cpio/CMakeLists.txt vendor/libarchive/dist/cpio/bsdcpio.1 vendor/libarchive/dist/cpio/cmdline.c vendor/libarchive/dist/cpio/cpio.c vendor/libarchive/dist/cpio/cpio.h vendor/libarchive/dist/cpio/cpio_platform.h vendor/libarchive/dist/cpio/cpio_windows.h vendor/libarchive/dist/cpio/test/CMakeLists.txt vendor/libarchive/dist/cpio/test/main.c vendor/libarchive/dist/cpio/test/test.h vendor/libarchive/dist/cpio/test/test_basic.c vendor/libarchive/dist/cpio/test/test_extract_cpio_lzo.c vendor/libarchive/dist/cpio/test/test_option_c.c vendor/libarchive/dist/cpio/test/test_option_version.c vendor/libarchive/dist/doc/mdoc2wiki.awk vendor/libarchive/dist/examples/minitar/Makefile vendor/libarchive/dist/examples/minitar/minitar.c vendor/libarchive/dist/examples/untar.c vendor/libarchive/dist/libarchive/CMakeLists.txt vendor/libarchive/dist/libarchive/archive.h vendor/libarchive/dist/libarchive/archive_endian.h vendor/libarchive/dist/libarchive/archive_entry.3 vendor/libarchive/dist/libarchive/archive_entry.c vendor/libarchive/dist/libarchive/archive_entry.h vendor/libarchive/dist/libarchive/archive_entry_acl.3 vendor/libarchive/dist/libarchive/archive_entry_copy_stat.c vendor/libarchive/dist/libarchive/archive_entry_paths.3 vendor/libarchive/dist/libarchive/archive_entry_perms.3 vendor/libarchive/dist/libarchive/archive_entry_private.h vendor/libarchive/dist/libarchive/archive_entry_sparse.c vendor/libarchive/dist/libarchive/archive_entry_stat.3 vendor/libarchive/dist/libarchive/archive_entry_time.3 vendor/libarchive/dist/libarchive/archive_entry_xattr.c vendor/libarchive/dist/libarchive/archive_getdate.c vendor/libarchive/dist/libarchive/archive_match.c vendor/libarchive/dist/libarchive/archive_options.c vendor/libarchive/dist/libarchive/archive_pathmatch.c vendor/libarchive/dist/libarchive/archive_platform.h vendor/libarchive/dist/libarchive/archive_private.h vendor/libarchive/dist/libarchive/archive_read.3 vendor/libarchive/dist/libarchive/archive_read.c vendor/libarchive/dist/libarchive/archive_read_append_filter.c vendor/libarchive/dist/libarchive/archive_read_data.3 vendor/libarchive/dist/libarchive/archive_read_disk_entry_from_file.c vendor/libarchive/dist/libarchive/archive_read_disk_posix.c vendor/libarchive/dist/libarchive/archive_read_disk_private.h vendor/libarchive/dist/libarchive/archive_read_disk_set_standard_lookup.c vendor/libarchive/dist/libarchive/archive_read_disk_windows.c vendor/libarchive/dist/libarchive/archive_read_extract.c vendor/libarchive/dist/libarchive/archive_read_filter.3 vendor/libarchive/dist/libarchive/archive_read_open.3 vendor/libarchive/dist/libarchive/archive_read_open_fd.c vendor/libarchive/dist/libarchive/archive_read_open_file.c vendor/libarchive/dist/libarchive/archive_read_open_filename.c vendor/libarchive/dist/libarchive/archive_read_open_memory.c vendor/libarchive/dist/libarchive/archive_read_private.h vendor/libarchive/dist/libarchive/archive_read_set_options.3 vendor/libarchive/dist/libarchive/archive_read_set_options.c vendor/libarchive/dist/libarchive/archive_read_support_filter_all.c vendor/libarchive/dist/libarchive/archive_read_support_filter_compress.c vendor/libarchive/dist/libarchive/archive_read_support_filter_lzop.c vendor/libarchive/dist/libarchive/archive_read_support_filter_uu.c vendor/libarchive/dist/libarchive/archive_read_support_filter_xz.c vendor/libarchive/dist/libarchive/archive_read_support_format_7zip.c vendor/libarchive/dist/libarchive/archive_read_support_format_all.c vendor/libarchive/dist/libarchive/archive_read_support_format_ar.c vendor/libarchive/dist/libarchive/archive_read_support_format_cab.c vendor/libarchive/dist/libarchive/archive_read_support_format_cpio.c vendor/libarchive/dist/libarchive/archive_read_support_format_empty.c vendor/libarchive/dist/libarchive/archive_read_support_format_iso9660.c vendor/libarchive/dist/libarchive/archive_read_support_format_lha.c vendor/libarchive/dist/libarchive/archive_read_support_format_mtree.c vendor/libarchive/dist/libarchive/archive_read_support_format_rar.c vendor/libarchive/dist/libarchive/archive_read_support_format_raw.c vendor/libarchive/dist/libarchive/archive_read_support_format_tar.c vendor/libarchive/dist/libarchive/archive_read_support_format_xar.c vendor/libarchive/dist/libarchive/archive_read_support_format_zip.c vendor/libarchive/dist/libarchive/archive_string.c vendor/libarchive/dist/libarchive/archive_util.c vendor/libarchive/dist/libarchive/archive_virtual.c vendor/libarchive/dist/libarchive/archive_windows.c vendor/libarchive/dist/libarchive/archive_windows.h vendor/libarchive/dist/libarchive/archive_write.3 vendor/libarchive/dist/libarchive/archive_write.c vendor/libarchive/dist/libarchive/archive_write_add_filter.c vendor/libarchive/dist/libarchive/archive_write_add_filter_by_name.c vendor/libarchive/dist/libarchive/archive_write_add_filter_bzip2.c vendor/libarchive/dist/libarchive/archive_write_add_filter_grzip.c vendor/libarchive/dist/libarchive/archive_write_add_filter_gzip.c vendor/libarchive/dist/libarchive/archive_write_add_filter_lrzip.c vendor/libarchive/dist/libarchive/archive_write_add_filter_lzop.c vendor/libarchive/dist/libarchive/archive_write_add_filter_program.c vendor/libarchive/dist/libarchive/archive_write_add_filter_xz.c vendor/libarchive/dist/libarchive/archive_write_data.3 vendor/libarchive/dist/libarchive/archive_write_disk.3 vendor/libarchive/dist/libarchive/archive_write_disk_acl.c vendor/libarchive/dist/libarchive/archive_write_disk_posix.c vendor/libarchive/dist/libarchive/archive_write_disk_set_standard_lookup.c vendor/libarchive/dist/libarchive/archive_write_disk_windows.c vendor/libarchive/dist/libarchive/archive_write_filter.3 vendor/libarchive/dist/libarchive/archive_write_finish_entry.3 vendor/libarchive/dist/libarchive/archive_write_format.3 vendor/libarchive/dist/libarchive/archive_write_open.3 vendor/libarchive/dist/libarchive/archive_write_open_filename.c vendor/libarchive/dist/libarchive/archive_write_private.h vendor/libarchive/dist/libarchive/archive_write_set_format.c vendor/libarchive/dist/libarchive/archive_write_set_format_7zip.c vendor/libarchive/dist/libarchive/archive_write_set_format_by_name.c vendor/libarchive/dist/libarchive/archive_write_set_format_gnutar.c vendor/libarchive/dist/libarchive/archive_write_set_format_iso9660.c vendor/libarchive/dist/libarchive/archive_write_set_format_mtree.c vendor/libarchive/dist/libarchive/archive_write_set_format_pax.c vendor/libarchive/dist/libarchive/archive_write_set_format_shar.c vendor/libarchive/dist/libarchive/archive_write_set_format_xar.c vendor/libarchive/dist/libarchive/archive_write_set_format_zip.c vendor/libarchive/dist/libarchive/archive_write_set_options.3 vendor/libarchive/dist/libarchive/filter_fork_windows.c vendor/libarchive/dist/libarchive/libarchive-formats.5 vendor/libarchive/dist/libarchive/libarchive.3 vendor/libarchive/dist/libarchive/libarchive_internals.3 vendor/libarchive/dist/libarchive/mtree.5 vendor/libarchive/dist/libarchive/tar.5 vendor/libarchive/dist/libarchive/test/CMakeLists.txt vendor/libarchive/dist/libarchive/test/README vendor/libarchive/dist/libarchive/test/main.c vendor/libarchive/dist/libarchive/test/read_open_memory.c vendor/libarchive/dist/libarchive/test/test.h vendor/libarchive/dist/libarchive/test/test_archive_getdate.c vendor/libarchive/dist/libarchive/test/test_archive_match_time.c vendor/libarchive/dist/libarchive/test/test_archive_pathmatch.c vendor/libarchive/dist/libarchive/test/test_archive_string.c vendor/libarchive/dist/libarchive/test/test_archive_write_add_filter_by_name.c vendor/libarchive/dist/libarchive/test/test_archive_write_set_format_by_name.c vendor/libarchive/dist/libarchive/test/test_compat_mac.c vendor/libarchive/dist/libarchive/test/test_compat_pax_libarchive_2x.c vendor/libarchive/dist/libarchive/test/test_compat_zip.c vendor/libarchive/dist/libarchive/test/test_fuzz.c vendor/libarchive/dist/libarchive/test/test_gnutar_filename_encoding.c vendor/libarchive/dist/libarchive/test/test_pax_filename_encoding.c vendor/libarchive/dist/libarchive/test/test_read_data_large.c vendor/libarchive/dist/libarchive/test/test_read_disk_directory_traversals.c vendor/libarchive/dist/libarchive/test/test_read_format_7zip.c vendor/libarchive/dist/libarchive/test/test_read_format_ar.c vendor/libarchive/dist/libarchive/test/test_read_format_cab.c vendor/libarchive/dist/libarchive/test/test_read_format_cab_filename.c vendor/libarchive/dist/libarchive/test/test_read_format_cpio_afio.c vendor/libarchive/dist/libarchive/test/test_read_format_cpio_bin.c vendor/libarchive/dist/libarchive/test/test_read_format_cpio_bin_Z.c vendor/libarchive/dist/libarchive/test/test_read_format_cpio_bin_be.c vendor/libarchive/dist/libarchive/test/test_read_format_cpio_bin_bz2.c vendor/libarchive/dist/libarchive/test/test_read_format_cpio_bin_gz.c vendor/libarchive/dist/libarchive/test/test_read_format_cpio_bin_lzip.c vendor/libarchive/dist/libarchive/test/test_read_format_cpio_bin_lzma.c vendor/libarchive/dist/libarchive/test/test_read_format_cpio_bin_xz.c vendor/libarchive/dist/libarchive/test/test_read_format_cpio_filename.c vendor/libarchive/dist/libarchive/test/test_read_format_cpio_odc.c vendor/libarchive/dist/libarchive/test/test_read_format_cpio_svr4_bzip2_rpm.c vendor/libarchive/dist/libarchive/test/test_read_format_cpio_svr4_gzip.c vendor/libarchive/dist/libarchive/test/test_read_format_cpio_svr4_gzip_rpm.c vendor/libarchive/dist/libarchive/test/test_read_format_cpio_svr4c_Z.c vendor/libarchive/dist/libarchive/test/test_read_format_empty.c vendor/libarchive/dist/libarchive/test/test_read_format_gtar_filename.c vendor/libarchive/dist/libarchive/test/test_read_format_gtar_gz.c vendor/libarchive/dist/libarchive/test/test_read_format_gtar_lzma.c vendor/libarchive/dist/libarchive/test/test_read_format_gtar_sparse.c vendor/libarchive/dist/libarchive/test/test_read_format_gtar_sparse_skip_entry.c vendor/libarchive/dist/libarchive/test/test_read_format_iso_Z.c vendor/libarchive/dist/libarchive/test/test_read_format_iso_multi_extent.c vendor/libarchive/dist/libarchive/test/test_read_format_iso_xorriso.c vendor/libarchive/dist/libarchive/test/test_read_format_isojoliet_bz2.c vendor/libarchive/dist/libarchive/test/test_read_format_isojoliet_long.c vendor/libarchive/dist/libarchive/test/test_read_format_isojoliet_rr.c vendor/libarchive/dist/libarchive/test/test_read_format_isojoliet_versioned.c vendor/libarchive/dist/libarchive/test/test_read_format_isorr_bz2.c vendor/libarchive/dist/libarchive/test/test_read_format_isorr_ce.c vendor/libarchive/dist/libarchive/test/test_read_format_isorr_new_bz2.c vendor/libarchive/dist/libarchive/test/test_read_format_isorr_rr_moved.c vendor/libarchive/dist/libarchive/test/test_read_format_isozisofs_bz2.c vendor/libarchive/dist/libarchive/test/test_read_format_lha.c vendor/libarchive/dist/libarchive/test/test_read_format_lha_filename.c vendor/libarchive/dist/libarchive/test/test_read_format_mtree.c vendor/libarchive/dist/libarchive/test/test_read_format_mtree.mtree.uu vendor/libarchive/dist/libarchive/test/test_read_format_pax_bz2.c vendor/libarchive/dist/libarchive/test/test_read_format_rar.c vendor/libarchive/dist/libarchive/test/test_read_format_raw.c vendor/libarchive/dist/libarchive/test/test_read_format_tar.c vendor/libarchive/dist/libarchive/test/test_read_format_tar_empty_filename.c vendor/libarchive/dist/libarchive/test/test_read_format_tar_filename.c vendor/libarchive/dist/libarchive/test/test_read_format_tbz.c vendor/libarchive/dist/libarchive/test/test_read_format_tgz.c vendor/libarchive/dist/libarchive/test/test_read_format_tlz.c vendor/libarchive/dist/libarchive/test/test_read_format_txz.c vendor/libarchive/dist/libarchive/test/test_read_format_tz.c vendor/libarchive/dist/libarchive/test/test_read_format_ustar_filename.c vendor/libarchive/dist/libarchive/test/test_read_format_xar.c vendor/libarchive/dist/libarchive/test/test_read_format_zip.c vendor/libarchive/dist/libarchive/test/test_read_format_zip_comment_stored.c vendor/libarchive/dist/libarchive/test/test_read_format_zip_filename.c vendor/libarchive/dist/libarchive/test/test_read_format_zip_mac_metadata.c vendor/libarchive/dist/libarchive/test/test_read_format_zip_sfx.c vendor/libarchive/dist/libarchive/test/test_read_truncated.c vendor/libarchive/dist/libarchive/test/test_sparse_basic.c vendor/libarchive/dist/libarchive/test/test_ustar_filename_encoding.c vendor/libarchive/dist/libarchive/test/test_write_disk_appledouble.c vendor/libarchive/dist/libarchive/test/test_write_disk_hfs_compression.c vendor/libarchive/dist/libarchive/test/test_write_disk_mac_metadata.c vendor/libarchive/dist/libarchive/test/test_write_disk_no_hfs_compression.c vendor/libarchive/dist/libarchive/test/test_write_disk_perms.c vendor/libarchive/dist/libarchive/test/test_write_disk_secure.c vendor/libarchive/dist/libarchive/test/test_write_filter_lzop.c vendor/libarchive/dist/libarchive/test/test_write_filter_program.c vendor/libarchive/dist/libarchive/test/test_write_format_ar.c vendor/libarchive/dist/libarchive/test/test_write_format_cpio_newc.c vendor/libarchive/dist/libarchive/test/test_write_format_gnutar.c vendor/libarchive/dist/libarchive/test/test_write_format_iso9660.c vendor/libarchive/dist/libarchive/test/test_write_format_iso9660_boot.c vendor/libarchive/dist/libarchive/test/test_write_format_mtree.c vendor/libarchive/dist/libarchive/test/test_write_format_pax.c vendor/libarchive/dist/libarchive/test/test_write_format_tar.c vendor/libarchive/dist/libarchive/test/test_write_format_tar_sparse.c vendor/libarchive/dist/libarchive/test/test_write_format_zip.c vendor/libarchive/dist/libarchive/test/test_zip_filename_encoding.c vendor/libarchive/dist/libarchive_fe/err.c vendor/libarchive/dist/libarchive_fe/err.h vendor/libarchive/dist/tar/CMakeLists.txt vendor/libarchive/dist/tar/bsdtar.1 vendor/libarchive/dist/tar/bsdtar.c vendor/libarchive/dist/tar/bsdtar.h vendor/libarchive/dist/tar/bsdtar_platform.h vendor/libarchive/dist/tar/bsdtar_windows.h vendor/libarchive/dist/tar/cmdline.c vendor/libarchive/dist/tar/creation_set.c vendor/libarchive/dist/tar/read.c vendor/libarchive/dist/tar/subst.c vendor/libarchive/dist/tar/test/CMakeLists.txt vendor/libarchive/dist/tar/test/main.c vendor/libarchive/dist/tar/test/test.h vendor/libarchive/dist/tar/test/test_option_X_upper.c vendor/libarchive/dist/tar/test/test_option_b.c vendor/libarchive/dist/tar/test/test_option_s.c vendor/libarchive/dist/tar/test/test_version.c vendor/libarchive/dist/tar/util.c vendor/libarchive/dist/tar/write.c Modified: vendor/libarchive/dist/.gitignore ============================================================================== --- vendor/libarchive/dist/.gitignore Wed May 11 10:18:56 2016 (r299424) +++ vendor/libarchive/dist/.gitignore Wed May 11 10:19:44 2016 (r299425) @@ -1,3 +1,4 @@ +*~ *.o *.exe *.lo @@ -7,6 +8,8 @@ Makefile Makefile.in aclocal.m4 autom4te.cache/ +bsdcat +bsdcat_test bsdcpio bsdcpio_test bsdtar @@ -24,9 +27,13 @@ build/autoconf/ltversion.m4 build/autoconf/lt~obsolete.m4 build/autoconf/missing build/pkgconfig/libarchive.pc +cat/.deps/ +cat/.dirstamp +cat/test/.deps/ +cat/test/.dirstamp +cat/test/list.h config.h config.h.in -config.h.in~ config.log config.status configure @@ -82,3 +89,17 @@ libarchive/libarchive.so libarchive/libarchive.so.* .DS_Store + +bsdcat_test.log +bsdcat_test.trs +bsdcpio_test.log +bsdcpio_test.trs +bsdtar_test.log +bsdtar_test.trs +build/autoconf/test-driver +libarchive_test.log +libarchive_test.trs +test-suite.log + +.sw? +.*.sw? Added: vendor/libarchive/dist/.travis.yml ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libarchive/dist/.travis.yml Wed May 11 10:19:44 2016 (r299425) @@ -0,0 +1,20 @@ +language: C +sudo: true +compiler: + - gcc + - clang +before_install: + - sudo add-apt-repository ppa:kubuntu-ppa/backports -y + - sudo apt-get update -qq +install: + - sudo apt-get install -y cmake=2.8.12.2-0ubuntu1~ubuntu12.04.1~ppa2 + - sudo apt-get install -y libbz2-dev libzip-dev liblzma-dev +before_script: + - BUILD_DIR=`pwd`/BUILD + - mkdir -p ${BUILD_DIR} + - cd ${BUILD_DIR} + - cmake .. +script: + - cd ${BUILD_DIR} + - make + - make test Modified: vendor/libarchive/dist/CMakeLists.txt ============================================================================== --- vendor/libarchive/dist/CMakeLists.txt Wed May 11 10:18:56 2016 (r299424) +++ vendor/libarchive/dist/CMakeLists.txt Wed May 11 10:19:44 2016 (r299425) @@ -1,5 +1,5 @@ # -CMAKE_MINIMUM_REQUIRED(VERSION 2.8.6 FATAL_ERROR) +CMAKE_MINIMUM_REQUIRED(VERSION 2.8.12 FATAL_ERROR) # PROJECT(libarchive C) # @@ -15,7 +15,7 @@ endif() # RelWithDebInfo : Release build with Debug Info # MinSizeRel : Release Min Size build IF(NOT CMAKE_BUILD_TYPE) - SET(CMAKE_BUILD_TYPE "Release" CACHE STRING "Build Type" FORCE) + SET(CMAKE_BUILD_TYPE "Debug" CACHE STRING "Build Type" FORCE) ENDIF(NOT CMAKE_BUILD_TYPE) # Set a value type to properly display CMAKE_BUILD_TYPE on GUI if the # value type is "UNINITIALIZED". @@ -35,6 +35,9 @@ ENDIF(NOT "${CMAKE_BUILD_TYPE}" # On MacOS, prefer MacPorts libraries to system libraries. # I haven't come up with a compelling argument for this to be conditional. list(APPEND CMAKE_PREFIX_PATH /opt/local) +# Enable @rpath in the install name. +# detail in "cmake --help-policy CMP0042" +SET(CMAKE_MACOSX_RPATH ON) # # Version - read from 'version' file. @@ -55,11 +58,12 @@ STRING(REGEX REPLACE "[0]*([^0]*[0-9])$" SET(VERSION "${_major}.${_trimmed_minor}.${_trimmed_revision}${_quality}") SET(BSDCPIO_VERSION_STRING "${VERSION}") SET(BSDTAR_VERSION_STRING "${VERSION}") +SET(BSDCAT_VERSION_STRING "${VERSION}") SET(LIBARCHIVE_VERSION_NUMBER "${_version_number}") SET(LIBARCHIVE_VERSION_STRING "${VERSION}") # INTERFACE_VERSION increments with every release -# libarchive 2.7 == interface version 9 = 2 + 7 +# libarchive 2.7 == interface version 9 = 2 + 7 # libarchive 2.8 == interface version 10 = 2 + 8 # libarchive 2.9 == interface version 11 = 2 + 9 # libarchive 3.0 == interface version 12 @@ -84,7 +88,7 @@ SET(CMAKE_REQUIRED_FLAGS) # Especially for early development, we want to be a little # aggressive about diagnosing build problems; this can get # relaxed somewhat in final shipping versions. -IF ("CMAKE_C_COMPILER_ID" MATCHES "^GNU$") +IF (CMAKE_C_COMPILER_ID MATCHES "^GNU$") SET(CMAKE_REQUIRED_FLAGS "-Wall -Wformat -Wformat-security") ################################################################# # Set compile flags for all build types. @@ -92,11 +96,43 @@ IF ("CMAKE_C_COMPILER_ID" MATCHES "^GNU$ ################################################################# # Set compile flags for debug build. # This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug" - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Werror -Wextra -Wunused") + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Werror") + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Wextra") + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Wunused") SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Wshadow") SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Wmissing-prototypes") SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Wcast-qual") -ENDIF ("CMAKE_C_COMPILER_ID" MATCHES "^GNU$") +ENDIF (CMAKE_C_COMPILER_ID MATCHES "^GNU$") +IF (CMAKE_C_COMPILER_ID MATCHES "^Clang$") + SET(CMAKE_REQUIRED_FLAGS "-Wall -Wformat -Wformat-security") + ################################################################# + # Set compile flags for all build types. + SET(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wall -Wformat -Wformat-security") + ################################################################# + # Set compile flags for debug build. + # This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug" + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -g") + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Werror") + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Wextra") + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Wunused") + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Wshadow") + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Wmissing-prototypes") + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -Wcast-qual") +ENDIF (CMAKE_C_COMPILER_ID MATCHES "^Clang$") +IF (CMAKE_C_COMPILER_ID MATCHES "^XL$") + SET(CMAKE_C_COMPILER "xlc_r") + SET(CMAKE_REQUIRED_FLAGS "-qflag=e:e -qformat=sec") + ################################################################# + # Set compile flags for all build types. + SET(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -qflag=e:e -qformat=sec") + ################################################################# + # Set compile flags for debug build. + # This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug" + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -g") + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -qhalt=w") + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -qflag=w:w") + SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -qinfo=pro:use") +ENDIF(CMAKE_C_COMPILER_ID MATCHES "^XL$") IF (MSVC) ################################################################# # Set compile flags for debug build. @@ -143,17 +179,36 @@ include(CTest) OPTION(ENABLE_NETTLE "Enable use of Nettle" ON) OPTION(ENABLE_OPENSSL "Enable use of OpenSSL" ON) +OPTION(ENABLE_LZMA "Enable the use of the system found LZMA library if found" ON) +OPTION(ENABLE_ZLIB "Enable the use of the system found ZLIB library if found" ON) +OPTION(ENABLE_BZip2 "Enable the use of the system found BZip2 library if found" ON) +OPTION(ENABLE_LIBXML2 "Enable the use of the system found libxml2 library if found" ON) +OPTION(ENABLE_EXPAT "Enable the use of the system found EXPAT library if found" ON) +OPTION(ENABLE_PCREPOSIX "Enable the use of the system found PCREPOSIX library if found" ON) +OPTION(ENABLE_LibGCC "Enable the use of the system found LibGCC library if found" ON) +# CNG is used for encrypt/decrypt Zip archives on Windows. +OPTION(ENABLE_CNG "Enable the use of CNG(Crypto Next Generation)" ON) + OPTION(ENABLE_TAR "Enable tar building" ON) OPTION(ENABLE_TAR_SHARED "Enable dynamic build of tar" FALSE) OPTION(ENABLE_CPIO "Enable cpio building" ON) OPTION(ENABLE_CPIO_SHARED "Enable dynamic build of cpio" FALSE) +OPTION(ENABLE_CAT "Enable cat building" ON) +OPTION(ENABLE_CAT_SHARED "Enable dynamic build of cat" FALSE) OPTION(ENABLE_XATTR "Enable extended attribute support" ON) OPTION(ENABLE_ACL "Enable ACL support" ON) OPTION(ENABLE_ICONV "Enable iconv support" ON) OPTION(ENABLE_TEST "Enable unit and regression tests" ON) +OPTION(ENABLE_COVERAGE "Enable code coverage (GCC only, automatically sets ENABLE_TEST to ON)" FALSE) +OPTION(ENABLE_INSTALL "Enable installing of libraries" ON) + SET(POSIX_REGEX_LIB "AUTO" CACHE STRING "Choose what library should provide POSIX regular expression support") SET(ENABLE_SAFESEH "AUTO" CACHE STRING "Enable use of /SAFESEH linker flag (MSVC only)") -SET(WINDOWS_VERSION "" CACHE STRING "Set Windows version to use (Windows only)") +SET(WINDOWS_VERSION "WIN7" CACHE STRING "Set Windows version to use (Windows only)") + +IF(ENABLE_COVERAGE) + include(LibarchiveCodeCoverage) +ENDIF(ENABLE_COVERAGE) IF(ENABLE_TEST) ENABLE_TESTING() @@ -161,22 +216,35 @@ ENDIF(ENABLE_TEST) IF(WIN32) IF(WINDOWS_VERSION STREQUAL "WIN8") + SET(NTDDI_VERSION 0x06020000) + SET(_WIN32_WINNT 0x0602) SET(WINVER 0x0602) ELSEIF(WINDOWS_VERSION STREQUAL "WIN7") + SET(NTDDI_VERSION 0x06010000) + SET(_WIN32_WINNT 0x0601) SET(WINVER 0x0601) ELSEIF(WINDOWS_VERSION STREQUAL "WS08") + SET(NTDDI_VERSION 0x06000100) + SET(_WIN32_WINNT 0x0600) SET(WINVER 0x0600) ELSEIF(WINDOWS_VERSION STREQUAL "VISTA") + SET(NTDDI_VERSION 0x06000000) + SET(_WIN32_WINNT 0x0600) SET(WINVER 0x0600) ELSEIF(WINDOWS_VERSION STREQUAL "WS03") + SET(NTDDI_VERSION 0x05020000) + SET(_WIN32_WINNT 0x0502) SET(WINVER 0x0502) ELSEIF(WINDOWS_VERSION STREQUAL "WINXP") + SET(NTDDI_VERSION 0x05010000) + SET(_WIN32_WINNT 0x0501) SET(WINVER 0x0501) ELSE(WINDOWS_VERSION STREQUAL "WIN8") - # The default is to use Windows 2000 API. - SET(WINVER 0x0500) + # Default to Windows Server 2003 API if we don't recognize the specifier + SET(NTDDI_VERSION 0x05020000) + SET(_WIN32_WINNT 0x0502) + SET(WINVER 0x0502) ENDIF(WINDOWS_VERSION STREQUAL "WIN8") - SET(_WIN32_WINNT ${WINVER}) ENDIF(WIN32) IF(MSVC) @@ -184,12 +252,12 @@ IF(MSVC) SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /SAFESEH") SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} /SAFESEH") SET(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} /SAFESEH") - SET(CMAKE_REQUIRED_LINKER_FLAGS "/SAFESEH") + SET(ENV{LDFLAGS} "$ENV{LDFLAGS} /SAFESEH") ELSEIF(ENABLE_SAFESEH STREQUAL "NO") SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /SAFESEH:NO") SET(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} /SAFESEH:NO") SET(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} /SAFESEH:NO") - SET(CMAKE_REQUIRED_LINKER_FLAGS "/SAFESEH:NO") + SET(ENV{LDFLAGS} "$ENV{LDFLAGS} /SAFESEH:NO") ENDIF(ENABLE_SAFESEH STREQUAL "YES") ENDIF(MSVC) @@ -198,8 +266,8 @@ IF("${CMAKE_C_PLATFORM_ID}" MATCHES "^(H ENDIF() # -INCLUDE(LibarchiveCheckCSourceCompiles) -INCLUDE(LibarchiveCheckCSourceRuns) +INCLUDE(CheckCSourceCompiles) +INCLUDE(CheckCSourceRuns) INCLUDE(CheckFileOffsetBits) INCLUDE(CheckFuncs) INCLUDE(CheckHeaderDirent) @@ -263,9 +331,9 @@ MACRO (TRY_MACRO_FOR_LIBRARY INCLUDES LI ENDIF(NOT "${PREV_VAR_WITH_LIB}" STREQUAL "${LIBRARIES}") # Check if the library can be used with the macro. IF("${TRY_TYPE}" MATCHES "COMPILES") - LIBARCHIVE_CHECK_C_SOURCE_COMPILES("${SAMPLE_SOURCE}" ${VAR}) + CHECK_C_SOURCE_COMPILES("${SAMPLE_SOURCE}" ${VAR}) ELSEIF("${TRY_TYPE}" MATCHES "RUNS") - LIBARCHIVE_CHECK_C_SOURCE_RUNS("${SAMPLE_SOURCE}" ${VAR}) + CHECK_C_SOURCE_RUNS("${SAMPLE_SOURCE}" ${VAR}) ELSE("${TRY_TYPE}" MATCHES "COMPILES") MESSAGE(FATAL_ERROR "UNKNOWN KEYWORD \"${TRY_TYPE}\" FOR TRY_TYPE") ENDIF("${TRY_TYPE}" MATCHES "COMPILES") @@ -297,11 +365,11 @@ IF(DEFINED __GNUWIN32PATH AND EXISTS "${ # e.g. # cmake -DCMAKE_PREFIX_PATH= # - # If compiling error occured in zconf.h, You may need patch to zconf.h. + # If compiling error occurred in zconf.h, You may need patch to zconf.h. #--- zconf.h.orig 2005-07-21 00:40:26.000000000 #+++ zconf.h 2009-01-19 11:39:10.093750000 #@@ -286,7 +286,7 @@ - # + # # #if 1 /* HAVE_UNISTD_H -- this line is updated by ./configure */ # # include /* for off_t */ #-# include /* for SEEK_* and off_t */ @@ -315,7 +383,11 @@ SET(ADDITIONAL_LIBS "") # # Find ZLIB # -FIND_PACKAGE(ZLIB) +IF(ENABLE_ZLIB) + FIND_PACKAGE(ZLIB) +ELSE() + SET(ZLIB_FOUND FALSE) # Override cached value +ENDIF() IF(ZLIB_FOUND) SET(HAVE_LIBZ 1) SET(HAVE_ZLIB_H 1) @@ -350,7 +422,11 @@ MARK_AS_ADVANCED(CLEAR ZLIB_LIBRARY) # # Find BZip2 # -FIND_PACKAGE(BZip2) +IF(ENABLE_BZip2) + FIND_PACKAGE(BZip2) +ELSE() + SET(BZIP2_FOUND FALSE) # Override cached value +ENDIF() IF(BZIP2_FOUND) SET(HAVE_LIBBZ2 1) SET(HAVE_BZLIB_H 1) @@ -370,10 +446,18 @@ IF(BZIP2_FOUND) ENDIF(BZIP2_FOUND) MARK_AS_ADVANCED(CLEAR BZIP2_INCLUDE_DIR) MARK_AS_ADVANCED(CLEAR BZIP2_LIBRARIES) + + # # Find LZMA # -FIND_PACKAGE(LZMA) +IF(ENABLE_LZMA) + FIND_PACKAGE(LZMA) +ELSE() + SET(LZMA_FOUND FALSE) # Override cached value + SET(LZMADEC_FOUND FALSE) # Override cached value +ENDIF() + IF(LZMA_FOUND) SET(HAVE_LIBLZMA 1) SET(HAVE_LZMA_H 1) @@ -393,6 +477,8 @@ ELSEIF(LZMADEC_FOUND) SET(HAVE_LZMADEC_H 1) INCLUDE_DIRECTORIES(${LZMADEC_INCLUDE_DIR}) LIST(APPEND ADDITIONAL_LIBS ${LZMADEC_LIBRARIES}) +ELSE(LZMA_FOUND) +# LZMA not found and will not be used. ENDIF(LZMA_FOUND) # # Find LZO2 @@ -418,6 +504,33 @@ IF(LZO2_FOUND) ENDIF(LZO2_FOUND) MARK_AS_ADVANCED(CLEAR LZO2_INCLUDE_DIR) MARK_AS_ADVANCED(CLEAR LZO2_LIBRARY) +# +# Find LZ4 +# +IF (LZ4_INCLUDE_DIR) + # Already in cache, be silent + SET(LZ4_FIND_QUIETLY TRUE) +ENDIF (LZ4_INCLUDE_DIR) + +FIND_PATH(LZ4_INCLUDE_DIR lz4.h) +FIND_LIBRARY(LZ4_LIBRARY NAMES lz4 liblz4) +INCLUDE(FindPackageHandleStandardArgs) +FIND_PACKAGE_HANDLE_STANDARD_ARGS(LZ4 DEFAULT_MSG LZ4_LIBRARY LZ4_INCLUDE_DIR) +IF(LZ4_FOUND) + SET(HAVE_LIBLZ4 1) + SET(HAVE_LZ4_H 1) + CMAKE_PUSH_CHECK_STATE() # Save the state of the variables + SET(CMAKE_REQUIRED_INCLUDES ${LZ4_INCLUDE_DIR}) + CHECK_INCLUDE_FILES("lz4hc.h" HAVE_LZ4HC_H) + CMAKE_POP_CHECK_STATE() # Restore the state of the variables + INCLUDE_DIRECTORIES(${LZ4_INCLUDE_DIR}) + LIST(APPEND ADDITIONAL_LIBS ${LZ4_LIBRARY}) + # + # TODO: test for static library. + # +ENDIF(LZ4_FOUND) +MARK_AS_ADVANCED(CLEAR LZ4_INCLUDE_DIR) +MARK_AS_ADVANCED(CLEAR LZ4_LIBRARY) # # Check headers @@ -444,7 +557,7 @@ LA_CHECK_INCLUDE_FILE("dlfcn.h" HAVE_DLF LA_CHECK_INCLUDE_FILE("errno.h" HAVE_ERRNO_H) LA_CHECK_INCLUDE_FILE("ext2fs/ext2_fs.h" HAVE_EXT2FS_EXT2_FS_H) -LIBARCHIVE_CHECK_C_SOURCE_COMPILES("#include +CHECK_C_SOURCE_COMPILES("#include #include int main(void) { return EXT2_IOC_GETFLAGS; }" HAVE_WORKING_EXT2_IOC_GETFLAGS) @@ -463,7 +576,9 @@ LA_CHECK_INCLUDE_FILE("memory.h" HAVE_ME LA_CHECK_INCLUDE_FILE("paths.h" HAVE_PATHS_H) LA_CHECK_INCLUDE_FILE("poll.h" HAVE_POLL_H) LA_CHECK_INCLUDE_FILE("process.h" HAVE_PROCESS_H) +LA_CHECK_INCLUDE_FILE("pthread.h" HAVE_PTHREAD_H) LA_CHECK_INCLUDE_FILE("pwd.h" HAVE_PWD_H) +LA_CHECK_INCLUDE_FILE("readpassphrase.h" HAVE_READPASSPHRASE_H) LA_CHECK_INCLUDE_FILE("regex.h" HAVE_REGEX_H) LA_CHECK_INCLUDE_FILE("signal.h" HAVE_SIGNAL_H) LA_CHECK_INCLUDE_FILE("spawn.h" HAVE_SPAWN_H) @@ -494,6 +609,11 @@ LA_CHECK_INCLUDE_FILE("utime.h" HAVE_UTI LA_CHECK_INCLUDE_FILE("wchar.h" HAVE_WCHAR_H) LA_CHECK_INCLUDE_FILE("wctype.h" HAVE_WCTYPE_H) LA_CHECK_INCLUDE_FILE("windows.h" HAVE_WINDOWS_H) +IF(ENABLE_CNG) + LA_CHECK_INCLUDE_FILE("Bcrypt.h" HAVE_BCRYPT_H) +ELSE(ENABLE_CNG) + UNSET(HAVE_BCRYPT_H CACHE) +ENDIF(ENABLE_CNG) # Following files need windwos.h, so we should test it after windows.h test. LA_CHECK_INCLUDE_FILE("wincrypt.h" HAVE_WINCRYPT_H) LA_CHECK_INCLUDE_FILE("winioctl.h" HAVE_WINIOCTL_H) @@ -507,7 +627,7 @@ FOREACH (it ${_HEADER}) SET(_INCLUDE_FILES "${_INCLUDE_FILES}#include <${it}>\n") ENDFOREACH (it) -LIBARCHIVE_CHECK_C_SOURCE_COMPILES( +CHECK_C_SOURCE_COMPILES( "#define __EXTENSIONS__ 1 ${_INCLUDE_FILES} int main() { return 0;}" @@ -520,11 +640,17 @@ IF(ENABLE_NETTLE) FIND_PACKAGE(Nettle) IF(NETTLE_FOUND) SET(HAVE_LIBNETTLE 1) - SET(HAVE_NETTLE_MD5_H 1) - SET(HAVE_NETTLE_RIPEMD160_H 1) - SET(HAVE_NETTLE_SHA_H 1) - INCLUDE_DIRECTORIES(${NETTLE_INCLUDE_DIR}) LIST(APPEND ADDITIONAL_LIBS ${NETTLE_LIBRARIES}) + INCLUDE_DIRECTORIES(${NETTLE_INCLUDE_DIR}) + + LIST(APPEND CMAKE_REQUIRED_INCLUDES ${NETTLE_INCLUDE_DIR}) + LA_CHECK_INCLUDE_FILE("nettle/aes.h" HAVE_NETTLE_AES_H) + LA_CHECK_INCLUDE_FILE("nettle/hmac.h" HAVE_NETTLE_HMAC_H) + LA_CHECK_INCLUDE_FILE("nettle/md5.h" HAVE_NETTLE_MD5_H) + LA_CHECK_INCLUDE_FILE("nettle/pbkdf2.h" HAVE_NETTLE_PBKDF2_H) + LA_CHECK_INCLUDE_FILE("nettle/ripemd160.h" HAVE_NETTLE_RIPEMD160_H) + LA_CHECK_INCLUDE_FILE("nettle/sha.h" HAVE_NETTLE_SHA_H) + ENDIF(NETTLE_FOUND) MARK_AS_ADVANCED(CLEAR NETTLE_INCLUDE_DIR) MARK_AS_ADVANCED(CLEAR NETTLE_LIBRARIES) @@ -536,6 +662,11 @@ ENDIF(ENABLE_NETTLE) # IF(ENABLE_OPENSSL AND NOT CMAKE_SYSTEM_NAME MATCHES "Darwin") FIND_PACKAGE(OpenSSL) + IF(OPENSSL_FOUND) + SET(HAVE_LIBCRYPTO 1) + INCLUDE_DIRECTORIES(${OPENSSL_INCLUDE_DIR}) + LIST(APPEND ADDITIONAL_LIBS ${OPENSSL_CRYPTO_LIBRARY}) + ENDIF(OPENSSL_FOUND) ELSE() SET(OPENSSL_FOUND FALSE) # Override cached value ENDIF() @@ -554,7 +685,7 @@ ENDIF(NOT OPENSSL_FOUND) # # How to prove that CRYPTO functions, which have several names on various -# platforms, just see if archive_crypto.c can compile and link against +# platforms, just see if archive_digest.c can compile and link against # required libraries. # MACRO(CHECK_CRYPTO ALGORITHMS IMPLEMENTATION) @@ -593,7 +724,7 @@ MACRO(CHECK_CRYPTO ALGORITHMS IMPLEMENTA ${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeTmp/confdefs.h) FILE(READ "${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeTmp/confdefs.h" CONFDEFS_H) - FILE(READ "${CMAKE_CURRENT_SOURCE_DIR}/libarchive/archive_crypto.c" + FILE(READ "${CMAKE_CURRENT_SOURCE_DIR}/libarchive/archive_digest.c" ARCHIVE_CRYPTO_C) SET(SOURCE "${CONFDEFS_H} @@ -619,16 +750,10 @@ main(int argc, char **argv) FILE(WRITE "${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeTmp/check_crypto_md.c" "${SOURCE}") MESSAGE(STATUS "Checking support for ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION}") - IF(CMAKE_REQUIRED_LINKER_FLAGS) - SET(CHECK_CRYPTO_ADD_LINKER_FLAGS - "-DCMAKE_EXE_LINKER_FLAGS:STRING=${CMAKE_REQUIRED_LINKER_FLAGS} -DCMAKE_SHARED_LINKER_FLAGS:STRING=${CMAKE_REQUIRED_LINKER_FLAGS} -DCMAKE_MODULE_LINKER_FLAGS:STRING=${CMAKE_REQUIRED_LINKER_FLAGS}") - ELSE(CMAKE_REQUIRED_LINKER_FLAGS) - SET(CHECK_CRYPTO_ADD_LINKER_FLAGS) - ENDIF(CMAKE_REQUIRED_LINKER_FLAGS) TRY_COMPILE(ARCHIVE_CRYPTO_${ALGORITHM}_${IMPLEMENTATION} ${CMAKE_BINARY_DIR} ${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeTmp/check_crypto_md.c - CMAKE_FLAGS ${CHECK_CRYPTO_ADD_LINKER_FLAGS} + CMAKE_FLAGS "${TRY_CRYPTO_REQUIRED_LIBS}" "${TRY_CRYPTO_REQUIRED_INCLUDES}" OUTPUT_VARIABLE OUTPUT) @@ -713,16 +838,10 @@ main(int argc, char **argv) FILE(WRITE "${SOURCE_FILE}" "${SOURCE}") MESSAGE(STATUS "Checking support for ARCHIVE_CRYPTO_${CRYPTO}_WIN") - IF(CMAKE_REQUIRED_LINKER_FLAGS) - SET(CHECK_CRYPTO_WIN_ADD_LINKER_FLAGS - "-DCMAKE_EXE_LINKER_FLAGS:STRING=${CMAKE_REQUIRED_LINKER_FLAGS} -DCMAKE_SHARED_LINKER_FLAGS:STRING=${CMAKE_REQUIRED_LINKER_FLAGS} -DCMAKE_MODULE_LINKER_FLAGS:STRING=${CMAKE_REQUIRED_LINKER_FLAGS}") - ELSE(CMAKE_REQUIRED_LINKER_FLAGS) - SET(CHECK_CRYPTO_WIN_ADD_LINKER_FLAGS) - ENDIF(CMAKE_REQUIRED_LINKER_FLAGS) TRY_COMPILE(ARCHIVE_CRYPTO_${CRYPTO}_WIN ${CMAKE_BINARY_DIR} ${SOURCE_FILE} - CMAKE_FLAGS "-DINCLUDE_DIRECTORIES:STRING=${CMAKE_BINARY_DIR};${CMAKE_CURRENT_SOURCE_DIR}/libarchive" ${CHECK_CRYPTO_WIN_ADD_LINKER_FLAGS} + CMAKE_FLAGS "-DINCLUDE_DIRECTORIES:STRING=${CMAKE_BINARY_DIR};${CMAKE_CURRENT_SOURCE_DIR}/libarchive" OUTPUT_VARIABLE OUTPUT) IF (ARCHIVE_CRYPTO_${CRYPTO}_WIN) @@ -755,20 +874,25 @@ ENDMACRO(CHECK_CRYPTO_WIN CRYPTO_LIST) MACRO(CHECK_ICONV LIB TRY_ICONV_CONST) IF(NOT HAVE_ICONV) CMAKE_PUSH_CHECK_STATE() # Save the state of the variables - IF ("CMAKE_C_COMPILER_ID" MATCHES "^GNU$") + IF (CMAKE_C_COMPILER_ID MATCHES "^GNU$" OR + CMAKE_C_COMPILER_ID MATCHES "^Clang$") # # During checking iconv proto type, we should use -Werror to avoid the # success of iconv detection with a warnig which success is a miss # detection. So this needs for all build mode(even it's a release mode). # SET(CMAKE_REQUIRED_FLAGS "${CMAKE_REQUIRED_FLAGS} -Werror") - ENDIF ("CMAKE_C_COMPILER_ID" MATCHES "^GNU$") + ENDIF (CMAKE_C_COMPILER_ID MATCHES "^GNU$" OR + CMAKE_C_COMPILER_ID MATCHES "^Clang$") + IF (CMAKE_C_COMPILER_ID MATCHES "^XL$") + SET(CMAKE_REQUIRED_FLAGS "${CMAKE_REQUIRED_FLAGS} -qhalt=w -qflag=w:w") + ENDIF (CMAKE_C_COMPILER_ID MATCHES "^XL$") IF (MSVC) # NOTE: /WX option is the same as gcc's -Werror option. SET(CMAKE_REQUIRED_FLAGS "${CMAKE_REQUIRED_FLAGS} /WX") ENDIF (MSVC) # - LIBARCHIVE_CHECK_C_SOURCE_COMPILES( + CHECK_C_SOURCE_COMPILES( "#include #include int main() { @@ -884,7 +1008,11 @@ ENDIF(ENABLE_ICONV) # # Find Libxml2 # -FIND_PACKAGE(LibXml2) +IF(ENABLE_LIBXML2) + FIND_PACKAGE(LibXml2) +ELSE() + SET(LIBXML2_FOUND FALSE) +ENDIF() IF(LIBXML2_FOUND) CMAKE_PUSH_CHECK_STATE() # Save the state of the variables INCLUDE_DIRECTORIES(${LIBXML2_INCLUDE_DIR}) @@ -909,7 +1037,11 @@ ELSE(LIBXML2_FOUND) # # Find Expat # - FIND_PACKAGE(EXPAT) + IF(ENABLE_EXPAT) + FIND_PACKAGE(EXPAT) + ELSE() + SET(EXPAT_FOUND FALSE) + ENDIF() IF(EXPAT_FOUND) CMAKE_PUSH_CHECK_STATE() # Save the state of the variables INCLUDE_DIRECTORIES(${EXPAT_INCLUDE_DIR}) @@ -979,8 +1111,16 @@ IF(NOT FOUND_POSIX_REGEX_LIB AND POSIX_R # # If requested, try finding library for PCREPOSIX # - FIND_PACKAGE(LibGCC) - FIND_PACKAGE(PCREPOSIX) + IF(ENABLE_LibGCC) + FIND_PACKAGE(LibGCC) + ELSE() + SET(LIBGCC_FOUND FALSE) # Override cached value + ENDIF() + IF(ENABLE_PCREPOSIX) + FIND_PACKAGE(PCREPOSIX) + ELSE() + SET(PCREPOSIX_FOUND FALSE) # Override cached value + ENDIF() IF(PCREPOSIX_FOUND) INCLUDE_DIRECTORIES(${PCRE_INCLUDE_DIR}) LIST(APPEND ADDITIONAL_LIBS ${PCREPOSIX_LIBRARIES}) @@ -1032,15 +1172,18 @@ ENDIF(NOT FOUND_POSIX_REGEX_LIB AND POSI # Check functions # CMAKE_PUSH_CHECK_STATE() # Save the state of the variables -IF ("CMAKE_C_COMPILER_ID" MATCHES "^GNU$") +IF (CMAKE_C_COMPILER_ID MATCHES "^GNU$" OR + CMAKE_C_COMPILER_ID MATCHES "^Clang$") # # During checking functions, we should use -fno-builtin to avoid the # failure of function detection which failure is an error "conflicting # types for built-in function" caused by using -Werror option. # SET(CMAKE_REQUIRED_FLAGS "${CMAKE_REQUIRED_FLAGS} -fno-builtin") -ENDIF ("CMAKE_C_COMPILER_ID" MATCHES "^GNU$") +ENDIF (CMAKE_C_COMPILER_ID MATCHES "^GNU$" OR + CMAKE_C_COMPILER_ID MATCHES "^Clang$") CHECK_SYMBOL_EXISTS(_CrtSetReportMode "crtdbg.h" HAVE__CrtSetReportMode) +CHECK_FUNCTION_EXISTS_GLIBC(arc4random_buf HAVE_ARC4RANDOM_BUF) CHECK_FUNCTION_EXISTS_GLIBC(chflags HAVE_CHFLAGS) CHECK_FUNCTION_EXISTS_GLIBC(chown HAVE_CHOWN) CHECK_FUNCTION_EXISTS_GLIBC(chroot HAVE_CHROOT) @@ -1088,6 +1231,7 @@ CHECK_FUNCTION_EXISTS_GLIBC(pipe HAVE_PI CHECK_FUNCTION_EXISTS_GLIBC(poll HAVE_POLL) CHECK_FUNCTION_EXISTS_GLIBC(posix_spawnp HAVE_POSIX_SPAWNP) CHECK_FUNCTION_EXISTS_GLIBC(readlink HAVE_READLINK) +CHECK_FUNCTION_EXISTS_GLIBC(readpassphrase HAVE_READPASSPHRASE) CHECK_FUNCTION_EXISTS_GLIBC(select HAVE_SELECT) CHECK_FUNCTION_EXISTS_GLIBC(setenv HAVE_SETENV) CHECK_FUNCTION_EXISTS_GLIBC(setlocale HAVE_SETLOCALE) @@ -1126,19 +1270,20 @@ CHECK_FUNCTION_EXISTS(strftime HAVE_STRF CHECK_FUNCTION_EXISTS(vprintf HAVE_VPRINTF) CHECK_FUNCTION_EXISTS(wmemcmp HAVE_WMEMCMP) CHECK_FUNCTION_EXISTS(wmemcpy HAVE_WMEMCPY) +CHECK_FUNCTION_EXISTS(wmemmove HAVE_WMEMMOVE) CMAKE_POP_CHECK_STATE() # Restore the state of the variables # Make sure we have the POSIX version of readdir_r, not the # older 2-argument version. -LIBARCHIVE_CHECK_C_SOURCE_COMPILES( +CHECK_C_SOURCE_COMPILES( "#include \nint main() {DIR *d = opendir(\".\"); struct dirent e,*r; return readdir_r(d,&e,&r);}" HAVE_READDIR_R) # Only detect readlinkat() if we also have AT_FDCWD in unistd.h. # NOTE: linux requires fcntl.h for AT_FDCWD. -LIBARCHIVE_CHECK_C_SOURCE_COMPILES( +CHECK_C_SOURCE_COMPILES( "#include \n#include \nint main() {char buf[10]; return readlinkat(AT_FDCWD, \"\", buf, 0);}" HAVE_READLINKAT) @@ -1147,10 +1292,10 @@ LIBARCHIVE_CHECK_C_SOURCE_COMPILES( # of interest and verify that the result can be linked. # CHECK_FUNCTION_EXISTS doesn't accept a header argument, # CHECK_SYMBOL_EXISTS doesn't test linkage. -LIBARCHIVE_CHECK_C_SOURCE_COMPILES( +CHECK_C_SOURCE_COMPILES( "#include \nint main() { return major(256); }" MAJOR_IN_MKDEV) -LIBARCHIVE_CHECK_C_SOURCE_COMPILES( +CHECK_C_SOURCE_COMPILES( "#include \nint main() { return major(256); }" MAJOR_IN_SYSMACROS) @@ -1171,10 +1316,15 @@ ENDIF(HAVE_INTTYPES_H) CHECK_SYMBOL_EXISTS(EFTYPE "errno.h" HAVE_EFTYPE) CHECK_SYMBOL_EXISTS(EILSEQ "errno.h" HAVE_EILSEQ) CHECK_SYMBOL_EXISTS(D_MD_ORDER "langinfo.h" HAVE_D_MD_ORDER) +CHECK_SYMBOL_EXISTS(INT32_MAX "${headers}" HAVE_DECL_INT32_MAX) +CHECK_SYMBOL_EXISTS(INT32_MIN "${headers}" HAVE_DECL_INT32_MIN) CHECK_SYMBOL_EXISTS(INT64_MAX "${headers}" HAVE_DECL_INT64_MAX) CHECK_SYMBOL_EXISTS(INT64_MIN "${headers}" HAVE_DECL_INT64_MIN) +CHECK_SYMBOL_EXISTS(INTMAX_MAX "${headers}" HAVE_DECL_INTMAX_MAX) +CHECK_SYMBOL_EXISTS(INTMAX_MIN "${headers}" HAVE_DECL_INTMAX_MIN) CHECK_SYMBOL_EXISTS(UINT32_MAX "${headers}" HAVE_DECL_UINT32_MAX) CHECK_SYMBOL_EXISTS(UINT64_MAX "${headers}" HAVE_DECL_UINT64_MAX) +CHECK_SYMBOL_EXISTS(UINTMAX_MAX "${headers}" HAVE_DECL_UINTMAX_MAX) CHECK_SYMBOL_EXISTS(SIZE_MAX "${headers}" HAVE_DECL_SIZE_MAX) CHECK_SYMBOL_EXISTS(SSIZE_MAX "limits.h" HAVE_DECL_SSIZE_MAX) @@ -1242,13 +1392,13 @@ CHECK_TYPE_SIZE("unsigned long long" SIZ CHECK_TYPE_SIZE("__int64" __INT64) CHECK_TYPE_SIZE("unsigned __int64" UNSIGNED___INT64) -CHECK_TYPE_SIZE(int16_t INT16_T) +CHECK_TYPE_SIZE(int16_t INT16_T) CHECK_TYPE_SIZE(int32_t INT32_T) CHECK_TYPE_SIZE(int64_t INT64_T) CHECK_TYPE_SIZE(intmax_t INTMAX_T) -CHECK_TYPE_SIZE(uint8_t UINT8_T) -CHECK_TYPE_SIZE(uint16_t UINT16_T) -CHECK_TYPE_SIZE(uint32_t UINT32_T) +CHECK_TYPE_SIZE(uint8_t UINT8_T) +CHECK_TYPE_SIZE(uint16_t UINT16_T) +CHECK_TYPE_SIZE(uint32_t UINT32_T) CHECK_TYPE_SIZE(uint64_t UINT64_T) CHECK_TYPE_SIZE(uintmax_t UINTMAX_T) @@ -1491,6 +1641,9 @@ CONFIGURE_FILE(${CMAKE_CURRENT_SOURCE_DI INCLUDE_DIRECTORIES(BEFORE ${CMAKE_CURRENT_BINARY_DIR}) ADD_DEFINITIONS(-DHAVE_CONFIG_H) +# Handle generation of the libarchive.pc file for pkg-config +INCLUDE(CreatePkgConfigFile) + # # Register installation of PDF documents. # @@ -1521,5 +1674,6 @@ IF(ENABLE_TEST) ENDIF(ENABLE_TEST) add_subdirectory(libarchive) +add_subdirectory(cat) add_subdirectory(tar) add_subdirectory(cpio) Added: vendor/libarchive/dist/CONTRIBUTING.md ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libarchive/dist/CONTRIBUTING.md Wed May 11 10:19:44 2016 (r299425) @@ -0,0 +1,98 @@ +Thank you for helping us improve libarchive. +The following guidelines will help ensure your contribution gets prompt attention. + +# Bugs and other Issues + +If you encounter any problems with libarchive, +[please file an issue on our issue tracker](https://github.com/libarchive/libarchive/issues). + +All bug reports should include the following information. You can copy the text below directly into the issue tracker to get started: + +``` +Basic Information + Version of libarchive: + How you obtained it: (build from source, pre-packaged binary, etc) + Operating system and version: + What compiler and/or IDE you are using (include version): + +If you are using a pre-packaged binary + Exact package name and version: + Repository you obtained it from: + +Description of the problem you are seeing: + What did you do? + What did you expect to happen? + What actually happened? + What log files or error messages were produced? + +How the libarchive developers can reproduce your problem: + What other software was involved? + What other files were involved? + How can we obtain any of the above? +``` + +Depending on the specific type of issue, other information will be helpful: + +## Test Failures + +If you see any test failures, please include the information above and also: + +* Names of the tests that failed. + +* Look for the .log files in the /tmp/libarchive_test_*date-and-time* directories. (On Mac OS, look in $TMPDIR which is different than /tmp.) + +Please paste the .log files you will find there directly into your report. + + +## Problems using libarchive in a program + +If you are trying to write a program using libarchive, please include the information above and also: + +* It will help us if we can actually run the program. This is easiest if you can provide source to a short program that illustrates your problem. + +* If you have a sufficiently short program that shows the problem, you can either paste it into the report or [put it into a gist](https://gist.github.com). + + +## Libarchive produced incorrect output + +Please tell us what program you ran, any command-line arguments you provided, and details of the input files (`ls -l` output is helpful here). If the problem involved a command-line program, please copy the full terminal text into the report, including the command line and any error messages. + +Please try to make the output file available to us. Unless it is very large, you can upload it into a fresh github repository and provide a link in your issue report. + + +## Libarchive could not read a particular input file + +Note: If you can provide a **very small** input file that reproduces the problem, we can add that to our test suite. This will ensure that the bug does not reappear in the future. + +A link to the relevant file is usually sufficient. + +If you cannot provide the input file or a link to the file, please let us know if there is some other way to obtain it. + + +## Documentation improvements + +We are always interested in improving the libarchive documentation. Please tell us about any errors you find, including: + +* Typos or errors in the manpages provided with libarchive source. + +* Mistakes in the [libarchive Wiki](https://github.com/libarchive/libarchive/wiki) + +* Problems with the PDF or Wiki files that are automatically generated from the manpages. + + +# Code Submissions + +We welcome all code submissions. But of course, some code submissions are easier for us to respond to than others. The best code submissions: + +* Address a single issue. There have been many cases where a simple fix to an obvious problem did not get handled for months because the patch that was provided also included an unrelated change affecting an especially complex area of the code. + +* Follow existing libarchive code style and conventions. Libarchive generally follows [BSD KNF](https://www.freebsd.org/cgi/man.cgi?query=style&sektion=9) for formatting code. + +* Do not make unnecessary changes to existing whitespace, capitalization, or spelling. + +* Include detailed instructions for reproducing the problem you're fixing. We do try to verify that a submission actually fixes a real problem. If we can't reproduce the problem, it will take us longer to evaluate the fix. For this reason, we encourage you to file an issue report first with details on reproducing the problem, then refer to that issue in your pull request. + +* Includes a test case. The libarchive Wiki has [detailed documentation for adding new test cases](https://github.com/libarchive/libarchive/wiki/LibarchiveAddingTest). + +* Are provided via Github pull requests. We welcome patches in almost any format, but github's pull request management makes it significantly easier for us to evaluate and test changes. + Modified: vendor/libarchive/dist/COPYING ============================================================================== --- vendor/libarchive/dist/COPYING Wed May 11 10:18:56 2016 (r299424) +++ vendor/libarchive/dist/COPYING Wed May 11 10:19:44 2016 (r299425) @@ -17,12 +17,11 @@ the actual statements in the files are c files for details: libarchive/archive_entry.c libarchive/archive_read_support_filter_compress.c - libarchive/archive_write_set_filter_compress.c + libarchive/archive_write_add_filter_compress.c libarchive/mtree.5 - tar/matching.c * The following source files are in the public domain: - tar/getdate.c + libarchive/archive_getdate.c * The build files---including Makefiles, configure scripts, and auxiliary scripts used as part of the compile process---have Modified: vendor/libarchive/dist/INSTALL ============================================================================== --- vendor/libarchive/dist/INSTALL Wed May 11 10:18:56 2016 (r299424) +++ vendor/libarchive/dist/INSTALL Wed May 11 10:19:44 2016 (r299425) @@ -1,5 +1,5 @@ More complete build documentation is available on the libarchive -Wiki: http://libarchive.googlecode.com/ +Wiki: https://github.com/libarchive/libarchive/wiki On most Unix-like systems, you should be able to install libarchive, bsdtar, and bsdcpio using the following common steps: Modified: vendor/libarchive/dist/Makefile.am ============================================================================== --- vendor/libarchive/dist/Makefile.am Wed May 11 10:18:56 2016 (r299424) +++ vendor/libarchive/dist/Makefile.am Wed May 11 10:19:44 2016 (r299425) @@ -8,24 +8,23 @@ ACLOCAL_AMFLAGS = -I build/autoconf # lib_LTLIBRARIES= libarchive.la noinst_LTLIBRARIES= libarchive_fe.la -bin_PROGRAMS= $(bsdtar_programs) $(bsdcpio_programs) -man_MANS= $(libarchive_man_MANS) $(bsdtar_man_MANS) $(bsdcpio_man_MANS) -BUILT_SOURCES= libarchive/test/list.h tar/test/list.h cpio/test/list.h +bin_PROGRAMS= $(bsdtar_programs) $(bsdcpio_programs) $(bsdcat_programs) +man_MANS= $(libarchive_man_MANS) $(bsdtar_man_MANS) $(bsdcpio_man_MANS) $(bsdcat_man_MANS) +BUILT_SOURCES= libarchive/test/list.h tar/test/list.h cpio/test/list.h cat/test/list.h # # What to test: We always test libarchive, test bsdtar and bsdcpio only # if we built them. # -check_PROGRAMS= libarchive_test $(bsdtar_test_programs) $(bsdcpio_test_programs) -TESTS= libarchive_test $(bsdtar_test_programs) $(bsdcpio_test_programs) -TESTS_ENVIRONMENT= $(libarchive_TESTS_ENVIRONMENT) $(bsdtar_TESTS_ENVIRONMENT) $(bsdcpio_TESTS_ENVIRONMENT) +check_PROGRAMS= libarchive_test $(bsdtar_test_programs) $(bsdcpio_test_programs) $(bsdcat_test_programs) +TESTS= libarchive_test $(bsdtar_test_programs) $(bsdcpio_test_programs) $(bsdcat_test_programs) +TESTS_ENVIRONMENT= $(libarchive_TESTS_ENVIRONMENT) $(bsdtar_TESTS_ENVIRONMENT) $(bsdcpio_TESTS_ENVIRONMENT) $(bsdcat_TESTS_ENVIRONMENT) # Always build and test both bsdtar and bsdcpio as part of 'distcheck' DISTCHECK_CONFIGURE_FLAGS = --enable-bsdtar --enable-bsdcpio -COMMON_CFLAGS=-Wall -Wformat -Wformat-security # The next line is commented out by default in shipping libarchive releases. # It is uncommented by default in trunk. -# DEV_CFLAGS=-Werror -Wextra -Wunused -Wshadow -Wmissing-prototypes -Wcast-qual -AM_CFLAGS=$(COMMON_CFLAGS) $(DEV_CFLAGS) +DEV_CFLAGS=-Werror -Wextra -Wunused -Wshadow -Wmissing-prototypes -Wcast-qual -g +AM_CFLAGS=$(DEV_CFLAGS) PLATFORMCPPFLAGS = @PLATFORMCPPFLAGS@ AM_CPPFLAGS=$(PLATFORMCPPFLAGS) @@ -33,21 +32,23 @@ AM_CPPFLAGS=$(PLATFORMCPPFLAGS) # What to include in the distribution # EXTRA_DIST= \ - CMakeLists.txt \ - build/autogen.sh \ - build/bump-version.sh \ - build/clean.sh \ - build/cmake \ - build/version \ - contrib \ - doc \ - examples \ - $(libarchive_EXTRA_DIST) \ - $(libarchive_test_EXTRA_DIST) \ - $(bsdtar_EXTRA_DIST) \ - $(bsdtar_test_EXTRA_DIST) \ - $(bsdcpio_EXTRA_DIST) \ - $(bsdcpio_test_EXTRA_DIST) + CMakeLists.txt \ + build/autogen.sh \ + build/bump-version.sh \ + build/clean.sh \ + build/cmake \ + build/version \ + contrib \ + doc \ + examples \ + $(libarchive_EXTRA_DIST) \ + $(libarchive_test_EXTRA_DIST) \ + $(bsdtar_EXTRA_DIST) \ + $(bsdtar_test_EXTRA_DIST) \ + $(bsdcpio_EXTRA_DIST) \ + $(bsdcpio_test_EXTRA_DIST) \ + $(bsdcat_EXTRA_DIST) \ + $(bsdcat_test_EXTRA_DIST) # a) Clean out some unneeded files and directories # b) Collect all documentation and format it for distribution. @@ -63,10 +64,11 @@ dist-hook: # # Extra rules for cleanup # -DISTCLEANFILES= \ - libarchive/test/list.h \ - tar/test/list.h \ - cpio/test/list.h +DISTCLEANFILES= \ + libarchive/test/list.h \ + tar/test/list.h \ + cpio/test/list.h \ + cat/test/list.h distclean-local: -rm -rf .ref @@ -78,6 +80,8 @@ distclean-local: -[ -f tar/test/Makefile ] && cd tar/test && make clean -[ -f cpio/Makefile ] && cd cpio && make clean -[ -f cpio/test/Makefile ] && cd cpio/test && make clean + -[ -f cat/Makefile ] && cd cat && make clean + -[ -f cpio/test/Makefile ] && cd cat/test && make clean # # Libarchive headers, source, etc. @@ -86,137 +90,157 @@ distclean-local: include_HEADERS= libarchive/archive.h libarchive/archive_entry.h -libarchive_la_SOURCES= \ - libarchive/archive_acl.c \ - libarchive/archive_acl_private.h \ - libarchive/archive_check_magic.c \ - libarchive/archive_cmdline.c \ - libarchive/archive_cmdline_private.h \ - libarchive/archive_crc32.h \ - libarchive/archive_crypto.c \ - libarchive/archive_crypto_private.h \ - libarchive/archive_endian.h \ - libarchive/archive_entry.c \ - libarchive/archive_entry.h \ - libarchive/archive_entry_copy_stat.c \ - libarchive/archive_entry_link_resolver.c \ - libarchive/archive_entry_locale.h \ - libarchive/archive_entry_private.h \ - libarchive/archive_entry_sparse.c \ - libarchive/archive_entry_stat.c \ - libarchive/archive_entry_strmode.c \ - libarchive/archive_entry_xattr.c \ - libarchive/archive_getdate.c \ - libarchive/archive_match.c \ - libarchive/archive_options.c \ - libarchive/archive_options_private.h \ - libarchive/archive_pathmatch.c \ - libarchive/archive_pathmatch.h \ - libarchive/archive_platform.h \ - libarchive/archive_ppmd_private.h \ - libarchive/archive_ppmd7.c \ - libarchive/archive_ppmd7_private.h \ - libarchive/archive_private.h \ - libarchive/archive_rb.c \ - libarchive/archive_rb.h \ - libarchive/archive_read.c \ - libarchive/archive_read_append_filter.c \ - libarchive/archive_read_data_into_fd.c \ - libarchive/archive_read_disk_entry_from_file.c \ - libarchive/archive_read_disk_posix.c \ - libarchive/archive_read_disk_private.h \ - libarchive/archive_read_disk_set_standard_lookup.c \ - libarchive/archive_read_extract.c \ - libarchive/archive_read_open_fd.c \ - libarchive/archive_read_open_file.c \ - libarchive/archive_read_open_filename.c \ - libarchive/archive_read_open_memory.c \ - libarchive/archive_read_private.h \ - libarchive/archive_read_set_format.c \ - libarchive/archive_read_set_options.c \ - libarchive/archive_read_support_filter_all.c \ - libarchive/archive_read_support_filter_bzip2.c \ - libarchive/archive_read_support_filter_compress.c \ - libarchive/archive_read_support_filter_grzip.c \ - libarchive/archive_read_support_filter_gzip.c \ - libarchive/archive_read_support_filter_lrzip.c \ - libarchive/archive_read_support_filter_lzop.c \ - libarchive/archive_read_support_filter_none.c \ - libarchive/archive_read_support_filter_program.c \ - libarchive/archive_read_support_filter_rpm.c \ - libarchive/archive_read_support_filter_uu.c \ - libarchive/archive_read_support_filter_xz.c \ - libarchive/archive_read_support_format_7zip.c \ - libarchive/archive_read_support_format_all.c \ - libarchive/archive_read_support_format_ar.c \ - libarchive/archive_read_support_format_by_code.c \ - libarchive/archive_read_support_format_cab.c \ *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Wed May 11 12:36:20 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 431E9B360CD; Wed, 11 May 2016 12:36:20 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 09F7F1254; Wed, 11 May 2016 12:36:19 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4BCaJ1j054680; Wed, 11 May 2016 12:36:19 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4BCaJ5N054679; Wed, 11 May 2016 12:36:19 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201605111236.u4BCaJ5N054679@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 11 May 2016 12:36:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299434 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 12:36:20 -0000 Author: mav Date: Wed May 11 12:36:19 2016 New Revision: 299434 URL: https://svnweb.freebsd.org/changeset/base/299434 Log: 6841 Undirty freed spill blocks Reviewed by: Brian Behlendorf Reviewed by: Dan McDonald Reviewed by: Matthew Ahrens Approved by: Robert Mustacchi Author: Tim Chase openzfs/openzfs@445e67805dd2ca6c3a2363b2ec9e163c62370233 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Wed May 11 11:35:37 2016 (r299433) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Wed May 11 12:36:19 2016 (r299434) @@ -908,8 +908,10 @@ dbuf_free_range(dnode_t *dn, uint64_t st dmu_buf_impl_t *db, *db_next; uint64_t txg = tx->tx_txg; avl_index_t where; + boolean_t freespill = + (start_blkid == DMU_SPILL_BLKID || end_blkid == DMU_SPILL_BLKID); - if (end_blkid > dn->dn_maxblkid && (end_blkid != DMU_SPILL_BLKID)) + if (end_blkid > dn->dn_maxblkid && !freespill) end_blkid = dn->dn_maxblkid; dprintf_dnode(dn, "start=%llu end=%llu\n", start_blkid, end_blkid); @@ -918,7 +920,7 @@ dbuf_free_range(dnode_t *dn, uint64_t st db_search.db_state = DB_SEARCH; mutex_enter(&dn->dn_dbufs_mtx); - if (start_blkid >= dn->dn_unlisted_l0_blkid) { + if (start_blkid >= dn->dn_unlisted_l0_blkid && !freespill) { /* There can't be any dbufs in this range; no need to search. */ #ifdef DEBUG db = avl_find(&dn->dn_dbufs, &db_search, &where); From owner-svn-src-vendor@freebsd.org Wed May 11 12:39:54 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F53CB361A9; Wed, 11 May 2016 12:39:54 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 716C816DD; Wed, 11 May 2016 12:39:54 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4BCdrgW054889; Wed, 11 May 2016 12:39:53 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4BCdrb5054888; Wed, 11 May 2016 12:39:53 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201605111239.u4BCdrb5054888@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 11 May 2016 12:39:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299436 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 12:39:54 -0000 Author: mav Date: Wed May 11 12:39:53 2016 New Revision: 299436 URL: https://svnweb.freebsd.org/changeset/base/299436 Log: 6843 Make xattr dir truncate and remove in one tx Reviewed by: Brian Behlendorf Reviewed by: Dan McDonald Reviewed by: Matthew Ahrens Approved by: Robert Mustacchi Author: Chunwei Chen openzfs/openzfs@399cc7d5d9aff97c714b708af3e3f0280ceab93f Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_dir.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_dir.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_dir.c Wed May 11 12:38:07 2016 (r299435) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_dir.c Wed May 11 12:39:53 2016 (r299436) @@ -606,19 +606,25 @@ zfs_rmnode(znode_t *zp) zfs_znode_free(zp); return; } - } - - /* - * Free up all the data in the file. - */ - error = dmu_free_long_range(os, zp->z_id, 0, DMU_OBJECT_END); - if (error) { + } else { /* - * Not enough space. Leave the file in the unlinked set. + * Free up all the data in the file. We don't do this for + * XATTR directories because we need truncate and remove to be + * in the same tx, like in zfs_znode_delete(). Otherwise, if + * we crash here we'll end up with an inconsistent truncated + * zap object in the delete queue. Note a truncated file is + * harmless since it only contains user data. */ - zfs_znode_dmu_fini(zp); - zfs_znode_free(zp); - return; + error = dmu_free_long_range(os, zp->z_id, 0, DMU_OBJECT_END); + if (error) { + /* + * Not enough space. Leave the file in the unlinked + * set. + */ + zfs_znode_dmu_fini(zp); + zfs_znode_free(zp); + return; + } } /* From owner-svn-src-vendor@freebsd.org Wed May 11 12:45:22 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 709B3B36397; Wed, 11 May 2016 12:45:22 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4A6321CC6; Wed, 11 May 2016 12:45:22 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4BCjL8g057763; Wed, 11 May 2016 12:45:21 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4BCjLtO057761; Wed, 11 May 2016 12:45:21 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201605111245.u4BCjLtO057761@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 11 May 2016 12:45:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299438 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 12:45:22 -0000 Author: mav Date: Wed May 11 12:45:21 2016 New Revision: 299438 URL: https://svnweb.freebsd.org/changeset/base/299438 Log: 6842 Fix empty xattr dir causing lockup Reviewed by: Brian Behlendorf Reviewed by: Dan McDonald Reviewed by: Matthew Ahrens Approved by: Robert Mustacchi Author: Chunwei Chen openzfs/openzfs@02525cd08fb3730fff3a69cb5376443d481f7839 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zap.c vendor-sys/illumos/dist/uts/common/fs/zfs/zap_micro.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zap.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zap.c Wed May 11 12:43:54 2016 (r299437) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zap.c Wed May 11 12:45:21 2016 (r299438) @@ -575,7 +575,14 @@ zap_deref_leaf(zap_t *zap, uint64_t h, d ASSERT(zap->zap_dbuf == NULL || zap_f_phys(zap) == zap->zap_dbuf->db_data); - ASSERT3U(zap_f_phys(zap)->zap_magic, ==, ZAP_MAGIC); + + /* Reality check for corrupt zap objects (leaf or header). */ + if ((zap_f_phys(zap)->zap_block_type != ZBT_LEAF && + zap_f_phys(zap)->zap_block_type != ZBT_HEADER) || + zap_f_phys(zap)->zap_magic != ZAP_MAGIC) { + return (SET_ERROR(EIO)); + } + idx = ZAP_HASH_IDX(h, zap_f_phys(zap)->zap_ptrtbl.zt_shift); err = zap_idx_to_blk(zap, idx, &blk); if (err != 0) Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zap_micro.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zap_micro.c Wed May 11 12:43:54 2016 (r299437) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zap_micro.c Wed May 11 12:45:21 2016 (r299438) @@ -367,6 +367,9 @@ mzap_open(objset_t *os, uint64_t obj, dm zap_t *winner; zap_t *zap; int i; + uint64_t *zap_hdr = (uint64_t *)db->db_data; + uint64_t zap_block_type = zap_hdr[0]; + uint64_t zap_magic = zap_hdr[1]; ASSERT3U(MZAP_ENT_LEN, ==, sizeof (mzap_ent_phys_t)); @@ -377,9 +380,13 @@ mzap_open(objset_t *os, uint64_t obj, dm zap->zap_object = obj; zap->zap_dbuf = db; - if (*(uint64_t *)db->db_data != ZBT_MICRO) { + if (zap_block_type != ZBT_MICRO) { mutex_init(&zap->zap_f.zap_num_entries_mtx, 0, 0, 0); zap->zap_f.zap_block_shift = highbit64(db->db_size) - 1; + if (zap_block_type != ZBT_HEADER || zap_magic != ZAP_MAGIC) { + winner = NULL; /* No actual winner here... */ + goto handle_winner; + } } else { zap->zap_ismicro = TRUE; } @@ -392,14 +399,8 @@ mzap_open(objset_t *os, uint64_t obj, dm dmu_buf_init_user(&zap->zap_dbu, zap_evict, &zap->zap_dbuf); winner = dmu_buf_set_user(db, &zap->zap_dbu); - if (winner != NULL) { - rw_exit(&zap->zap_rwlock); - rw_destroy(&zap->zap_rwlock); - if (!zap->zap_ismicro) - mutex_destroy(&zap->zap_f.zap_num_entries_mtx); - kmem_free(zap, sizeof (zap_t)); - return (winner); - } + if (winner != NULL) + goto handle_winner; if (zap->zap_ismicro) { zap->zap_salt = zap_m_phys(zap)->mz_salt; @@ -446,6 +447,14 @@ mzap_open(objset_t *os, uint64_t obj, dm } rw_exit(&zap->zap_rwlock); return (zap); + +handle_winner: + rw_exit(&zap->zap_rwlock); + rw_destroy(&zap->zap_rwlock); + if (!zap->zap_ismicro) + mutex_destroy(&zap->zap_f.zap_num_entries_mtx); + kmem_free(zap, sizeof (zap_t)); + return (winner); } int @@ -472,8 +481,17 @@ zap_lockdir(objset_t *os, uint64_t obj, #endif zap = dmu_buf_get_user(db); - if (zap == NULL) + if (zap == NULL) { zap = mzap_open(os, obj, db); + if (zap == NULL) { + /* + * mzap_open() didn't like what it saw on-disk. + * Check for corruption! + */ + dmu_buf_rele(db, NULL); + return (SET_ERROR(EIO)); + } + } /* * We're checking zap_ismicro without the lock held, in order to From owner-svn-src-vendor@freebsd.org Wed May 11 12:51:00 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1EDBB36495; Wed, 11 May 2016 12:51:00 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6C44A105E; Wed, 11 May 2016 12:51:00 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4BCoxle059145; Wed, 11 May 2016 12:50:59 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4BCow6I058056; Wed, 11 May 2016 12:50:58 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201605111250.u4BCow6I058056@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 11 May 2016 12:50:58 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299440 - in vendor-sys/illumos/dist/uts/common: fs/zfs fs/zfs/sys sys/fs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 12:51:00 -0000 Author: mav Date: Wed May 11 12:50:58 2016 New Revision: 299440 URL: https://svnweb.freebsd.org/changeset/base/299440 Log: 6736 ZFS per-vdev ZAPs Reviewed by: Matthew Ahrens Reviewed by: John Kennedy Reviewed by: George Wilson Reviewed by: Don Brady Reviewed by: Dan McDonald Approved by: Richard Lowe Author: Joe Stein openzfs/openzfs@215198a6ad15cf4832370e2f19247abeb36b951a Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_config.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c vendor-sys/illumos/dist/uts/common/fs/zfs/zap.c vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Wed May 11 12:46:07 2016 (r299439) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Wed May 11 12:50:58 2016 (r299440) @@ -1610,6 +1610,19 @@ spa_check_removed(vdev_t *vd) } } +static void +spa_config_valid_zaps(vdev_t *vd, vdev_t *mvd) +{ + ASSERT3U(vd->vdev_children, ==, mvd->vdev_children); + + vd->vdev_top_zap = mvd->vdev_top_zap; + vd->vdev_leaf_zap = mvd->vdev_leaf_zap; + + for (uint64_t i = 0; i < vd->vdev_children; i++) { + spa_config_valid_zaps(vd->vdev_child[i], mvd->vdev_child[i]); + } +} + /* * Validate the current config against the MOS config */ @@ -1713,16 +1726,25 @@ spa_config_valid(spa_t *spa, nvlist_t *c spa_config_enter(spa, SCL_ALL, FTAG, RW_WRITER); vdev_reopen(rvd); - } else if (mtvd->vdev_islog) { + } else { + if (mtvd->vdev_islog) { + /* + * Load the slog device's state from the MOS + * config since it's possible that the label + * does not contain the most up-to-date + * information. + */ + vdev_load_log_state(tvd, mtvd); + vdev_reopen(tvd); + } + /* - * Load the slog device's state from the MOS config - * since it's possible that the label does not - * contain the most up-to-date information. + * Per-vdev ZAP info is stored exclusively in the MOS. */ - vdev_load_log_state(tvd, mtvd); - vdev_reopen(tvd); + spa_config_valid_zaps(tvd, mtvd); } } + vdev_free(mrvd); spa_config_exit(spa, SCL_ALL, FTAG); @@ -2140,6 +2162,34 @@ spa_load(spa_t *spa, spa_load_state_t st } /* + * Count the number of per-vdev ZAPs associated with all of the vdevs in the + * vdev tree rooted in the given vd, and ensure that each ZAP is present in the + * spa's per-vdev ZAP list. + */ +static uint64_t +vdev_count_verify_zaps(vdev_t *vd) +{ + spa_t *spa = vd->vdev_spa; + uint64_t total = 0; + if (vd->vdev_top_zap != 0) { + total++; + ASSERT0(zap_lookup_int(spa->spa_meta_objset, + spa->spa_all_vdev_zaps, vd->vdev_top_zap)); + } + if (vd->vdev_leaf_zap != 0) { + total++; + ASSERT0(zap_lookup_int(spa->spa_meta_objset, + spa->spa_all_vdev_zaps, vd->vdev_leaf_zap)); + } + + for (uint64_t i = 0; i < vd->vdev_children; i++) { + total += vdev_count_verify_zaps(vd->vdev_child[i]); + } + + return (total); +} + +/* * Load an existing storage pool, using the pool's builtin spa_config as a * source of configuration information. */ @@ -2568,6 +2618,39 @@ spa_load_impl(spa_t *spa, uint64_t pool_ return (spa_vdev_err(rvd, VDEV_AUX_CORRUPT_DATA, EIO)); /* + * Load the per-vdev ZAP map. If we have an older pool, this will not + * be present; in this case, defer its creation to a later time to + * avoid dirtying the MOS this early / out of sync context. See + * spa_sync_config_object. + */ + + /* The sentinel is only available in the MOS config. */ + nvlist_t *mos_config; + if (load_nvlist(spa, spa->spa_config_object, &mos_config) != 0) + return (spa_vdev_err(rvd, VDEV_AUX_CORRUPT_DATA, EIO)); + + error = spa_dir_prop(spa, DMU_POOL_VDEV_ZAP_MAP, + &spa->spa_all_vdev_zaps); + + if (error != ENOENT && error != 0) { + return (spa_vdev_err(rvd, VDEV_AUX_CORRUPT_DATA, EIO)); + } else if (error == 0 && !nvlist_exists(mos_config, + ZPOOL_CONFIG_HAS_PER_VDEV_ZAPS)) { + /* + * An older version of ZFS overwrote the sentinel value, so + * we have orphaned per-vdev ZAPs in the MOS. Defer their + * destruction to later; see spa_sync_config_object. + */ + spa->spa_avz_action = AVZ_ACTION_DESTROY; + /* + * We're assuming that no vdevs have had their ZAPs created + * before this. Better be sure of it. + */ + ASSERT0(vdev_count_verify_zaps(spa->spa_root_vdev)); + } + nvlist_free(mos_config); + + /* * If we're assembling the pool from the split-off vdevs of * an existing pool, we don't want to attach the spares & cache * devices. @@ -5039,6 +5122,16 @@ spa_vdev_split_mirror(spa_t *spa, char * vml[c]->vdev_top->vdev_asize) == 0); VERIFY(nvlist_add_uint64(child[c], ZPOOL_CONFIG_ASHIFT, vml[c]->vdev_top->vdev_ashift) == 0); + + /* transfer per-vdev ZAPs */ + ASSERT3U(vml[c]->vdev_leaf_zap, !=, 0); + VERIFY0(nvlist_add_uint64(child[c], + ZPOOL_CONFIG_VDEV_LEAF_ZAP, vml[c]->vdev_leaf_zap)); + + ASSERT3U(vml[c]->vdev_top->vdev_top_zap, !=, 0); + VERIFY0(nvlist_add_uint64(child[c], + ZPOOL_CONFIG_VDEV_TOP_ZAP, + vml[c]->vdev_parent->vdev_top_zap)); } if (error != 0) { @@ -5080,11 +5173,13 @@ spa_vdev_split_mirror(spa_t *spa, char * spa->spa_config_txg) == 0); VERIFY(nvlist_add_uint64(config, ZPOOL_CONFIG_POOL_GUID, spa_generate_guid(NULL)) == 0); + VERIFY0(nvlist_add_boolean(config, ZPOOL_CONFIG_HAS_PER_VDEV_ZAPS)); (void) nvlist_lookup_string(props, zpool_prop_to_name(ZPOOL_PROP_ALTROOT), &altroot); /* add the new pool to the namespace */ newspa = spa_add(newname, config, altroot); + newspa->spa_avz_action = AVZ_ACTION_REBUILD; newspa->spa_config_txg = spa->spa_config_txg; spa_set_log_state(newspa, SPA_LOG_CLEAR); @@ -5142,9 +5237,11 @@ spa_vdev_split_mirror(spa_t *spa, char * if (error == 0) spa_history_log_internal(spa, "detach", tx, "vdev=%s", vml[c]->vdev_path); + vdev_free(vml[c]); } } + spa->spa_avz_action = AVZ_ACTION_REBUILD; vdev_config_dirty(spa->spa_root_vdev); spa->spa_config_splitting = NULL; nvlist_free(nvl); @@ -5983,16 +6080,118 @@ spa_sync_aux_dev(spa_t *spa, spa_aux_vde sav->sav_sync = B_FALSE; } +/* + * Rebuild spa's all-vdev ZAP from the vdev ZAPs indicated in each vdev_t. + * The all-vdev ZAP must be empty. + */ +static void +spa_avz_build(vdev_t *vd, uint64_t avz, dmu_tx_t *tx) +{ + spa_t *spa = vd->vdev_spa; + if (vd->vdev_top_zap != 0) { + VERIFY0(zap_add_int(spa->spa_meta_objset, avz, + vd->vdev_top_zap, tx)); + } + if (vd->vdev_leaf_zap != 0) { + VERIFY0(zap_add_int(spa->spa_meta_objset, avz, + vd->vdev_leaf_zap, tx)); + } + for (uint64_t i = 0; i < vd->vdev_children; i++) { + spa_avz_build(vd->vdev_child[i], avz, tx); + } +} + static void spa_sync_config_object(spa_t *spa, dmu_tx_t *tx) { nvlist_t *config; - if (list_is_empty(&spa->spa_config_dirty_list)) + /* + * If the pool is being imported from a pre-per-vdev-ZAP version of ZFS, + * its config may not be dirty but we still need to build per-vdev ZAPs. + * Similarly, if the pool is being assembled (e.g. after a split), we + * need to rebuild the AVZ although the config may not be dirty. + */ + if (list_is_empty(&spa->spa_config_dirty_list) && + spa->spa_avz_action == AVZ_ACTION_NONE) return; spa_config_enter(spa, SCL_STATE, FTAG, RW_READER); + ASSERT(spa->spa_avz_action == AVZ_ACTION_NONE || + spa->spa_all_vdev_zaps != 0); + + if (spa->spa_avz_action == AVZ_ACTION_REBUILD) { + /* Make and build the new AVZ */ + uint64_t new_avz = zap_create(spa->spa_meta_objset, + DMU_OTN_ZAP_METADATA, DMU_OT_NONE, 0, tx); + spa_avz_build(spa->spa_root_vdev, new_avz, tx); + + /* Diff old AVZ with new one */ + zap_cursor_t zc; + zap_attribute_t za; + + for (zap_cursor_init(&zc, spa->spa_meta_objset, + spa->spa_all_vdev_zaps); + zap_cursor_retrieve(&zc, &za) == 0; + zap_cursor_advance(&zc)) { + uint64_t vdzap = za.za_first_integer; + if (zap_lookup_int(spa->spa_meta_objset, new_avz, + vdzap) == ENOENT) { + /* + * ZAP is listed in old AVZ but not in new one; + * destroy it + */ + VERIFY0(zap_destroy(spa->spa_meta_objset, vdzap, + tx)); + } + } + + zap_cursor_fini(&zc); + + /* Destroy the old AVZ */ + VERIFY0(zap_destroy(spa->spa_meta_objset, + spa->spa_all_vdev_zaps, tx)); + + /* Replace the old AVZ in the dir obj with the new one */ + VERIFY0(zap_update(spa->spa_meta_objset, + DMU_POOL_DIRECTORY_OBJECT, DMU_POOL_VDEV_ZAP_MAP, + sizeof (new_avz), 1, &new_avz, tx)); + + spa->spa_all_vdev_zaps = new_avz; + } else if (spa->spa_avz_action == AVZ_ACTION_DESTROY) { + zap_cursor_t zc; + zap_attribute_t za; + + /* Walk through the AVZ and destroy all listed ZAPs */ + for (zap_cursor_init(&zc, spa->spa_meta_objset, + spa->spa_all_vdev_zaps); + zap_cursor_retrieve(&zc, &za) == 0; + zap_cursor_advance(&zc)) { + uint64_t zap = za.za_first_integer; + VERIFY0(zap_destroy(spa->spa_meta_objset, zap, tx)); + } + + zap_cursor_fini(&zc); + + /* Destroy and unlink the AVZ itself */ + VERIFY0(zap_destroy(spa->spa_meta_objset, + spa->spa_all_vdev_zaps, tx)); + VERIFY0(zap_remove(spa->spa_meta_objset, + DMU_POOL_DIRECTORY_OBJECT, DMU_POOL_VDEV_ZAP_MAP, tx)); + spa->spa_all_vdev_zaps = 0; + } + + if (spa->spa_all_vdev_zaps == 0) { + spa->spa_all_vdev_zaps = zap_create_link(spa->spa_meta_objset, + DMU_OTN_ZAP_METADATA, DMU_POOL_DIRECTORY_OBJECT, + DMU_POOL_VDEV_ZAP_MAP, tx); + } + spa->spa_avz_action = AVZ_ACTION_NONE; + + /* Create ZAPs for vdevs that don't have them. */ + vdev_construct_zaps(spa->spa_root_vdev, tx); + config = spa_config_generate(spa, spa->spa_root_vdev, dmu_tx_get_txg(tx), B_FALSE); @@ -6393,6 +6592,21 @@ spa_sync(spa_t *spa, uint64_t txg) } while (dmu_objset_is_dirty(mos, txg)); + if (!list_is_empty(&spa->spa_config_dirty_list)) { + /* + * Make sure that the number of ZAPs for all the vdevs matches + * the number of ZAPs in the per-vdev ZAP list. This only gets + * called if the config is dirty; otherwise there may be + * outstanding AVZ operations that weren't completed in + * spa_sync_config_object. + */ + uint64_t all_vdev_zap_entry_count; + ASSERT0(zap_count(spa->spa_meta_objset, + spa->spa_all_vdev_zaps, &all_vdev_zap_entry_count)); + ASSERT3U(vdev_count_verify_zaps(spa->spa_root_vdev), ==, + all_vdev_zap_entry_count); + } + /* * Rewrite the vdev configuration (which includes the uberblock) * to commit the transaction group. Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_config.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa_config.c Wed May 11 12:46:07 2016 (r299439) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa_config.c Wed May 11 12:50:58 2016 (r299440) @@ -22,7 +22,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright 2011 Nexenta Systems, Inc. All rights reserved. - * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright (c) 2011, 2015 by Delphix. All rights reserved. */ #include @@ -124,7 +124,7 @@ spa_config_load(void) if (nvpair_type(nvpair) != DATA_TYPE_NVLIST) continue; - VERIFY(nvpair_value_nvlist(nvpair, &child) == 0); + child = fnvpair_value_nvlist(nvpair); if (spa_lookup(nvpair_name(nvpair)) != NULL) continue; @@ -162,14 +162,9 @@ spa_config_write(spa_config_dirent_t *dp /* * Pack the configuration into a buffer. */ - VERIFY(nvlist_size(nvl, &buflen, NV_ENCODE_XDR) == 0); - - buf = kmem_alloc(buflen, KM_SLEEP); + buf = fnvlist_pack(nvl, &buflen); temp = kmem_zalloc(MAXPATHLEN, KM_SLEEP); - VERIFY(nvlist_pack(nvl, &buf, &buflen, NV_ENCODE_XDR, - KM_SLEEP) == 0); - /* * Write the configuration to disk. We need to do the traditional * 'write to temporary file, sync, move over original' to make sure we @@ -191,7 +186,7 @@ spa_config_write(spa_config_dirent_t *dp (void) vn_remove(temp, UIO_SYSSPACE, RMFILE); - kmem_free(buf, buflen); + fnvlist_pack_free(buf, buflen); kmem_free(temp, MAXPATHLEN); return (err); } @@ -256,11 +251,10 @@ spa_config_sync(spa_t *target, boolean_t } if (nvl == NULL) - VERIFY(nvlist_alloc(&nvl, NV_UNIQUE_NAME, - KM_SLEEP) == 0); + nvl = fnvlist_alloc(); - VERIFY(nvlist_add_nvlist(nvl, spa->spa_name, - spa->spa_config) == 0); + fnvlist_add_nvlist(nvl, spa->spa_name, + spa->spa_config); mutex_exit(&spa->spa_props_lock); } @@ -322,15 +316,15 @@ spa_all_configs(uint64_t *generation) if (*generation == spa_config_generation) return (NULL); - VERIFY(nvlist_alloc(&pools, NV_UNIQUE_NAME, KM_SLEEP) == 0); + pools = fnvlist_alloc(); mutex_enter(&spa_namespace_lock); while ((spa = spa_next(spa)) != NULL) { if (INGLOBALZONE(curproc) || zone_dataset_visible(spa_name(spa), NULL)) { mutex_enter(&spa->spa_props_lock); - VERIFY(nvlist_add_nvlist(pools, spa_name(spa), - spa->spa_config) == 0); + fnvlist_add_nvlist(pools, spa_name(spa), + spa->spa_config); mutex_exit(&spa->spa_props_lock); } } @@ -379,21 +373,17 @@ spa_config_generate(spa_t *spa, vdev_t * if (txg == -1ULL) txg = spa->spa_config_txg; - VERIFY(nvlist_alloc(&config, NV_UNIQUE_NAME, KM_SLEEP) == 0); - - VERIFY(nvlist_add_uint64(config, ZPOOL_CONFIG_VERSION, - spa_version(spa)) == 0); - VERIFY(nvlist_add_string(config, ZPOOL_CONFIG_POOL_NAME, - spa_name(spa)) == 0); - VERIFY(nvlist_add_uint64(config, ZPOOL_CONFIG_POOL_STATE, - spa_state(spa)) == 0); - VERIFY(nvlist_add_uint64(config, ZPOOL_CONFIG_POOL_TXG, - txg) == 0); - VERIFY(nvlist_add_uint64(config, ZPOOL_CONFIG_POOL_GUID, - spa_guid(spa)) == 0); - VERIFY(spa->spa_comment == NULL || nvlist_add_string(config, - ZPOOL_CONFIG_COMMENT, spa->spa_comment) == 0); + config = fnvlist_alloc(); + fnvlist_add_uint64(config, ZPOOL_CONFIG_VERSION, spa_version(spa)); + fnvlist_add_string(config, ZPOOL_CONFIG_POOL_NAME, spa_name(spa)); + fnvlist_add_uint64(config, ZPOOL_CONFIG_POOL_STATE, spa_state(spa)); + fnvlist_add_uint64(config, ZPOOL_CONFIG_POOL_TXG, txg); + fnvlist_add_uint64(config, ZPOOL_CONFIG_POOL_GUID, spa_guid(spa)); + if (spa->spa_comment != NULL) { + fnvlist_add_string(config, ZPOOL_CONFIG_COMMENT, + spa->spa_comment); + } #ifdef _KERNEL hostid = zone_get_hostid(NULL); @@ -405,23 +395,24 @@ spa_config_generate(spa_t *spa, vdev_t * (void) ddi_strtoul(hw_serial, NULL, 10, &hostid); #endif /* _KERNEL */ if (hostid != 0) { - VERIFY(nvlist_add_uint64(config, ZPOOL_CONFIG_HOSTID, - hostid) == 0); + fnvlist_add_uint64(config, ZPOOL_CONFIG_HOSTID, hostid); } - VERIFY(nvlist_add_string(config, ZPOOL_CONFIG_HOSTNAME, - utsname.nodename) == 0); + fnvlist_add_string(config, ZPOOL_CONFIG_HOSTNAME, utsname.nodename); + int config_gen_flags = 0; if (vd != rvd) { - VERIFY(nvlist_add_uint64(config, ZPOOL_CONFIG_TOP_GUID, - vd->vdev_top->vdev_guid) == 0); - VERIFY(nvlist_add_uint64(config, ZPOOL_CONFIG_GUID, - vd->vdev_guid) == 0); - if (vd->vdev_isspare) - VERIFY(nvlist_add_uint64(config, ZPOOL_CONFIG_IS_SPARE, - 1ULL) == 0); - if (vd->vdev_islog) - VERIFY(nvlist_add_uint64(config, ZPOOL_CONFIG_IS_LOG, - 1ULL) == 0); + fnvlist_add_uint64(config, ZPOOL_CONFIG_TOP_GUID, + vd->vdev_top->vdev_guid); + fnvlist_add_uint64(config, ZPOOL_CONFIG_GUID, + vd->vdev_guid); + if (vd->vdev_isspare) { + fnvlist_add_uint64(config, + ZPOOL_CONFIG_IS_SPARE, 1ULL); + } + if (vd->vdev_islog) { + fnvlist_add_uint64(config, + ZPOOL_CONFIG_IS_LOG, 1ULL); + } vd = vd->vdev_top; /* label contains top config */ } else { /* @@ -429,8 +420,12 @@ spa_config_generate(spa_t *spa, vdev_t * * in the mos config, and not in the vdev labels */ if (spa->spa_config_splitting != NULL) - VERIFY(nvlist_add_nvlist(config, ZPOOL_CONFIG_SPLIT, - spa->spa_config_splitting) == 0); + fnvlist_add_nvlist(config, ZPOOL_CONFIG_SPLIT, + spa->spa_config_splitting); + fnvlist_add_boolean(config, + ZPOOL_CONFIG_HAS_PER_VDEV_ZAPS); + + config_gen_flags |= VDEV_CONFIG_MOS; } /* @@ -445,19 +440,19 @@ spa_config_generate(spa_t *spa, vdev_t * if (spa->spa_config_splitting != NULL && nvlist_lookup_uint64(spa->spa_config_splitting, ZPOOL_CONFIG_SPLIT_GUID, &split_guid) == 0) { - VERIFY(nvlist_add_uint64(config, ZPOOL_CONFIG_SPLIT_GUID, - split_guid) == 0); + fnvlist_add_uint64(config, ZPOOL_CONFIG_SPLIT_GUID, + split_guid); } - nvroot = vdev_config_generate(spa, vd, getstats, 0); - VERIFY(nvlist_add_nvlist(config, ZPOOL_CONFIG_VDEV_TREE, nvroot) == 0); + nvroot = vdev_config_generate(spa, vd, getstats, config_gen_flags); + fnvlist_add_nvlist(config, ZPOOL_CONFIG_VDEV_TREE, nvroot); nvlist_free(nvroot); /* * Store what's necessary for reading the MOS in the label. */ - VERIFY(nvlist_add_nvlist(config, ZPOOL_CONFIG_FEATURES_FOR_READ, - spa->spa_label_features) == 0); + fnvlist_add_nvlist(config, ZPOOL_CONFIG_FEATURES_FOR_READ, + spa->spa_label_features); if (getstats && spa_load_state(spa) == SPA_LOAD_NONE) { ddt_histogram_t *ddh; @@ -466,23 +461,23 @@ spa_config_generate(spa_t *spa, vdev_t * ddh = kmem_zalloc(sizeof (ddt_histogram_t), KM_SLEEP); ddt_get_dedup_histogram(spa, ddh); - VERIFY(nvlist_add_uint64_array(config, + fnvlist_add_uint64_array(config, ZPOOL_CONFIG_DDT_HISTOGRAM, - (uint64_t *)ddh, sizeof (*ddh) / sizeof (uint64_t)) == 0); + (uint64_t *)ddh, sizeof (*ddh) / sizeof (uint64_t)); kmem_free(ddh, sizeof (ddt_histogram_t)); ddo = kmem_zalloc(sizeof (ddt_object_t), KM_SLEEP); ddt_get_dedup_object_stats(spa, ddo); - VERIFY(nvlist_add_uint64_array(config, + fnvlist_add_uint64_array(config, ZPOOL_CONFIG_DDT_OBJ_STATS, - (uint64_t *)ddo, sizeof (*ddo) / sizeof (uint64_t)) == 0); + (uint64_t *)ddo, sizeof (*ddo) / sizeof (uint64_t)); kmem_free(ddo, sizeof (ddt_object_t)); dds = kmem_zalloc(sizeof (ddt_stat_t), KM_SLEEP); ddt_get_dedup_stats(spa, dds); - VERIFY(nvlist_add_uint64_array(config, + fnvlist_add_uint64_array(config, ZPOOL_CONFIG_DDT_STATS, - (uint64_t *)dds, sizeof (*dds) / sizeof (uint64_t)) == 0); + (uint64_t *)dds, sizeof (*dds) / sizeof (uint64_t)); kmem_free(dds, sizeof (ddt_stat_t)); } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h Wed May 11 12:46:07 2016 (r299439) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h Wed May 11 12:50:58 2016 (r299440) @@ -321,6 +321,7 @@ typedef struct dmu_buf { #define DMU_POOL_BPTREE_OBJ "bptree_obj" #define DMU_POOL_EMPTY_BPOBJ "empty_bpobj" #define DMU_POOL_CHECKSUM_SALT "org.illumos:checksum_salt" +#define DMU_POOL_VDEV_ZAP_MAP "com.delphix:vdev_zap_map" /* * Allocate an object from this objset. The range of object numbers Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h Wed May 11 12:46:07 2016 (r299439) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h Wed May 11 12:50:58 2016 (r299440) @@ -116,6 +116,12 @@ typedef struct spa_taskqs { taskq_t **stqs_taskq; } spa_taskqs_t; +typedef enum spa_all_vdev_zap_action { + AVZ_ACTION_NONE = 0, + AVZ_ACTION_DESTROY, /* Destroy all per-vdev ZAPs and the AVZ. */ + AVZ_ACTION_REBUILD /* Populate the new AVZ, see spa_avz_rebuild */ +} spa_avz_action_t; + struct spa { /* * Fields protected by spa_namespace_lock. @@ -253,6 +259,8 @@ struct spa { uint64_t spa_deadman_calls; /* number of deadman calls */ hrtime_t spa_sync_starttime; /* starting time fo spa_sync */ uint64_t spa_deadman_synctime; /* deadman expiration timer */ + uint64_t spa_all_vdev_zaps; /* ZAP of per-vd ZAP obj #s */ + spa_avz_action_t spa_avz_action; /* destroy/rebuild AVZ? */ /* * spa_iokstat_lock protects spa_iokstat and Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev.h Wed May 11 12:46:07 2016 (r299439) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev.h Wed May 11 12:50:58 2016 (r299440) @@ -71,6 +71,10 @@ extern void vdev_dtl_reassess(vdev_t *vd extern boolean_t vdev_dtl_required(vdev_t *vd); extern boolean_t vdev_resilver_needed(vdev_t *vd, uint64_t *minp, uint64_t *maxp); +extern void vdev_destroy_unlink_zap(vdev_t *vd, uint64_t zapobj, + dmu_tx_t *tx); +extern uint64_t vdev_create_link_zap(vdev_t *vd, dmu_tx_t *tx); +extern void vdev_construct_zaps(vdev_t *vd, dmu_tx_t *tx); extern void vdev_hold(vdev_t *); extern void vdev_rele(vdev_t *); @@ -130,7 +134,8 @@ extern void vdev_state_clean(vdev_t *vd) typedef enum vdev_config_flag { VDEV_CONFIG_SPARE = 1 << 0, VDEV_CONFIG_L2CACHE = 1 << 1, - VDEV_CONFIG_REMOVING = 1 << 2 + VDEV_CONFIG_REMOVING = 1 << 2, + VDEV_CONFIG_MOS = 1 << 3 } vdev_config_flag_t; extern void vdev_top_config_generate(spa_t *spa, nvlist_t *config); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h Wed May 11 12:46:07 2016 (r299439) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h Wed May 11 12:50:58 2016 (r299440) @@ -172,6 +172,7 @@ struct vdev { uint64_t vdev_islog; /* is an intent log device */ uint64_t vdev_removing; /* device is being removed? */ boolean_t vdev_ishole; /* is a hole in the namespace */ + uint64_t vdev_top_zap; /* * Leaf vdev state. @@ -210,6 +211,7 @@ struct vdev { spa_aux_vdev_t *vdev_aux; /* for l2cache and spares vdevs */ zio_t *vdev_probe_zio; /* root of current probe */ vdev_aux_t vdev_label_aux; /* on-disk aux state */ + uint64_t vdev_leaf_zap; /* * For DTrace to work in userland (libzpool) context, these fields must Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Wed May 11 12:46:07 2016 (r299439) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Wed May 11 12:50:58 2016 (r299440) @@ -515,6 +515,10 @@ vdev_alloc(spa_t *spa, vdev_t **vdp, nvl &vd->vdev_asize); (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_REMOVING, &vd->vdev_removing); + (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_VDEV_TOP_ZAP, + &vd->vdev_top_zap); + } else { + ASSERT0(vd->vdev_top_zap); } if (parent && !parent->vdev_parent && alloctype != VDEV_ALLOC_ATTACH) { @@ -526,9 +530,18 @@ vdev_alloc(spa_t *spa, vdev_t **vdp, nvl spa_log_class(spa) : spa_normal_class(spa), vd); } + if (vd->vdev_ops->vdev_op_leaf && + (alloctype == VDEV_ALLOC_LOAD || alloctype == VDEV_ALLOC_SPLIT)) { + (void) nvlist_lookup_uint64(nv, + ZPOOL_CONFIG_VDEV_LEAF_ZAP, &vd->vdev_leaf_zap); + } else { + ASSERT0(vd->vdev_leaf_zap); + } + /* * If we're a leaf vdev, try to load the DTL object and other state. */ + if (vd->vdev_ops->vdev_op_leaf && (alloctype == VDEV_ALLOC_LOAD || alloctype == VDEV_ALLOC_L2CACHE || alloctype == VDEV_ALLOC_ROOTPOOL)) { @@ -689,10 +702,12 @@ vdev_top_transfer(vdev_t *svd, vdev_t *t tvd->vdev_ms_array = svd->vdev_ms_array; tvd->vdev_ms_shift = svd->vdev_ms_shift; tvd->vdev_ms_count = svd->vdev_ms_count; + tvd->vdev_top_zap = svd->vdev_top_zap; svd->vdev_ms_array = 0; svd->vdev_ms_shift = 0; svd->vdev_ms_count = 0; + svd->vdev_top_zap = 0; if (tvd->vdev_mg) ASSERT3P(tvd->vdev_mg, ==, svd->vdev_mg); @@ -1934,6 +1949,49 @@ vdev_dtl_load(vdev_t *vd) } void +vdev_destroy_unlink_zap(vdev_t *vd, uint64_t zapobj, dmu_tx_t *tx) +{ + spa_t *spa = vd->vdev_spa; + + VERIFY0(zap_destroy(spa->spa_meta_objset, zapobj, tx)); + VERIFY0(zap_remove_int(spa->spa_meta_objset, spa->spa_all_vdev_zaps, + zapobj, tx)); +} + +uint64_t +vdev_create_link_zap(vdev_t *vd, dmu_tx_t *tx) +{ + spa_t *spa = vd->vdev_spa; + uint64_t zap = zap_create(spa->spa_meta_objset, DMU_OTN_ZAP_METADATA, + DMU_OT_NONE, 0, tx); + + ASSERT(zap != 0); + VERIFY0(zap_add_int(spa->spa_meta_objset, spa->spa_all_vdev_zaps, + zap, tx)); + + return (zap); +} + +void +vdev_construct_zaps(vdev_t *vd, dmu_tx_t *tx) +{ + if (vd->vdev_ops != &vdev_hole_ops && + vd->vdev_ops != &vdev_missing_ops && + vd->vdev_ops != &vdev_root_ops && + !vd->vdev_top->vdev_removing) { + if (vd->vdev_ops->vdev_op_leaf && vd->vdev_leaf_zap == 0) { + vd->vdev_leaf_zap = vdev_create_link_zap(vd, tx); + } + if (vd == vd->vdev_top && vd->vdev_top_zap == 0) { + vd->vdev_top_zap = vdev_create_link_zap(vd, tx); + } + } + for (uint64_t i = 0; i < vd->vdev_children; i++) { + vdev_construct_zaps(vd->vdev_child[i], tx); + } +} + +void vdev_dtl_sync(vdev_t *vd, uint64_t txg) { spa_t *spa = vd->vdev_spa; @@ -1955,6 +2013,18 @@ vdev_dtl_sync(vdev_t *vd, uint64_t txg) space_map_close(vd->vdev_dtl_sm); vd->vdev_dtl_sm = NULL; mutex_exit(&vd->vdev_dtl_lock); + + /* + * We only destroy the leaf ZAP for detached leaves or for + * removed log devices. Removed data devices handle leaf ZAP + * cleanup later, once cancellation is no longer possible. + */ + if (vd->vdev_leaf_zap != 0 && (vd->vdev_detached || + vd->vdev_top->vdev_islog)) { + vdev_destroy_unlink_zap(vd, vd->vdev_leaf_zap, tx); + vd->vdev_leaf_zap = 0; + } + dmu_tx_commit(tx); return; } @@ -2157,6 +2227,8 @@ vdev_remove(vdev_t *vd, uint64_t txg) dmu_tx_t *tx; tx = dmu_tx_create_assigned(spa_get_dsl(spa), txg); + ASSERT(vd == vd->vdev_top); + ASSERT3U(txg, ==, spa_syncing_txg(spa)); if (vd->vdev_ms != NULL) { metaslab_group_t *mg = vd->vdev_mg; @@ -2198,6 +2270,11 @@ vdev_remove(vdev_t *vd, uint64_t txg) (void) dmu_object_free(mos, vd->vdev_ms_array, tx); vd->vdev_ms_array = 0; } + + if (vd->vdev_islog && vd->vdev_top_zap != 0) { + vdev_destroy_unlink_zap(vd, vd->vdev_top_zap, tx); + vd->vdev_top_zap = 0; + } dmu_tx_commit(tx); } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c Wed May 11 12:46:07 2016 (r299439) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c Wed May 11 12:50:58 2016 (r299440) @@ -291,6 +291,20 @@ vdev_config_generate(spa_t *spa, vdev_t if (vd->vdev_crtxg) fnvlist_add_uint64(nv, ZPOOL_CONFIG_CREATE_TXG, vd->vdev_crtxg); + if (flags & VDEV_CONFIG_MOS) { + if (vd->vdev_leaf_zap != 0) { + ASSERT(vd->vdev_ops->vdev_op_leaf); + fnvlist_add_uint64(nv, ZPOOL_CONFIG_VDEV_LEAF_ZAP, + vd->vdev_leaf_zap); + } + + if (vd->vdev_top_zap != 0) { + ASSERT(vd == vd->vdev_top); + fnvlist_add_uint64(nv, ZPOOL_CONFIG_VDEV_TOP_ZAP, + vd->vdev_top_zap); + } + } + if (getstats) { vdev_stat_t vs; pool_scan_stat_t ps; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zap.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zap.c Wed May 11 12:46:07 2016 (r299439) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zap.c Wed May 11 12:50:58 2016 (r299440) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012, 2014 by Delphix. All rights reserved. + * Copyright (c) 2012, 2015 by Delphix. All rights reserved. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. */ @@ -962,8 +962,8 @@ zap_create_link(objset_t *os, dmu_object uint64_t new_obj; VERIFY((new_obj = zap_create(os, ot, DMU_OT_NONE, 0, tx)) > 0); - VERIFY(zap_add(os, parent_obj, name, sizeof (uint64_t), 1, &new_obj, - tx) == 0); + VERIFY0(zap_add(os, parent_obj, name, sizeof (uint64_t), 1, &new_obj, + tx)); return (new_obj); } Modified: vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Wed May 11 12:46:07 2016 (r299439) +++ vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Wed May 11 12:50:58 2016 (r299440) @@ -555,6 +555,9 @@ typedef struct zpool_rewind_policy { #define ZPOOL_CONFIG_CAN_RDONLY "can_rdonly" /* not stored on disk */ #define ZPOOL_CONFIG_FEATURES_FOR_READ "features_for_read" #define ZPOOL_CONFIG_FEATURE_STATS "feature_stats" /* not stored on disk */ +#define ZPOOL_CONFIG_VDEV_TOP_ZAP "com.delphix:vdev_zap_top" +#define ZPOOL_CONFIG_VDEV_LEAF_ZAP "com.delphix:vdev_zap_leaf" +#define ZPOOL_CONFIG_HAS_PER_VDEV_ZAPS "com.delphix:has_per_vdev_zaps" /* * The persistent vdev state is stored as separate values rather than a single * 'vdev_state' entry. This is because a device can be in multiple states, such From owner-svn-src-vendor@freebsd.org Wed May 11 12:58:13 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 89C80B367D3; Wed, 11 May 2016 12:58:13 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5609917C1; Wed, 11 May 2016 12:58:13 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4BCwCqI061312; Wed, 11 May 2016 12:58:12 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4BCwCme061309; Wed, 11 May 2016 12:58:12 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201605111258.u4BCwCme061309@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 11 May 2016 12:58:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299442 - in vendor-sys/illumos/dist: common/acl uts/common/fs/zfs uts/common/sys X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 12:58:13 -0000 Author: mav Date: Wed May 11 12:58:12 2016 New Revision: 299442 URL: https://svnweb.freebsd.org/changeset/base/299442 Log: 6762 POSIX write should imply DELETE_CHILD on directories - and some additional considerations Reviewed by: Gordon Ross Reviewed by: Yuri Pankov Author: Kevin Crowe openzfs/openzfs@d316fffc9c361532a482208561bbb614dac7f916 Modified: vendor-sys/illumos/dist/common/acl/acl_common.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c vendor-sys/illumos/dist/uts/common/sys/acl.h Modified: vendor-sys/illumos/dist/common/acl/acl_common.c ============================================================================== --- vendor-sys/illumos/dist/common/acl/acl_common.c Wed May 11 12:54:00 2016 (r299441) +++ vendor-sys/illumos/dist/common/acl/acl_common.c Wed May 11 12:58:12 2016 (r299442) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright 2011 Nexenta Systems, Inc. All rights reserved. + * Copyright 2014 Nexenta Systems, Inc. All rights reserved. */ #include @@ -1575,7 +1575,8 @@ acl_trivial_access_masks(mode_t mode, bo uint32_t write_mask = ACE_WRITE_DATA|ACE_APPEND_DATA; uint32_t execute_mask = ACE_EXECUTE; - (void) isdir; /* will need this later */ + if (isdir) + write_mask |= ACE_DELETE_CHILD; masks->deny1 = 0; if (!(mode & S_IRUSR) && (mode & (S_IRGRP|S_IROTH))) @@ -1719,10 +1720,17 @@ ace_trivial_common(void *acep, int aclcn return (1); /* - * Delete permissions are never set by default + * Delete permission is never set by default + */ + if (mask & ACE_DELETE) + return (1); + + /* + * Child delete permission should be accompanied by write */ - if (mask & (ACE_DELETE|ACE_DELETE_CHILD)) + if ((mask & ACE_DELETE_CHILD) && !(mask & ACE_WRITE_DATA)) return (1); + /* * only allow owner@ to have * write_acl/write_owner/write_attributes/write_xattr/ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c Wed May 11 12:54:00 2016 (r299441) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c Wed May 11 12:58:12 2016 (r299442) @@ -20,8 +20,8 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright 2011 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright 2014 Nexenta Systems, Inc. All rights reserved. */ #include @@ -2080,7 +2080,7 @@ zfs_zaccess_dataset_check(znode_t *zp, u * placed into the working_mode, giving the caller a mask of denied * accesses. Returns: * 0 if all AoI granted - * EACCESS if the denied mask is non-zero + * EACCES if the denied mask is non-zero * other error if abnormal failure (e.g., IO error) * * A secondary usage of the function is to determine if any of the @@ -2517,46 +2517,30 @@ zfs_zaccess_unix(znode_t *zp, mode_t mod return (zfs_zaccess(zp, v4_mode, 0, B_FALSE, cr)); } -static int -zfs_delete_final_check(znode_t *zp, znode_t *dzp, - mode_t available_perms, cred_t *cr) -{ - int error; - uid_t downer; - - downer = zfs_fuid_map_id(dzp->z_zfsvfs, dzp->z_uid, cr, ZFS_OWNER); - - error = secpolicy_vnode_access2(cr, ZTOV(dzp), - downer, available_perms, VWRITE|VEXEC); - - if (error == 0) - error = zfs_sticky_remove_access(dzp, zp, cr); - - return (error); -} +/* See zfs_zaccess_delete() */ +int zfs_write_implies_delete_child = 1; /* - * Determine whether Access should be granted/deny, without - * consulting least priv subsystem. + * Determine whether delete access should be granted. * * The following chart is the recommended NFSv4 enforcement for * ability to delete an object. * * ------------------------------------------------------- - * | Parent Dir | Target Object Permissions | + * | Parent Dir | Target Object Permissions | * | permissions | | * ------------------------------------------------------- * | | ACL Allows | ACL Denies| Delete | * | | Delete | Delete | unspecified| * ------------------------------------------------------- - * | ACL Allows | Permit | Permit | Permit | - * | DELETE_CHILD | | + * | ACL Allows | Permit | Permit * | Permit | + * | DELETE_CHILD | | | | * ------------------------------------------------------- - * | ACL Denies | Permit | Deny | Deny | + * | ACL Denies | Permit * | Deny | Deny | * | DELETE_CHILD | | | | * ------------------------------------------------------- * | ACL specifies | | | | - * | only allow | Permit | Permit | Permit | + * | only allow | Permit | Permit * | Permit | * | write and | | | | * | execute | | | | * ------------------------------------------------------- @@ -2566,91 +2550,174 @@ zfs_delete_final_check(znode_t *zp, znod * ------------------------------------------------------- * ^ * | - * No search privilege, can't even look up file? + * Re. execute permission on the directory: if that's missing, + * the vnode lookup of the target will fail before we get here. + * + * Re [*] in the table above: We are intentionally disregarding the + * NFSv4 committee recommendation for these three cells of the matrix + * because that recommendation conflicts with the behavior expected + * by Windows clients for ACL evaluation. See acl.h for notes on + * which ACE_... flags should be checked for which operations. + * Specifically, the NFSv4 committee recommendation is in conflict + * with the Windows interpretation of DENY ACEs, where DENY ACEs + * should take precedence ahead of ALLOW ACEs. + * + * This implementation takes a conservative approach by checking for + * DENY ACEs on both the target object and it's container; checking + * the ACE_DELETE on the target object, and ACE_DELETE_CHILD on the + * container. If a DENY ACE is found for either of those, delete + * access is denied. (Note that DENY ACEs are very rare.) + * + * Note that after these changes, entire the second row and the + * entire middle column of the table above change to Deny. + * Accordingly, the logic here is somewhat simplified. * + * First check for DENY ACEs that apply. + * If either target or container has a deny, EACCES. + * + * Delete access can then be summarized as follows: + * 1: The object to be deleted grants ACE_DELETE, or + * 2: The containing directory grants ACE_DELETE_CHILD. + * In a Windows system, that would be the end of the story. + * In this system, (2) has some complications... + * 2a: "sticky" bit on a directory adds restrictions, and + * 2b: existing ACEs from previous versions of ZFS may + * not carry ACE_DELETE_CHILD where they should, so we + * also allow delete when ACE_WRITE_DATA is granted. + * + * Note: 2b is technically a work-around for a prior bug, + * which hopefully can go away some day. For those who + * no longer need the work around, and for testing, this + * work-around is made conditional via the tunable: + * zfs_write_implies_delete_child */ int zfs_zaccess_delete(znode_t *dzp, znode_t *zp, cred_t *cr) { + uint32_t wanted_dirperms; uint32_t dzp_working_mode = 0; uint32_t zp_working_mode = 0; int dzp_error, zp_error; - mode_t available_perms; - boolean_t dzpcheck_privs = B_TRUE; - boolean_t zpcheck_privs = B_TRUE; - - /* - * We want specific DELETE permissions to - * take precedence over WRITE/EXECUTE. We don't - * want an ACL such as this to mess us up. - * user:joe:write_data:deny,user:joe:delete:allow - * - * However, deny permissions may ultimately be overridden - * by secpolicy_vnode_access(). - * - * We will ask for all of the necessary permissions and then - * look at the working modes from the directory and target object - * to determine what was found. - */ + boolean_t dzpcheck_privs; + boolean_t zpcheck_privs; if (zp->z_pflags & (ZFS_IMMUTABLE | ZFS_NOUNLINK)) return (SET_ERROR(EPERM)); /* - * First row - * If the directory permissions allow the delete, we are done. - */ - if ((dzp_error = zfs_zaccess_common(dzp, ACE_DELETE_CHILD, - &dzp_working_mode, &dzpcheck_privs, B_FALSE, cr)) == 0) - return (0); + * Case 1: + * If target object grants ACE_DELETE then we are done. This is + * indicated by a return value of 0. For this case we don't worry + * about the sticky bit because sticky only applies to the parent + * directory and this is the child access result. + * + * If we encounter a DENY ACE here, we're also done (EACCES). + * Note that if we hit a DENY ACE here (on the target) it should + * take precedence over a DENY ACE on the container, so that when + * we have more complete auditing support we will be able to + * report an access failure against the specific target. + * (This is part of why we're checking the target first.) + */ + zp_error = zfs_zaccess_common(zp, ACE_DELETE, &zp_working_mode, + &zpcheck_privs, B_FALSE, cr); + if (zp_error == EACCES) { + /* We hit a DENY ACE. */ + if (!zpcheck_privs) + return (SET_ERROR(zp_error)); + return (secpolicy_vnode_remove(cr)); - /* - * If target object has delete permission then we are done - */ - if ((zp_error = zfs_zaccess_common(zp, ACE_DELETE, &zp_working_mode, - &zpcheck_privs, B_FALSE, cr)) == 0) + } + if (zp_error == 0) return (0); - ASSERT(dzp_error && zp_error); + /* + * Case 2: + * If the containing directory grants ACE_DELETE_CHILD, + * or we're in backward compatibility mode and the + * containing directory has ACE_WRITE_DATA, allow. + * Case 2b is handled with wanted_dirperms. + */ + wanted_dirperms = ACE_DELETE_CHILD; + if (zfs_write_implies_delete_child) + wanted_dirperms |= ACE_WRITE_DATA; + dzp_error = zfs_zaccess_common(dzp, wanted_dirperms, + &dzp_working_mode, &dzpcheck_privs, B_FALSE, cr); + if (dzp_error == EACCES) { + /* We hit a DENY ACE. */ + if (!dzpcheck_privs) + return (SET_ERROR(dzp_error)); + return (secpolicy_vnode_remove(cr)); + } - if (!dzpcheck_privs) - return (dzp_error); - if (!zpcheck_privs) - return (zp_error); + /* + * Cases 2a, 2b (continued) + * + * Note: dzp_working_mode now contains any permissions + * that were NOT granted. Therefore, if any of the + * wanted_dirperms WERE granted, we will have: + * dzp_working_mode != wanted_dirperms + * We're really asking if ANY of those permissions + * were granted, and if so, grant delete access. + */ + if (dzp_working_mode != wanted_dirperms) + dzp_error = 0; /* - * Second row + * dzp_error is 0 if the container granted us permissions to "modify". + * If we do not have permission via one or more ACEs, our current + * privileges may still permit us to modify the container. * - * If directory returns EACCES then delete_child was denied - * due to deny delete_child. In this case send the request through - * secpolicy_vnode_remove(). We don't use zfs_delete_final_check() - * since that *could* allow the delete based on write/execute permission - * and we want delete permissions to override write/execute. + * dzpcheck_privs is false when i.e. the FS is read-only. + * Otherwise, do privilege checks for the container. */ + if (dzp_error != 0 && dzpcheck_privs) { + uid_t owner; - if (dzp_error == EACCES) - return (secpolicy_vnode_remove(cr)); + /* + * The secpolicy call needs the requested access and + * the current access mode of the container, but it + * only knows about Unix-style modes (VEXEC, VWRITE), + * so this must condense the fine-grained ACE bits into + * Unix modes. + * + * The VEXEC flag is easy, because we know that has + * always been checked before we get here (during the + * lookup of the target vnode). The container has not + * granted us permissions to "modify", so we do not set + * the VWRITE flag in the current access mode. + */ + owner = zfs_fuid_map_id(dzp->z_zfsvfs, dzp->z_uid, cr, + ZFS_OWNER); + dzp_error = secpolicy_vnode_access2(cr, ZTOV(dzp), + owner, VEXEC, VWRITE|VEXEC); + } + if (dzp_error != 0) { + /* + * Note: We may have dzp_error = -1 here (from + * zfs_zacess_common). Don't return that. + */ + return (SET_ERROR(EACCES)); + } /* - * Third Row - * only need to see if we have write/execute on directory. + * At this point, we know that the directory permissions allow + * us to modify, but we still need to check for the additional + * restrictions that apply when the "sticky bit" is set. + * + * Yes, zfs_sticky_remove_access() also checks this bit, but + * checking it here and skipping the call below is nice when + * you're watching all of this with dtrace. */ - - dzp_error = zfs_zaccess_common(dzp, ACE_EXECUTE|ACE_WRITE_DATA, - &dzp_working_mode, &dzpcheck_privs, B_FALSE, cr); - - if (dzp_error != 0 && !dzpcheck_privs) - return (dzp_error); + if ((dzp->z_mode & S_ISVTX) == 0) + return (0); /* - * Fourth row + * zfs_sticky_remove_access will succeed if: + * 1. The sticky bit is absent. + * 2. We pass the sticky bit restrictions. + * 3. We have privileges that always allow file removal. */ - - available_perms = (dzp_working_mode & ACE_WRITE_DATA) ? 0 : VWRITE; - available_perms |= (dzp_working_mode & ACE_EXECUTE) ? 0 : VEXEC; - - return (zfs_delete_final_check(zp, dzp, available_perms, cr)); - + return (zfs_sticky_remove_access(dzp, zp, cr)); } int Modified: vendor-sys/illumos/dist/uts/common/sys/acl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/sys/acl.h Wed May 11 12:54:00 2016 (r299441) +++ vendor-sys/illumos/dist/uts/common/sys/acl.h Wed May 11 12:58:12 2016 (r299442) @@ -23,6 +23,8 @@ * * Copyright 2009 Sun Microsystems, Inc. All rights reserved. * Use is subject to license terms. + * + * Copyright 2014 Nexenta Systems, Inc. All rights reserved. */ #ifndef _SYS_ACL_H @@ -76,37 +78,55 @@ typedef struct acl_info acl_t; /* * The following are defined for ace_t. + * + * Note, these are intentionally the same as the Windows + * "File Access Rights Constants" you can find on MSDN. + * (See also: "Standard Access Rights" on MSDN). + * + * The equivalent Windows names for these are just like + * those show below, with FILE_ in place of ACE_, except + * as noted below. Also note that Windows uses a special + * privilege: BYPASS_TRAVERSE_CHECKING, normally granted + * to everyone, that causes the absence of ACE_TRAVERSE + * to be ignored. + */ +#define ACE_READ_DATA 0x00000001 /* file: read data */ +#define ACE_LIST_DIRECTORY 0x00000001 /* dir: list files */ +#define ACE_WRITE_DATA 0x00000002 /* file: write data */ +#define ACE_ADD_FILE 0x00000002 /* dir: create file */ +#define ACE_APPEND_DATA 0x00000004 /* file: append data */ +#define ACE_ADD_SUBDIRECTORY 0x00000004 /* dir: create subdir */ +#define ACE_READ_NAMED_ATTRS 0x00000008 /* FILE_READ_EA */ +#define ACE_WRITE_NAMED_ATTRS 0x00000010 /* FILE_WRITE_EA */ +#define ACE_EXECUTE 0x00000020 /* file: execute */ +#define ACE_TRAVERSE 0x00000020 /* dir: lookup name */ +#define ACE_DELETE_CHILD 0x00000040 /* dir: unlink child */ +#define ACE_READ_ATTRIBUTES 0x00000080 /* (all) stat, etc. */ +#define ACE_WRITE_ATTRIBUTES 0x00000100 /* (all) utimes, etc. */ +#define ACE_DELETE 0x00010000 /* (all) unlink self */ +#define ACE_READ_ACL 0x00020000 /* (all) getsecattr */ +#define ACE_WRITE_ACL 0x00040000 /* (all) setsecattr */ +#define ACE_WRITE_OWNER 0x00080000 /* (all) chown */ +#define ACE_SYNCHRONIZE 0x00100000 /* (all) see MSDN */ + +/* + * Some of the following are the same as Windows uses. (but NOT ALL!) + * See the "ACE_HEADER" structure description on MSDN for details. + * Comments show relations to the MSDN names. */ -#define ACE_READ_DATA 0x00000001 -#define ACE_LIST_DIRECTORY 0x00000001 -#define ACE_WRITE_DATA 0x00000002 -#define ACE_ADD_FILE 0x00000002 -#define ACE_APPEND_DATA 0x00000004 -#define ACE_ADD_SUBDIRECTORY 0x00000004 -#define ACE_READ_NAMED_ATTRS 0x00000008 -#define ACE_WRITE_NAMED_ATTRS 0x00000010 -#define ACE_EXECUTE 0x00000020 -#define ACE_DELETE_CHILD 0x00000040 -#define ACE_READ_ATTRIBUTES 0x00000080 -#define ACE_WRITE_ATTRIBUTES 0x00000100 -#define ACE_DELETE 0x00010000 -#define ACE_READ_ACL 0x00020000 -#define ACE_WRITE_ACL 0x00040000 -#define ACE_WRITE_OWNER 0x00080000 -#define ACE_SYNCHRONIZE 0x00100000 - -#define ACE_FILE_INHERIT_ACE 0x0001 -#define ACE_DIRECTORY_INHERIT_ACE 0x0002 -#define ACE_NO_PROPAGATE_INHERIT_ACE 0x0004 -#define ACE_INHERIT_ONLY_ACE 0x0008 +#define ACE_FILE_INHERIT_ACE 0x0001 /* = OBJECT_INHERIT_ACE */ +#define ACE_DIRECTORY_INHERIT_ACE 0x0002 /* = CONTAINER_INHERIT_ACE */ +#define ACE_NO_PROPAGATE_INHERIT_ACE 0x0004 /* = NO_PROPAGATE_INHERIT_ACE */ +#define ACE_INHERIT_ONLY_ACE 0x0008 /* = INHERIT_ONLY_ACE */ #define ACE_SUCCESSFUL_ACCESS_ACE_FLAG 0x0010 #define ACE_FAILED_ACCESS_ACE_FLAG 0x0020 #define ACE_IDENTIFIER_GROUP 0x0040 -#define ACE_INHERITED_ACE 0x0080 +#define ACE_INHERITED_ACE 0x0080 /* INHERITED_ACE, 0x10 on NT */ #define ACE_OWNER 0x1000 #define ACE_GROUP 0x2000 #define ACE_EVERYONE 0x4000 +/* These four are the same as Windows, but with an ACE_ prefix added. */ #define ACE_ACCESS_ALLOWED_ACE_TYPE 0x0000 #define ACE_ACCESS_DENIED_ACE_TYPE 0x0001 #define ACE_SYSTEM_AUDIT_ACE_TYPE 0x0002 @@ -122,6 +142,7 @@ typedef struct acl_info acl_t; /* * These are only applicable in a CIFS context. + * Here again, same as Windows, but with an ACE_ prefix added. */ #define ACE_ACCESS_ALLOWED_COMPOUND_ACE_TYPE 0x04 #define ACE_ACCESS_ALLOWED_OBJECT_ACE_TYPE 0x05 From owner-svn-src-vendor@freebsd.org Wed May 11 13:46:45 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58A04B3677C; Wed, 11 May 2016 13:46:45 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 27F4C1FA5; Wed, 11 May 2016 13:46:45 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4BDki2r077002; Wed, 11 May 2016 13:46:44 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4BDkiue077001; Wed, 11 May 2016 13:46:44 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201605111346.u4BDkiue077001@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 11 May 2016 13:46:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299449 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 13:46:45 -0000 Author: mav Date: Wed May 11 13:46:44 2016 New Revision: 299449 URL: https://svnweb.freebsd.org/changeset/base/299449 Log: 6763 aclinherit=restricted masks inherited permissions by group perms (groupmask) Reviewed by: Gordon Ross Reviewed by: Yuri Pankov Author: Albert Lee openzfs/openzfs@eebb483d0cd68bdc4cf03c01fdeba9af160c17af Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c Wed May 11 13:43:20 2016 (r299448) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c Wed May 11 13:46:44 2016 (r299449) @@ -1411,11 +1411,10 @@ zfs_acl_chmod(vtype_t vtype, uint64_t mo } else { /* - * Limit permissions to be no greater than - * group permissions. - * The "aclinherit" and "aclmode" properties - * affect policy for create and chmod(2), - * respectively. + * Limit permissions granted by ACEs to be no greater + * than permissions of the requested group mode. + * Applies when the "aclmode" property is set to + * "groupmask". */ if ((type == ALLOW) && trim) access_mask &= masks.group; @@ -1729,7 +1728,7 @@ zfs_acl_ids_create(znode_t *dzp, int fla acl_ids->z_aclp->z_hints |= (vap->va_type == VDIR) ? ZFS_ACL_AUTO_INHERIT : 0; zfs_acl_chmod(vap->va_type, acl_ids->z_mode, - (zfsvfs->z_acl_inherit == ZFS_ACL_RESTRICTED), + (zfsvfs->z_acl_mode == ZFS_ACL_GROUPMASK), acl_ids->z_aclp); } } From owner-svn-src-vendor@freebsd.org Wed May 11 13:49:51 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4E090B368BF; Wed, 11 May 2016 13:49:51 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1052D12CD; Wed, 11 May 2016 13:49:50 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4BDnocZ077224; Wed, 11 May 2016 13:49:50 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4BDnoqk077223; Wed, 11 May 2016 13:49:50 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201605111349.u4BDnoqk077223@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 11 May 2016 13:49:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299451 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 13:49:51 -0000 Author: mav Date: Wed May 11 13:49:50 2016 New Revision: 299451 URL: https://svnweb.freebsd.org/changeset/base/299451 Log: 6764 zfs issues with inheritance flags during chmod(2) with aclmode=passthrough Reviewed by: Gordon Ross Reviewed by: Yuri Pankov Author: Albert Lee openzfs/openzfs@1bcf0d240bdebed61b4261f7c0ee323e07c8dfac Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c Wed May 11 13:48:15 2016 (r299450) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c Wed May 11 13:49:50 2016 (r299451) @@ -887,7 +887,6 @@ zfs_set_ace(zfs_acl_t *aclp, void *acep, /* * Determine mode of file based on ACL. - * Also, create FUIDs for any User/Group ACEs */ uint64_t zfs_mode_compute(uint64_t fmode, zfs_acl_t *aclp, @@ -913,11 +912,9 @@ zfs_mode_compute(uint64_t fmode, zfs_acl entry_type = (iflags & ACE_TYPE_FLAGS); /* - * Skip over owner@, group@ or everyone@ inherit only ACEs + * Skip over any inherit_only ACEs */ - if ((iflags & ACE_INHERIT_ONLY_ACE) && - (entry_type == ACE_OWNER || entry_type == ACE_EVERYONE || - entry_type == OWNING_GROUP)) + if (iflags & ACE_INHERIT_ONLY_ACE) continue; if (entry_type == ACE_OWNER || (entry_type == 0 && @@ -1333,7 +1330,8 @@ zfs_aclset_common(znode_t *zp, zfs_acl_t } static void -zfs_acl_chmod(vtype_t vtype, uint64_t mode, boolean_t trim, zfs_acl_t *aclp) +zfs_acl_chmod(vtype_t vtype, uint64_t mode, boolean_t split, boolean_t trim, + zfs_acl_t *aclp) { void *acep = NULL; uint64_t who; @@ -1378,15 +1376,27 @@ zfs_acl_chmod(vtype_t vtype, uint64_t mo while (acep = zfs_acl_next_ace(aclp, acep, &who, &access_mask, &iflags, &type)) { - uint16_t inherit_flags; - entry_type = (iflags & ACE_TYPE_FLAGS); - inherit_flags = (iflags & ALL_INHERIT); - - if ((entry_type == ACE_OWNER || entry_type == ACE_EVERYONE || - (entry_type == OWNING_GROUP)) && - ((inherit_flags & ACE_INHERIT_ONLY_ACE) == 0)) { - continue; + /* + * ACEs used to represent the file mode may be divided + * into an equivalent pair of inherit-only and regular + * ACEs, if they are inheritable. + * Skip regular ACEs, which are replaced by the new mode. + */ + if (split && (entry_type == ACE_OWNER || + entry_type == OWNING_GROUP || + entry_type == ACE_EVERYONE)) { + if (!isdir || !(iflags & + (ACE_FILE_INHERIT_ACE|ACE_DIRECTORY_INHERIT_ACE))) + continue; + /* + * We preserve owner@, group@, or @everyone + * permissions, if they are inheritable, by + * copying them to inherit_only ACEs. This + * prevents inheritable permissions from being + * altered along with the file mode. + */ + iflags |= ACE_INHERIT_ONLY_ACE; } /* @@ -1394,12 +1404,12 @@ zfs_acl_chmod(vtype_t vtype, uint64_t mo * the hints (which are later masked into the pflags) * so create knows to do inheritance. */ - if (isdir && (inherit_flags & + if (isdir && (iflags & (ACE_FILE_INHERIT_ACE|ACE_DIRECTORY_INHERIT_ACE))) aclp->z_hints |= ZFS_INHERIT_ACE; if ((type != ALLOW && type != DENY) || - (inherit_flags & ACE_INHERIT_ONLY_ACE)) { + (iflags & ACE_INHERIT_ONLY_ACE)) { switch (type) { case ACE_ACCESS_ALLOWED_OBJECT_ACE_TYPE: case ACE_ACCESS_DENIED_OBJECT_ACE_TYPE: @@ -1409,7 +1419,6 @@ zfs_acl_chmod(vtype_t vtype, uint64_t mo break; } } else { - /* * Limit permissions granted by ACEs to be no greater * than permissions of the requested group mode. @@ -1425,11 +1434,11 @@ zfs_acl_chmod(vtype_t vtype, uint64_t mo new_count++; new_bytes += ace_size; } - zfs_set_ace(aclp, zacep, masks.owner, 0, -1, ACE_OWNER); + zfs_set_ace(aclp, zacep, masks.owner, ALLOW, -1, ACE_OWNER); zacep = (void *)((uintptr_t)zacep + abstract_size); - zfs_set_ace(aclp, zacep, masks.group, 0, -1, OWNING_GROUP); + zfs_set_ace(aclp, zacep, masks.group, ALLOW, -1, OWNING_GROUP); zacep = (void *)((uintptr_t)zacep + abstract_size); - zfs_set_ace(aclp, zacep, masks.everyone, 0, -1, ACE_EVERYONE); + zfs_set_ace(aclp, zacep, masks.everyone, ALLOW, -1, ACE_EVERYONE); new_count += 3; new_bytes += abstract_size * 3; @@ -1455,7 +1464,7 @@ zfs_acl_chmod_setattr(znode_t *zp, zfs_a if (error == 0) { (*aclp)->z_hints = zp->z_pflags & V4_ACL_WIDE_FLAGS; - zfs_acl_chmod(ZTOV(zp)->v_type, mode, + zfs_acl_chmod(ZTOV(zp)->v_type, mode, B_TRUE, (zp->z_zfsvfs->z_acl_mode == ZFS_ACL_GROUPMASK), *aclp); } mutex_exit(&zp->z_lock); @@ -1465,21 +1474,6 @@ zfs_acl_chmod_setattr(znode_t *zp, zfs_a } /* - * strip off write_owner and write_acl - */ -static void -zfs_restricted_update(zfsvfs_t *zfsvfs, zfs_acl_t *aclp, void *acep) -{ - uint32_t mask = aclp->z_ops.ace_mask_get(acep); - - if ((zfsvfs->z_acl_inherit == ZFS_ACL_RESTRICTED) && - (aclp->z_ops.ace_type_get(acep) == ALLOW)) { - mask &= ~RESTRICTED_CLEAR; - aclp->z_ops.ace_mask_set(acep, mask); - } -} - -/* * Should ACE be inherited? */ static int @@ -1500,9 +1494,9 @@ zfs_ace_can_use(vtype_t vtype, uint16_t */ static zfs_acl_t * zfs_acl_inherit(zfsvfs_t *zfsvfs, vtype_t vtype, zfs_acl_t *paclp, - uint64_t mode, boolean_t *need_chmod) + uint64_t mode) { - void *pacep; + void *pacep = NULL; void *acep; zfs_acl_node_t *aclnode; zfs_acl_t *aclp = NULL; @@ -1512,22 +1506,14 @@ zfs_acl_inherit(zfsvfs_t *zfsvfs, vtype_ size_t ace_size; void *data1, *data2; size_t data1sz, data2sz; - boolean_t vdir = vtype == VDIR; - boolean_t vreg = vtype == VREG; - boolean_t passthrough, passthrough_x, noallow; - - passthrough_x = - zfsvfs->z_acl_inherit == ZFS_ACL_PASSTHROUGH_X; - passthrough = passthrough_x || - zfsvfs->z_acl_inherit == ZFS_ACL_PASSTHROUGH; - noallow = - zfsvfs->z_acl_inherit == ZFS_ACL_NOALLOW; + uint_t aclinherit; + boolean_t isdir = (vtype == VDIR); - *need_chmod = B_TRUE; - pacep = NULL; aclp = zfs_acl_alloc(paclp->z_version); - if (zfsvfs->z_acl_inherit == ZFS_ACL_DISCARD || vtype == VLNK) + aclinherit = zfsvfs->z_acl_inherit; + if (aclinherit == ZFS_ACL_DISCARD || vtype == VLNK) return (aclp); + while (pacep = zfs_acl_next_ace(paclp, pacep, &who, &access_mask, &iflags, &type)) { @@ -1537,31 +1523,31 @@ zfs_acl_inherit(zfsvfs_t *zfsvfs, vtype_ if (!zfs_acl_valid_ace_type(type, iflags)) continue; - if (noallow && type == ALLOW) - continue; - - ace_size = aclp->z_ops.ace_size(pacep); - - if (!zfs_ace_can_use(vtype, iflags)) + /* + * Check if ACE is inheritable by this vnode + */ + if ((aclinherit == ZFS_ACL_NOALLOW && type == ALLOW) || + !zfs_ace_can_use(vtype, iflags)) continue; /* - * If owner@, group@, or everyone@ inheritable - * then zfs_acl_chmod() isn't needed. + * Strip inherited execute permission from file if + * not in mode */ - if (passthrough && - ((iflags & (ACE_OWNER|ACE_EVERYONE)) || - ((iflags & OWNING_GROUP) == - OWNING_GROUP)) && (vreg || (vdir && (iflags & - ACE_DIRECTORY_INHERIT_ACE)))) { - *need_chmod = B_FALSE; + if (aclinherit == ZFS_ACL_PASSTHROUGH_X && type == ALLOW && + !isdir && ((mode & (S_IXUSR|S_IXGRP|S_IXOTH)) == 0)) { + access_mask &= ~ACE_EXECUTE; } - if (!vdir && passthrough_x && - ((mode & (S_IXUSR | S_IXGRP | S_IXOTH)) == 0)) { - access_mask &= ~ACE_EXECUTE; + /* + * Strip write_acl and write_owner from permissions + * when inheriting an ACE + */ + if (aclinherit == ZFS_ACL_RESTRICTED && type == ALLOW) { + access_mask &= ~RESTRICTED_CLEAR; } + ace_size = aclp->z_ops.ace_size(pacep); aclnode = zfs_acl_node_alloc(ace_size); list_insert_tail(&aclp->z_acl, aclnode); acep = aclnode->z_acldata; @@ -1583,18 +1569,21 @@ zfs_acl_inherit(zfsvfs_t *zfsvfs, vtype_ aclp->z_acl_bytes += aclnode->z_size; newflags = aclp->z_ops.ace_flags_get(acep); - if (vdir) - aclp->z_hints |= ZFS_INHERIT_ACE; - - if ((iflags & ACE_NO_PROPAGATE_INHERIT_ACE) || !vdir) { + /* + * If ACE is not to be inherited further, or if the vnode is + * not a directory, remove all inheritance flags + */ + if (!isdir || (iflags & ACE_NO_PROPAGATE_INHERIT_ACE)) { newflags &= ~ALL_INHERIT; aclp->z_ops.ace_flags_set(acep, newflags|ACE_INHERITED_ACE); - zfs_restricted_update(zfsvfs, aclp, acep); continue; } - ASSERT(vdir); + /* + * This directory has an inheritable ACE + */ + aclp->z_hints |= ZFS_INHERIT_ACE; /* * If only FILE_INHERIT is set then turn on @@ -1611,12 +1600,14 @@ zfs_acl_inherit(zfsvfs_t *zfsvfs, vtype_ newflags|ACE_INHERITED_ACE); } } + return (aclp); } /* * Create file system object initial permissions * including inheritable ACEs. + * Also, create FUIDs for owner and group. */ int zfs_acl_ids_create(znode_t *dzp, int flag, vattr_t *vap, cred_t *cr, @@ -1626,7 +1617,7 @@ zfs_acl_ids_create(znode_t *dzp, int fla zfsvfs_t *zfsvfs = dzp->z_zfsvfs; zfs_acl_t *paclp; gid_t gid; - boolean_t need_chmod = B_TRUE; + boolean_t trim = B_FALSE; boolean_t inherited = B_FALSE; bzero(acl_ids, sizeof (zfs_acl_ids_t)); @@ -1715,7 +1706,7 @@ zfs_acl_ids_create(znode_t *dzp, int fla VERIFY(0 == zfs_acl_node_read(dzp, B_TRUE, &paclp, B_FALSE)); acl_ids->z_aclp = zfs_acl_inherit(zfsvfs, - vap->va_type, paclp, acl_ids->z_mode, &need_chmod); + vap->va_type, paclp, acl_ids->z_mode); inherited = B_TRUE; } else { acl_ids->z_aclp = @@ -1724,13 +1715,16 @@ zfs_acl_ids_create(znode_t *dzp, int fla } mutex_exit(&dzp->z_lock); mutex_exit(&dzp->z_acl_lock); - if (need_chmod) { - acl_ids->z_aclp->z_hints |= (vap->va_type == VDIR) ? - ZFS_ACL_AUTO_INHERIT : 0; - zfs_acl_chmod(vap->va_type, acl_ids->z_mode, - (zfsvfs->z_acl_mode == ZFS_ACL_GROUPMASK), - acl_ids->z_aclp); - } + + if (vap->va_type == VDIR) + acl_ids->z_aclp->z_hints |= ZFS_ACL_AUTO_INHERIT; + + if (zfsvfs->z_acl_mode == ZFS_ACL_GROUPMASK && + zfsvfs->z_acl_inherit != ZFS_ACL_PASSTHROUGH && + zfsvfs->z_acl_inherit != ZFS_ACL_PASSTHROUGH_X) + trim = B_TRUE; + zfs_acl_chmod(vap->va_type, acl_ids->z_mode, B_FALSE, trim, + acl_ids->z_aclp); } if (inherited || vsecp) { From owner-svn-src-vendor@freebsd.org Wed May 11 13:51:54 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D9B61B36A99; Wed, 11 May 2016 13:51:54 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9A7CE197F; Wed, 11 May 2016 13:51:54 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4BDprRI077407; Wed, 11 May 2016 13:51:53 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4BDproS077406; Wed, 11 May 2016 13:51:53 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201605111351.u4BDproS077406@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 11 May 2016 13:51:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299453 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 13:51:55 -0000 Author: mav Date: Wed May 11 13:51:53 2016 New Revision: 299453 URL: https://svnweb.freebsd.org/changeset/base/299453 Log: 6765 zfs_zaccess_delete() comments do not accurately reflect delete permissions for ACLs Reviewed by: Gordon Ross Reviewed by: Yuri Pankov Author: Kevin Crowe openzfs/openzfs@a40149b935cbbe87bf95e2cc44b3bc99d400513a Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c Wed May 11 13:50:34 2016 (r299452) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_acl.c Wed May 11 13:51:53 2016 (r299453) @@ -2516,8 +2516,10 @@ int zfs_write_implies_delete_child = 1; /* * Determine whether delete access should be granted. * - * The following chart is the recommended NFSv4 enforcement for - * ability to delete an object. + * The following chart outlines how we handle delete permissions which is + * how recent versions of windows (Windows 2008) handles it. The efficiency + * comes from not having to check the parent ACL where the object itself grants + * delete: * * ------------------------------------------------------- * | Parent Dir | Target Object Permissions | @@ -2526,14 +2528,14 @@ int zfs_write_implies_delete_child = 1; * | | ACL Allows | ACL Denies| Delete | * | | Delete | Delete | unspecified| * ------------------------------------------------------- - * | ACL Allows | Permit | Permit * | Permit | - * | DELETE_CHILD | | | | + * | ACL Allows | Permit | Deny * | Permit | + * | DELETE_CHILD | | | | * ------------------------------------------------------- - * | ACL Denies | Permit * | Deny | Deny | - * | DELETE_CHILD | | | | + * | ACL Denies | Permit | Deny | Deny | + * | DELETE_CHILD | | | | * ------------------------------------------------------- * | ACL specifies | | | | - * | only allow | Permit | Permit * | Permit | + * | only allow | Permit | Deny * | Permit | * | write and | | | | * | execute | | | | * ------------------------------------------------------- @@ -2546,24 +2548,21 @@ int zfs_write_implies_delete_child = 1; * Re. execute permission on the directory: if that's missing, * the vnode lookup of the target will fail before we get here. * - * Re [*] in the table above: We are intentionally disregarding the - * NFSv4 committee recommendation for these three cells of the matrix - * because that recommendation conflicts with the behavior expected - * by Windows clients for ACL evaluation. See acl.h for notes on - * which ACE_... flags should be checked for which operations. - * Specifically, the NFSv4 committee recommendation is in conflict - * with the Windows interpretation of DENY ACEs, where DENY ACEs + * Re [*] in the table above: NFSv4 would normally Permit delete for + * these two cells of the matrix. + * See acl.h for notes on which ACE_... flags should be checked for which + * operations. Specifically, the NFSv4 committee recommendation is in + * conflict with the Windows interpretation of DENY ACEs, where DENY ACEs * should take precedence ahead of ALLOW ACEs. * - * This implementation takes a conservative approach by checking for - * DENY ACEs on both the target object and it's container; checking - * the ACE_DELETE on the target object, and ACE_DELETE_CHILD on the - * container. If a DENY ACE is found for either of those, delete - * access is denied. (Note that DENY ACEs are very rare.) - * - * Note that after these changes, entire the second row and the - * entire middle column of the table above change to Deny. - * Accordingly, the logic here is somewhat simplified. + * This implementation always consults the target object's ACL first. + * If a DENY ACE is present on the target object that specifies ACE_DELETE, + * delete access is denied. If an ALLOW ACE with ACE_DELETE is present on + * the target object, access is allowed. If and only if no entries with + * ACE_DELETE are present in the object's ACL, check the container's ACL + * for entries with ACE_DELETE_CHILD. + * + * A summary of the logic implemented from the table above is as follows: * * First check for DENY ACEs that apply. * If either target or container has a deny, EACCES. From owner-svn-src-vendor@freebsd.org Thu May 12 09:44:22 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B403FB37F2D; Thu, 12 May 2016 09:44:22 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6798F1807; Thu, 12 May 2016 09:44:22 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4C9iLUQ047811; Thu, 12 May 2016 09:44:21 GMT (envelope-from mm@FreeBSD.org) Received: (from mm@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4C9iLwP047810; Thu, 12 May 2016 09:44:21 GMT (envelope-from mm@FreeBSD.org) Message-Id: <201605120944.u4C9iLwP047810@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mm set sender to mm@FreeBSD.org using -f From: Martin Matuska Date: Thu, 12 May 2016 09:44:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299528 - vendor/libarchive/3.2.0 X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2016 09:44:22 -0000 Author: mm Date: Thu May 12 09:44:21 2016 New Revision: 299528 URL: https://svnweb.freebsd.org/changeset/base/299528 Log: Tag libarchive 3.2.0 Added: vendor/libarchive/3.2.0/ - copied from r299433, vendor/libarchive/dist/ From owner-svn-src-vendor@freebsd.org Thu May 12 16:03:56 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D68FBB37104; Thu, 12 May 2016 16:03:56 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9AF16145A; Thu, 12 May 2016 16:03:56 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4CG3tjL063838; Thu, 12 May 2016 16:03:55 GMT (envelope-from mm@FreeBSD.org) Received: (from mm@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4CG3t4n063834; Thu, 12 May 2016 16:03:55 GMT (envelope-from mm@FreeBSD.org) Message-Id: <201605121603.u4CG3t4n063834@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mm set sender to mm@FreeBSD.org using -f From: Martin Matuska Date: Thu, 12 May 2016 16:03:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299539 - in vendor/libarchive/dist: cat libarchive libarchive_fe X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2016 16:03:56 -0000 Author: mm Date: Thu May 12 16:03:55 2016 New Revision: 299539 URL: https://svnweb.freebsd.org/changeset/base/299539 Log: Update vendor/libarchive/dist to git commit f48d99b Integrates my pull requests #701, #702 and #703 Modified: vendor/libarchive/dist/cat/bsdcat.c vendor/libarchive/dist/cat/bsdcat.h vendor/libarchive/dist/libarchive/archive_read_support_format_mtree.c vendor/libarchive/dist/libarchive_fe/err.c Modified: vendor/libarchive/dist/cat/bsdcat.c ============================================================================== --- vendor/libarchive/dist/cat/bsdcat.c Thu May 12 16:01:33 2016 (r299538) +++ vendor/libarchive/dist/cat/bsdcat.c Thu May 12 16:03:55 2016 (r299539) @@ -42,10 +42,10 @@ __FBSDID("$FreeBSD$"); #define BYTES_PER_BLOCK (20*512) -struct archive *a; -struct archive_entry *ae; -char *bsdcat_current_path; -int exit_status = 0; +static struct archive *a; +static struct archive_entry *ae; +static const char *bsdcat_current_path; +static int exit_status = 0; void @@ -68,7 +68,7 @@ version(void) } void -bsdcat_next() +bsdcat_next(void) { a = archive_read_new(); archive_read_support_filter_all(a); @@ -85,7 +85,7 @@ bsdcat_print_error(void) } void -bsdcat_read_to_stdout(char* filename) +bsdcat_read_to_stdout(const char* filename) { int r; Modified: vendor/libarchive/dist/cat/bsdcat.h ============================================================================== --- vendor/libarchive/dist/cat/bsdcat.h Thu May 12 16:01:33 2016 (r299538) +++ vendor/libarchive/dist/cat/bsdcat.h Thu May 12 16:03:55 2016 (r299539) @@ -53,4 +53,4 @@ int bsdcat_getopt(struct bsdcat *); void usage(FILE *stream, int eval); void bsdcat_next(void); void bsdcat_print_error(void); -void bsdcat_read_to_stdout(char* filename); +void bsdcat_read_to_stdout(const char* filename); Modified: vendor/libarchive/dist/libarchive/archive_read_support_format_mtree.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_read_support_format_mtree.c Thu May 12 16:01:33 2016 (r299538) +++ vendor/libarchive/dist/libarchive/archive_read_support_format_mtree.c Thu May 12 16:03:55 2016 (r299539) @@ -1342,7 +1342,7 @@ parse_line(struct archive_read *a, struc /* strsep() is not in C90, but strcspn() is. */ /* Taken from http://unixpapa.com/incnote/string.html */ static char * -la_strsep(char **sp, char *sep) +la_strsep(char **sp, const char *sep) { char *p, *s; if (sp == NULL || *sp == NULL || **sp == '\0') Modified: vendor/libarchive/dist/libarchive_fe/err.c ============================================================================== --- vendor/libarchive/dist/libarchive_fe/err.c Thu May 12 16:01:33 2016 (r299538) +++ vendor/libarchive/dist/libarchive_fe/err.c Thu May 12 16:03:55 2016 (r299539) @@ -42,7 +42,7 @@ __FBSDID("$FreeBSD$"); static void lafe_vwarnc(int, const char *, va_list) __LA_PRINTFLIKE(2, 0); -const char *lafe_progname; +static const char *lafe_progname; const char * lafe_getprogname(void) From owner-svn-src-vendor@freebsd.org Sat May 14 05:25:50 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D6F1CB3AAB9; Sat, 14 May 2016 05:25:50 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 787E81043; Sat, 14 May 2016 05:25:50 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4E5PnFa054762; Sat, 14 May 2016 05:25:49 GMT (envelope-from delphij@FreeBSD.org) Received: (from delphij@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4E5Pl9V054742; Sat, 14 May 2016 05:25:47 GMT (envelope-from delphij@FreeBSD.org) Message-Id: <201605140525.u4E5Pl9V054742@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: delphij set sender to delphij@FreeBSD.org using -f From: Xin LI Date: Sat, 14 May 2016 05:25:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299716 - in vendor/file/dist: . magic/Magdir src tests X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 May 2016 05:25:50 -0000 Author: delphij Date: Sat May 14 05:25:47 2016 New Revision: 299716 URL: https://svnweb.freebsd.org/changeset/base/299716 Log: Vendor import of file 4.27. Modified: vendor/file/dist/ChangeLog vendor/file/dist/configure vendor/file/dist/configure.ac vendor/file/dist/magic/Magdir/archive vendor/file/dist/magic/Magdir/commands vendor/file/dist/magic/Magdir/console vendor/file/dist/magic/Magdir/os2 vendor/file/dist/magic/Magdir/xenix vendor/file/dist/src/ascmagic.c vendor/file/dist/src/cdf.c vendor/file/dist/src/cdf.h vendor/file/dist/src/compress.c vendor/file/dist/src/der.c vendor/file/dist/src/magic.c vendor/file/dist/src/magic.h vendor/file/dist/src/magic.h.in vendor/file/dist/src/readcdf.c vendor/file/dist/src/softmagic.c vendor/file/dist/tests/Makefile.am vendor/file/dist/tests/Makefile.in Modified: vendor/file/dist/ChangeLog ============================================================================== --- vendor/file/dist/ChangeLog Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/ChangeLog Sat May 14 05:25:47 2016 (r299716) @@ -1,6 +1,19 @@ +2016-05-13 12:00 Christos Zoulas + + * release 5.27 + +2016-04-18 9:35 Christos Zoulas + + * Errors comparing DER entries or computing offsets + are just indications of malformed non-DER files. + Don't print them. + * Offset comparison was off-by-one. + * Fix compression code (Werner Fink) + * Put new bytes constant in the right file (not the generated one) + 2016-04-16 18:34 Christos Zoulas - * release 5.25 + * release 5.26 2016-03-31 13:50 Christos Zoulas Modified: vendor/file/dist/configure ============================================================================== --- vendor/file/dist/configure Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/configure Sat May 14 05:25:47 2016 (r299716) @@ -1,6 +1,6 @@ #! /bin/sh # Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.69 for file 5.26. +# Generated by GNU Autoconf 2.69 for file 5.27. # # Report bugs to . # @@ -590,8 +590,8 @@ MAKEFLAGS= # Identity of this package. PACKAGE_NAME='file' PACKAGE_TARNAME='file' -PACKAGE_VERSION='5.26' -PACKAGE_STRING='file 5.26' +PACKAGE_VERSION='5.27' +PACKAGE_STRING='file 5.27' PACKAGE_BUGREPORT='christos@astron.com' PACKAGE_URL='' @@ -1327,7 +1327,7 @@ if test "$ac_init_help" = "long"; then # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF -\`configure' configures file 5.26 to adapt to many kinds of systems. +\`configure' configures file 5.27 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... @@ -1397,7 +1397,7 @@ fi if test -n "$ac_init_help"; then case $ac_init_help in - short | recursive ) echo "Configuration of file 5.26:";; + short | recursive ) echo "Configuration of file 5.27:";; esac cat <<\_ACEOF @@ -1507,7 +1507,7 @@ fi test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF -file configure 5.26 +file configure 5.27 generated by GNU Autoconf 2.69 Copyright (C) 2012 Free Software Foundation, Inc. @@ -2163,7 +2163,7 @@ cat >config.log <<_ACEOF This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. -It was created by file $as_me 5.26, which was +It was created by file $as_me 5.27, which was generated by GNU Autoconf 2.69. Invocation command line was $ $0 $@ @@ -3029,7 +3029,7 @@ fi # Define the identity of the package. PACKAGE='file' - VERSION='5.26' + VERSION='5.27' cat >>confdefs.h <<_ACEOF @@ -15049,7 +15049,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_wri # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" -This file was extended by file $as_me 5.26, which was +This file was extended by file $as_me 5.27, which was generated by GNU Autoconf 2.69. Invocation command line was CONFIG_FILES = $CONFIG_FILES @@ -15115,7 +15115,7 @@ _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ -file config.status 5.26 +file config.status 5.27 configured by $0, generated by GNU Autoconf 2.69, with options \\"\$ac_cs_config\\" Modified: vendor/file/dist/configure.ac ============================================================================== --- vendor/file/dist/configure.ac Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/configure.ac Sat May 14 05:25:47 2016 (r299716) @@ -1,5 +1,5 @@ dnl Process this file with autoconf to produce a configure script. -AC_INIT([file],[5.26],[christos@astron.com]) +AC_INIT([file],[5.27],[christos@astron.com]) AM_INIT_AUTOMAKE([subdir-objects foreign]) m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) Modified: vendor/file/dist/magic/Magdir/archive ============================================================================== --- vendor/file/dist/magic/Magdir/archive Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/magic/Magdir/archive Sat May 14 05:25:47 2016 (r299716) @@ -1,5 +1,5 @@ #------------------------------------------------------------------------------ -# $File: archive,v 1.102 2016/01/11 20:59:24 christos Exp $ +# $File: archive,v 1.103 2016/05/05 17:07:40 christos Exp $ # archive: file(1) magic for archive formats (see also "msdos" for self- # extracting compressed archives) # @@ -891,6 +891,14 @@ >(26.s+30) leshort 0xcafe Java archive data (JAR) !:mime application/java-archive +# iOS App +>(26.s+30) leshort !0xcafe +>>26 string !\x8\0\0\0mimetype +>>>30 string Payload/ +>>>>38 search/64 .app/ iOS App +!:mime application/x-ios-app + + # Generic zip archives (Greg Roelofs, c/o zip-bugs@wkuvx1.wku.edu) # Next line excludes specialized formats: >(26.s+30) leshort !0xcafe Modified: vendor/file/dist/magic/Magdir/commands ============================================================================== --- vendor/file/dist/magic/Magdir/commands Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/magic/Magdir/commands Sat May 14 05:25:47 2016 (r299716) @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: commands,v 1.53 2016/02/23 12:35:20 christos Exp $ +# $File: commands,v 1.54 2016/04/19 13:40:02 christos Exp $ # commands: file(1) magic for various shells and interpreters # #0 string/w : shell archive or script for antique kernel text @@ -101,7 +101,7 @@ 0 string =5 regex [\ \n] >>6 string /*\ Smarty\ version Smarty compiled template ->24 regex [0-9.]+ \b, version %s +>>>24 regex [0-9.]+ \b, version %s !:mime text/x-php 0 string Zend\x00 PHP script Zend Optimizer data Modified: vendor/file/dist/magic/Magdir/console ============================================================================== --- vendor/file/dist/magic/Magdir/console Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/magic/Magdir/console Sat May 14 05:25:47 2016 (r299716) @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: console,v 1.24 2016/03/23 15:29:20 christos Exp $ +# $File: console,v 1.25 2016/04/18 20:22:10 christos Exp $ # Console game magic # Toby Deshane @@ -53,7 +53,7 @@ # so most of the data isn't easily parseable. # 0 string UNIF -4 lelong <16 UNIF v%d format NES ROM image +>4 lelong <16 UNIF v%d format NES ROM image #------------------------------------------------------------------------------ # gameboy: file(1) magic for the Nintendo (Color) Gameboy raw ROM format Modified: vendor/file/dist/magic/Magdir/os2 ============================================================================== --- vendor/file/dist/magic/Magdir/os2 Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/magic/Magdir/os2 Sat May 14 05:25:47 2016 (r299716) @@ -1,12 +1,12 @@ #------------------------------------------------------------------------------ -# $File: os2,v 1.8 2015/01/05 00:17:13 christos Exp $ +# $File: os2,v 1.9 2016/05/11 15:51:57 christos Exp $ # os2: file(1) magic for OS/2 files # # Provided 1998/08/22 by # David Mediavilla -1 search/1 InternetShortcut MS Windows 95 Internet shortcut text +1 search/100 InternetShortcut MS Windows 95 Internet shortcut text >17 search/100 URL= (URL=< >>&0 string x \b%s>) Modified: vendor/file/dist/magic/Magdir/xenix ============================================================================== --- vendor/file/dist/magic/Magdir/xenix Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/magic/Magdir/xenix Sat May 14 05:25:47 2016 (r299716) @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: xenix,v 1.9 2009/09/19 16:28:13 christos Exp $ +# $File: xenix,v 1.10 2016/04/19 18:14:19 christos Exp $ # xenix: file(1) magic for Microsoft Xenix # # "Middle model" stuff, and "Xenix 8086 relocatable or 80286 small @@ -12,7 +12,26 @@ # XXX - "x.out" collides with PDP-11 archives # 0 string core core file (Xenix) -0 byte 0x80 8086 relocatable (Microsoft) +# URL: http://www.polarhome.com/service/man/?qf=86rel&tf=2&of=Xenix +# Reference: http://www.azillionmonkeys.com/qed/Omfg.pdf +# Update: Joerg Jenderek +# recordtype~TranslatorHEADerRecord +0 byte 0x80 +# GRR: line above is too general as it catches also Extensible storage engine DataBase +# skip examples like GENA.SND Switch.Snd by looking for record length maximal 1024-3 +>1 uleshort <1022 +# skip examples like GAME.PICTURE Strange.Pic by looking for positiv record length +>>1 uleshort >0 +# skip examples like Xtable.Data FRACTAL.GEN SHR.VIEW by looking for positiv string length +>>>3 ubyte >0 +# skip examples like OMBRE.6 with "UUUUUU" by looking for filename like "hello.c" +>>>>4 regex [a-zA-Z_/]{1,8}[.] 8086 relocatable (Microsoft) +#!:mime application/octet-stream +!:mime application/x-object +!:ext o/a +>>>>>3 pstring x \b, "%s" +# checksum +#>>>>>(3.b+4) ubyte x \b, checksum 0x%2.2x 0 leshort 0xff65 x.out >2 string __.SYMDEF randomized >0 byte x archive Modified: vendor/file/dist/src/ascmagic.c ============================================================================== --- vendor/file/dist/src/ascmagic.c Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/src/ascmagic.c Sat May 14 05:25:47 2016 (r299716) @@ -35,7 +35,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: ascmagic.c,v 1.94 2016/03/31 17:51:12 christos Exp $") +FILE_RCSID("@(#)$File: ascmagic.c,v 1.95 2016/05/03 16:10:37 christos Exp $") #endif /* lint */ #include "magic.h" @@ -79,9 +79,6 @@ file_ascmagic(struct magic_set *ms, cons const char *code_mime = NULL; const char *type = NULL; - if (ms->flags & (MAGIC_APPLE|MAGIC_EXTENSION)) - return 0; - nbytes = trim_nuls(buf, nbytes); /* If file doesn't look like any sort of text, give up. */ @@ -123,9 +120,6 @@ file_ascmagic_with_encoding(struct magic size_t last_line_end = (size_t)-1; int has_long_lines = 0; - if (ms->flags & (MAGIC_APPLE|MAGIC_EXTENSION)) - return 0; - nbytes = trim_nuls(buf, nbytes); /* If we have fewer than 2 bytes, give up. */ @@ -150,7 +144,11 @@ file_ascmagic_with_encoding(struct magic (size_t)(utf8_end - utf8_buf), NULL, NULL, TEXTTEST, text)) == 0) rv = -1; + if ((ms->flags & (MAGIC_APPLE|MAGIC_EXTENSION))) + return rv == -1 ? 0 : 1; } + if ((ms->flags & (MAGIC_APPLE|MAGIC_EXTENSION))) + return 0; /* Now try to discover other details about the file. */ for (i = 0; i < ulen; i++) { Modified: vendor/file/dist/src/cdf.c ============================================================================== --- vendor/file/dist/src/cdf.c Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/src/cdf.c Sat May 14 05:25:47 2016 (r299716) @@ -35,7 +35,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: cdf.c,v 1.76 2015/02/28 00:18:02 christos Exp $") +FILE_RCSID("@(#)$File: cdf.c,v 1.80 2016/05/06 15:17:10 christos Exp $") #endif #include @@ -267,13 +267,32 @@ cdf_unpack_dir(cdf_directory_t *d, char } static int +cdf_zero_stream(cdf_stream_t *scn) +{ + scn->sst_len = 0; + scn->sst_dirlen = 0; + scn->sst_ss = 0; + free(scn->sst_tab); + scn->sst_tab = NULL; + return -1; +} + +static size_t +cdf_check_stream(const cdf_stream_t *sst, const cdf_header_t *h) +{ + size_t ss = sst->sst_dirlen < h->h_min_size_standard_stream ? + CDF_SHORT_SEC_SIZE(h) : CDF_SEC_SIZE(h); + assert(ss == sst->sst_ss); + return sst->sst_ss; +} + +static int cdf_check_stream_offset(const cdf_stream_t *sst, const cdf_header_t *h, const void *p, size_t tail, int line) { const char *b = (const char *)sst->sst_tab; const char *e = ((const char *)p) + tail; - size_t ss = sst->sst_dirlen < h->h_min_size_standard_stream ? - CDF_SHORT_SEC_SIZE(h) : CDF_SEC_SIZE(h); + size_t ss = cdf_check_stream(sst, h); /*LINTED*/(void)&line; if (e >= b && (size_t)(e - b) <= ss * sst->sst_len) return 0; @@ -290,10 +309,8 @@ cdf_read(const cdf_info_t *info, off_t o { size_t siz = (size_t)off + len; - if ((off_t)(off + len) != (off_t)siz) { - errno = EINVAL; - return -1; - } + if ((off_t)(off + len) != (off_t)siz) + goto out; if (info->i_buf != NULL && info->i_len >= siz) { (void)memcpy(buf, &info->i_buf[off], len); @@ -301,12 +318,15 @@ cdf_read(const cdf_info_t *info, off_t o } if (info->i_fd == -1) - return -1; + goto out; if (pread(info->i_fd, buf, len, off) != (ssize_t)len) return -1; return (ssize_t)len; +out: + errno = EINVAL; + return -1; } int @@ -363,11 +383,14 @@ cdf_read_short_sector(const cdf_stream_t DPRINTF(("Out of bounds read %" SIZE_T_FORMAT "u > %" SIZE_T_FORMAT "u\n", pos + len, CDF_SEC_SIZE(h) * sst->sst_len)); - return -1; + goto out; } (void)memcpy(((char *)buf) + offs, ((const char *)sst->sst_tab) + pos, len); return len; +out: + errno = EFTYPE; + return -1; } /* @@ -421,8 +444,7 @@ cdf_read_sat(const cdf_info_t *info, cdf goto out; if (j >= CDF_LOOP_LIMIT) { DPRINTF(("Reading master sector loop limit")); - errno = EFTYPE; - goto out2; + goto out3; } if (cdf_read_sector(info, msa, 0, ss, h, mid) != (ssize_t)ss) { DPRINTF(("Reading master sector %d", mid)); @@ -435,8 +457,7 @@ cdf_read_sat(const cdf_info_t *info, cdf if (i >= sat->sat_len) { DPRINTF(("Out of bounds reading MSA %" SIZE_T_FORMAT "u >= %" SIZE_T_FORMAT "u", i, sat->sat_len)); - errno = EFTYPE; - goto out2; + goto out3; } if (cdf_read_sector(info, sat->sat_tab, ss * i, ss, h, sec) != (ssize_t)ss) { @@ -451,6 +472,8 @@ out: sat->sat_len = i; free(msa); return 0; +out3: + errno = EFTYPE; out2: free(msa); out1: @@ -476,23 +499,24 @@ cdf_count_chain(const cdf_sat_t *sat, cd DPRINTF((" %d", sid)); if (j >= CDF_LOOP_LIMIT) { DPRINTF(("Counting chain loop limit")); - errno = EFTYPE; - return (size_t)-1; + goto out; } if (sid >= maxsector) { DPRINTF(("Sector %d >= %d\n", sid, maxsector)); - errno = EFTYPE; - return (size_t)-1; + goto out; } sid = CDF_TOLE4((uint32_t)sat->sat_tab[sid]); } if (i == 0) { DPRINTF((" none, sid: %d\n", sid)); - return (size_t)-1; + goto out; } DPRINTF(("\n")); return i; +out: + errno = EFTYPE; + return (size_t)-1; } int @@ -501,27 +525,27 @@ cdf_read_long_sector_chain(const cdf_inf { size_t ss = CDF_SEC_SIZE(h), i, j; ssize_t nr; + scn->sst_tab = NULL; scn->sst_len = cdf_count_chain(sat, sid, ss); scn->sst_dirlen = len; + scn->sst_ss = ss; if (scn->sst_len == (size_t)-1) - return -1; + goto out; scn->sst_tab = calloc(scn->sst_len, ss); if (scn->sst_tab == NULL) - return -1; + return cdf_zero_stream(scn); for (j = i = 0; sid >= 0; i++, j++) { if (j >= CDF_LOOP_LIMIT) { DPRINTF(("Read long sector chain loop limit")); - errno = EFTYPE; goto out; } if (i >= scn->sst_len) { DPRINTF(("Out of bounds reading long sector chain " "%" SIZE_T_FORMAT "u > %" SIZE_T_FORMAT "u\n", i, scn->sst_len)); - errno = EFTYPE; goto out; } if ((nr = cdf_read_sector(info, scn->sst_tab, i * ss, ss, h, @@ -537,8 +561,8 @@ cdf_read_long_sector_chain(const cdf_inf } return 0; out: - free(scn->sst_tab); - return -1; + errno = EFTYPE; + return cdf_zero_stream(scn); } int @@ -547,27 +571,27 @@ cdf_read_short_sector_chain(const cdf_he cdf_secid_t sid, size_t len, cdf_stream_t *scn) { size_t ss = CDF_SHORT_SEC_SIZE(h), i, j; - scn->sst_len = cdf_count_chain(ssat, sid, CDF_SEC_SIZE(h)); + scn->sst_tab = NULL; + scn->sst_len = cdf_count_chain(ssat, sid, ss); scn->sst_dirlen = len; + scn->sst_ss = ss; if (sst->sst_tab == NULL || scn->sst_len == (size_t)-1) - return -1; + goto out; scn->sst_tab = calloc(scn->sst_len, ss); if (scn->sst_tab == NULL) - return -1; + return cdf_zero_stream(scn); for (j = i = 0; sid >= 0; i++, j++) { if (j >= CDF_LOOP_LIMIT) { DPRINTF(("Read short sector chain loop limit")); - errno = EFTYPE; goto out; } if (i >= scn->sst_len) { DPRINTF(("Out of bounds reading short sector chain " "%" SIZE_T_FORMAT "u > %" SIZE_T_FORMAT "u\n", i, scn->sst_len)); - errno = EFTYPE; goto out; } if (cdf_read_short_sector(sst, scn->sst_tab, i * ss, ss, h, @@ -579,8 +603,8 @@ cdf_read_short_sector_chain(const cdf_he } return 0; out: - free(scn->sst_tab); - return -1; + errno = EFTYPE; + return cdf_zero_stream(scn); } int @@ -625,7 +649,6 @@ cdf_read_dir(const cdf_info_t *info, con for (j = i = 0; i < ns; i++, j++) { if (j >= CDF_LOOP_LIMIT) { DPRINTF(("Read dir loop limit")); - errno = EFTYPE; goto out; } if (cdf_read_sector(info, buf, 0, ss, h, sid) != (ssize_t)ss) { @@ -646,6 +669,7 @@ cdf_read_dir(const cdf_info_t *info, con out: free(dir->dir_tab); free(buf); + errno = EFTYPE; return -1; } @@ -658,36 +682,37 @@ cdf_read_ssat(const cdf_info_t *info, co size_t ss = CDF_SEC_SIZE(h); cdf_secid_t sid = h->h_secid_first_sector_in_short_sat; + ssat->sat_tab = NULL; ssat->sat_len = cdf_count_chain(sat, sid, CDF_SEC_SIZE(h)); if (ssat->sat_len == (size_t)-1) - return -1; + goto out; ssat->sat_tab = CAST(cdf_secid_t *, calloc(ssat->sat_len, ss)); if (ssat->sat_tab == NULL) - return -1; + goto out1; for (j = i = 0; sid >= 0; i++, j++) { if (j >= CDF_LOOP_LIMIT) { DPRINTF(("Read short sat sector loop limit")); - errno = EFTYPE; goto out; } if (i >= ssat->sat_len) { DPRINTF(("Out of bounds reading short sector chain " "%" SIZE_T_FORMAT "u > %" SIZE_T_FORMAT "u\n", i, ssat->sat_len)); - errno = EFTYPE; goto out; } if (cdf_read_sector(info, ssat->sat_tab, i * ss, ss, h, sid) != (ssize_t)ss) { DPRINTF(("Reading short sat sector %d", sid)); - goto out; + goto out1; } sid = CDF_TOLE4((uint32_t)sat->sat_tab[sid]); } return 0; out: + errno = EFTYPE; +out1: free(ssat->sat_tab); return -1; } @@ -715,13 +740,13 @@ cdf_read_short_stream(const cdf_info_t * if (d->d_stream_first_sector < 0) goto out; - return cdf_read_long_sector_chain(info, h, sat, + return cdf_read_long_sector_chain(info, h, sat, d->d_stream_first_sector, d->d_size, scn); out: scn->sst_tab = NULL; - scn->sst_len = 0; - scn->sst_dirlen = 0; - return 0; + (void)cdf_zero_stream(scn); + errno = EFTYPE; + return -1; } static int @@ -750,8 +775,10 @@ cdf_read_user_stream(const cdf_info_t *i const cdf_directory_t *d; int i = cdf_find_stream(dir, name, CDF_DIR_TYPE_USER_STREAM); - if (i <= 0) + if (i <= 0) { + memset(scn, 0, sizeof(*scn)); return -1; + } d = &dir->dir_tab[i - 1]; return cdf_read_sector_chain(info, h, sat, ssat, sst, @@ -824,7 +851,7 @@ cdf_read_property_info(const cdf_stream_ malloc(*maxcount * sizeof(*inp))); } if (inp == NULL) - goto out; + goto out1; *info = inp; inp += *count; *count += sh.sh_properties; @@ -931,7 +958,7 @@ cdf_read_property_info(const cdf_stream_ inp = CAST(cdf_property_info_t *, realloc(*info, *maxcount * sizeof(*inp))); if (inp == NULL) - goto out; + goto out1; *info = inp; inp = *info + nelem; } @@ -976,6 +1003,8 @@ cdf_read_property_info(const cdf_stream_ } return 0; out: + errno = EFTYPE; +out1: free(*info); return -1; } @@ -1022,8 +1051,7 @@ int cdf_unpack_catalog(const cdf_header_t *h, const cdf_stream_t *sst, cdf_catalog_t **cat) { - size_t ss = sst->sst_dirlen < h->h_min_size_standard_stream ? - CDF_SHORT_SEC_SIZE(h) : CDF_SEC_SIZE(h); + size_t ss = cdf_check_stream(sst, h); const char *b = CAST(const char *, sst->sst_tab); const char *eb = b + ss * sst->sst_len; size_t nr, i, j, k; @@ -1043,6 +1071,8 @@ cdf_unpack_catalog(const cdf_header_t *h nr--; *cat = CAST(cdf_catalog_t *, malloc(sizeof(cdf_catalog_t) + nr * sizeof(*ce))); + if (*cat == NULL) + return -1; ce = (*cat)->cat_e; memset(ce, 0, nr * sizeof(*ce)); b = CAST(const char *, sst->sst_tab); @@ -1245,8 +1275,7 @@ cdf_dump(const void *v, size_t len) void cdf_dump_stream(const cdf_header_t *h, const cdf_stream_t *sst) { - size_t ss = sst->sst_dirlen < h->h_min_size_standard_stream ? - CDF_SHORT_SEC_SIZE(h) : CDF_SEC_SIZE(h); + size_t ss = sst->sst_ss; cdf_dump(sst->sst_tab, ss * sst->sst_len); } Modified: vendor/file/dist/src/cdf.h ============================================================================== --- vendor/file/dist/src/cdf.h Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/src/cdf.h Sat May 14 05:25:47 2016 (r299716) @@ -129,6 +129,7 @@ typedef struct { void *sst_tab; size_t sst_len; size_t sst_dirlen; + size_t sst_ss; } cdf_stream_t; typedef struct { @@ -277,7 +278,7 @@ typedef struct { typedef struct { size_t cat_num; - cdf_catalog_entry_t cat_e[0]; + cdf_catalog_entry_t cat_e[1]; } cdf_catalog_t; struct timespec; Modified: vendor/file/dist/src/compress.c ============================================================================== --- vendor/file/dist/src/compress.c Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/src/compress.c Sat May 14 05:25:47 2016 (r299716) @@ -35,7 +35,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: compress.c,v 1.93 2016/03/31 17:51:12 christos Exp $") +FILE_RCSID("@(#)$File: compress.c,v 1.96 2016/04/20 00:00:26 christos Exp $") #endif #include "magic.h" @@ -187,7 +187,7 @@ file_zmagic(struct magic_set *ms, int fd size_t i, nsz; char *rbuf; file_pushbuf_t *pb; - int rv = 0; + int urv, prv, rv = 0; int mime = ms->flags & MAGIC_MIME; #ifdef HAVE_SIGNAL_H sig_t osigpipe; @@ -214,22 +214,22 @@ file_zmagic(struct magic_set *ms, int fd if (!zm) continue; nsz = nbytes; - rv = uncompressbuf(fd, ms->bytes_max, i, buf, &newbuf, &nsz); - DPRINTF("uncompressbuf = %d, %s, %zu\n", rv, (char *)newbuf, + urv = uncompressbuf(fd, ms->bytes_max, i, buf, &newbuf, &nsz); + DPRINTF("uncompressbuf = %d, %s, %zu\n", urv, (char *)newbuf, nsz); - switch (rv) { + switch (urv) { case OKDATA: case ERRDATA: ms->flags &= ~MAGIC_COMPRESS; - if (rv == ERRDATA) - rv = file_printf(ms, "%s ERROR: %s", + if (urv == ERRDATA) + prv = file_printf(ms, "%s ERROR: %s", methodname(i), newbuf); else - rv = file_buffer(ms, -1, name, newbuf, nsz); - if (rv == -1) + prv = file_buffer(ms, -1, name, newbuf, nsz); + if (prv == -1) goto error; - DPRINTF("rv = %d\n", rv); + rv = 1; if ((ms->flags & MAGIC_COMPRESS_TRANSP) != 0) goto out; if (mime != MAGIC_MIME && mime != 0) @@ -239,6 +239,10 @@ file_zmagic(struct magic_set *ms, int fd goto error; if ((pb = file_push_buffer(ms)) == NULL) goto error; + /* + * XXX: If file_buffer fails here, we overwrite + * the compressed text. FIXME. + */ if (file_buffer(ms, -1, NULL, buf, nbytes) == -1) goto error; if ((rbuf = file_pop_buffer(ms, pb)) != NULL) { @@ -250,16 +254,20 @@ file_zmagic(struct magic_set *ms, int fd } if (!mime && file_printf(ms, ")") == -1) goto error; - goto out; + /*FALLTHROUGH*/ case NODATA: - goto out; + break; default: abort(); + /*NOTREACHED*/ + error: + rv = -1; + break; } } out: - rv = 1; -error: + DPRINTF("rv = %d\n", rv); + #ifdef HAVE_SIGNAL_H (void)signal(SIGPIPE, osigpipe); #endif Modified: vendor/file/dist/src/der.c ============================================================================== --- vendor/file/dist/src/der.c Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/src/der.c Sat May 14 05:25:47 2016 (r299716) @@ -35,7 +35,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: der.c,v 1.4 2016/03/21 23:04:40 christos Exp $") +FILE_RCSID("@(#)$File: der.c,v 1.6 2016/04/21 14:26:03 christos Exp $") #endif #endif @@ -44,7 +44,6 @@ FILE_RCSID("@(#)$File: der.c,v 1.4 2016/ #include #include -#include #include #include #include @@ -53,6 +52,8 @@ FILE_RCSID("@(#)$File: der.c,v 1.4 2016/ #ifndef TEST_DER #include "magic.h" #include "der.h" +#else +#include #endif #define DER_BAD ((uint32_t)-1) @@ -61,11 +62,15 @@ FILE_RCSID("@(#)$File: der.c,v 1.4 2016/ #define DER_CLASS_APPLICATION 1 #define DER_CLASS_CONTEXT 2 #define DER_CLASS_PRIVATE 3 +#ifdef DEBUG_DER static const char der_class[] = "UACP"; +#endif #define DER_TYPE_PRIMITIVE 0 #define DER_TYPE_CONSTRUCTED 1 +#ifdef DEBUG_DER static const char der_type[] = "PC"; +#endif #define DER_TAG_EOC 0x00 #define DER_TAG_BOOLEAN 0x01 Modified: vendor/file/dist/src/magic.c ============================================================================== --- vendor/file/dist/src/magic.c Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/src/magic.c Sat May 14 05:25:47 2016 (r299716) @@ -33,7 +33,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: magic.c,v 1.97 2016/03/31 17:51:12 christos Exp $") +FILE_RCSID("@(#)$File: magic.c,v 1.99 2016/05/03 16:09:38 christos Exp $") #endif /* lint */ #include "magic.h" @@ -523,9 +523,11 @@ file_or_fd(struct magic_set *ms, const c rv = 0; done: free(buf); - if (pos != (off_t)-1) - (void)lseek(fd, pos, SEEK_SET); - close_and_restore(ms, inname, fd, &sb); + if (fd != -1) { + if (pos != (off_t)-1) + (void)lseek(fd, pos, SEEK_SET); + close_and_restore(ms, inname, fd, &sb); + } out: return rv == 0 ? file_getbuffer(ms) : NULL; } Modified: vendor/file/dist/src/magic.h ============================================================================== --- vendor/file/dist/src/magic.h Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/src/magic.h Sat May 14 05:25:47 2016 (r299716) @@ -80,7 +80,7 @@ #define MAGIC_NO_CHECK_FORTRAN 0x000000 /* Don't check ascii/fortran */ #define MAGIC_NO_CHECK_TROFF 0x000000 /* Don't check ascii/troff */ -#define MAGIC_VERSION 525 /* This implementation */ +#define MAGIC_VERSION 526 /* This implementation */ #ifdef __cplusplus @@ -114,7 +114,7 @@ int magic_errno(magic_t); #define MAGIC_PARAM_ELF_SHNUM_MAX 3 #define MAGIC_PARAM_ELF_NOTES_MAX 4 #define MAGIC_PARAM_REGEX_MAX 5 -#define MAGIC_PARAM_BYTES_MAX 6 +#define MAGIC_PARAM_BYTES_MAX 6 int magic_setparam(magic_t, int, const void *); int magic_getparam(magic_t, int, void *); Modified: vendor/file/dist/src/magic.h.in ============================================================================== --- vendor/file/dist/src/magic.h.in Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/src/magic.h.in Sat May 14 05:25:47 2016 (r299716) @@ -114,6 +114,7 @@ int magic_errno(magic_t); #define MAGIC_PARAM_ELF_SHNUM_MAX 3 #define MAGIC_PARAM_ELF_NOTES_MAX 4 #define MAGIC_PARAM_REGEX_MAX 5 +#define MAGIC_PARAM_BYTES_MAX 6 int magic_setparam(magic_t, int, const void *); int magic_getparam(magic_t, int, void *); Modified: vendor/file/dist/src/readcdf.c ============================================================================== --- vendor/file/dist/src/readcdf.c Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/src/readcdf.c Sat May 14 05:25:47 2016 (r299716) @@ -26,7 +26,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: readcdf.c,v 1.56 2016/03/03 22:20:03 christos Exp $") +FILE_RCSID("@(#)$File: readcdf.c,v 1.57 2016/05/03 16:08:49 christos Exp $") #endif #include @@ -373,7 +373,7 @@ cdf_file_catalog_info(struct magic_set * dir, "Catalog", scn)) == -1) return i; #ifdef CDF_DEBUG - cdf_dump_catalog(&h, &scn); + cdf_dump_catalog(&h, scn); #endif if ((i = cdf_file_catalog(ms, h, scn)) == -1) return -1; Modified: vendor/file/dist/src/softmagic.c ============================================================================== --- vendor/file/dist/src/softmagic.c Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/src/softmagic.c Sat May 14 05:25:47 2016 (r299716) @@ -32,7 +32,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: softmagic.c,v 1.230 2016/04/18 15:10:34 christos Exp $") +FILE_RCSID("@(#)$File: softmagic.c,v 1.231 2016/04/21 15:23:31 christos Exp $") #endif /* lint */ #include "magic.h" Modified: vendor/file/dist/tests/Makefile.am ============================================================================== --- vendor/file/dist/tests/Makefile.am Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/tests/Makefile.am Sat May 14 05:25:47 2016 (r299716) @@ -13,4 +13,4 @@ issue311docx.testfile T = $(top_srcdir)/tests check-local: MAGIC=$(top_builddir)/magic/magic ./test - for i in $T/*.testfile; do echo Running test: $$i; MAGIC=$(top_builddir)/magic/magic ./test $$i $${i%%.testfile}.result; done + set -e; for i in $T/*.testfile; do echo Running test: $$i; TZ=UTC MAGIC=$(top_builddir)/magic/magic ./test $$i $${i%%.testfile}.result; done Modified: vendor/file/dist/tests/Makefile.in ============================================================================== --- vendor/file/dist/tests/Makefile.in Sat May 14 05:00:17 2016 (r299715) +++ vendor/file/dist/tests/Makefile.in Sat May 14 05:25:47 2016 (r299716) @@ -608,7 +608,7 @@ uninstall-am: check-local: MAGIC=$(top_builddir)/magic/magic ./test - for i in $T/*.testfile; do echo Running test: $$i; MAGIC=$(top_builddir)/magic/magic ./test $$i $${i%%.testfile}.result; done + set -e; for i in $T/*.testfile; do echo Running test: $$i; TZ=UTC MAGIC=$(top_builddir)/magic/magic ./test $$i $${i%%.testfile}.result; done # Tell versions [3.59,3.63) of GNU make to not export all variables. # Otherwise a system limit (for SysV at least) may be exceeded. From owner-svn-src-vendor@freebsd.org Sat May 14 05:27:09 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C3CDB3AB3E; Sat, 14 May 2016 05:27:09 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D9BB5119E; Sat, 14 May 2016 05:27:08 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u4E5R8Hu054878; Sat, 14 May 2016 05:27:08 GMT (envelope-from delphij@FreeBSD.org) Received: (from delphij@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u4E5R8vg054877; Sat, 14 May 2016 05:27:08 GMT (envelope-from delphij@FreeBSD.org) Message-Id: <201605140527.u4E5R8vg054877@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: delphij set sender to delphij@FreeBSD.org using -f From: Xin LI Date: Sat, 14 May 2016 05:27:08 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r299717 - vendor/file/5.27 X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 May 2016 05:27:09 -0000 Author: delphij Date: Sat May 14 05:27:07 2016 New Revision: 299717 URL: https://svnweb.freebsd.org/changeset/base/299717 Log: Tag file 5.27. Added: vendor/file/5.27/ - copied from r299716, vendor/file/dist/ From owner-svn-src-vendor@freebsd.org Sat May 14 10:08:07 2016 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1CF88B3A7B9; Sat, 14 May 2016 10:08:07 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9C26019B7; Sat, 14 May 2016 10:08:06 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1b1WUI-000BXm-Ja; Sat, 14 May 2016 13:07:58 +0300 Date: Sat, 14 May 2016 13:07:58 +0300 From: Slawa Olhovchenkov To: Xin LI Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: Re: svn commit: r299716 - in vendor/file/dist: . magic/Magdir src tests Message-ID: <20160514100758.GB39874@zxy.spb.ru> References: <201605140525.u4E5Pl9V054742@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201605140525.u4E5Pl9V054742@repo.freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 May 2016 10:08:07 -0000 On Sat, May 14, 2016 at 05:25:47AM +0000, Xin LI wrote: > Author: delphij > Date: Sat May 14 05:25:47 2016 > New Revision: 299716 > URL: https://svnweb.freebsd.org/changeset/base/299716 > > Log: > Vendor import of file 4.27. 4.27? > Modified: > vendor/file/dist/ChangeLog > vendor/file/dist/configure > vendor/file/dist/configure.ac > vendor/file/dist/magic/Magdir/archive > vendor/file/dist/magic/Magdir/commands > vendor/file/dist/magic/Magdir/console > vendor/file/dist/magic/Magdir/os2 > vendor/file/dist/magic/Magdir/xenix > vendor/file/dist/src/ascmagic.c > vendor/file/dist/src/cdf.c > vendor/file/dist/src/cdf.h > vendor/file/dist/src/compress.c > vendor/file/dist/src/der.c > vendor/file/dist/src/magic.c > vendor/file/dist/src/magic.h > vendor/file/dist/src/magic.h.in > vendor/file/dist/src/readcdf.c > vendor/file/dist/src/softmagic.c > vendor/file/dist/tests/Makefile.am > vendor/file/dist/tests/Makefile.in > > Modified: vendor/file/dist/ChangeLog > ============================================================================== > --- vendor/file/dist/ChangeLog Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/ChangeLog Sat May 14 05:25:47 2016 (r299716) > @@ -1,6 +1,19 @@ > +2016-05-13 12:00 Christos Zoulas > + > + * release 5.27 > + > +2016-04-18 9:35 Christos Zoulas > + > + * Errors comparing DER entries or computing offsets > + are just indications of malformed non-DER files. > + Don't print them. > + * Offset comparison was off-by-one. > + * Fix compression code (Werner Fink) > + * Put new bytes constant in the right file (not the generated one) > + > 2016-04-16 18:34 Christos Zoulas > > - * release 5.25 > + * release 5.26 > > 2016-03-31 13:50 Christos Zoulas > > > Modified: vendor/file/dist/configure > ============================================================================== > --- vendor/file/dist/configure Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/configure Sat May 14 05:25:47 2016 (r299716) > @@ -1,6 +1,6 @@ > #! /bin/sh > # Guess values for system-dependent variables and create Makefiles. > -# Generated by GNU Autoconf 2.69 for file 5.26. > +# Generated by GNU Autoconf 2.69 for file 5.27. > # > # Report bugs to . > # > @@ -590,8 +590,8 @@ MAKEFLAGS= > # Identity of this package. > PACKAGE_NAME='file' > PACKAGE_TARNAME='file' > -PACKAGE_VERSION='5.26' > -PACKAGE_STRING='file 5.26' > +PACKAGE_VERSION='5.27' > +PACKAGE_STRING='file 5.27' > PACKAGE_BUGREPORT='christos@astron.com' > PACKAGE_URL='' > > @@ -1327,7 +1327,7 @@ if test "$ac_init_help" = "long"; then > # Omit some internal or obsolete options to make the list less imposing. > # This message is too long to be a string in the A/UX 3.1 sh. > cat <<_ACEOF > -\`configure' configures file 5.26 to adapt to many kinds of systems. > +\`configure' configures file 5.27 to adapt to many kinds of systems. > > Usage: $0 [OPTION]... [VAR=VALUE]... > > @@ -1397,7 +1397,7 @@ fi > > if test -n "$ac_init_help"; then > case $ac_init_help in > - short | recursive ) echo "Configuration of file 5.26:";; > + short | recursive ) echo "Configuration of file 5.27:";; > esac > cat <<\_ACEOF > > @@ -1507,7 +1507,7 @@ fi > test -n "$ac_init_help" && exit $ac_status > if $ac_init_version; then > cat <<\_ACEOF > -file configure 5.26 > +file configure 5.27 > generated by GNU Autoconf 2.69 > > Copyright (C) 2012 Free Software Foundation, Inc. > @@ -2163,7 +2163,7 @@ cat >config.log <<_ACEOF > This file contains any messages produced by compilers while > running configure, to aid debugging if configure makes a mistake. > > -It was created by file $as_me 5.26, which was > +It was created by file $as_me 5.27, which was > generated by GNU Autoconf 2.69. Invocation command line was > > $ $0 $@ > @@ -3029,7 +3029,7 @@ fi > > # Define the identity of the package. > PACKAGE='file' > - VERSION='5.26' > + VERSION='5.27' > > > cat >>confdefs.h <<_ACEOF > @@ -15049,7 +15049,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_wri > # report actual input values of CONFIG_FILES etc. instead of their > # values after options handling. > ac_log=" > -This file was extended by file $as_me 5.26, which was > +This file was extended by file $as_me 5.27, which was > generated by GNU Autoconf 2.69. Invocation command line was > > CONFIG_FILES = $CONFIG_FILES > @@ -15115,7 +15115,7 @@ _ACEOF > cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 > ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" > ac_cs_version="\\ > -file config.status 5.26 > +file config.status 5.27 > configured by $0, generated by GNU Autoconf 2.69, > with options \\"\$ac_cs_config\\" > > > Modified: vendor/file/dist/configure.ac > ============================================================================== > --- vendor/file/dist/configure.ac Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/configure.ac Sat May 14 05:25:47 2016 (r299716) > @@ -1,5 +1,5 @@ > dnl Process this file with autoconf to produce a configure script. > -AC_INIT([file],[5.26],[christos@astron.com]) > +AC_INIT([file],[5.27],[christos@astron.com]) > AM_INIT_AUTOMAKE([subdir-objects foreign]) > m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) > > > Modified: vendor/file/dist/magic/Magdir/archive > ============================================================================== > --- vendor/file/dist/magic/Magdir/archive Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/magic/Magdir/archive Sat May 14 05:25:47 2016 (r299716) > @@ -1,5 +1,5 @@ > #------------------------------------------------------------------------------ > -# $File: archive,v 1.102 2016/01/11 20:59:24 christos Exp $ > +# $File: archive,v 1.103 2016/05/05 17:07:40 christos Exp $ > # archive: file(1) magic for archive formats (see also "msdos" for self- > # extracting compressed archives) > # > @@ -891,6 +891,14 @@ > >(26.s+30) leshort 0xcafe Java archive data (JAR) > !:mime application/java-archive > > +# iOS App > +>(26.s+30) leshort !0xcafe > +>>26 string !\x8\0\0\0mimetype > +>>>30 string Payload/ > +>>>>38 search/64 .app/ iOS App > +!:mime application/x-ios-app > + > + > # Generic zip archives (Greg Roelofs, c/o zip-bugs@wkuvx1.wku.edu) > # Next line excludes specialized formats: > >(26.s+30) leshort !0xcafe > > Modified: vendor/file/dist/magic/Magdir/commands > ============================================================================== > --- vendor/file/dist/magic/Magdir/commands Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/magic/Magdir/commands Sat May 14 05:25:47 2016 (r299716) > @@ -1,6 +1,6 @@ > > #------------------------------------------------------------------------------ > -# $File: commands,v 1.53 2016/02/23 12:35:20 christos Exp $ > +# $File: commands,v 1.54 2016/04/19 13:40:02 christos Exp $ > # commands: file(1) magic for various shells and interpreters > # > #0 string/w : shell archive or script for antique kernel text > @@ -101,7 +101,7 @@ > 0 string = >5 regex [\ \n] > >>6 string /*\ Smarty\ version Smarty compiled template > ->24 regex [0-9.]+ \b, version %s > +>>>24 regex [0-9.]+ \b, version %s > !:mime text/x-php > > 0 string Zend\x00 PHP script Zend Optimizer data > > Modified: vendor/file/dist/magic/Magdir/console > ============================================================================== > --- vendor/file/dist/magic/Magdir/console Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/magic/Magdir/console Sat May 14 05:25:47 2016 (r299716) > @@ -1,6 +1,6 @@ > > #------------------------------------------------------------------------------ > -# $File: console,v 1.24 2016/03/23 15:29:20 christos Exp $ > +# $File: console,v 1.25 2016/04/18 20:22:10 christos Exp $ > # Console game magic > # Toby Deshane > > @@ -53,7 +53,7 @@ > # so most of the data isn't easily parseable. > # > 0 string UNIF > -4 lelong <16 UNIF v%d format NES ROM image > +>4 lelong <16 UNIF v%d format NES ROM image > > #------------------------------------------------------------------------------ > # gameboy: file(1) magic for the Nintendo (Color) Gameboy raw ROM format > > Modified: vendor/file/dist/magic/Magdir/os2 > ============================================================================== > --- vendor/file/dist/magic/Magdir/os2 Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/magic/Magdir/os2 Sat May 14 05:25:47 2016 (r299716) > @@ -1,12 +1,12 @@ > > #------------------------------------------------------------------------------ > -# $File: os2,v 1.8 2015/01/05 00:17:13 christos Exp $ > +# $File: os2,v 1.9 2016/05/11 15:51:57 christos Exp $ > # os2: file(1) magic for OS/2 files > # > > # Provided 1998/08/22 by > # David Mediavilla > -1 search/1 InternetShortcut MS Windows 95 Internet shortcut text > +1 search/100 InternetShortcut MS Windows 95 Internet shortcut text > >17 search/100 URL= (URL=< > >>&0 string x \b%s>) > > > Modified: vendor/file/dist/magic/Magdir/xenix > ============================================================================== > --- vendor/file/dist/magic/Magdir/xenix Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/magic/Magdir/xenix Sat May 14 05:25:47 2016 (r299716) > @@ -1,6 +1,6 @@ > > #------------------------------------------------------------------------------ > -# $File: xenix,v 1.9 2009/09/19 16:28:13 christos Exp $ > +# $File: xenix,v 1.10 2016/04/19 18:14:19 christos Exp $ > # xenix: file(1) magic for Microsoft Xenix > # > # "Middle model" stuff, and "Xenix 8086 relocatable or 80286 small > @@ -12,7 +12,26 @@ > # XXX - "x.out" collides with PDP-11 archives > # > 0 string core core file (Xenix) > -0 byte 0x80 8086 relocatable (Microsoft) > +# URL: http://www.polarhome.com/service/man/?qf=86rel&tf=2&of=Xenix > +# Reference: http://www.azillionmonkeys.com/qed/Omfg.pdf > +# Update: Joerg Jenderek > +# recordtype~TranslatorHEADerRecord > +0 byte 0x80 > +# GRR: line above is too general as it catches also Extensible storage engine DataBase > +# skip examples like GENA.SND Switch.Snd by looking for record length maximal 1024-3 > +>1 uleshort <1022 > +# skip examples like GAME.PICTURE Strange.Pic by looking for positiv record length > +>>1 uleshort >0 > +# skip examples like Xtable.Data FRACTAL.GEN SHR.VIEW by looking for positiv string length > +>>>3 ubyte >0 > +# skip examples like OMBRE.6 with "UUUUUU" by looking for filename like "hello.c" > +>>>>4 regex [a-zA-Z_/]{1,8}[.] 8086 relocatable (Microsoft) > +#!:mime application/octet-stream > +!:mime application/x-object > +!:ext o/a > +>>>>>3 pstring x \b, "%s" > +# checksum > +#>>>>>(3.b+4) ubyte x \b, checksum 0x%2.2x > 0 leshort 0xff65 x.out > >2 string __.SYMDEF randomized > >0 byte x archive > > Modified: vendor/file/dist/src/ascmagic.c > ============================================================================== > --- vendor/file/dist/src/ascmagic.c Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/src/ascmagic.c Sat May 14 05:25:47 2016 (r299716) > @@ -35,7 +35,7 @@ > #include "file.h" > > #ifndef lint > -FILE_RCSID("@(#)$File: ascmagic.c,v 1.94 2016/03/31 17:51:12 christos Exp $") > +FILE_RCSID("@(#)$File: ascmagic.c,v 1.95 2016/05/03 16:10:37 christos Exp $") > #endif /* lint */ > > #include "magic.h" > @@ -79,9 +79,6 @@ file_ascmagic(struct magic_set *ms, cons > const char *code_mime = NULL; > const char *type = NULL; > > - if (ms->flags & (MAGIC_APPLE|MAGIC_EXTENSION)) > - return 0; > - > nbytes = trim_nuls(buf, nbytes); > > /* If file doesn't look like any sort of text, give up. */ > @@ -123,9 +120,6 @@ file_ascmagic_with_encoding(struct magic > size_t last_line_end = (size_t)-1; > int has_long_lines = 0; > > - if (ms->flags & (MAGIC_APPLE|MAGIC_EXTENSION)) > - return 0; > - > nbytes = trim_nuls(buf, nbytes); > > /* If we have fewer than 2 bytes, give up. */ > @@ -150,7 +144,11 @@ file_ascmagic_with_encoding(struct magic > (size_t)(utf8_end - utf8_buf), NULL, NULL, > TEXTTEST, text)) == 0) > rv = -1; > + if ((ms->flags & (MAGIC_APPLE|MAGIC_EXTENSION))) > + return rv == -1 ? 0 : 1; > } > + if ((ms->flags & (MAGIC_APPLE|MAGIC_EXTENSION))) > + return 0; > > /* Now try to discover other details about the file. */ > for (i = 0; i < ulen; i++) { > > Modified: vendor/file/dist/src/cdf.c > ============================================================================== > --- vendor/file/dist/src/cdf.c Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/src/cdf.c Sat May 14 05:25:47 2016 (r299716) > @@ -35,7 +35,7 @@ > #include "file.h" > > #ifndef lint > -FILE_RCSID("@(#)$File: cdf.c,v 1.76 2015/02/28 00:18:02 christos Exp $") > +FILE_RCSID("@(#)$File: cdf.c,v 1.80 2016/05/06 15:17:10 christos Exp $") > #endif > > #include > @@ -267,13 +267,32 @@ cdf_unpack_dir(cdf_directory_t *d, char > } > > static int > +cdf_zero_stream(cdf_stream_t *scn) > +{ > + scn->sst_len = 0; > + scn->sst_dirlen = 0; > + scn->sst_ss = 0; > + free(scn->sst_tab); > + scn->sst_tab = NULL; > + return -1; > +} > + > +static size_t > +cdf_check_stream(const cdf_stream_t *sst, const cdf_header_t *h) > +{ > + size_t ss = sst->sst_dirlen < h->h_min_size_standard_stream ? > + CDF_SHORT_SEC_SIZE(h) : CDF_SEC_SIZE(h); > + assert(ss == sst->sst_ss); > + return sst->sst_ss; > +} > + > +static int > cdf_check_stream_offset(const cdf_stream_t *sst, const cdf_header_t *h, > const void *p, size_t tail, int line) > { > const char *b = (const char *)sst->sst_tab; > const char *e = ((const char *)p) + tail; > - size_t ss = sst->sst_dirlen < h->h_min_size_standard_stream ? > - CDF_SHORT_SEC_SIZE(h) : CDF_SEC_SIZE(h); > + size_t ss = cdf_check_stream(sst, h); > /*LINTED*/(void)&line; > if (e >= b && (size_t)(e - b) <= ss * sst->sst_len) > return 0; > @@ -290,10 +309,8 @@ cdf_read(const cdf_info_t *info, off_t o > { > size_t siz = (size_t)off + len; > > - if ((off_t)(off + len) != (off_t)siz) { > - errno = EINVAL; > - return -1; > - } > + if ((off_t)(off + len) != (off_t)siz) > + goto out; > > if (info->i_buf != NULL && info->i_len >= siz) { > (void)memcpy(buf, &info->i_buf[off], len); > @@ -301,12 +318,15 @@ cdf_read(const cdf_info_t *info, off_t o > } > > if (info->i_fd == -1) > - return -1; > + goto out; > > if (pread(info->i_fd, buf, len, off) != (ssize_t)len) > return -1; > > return (ssize_t)len; > +out: > + errno = EINVAL; > + return -1; > } > > int > @@ -363,11 +383,14 @@ cdf_read_short_sector(const cdf_stream_t > DPRINTF(("Out of bounds read %" SIZE_T_FORMAT "u > %" > SIZE_T_FORMAT "u\n", > pos + len, CDF_SEC_SIZE(h) * sst->sst_len)); > - return -1; > + goto out; > } > (void)memcpy(((char *)buf) + offs, > ((const char *)sst->sst_tab) + pos, len); > return len; > +out: > + errno = EFTYPE; > + return -1; > } > > /* > @@ -421,8 +444,7 @@ cdf_read_sat(const cdf_info_t *info, cdf > goto out; > if (j >= CDF_LOOP_LIMIT) { > DPRINTF(("Reading master sector loop limit")); > - errno = EFTYPE; > - goto out2; > + goto out3; > } > if (cdf_read_sector(info, msa, 0, ss, h, mid) != (ssize_t)ss) { > DPRINTF(("Reading master sector %d", mid)); > @@ -435,8 +457,7 @@ cdf_read_sat(const cdf_info_t *info, cdf > if (i >= sat->sat_len) { > DPRINTF(("Out of bounds reading MSA %" SIZE_T_FORMAT > "u >= %" SIZE_T_FORMAT "u", i, sat->sat_len)); > - errno = EFTYPE; > - goto out2; > + goto out3; > } > if (cdf_read_sector(info, sat->sat_tab, ss * i, ss, h, > sec) != (ssize_t)ss) { > @@ -451,6 +472,8 @@ out: > sat->sat_len = i; > free(msa); > return 0; > +out3: > + errno = EFTYPE; > out2: > free(msa); > out1: > @@ -476,23 +499,24 @@ cdf_count_chain(const cdf_sat_t *sat, cd > DPRINTF((" %d", sid)); > if (j >= CDF_LOOP_LIMIT) { > DPRINTF(("Counting chain loop limit")); > - errno = EFTYPE; > - return (size_t)-1; > + goto out; > } > if (sid >= maxsector) { > DPRINTF(("Sector %d >= %d\n", sid, maxsector)); > - errno = EFTYPE; > - return (size_t)-1; > + goto out; > } > sid = CDF_TOLE4((uint32_t)sat->sat_tab[sid]); > } > if (i == 0) { > DPRINTF((" none, sid: %d\n", sid)); > - return (size_t)-1; > + goto out; > > } > DPRINTF(("\n")); > return i; > +out: > + errno = EFTYPE; > + return (size_t)-1; > } > > int > @@ -501,27 +525,27 @@ cdf_read_long_sector_chain(const cdf_inf > { > size_t ss = CDF_SEC_SIZE(h), i, j; > ssize_t nr; > + scn->sst_tab = NULL; > scn->sst_len = cdf_count_chain(sat, sid, ss); > scn->sst_dirlen = len; > + scn->sst_ss = ss; > > if (scn->sst_len == (size_t)-1) > - return -1; > + goto out; > > scn->sst_tab = calloc(scn->sst_len, ss); > if (scn->sst_tab == NULL) > - return -1; > + return cdf_zero_stream(scn); > > for (j = i = 0; sid >= 0; i++, j++) { > if (j >= CDF_LOOP_LIMIT) { > DPRINTF(("Read long sector chain loop limit")); > - errno = EFTYPE; > goto out; > } > if (i >= scn->sst_len) { > DPRINTF(("Out of bounds reading long sector chain " > "%" SIZE_T_FORMAT "u > %" SIZE_T_FORMAT "u\n", i, > scn->sst_len)); > - errno = EFTYPE; > goto out; > } > if ((nr = cdf_read_sector(info, scn->sst_tab, i * ss, ss, h, > @@ -537,8 +561,8 @@ cdf_read_long_sector_chain(const cdf_inf > } > return 0; > out: > - free(scn->sst_tab); > - return -1; > + errno = EFTYPE; > + return cdf_zero_stream(scn); > } > > int > @@ -547,27 +571,27 @@ cdf_read_short_sector_chain(const cdf_he > cdf_secid_t sid, size_t len, cdf_stream_t *scn) > { > size_t ss = CDF_SHORT_SEC_SIZE(h), i, j; > - scn->sst_len = cdf_count_chain(ssat, sid, CDF_SEC_SIZE(h)); > + scn->sst_tab = NULL; > + scn->sst_len = cdf_count_chain(ssat, sid, ss); > scn->sst_dirlen = len; > + scn->sst_ss = ss; > > if (sst->sst_tab == NULL || scn->sst_len == (size_t)-1) > - return -1; > + goto out; > > scn->sst_tab = calloc(scn->sst_len, ss); > if (scn->sst_tab == NULL) > - return -1; > + return cdf_zero_stream(scn); > > for (j = i = 0; sid >= 0; i++, j++) { > if (j >= CDF_LOOP_LIMIT) { > DPRINTF(("Read short sector chain loop limit")); > - errno = EFTYPE; > goto out; > } > if (i >= scn->sst_len) { > DPRINTF(("Out of bounds reading short sector chain " > "%" SIZE_T_FORMAT "u > %" SIZE_T_FORMAT "u\n", > i, scn->sst_len)); > - errno = EFTYPE; > goto out; > } > if (cdf_read_short_sector(sst, scn->sst_tab, i * ss, ss, h, > @@ -579,8 +603,8 @@ cdf_read_short_sector_chain(const cdf_he > } > return 0; > out: > - free(scn->sst_tab); > - return -1; > + errno = EFTYPE; > + return cdf_zero_stream(scn); > } > > int > @@ -625,7 +649,6 @@ cdf_read_dir(const cdf_info_t *info, con > for (j = i = 0; i < ns; i++, j++) { > if (j >= CDF_LOOP_LIMIT) { > DPRINTF(("Read dir loop limit")); > - errno = EFTYPE; > goto out; > } > if (cdf_read_sector(info, buf, 0, ss, h, sid) != (ssize_t)ss) { > @@ -646,6 +669,7 @@ cdf_read_dir(const cdf_info_t *info, con > out: > free(dir->dir_tab); > free(buf); > + errno = EFTYPE; > return -1; > } > > @@ -658,36 +682,37 @@ cdf_read_ssat(const cdf_info_t *info, co > size_t ss = CDF_SEC_SIZE(h); > cdf_secid_t sid = h->h_secid_first_sector_in_short_sat; > > + ssat->sat_tab = NULL; > ssat->sat_len = cdf_count_chain(sat, sid, CDF_SEC_SIZE(h)); > if (ssat->sat_len == (size_t)-1) > - return -1; > + goto out; > > ssat->sat_tab = CAST(cdf_secid_t *, calloc(ssat->sat_len, ss)); > if (ssat->sat_tab == NULL) > - return -1; > + goto out1; > > for (j = i = 0; sid >= 0; i++, j++) { > if (j >= CDF_LOOP_LIMIT) { > DPRINTF(("Read short sat sector loop limit")); > - errno = EFTYPE; > goto out; > } > if (i >= ssat->sat_len) { > DPRINTF(("Out of bounds reading short sector chain " > "%" SIZE_T_FORMAT "u > %" SIZE_T_FORMAT "u\n", i, > ssat->sat_len)); > - errno = EFTYPE; > goto out; > } > if (cdf_read_sector(info, ssat->sat_tab, i * ss, ss, h, sid) != > (ssize_t)ss) { > DPRINTF(("Reading short sat sector %d", sid)); > - goto out; > + goto out1; > } > sid = CDF_TOLE4((uint32_t)sat->sat_tab[sid]); > } > return 0; > out: > + errno = EFTYPE; > +out1: > free(ssat->sat_tab); > return -1; > } > @@ -715,13 +740,13 @@ cdf_read_short_stream(const cdf_info_t * > if (d->d_stream_first_sector < 0) > goto out; > > - return cdf_read_long_sector_chain(info, h, sat, > + return cdf_read_long_sector_chain(info, h, sat, > d->d_stream_first_sector, d->d_size, scn); > out: > scn->sst_tab = NULL; > - scn->sst_len = 0; > - scn->sst_dirlen = 0; > - return 0; > + (void)cdf_zero_stream(scn); > + errno = EFTYPE; > + return -1; > } > > static int > @@ -750,8 +775,10 @@ cdf_read_user_stream(const cdf_info_t *i > const cdf_directory_t *d; > int i = cdf_find_stream(dir, name, CDF_DIR_TYPE_USER_STREAM); > > - if (i <= 0) > + if (i <= 0) { > + memset(scn, 0, sizeof(*scn)); > return -1; > + } > > d = &dir->dir_tab[i - 1]; > return cdf_read_sector_chain(info, h, sat, ssat, sst, > @@ -824,7 +851,7 @@ cdf_read_property_info(const cdf_stream_ > malloc(*maxcount * sizeof(*inp))); > } > if (inp == NULL) > - goto out; > + goto out1; > *info = inp; > inp += *count; > *count += sh.sh_properties; > @@ -931,7 +958,7 @@ cdf_read_property_info(const cdf_stream_ > inp = CAST(cdf_property_info_t *, > realloc(*info, *maxcount * sizeof(*inp))); > if (inp == NULL) > - goto out; > + goto out1; > *info = inp; > inp = *info + nelem; > } > @@ -976,6 +1003,8 @@ cdf_read_property_info(const cdf_stream_ > } > return 0; > out: > + errno = EFTYPE; > +out1: > free(*info); > return -1; > } > @@ -1022,8 +1051,7 @@ int > cdf_unpack_catalog(const cdf_header_t *h, const cdf_stream_t *sst, > cdf_catalog_t **cat) > { > - size_t ss = sst->sst_dirlen < h->h_min_size_standard_stream ? > - CDF_SHORT_SEC_SIZE(h) : CDF_SEC_SIZE(h); > + size_t ss = cdf_check_stream(sst, h); > const char *b = CAST(const char *, sst->sst_tab); > const char *eb = b + ss * sst->sst_len; > size_t nr, i, j, k; > @@ -1043,6 +1071,8 @@ cdf_unpack_catalog(const cdf_header_t *h > nr--; > *cat = CAST(cdf_catalog_t *, > malloc(sizeof(cdf_catalog_t) + nr * sizeof(*ce))); > + if (*cat == NULL) > + return -1; > ce = (*cat)->cat_e; > memset(ce, 0, nr * sizeof(*ce)); > b = CAST(const char *, sst->sst_tab); > @@ -1245,8 +1275,7 @@ cdf_dump(const void *v, size_t len) > void > cdf_dump_stream(const cdf_header_t *h, const cdf_stream_t *sst) > { > - size_t ss = sst->sst_dirlen < h->h_min_size_standard_stream ? > - CDF_SHORT_SEC_SIZE(h) : CDF_SEC_SIZE(h); > + size_t ss = sst->sst_ss; > cdf_dump(sst->sst_tab, ss * sst->sst_len); > } > > > Modified: vendor/file/dist/src/cdf.h > ============================================================================== > --- vendor/file/dist/src/cdf.h Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/src/cdf.h Sat May 14 05:25:47 2016 (r299716) > @@ -129,6 +129,7 @@ typedef struct { > void *sst_tab; > size_t sst_len; > size_t sst_dirlen; > + size_t sst_ss; > } cdf_stream_t; > > typedef struct { > @@ -277,7 +278,7 @@ typedef struct { > > typedef struct { > size_t cat_num; > - cdf_catalog_entry_t cat_e[0]; > + cdf_catalog_entry_t cat_e[1]; > } cdf_catalog_t; > > struct timespec; > > Modified: vendor/file/dist/src/compress.c > ============================================================================== > --- vendor/file/dist/src/compress.c Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/src/compress.c Sat May 14 05:25:47 2016 (r299716) > @@ -35,7 +35,7 @@ > #include "file.h" > > #ifndef lint > -FILE_RCSID("@(#)$File: compress.c,v 1.93 2016/03/31 17:51:12 christos Exp $") > +FILE_RCSID("@(#)$File: compress.c,v 1.96 2016/04/20 00:00:26 christos Exp $") > #endif > > #include "magic.h" > @@ -187,7 +187,7 @@ file_zmagic(struct magic_set *ms, int fd > size_t i, nsz; > char *rbuf; > file_pushbuf_t *pb; > - int rv = 0; > + int urv, prv, rv = 0; > int mime = ms->flags & MAGIC_MIME; > #ifdef HAVE_SIGNAL_H > sig_t osigpipe; > @@ -214,22 +214,22 @@ file_zmagic(struct magic_set *ms, int fd > if (!zm) > continue; > nsz = nbytes; > - rv = uncompressbuf(fd, ms->bytes_max, i, buf, &newbuf, &nsz); > - DPRINTF("uncompressbuf = %d, %s, %zu\n", rv, (char *)newbuf, > + urv = uncompressbuf(fd, ms->bytes_max, i, buf, &newbuf, &nsz); > + DPRINTF("uncompressbuf = %d, %s, %zu\n", urv, (char *)newbuf, > nsz); > - switch (rv) { > + switch (urv) { > case OKDATA: > case ERRDATA: > > ms->flags &= ~MAGIC_COMPRESS; > - if (rv == ERRDATA) > - rv = file_printf(ms, "%s ERROR: %s", > + if (urv == ERRDATA) > + prv = file_printf(ms, "%s ERROR: %s", > methodname(i), newbuf); > else > - rv = file_buffer(ms, -1, name, newbuf, nsz); > - if (rv == -1) > + prv = file_buffer(ms, -1, name, newbuf, nsz); > + if (prv == -1) > goto error; > - DPRINTF("rv = %d\n", rv); > + rv = 1; > if ((ms->flags & MAGIC_COMPRESS_TRANSP) != 0) > goto out; > if (mime != MAGIC_MIME && mime != 0) > @@ -239,6 +239,10 @@ file_zmagic(struct magic_set *ms, int fd > goto error; > if ((pb = file_push_buffer(ms)) == NULL) > goto error; > + /* > + * XXX: If file_buffer fails here, we overwrite > + * the compressed text. FIXME. > + */ > if (file_buffer(ms, -1, NULL, buf, nbytes) == -1) > goto error; > if ((rbuf = file_pop_buffer(ms, pb)) != NULL) { > @@ -250,16 +254,20 @@ file_zmagic(struct magic_set *ms, int fd > } > if (!mime && file_printf(ms, ")") == -1) > goto error; > - goto out; > + /*FALLTHROUGH*/ > case NODATA: > - goto out; > + break; > default: > abort(); > + /*NOTREACHED*/ > + error: > + rv = -1; > + break; > } > } > out: > - rv = 1; > -error: > + DPRINTF("rv = %d\n", rv); > + > #ifdef HAVE_SIGNAL_H > (void)signal(SIGPIPE, osigpipe); > #endif > > Modified: vendor/file/dist/src/der.c > ============================================================================== > --- vendor/file/dist/src/der.c Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/src/der.c Sat May 14 05:25:47 2016 (r299716) > @@ -35,7 +35,7 @@ > #include "file.h" > > #ifndef lint > -FILE_RCSID("@(#)$File: der.c,v 1.4 2016/03/21 23:04:40 christos Exp $") > +FILE_RCSID("@(#)$File: der.c,v 1.6 2016/04/21 14:26:03 christos Exp $") > #endif > #endif > > @@ -44,7 +44,6 @@ FILE_RCSID("@(#)$File: der.c,v 1.4 2016/ > #include > > #include > -#include > #include > #include > #include > @@ -53,6 +52,8 @@ FILE_RCSID("@(#)$File: der.c,v 1.4 2016/ > #ifndef TEST_DER > #include "magic.h" > #include "der.h" > +#else > +#include > #endif > > #define DER_BAD ((uint32_t)-1) > @@ -61,11 +62,15 @@ FILE_RCSID("@(#)$File: der.c,v 1.4 2016/ > #define DER_CLASS_APPLICATION 1 > #define DER_CLASS_CONTEXT 2 > #define DER_CLASS_PRIVATE 3 > +#ifdef DEBUG_DER > static const char der_class[] = "UACP"; > +#endif > > #define DER_TYPE_PRIMITIVE 0 > #define DER_TYPE_CONSTRUCTED 1 > +#ifdef DEBUG_DER > static const char der_type[] = "PC"; > +#endif > > #define DER_TAG_EOC 0x00 > #define DER_TAG_BOOLEAN 0x01 > > Modified: vendor/file/dist/src/magic.c > ============================================================================== > --- vendor/file/dist/src/magic.c Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/src/magic.c Sat May 14 05:25:47 2016 (r299716) > @@ -33,7 +33,7 @@ > #include "file.h" > > #ifndef lint > -FILE_RCSID("@(#)$File: magic.c,v 1.97 2016/03/31 17:51:12 christos Exp $") > +FILE_RCSID("@(#)$File: magic.c,v 1.99 2016/05/03 16:09:38 christos Exp $") > #endif /* lint */ > > #include "magic.h" > @@ -523,9 +523,11 @@ file_or_fd(struct magic_set *ms, const c > rv = 0; > done: > free(buf); > - if (pos != (off_t)-1) > - (void)lseek(fd, pos, SEEK_SET); > - close_and_restore(ms, inname, fd, &sb); > + if (fd != -1) { > + if (pos != (off_t)-1) > + (void)lseek(fd, pos, SEEK_SET); > + close_and_restore(ms, inname, fd, &sb); > + } > out: > return rv == 0 ? file_getbuffer(ms) : NULL; > } > > Modified: vendor/file/dist/src/magic.h > ============================================================================== > --- vendor/file/dist/src/magic.h Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/src/magic.h Sat May 14 05:25:47 2016 (r299716) > @@ -80,7 +80,7 @@ > #define MAGIC_NO_CHECK_FORTRAN 0x000000 /* Don't check ascii/fortran */ > #define MAGIC_NO_CHECK_TROFF 0x000000 /* Don't check ascii/troff */ > > -#define MAGIC_VERSION 525 /* This implementation */ > +#define MAGIC_VERSION 526 /* This implementation */ > > > #ifdef __cplusplus > @@ -114,7 +114,7 @@ int magic_errno(magic_t); > #define MAGIC_PARAM_ELF_SHNUM_MAX 3 > #define MAGIC_PARAM_ELF_NOTES_MAX 4 > #define MAGIC_PARAM_REGEX_MAX 5 > -#define MAGIC_PARAM_BYTES_MAX 6 > +#define MAGIC_PARAM_BYTES_MAX 6 > > int magic_setparam(magic_t, int, const void *); > int magic_getparam(magic_t, int, void *); > > Modified: vendor/file/dist/src/magic.h.in > ============================================================================== > --- vendor/file/dist/src/magic.h.in Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/src/magic.h.in Sat May 14 05:25:47 2016 (r299716) > @@ -114,6 +114,7 @@ int magic_errno(magic_t); > #define MAGIC_PARAM_ELF_SHNUM_MAX 3 > #define MAGIC_PARAM_ELF_NOTES_MAX 4 > #define MAGIC_PARAM_REGEX_MAX 5 > +#define MAGIC_PARAM_BYTES_MAX 6 > > int magic_setparam(magic_t, int, const void *); > int magic_getparam(magic_t, int, void *); > > Modified: vendor/file/dist/src/readcdf.c > ============================================================================== > --- vendor/file/dist/src/readcdf.c Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/src/readcdf.c Sat May 14 05:25:47 2016 (r299716) > @@ -26,7 +26,7 @@ > #include "file.h" > > #ifndef lint > -FILE_RCSID("@(#)$File: readcdf.c,v 1.56 2016/03/03 22:20:03 christos Exp $") > +FILE_RCSID("@(#)$File: readcdf.c,v 1.57 2016/05/03 16:08:49 christos Exp $") > #endif > > #include > @@ -373,7 +373,7 @@ cdf_file_catalog_info(struct magic_set * > dir, "Catalog", scn)) == -1) > return i; > #ifdef CDF_DEBUG > - cdf_dump_catalog(&h, &scn); > + cdf_dump_catalog(&h, scn); > #endif > if ((i = cdf_file_catalog(ms, h, scn)) == -1) > return -1; > > Modified: vendor/file/dist/src/softmagic.c > ============================================================================== > --- vendor/file/dist/src/softmagic.c Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/src/softmagic.c Sat May 14 05:25:47 2016 (r299716) > @@ -32,7 +32,7 @@ > #include "file.h" > > #ifndef lint > -FILE_RCSID("@(#)$File: softmagic.c,v 1.230 2016/04/18 15:10:34 christos Exp $") > +FILE_RCSID("@(#)$File: softmagic.c,v 1.231 2016/04/21 15:23:31 christos Exp $") > #endif /* lint */ > > #include "magic.h" > > Modified: vendor/file/dist/tests/Makefile.am > ============================================================================== > --- vendor/file/dist/tests/Makefile.am Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/tests/Makefile.am Sat May 14 05:25:47 2016 (r299716) > @@ -13,4 +13,4 @@ issue311docx.testfile > T = $(top_srcdir)/tests > check-local: > MAGIC=$(top_builddir)/magic/magic ./test > - for i in $T/*.testfile; do echo Running test: $$i; MAGIC=$(top_builddir)/magic/magic ./test $$i $${i%%.testfile}.result; done > + set -e; for i in $T/*.testfile; do echo Running test: $$i; TZ=UTC MAGIC=$(top_builddir)/magic/magic ./test $$i $${i%%.testfile}.result; done > > Modified: vendor/file/dist/tests/Makefile.in > ============================================================================== > --- vendor/file/dist/tests/Makefile.in Sat May 14 05:00:17 2016 (r299715) > +++ vendor/file/dist/tests/Makefile.in Sat May 14 05:25:47 2016 (r299716) > @@ -608,7 +608,7 @@ uninstall-am: > > check-local: > MAGIC=$(top_builddir)/magic/magic ./test > - for i in $T/*.testfile; do echo Running test: $$i; MAGIC=$(top_builddir)/magic/magic ./test $$i $${i%%.testfile}.result; done > + set -e; for i in $T/*.testfile; do echo Running test: $$i; TZ=UTC MAGIC=$(top_builddir)/magic/magic ./test $$i $${i%%.testfile}.result; done > > # Tell versions [3.59,3.63) of GNU make to not export all variables. > # Otherwise a system limit (for SysV at least) may be exceeded. > _______________________________________________ > svn-src-all@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/svn-src-all > To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"