From owner-svn-src-vendor@freebsd.org Sun Sep 17 15:57:08 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1367E1AB2F; Sun, 17 Sep 2017 15:57:08 +0000 (UTC) (envelope-from gordon@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 781566354A; Sun, 17 Sep 2017 15:57:08 +0000 (UTC) (envelope-from gordon@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8HFv7hg025812; Sun, 17 Sep 2017 15:57:07 GMT (envelope-from gordon@FreeBSD.org) Received: (from gordon@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8HFv6eC025799; Sun, 17 Sep 2017 15:57:06 GMT (envelope-from gordon@FreeBSD.org) Message-Id: <201709171557.v8HFv6eC025799@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: gordon set sender to gordon@FreeBSD.org using -f From: Gordon Tetlow Date: Sun, 17 Sep 2017 15:57:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323678 - in vendor/file/dist: . doc magic magic/Magdir python src tests X-SVN-Group: vendor X-SVN-Commit-Author: gordon X-SVN-Commit-Paths: in vendor/file/dist: . doc magic magic/Magdir python src tests X-SVN-Commit-Revision: 323678 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Sep 2017 15:57:08 -0000 Author: gordon Date: Sun Sep 17 15:57:06 2017 New Revision: 323678 URL: https://svnweb.freebsd.org/changeset/base/323678 Log: Vendor import of file 5.32. Added: vendor/file/dist/magic/Magdir/apache (contents, props changed) vendor/file/dist/magic/Magdir/bhl vendor/file/dist/magic/Magdir/gconv vendor/file/dist/magic/Magdir/gpu vendor/file/dist/magic/Magdir/webassembly vendor/file/dist/magic/Magdir/yara vendor/file/dist/tests/hddrawcopytool.result vendor/file/dist/tests/hddrawcopytool.testfile (contents, props changed) Modified: vendor/file/dist/ChangeLog vendor/file/dist/aclocal.m4 vendor/file/dist/config.guess vendor/file/dist/config.sub vendor/file/dist/configure vendor/file/dist/configure.ac vendor/file/dist/doc/file.man vendor/file/dist/doc/libmagic.man vendor/file/dist/doc/magic.man vendor/file/dist/magic/Localstuff vendor/file/dist/magic/Magdir/adventure vendor/file/dist/magic/Magdir/amanda vendor/file/dist/magic/Magdir/amigaos vendor/file/dist/magic/Magdir/android vendor/file/dist/magic/Magdir/animation vendor/file/dist/magic/Magdir/apple vendor/file/dist/magic/Magdir/archive vendor/file/dist/magic/Magdir/att3b vendor/file/dist/magic/Magdir/audio vendor/file/dist/magic/Magdir/blackberry vendor/file/dist/magic/Magdir/blender vendor/file/dist/magic/Magdir/c-lang vendor/file/dist/magic/Magdir/cad vendor/file/dist/magic/Magdir/cafebabe vendor/file/dist/magic/Magdir/clipper vendor/file/dist/magic/Magdir/coff vendor/file/dist/magic/Magdir/commands vendor/file/dist/magic/Magdir/compress vendor/file/dist/magic/Magdir/console vendor/file/dist/magic/Magdir/cups vendor/file/dist/magic/Magdir/database vendor/file/dist/magic/Magdir/der vendor/file/dist/magic/Magdir/diff vendor/file/dist/magic/Magdir/dolby vendor/file/dist/magic/Magdir/dump vendor/file/dist/magic/Magdir/dyadic vendor/file/dist/magic/Magdir/editors vendor/file/dist/magic/Magdir/filesystems vendor/file/dist/magic/Magdir/flash vendor/file/dist/magic/Magdir/fonts vendor/file/dist/magic/Magdir/fsav vendor/file/dist/magic/Magdir/games vendor/file/dist/magic/Magdir/geo vendor/file/dist/magic/Magdir/gnu vendor/file/dist/magic/Magdir/gpt vendor/file/dist/magic/Magdir/gringotts vendor/file/dist/magic/Magdir/hitachi-sh vendor/file/dist/magic/Magdir/ibm370 vendor/file/dist/magic/Magdir/ibm6000 vendor/file/dist/magic/Magdir/icc vendor/file/dist/magic/Magdir/images vendor/file/dist/magic/Magdir/intel vendor/file/dist/magic/Magdir/isz vendor/file/dist/magic/Magdir/jpeg vendor/file/dist/magic/Magdir/kerberos vendor/file/dist/magic/Magdir/kml vendor/file/dist/magic/Magdir/linux vendor/file/dist/magic/Magdir/lisp vendor/file/dist/magic/Magdir/m4 vendor/file/dist/magic/Magdir/macintosh vendor/file/dist/magic/Magdir/make vendor/file/dist/magic/Magdir/maple vendor/file/dist/magic/Magdir/marc21 vendor/file/dist/magic/Magdir/mathematica vendor/file/dist/magic/Magdir/metastore vendor/file/dist/magic/Magdir/meteorological vendor/file/dist/magic/Magdir/microfocus vendor/file/dist/magic/Magdir/mime vendor/file/dist/magic/Magdir/misctools vendor/file/dist/magic/Magdir/modem vendor/file/dist/magic/Magdir/mozilla vendor/file/dist/magic/Magdir/msdos vendor/file/dist/magic/Magdir/msvc vendor/file/dist/magic/Magdir/msx vendor/file/dist/magic/Magdir/mup vendor/file/dist/magic/Magdir/nasa vendor/file/dist/magic/Magdir/netbsd vendor/file/dist/magic/Magdir/netscape vendor/file/dist/magic/Magdir/nitpicker vendor/file/dist/magic/Magdir/os2 vendor/file/dist/magic/Magdir/os9 vendor/file/dist/magic/Magdir/pbf vendor/file/dist/magic/Magdir/pc88 vendor/file/dist/magic/Magdir/pc98 vendor/file/dist/magic/Magdir/pdf vendor/file/dist/magic/Magdir/pdp vendor/file/dist/magic/Magdir/perl vendor/file/dist/magic/Magdir/pgf vendor/file/dist/magic/Magdir/pgp vendor/file/dist/magic/Magdir/printer vendor/file/dist/magic/Magdir/project vendor/file/dist/magic/Magdir/psdbms vendor/file/dist/magic/Magdir/python vendor/file/dist/magic/Magdir/riff vendor/file/dist/magic/Magdir/ruby vendor/file/dist/magic/Magdir/sccs vendor/file/dist/magic/Magdir/scientific vendor/file/dist/magic/Magdir/sendmail vendor/file/dist/magic/Magdir/sequent vendor/file/dist/magic/Magdir/sgml vendor/file/dist/magic/Magdir/sharc vendor/file/dist/magic/Magdir/sketch vendor/file/dist/magic/Magdir/sql vendor/file/dist/magic/Magdir/ssl vendor/file/dist/magic/Magdir/sysex vendor/file/dist/magic/Magdir/terminfo vendor/file/dist/magic/Magdir/vms vendor/file/dist/magic/Magdir/vmware vendor/file/dist/magic/Magdir/vorbis vendor/file/dist/magic/Magdir/windows vendor/file/dist/magic/Magdir/xenix vendor/file/dist/magic/Magdir/xilinx vendor/file/dist/magic/Magdir/xwindows vendor/file/dist/magic/Makefile.am vendor/file/dist/magic/Makefile.in vendor/file/dist/python/magic.py vendor/file/dist/src/apprentice.c vendor/file/dist/src/cdf.c vendor/file/dist/src/cdf.h vendor/file/dist/src/cdf_time.c vendor/file/dist/src/compress.c vendor/file/dist/src/der.c vendor/file/dist/src/file.h vendor/file/dist/src/fsmagic.c vendor/file/dist/src/funcs.c vendor/file/dist/src/is_tar.c vendor/file/dist/src/magic.c vendor/file/dist/src/magic.h.in vendor/file/dist/src/print.c vendor/file/dist/src/readcdf.c vendor/file/dist/src/readelf.c vendor/file/dist/src/readelf.h vendor/file/dist/src/softmagic.c vendor/file/dist/src/vasprintf.c vendor/file/dist/tests/Makefile.am vendor/file/dist/tests/Makefile.in vendor/file/dist/tests/test.c Modified: vendor/file/dist/ChangeLog ============================================================================== --- vendor/file/dist/ChangeLog Sun Sep 17 15:45:39 2017 (r323677) +++ vendor/file/dist/ChangeLog Sun Sep 17 15:57:06 2017 (r323678) @@ -1,6 +1,60 @@ +2017-09-02 11:53 Christos Zoulas + + * release 5.32 + +2017-08-28 16:37 Christos Zoulas + + * Always reset state in {file,buffer}_apprentice (Krzysztof Wilczynski) + +2017-08-27 03:55 Christos Zoulas + + * Fix always true condition (Thomas Jarosch) + +2017-05-24 17:30 Christos Zoulas + + * pickier parsing of numeric values in magic files. + +2017-05-23 17:55 Christos Zoulas + + * PR/615 add magic_getflags() + +2017-05-23 13:55 Christos Zoulas + + * release 5.31 + +2017-03-17 20:32 Christos Zoulas + + * remove trailing spaces from magic files + * refactor is_tar + * better bounds checks for cdf + +2017-02-10 12:24 Christos Zoulas + + * release 5.30 + +2017-02-07 23:27 Christos Zoulas + + * If we exceeded the offset in a search return no match + (Christoph Biedl) + * Be more lenient on corrupt CDF files (Christoph Biedl) + +2017-02-04 16:46 Christos Zoulas + + * pacify ubsan sign extension (oss-fuzz/524) + +2017-02-01 12:42 Christos Zoulas + + * off by one in cdf parsing (PR/593) + * report debugging sections in elf (PR/591) + +2016-11-06 10:52 Christos Zoulas + + * Allow @@@ in extensions + * Add missing overflow check in der magic (Jonas Wagner) + 2016-10-25 10:40 Christos Zoulas - * release 5.28 + * release 5.29 2016-10-24 11:20 Christos Zoulas @@ -387,7 +441,7 @@ ` 2013-11-06 14:40 Christos Zoulas - * fix erroneous non-zero exit code from non-existant file and message + * fix erroneous non-zero exit code from non-existent file and message 2013-10-29 14:25 Christos Zoulas Modified: vendor/file/dist/aclocal.m4 ============================================================================== --- vendor/file/dist/aclocal.m4 Sun Sep 17 15:45:39 2017 (r323677) +++ vendor/file/dist/aclocal.m4 Sun Sep 17 15:57:06 2017 (r323678) @@ -21,7 +21,7 @@ If you have problems, you may need to regenerate the b To do so, use the procedure documented by the package, typically 'autoreconf'.])]) # visibility.m4 serial 5 (gettext-0.18.2) -dnl Copyright (C) 2005, 2008, 2010-2014 Free Software Foundation, Inc. +dnl Copyright (C) 2005, 2008, 2010-2016 Free Software Foundation, Inc. dnl This file is free software; the Free Software Foundation dnl gives unlimited permission to copy and/or distribute it, dnl with or without modifications, as long as this notice is preserved. Modified: vendor/file/dist/config.guess ============================================================================== --- vendor/file/dist/config.guess Sun Sep 17 15:45:39 2017 (r323677) +++ vendor/file/dist/config.guess Sun Sep 17 15:57:06 2017 (r323678) @@ -1,8 +1,8 @@ #! /bin/sh # Attempt to guess a canonical system name. -# Copyright 1992-2015 Free Software Foundation, Inc. +# Copyright 1992-2017 Free Software Foundation, Inc. -timestamp='2015-03-04' +timestamp='2017-01-01' # This file is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by @@ -27,7 +27,7 @@ timestamp='2015-03-04' # Originally written by Per Bothner; maintained since 2000 by Ben Elliston. # # You can get the latest version of this script from: -# http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD +# http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess # # Please send patches to . @@ -50,7 +50,7 @@ version="\ GNU config.guess ($timestamp) Originally written by Per Bothner. -Copyright 1992-2015 Free Software Foundation, Inc. +Copyright 1992-2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." @@ -186,9 +186,12 @@ case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE *) machine=${UNAME_MACHINE_ARCH}-unknown ;; esac # The Operating System including object format, if it has switched - # to ELF recently, or will in the future. + # to ELF recently (or will in the future) and ABI. case "${UNAME_MACHINE_ARCH}" in - arm*|earm*|i386|m68k|ns32k|sh3*|sparc|vax) + earm*) + os=netbsdelf + ;; + arm*|i386|m68k|ns32k|sh3*|sparc|vax) eval $set_cc_for_build if echo __ELF__ | $CC_FOR_BUILD -E - 2>/dev/null \ | grep -q __ELF__ @@ -221,7 +224,7 @@ case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE release='-gnu' ;; *) - release=`echo ${UNAME_RELEASE}|sed -e 's/[-_].*/\./'` + release=`echo ${UNAME_RELEASE} | sed -e 's/[-_].*//' | cut -d. -f1,2` ;; esac # Since CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM: @@ -237,6 +240,10 @@ case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE UNAME_MACHINE_ARCH=`arch | sed 's/OpenBSD.//'` echo ${UNAME_MACHINE_ARCH}-unknown-openbsd${UNAME_RELEASE} exit ;; + *:LibertyBSD:*:*) + UNAME_MACHINE_ARCH=`arch | sed 's/^.*BSD\.//'` + echo ${UNAME_MACHINE_ARCH}-unknown-libertybsd${UNAME_RELEASE} + exit ;; *:ekkoBSD:*:*) echo ${UNAME_MACHINE}-unknown-ekkobsd${UNAME_RELEASE} exit ;; @@ -249,6 +256,9 @@ case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE *:MirBSD:*:*) echo ${UNAME_MACHINE}-unknown-mirbsd${UNAME_RELEASE} exit ;; + *:Sortix:*:*) + echo ${UNAME_MACHINE}-unknown-sortix + exit ;; alpha:OSF1:*:*) case $UNAME_RELEASE in *4.0) @@ -265,42 +275,42 @@ case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE ALPHA_CPU_TYPE=`/usr/sbin/psrinfo -v | sed -n -e 's/^ The alpha \(.*\) processor.*$/\1/p' | head -n 1` case "$ALPHA_CPU_TYPE" in "EV4 (21064)") - UNAME_MACHINE="alpha" ;; + UNAME_MACHINE=alpha ;; "EV4.5 (21064)") - UNAME_MACHINE="alpha" ;; + UNAME_MACHINE=alpha ;; "LCA4 (21066/21068)") - UNAME_MACHINE="alpha" ;; + UNAME_MACHINE=alpha ;; "EV5 (21164)") - UNAME_MACHINE="alphaev5" ;; + UNAME_MACHINE=alphaev5 ;; "EV5.6 (21164A)") - UNAME_MACHINE="alphaev56" ;; + UNAME_MACHINE=alphaev56 ;; "EV5.6 (21164PC)") - UNAME_MACHINE="alphapca56" ;; + UNAME_MACHINE=alphapca56 ;; "EV5.7 (21164PC)") - UNAME_MACHINE="alphapca57" ;; + UNAME_MACHINE=alphapca57 ;; "EV6 (21264)") - UNAME_MACHINE="alphaev6" ;; + UNAME_MACHINE=alphaev6 ;; "EV6.7 (21264A)") - UNAME_MACHINE="alphaev67" ;; + UNAME_MACHINE=alphaev67 ;; "EV6.8CB (21264C)") - UNAME_MACHINE="alphaev68" ;; + UNAME_MACHINE=alphaev68 ;; "EV6.8AL (21264B)") - UNAME_MACHINE="alphaev68" ;; + UNAME_MACHINE=alphaev68 ;; "EV6.8CX (21264D)") - UNAME_MACHINE="alphaev68" ;; + UNAME_MACHINE=alphaev68 ;; "EV6.9A (21264/EV69A)") - UNAME_MACHINE="alphaev69" ;; + UNAME_MACHINE=alphaev69 ;; "EV7 (21364)") - UNAME_MACHINE="alphaev7" ;; + UNAME_MACHINE=alphaev7 ;; "EV7.9 (21364A)") - UNAME_MACHINE="alphaev79" ;; + UNAME_MACHINE=alphaev79 ;; esac # A Pn.n version is a patched version. # A Vn.n version is a released version. # A Tn.n version is a released field test version. # A Xn.n version is an unreleased experimental baselevel. # 1.2 uses "1.2" for uname -r. - echo ${UNAME_MACHINE}-dec-osf`echo ${UNAME_RELEASE} | sed -e 's/^[PVTX]//' | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` + echo ${UNAME_MACHINE}-dec-osf`echo ${UNAME_RELEASE} | sed -e 's/^[PVTX]//' | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz` # Reset EXIT trap before exiting to avoid spurious non-zero exit code. exitcode=$? trap '' 0 @@ -373,16 +383,16 @@ case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE exit ;; i86pc:SunOS:5.*:* | i86xen:SunOS:5.*:*) eval $set_cc_for_build - SUN_ARCH="i386" + SUN_ARCH=i386 # If there is a compiler, see if it is configured for 64-bit objects. # Note that the Sun cc does not turn __LP64__ into 1 like gcc does. # This test works for both compilers. - if [ "$CC_FOR_BUILD" != 'no_compiler_found' ]; then + if [ "$CC_FOR_BUILD" != no_compiler_found ]; then if (echo '#ifdef __amd64'; echo IS_64BIT_ARCH; echo '#endif') | \ - (CCOPTS= $CC_FOR_BUILD -E - 2>/dev/null) | \ + (CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | \ grep IS_64BIT_ARCH >/dev/null then - SUN_ARCH="x86_64" + SUN_ARCH=x86_64 fi fi echo ${SUN_ARCH}-pc-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` @@ -407,7 +417,7 @@ case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE exit ;; sun*:*:4.2BSD:*) UNAME_RELEASE=`(sed 1q /etc/motd | awk '{print substr($5,1,3)}') 2>/dev/null` - test "x${UNAME_RELEASE}" = "x" && UNAME_RELEASE=3 + test "x${UNAME_RELEASE}" = x && UNAME_RELEASE=3 case "`/bin/arch`" in sun3) echo m68k-sun-sunos${UNAME_RELEASE} @@ -632,13 +642,13 @@ EOF sc_cpu_version=`/usr/bin/getconf SC_CPU_VERSION 2>/dev/null` sc_kernel_bits=`/usr/bin/getconf SC_KERNEL_BITS 2>/dev/null` case "${sc_cpu_version}" in - 523) HP_ARCH="hppa1.0" ;; # CPU_PA_RISC1_0 - 528) HP_ARCH="hppa1.1" ;; # CPU_PA_RISC1_1 + 523) HP_ARCH=hppa1.0 ;; # CPU_PA_RISC1_0 + 528) HP_ARCH=hppa1.1 ;; # CPU_PA_RISC1_1 532) # CPU_PA_RISC2_0 case "${sc_kernel_bits}" in - 32) HP_ARCH="hppa2.0n" ;; - 64) HP_ARCH="hppa2.0w" ;; - '') HP_ARCH="hppa2.0" ;; # HP-UX 10.20 + 32) HP_ARCH=hppa2.0n ;; + 64) HP_ARCH=hppa2.0w ;; + '') HP_ARCH=hppa2.0 ;; # HP-UX 10.20 esac ;; esac fi @@ -677,11 +687,11 @@ EOF exit (0); } EOF - (CCOPTS= $CC_FOR_BUILD -o $dummy $dummy.c 2>/dev/null) && HP_ARCH=`$dummy` + (CCOPTS="" $CC_FOR_BUILD -o $dummy $dummy.c 2>/dev/null) && HP_ARCH=`$dummy` test -z "$HP_ARCH" && HP_ARCH=hppa fi ;; esac - if [ ${HP_ARCH} = "hppa2.0w" ] + if [ ${HP_ARCH} = hppa2.0w ] then eval $set_cc_for_build @@ -694,12 +704,12 @@ EOF # $ CC_FOR_BUILD="cc +DA2.0w" ./config.guess # => hppa64-hp-hpux11.23 - if echo __LP64__ | (CCOPTS= $CC_FOR_BUILD -E - 2>/dev/null) | + if echo __LP64__ | (CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | grep -q __LP64__ then - HP_ARCH="hppa2.0w" + HP_ARCH=hppa2.0w else - HP_ARCH="hppa64" + HP_ARCH=hppa64 fi fi echo ${HP_ARCH}-hp-hpux${HPUX_REV} @@ -804,14 +814,14 @@ EOF echo craynv-cray-unicosmp${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' exit ;; F30[01]:UNIX_System_V:*:* | F700:UNIX_System_V:*:*) - FUJITSU_PROC=`uname -m | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` - FUJITSU_SYS=`uname -p | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/\///'` + FUJITSU_PROC=`uname -m | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz` + FUJITSU_SYS=`uname -p | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/\///'` FUJITSU_REL=`echo ${UNAME_RELEASE} | sed -e 's/ /_/'` echo "${FUJITSU_PROC}-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}" exit ;; 5000:UNIX_System_V:4.*:*) - FUJITSU_SYS=`uname -p | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/\///'` - FUJITSU_REL=`echo ${UNAME_RELEASE} | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/ /_/'` + FUJITSU_SYS=`uname -p | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/\///'` + FUJITSU_REL=`echo ${UNAME_RELEASE} | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/ /_/'` echo "sparc-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}" exit ;; i*86:BSD/386:*:* | i*86:BSD/OS:*:* | *:Ascend\ Embedded/OS:*:*) @@ -893,7 +903,7 @@ EOF exit ;; *:GNU/*:*:*) # other systems with GNU libc and userland - echo ${UNAME_MACHINE}-unknown-`echo ${UNAME_SYSTEM} | sed 's,^[^/]*/,,' | tr '[A-Z]' '[a-z]'``echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`-${LIBC} + echo ${UNAME_MACHINE}-unknown-`echo ${UNAME_SYSTEM} | sed 's,^[^/]*/,,' | tr "[:upper:]" "[:lower:]"``echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`-${LIBC} exit ;; i*86:Minix:*:*) echo ${UNAME_MACHINE}-pc-minix @@ -916,7 +926,7 @@ EOF EV68*) UNAME_MACHINE=alphaev68 ;; esac objdump --private-headers /bin/sh | grep -q ld.so.1 - if test "$?" = 0 ; then LIBC="gnulibc1" ; fi + if test "$?" = 0 ; then LIBC=gnulibc1 ; fi echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; arc:Linux:*:* | arceb:Linux:*:*) @@ -962,6 +972,9 @@ EOF ia64:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; + k1om:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; m32r*:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; @@ -987,6 +1000,9 @@ EOF eval `$CC_FOR_BUILD -E $dummy.c 2>/dev/null | grep '^CPU'` test x"${CPU}" != x && { echo "${CPU}-unknown-linux-${LIBC}"; exit; } ;; + mips64el:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; openrisc*:Linux:*:*) echo or1k-unknown-linux-${LIBC} exit ;; @@ -1019,6 +1035,9 @@ EOF ppcle:Linux:*:*) echo powerpcle-unknown-linux-${LIBC} exit ;; + riscv32:Linux:*:* | riscv64:Linux:*:*) + echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + exit ;; s390:Linux:*:* | s390x:Linux:*:*) echo ${UNAME_MACHINE}-ibm-linux-${LIBC} exit ;; @@ -1038,7 +1057,7 @@ EOF echo ${UNAME_MACHINE}-dec-linux-${LIBC} exit ;; x86_64:Linux:*:*) - echo ${UNAME_MACHINE}-unknown-linux-${LIBC} + echo ${UNAME_MACHINE}-pc-linux-${LIBC} exit ;; xtensa*:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} @@ -1117,7 +1136,7 @@ EOF # uname -m prints for DJGPP always 'pc', but it prints nothing about # the processor, so we play safe by assuming i586. # Note: whatever this is, it MUST be the same as what config.sub - # prints for the "djgpp" host, or else GDB configury will decide that + # prints for the "djgpp" host, or else GDB configure will decide that # this is a cross-build. echo i586-pc-msdosdjgpp exit ;; @@ -1266,6 +1285,9 @@ EOF SX-8R:SUPER-UX:*:*) echo sx8r-nec-superux${UNAME_RELEASE} exit ;; + SX-ACE:SUPER-UX:*:*) + echo sxace-nec-superux${UNAME_RELEASE} + exit ;; Power*:Rhapsody:*:*) echo powerpc-apple-rhapsody${UNAME_RELEASE} exit ;; @@ -1279,9 +1301,9 @@ EOF UNAME_PROCESSOR=powerpc fi if test `echo "$UNAME_RELEASE" | sed -e 's/\..*//'` -le 10 ; then - if [ "$CC_FOR_BUILD" != 'no_compiler_found' ]; then + if [ "$CC_FOR_BUILD" != no_compiler_found ]; then if (echo '#ifdef __LP64__'; echo IS_64BIT_ARCH; echo '#endif') | \ - (CCOPTS= $CC_FOR_BUILD -E - 2>/dev/null) | \ + (CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | \ grep IS_64BIT_ARCH >/dev/null then case $UNAME_PROCESSOR in @@ -1303,7 +1325,7 @@ EOF exit ;; *:procnto*:*:* | *:QNX:[0123456789]*:*) UNAME_PROCESSOR=`uname -p` - if test "$UNAME_PROCESSOR" = "x86"; then + if test "$UNAME_PROCESSOR" = x86; then UNAME_PROCESSOR=i386 UNAME_MACHINE=pc fi @@ -1334,7 +1356,7 @@ EOF # "uname -m" is not consistent, so use $cputype instead. 386 # is converted to i386 for consistency with other x86 # operating systems. - if test "$cputype" = "386"; then + if test "$cputype" = 386; then UNAME_MACHINE=i386 else UNAME_MACHINE="$cputype" @@ -1376,7 +1398,7 @@ EOF echo i386-pc-xenix exit ;; i*86:skyos:*:*) - echo ${UNAME_MACHINE}-pc-skyos`echo ${UNAME_RELEASE}` | sed -e 's/ .*$//' + echo ${UNAME_MACHINE}-pc-skyos`echo ${UNAME_RELEASE} | sed -e 's/ .*$//'` exit ;; i*86:rdos:*:*) echo ${UNAME_MACHINE}-pc-rdos @@ -1387,23 +1409,25 @@ EOF x86_64:VMkernel:*:*) echo ${UNAME_MACHINE}-unknown-esx exit ;; + amd64:Isilon\ OneFS:*:*) + echo x86_64-unknown-onefs + exit ;; esac cat >&2 < in order to provide the needed -information to handle your system. +If $0 has already been updated, send the following data and any +information you think might be pertinent to config-patches@gnu.org to +provide the necessary information to handle your system. config.guess timestamp = $timestamp Modified: vendor/file/dist/config.sub ============================================================================== --- vendor/file/dist/config.sub Sun Sep 17 15:45:39 2017 (r323677) +++ vendor/file/dist/config.sub Sun Sep 17 15:57:06 2017 (r323678) @@ -1,8 +1,8 @@ #! /bin/sh # Configuration validation subroutine script. -# Copyright 1992-2015 Free Software Foundation, Inc. +# Copyright 1992-2017 Free Software Foundation, Inc. -timestamp='2015-03-08' +timestamp='2017-01-01' # This file is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by @@ -33,7 +33,7 @@ timestamp='2015-03-08' # Otherwise, we print the canonical config type on stdout and succeed. # You can get the latest version of this script from: -# http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD +# http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub # This file is supposed to be the same for all GNU packages # and recognize all the CPU types, system types and aliases @@ -53,8 +53,7 @@ timestamp='2015-03-08' me=`echo "$0" | sed -e 's,.*/,,'` usage="\ -Usage: $0 [OPTION] CPU-MFR-OPSYS - $0 [OPTION] ALIAS +Usage: $0 [OPTION] CPU-MFR-OPSYS or ALIAS Canonicalize a configuration name. @@ -68,7 +67,7 @@ Report bugs and patches to ." version="\ GNU config.sub ($timestamp) -Copyright 1992-2015 Free Software Foundation, Inc. +Copyright 1992-2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." @@ -118,7 +117,7 @@ case $maybe_os in nto-qnx* | linux-gnu* | linux-android* | linux-dietlibc | linux-newlib* | \ linux-musl* | linux-uclibc* | uclinux-uclibc* | uclinux-gnu* | kfreebsd*-gnu* | \ knetbsd*-gnu* | netbsd*-gnu* | netbsd*-eabi* | \ - kopensolaris*-gnu* | \ + kopensolaris*-gnu* | cloudabi*-eabi* | \ storm-chaos* | os2-emx* | rtmk-nova*) os=-$maybe_os basic_machine=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\1/'` @@ -255,6 +254,7 @@ case $basic_machine in | arc | arceb \ | arm | arm[bl]e | arme[lb] | armv[2-8] | armv[3-8][lb] | armv7[arm] \ | avr | avr32 \ + | ba \ | be32 | be64 \ | bfin \ | c4x | c8051 | clipper \ @@ -301,11 +301,12 @@ case $basic_machine in | open8 | or1k | or1knd | or32 \ | pdp10 | pdp11 | pj | pjl \ | powerpc | powerpc64 | powerpc64le | powerpcle \ + | pru \ | pyramid \ | riscv32 | riscv64 \ | rl78 | rx \ | score \ - | sh | sh[1234] | sh[24]a | sh[24]aeb | sh[23]e | sh[34]eb | sheb | shbe | shle | sh[1234]le | sh3ele \ + | sh | sh[1234] | sh[24]a | sh[24]aeb | sh[23]e | sh[234]eb | sheb | shbe | shle | sh[1234]le | sh3ele \ | sh64 | sh64le \ | sparc | sparc64 | sparc64b | sparc64v | sparc86x | sparclet | sparclite \ | sparcv8 | sparcv9 | sparcv9b | sparcv9v \ @@ -376,6 +377,7 @@ case $basic_machine in | alphapca5[67]-* | alpha64pca5[67]-* | arc-* | arceb-* \ | arm-* | armbe-* | armle-* | armeb-* | armv*-* \ | avr-* | avr32-* \ + | ba-* \ | be32-* | be64-* \ | bfin-* | bs2000-* \ | c[123]* | c30-* | [cjt]90-* | c4x-* \ @@ -427,13 +429,15 @@ case $basic_machine in | orion-* \ | pdp10-* | pdp11-* | pj-* | pjl-* | pn-* | power-* \ | powerpc-* | powerpc64-* | powerpc64le-* | powerpcle-* \ + | pru-* \ | pyramid-* \ + | riscv32-* | riscv64-* \ | rl78-* | romp-* | rs6000-* | rx-* \ | sh-* | sh[1234]-* | sh[24]a-* | sh[24]aeb-* | sh[23]e-* | sh[34]eb-* | sheb-* | shbe-* \ | shle-* | sh[1234]le-* | sh3ele-* | sh64-* | sh64le-* \ | sparc-* | sparc64-* | sparc64b-* | sparc64v-* | sparc86x-* | sparclet-* \ | sparclite-* \ - | sparcv8-* | sparcv9-* | sparcv9b-* | sparcv9v-* | sv1-* | sx?-* \ + | sparcv8-* | sparcv9-* | sparcv9b-* | sparcv9v-* | sv1-* | sx*-* \ | tahoe-* \ | tic30-* | tic4x-* | tic54x-* | tic55x-* | tic6x-* | tic80-* \ | tile*-* \ @@ -518,7 +522,7 @@ case $basic_machine in basic_machine=i386-pc os=-aros ;; - asmjs) + asmjs) basic_machine=asmjs-unknown ;; aux) @@ -641,6 +645,14 @@ case $basic_machine in basic_machine=m68k-bull os=-sysv3 ;; + e500v[12]) + basic_machine=powerpc-unknown + os=$os"spe" + ;; + e500v[12]-*) + basic_machine=powerpc-`echo $basic_machine | sed 's/^[^-]*-//'` + os=$os"spe" + ;; ebmon29k) basic_machine=a29k-amd os=-ebmon @@ -1020,7 +1032,7 @@ case $basic_machine in ppc-* | ppcbe-*) basic_machine=powerpc-`echo $basic_machine | sed 's/^[^-]*-//'` ;; - ppcle | powerpclittle | ppc-le | powerpc-little) + ppcle | powerpclittle) basic_machine=powerpcle-unknown ;; ppcle-* | powerpclittle-*) @@ -1030,7 +1042,7 @@ case $basic_machine in ;; ppc64-*) basic_machine=powerpc64-`echo $basic_machine | sed 's/^[^-]*-//'` ;; - ppc64le | powerpc64little | ppc64-le | powerpc64-little) + ppc64le | powerpc64little) basic_machine=powerpc64le-unknown ;; ppc64le-* | powerpc64little-*) @@ -1376,18 +1388,18 @@ case $os in | -hpux* | -unos* | -osf* | -luna* | -dgux* | -auroraux* | -solaris* \ | -sym* | -kopensolaris* | -plan9* \ | -amigaos* | -amigados* | -msdos* | -newsos* | -unicos* | -aof* \ - | -aos* | -aros* | -cloudabi* \ + | -aos* | -aros* | -cloudabi* | -sortix* \ | -nindy* | -vxsim* | -vxworks* | -ebmon* | -hms* | -mvs* \ | -clix* | -riscos* | -uniplus* | -iris* | -rtu* | -xenix* \ | -hiux* | -386bsd* | -knetbsd* | -mirbsd* | -netbsd* \ - | -bitrig* | -openbsd* | -solidbsd* \ + | -bitrig* | -openbsd* | -solidbsd* | -libertybsd* \ | -ekkobsd* | -kfreebsd* | -freebsd* | -riscix* | -lynxos* \ | -bosx* | -nextstep* | -cxux* | -aout* | -elf* | -oabi* \ | -ptx* | -coff* | -ecoff* | -winnt* | -domain* | -vsta* \ | -udi* | -eabi* | -lites* | -ieee* | -go32* | -aux* \ - | -chorusos* | -chorusrdb* | -cegcc* \ + | -chorusos* | -chorusrdb* | -cegcc* | -glidix* \ | -cygwin* | -msys* | -pe* | -psos* | -moss* | -proelf* | -rtems* \ - | -mingw32* | -mingw64* | -linux-gnu* | -linux-android* \ + | -midipix* | -mingw32* | -mingw64* | -linux-gnu* | -linux-android* \ | -linux-newlib* | -linux-musl* | -linux-uclibc* \ | -uxpv* | -beos* | -mpeix* | -udk* | -moxiebox* \ | -interix* | -uwin* | -mks* | -rhapsody* | -darwin* | -opened* \ @@ -1396,7 +1408,8 @@ case $os in | -os2* | -vos* | -palmos* | -uclinux* | -nucleus* \ | -morphos* | -superux* | -rtmk* | -rtmk-nova* | -windiss* \ | -powermax* | -dnix* | -nx6 | -nx7 | -sei* | -dragonfly* \ - | -skyos* | -haiku* | -rdos* | -toppers* | -drops* | -es* | -tirtos*) + | -skyos* | -haiku* | -rdos* | -toppers* | -drops* | -es* \ + | -onefs* | -tirtos* | -phoenix* | -fuchsia* | -redox*) # Remember, each alternative MUST END IN *, to match a version number. ;; -qnx*) @@ -1528,6 +1541,8 @@ case $os in ;; -nacl*) ;; + -ios) + ;; -none) ;; *) @@ -1622,6 +1637,9 @@ case $basic_machine in ;; sparc-* | *-sun) os=-sunos4.1.1 + ;; + pru-*) + os=-elf ;; *-be) os=-beos Modified: vendor/file/dist/configure ============================================================================== --- vendor/file/dist/configure Sun Sep 17 15:45:39 2017 (r323677) +++ vendor/file/dist/configure Sun Sep 17 15:57:06 2017 (r323678) @@ -1,6 +1,6 @@ #! /bin/sh # Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.69 for file 5.29. +# Generated by GNU Autoconf 2.69 for file 5.32. # # Report bugs to . # @@ -590,8 +590,8 @@ MAKEFLAGS= # Identity of this package. PACKAGE_NAME='file' PACKAGE_TARNAME='file' -PACKAGE_VERSION='5.29' -PACKAGE_STRING='file 5.29' +PACKAGE_VERSION='5.32' +PACKAGE_STRING='file 5.32' PACKAGE_BUGREPORT='christos@astron.com' PACKAGE_URL='' @@ -1328,7 +1328,7 @@ if test "$ac_init_help" = "long"; then # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF -\`configure' configures file 5.29 to adapt to many kinds of systems. +\`configure' configures file 5.32 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... @@ -1398,7 +1398,7 @@ fi if test -n "$ac_init_help"; then case $ac_init_help in - short | recursive ) echo "Configuration of file 5.29:";; + short | recursive ) echo "Configuration of file 5.32:";; esac cat <<\_ACEOF @@ -1509,7 +1509,7 @@ fi test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF -file configure 5.29 +file configure 5.32 generated by GNU Autoconf 2.69 Copyright (C) 2012 Free Software Foundation, Inc. @@ -2165,7 +2165,7 @@ cat >config.log <<_ACEOF This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. -It was created by file $as_me 5.29, which was +It was created by file $as_me 5.32, which was generated by GNU Autoconf 2.69. Invocation command line was $ $0 $@ @@ -3031,7 +3031,7 @@ fi # Define the identity of the package. PACKAGE='file' - VERSION='5.29' + VERSION='5.32' cat >>confdefs.h <<_ACEOF @@ -15075,7 +15075,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" -This file was extended by file $as_me 5.29, which was +This file was extended by file $as_me 5.32, which was generated by GNU Autoconf 2.69. Invocation command line was CONFIG_FILES = $CONFIG_FILES @@ -15141,7 +15141,7 @@ _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ -file config.status 5.29 +file config.status 5.32 configured by $0, generated by GNU Autoconf 2.69, with options \\"\$ac_cs_config\\" Modified: vendor/file/dist/configure.ac ============================================================================== --- vendor/file/dist/configure.ac Sun Sep 17 15:45:39 2017 (r323677) +++ vendor/file/dist/configure.ac Sun Sep 17 15:57:06 2017 (r323678) @@ -1,5 +1,5 @@ dnl Process this file with autoconf to produce a configure script. -AC_INIT([file],[5.29],[christos@astron.com]) +AC_INIT([file],[5.32],[christos@astron.com]) AM_INIT_AUTOMAKE([subdir-objects foreign]) m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) Modified: vendor/file/dist/doc/file.man ============================================================================== --- vendor/file/dist/doc/file.man Sun Sep 17 15:45:39 2017 (r323677) +++ vendor/file/dist/doc/file.man Sun Sep 17 15:57:06 2017 (r323678) @@ -1,4 +1,4 @@ -.\" $File: file.man,v 1.124 2016/10/19 20:52:45 christos Exp $ +.\" $File: file.man,v 1.125 2017/01/03 11:24:46 christos Exp $ .Dd October 19, 2016 .Dt FILE __CSECTION__ .Os @@ -238,8 +238,8 @@ or at least one filename argument must be present; to test the standard input, use .Sq - as a filename argument. -Please note that -.Ar namefile +Please note that +.Ar namefile is unwrapped and the enclosed filenames are processed when this option is encountered and before any further options processing is done. This allows one to process multiple lists of files with different command line @@ -411,10 +411,10 @@ and .Fl h options. .Sh SEE ALSO -.Xr magic __FSECTION__ , .Xr hexdump 1 , .Xr od 1 , .Xr strings 1 , +.Xr magic __FSECTION__ .Sh STANDARDS CONFORMANCE This program is believed to exceed the System V Interface Definition of FILE(CMD), as near as one can determine from the vague language @@ -530,16 +530,15 @@ John Gilmore revised the code extensively, making it b the first version. Geoff Collyer found several inadequacies and provided some magic file entries. -Contributions by the +Contributions of the .Sq \*[Am] operator by Rob McMahon, .Aq cudcv@warwick.ac.uk , 1989. .Pp -Guy Harris, +Guy Harris, .Aq guy@netapp.com , made many changes from 1993 to the present. -1989. .Pp Primary development and maintenance from 1990 to the present by Christos Zoulas @@ -587,7 +586,6 @@ program, and are not covered by the above license. .Nm returns 0 on success, and non-zero on error. .Sh BUGS -.Pp Please report bugs and send patches to the bug tracker at .Pa http://bugs.gw.com/ or the mailing list at @@ -596,7 +594,6 @@ or the mailing list at .Pa http://mx.gw.com/mailman/listinfo/file first to subscribe). .Sh TODO -.Pp Fix output so that tests for MIME and APPLE flags are not needed all over the place, and actual output is only done in one place. This needs a design. @@ -645,16 +642,16 @@ Fix .Dq name and .Dq use -to check for consistency at compile time (duplicate +to check for consistency at compile time (duplicate .Dq name , .Dq use pointing to undefined .Dq name ). -Make +Make .Dq name / -.Dq use +.Dq use more efficient by keeping a sorted list of names. Special-case ^ to flip endianness in the parser so that it does not have to be escaped, and document it. Modified: vendor/file/dist/doc/libmagic.man ============================================================================== --- vendor/file/dist/doc/libmagic.man Sun Sep 17 15:45:39 2017 (r323677) +++ vendor/file/dist/doc/libmagic.man Sun Sep 17 15:57:06 2017 (r323678) @@ -1,4 +1,4 @@ -.\" $File: libmagic.man,v 1.40 2016/03/31 17:51:12 christos Exp $ +.\" $File: libmagic.man,v 1.41 2017/05/23 21:54:07 christos Exp $ .\" .\" Copyright (c) Christos Zoulas 2003. .\" All Rights Reserved. @@ -25,7 +25,7 @@ .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" -.Dd September 11, 2015 +.Dd May 23, 2017 .Dt LIBMAGIC 3 .Os .Sh NAME @@ -35,6 +35,7 @@ .Nm magic_errno , .Nm magic_descriptor , .Nm magic_buffer , +.Nm magic_getflags , .Nm magic_setflags , .Nm magic_check , .Nm magic_compile , @@ -64,6 +65,8 @@ .Ft const char * .Fn magic_buffer "magic_t cookie" "const void *buffer" "size_t length" .Ft int +.Fn magic_getflags "magic_t cookie" +.Ft int .Fn magic_setflags "magic_t cookie" "int flags" .Ft int .Fn magic_check "magic_t cookie" "const char *filename" @@ -204,6 +207,12 @@ function returns a textual description of the contents argument with .Ar length bytes size. +.Pp +The +.Fn magic_getflags +functions returns a value representing current +.Ar flags +set. .Pp The .Fn magic_setflags Modified: vendor/file/dist/doc/magic.man ============================================================================== --- vendor/file/dist/doc/magic.man Sun Sep 17 15:45:39 2017 (r323677) +++ vendor/file/dist/doc/magic.man Sun Sep 17 15:57:06 2017 (r323678) @@ -1,5 +1,5 @@ -.\" $File: magic.man,v 1.88 2016/07/27 09:42:49 rrt Exp $ -.Dd July 20, 2016 +.\" $File: magic.man,v 1.91 2017/02/12 15:30:08 christos Exp $ +.Dd February 12, 2017 .Dt MAGIC __FSECTION__ .Os .\" install as magic.4 on USG, magic.5 on V7, Berkeley and Linux systems. @@ -7,7 +7,7 @@ .Nm magic .Nd file command's magic pattern file .Sh DESCRIPTION -This manual page documents the format of the magic file as +This manual page documents the format of magic files as used by the .Xr file __CSECTION__ command, version __VERSION__. @@ -17,13 +17,19 @@ command identifies the type of a file using, among other tests, a test for whether the file contains certain .Dq "magic patterns" . -The file -.Pa __MAGIC__ -specifies what patterns are to be tested for, what message or +The database of these +.Dq "magic patterns" +is usually located in a binary file in +.Pa __MAGIC__.mgc +or a directory of source text magic pattern fragment files in +.Pa __MAGIC__ . +The database specifies what patterns are to be tested for, what message or MIME type to print if a particular pattern is found, and additional information to extract from the file. .Pp -Each line of the file specifies a test to be performed. +The format of the source fragment files that are used to build this database +is as follows: +Each line of a fragment file specifies a test to be performed. A test compares the data starting at a particular offset in the file with a byte value, a string or a numeric value. If the test succeeds, a message is printed. @@ -98,13 +104,13 @@ The following modifiers are supported: .It B A byte length (default). .It H -A 2 byte big endian length. +A 4 byte big endian length. .It h -A 2 byte big little length. +A 2 byte big endian length. .It L -A 4 byte big endian length. +A 4 byte little endian length. .It l -A 4 byte big little length. +A 2 byte little endian length. .It J The length includes itself in its count. .El @@ -651,7 +657,7 @@ start of the main indirect offset. \*[Gt]\*[Gt]\*[Gt]\*[Gt](\*[Am]0xe.l+(-4)) string PK\e3\e4 \eb, ZIP self-extracting archive .Ed .Pp -If you have a list of known avalues at a particular continuation level, +If you have a list of known values at a particular continuation level, and you want to provide a switch-like default case: .Bd -literal -offset indent # clear that continuation level match Modified: vendor/file/dist/magic/Localstuff ============================================================================== --- vendor/file/dist/magic/Localstuff Sun Sep 17 15:45:39 2017 (r323677) +++ vendor/file/dist/magic/Localstuff Sun Sep 17 15:57:06 2017 (r323678) @@ -2,6 +2,6 @@ #------------------------------------------------------------------------------ # Localstuff: file(1) magic for locally observed files # -# $File: Localstuff,v 1.4 2003/03/23 04:17:27 christos Exp $ +# $File: Localstuff,v 1.5 2007/01/12 17:38:27 christos Exp $ # Add any locally observed files here. Remember: # text if readable, executable if runnable binary, data if unreadable. Modified: vendor/file/dist/magic/Magdir/adventure ============================================================================== --- vendor/file/dist/magic/Magdir/adventure Sun Sep 17 15:45:39 2017 (r323677) +++ vendor/file/dist/magic/Magdir/adventure Sun Sep 17 15:57:06 2017 (r323678) @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: adventure,v 1.15 2015/09/07 10:03:21 christos Exp $ +# $File: adventure,v 1.17 2017/07/03 16:03:40 christos Exp $ # adventure: file(1) magic for Adventure game files # # from Allen Garvin @@ -36,11 +36,12 @@ >0 ubyte <9 >>16 belong&0xfe00f0f0 0x3030 >>>0 ubyte < 10 ->>>>2 ubeshort < 10 +>>>>2 ubeshort x >>>>>18 regex [0-9][0-9][0-9][0-9][0-9][0-9] ->>>>>>0 ubyte < 10 Infocom (Z-machine %d, ->>>>>>>2 ubeshort < 10 Release %d / ->>>>>>>>18 string >\0 Serial %.6s) +>>>>>>0 ubyte < 10 Infocom (Z-machine %d +>>>>>>>2 ubeshort x \b, Release %d +>>>>>>>>18 string >\0 \b, Serial %.6s +>>>>>>>>18 string x \b) !:strength + 40 !:mime application/x-zmachine @@ -78,7 +79,7 @@ !:mime application/x-tads # Some saved game files start with "TADS2 save/g\n\r\032\0", a little-endian # 2-byte length N, the N-char name of the game file *without* a NUL (darn!), -# "TADS2 save\n\r\032\0" and the interpreter version. +# "TADS2 save\n\r\032\0" and the interpreter version. 0 string TADS2\ save/g TADS >12 belong !0x0A0D1A00 saved game data, CORRUPTED >12 belong 0x0A0D1A00 *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Sun Sep 17 15:58:18 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0A40DE1AC84; Sun, 17 Sep 2017 15:58:18 +0000 (UTC) (envelope-from gordon@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B2D2263697; Sun, 17 Sep 2017 15:58:17 +0000 (UTC) (envelope-from gordon@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8HFwGHS025889; Sun, 17 Sep 2017 15:58:16 GMT (envelope-from gordon@FreeBSD.org) Received: (from gordon@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8HFwGlV025888; Sun, 17 Sep 2017 15:58:16 GMT (envelope-from gordon@FreeBSD.org) Message-Id: <201709171558.v8HFwGlV025888@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: gordon set sender to gordon@FreeBSD.org using -f From: Gordon Tetlow Date: Sun, 17 Sep 2017 15:58:16 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323679 - vendor/file/5.32 X-SVN-Group: vendor X-SVN-Commit-Author: gordon X-SVN-Commit-Paths: vendor/file/5.32 X-SVN-Commit-Revision: 323679 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Sep 2017 15:58:18 -0000 Author: gordon Date: Sun Sep 17 15:58:16 2017 New Revision: 323679 URL: https://svnweb.freebsd.org/changeset/base/323679 Log: Tag file 5.32. Added: vendor/file/5.32/ - copied from r323678, vendor/file/dist/ From owner-svn-src-vendor@freebsd.org Mon Sep 18 04:10:58 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 079F3E1D09F; Mon, 18 Sep 2017 04:10:58 +0000 (UTC) (envelope-from gordon@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A13717E1EE; Mon, 18 Sep 2017 04:10:57 +0000 (UTC) (envelope-from gordon@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8I4AucO028227; Mon, 18 Sep 2017 04:10:56 GMT (envelope-from gordon@FreeBSD.org) Received: (from gordon@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8I4Aso0028205; Mon, 18 Sep 2017 04:10:54 GMT (envelope-from gordon@FreeBSD.org) Message-Id: <201709180410.v8I4Aso0028205@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: gordon set sender to gordon@FreeBSD.org using -f From: Gordon Tetlow Date: Mon, 18 Sep 2017 04:10:54 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323696 - in vendor/tcpdump/dist: . tests X-SVN-Group: vendor X-SVN-Commit-Author: gordon X-SVN-Commit-Paths: in vendor/tcpdump/dist: . tests X-SVN-Commit-Revision: 323696 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Sep 2017 04:10:58 -0000 Author: gordon Date: Mon Sep 18 04:10:54 2017 New Revision: 323696 URL: https://svnweb.freebsd.org/changeset/base/323696 Log: Vendor import of tcpdump 4.9.2. Approved by: emaste (mentor) Added: vendor/tcpdump/dist/funcattrs.h (contents, props changed) vendor/tcpdump/dist/tests/802_15_4-data.out vendor/tcpdump/dist/tests/802_15_4-data.pcap (contents, props changed) vendor/tcpdump/dist/tests/802_15_4-oobr-1.out vendor/tcpdump/dist/tests/802_15_4-oobr-1.pcap (contents, props changed) vendor/tcpdump/dist/tests/802_15_4-oobr-2.out vendor/tcpdump/dist/tests/802_15_4-oobr-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/802_15_4_beacon.out vendor/tcpdump/dist/tests/802_15_4_beacon.pcap (contents, props changed) vendor/tcpdump/dist/tests/arp-oobr.out vendor/tcpdump/dist/tests/arp-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/beep-oobr.out vendor/tcpdump/dist/tests/beep-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/bgp-aigp-oobr-nossl.out vendor/tcpdump/dist/tests/bgp-aigp-oobr-ssl.out vendor/tcpdump/dist/tests/bgp-aigp-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/bgp-as-path-oobr-nossl.out vendor/tcpdump/dist/tests/bgp-as-path-oobr-ssl.out vendor/tcpdump/dist/tests/bgp-as-path-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/bgp_mvpn_6_and_7.out vendor/tcpdump/dist/tests/bgp_mvpn_6_and_7.pcap (contents, props changed) vendor/tcpdump/dist/tests/bgp_pmsi_tunnel-oobr.out vendor/tcpdump/dist/tests/bgp_pmsi_tunnel-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/bgp_vpn_rt-oobr.out vendor/tcpdump/dist/tests/bgp_vpn_rt-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/bootp_asan-2.out vendor/tcpdump/dist/tests/bootp_asan-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/bootp_asan.out vendor/tcpdump/dist/tests/bootp_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/cfm_sender_id-oobr.out vendor/tcpdump/dist/tests/cfm_sender_id-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/decnet-oobr.out vendor/tcpdump/dist/tests/decnet-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/decnet-shorthdr-oobr.out vendor/tcpdump/dist/tests/decnet-shorthdr-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/dhcp6_reconf_asan.out vendor/tcpdump/dist/tests/dhcp6_reconf_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/dns-zlip-1.out vendor/tcpdump/dist/tests/dns-zlip-1.pcap (contents, props changed) vendor/tcpdump/dist/tests/dns-zlip-2.out vendor/tcpdump/dist/tests/dns-zlip-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/dns-zlip-3.out vendor/tcpdump/dist/tests/dns-zlip-3.pcap (contents, props changed) vendor/tcpdump/dist/tests/dns_fwdptr.out vendor/tcpdump/dist/tests/dns_fwdptr.pcap (contents, props changed) vendor/tcpdump/dist/tests/eap_extract_read2_asan.out vendor/tcpdump/dist/tests/eap_extract_read2_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/eigrp-tlv-oobr.out vendor/tcpdump/dist/tests/eigrp-tlv-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/esis_opt_prot-oobr.out vendor/tcpdump/dist/tests/esis_opt_prot-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/esis_snpa_asan-2.out vendor/tcpdump/dist/tests/esis_snpa_asan-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/esis_snpa_asan-3.out vendor/tcpdump/dist/tests/esis_snpa_asan-3.pcap (contents, props changed) vendor/tcpdump/dist/tests/esis_snpa_asan-4.out vendor/tcpdump/dist/tests/esis_snpa_asan-4.pcap (contents, props changed) vendor/tcpdump/dist/tests/esis_snpa_asan-5.out vendor/tcpdump/dist/tests/esis_snpa_asan-5.pcap (contents, props changed) vendor/tcpdump/dist/tests/esis_snpa_asan.out vendor/tcpdump/dist/tests/esis_snpa_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/extract_read2_asan.out vendor/tcpdump/dist/tests/extract_read2_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/getname_2_read4_asan.out vendor/tcpdump/dist/tests/getname_2_read4_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/hncp_dhcpv4data-oobr.out vendor/tcpdump/dist/tests/hncp_dhcpv4data-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/hncp_dhcpv6data-oobr.out vendor/tcpdump/dist/tests/hncp_dhcpv6data-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/hoobr_aodv_extension.out vendor/tcpdump/dist/tests/hoobr_aodv_extension.pcap (contents, props changed) vendor/tcpdump/dist/tests/hoobr_chdlc_print.out vendor/tcpdump/dist/tests/hoobr_chdlc_print.pcap (contents, props changed) vendor/tcpdump/dist/tests/hoobr_juniper.out vendor/tcpdump/dist/tests/hoobr_juniper.pcap (contents, props changed) vendor/tcpdump/dist/tests/hoobr_juniper2.out vendor/tcpdump/dist/tests/hoobr_juniper2.pcap (contents, props changed) vendor/tcpdump/dist/tests/hoobr_juniper3.out vendor/tcpdump/dist/tests/hoobr_juniper3.pcap (contents, props changed) vendor/tcpdump/dist/tests/hoobr_juniper4.out vendor/tcpdump/dist/tests/hoobr_juniper4.pcap (contents, props changed) vendor/tcpdump/dist/tests/hoobr_lookup_nsap.out vendor/tcpdump/dist/tests/hoobr_lookup_nsap.pcap (contents, props changed) vendor/tcpdump/dist/tests/hoobr_nfs_printfh.out vendor/tcpdump/dist/tests/hoobr_nfs_printfh.pcap (contents, props changed) vendor/tcpdump/dist/tests/hoobr_nfs_xid_map_enter.out vendor/tcpdump/dist/tests/hoobr_nfs_xid_map_enter.pcap (contents, props changed) vendor/tcpdump/dist/tests/hoobr_parse_field.out vendor/tcpdump/dist/tests/hoobr_parse_field.pcap (contents, props changed) vendor/tcpdump/dist/tests/hoobr_pimv1.out vendor/tcpdump/dist/tests/hoobr_pimv1.pcap (contents, props changed) vendor/tcpdump/dist/tests/hoobr_ripng_print.out vendor/tcpdump/dist/tests/hoobr_ripng_print.pcap (contents, props changed) vendor/tcpdump/dist/tests/hoobr_rt6_print.out vendor/tcpdump/dist/tests/hoobr_rt6_print.pcap (contents, props changed) vendor/tcpdump/dist/tests/hoobr_safeputs.out vendor/tcpdump/dist/tests/hoobr_safeputs.pcap (contents, props changed) vendor/tcpdump/dist/tests/icmp-cksum-oobr-1.out vendor/tcpdump/dist/tests/icmp-cksum-oobr-1.pcap (contents, props changed) vendor/tcpdump/dist/tests/icmp-cksum-oobr-2.out vendor/tcpdump/dist/tests/icmp-cksum-oobr-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/icmp-cksum-oobr-3.out vendor/tcpdump/dist/tests/icmp-cksum-oobr-3.pcap (contents, props changed) vendor/tcpdump/dist/tests/icmp-cksum-oobr-4.out vendor/tcpdump/dist/tests/icmp-cksum-oobr-4.pcap (contents, props changed) vendor/tcpdump/dist/tests/icmp6_mobileprefix_asan.out vendor/tcpdump/dist/tests/icmp6_mobileprefix_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/icmp6_nodeinfo_oobr.out vendor/tcpdump/dist/tests/icmp6_nodeinfo_oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/ieee802.11_rates_oobr.out vendor/tcpdump/dist/tests/ieee802.11_rates_oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/ieee802.11_tim_ie_oobr.out vendor/tcpdump/dist/tests/ieee802.11_tim_ie_oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/ikev1_id_ipv6_addr_subnet-oobr.out vendor/tcpdump/dist/tests/ikev1_id_ipv6_addr_subnet-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/ip6_frag_asan.out vendor/tcpdump/dist/tests/ip6_frag_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/ip_printroute_asan.out vendor/tcpdump/dist/tests/ip_printroute_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/ip_ts_opts_asan.out vendor/tcpdump/dist/tests/ip_ts_opts_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/ipv6-mobility-header-oobr.out vendor/tcpdump/dist/tests/ipv6-mobility-header-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/ipv6-next-header-oobr-1.out vendor/tcpdump/dist/tests/ipv6-next-header-oobr-1.pcap (contents, props changed) vendor/tcpdump/dist/tests/ipv6-next-header-oobr-2.out vendor/tcpdump/dist/tests/ipv6-next-header-oobr-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/ipv6-rthdr-oobr.out vendor/tcpdump/dist/tests/ipv6-rthdr-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/isakmp-3948-oobr-2.out vendor/tcpdump/dist/tests/isakmp-3948-oobr-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/isakmp-no-none-np.out vendor/tcpdump/dist/tests/isakmp-no-none-np.pcap (contents, props changed) vendor/tcpdump/dist/tests/isakmp-rfc3948-oobr.out vendor/tcpdump/dist/tests/isakmp-rfc3948-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/isakmp-various-oobr.out vendor/tcpdump/dist/tests/isakmp-various-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/isakmpv1-attr-oobr.out vendor/tcpdump/dist/tests/isakmpv1-attr-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/isis-areaaddr-oobr-1.out vendor/tcpdump/dist/tests/isis-areaaddr-oobr-1.pcap (contents, props changed) vendor/tcpdump/dist/tests/isis-areaaddr-oobr-2.out vendor/tcpdump/dist/tests/isis-areaaddr-oobr-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/isis-extd-ipreach-oobr.out vendor/tcpdump/dist/tests/isis-extd-ipreach-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/isis-extd-isreach-oobr.out vendor/tcpdump/dist/tests/isis-extd-isreach-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/isis_stlv_asan-2.out vendor/tcpdump/dist/tests/isis_stlv_asan-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/isis_stlv_asan-3.out vendor/tcpdump/dist/tests/isis_stlv_asan-3.pcap (contents, props changed) vendor/tcpdump/dist/tests/isis_stlv_asan-4.out vendor/tcpdump/dist/tests/isis_stlv_asan-4.pcap (contents, props changed) vendor/tcpdump/dist/tests/isis_stlv_asan.out vendor/tcpdump/dist/tests/isis_stlv_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/isis_sysid_asan.out vendor/tcpdump/dist/tests/isis_sysid_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/isoclns-oobr.out vendor/tcpdump/dist/tests/isoclns-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/juniper_atm1.out vendor/tcpdump/dist/tests/juniper_atm1.pcap (contents, props changed) vendor/tcpdump/dist/tests/juniper_es.out vendor/tcpdump/dist/tests/juniper_es.pcap (contents, props changed) vendor/tcpdump/dist/tests/l2tp-avp-overflow.out vendor/tcpdump/dist/tests/l2tp-avp-overflow.pcap (contents, props changed) vendor/tcpdump/dist/tests/lldp-infinite-loop-1.out vendor/tcpdump/dist/tests/lldp-infinite-loop-1.pcap (contents, props changed) vendor/tcpdump/dist/tests/lldp-infinite-loop-2.out vendor/tcpdump/dist/tests/lldp-infinite-loop-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/lldp_8023_mtu-oobr.out vendor/tcpdump/dist/tests/lldp_8023_mtu-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/lldp_asan.out vendor/tcpdump/dist/tests/lldp_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/lldp_mgmt_addr_tlv_asan.out vendor/tcpdump/dist/tests/lldp_mgmt_addr_tlv_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/lmpv1_busyloop.out vendor/tcpdump/dist/tests/lmpv1_busyloop.pcap (contents, props changed) vendor/tcpdump/dist/tests/mlppp-oobr.out vendor/tcpdump/dist/tests/mlppp-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/mobility_opt_asan.out vendor/tcpdump/dist/tests/mobility_opt_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/mobility_opt_asan_2.out vendor/tcpdump/dist/tests/mobility_opt_asan_2.pcap (contents, props changed) vendor/tcpdump/dist/tests/mobility_opt_asan_3.out vendor/tcpdump/dist/tests/mobility_opt_asan_3.pcap (contents, props changed) vendor/tcpdump/dist/tests/mobility_opt_asan_4.out vendor/tcpdump/dist/tests/mobility_opt_asan_4.pcap (contents, props changed) vendor/tcpdump/dist/tests/mobility_opt_asan_5.out vendor/tcpdump/dist/tests/mobility_opt_asan_5.pcap (contents, props changed) vendor/tcpdump/dist/tests/mobility_opt_asan_6.out vendor/tcpdump/dist/tests/mobility_opt_asan_6.pcap (contents, props changed) vendor/tcpdump/dist/tests/mobility_opt_asan_7.out vendor/tcpdump/dist/tests/mobility_opt_asan_7.pcap (contents, props changed) vendor/tcpdump/dist/tests/mobility_opt_asan_8.out vendor/tcpdump/dist/tests/mobility_opt_asan_8.pcap (contents, props changed) vendor/tcpdump/dist/tests/mptcp-dss-oobr.out vendor/tcpdump/dist/tests/mptcp-dss-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/nbns-valgrind.out vendor/tcpdump/dist/tests/nbns-valgrind.pcap (contents, props changed) vendor/tcpdump/dist/tests/nfs-attr-oobr.out vendor/tcpdump/dist/tests/nfs-attr-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/olsr-oobr-1.out vendor/tcpdump/dist/tests/olsr-oobr-1.pcap (contents, props changed) vendor/tcpdump/dist/tests/olsr-oobr-2.out vendor/tcpdump/dist/tests/olsr-oobr-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/oobr_parse_elements.out vendor/tcpdump/dist/tests/oobr_parse_elements.pcap (contents, props changed) vendor/tcpdump/dist/tests/ospf6_decode_v3_asan.out vendor/tcpdump/dist/tests/ospf6_decode_v3_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/pgm_group_addr_asan.out vendor/tcpdump/dist/tests/pgm_group_addr_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/pgm_opts_asan.out vendor/tcpdump/dist/tests/pgm_opts_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/pgm_opts_asan_2.out vendor/tcpdump/dist/tests/pgm_opts_asan_2.pcap (contents, props changed) vendor/tcpdump/dist/tests/pgm_opts_asan_3.out vendor/tcpdump/dist/tests/pgm_opts_asan_3.pcap (contents, props changed) vendor/tcpdump/dist/tests/pim_header_asan-2.out vendor/tcpdump/dist/tests/pim_header_asan-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/pim_header_asan-3.out vendor/tcpdump/dist/tests/pim_header_asan-3.pcap (contents, props changed) vendor/tcpdump/dist/tests/pim_header_asan-4.out vendor/tcpdump/dist/tests/pim_header_asan-4.pcap (contents, props changed) vendor/tcpdump/dist/tests/pim_header_asan.out vendor/tcpdump/dist/tests/pim_header_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/pimv2-oobr-1.out vendor/tcpdump/dist/tests/pimv2-oobr-1.pcap (contents, props changed) vendor/tcpdump/dist/tests/pimv2-oobr-2.out vendor/tcpdump/dist/tests/pimv2-oobr-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/pimv2-oobr-3.out vendor/tcpdump/dist/tests/pimv2-oobr-3.pcap (contents, props changed) vendor/tcpdump/dist/tests/pimv2-oobr-4.out vendor/tcpdump/dist/tests/pimv2-oobr-4.pcap (contents, props changed) vendor/tcpdump/dist/tests/pktap-heap-overflow.out vendor/tcpdump/dist/tests/pktap-heap-overflow.pcap (contents, props changed) vendor/tcpdump/dist/tests/ppp_ccp_config_deflate_option_asan.out vendor/tcpdump/dist/tests/ppp_ccp_config_deflate_option_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/radius_attr_asan.out vendor/tcpdump/dist/tests/radius_attr_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/resp_4_infiniteloop.out vendor/tcpdump/dist/tests/resp_4_infiniteloop.pcap (contents, props changed) vendor/tcpdump/dist/tests/rpki-rtr-oob.out vendor/tcpdump/dist/tests/rpki-rtr-oob.pcap (contents, props changed) vendor/tcpdump/dist/tests/rsvp_fast_reroute-oobr.out vendor/tcpdump/dist/tests/rsvp_fast_reroute-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/rsvp_uni-oobr-1.out vendor/tcpdump/dist/tests/rsvp_uni-oobr-1.pcap (contents, props changed) vendor/tcpdump/dist/tests/rsvp_uni-oobr-2.out vendor/tcpdump/dist/tests/rsvp_uni-oobr-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/rsvp_uni-oobr-3.out vendor/tcpdump/dist/tests/rsvp_uni-oobr-3.pcap (contents, props changed) vendor/tcpdump/dist/tests/rx_ubik-oobr.out vendor/tcpdump/dist/tests/rx_ubik-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/slip-bad-direction.out vendor/tcpdump/dist/tests/slip-bad-direction.pcap (contents, props changed) vendor/tcpdump/dist/tests/stp-v4-length-sigsegv.out vendor/tcpdump/dist/tests/stp-v4-length-sigsegv.pcap (contents, props changed) vendor/tcpdump/dist/tests/telnet-iac-check-oobr.out vendor/tcpdump/dist/tests/telnet-iac-check-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/tok2str-oobr-1.out vendor/tcpdump/dist/tests/tok2str-oobr-1.pcap (contents, props changed) vendor/tcpdump/dist/tests/tok2str-oobr-2.out vendor/tcpdump/dist/tests/tok2str-oobr-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/vqp-oobr.out vendor/tcpdump/dist/tests/vqp-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/vtp_asan-2.out vendor/tcpdump/dist/tests/vtp_asan-2.pcap (contents, props changed) vendor/tcpdump/dist/tests/vtp_asan-3.out vendor/tcpdump/dist/tests/vtp_asan-3.pcap (contents, props changed) vendor/tcpdump/dist/tests/vtp_asan.out vendor/tcpdump/dist/tests/vtp_asan.pcap (contents, props changed) vendor/tcpdump/dist/tests/wb-oobr.out vendor/tcpdump/dist/tests/wb-oobr.pcap (contents, props changed) vendor/tcpdump/dist/tests/zephyr-oobr.out vendor/tcpdump/dist/tests/zephyr-oobr.pcap (contents, props changed) Modified: vendor/tcpdump/dist/CHANGES vendor/tcpdump/dist/CONTRIBUTING vendor/tcpdump/dist/CREDITS vendor/tcpdump/dist/INSTALL.txt vendor/tcpdump/dist/Makefile.in vendor/tcpdump/dist/PLATFORMS vendor/tcpdump/dist/README.md vendor/tcpdump/dist/VERSION vendor/tcpdump/dist/aclocal.m4 vendor/tcpdump/dist/addrtoname.c vendor/tcpdump/dist/addrtoname.h vendor/tcpdump/dist/addrtostr.c vendor/tcpdump/dist/af.c vendor/tcpdump/dist/af.h vendor/tcpdump/dist/checksum.c vendor/tcpdump/dist/config.h.in vendor/tcpdump/dist/configure vendor/tcpdump/dist/configure.in vendor/tcpdump/dist/extract.h vendor/tcpdump/dist/gmpls.c vendor/tcpdump/dist/gmpls.h vendor/tcpdump/dist/ip6.h vendor/tcpdump/dist/ipproto.c vendor/tcpdump/dist/ipproto.h vendor/tcpdump/dist/l2vpn.c vendor/tcpdump/dist/l2vpn.h vendor/tcpdump/dist/netdissect-stdinc.h vendor/tcpdump/dist/netdissect.h vendor/tcpdump/dist/nlpid.c vendor/tcpdump/dist/nlpid.h vendor/tcpdump/dist/oui.c vendor/tcpdump/dist/oui.h vendor/tcpdump/dist/print-802_11.c vendor/tcpdump/dist/print-802_15_4.c vendor/tcpdump/dist/print-aodv.c vendor/tcpdump/dist/print-arp.c vendor/tcpdump/dist/print-atm.c vendor/tcpdump/dist/print-beep.c vendor/tcpdump/dist/print-bfd.c vendor/tcpdump/dist/print-bgp.c vendor/tcpdump/dist/print-bootp.c vendor/tcpdump/dist/print-cfm.c vendor/tcpdump/dist/print-chdlc.c vendor/tcpdump/dist/print-cnfp.c vendor/tcpdump/dist/print-decnet.c vendor/tcpdump/dist/print-dhcp6.c vendor/tcpdump/dist/print-domain.c vendor/tcpdump/dist/print-eap.c vendor/tcpdump/dist/print-eigrp.c vendor/tcpdump/dist/print-esp.c vendor/tcpdump/dist/print-ether.c vendor/tcpdump/dist/print-fr.c vendor/tcpdump/dist/print-frag6.c vendor/tcpdump/dist/print-gre.c vendor/tcpdump/dist/print-hncp.c vendor/tcpdump/dist/print-icmp.c vendor/tcpdump/dist/print-icmp6.c vendor/tcpdump/dist/print-ip.c vendor/tcpdump/dist/print-ip6.c vendor/tcpdump/dist/print-ip6opts.c vendor/tcpdump/dist/print-isakmp.c vendor/tcpdump/dist/print-isoclns.c vendor/tcpdump/dist/print-juniper.c vendor/tcpdump/dist/print-l2tp.c vendor/tcpdump/dist/print-ldp.c vendor/tcpdump/dist/print-llc.c vendor/tcpdump/dist/print-lldp.c vendor/tcpdump/dist/print-lmp.c vendor/tcpdump/dist/print-lspping.c vendor/tcpdump/dist/print-m3ua.c vendor/tcpdump/dist/print-mobility.c vendor/tcpdump/dist/print-mpcp.c vendor/tcpdump/dist/print-mpls.c vendor/tcpdump/dist/print-mptcp.c vendor/tcpdump/dist/print-nfs.c vendor/tcpdump/dist/print-null.c vendor/tcpdump/dist/print-olsr.c vendor/tcpdump/dist/print-ospf6.c vendor/tcpdump/dist/print-pgm.c vendor/tcpdump/dist/print-pim.c vendor/tcpdump/dist/print-pktap.c vendor/tcpdump/dist/print-ppp.c vendor/tcpdump/dist/print-radius.c vendor/tcpdump/dist/print-resp.c vendor/tcpdump/dist/print-ripng.c vendor/tcpdump/dist/print-rpki-rtr.c vendor/tcpdump/dist/print-rsvp.c vendor/tcpdump/dist/print-rt6.c vendor/tcpdump/dist/print-rx.c vendor/tcpdump/dist/print-sip.c vendor/tcpdump/dist/print-sl.c vendor/tcpdump/dist/print-slow.c vendor/tcpdump/dist/print-stp.c vendor/tcpdump/dist/print-syslog.c vendor/tcpdump/dist/print-telnet.c vendor/tcpdump/dist/print-tftp.c vendor/tcpdump/dist/print-vqp.c vendor/tcpdump/dist/print-vtp.c vendor/tcpdump/dist/print-wb.c vendor/tcpdump/dist/print-zephyr.c vendor/tcpdump/dist/print.c vendor/tcpdump/dist/signature.c vendor/tcpdump/dist/signature.h vendor/tcpdump/dist/smbutil.c vendor/tcpdump/dist/tcpdump.1.in vendor/tcpdump/dist/tcpdump.c vendor/tcpdump/dist/tests/TESTLIST vendor/tcpdump/dist/tests/TESTonce vendor/tcpdump/dist/tests/TESTrun.sh vendor/tcpdump/dist/tests/crypto.sh vendor/tcpdump/dist/tests/heapoverflow-in_checksum.out vendor/tcpdump/dist/tests/isis-seg-fault-1-v.out vendor/tcpdump/dist/tests/isis-seg-fault-1-v.sh vendor/tcpdump/dist/tests/isis-seg-fault-2-v.out vendor/tcpdump/dist/tests/isis-seg-fault-3-v.out vendor/tcpdump/dist/tests/isis_infloop-v.out vendor/tcpdump/dist/tests/kday2.out vendor/tcpdump/dist/tests/kday3.out vendor/tcpdump/dist/tests/kday4.out vendor/tcpdump/dist/tests/kday5.out vendor/tcpdump/dist/tests/kday6.out vendor/tcpdump/dist/tests/kday7.out vendor/tcpdump/dist/tests/kday8.out vendor/tcpdump/dist/tests/lmp-v.sh vendor/tcpdump/dist/tests/nflog-e.sh vendor/tcpdump/dist/util-print.c Modified: vendor/tcpdump/dist/CHANGES ============================================================================== --- vendor/tcpdump/dist/CHANGES Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/CHANGES Mon Sep 18 04:10:54 2017 (r323696) @@ -1,10 +1,119 @@ +Sunday September 3, 2017 denis@ovsienko.info + Summary for 4.9.2 tcpdump release + Do not use getprotobynumber() for protocol name resolution. Do not do + any protocol name resolution if -n is specified. + Improve errors detection in the test scripts. + Fix a segfault with OpenSSL 1.1 and improve OpenSSL usage. + Clean up IS-IS printing. + Fix buffer overflow vulnerabilities: + CVE-2017-11543 (SLIP) + CVE-2017-13011 (bittok2str_internal) + Fix infinite loop vulnerabilities: + CVE-2017-12989 (RESP) + CVE-2017-12990 (ISAKMP) + CVE-2017-12995 (DNS) + CVE-2017-12997 (LLDP) + Fix buffer over-read vulnerabilities: + CVE-2017-11541 (safeputs) + CVE-2017-11542 (PIMv1) + CVE-2017-12893 (SMB/CIFS) + CVE-2017-12894 (lookup_bytestring) + CVE-2017-12895 (ICMP) + CVE-2017-12896 (ISAKMP) + CVE-2017-12897 (ISO CLNS) + CVE-2017-12898 (NFS) + CVE-2017-12899 (DECnet) + CVE-2017-12900 (tok2strbuf) + CVE-2017-12901 (EIGRP) + CVE-2017-12902 (Zephyr) + CVE-2017-12985 (IPv6) + CVE-2017-12986 (IPv6 routing headers) + CVE-2017-12987 (IEEE 802.11) + CVE-2017-12988 (telnet) + CVE-2017-12991 (BGP) + CVE-2017-12992 (RIPng) + CVE-2017-12993 (Juniper) + CVE-2017-11542 (PIMv1) + CVE-2017-11541 (safeputs) + CVE-2017-12994 (BGP) + CVE-2017-12996 (PIMv2) + CVE-2017-12998 (ISO IS-IS) + CVE-2017-12999 (ISO IS-IS) + CVE-2017-13000 (IEEE 802.15.4) + CVE-2017-13001 (NFS) + CVE-2017-13002 (AODV) + CVE-2017-13003 (LMP) + CVE-2017-13004 (Juniper) + CVE-2017-13005 (NFS) + CVE-2017-13006 (L2TP) + CVE-2017-13007 (Apple PKTAP) + CVE-2017-13008 (IEEE 802.11) + CVE-2017-13009 (IPv6 mobility) + CVE-2017-13010 (BEEP) + CVE-2017-13012 (ICMP) + CVE-2017-13013 (ARP) + CVE-2017-13014 (White Board) + CVE-2017-13015 (EAP) + CVE-2017-11543 (SLIP) + CVE-2017-13016 (ISO ES-IS) + CVE-2017-13017 (DHCPv6) + CVE-2017-13018 (PGM) + CVE-2017-13019 (PGM) + CVE-2017-13020 (VTP) + CVE-2017-13021 (ICMPv6) + CVE-2017-13022 (IP) + CVE-2017-13023 (IPv6 mobility) + CVE-2017-13024 (IPv6 mobility) + CVE-2017-13025 (IPv6 mobility) + CVE-2017-13026 (ISO IS-IS) + CVE-2017-13027 (LLDP) + CVE-2017-13028 (BOOTP) + CVE-2017-13029 (PPP) + CVE-2017-13030 (PIM) + CVE-2017-13031 (IPv6 fragmentation header) + CVE-2017-13032 (RADIUS) + CVE-2017-13033 (VTP) + CVE-2017-13034 (PGM) + CVE-2017-13035 (ISO IS-IS) + CVE-2017-13036 (OSPFv3) + CVE-2017-13037 (IP) + CVE-2017-13038 (PPP) + CVE-2017-13039 (ISAKMP) + CVE-2017-13040 (MPTCP) + CVE-2017-13041 (ICMPv6) + CVE-2017-13042 (HNCP) + CVE-2017-13043 (BGP) + CVE-2017-13044 (HNCP) + CVE-2017-13045 (VQP) + CVE-2017-13046 (BGP) + CVE-2017-13047 (ISO ES-IS) + CVE-2017-13048 (RSVP) + CVE-2017-13049 (Rx) + CVE-2017-13050 (RPKI-Router) + CVE-2017-13051 (RSVP) + CVE-2017-13052 (CFM) + CVE-2017-13053 (BGP) + CVE-2017-13054 (LLDP) + CVE-2017-13055 (ISO IS-IS) + CVE-2017-13687 (Cisco HDLC) + CVE-2017-13688 (OLSR) + CVE-2017-13689 (IKEv1) + CVE-2017-13690 (IKEv2) + CVE-2017-13725 (IPv6 routing headers) + +Sunday July 23, 2017 denis@ovsienko.info + Summary for 4.9.1 tcpdump release + CVE-2017-11108/Fix bounds checking for STP. + Make assorted documentation updates and fix a few typos in tcpdump output. + Fixup -C for file size >2GB (GH #488). + Show AddressSanitizer presence in version output. + Fix a bug in test scripts (exposed in GH #613). + On FreeBSD adjust Capsicum capabilities for netmap. + On Linux fix a use-after-free when the requested interface does not exist. + Wednesday January 18, 2017 devel.fx.lebail@orange.fr Summary for 4.9.0 tcpdump release General updates: - Improve separation frontend/backend (tcpdump/libnetdissect) - Don't require IPv6 library support in order to support IPv6 addresses - Introduce data types to use for integral values in packet structures - Fix display of timestamps with -tt, -ttt and -ttttt options Fix some heap overflows found with American Fuzzy Lop by Hanno Boeck and others (More information in the log with CVE-2016-* and CVE-2017-*) Change the way protocols print link-layer addresses (Fix heap overflows @@ -35,14 +144,6 @@ Wednesday January 18, 2017 devel.fx.lebail@orange.fr Don't drop CAP_SYS_CHROOT before chrooting Fixes issue where statistics not reported when -G and -W options used - New printers supporting: - Generic Protocol Extension for VXLAN (VXLAN-GPE) - Home Networking Control Protocol (HNCP), RFCs 7787 and 7788 - Locator/Identifier Separation Protocol (LISP), type 3 and type 4 packets - Marvell Extended Distributed Switch Architecture header (MEDSA) - Network Service Header (NSH) - REdis Serialization Protocol (RESP) - Updated printers: 802.11: Beginnings of 11ac radiotap support 802.11: Check the Protected bit for management frames @@ -61,7 +162,6 @@ Wednesday January 18, 2017 devel.fx.lebail@orange.fr ATM: Fix an incorrect bounds check BFD: Update specification from draft to RFC 5880 BFD: Update to print optional authentication field - BGP: Add decoding of ADD-PATH capability BGP: Add support for the AIGP attribute (RFC7311) BGP: Print LARGE_COMMUNITY Path Attribute BGP: Update BGP numbers from IANA; Print minor values for FSM notification @@ -78,7 +178,6 @@ Wednesday January 18, 2017 devel.fx.lebail@orange.fr DTP: Improve packet integrity checks EGP: Fix bounds checks ESP: Don't use OpenSSL_add_all_algorithms() in OpenSSL 1.1.0 or later - ESP: Handle OpenSSL 1.1.x Ethernet: Add some bounds checking before calling isoclns_print (Fix a heap overflow) Ethernet: Print the Length/Type field as length when needed FDDI: Fix -e output for FDDI @@ -87,7 +186,6 @@ Wednesday January 18, 2017 devel.fx.lebail@orange.fr Geneve: Fix error message with invalid option length; Update list option classes HNCP: Fix incorrect time interval format. Fix handling of IPv4 prefixes ICMP6: Fetch a 32-bit big-endian quantity with EXTRACT_32BITS() - ICMP6: dagid is always an IPv6 address, not an opaque 128-bit string IGMP: Add a length check IP: Add a bounds check (Fix a heap overflow) IP: Check before fetching the protocol version (Fix a heap overflow) @@ -115,7 +213,6 @@ Wednesday January 18, 2017 devel.fx.lebail@orange.fr MPLS LSP ping: Update printing for RFC 4379, bug fixes, more bounds checks MPLS: "length" is now the *remaining* packet length MPLS: Add bounds and length checks (Fix a heap overflow) - NFS: Add a test that makes unaligned accesses NFS: Don't assume the ONC RPC header is nicely aligned NFS: Don't overflow the Opaque_Handle buffer (Fix a segmentation fault) NFS: Don't run past the end of an NFSv3 file handle @@ -130,7 +227,6 @@ Wednesday January 18, 2017 devel.fx.lebail@orange.fr PGM: Print the formatted IP address, not the raw binary address, as a string PIM: Add some bounds checking (Fix a heap overflow) PIMv2: Fix checksumming of Register messages - PPI: Pass an adjusted struct pcap_pkthdr to the sub-printer PPP: Add some bounds checks (Fix a heap overflow) PPP: Report invalid PAP AACK/ANAK packets Q.933: Add a missing bounds check @@ -171,16 +267,46 @@ Wednesday January 18, 2017 devel.fx.lebail@orange.fr UDLD: Fix an infinite loop UDP: Add a bounds check (Fix a heap overflow) UDP: Check against the packet length first - UDP: Don't do the DDP-over-UDP heuristic check up front VAT: Add some bounds checks VTP: Add a test on Mgmt Domain Name length VTP: Add bounds checks and filter out non-printable characters VXLAN: Add a bound check and a test case ZeroMQ: Fix an infinite loop -Tuesday April 14, 2015 guy@alum.mit.edu - Summary for 4.8.0 tcpdump release +Tuesday October 25, 2016 mcr@sandelman.ca + Summary for 4.8.1 tcpdump release Fix "-x" for Apple PKTAP and PPI packets + Improve separation frontend/backend (tcpdump/libnetdissect) + Fix display of timestamps with -tt, -ttt and -ttttt options + Add support for the Marvell Extended Distributed Switch Architecture header + Use PRIx64 to print a 64-bit number in hex. + Printer for HNCP (RFCs 7787 and 7788). + dagid is always an IPv6 address, not an opaque 128-bit string, and other fixes to RPL printer. + RSVP: Add bounds and length checks + OSPF: Do more bounds checking + Handle OpenSSL 1.1.x. + Initial support for the REdis Serialization Protocol known as RESP. + Add printing function for Generic Protocol Extension for VXLAN + draft-ietf-nvo3-vxlan-gpe-01 + Network Service Header: draft-ietf-sfc-nsh-01 + Don't recompile the filter if the new file has the same DLT. + Pass an adjusted struct pcap_pkthdr to the sub-printer. + Add three test cases for already fixed CVEs + CVE-2014-8767: OLSR + CVE-2014-8768: Geonet + CVE-2014-8769: AODV + Don't do the DDP-over-UDP heuristic first: GitHub issue #499. + Use the new debugging routines in libpcap. + Harmonize TCP source or destination ports tests with UDP ones + Introduce data types to use for integral values in packet structures. + RSVP: Fix an infinite loop + Support of Type 3 and Type 4 LISP packets. + Don't require IPv6 library support in order to support IPv6 addresses. + Many many changes to support libnetdissect usage. + Add a test that makes unaligned accesses: GitHub issue #478. + add a DNSSEC test case: GH #445 and GH #467. + BGP: add decoding of ADD-PATH capability + fixes to LLC header printing, and RFC948-style IP packets Friday April 10, 2015 guy@alum.mit.edu Summary for 4.7.4 tcpdump release Modified: vendor/tcpdump/dist/CONTRIBUTING ============================================================================== --- vendor/tcpdump/dist/CONTRIBUTING Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/CONTRIBUTING Mon Sep 18 04:10:54 2017 (r323696) @@ -3,6 +3,44 @@ Some Information for Contributors You want to contribute to Tcpdump, Thanks! Please, read these lines. + +How to report bugs and other problems +------------------------------------- +To report a security issue (segfault, buffer overflow, infinite loop, arbitrary +code execution etc) please send an e-mail to security@tcpdump.org, do not use +the bug tracker! + +To report a non-security problem (failure to compile, incorrect output in the +protocol printout, missing support for a particular protocol etc) please check +first that it reproduces with the latest stable release of tcpdump and the latest +stable release of libpcap. If it does, please check that the problem reproduces +with the current git master branch of tcpdump and the current git master branch of +libpcap. If it does (and it is not a security-related problem, otherwise see +above), please navigate to https://github.com/the-tcpdump-group/tcpdump/issues +and check if the problem has already been reported. If it has not, please open +a new issue and provide the following details: + +* tcpdump and libpcap version (tcpdump --version) +* operating system name and version and any other details that may be relevant + (uname -a, compiler name and version, CPU type etc.) +* configure flags if any were used +* statement of the problem +* steps to reproduce + +Please note that if you know exactly how to solve the problem and the solution +would not be too intrusive, it would be best to contribute some development time +and open a pull request instead as discussed below. + +Still not sure how to do? Feel free to [subscribe](http://www.tcpdump.org/#mailing-lists) +to the mailing list tcpdump-workers@lists.tcpdump.org and ask! + + +How to add new code and to update existing code +----------------------------------------------- + +0) Check that there isn't a pull request already opened for the changes you + intend to make. + 1) Fork the Tcpdump repository on GitHub from https://github.com/the-tcpdump-group/tcpdump (See https://help.github.com/articles/fork-a-repo/) @@ -12,8 +50,11 @@ Please, read these lines. on Linux and OSX before sending pull requests. (See http://docs.travis-ci.com/user/getting-started/) -3) Clone your repository +3) Setup your git working copy git clone https://github.com//tcpdump.git + cd tcpdump + git remote add upstream https://github.com/the-tcpdump-group/tcpdump + git fetch upstream 4) Do a 'touch .devel' in your working directory. Currently, the effect is @@ -47,19 +88,26 @@ Please, read these lines. 7) Test with 'make check' Don't send a pull request if 'make check' gives failed tests. -8) Rebase your commits against upstream/master - (To keep linearity) +8) Try to rebase your commits to keep the history simple. + git rebase upstream/master + (If the rebase fails and you cannot resolve, issue "git rebase --abort" + and ask for help in the pull request comment.) -9) Initiate and send a pull request +9) Once 100% happy, put your work into your forked repository. + git push + +10) Initiate and send a pull request (See https://help.github.com/articles/using-pull-requests/) -Some remarks ------------- + +Code style and generic remarks +------------------------------ a) A thorough reading of some other printers code is useful. b) Put the normative reference if any as comments (RFC, etc.). -c) Put the format of packets/headers/options as comments. +c) Put the format of packets/headers/options as comments if there is no + published normative reference. d) The printer may receive incomplete packet in the buffer, truncated at any random position, for example by capturing with '-s size' option. Modified: vendor/tcpdump/dist/CREDITS ============================================================================== --- vendor/tcpdump/dist/CREDITS Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/CREDITS Mon Sep 18 04:10:54 2017 (r323696) @@ -5,7 +5,7 @@ The current maintainers: Denis Ovsienko Fulvio Risso Guy Harris - Hannes Gredler + Hannes Gredler Michael Richardson Francois-Xavier Le Bail @@ -39,6 +39,7 @@ Additional people who have contributed patches: Bjoern A. Zeeb Bram Brent L. Bates + Brian Carpenter Brian Ginsbach Bruce M. Simpson Carles Kishimoto Bisbe @@ -54,6 +55,7 @@ Additional people who have contributed patches: Craig Rodrigues Crist J. Clark Daniel Hagerty + Daniel Lee Darren Reed David Binderman David Horn @@ -85,6 +87,7 @@ Additional people who have contributed patches: Greg Stark Hank Leininger Hannes Viertel + Hanno Böck Harry Raaymakers Heinz-Ado Arnolds Hendrik Scholz @@ -111,6 +114,7 @@ Additional people who have contributed patches: Juliusz Chroboczek Kaarthik Sivakumar Kaladhar Musunuru + Kamil Frankowicz Karl Norby Kazushi Sugyo Kelly Carmichael @@ -123,7 +127,6 @@ Additional people who have contributed patches: Larry Lile Lennert Buytenhek Loganaden Velvindron - Daniel Lee Loris Degioanni Love Hörnquist-Åstrand Lucas C. Villa Real @@ -166,6 +169,7 @@ Additional people who have contributed patches: Paolo Abeni Pascal Hennequin Pasvorn Boonmark + Patrik Lundquist Paul Ferrell Paul Mundt Paul S. Traina Modified: vendor/tcpdump/dist/INSTALL.txt ============================================================================== --- vendor/tcpdump/dist/INSTALL.txt Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/INSTALL.txt Mon Sep 18 04:10:54 2017 (r323696) @@ -37,6 +37,7 @@ Please see "PLATFORMS" for notes about tested platform FILES ----- CHANGES - description of differences between releases +CONTRIBUTING - guidelines for contributing CREDITS - people that have helped tcpdump along INSTALL.txt - this file LICENSE - the license under which tcpdump is distributed Modified: vendor/tcpdump/dist/Makefile.in ============================================================================== --- vendor/tcpdump/dist/Makefile.in Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/Makefile.in Mon Sep 18 04:10:54 2017 (r323696) @@ -263,6 +263,7 @@ HDR = \ ether.h \ ethertype.h \ extract.h \ + funcattrs.h \ getopt_long.h \ gmpls.h \ gmt2local.h \ Modified: vendor/tcpdump/dist/PLATFORMS ============================================================================== --- vendor/tcpdump/dist/PLATFORMS Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/PLATFORMS Mon Sep 18 04:10:54 2017 (r323696) @@ -1,9 +1,16 @@ -== Tested platforms == -NetBSD 5.1/i386 (mcr - 2012/4/1) -Debian Linux (squeeze/i386) (mcr - 2012/4/1) +In many operating systems tcpdump is available as a native package or port, +which simplifies installation of updates and long-term maintenance. However, +the native packages are sometimes a few versions behind and to try a more +recent snapshot it will take to compile tcpdump from the source code. ---- -RedHat Linux 6.1/i386 (assar) -FreeBSD 2.2.8/i386 (itojun) +tcpdump compiles and works on at least the following platforms: - +* AIX +* FreeBSD +* HP-UX 11i +* Linux (any) with glibc (usually just works) +* Linux (any) with musl libc (sometimes fails to compile, please report any bugs) +* Mac OS X / macOS +* NetBSD +* OpenWrt +* Solaris Modified: vendor/tcpdump/dist/README.md ============================================================================== --- vendor/tcpdump/dist/README.md Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/README.md Mon Sep 18 04:10:54 2017 (r323696) @@ -3,25 +3,21 @@ [![Build Status](https://travis-ci.org/the-tcpdump-group/tcpdump.png)](https://travis-ci.org/the-tcpdump-group/tcpdump) -TCPDUMP 4.x.y -Now maintained by "The Tcpdump Group" -See www.tcpdump.org +To report a security issue please send an e-mail to security@tcpdump.org. -Please send inquiries/comments/reports to: +To report bugs and other problems, contribute patches, request a +feature, provide generic feedback etc please see the file +CONTRIBUTING in the tcpdump source tree root. -* tcpdump-workers@lists.tcpdump.org +TCPDUMP 4.x.y +Now maintained by "The Tcpdump Group" +See www.tcpdump.org Anonymous Git is available via: git clone git://bpf.tcpdump.org/tcpdump -Please submit patches by forking the branch on GitHub at: - -* http://github.com/the-tcpdump-group/tcpdump/tree/master - -and issuing a pull request. - -formerly from Lawrence Berkeley National Laboratory +formerly from Lawrence Berkeley National Laboratory Network Research Group ftp://ftp.ee.lbl.gov/old/tcpdump.tar.Z (3.4) @@ -70,20 +66,6 @@ Another tool that tcpdump users might find useful is t It is a program that can be used to extract portions of tcpdump binary trace files. See the above distribution for further details and documentation. - -Problems, bugs, questions, desirable enhancements, etc. should be sent -to the address "tcpdump-workers@lists.tcpdump.org". Bugs, support -requests, and feature requests may also be submitted on the GitHub issue -tracker for tcpdump at: - -* https://github.com/the-tcpdump-group/tcpdump/issues - -Source code contributions, etc. should be sent to the email address -above or submitted by forking the branch on GitHub at: - -* http://github.com/the-tcpdump-group/tcpdump/tree/master - -and issuing a pull request. Current versions can be found at www.tcpdump.org. Modified: vendor/tcpdump/dist/VERSION ============================================================================== --- vendor/tcpdump/dist/VERSION Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/VERSION Mon Sep 18 04:10:54 2017 (r323696) @@ -1 +1 @@ -4.9.0 +4.9.2 Modified: vendor/tcpdump/dist/aclocal.m4 ============================================================================== --- vendor/tcpdump/dist/aclocal.m4 Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/aclocal.m4 Mon Sep 18 04:10:54 2017 (r323696) @@ -655,7 +655,7 @@ AC_DEFUN(AC_LBL_LIBPCAP, AC_CHECK_FUNC(pcap_loop,, [ AC_MSG_ERROR( -[Report this to tcpdump-workers@lists.tcpdump.org, and include the +[This is a bug, please follow the guidelines in CONTRIBUTING and include the config.log file in your report. If you have downloaded libpcap from tcpdump.org, and built it yourself, please also include the config.log file from the libpcap source directory, the Makefile from the libpcap Modified: vendor/tcpdump/dist/addrtoname.c ============================================================================== --- vendor/tcpdump/dist/addrtoname.c Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/addrtoname.c Mon Sep 18 04:10:54 2017 (r323696) @@ -145,14 +145,24 @@ struct enamemem { u_short e_addr2; const char *e_name; u_char *e_nsap; /* used only for nsaptable[] */ -#define e_bs e_nsap /* for bytestringtable */ struct enamemem *e_nxt; }; static struct enamemem enametable[HASHNAMESIZE]; static struct enamemem nsaptable[HASHNAMESIZE]; -static struct enamemem bytestringtable[HASHNAMESIZE]; +struct bsnamemem { + u_short bs_addr0; + u_short bs_addr1; + u_short bs_addr2; + const char *bs_name; + u_char *bs_bytes; + unsigned int bs_nbytes; + struct bsnamemem *bs_nxt; +}; + +static struct bsnamemem bytestringtable[HASHNAMESIZE]; + struct protoidmem { uint32_t p_oui; u_short p_proto; @@ -321,7 +331,7 @@ getname6(netdissect_options *ndo, const u_char *ap) return (p->name); } -static const char hex[] = "0123456789abcdef"; +static const char hex[16] = "0123456789abcdef"; /* Find the hash node that corresponds the ether address 'ep' */ @@ -359,11 +369,11 @@ lookup_emem(netdissect_options *ndo, const u_char *ep) * with length 'nlen' */ -static inline struct enamemem * +static inline struct bsnamemem * lookup_bytestring(netdissect_options *ndo, register const u_char *bs, const unsigned int nlen) { - struct enamemem *tp; + struct bsnamemem *tp; register u_int i, j, k; if (nlen >= 6) { @@ -378,26 +388,28 @@ lookup_bytestring(netdissect_options *ndo, register co i = j = k = 0; tp = &bytestringtable[(i ^ j) & (HASHNAMESIZE-1)]; - while (tp->e_nxt) - if (tp->e_addr0 == i && - tp->e_addr1 == j && - tp->e_addr2 == k && - memcmp((const char *)bs, (const char *)(tp->e_bs), nlen) == 0) + while (tp->bs_nxt) + if (nlen == tp->bs_nbytes && + tp->bs_addr0 == i && + tp->bs_addr1 == j && + tp->bs_addr2 == k && + memcmp((const char *)bs, (const char *)(tp->bs_bytes), nlen) == 0) return tp; else - tp = tp->e_nxt; + tp = tp->bs_nxt; - tp->e_addr0 = i; - tp->e_addr1 = j; - tp->e_addr2 = k; + tp->bs_addr0 = i; + tp->bs_addr1 = j; + tp->bs_addr2 = k; - tp->e_bs = (u_char *) calloc(1, nlen + 1); - if (tp->e_bs == NULL) + tp->bs_bytes = (u_char *) calloc(1, nlen); + if (tp->bs_bytes == NULL) (*ndo->ndo_error)(ndo, "lookup_bytestring: calloc"); - memcpy(tp->e_bs, bs, nlen); - tp->e_nxt = (struct enamemem *)calloc(1, sizeof(*tp)); - if (tp->e_nxt == NULL) + memcpy(tp->bs_bytes, bs, nlen); + tp->bs_nbytes = nlen; + tp->bs_nxt = (struct bsnamemem *)calloc(1, sizeof(*tp)); + if (tp->bs_nxt == NULL) (*ndo->ndo_error)(ndo, "lookup_bytestring: calloc"); return tp; @@ -424,11 +436,11 @@ lookup_nsap(netdissect_options *ndo, register const u_ tp = &nsaptable[(i ^ j) & (HASHNAMESIZE-1)]; while (tp->e_nxt) - if (tp->e_addr0 == i && + if (nsap_length == tp->e_nsap[0] && + tp->e_addr0 == i && tp->e_addr1 == j && tp->e_addr2 == k && - tp->e_nsap[0] == nsap_length && - memcmp((const char *)&(nsap[1]), + memcmp((const char *)nsap, (char *)&(tp->e_nsap[1]), nsap_length) == 0) return tp; else @@ -528,12 +540,12 @@ le64addr_string(netdissect_options *ndo, const u_char const unsigned int len = 8; register u_int i; register char *cp; - register struct enamemem *tp; + register struct bsnamemem *tp; char buf[BUFSIZE]; tp = lookup_bytestring(ndo, ep, len); - if (tp->e_name) - return (tp->e_name); + if (tp->bs_name) + return (tp->bs_name); cp = buf; for (i = len; i > 0 ; --i) { @@ -545,11 +557,11 @@ le64addr_string(netdissect_options *ndo, const u_char *cp = '\0'; - tp->e_name = strdup(buf); - if (tp->e_name == NULL) + tp->bs_name = strdup(buf); + if (tp->bs_name == NULL) (*ndo->ndo_error)(ndo, "le64addr_string: strdup(buf)"); - return (tp->e_name); + return (tp->bs_name); } const char * @@ -558,7 +570,7 @@ linkaddr_string(netdissect_options *ndo, const u_char { register u_int i; register char *cp; - register struct enamemem *tp; + register struct bsnamemem *tp; if (len == 0) return (""); @@ -570,11 +582,11 @@ linkaddr_string(netdissect_options *ndo, const u_char return (q922_string(ndo, ep, len)); tp = lookup_bytestring(ndo, ep, len); - if (tp->e_name) - return (tp->e_name); + if (tp->bs_name) + return (tp->bs_name); - tp->e_name = cp = (char *)malloc(len*3); - if (tp->e_name == NULL) + tp->bs_name = cp = (char *)malloc(len*3); + if (tp->bs_name == NULL) (*ndo->ndo_error)(ndo, "linkaddr_string: malloc"); *cp++ = hex[*ep >> 4]; *cp++ = hex[*ep++ & 0xf]; @@ -584,7 +596,7 @@ linkaddr_string(netdissect_options *ndo, const u_char *cp++ = hex[*ep++ & 0xf]; } *cp = '\0'; - return (tp->e_name); + return (tp->bs_name); } const char * Modified: vendor/tcpdump/dist/addrtoname.h ============================================================================== --- vendor/tcpdump/dist/addrtoname.h Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/addrtoname.h Mon Sep 18 04:10:54 2017 (r323696) @@ -33,7 +33,8 @@ enum { LINKADDR_ETHER, LINKADDR_FRELAY, LINKADDR_IEEE1394, - LINKADDR_ATM + LINKADDR_ATM, + LINKADDR_OTHER }; #define BUFSIZE 128 Modified: vendor/tcpdump/dist/addrtostr.c ============================================================================== --- vendor/tcpdump/dist/addrtostr.c Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/addrtostr.c Mon Sep 18 04:10:54 2017 (r323696) @@ -110,25 +110,24 @@ addrtostr6 (const void *src, char *dst, size_t size) size_t space_left, added_space; int snprintfed; struct { - long base; - long len; + int base; + int len; } best, cur; - u_long words [IN6ADDRSZ / INT16SZ]; - u_int i; + uint16_t words [IN6ADDRSZ / INT16SZ]; + int i; /* Preprocess: * Copy the input (bytewise) array into a wordwise array. * Find the longest run of 0x00's in src[] for :: shorthanding. */ - memset (words, 0, sizeof(words)); - for (i = 0; i < IN6ADDRSZ; i++) - words[i/2] |= (srcaddr[i] << ((1 - (i % 2)) << 3)); + for (i = 0; i < (IN6ADDRSZ / INT16SZ); i++) + words[i] = (srcaddr[2*i] << 8) | srcaddr[2*i + 1]; best.len = 0; best.base = -1; cur.len = 0; cur.base = -1; - for (i = 0; i < (IN6ADDRSZ / INT16SZ); i++) + for (i = 0; i < (int)(IN6ADDRSZ / INT16SZ); i++) { if (words[i] == 0) { @@ -161,7 +160,7 @@ addrtostr6 (const void *src, char *dst, size_t size) *dp++ = c; \ space_left--; \ } - for (i = 0; i < (IN6ADDRSZ / INT16SZ); i++) + for (i = 0; i < (int)(IN6ADDRSZ / INT16SZ); i++) { /* Are we inside the best run of 0x00's? */ @@ -192,7 +191,7 @@ addrtostr6 (const void *src, char *dst, size_t size) space_left -= added_space; break; } - snprintfed = snprintf (dp, space_left, "%lx", words[i]); + snprintfed = snprintf (dp, space_left, "%x", words[i]); if (snprintfed < 0) return (NULL); if ((size_t) snprintfed >= space_left) Modified: vendor/tcpdump/dist/af.c ============================================================================== --- vendor/tcpdump/dist/af.c Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/af.c Mon Sep 18 04:10:54 2017 (r323696) @@ -12,7 +12,7 @@ * LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS * FOR A PARTICULAR PURPOSE. * - * Original code by Hannes Gredler (hannes@juniper.net) + * Original code by Hannes Gredler (hannes@gredler.at) */ #ifdef HAVE_CONFIG_H Modified: vendor/tcpdump/dist/af.h ============================================================================== --- vendor/tcpdump/dist/af.h Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/af.h Mon Sep 18 04:10:54 2017 (r323696) @@ -12,7 +12,7 @@ * LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS * FOR A PARTICULAR PURPOSE. * - * Original code by Hannes Gredler (hannes@juniper.net) + * Original code by Hannes Gredler (hannes@gredler.at) */ extern const struct tok af_values[]; Modified: vendor/tcpdump/dist/checksum.c ============================================================================== --- vendor/tcpdump/dist/checksum.c Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/checksum.c Mon Sep 18 04:10:54 2017 (r323696) @@ -14,7 +14,7 @@ * * miscellaneous checksumming routines * - * Original code by Hannes Gredler (hannes@juniper.net) + * Original code by Hannes Gredler (hannes@gredler.at) */ #ifdef HAVE_CONFIG_H Modified: vendor/tcpdump/dist/config.h.in ============================================================================== --- vendor/tcpdump/dist/config.h.in Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/config.h.in Mon Sep 18 04:10:54 2017 (r323696) @@ -34,6 +34,9 @@ /* Define to 1 if you have the `ether_ntohost' function. */ #undef HAVE_ETHER_NTOHOST +/* Define to 1 if you have the `EVP_CipherInit_ex' function. */ +#undef HAVE_EVP_CIPHERINIT_EX + /* Define to 1 if you have the `EVP_CIPHER_CTX_new' function. */ #undef HAVE_EVP_CIPHER_CTX_NEW Modified: vendor/tcpdump/dist/configure ============================================================================== --- vendor/tcpdump/dist/configure Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/configure Mon Sep 18 04:10:54 2017 (r323696) @@ -5801,7 +5801,7 @@ if test "x$ac_cv_func_pcap_loop" = xyes; then : else - as_fn_error $? "Report this to tcpdump-workers@lists.tcpdump.org, and include the + as_fn_error $? "This is a bug, please follow the guidelines in CONTRIBUTING and include the config.log file in your report. If you have downloaded libpcap from tcpdump.org, and built it yourself, please also include the config.log file from the libpcap source directory, the Makefile from the libpcap @@ -8116,17 +8116,32 @@ fi done # - # OK, do we have EVP_CIPHER_CTX_new? + # OK, then: + # + # 1) do we have EVP_CIPHER_CTX_new? # If so, we use it to allocate an # EVP_CIPHER_CTX, as EVP_CIPHER_CTX may be # opaque; otherwise, we allocate it ourselves. # - for ac_func in EVP_CIPHER_CTX_new + # 2) do we have EVP_CipherInit_ex()? + # If so, we use it, because we need to be + # able to make two "initialize the cipher" + # calls, one with the cipher and key, and + # one with the IV, and, as of OpenSSL 1.1, + # You Can't Do That with EVP_CipherInit(), + # because a call to EVP_CipherInit() will + # unconditionally clear the context, and + # if you don't supply a cipher, it'll + # clear the cipher, rendering the context + # unusable and causing a crash. + # + for ac_func in EVP_CIPHER_CTX_new EVP_CipherInit_ex do : - ac_fn_c_check_func "$LINENO" "EVP_CIPHER_CTX_new" "ac_cv_func_EVP_CIPHER_CTX_new" -if test "x$ac_cv_func_EVP_CIPHER_CTX_new" = xyes; then : + as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh` +ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var" +if eval test \"x\$"$as_ac_var"\" = x"yes"; then : cat >>confdefs.h <<_ACEOF -#define HAVE_EVP_CIPHER_CTX_NEW 1 +#define `$as_echo "HAVE_$ac_func" | $as_tr_cpp` 1 _ACEOF fi Modified: vendor/tcpdump/dist/configure.in ============================================================================== --- vendor/tcpdump/dist/configure.in Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/configure.in Mon Sep 18 04:10:54 2017 (r323696) @@ -935,12 +935,26 @@ if test "$want_libcrypto" != "no"; then if test "$ac_cv_lib_crypto_DES_cbc_encrypt" = "yes"; then AC_CHECK_HEADERS(openssl/evp.h) # - # OK, do we have EVP_CIPHER_CTX_new? + # OK, then: + # + # 1) do we have EVP_CIPHER_CTX_new? # If so, we use it to allocate an # EVP_CIPHER_CTX, as EVP_CIPHER_CTX may be # opaque; otherwise, we allocate it ourselves. # - AC_CHECK_FUNCS(EVP_CIPHER_CTX_new) + # 2) do we have EVP_CipherInit_ex()? + # If so, we use it, because we need to be + # able to make two "initialize the cipher" + # calls, one with the cipher and key, and + # one with the IV, and, as of OpenSSL 1.1, + # You Can't Do That with EVP_CipherInit(), + # because a call to EVP_CipherInit() will + # unconditionally clear the context, and + # if you don't supply a cipher, it'll + # clear the cipher, rendering the context + # unusable and causing a crash. + # + AC_CHECK_FUNCS(EVP_CIPHER_CTX_new EVP_CipherInit_ex) fi ]) fi Modified: vendor/tcpdump/dist/extract.h ============================================================================== --- vendor/tcpdump/dist/extract.h Mon Sep 18 04:09:59 2017 (r323695) +++ vendor/tcpdump/dist/extract.h Mon Sep 18 04:10:54 2017 (r323696) @@ -20,8 +20,48 @@ */ /* - * Macros to extract possibly-unaligned big-endian integral values. + * For 8-bit values; provided for the sake of completeness. Byte order + * isn't relevant, and alignment isn't an issue. */ +#define EXTRACT_8BITS(p) (*(p)) +#define EXTRACT_LE_8BITS(p) (*(p)) + +/* + * Inline functions or macros to extract possibly-unaligned big-endian + * integral values. + */ +#include "funcattrs.h" + +/* + * If we have versions of GCC or Clang that support an __attribute__ + * to say "if we're building with unsigned behavior sanitization, + * don't complain about undefined behavior in this function", we + * label these functions with that attribute - we *know* it's undefined + * in the C standard, but we *also* know it does what we want with + * the ISA we're targeting and the compiler we're using. + * + * For GCC 4.9.0 and later, we use __attribute__((no_sanitize_undefined)); + * pre-5.0 GCC doesn't have __has_attribute, and I'm not sure whether + * GCC or Clang first had __attribute__((no_sanitize(XXX)). + * + * For Clang, we check for __attribute__((no_sanitize(XXX)) with + * __has_attribute, as there are versions of Clang that support + * __attribute__((no_sanitize("undefined")) but don't support + * __attribute__((no_sanitize_undefined)). + * + * We define this here, rather than in funcattrs.h, because we + * only want it used here, we don't want it to be broadly used. + * (Any printer will get this defined, but this should at least + * make it harder for people to find.) + */ +#if defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 409) +#define UNALIGNED_OK __attribute__((no_sanitize_undefined)) +#elif __has_attribute(no_sanitize) +#define UNALIGNED_OK __attribute__((no_sanitize("undefined"))) +#else +#define UNALIGNED_OK +#endif + #ifdef LBL_ALIGN /* * The processor doesn't natively handle unaligned loads. @@ -31,7 +71,7 @@ defined(__mips) || defined(__mips__)) /* - * This is a GCC-compatible compiler and we have __attribute__, which +* This is a GCC-compatible compiler and we have __attribute__, which * we assume that mean we have __attribute__((packed)), and this is * MIPS or Alpha, which has instructions that can help when doing * unaligned loads. @@ -88,19 +128,19 @@ typedef struct { uint32_t val; } __attribute__((packed)) unaligned_uint32_t; -static inline uint16_t +UNALIGNED_OK static inline uint16_t EXTRACT_16BITS(const void *p) { return ((uint16_t)ntohs(((const unaligned_uint16_t *)(p))->val)); } -static inline uint32_t +UNALIGNED_OK static inline uint32_t EXTRACT_32BITS(const void *p) { return ((uint32_t)ntohl(((const unaligned_uint32_t *)(p))->val)); } -static inline uint64_t +UNALIGNED_OK static inline uint64_t EXTRACT_64BITS(const void *p) { return ((uint64_t)(((uint64_t)ntohl(((const unaligned_uint32_t *)(p) + 0)->val)) << 32 | @@ -138,19 +178,19 @@ EXTRACT_64BITS(const void *p) * The processor natively handles unaligned loads, so we can just * cast the pointer and fetch through it. */ -static inline uint16_t +static inline uint16_t UNALIGNED_OK EXTRACT_16BITS(const void *p) { return ((uint16_t)ntohs(*(const uint16_t *)(p))); } -static inline uint32_t +static inline uint32_t UNALIGNED_OK EXTRACT_32BITS(const void *p) { return ((uint32_t)ntohl(*(const uint32_t *)(p))); } -static inline uint64_t +static inline uint64_t UNALIGNED_OK EXTRACT_64BITS(const void *p) { return ((uint64_t)(((uint64_t)ntohl(*((const uint32_t *)(p) + 0))) << 32 | @@ -193,7 +233,6 @@ EXTRACT_64BITS(const void *p) * Macros to extract possibly-unaligned little-endian integral values. * XXX - do loads on little-endian machines that support unaligned loads? */ -#define EXTRACT_LE_8BITS(p) (*(p)) #define EXTRACT_LE_16BITS(p) \ ((uint16_t)(((uint16_t)(*((const uint8_t *)(p) + 1)) << 8) | \ ((uint16_t)(*((const uint8_t *)(p) + 0)) << 0))) @@ -242,3 +281,6 @@ EXTRACT_64BITS(const void *p) #define ND_TTEST_64BITS(p) ND_TTEST2(*(p), 8) #define ND_TCHECK_64BITS(p) ND_TCHECK2(*(p), 8) + +#define ND_TTEST_128BITS(p) ND_TTEST2(*(p), 16) +#define ND_TCHECK_128BITS(p) ND_TCHECK2(*(p), 16) Added: vendor/tcpdump/dist/funcattrs.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/tcpdump/dist/funcattrs.h Mon Sep 18 04:10:54 2017 (r323696) @@ -0,0 +1,122 @@ +/* -*- Mode: c; tab-width: 8; indent-tabs-mode: 1; c-basic-offset: 8; -*- */ +/* + * Copyright (c) 1993, 1994, 1995, 1996, 1997 + * The Regents of the University of California. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Mon Sep 18 04:12:47 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BDB48E1D25D; Mon, 18 Sep 2017 04:12:47 +0000 (UTC) (envelope-from gordon@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 71D217E442; Mon, 18 Sep 2017 04:12:47 +0000 (UTC) (envelope-from gordon@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8I4CkrM031356; Mon, 18 Sep 2017 04:12:46 GMT (envelope-from gordon@FreeBSD.org) Received: (from gordon@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8I4Ck04031355; Mon, 18 Sep 2017 04:12:46 GMT (envelope-from gordon@FreeBSD.org) Message-Id: <201709180412.v8I4Ck04031355@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: gordon set sender to gordon@FreeBSD.org using -f From: Gordon Tetlow Date: Mon, 18 Sep 2017 04:12:46 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323697 - vendor/tcpdump/4.9.2 X-SVN-Group: vendor X-SVN-Commit-Author: gordon X-SVN-Commit-Paths: vendor/tcpdump/4.9.2 X-SVN-Commit-Revision: 323697 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Sep 2017 04:12:47 -0000 Author: gordon Date: Mon Sep 18 04:12:46 2017 New Revision: 323697 URL: https://svnweb.freebsd.org/changeset/base/323697 Log: Tag tcpdump-4.9.2. Approved by: emaste (mentor) Added: vendor/tcpdump/4.9.2/ - copied from r323696, vendor/tcpdump/dist/ From owner-svn-src-vendor@freebsd.org Wed Sep 20 06:34:49 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8B681E25E2D; Wed, 20 Sep 2017 06:34:49 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6789F82446; Wed, 20 Sep 2017 06:34:49 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8K6Ymgm087166; Wed, 20 Sep 2017 06:34:48 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8K6Ym39087165; Wed, 20 Sep 2017 06:34:48 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201709200634.v8K6Ym39087165@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Wed, 20 Sep 2017 06:34:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323789 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 323789 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Sep 2017 06:34:49 -0000 Author: avg Date: Wed Sep 20 06:34:48 2017 New Revision: 323789 URL: https://svnweb.freebsd.org/changeset/base/323789 Log: 8473 scrub does not detect errors on active spares illumos/illumos-gate@554675eee75dd2d7398d960aa5c81083ceb8505a https://github.com/illumos/illumos-gate/commit/554675eee75dd2d7398d960aa5c81083ceb8505a https://www.illumos.org/issues/8473 Scrubbing is supposed to detect and repair all errors in the pool. However, it wrongly ignores active spare devices. The problem can easily be reproduced in OpenZFS at git rev 0ef125d with these commands: truncate -s 64m /tmp/a /tmp/b /tmp/c sudo zpool create testpool mirror /tmp/a /tmp/b spare /tmp/c sudo zpool replace testpool /tmp/a /tmp/c /bin/dd if=/dev/zero bs=1024k count=63 oseek=1 conv=notrunc of=/tmp/c sync sudo zpool scrub testpool zpool status testpool # Will show 0 errors, which is wrong sudo zpool offline testpool /tmp/a sudo zpool scrub testpool zpool status testpool # Will show errors on /tmp/c, which should've already been fixed FreeBSD head is partially affected: the first scrub will detect some errors, but the second scrub will detect more. Reviewed by: Andy Stormont Reviewed by: Matt Ahrens Reviewed by: George Wilson Approved by: Richard Lowe Author: Alan Somers Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c Wed Sep 20 06:29:11 2017 (r323788) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c Wed Sep 20 06:34:48 2017 (r323789) @@ -29,6 +29,9 @@ #include #include +#include +#include +#include #include #include #include @@ -49,7 +52,7 @@ typedef struct mirror_child { typedef struct mirror_map { int mm_children; - int mm_replacing; + int mm_resilvering; int mm_preferred; int mm_root; mirror_child_t mm_child[1]; @@ -86,7 +89,7 @@ vdev_mirror_map_alloc(zio_t *zio) mm = kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP); mm->mm_children = c; - mm->mm_replacing = B_FALSE; + mm->mm_resilvering = B_FALSE; mm->mm_preferred = spa_get_random(c); mm->mm_root = B_TRUE; @@ -109,13 +112,51 @@ vdev_mirror_map_alloc(zio_t *zio) mc->mc_offset = DVA_GET_OFFSET(&dva[c]); } } else { + int replacing; + c = vd->vdev_children; mm = kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP); mm->mm_children = c; - mm->mm_replacing = (vd->vdev_ops == &vdev_replacing_ops || + /* + * If we are resilvering, then we should handle scrub reads + * differently; we shouldn't issue them to the resilvering + * device because it might not have those blocks. + * + * We are resilvering iff: + * 1) We are a replacing vdev (ie our name is "replacing-1" or + * "spare-1" or something like that), and + * 2) The pool is currently being resilvered. + * + * We cannot simply check vd->vdev_resilver_txg, because it's + * not set in this path. + * + * Nor can we just check our vdev_ops; there are cases (such as + * when a user types "zpool replace pool odev spare_dev" and + * spare_dev is in the spare list, or when a spare device is + * automatically used to replace a DEGRADED device) when + * resilvering is complete but both the original vdev and the + * spare vdev remain in the pool. That behavior is intentional. + * It helps implement the policy that a spare should be + * automatically removed from the pool after the user replaces + * the device that originally failed. + */ + replacing = (vd->vdev_ops == &vdev_replacing_ops || vd->vdev_ops == &vdev_spare_ops); - mm->mm_preferred = mm->mm_replacing ? 0 : + /* + * If a spa load is in progress, then spa_dsl_pool may be + * uninitialized. But we shouldn't be resilvering during a spa + * load anyway. + */ + if (replacing && + (spa_load_state(vd->vdev_spa) == SPA_LOAD_NONE) && + dsl_scan_resilvering(vd->vdev_spa->spa_dsl_pool)) { + mm->mm_resilvering = B_TRUE; + } else { + mm->mm_resilvering = B_FALSE; + } + + mm->mm_preferred = mm->mm_resilvering ? 0 : (zio->io_offset >> vdev_mirror_shift) % c; mm->mm_root = B_FALSE; @@ -271,7 +312,7 @@ vdev_mirror_io_start(zio_t *zio) mm = vdev_mirror_map_alloc(zio); if (zio->io_type == ZIO_TYPE_READ) { - if ((zio->io_flags & ZIO_FLAG_SCRUB) && !mm->mm_replacing) { + if ((zio->io_flags & ZIO_FLAG_SCRUB) && !mm->mm_resilvering) { /* * For scrubbing reads we need to allocate a read * buffer for each child and issue reads to all @@ -408,7 +449,7 @@ vdev_mirror_io_done(zio_t *zio) if (good_copies && spa_writeable(zio->io_spa) && (unexpected_errors || (zio->io_flags & ZIO_FLAG_RESILVER) || - ((zio->io_flags & ZIO_FLAG_SCRUB) && mm->mm_replacing))) { + ((zio->io_flags & ZIO_FLAG_SCRUB) && mm->mm_resilvering))) { /* * Use the good data we have in hand to repair damaged children. */ From owner-svn-src-vendor@freebsd.org Wed Sep 20 07:18:10 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8A2D2E273A2; Wed, 20 Sep 2017 07:18:10 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BE2483699; Wed, 20 Sep 2017 07:18:10 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8K7I9fK003653; Wed, 20 Sep 2017 07:18:09 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8K7I99n003651; Wed, 20 Sep 2017 07:18:09 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201709200718.v8K7I99n003651@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Wed, 20 Sep 2017 07:18:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323790 - in vendor/illumos/dist: cmd/zpool lib/libzfs/common X-SVN-Group: vendor X-SVN-Commit-Author: avg X-SVN-Commit-Paths: in vendor/illumos/dist: cmd/zpool lib/libzfs/common X-SVN-Commit-Revision: 323790 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Sep 2017 07:18:10 -0000 Author: avg Date: Wed Sep 20 07:18:09 2017 New Revision: 323790 URL: https://svnweb.freebsd.org/changeset/base/323790 Log: 8567 Inconsistent return value in zpool_read_label illumos/illumos-gate@c861bfbd77c4ae780a0341e9cb6926d8b74341cf https://github.com/illumos/illumos-gate/commit/c861bfbd77c4ae780a0341e9cb6926d8b74341cf https://www.illumos.org/issues/8567 If fstat64 fails, pread64 fails, or the label is unintelligible, zpool_read_label will return 0. But if malloc fails, it will return -1. For consistency, it should always return -1 on failure or 0 on success. Reviewed by: Prakash Surya Reviewed by: Matthew Ahrens Approved by: Robert Mustacchi Author: Alan Somers Modified: vendor/illumos/dist/cmd/zpool/zpool_main.c vendor/illumos/dist/lib/libzfs/common/libzfs_import.c Modified: vendor/illumos/dist/cmd/zpool/zpool_main.c ============================================================================== --- vendor/illumos/dist/cmd/zpool/zpool_main.c Wed Sep 20 06:34:48 2017 (r323789) +++ vendor/illumos/dist/cmd/zpool/zpool_main.c Wed Sep 20 07:18:09 2017 (r323790) @@ -708,7 +708,7 @@ zpool_do_labelclear(int argc, char **argv) return (1); } - if (zpool_read_label(fd, &config) != 0 || config == NULL) { + if (zpool_read_label(fd, &config) != 0) { (void) fprintf(stderr, gettext("failed to read label from %s\n"), vdev); return (1); Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_import.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_import.c Wed Sep 20 06:34:48 2017 (r323789) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_import.c Wed Sep 20 07:18:09 2017 (r323790) @@ -841,6 +841,7 @@ label_offset(uint64_t size, int l) /* * Given a file descriptor, read the label information and return an nvlist * describing the configuration, if there is one. + * Return 0 on success, or -1 on failure */ int zpool_read_label(int fd, nvlist_t **config) @@ -853,7 +854,7 @@ zpool_read_label(int fd, nvlist_t **config) *config = NULL; if (fstat64(fd, &statbuf) == -1) - return (0); + return (-1); size = P2ALIGN_TYPED(statbuf.st_size, sizeof (vdev_label_t), uint64_t); if ((label = malloc(sizeof (vdev_label_t))) == NULL) @@ -887,7 +888,7 @@ zpool_read_label(int fd, nvlist_t **config) free(label); *config = NULL; - return (0); + return (-1); } typedef struct rdsk_node { @@ -1052,7 +1053,7 @@ zpool_open_func(void *arg) check_slices(rn->rn_avl, fd, rn->rn_name); } - if ((zpool_read_label(fd, &config)) != 0) { + if ((zpool_read_label(fd, &config)) != 0 && errno == ENOMEM) { (void) close(fd); (void) no_memory(rn->rn_hdl); return; @@ -1517,7 +1518,7 @@ zpool_in_use(libzfs_handle_t *hdl, int fd, pool_state_ *inuse = B_FALSE; - if (zpool_read_label(fd, &config) != 0) { + if (zpool_read_label(fd, &config) != 0 && errno == ENOMEM) { (void) no_memory(hdl); return (-1); } From owner-svn-src-vendor@freebsd.org Wed Sep 20 07:24:58 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38502E27808; Wed, 20 Sep 2017 07:24:58 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0733883C5B; Wed, 20 Sep 2017 07:24:57 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8K7Ovti007557; Wed, 20 Sep 2017 07:24:57 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8K7OvuO007556; Wed, 20 Sep 2017 07:24:57 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201709200724.v8K7OvuO007556@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Wed, 20 Sep 2017 07:24:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323792 - vendor-sys/illumos/dist/uts/common/fs/zfs/sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs/sys X-SVN-Commit-Revision: 323792 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Sep 2017 07:24:58 -0000 Author: avg Date: Wed Sep 20 07:24:57 2017 New Revision: 323792 URL: https://svnweb.freebsd.org/changeset/base/323792 Log: 8602 remove unused "dp_early_sync_tasks" field from "dsl_pool" structure illumos/illumos-gate@2bcb5458541cc6e8bf7dc541303da29297b82e8b https://github.com/illumos/illumos-gate/commit/2bcb5458541cc6e8bf7dc541303da29297b82e8b https://www.illumos.org/issues/8602 When I landed the fix for 8558, I incorrectly added the "dp_early_sync_tasks" field to the "dsl_pool" structure. This field is used in DelphixOS, but not in illumos. It was incorrectly pulled into illumos, so this bug is to remove it from the structure. Reviewed by: Serapheim Dimitropoulos Reviewed by: Matthew Ahrens Approved by: Robert Mustacchi Author: Prakash Surya Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_pool.h Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_pool.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_pool.h Wed Sep 20 07:23:50 2017 (r323791) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_pool.h Wed Sep 20 07:24:57 2017 (r323792) @@ -123,7 +123,6 @@ typedef struct dsl_pool { txg_list_t dp_sync_tasks; taskq_t *dp_sync_taskq; taskq_t *dp_zil_clean_taskq; - txg_list_t dp_early_sync_tasks; /* * Protects administrative changes (properties, namespace) From owner-svn-src-vendor@freebsd.org Wed Sep 20 07:27:46 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9090E27A0A; Wed, 20 Sep 2017 07:27:46 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 74F1483F9D; Wed, 20 Sep 2017 07:27:46 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8K7RjZW007773; Wed, 20 Sep 2017 07:27:45 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8K7Rj0Z007772; Wed, 20 Sep 2017 07:27:45 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201709200727.v8K7Rj0Z007772@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Wed, 20 Sep 2017 07:27:45 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323794 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/man/man1m X-SVN-Group: vendor X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/man/man1m X-SVN-Commit-Revision: 323794 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Sep 2017 07:27:46 -0000 Author: avg Date: Wed Sep 20 07:27:45 2017 New Revision: 323794 URL: https://svnweb.freebsd.org/changeset/base/323794 Log: 8605 zfs channel programs: zfs.exists undocumented and non-working illumos/illumos-gate@5f39f884e2035d671ec02148fc4d8420c670bcb4 https://github.com/illumos/illumos-gate/commit/5f39f884e2035d671ec02148fc4d8420c670bcb4 https://www.illumos.org/issues/8605 zfs.exists() in channel programs doesn't return any result, and should have a man page entry. Reviewed by: Paul Dagnelie Reviewed by: Dan Kimmel Reviewed by: Matt Ahrens Approved by: Robert Mustacchi Author: Chris Williamson Modified: vendor/illumos/dist/man/man1m/zfs-program.1m Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c Modified: vendor/illumos/dist/man/man1m/zfs-program.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zfs-program.1m Wed Sep 20 07:26:52 2017 (r323793) +++ vendor/illumos/dist/man/man1m/zfs-program.1m Wed Sep 20 07:27:45 2017 (r323794) @@ -289,6 +289,18 @@ msg (string) .Bd -ragged -compact -offset "xxxx" Debug message to be printed. .Ed +.It Em zfs.exists(dataset) +Returns true if the given dataset exists, or false if it doesn't. +A fatal error will be thrown if the dataset is not in the target pool. +That is, in a channel program running on rpool, +zfs.exists("rpool/nonexistent_fs") returns false, but +zfs.exists("somepool/fs_that_may_exist") will error. +.Pp +dataset (string) +.Bd -ragged -compact -offset "xxxx" +Dataset to check for existence. +Must be in the target pool. +.Ed .It Em zfs.get_prop(dataset, property) Returns two values. First, a string, number or table containing the property value for the given From owner-svn-src-vendor@freebsd.org Wed Sep 20 07:27:46 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 610C1E27A07; Wed, 20 Sep 2017 07:27:46 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2D26283F9C; Wed, 20 Sep 2017 07:27:46 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8K7Rjn5007767; Wed, 20 Sep 2017 07:27:45 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8K7RjL1007766; Wed, 20 Sep 2017 07:27:45 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201709200727.v8K7RjL1007766@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Wed, 20 Sep 2017 07:27:45 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323794 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/man/man1m X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/man/man1m X-SVN-Commit-Revision: 323794 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Sep 2017 07:27:46 -0000 Author: avg Date: Wed Sep 20 07:27:45 2017 New Revision: 323794 URL: https://svnweb.freebsd.org/changeset/base/323794 Log: 8605 zfs channel programs: zfs.exists undocumented and non-working illumos/illumos-gate@5f39f884e2035d671ec02148fc4d8420c670bcb4 https://github.com/illumos/illumos-gate/commit/5f39f884e2035d671ec02148fc4d8420c670bcb4 https://www.illumos.org/issues/8605 zfs.exists() in channel programs doesn't return any result, and should have a man page entry. Reviewed by: Paul Dagnelie Reviewed by: Dan Kimmel Reviewed by: Matt Ahrens Approved by: Robert Mustacchi Author: Chris Williamson Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/man/man1m/zfs-program.1m Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c Wed Sep 20 07:26:52 2017 (r323793) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c Wed Sep 20 07:27:45 2017 (r323794) @@ -710,7 +710,7 @@ zcp_exists(lua_State *state) return (luaL_error(state, "unexpected error %d", error)); } - return (0); + return (1); } /* From owner-svn-src-vendor@freebsd.org Wed Sep 20 07:28:20 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9DC77E27AAB; Wed, 20 Sep 2017 07:28:20 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FDB0168; Wed, 20 Sep 2017 07:28:20 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8K7SJO9007839; Wed, 20 Sep 2017 07:28:19 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8K7SJQS007836; Wed, 20 Sep 2017 07:28:19 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201709200728.v8K7SJQS007836@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Wed, 20 Sep 2017 07:28:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323795 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 323795 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Sep 2017 07:28:20 -0000 Author: avg Date: Wed Sep 20 07:28:18 2017 New Revision: 323795 URL: https://svnweb.freebsd.org/changeset/base/323795 Log: 8604 Avoid unnecessary work search in VFS when unmounting snapshots illumos/illumos-gate@ed992b0aac4e5b70dc1273b1d055c0d471fbb4b1 https://github.com/illumos/illumos-gate/commit/ed992b0aac4e5b70dc1273b1d055c0d471fbb4b1 https://www.illumos.org/issues/8604 Every time we want to unmount a snapshot (happens during snapshot deletion or renaming) we unnecessarily iterate through all the mountpoints in the VFS layer (see zfs_get_vfs). Ideally we would just put a hold on the snapshot and access its respective VFS resource directly. gwilson_snap_unmount.svg - Flamegraph indicating the issue discussed (138 KB) Serapheim Dimitropoulos, 2017-09-14 06:36 PM Reviewed by: Matt Ahrens Reviewed by: George Wilson Reviewed by: Andy Stormont Approved by: Robert Mustacchi Author: Serapheim Dimitropoulos Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_ioctl.h vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c Wed Sep 20 07:27:45 2017 (r323794) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c Wed Sep 20 07:28:18 2017 (r323795) @@ -488,23 +488,29 @@ dsl_destroy_snapshots_nvl(nvlist_t *snaps, boolean_t d if (nvlist_next_nvpair(snaps, NULL) == NULL) return (0); - nvlist_t *arg = fnvlist_alloc(); - nvlist_t *snaps_normalized = fnvlist_alloc(); /* * lzc_destroy_snaps() is documented to take an nvlist whose - * values "don't matter". We need to convert that nvlist to one - * that we know can be converted to LUA. + * values "don't matter". We need to convert that nvlist to + * one that we know can be converted to LUA. We also don't + * care about any duplicate entries because the nvlist will + * be converted to a LUA table which should take care of this. */ + nvlist_t *snaps_normalized; + VERIFY0(nvlist_alloc(&snaps_normalized, 0, KM_SLEEP)); for (nvpair_t *pair = nvlist_next_nvpair(snaps, NULL); pair != NULL; pair = nvlist_next_nvpair(snaps, pair)) { fnvlist_add_boolean_value(snaps_normalized, nvpair_name(pair), B_TRUE); } + + nvlist_t *arg; + VERIFY0(nvlist_alloc(&arg, 0, KM_SLEEP)); fnvlist_add_nvlist(arg, "snaps", snaps_normalized); fnvlist_free(snaps_normalized); fnvlist_add_boolean_value(arg, "defer", defer); - nvlist_t *wrapper = fnvlist_alloc(); + nvlist_t *wrapper; + VERIFY0(nvlist_alloc(&wrapper, 0, KM_SLEEP)); fnvlist_add_nvlist(wrapper, ZCP_ARG_ARGLIST, arg); fnvlist_free(arg); @@ -538,7 +544,7 @@ dsl_destroy_snapshots_nvl(nvlist_t *snaps, boolean_t d program, 0, zfs_lua_max_memlimit, - fnvlist_lookup_nvpair(wrapper, ZCP_ARG_ARGLIST), result); + nvlist_next_nvpair(wrapper, NULL), result); if (error != 0) { char *errorstr = NULL; (void) nvlist_lookup_string(result, ZCP_RET_ERROR, &errorstr); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_ioctl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_ioctl.h Wed Sep 20 07:27:45 2017 (r323794) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_ioctl.h Wed Sep 20 07:28:18 2017 (r323795) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2011-2012 Pawel Jakub Dawidek. All rights reserved. - * Copyright (c) 2012, 2016 by Delphix. All rights reserved. + * Copyright (c) 2012, 2017 by Delphix. All rights reserved. * Copyright 2016 RackTop Systems. * Copyright (c) 2014 Integros [integros.com] */ @@ -428,9 +428,10 @@ extern int zfs_secpolicy_snapshot_perms(const char *, extern int zfs_secpolicy_rename_perms(const char *, const char *, cred_t *); extern int zfs_secpolicy_destroy_perms(const char *, cred_t *); extern int zfs_busy(void); -extern int zfs_unmount_snap(const char *); +extern void zfs_unmount_snap(const char *); extern void zfs_destroy_unmount_origin(const char *); extern int getzfsvfs_impl(struct objset *, struct zfsvfs **); +extern int getzfsvfs(const char *, struct zfsvfs **); /* * ZFS minor numbers can refer to either a control device instance or Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Wed Sep 20 07:27:45 2017 (r323794) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Wed Sep 20 07:28:18 2017 (r323795) @@ -1417,7 +1417,7 @@ getzfsvfs_impl(objset_t *os, zfsvfs_t **zfvp) return (error); } -static int +int getzfsvfs(const char *dsname, zfsvfs_t **zfvp) { objset_t *os; @@ -2988,31 +2988,6 @@ zfs_ioc_get_fsacl(zfs_cmd_t *zc) return (error); } -/* - * Search the vfs list for a specified resource. Returns a pointer to it - * or NULL if no suitable entry is found. The caller of this routine - * is responsible for releasing the returned vfs pointer. - */ -static vfs_t * -zfs_get_vfs(const char *resource) -{ - struct vfs *vfsp; - struct vfs *vfs_found = NULL; - - vfs_list_read_lock(); - vfsp = rootvfs; - do { - if (strcmp(refstr_value(vfsp->vfs_resource), resource) == 0) { - VFS_HOLD(vfsp); - vfs_found = vfsp; - break; - } - vfsp = vfsp->vfs_next; - } while (vfsp != rootvfs); - vfs_list_unlock(); - return (vfs_found); -} - /* ARGSUSED */ static void zfs_create_cb(objset_t *os, void *arg, cred_t *cr, dmu_tx_t *tx) @@ -3439,40 +3414,41 @@ zfs_ioc_log_history(const char *unused, nvlist_t *innv * Returns 0 if the argument is not a snapshot, or it is not currently a * filesystem, or we were able to unmount it. Returns error code otherwise. */ -int +void zfs_unmount_snap(const char *snapname) { - vfs_t *vfsp; - zfsvfs_t *zfsvfs; - int err; + vfs_t *vfsp = NULL; + zfsvfs_t *zfsvfs = NULL; if (strchr(snapname, '@') == NULL) - return (0); + return; - vfsp = zfs_get_vfs(snapname); - if (vfsp == NULL) - return (0); + int err = getzfsvfs(snapname, &zfsvfs); + if (err != 0) { + ASSERT3P(zfsvfs, ==, NULL); + return; + } + vfsp = zfsvfs->z_vfs; - zfsvfs = vfsp->vfs_data; ASSERT(!dsl_pool_config_held(dmu_objset_pool(zfsvfs->z_os))); err = vn_vfswlock(vfsp->vfs_vnodecovered); VFS_RELE(vfsp); if (err != 0) - return (SET_ERROR(err)); + return; /* * Always force the unmount for snapshots. */ (void) dounmount(vfsp, MS_FORCE, kcred); - return (0); } /* ARGSUSED */ static int zfs_unmount_snap_cb(const char *snapname, void *arg) { - return (zfs_unmount_snap(snapname)); + zfs_unmount_snap(snapname); + return (0); } /* @@ -3495,7 +3471,7 @@ zfs_destroy_unmount_origin(const char *fsname) char originname[ZFS_MAX_DATASET_NAME_LEN]; dsl_dataset_name(ds->ds_prev, originname); dmu_objset_rele(os, FTAG); - (void) zfs_unmount_snap(originname); + zfs_unmount_snap(originname); } else { dmu_objset_rele(os, FTAG); } @@ -3524,7 +3500,7 @@ zfs_ioc_destroy_snaps(const char *poolname, nvlist_t * for (pair = nvlist_next_nvpair(snaps, NULL); pair != NULL; pair = nvlist_next_nvpair(snaps, pair)) { - (void) zfs_unmount_snap(nvpair_name(pair)); + zfs_unmount_snap(nvpair_name(pair)); } return (dsl_destroy_snapshots_nvl(snaps, defer, outnvl)); @@ -3667,11 +3643,8 @@ zfs_ioc_destroy(zfs_cmd_t *zc) { int err; - if (zc->zc_objset_type == DMU_OST_ZFS) { - err = zfs_unmount_snap(zc->zc_name); - if (err != 0) - return (err); - } + if (zc->zc_objset_type == DMU_OST_ZFS) + zfs_unmount_snap(zc->zc_name); if (strchr(zc->zc_name, '@')) err = dsl_destroy_snapshot(zc->zc_name, zc->zc_defer_destroy); @@ -3735,7 +3708,9 @@ recursive_unmount(const char *fsname, void *arg) char fullname[ZFS_MAX_DATASET_NAME_LEN]; (void) snprintf(fullname, sizeof (fullname), "%s@%s", fsname, snapname); - return (zfs_unmount_snap(fullname)); + zfs_unmount_snap(fullname); + + return (0); } /* From owner-svn-src-vendor@freebsd.org Fri Sep 22 08:13:11 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 288A2E1DC29; Fri, 22 Sep 2017 08:13:11 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E9562838CD; Fri, 22 Sep 2017 08:13:10 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8M8DA8H021680; Fri, 22 Sep 2017 08:13:10 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8M8DAOT021679; Fri, 22 Sep 2017 08:13:10 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201709220813.v8M8DAOT021679@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Fri, 22 Sep 2017 08:13:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323911 - vendor/illumos/dist/lib/libzfs/common X-SVN-Group: vendor X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor/illumos/dist/lib/libzfs/common X-SVN-Commit-Revision: 323911 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Sep 2017 08:13:11 -0000 Author: avg Date: Fri Sep 22 08:13:09 2017 New Revision: 323911 URL: https://svnweb.freebsd.org/changeset/base/323911 Log: 8502 illumos#7955 broke delegated datasets when libshare is not present illumos/illumos-gate@1c18e8fbd8db41a9fb39bd3ef7a18ee71ece20da https://github.com/illumos/illumos-gate/commit/1c18e8fbd8db41a9fb39bd3ef7a18ee71ece20da https://www.illumos.org/issues/8502 The code in lib/libzfs/common/libzfs_mount.c already basically handles the case when libshare is not installed. We just need to not fail in zfs_init_libshare_impl. I tested this in lx and things work as expected. I also tested there trying to set sharenfs and sharesmb on the delegated dataset. Neither is allowed from within a zone. The spew of msgs from a native zone is not ZFS specific. I see the same spew simply running the share command. Reviewed by: Robert Mustacchi Reviewed by: Yuri Pankov Approved by: Richard Lowe Author: Jerry Jelinek Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c Fri Sep 22 08:12:08 2017 (r323910) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c Fri Sep 22 08:13:09 2017 (r323911) @@ -24,6 +24,7 @@ * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2014, 2016 by Delphix. All rights reserved. * Copyright 2016 Igor Kozhukhov + * Copyright 2017 Joyent, Inc. * Copyright 2017 RackTop Systems. */ @@ -664,8 +665,14 @@ _zfs_init_libshare(void) static int zfs_init_libshare_impl(libzfs_handle_t *zhandle, int service, void *arg) { + /* + * libshare is either not installed or we're in a branded zone. The + * rest of the wrapper functions around the libshare calls already + * handle NULL function pointers, but we don't want the callers of + * zfs_init_libshare() to fail prematurely if libshare is not available. + */ if (_sa_init == NULL) - return (SA_CONFIG_ERR); + return (SA_OK); /* * Attempt to refresh libshare. This is necessary if there was a cache From owner-svn-src-vendor@freebsd.org Fri Sep 22 08:15:36 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED9BCE1DD5B; Fri, 22 Sep 2017 08:15:36 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BAF0E83A75; Fri, 22 Sep 2017 08:15:36 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8M8FZ6m021975; Fri, 22 Sep 2017 08:15:35 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8M8FZb7021974; Fri, 22 Sep 2017 08:15:35 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201709220815.v8M8FZb7021974@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Fri, 22 Sep 2017 08:15:35 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323912 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/man/man1m X-SVN-Group: vendor X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/man/man1m X-SVN-Commit-Revision: 323912 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Sep 2017 08:15:37 -0000 Author: avg Date: Fri Sep 22 08:15:35 2017 New Revision: 323912 URL: https://svnweb.freebsd.org/changeset/base/323912 Log: 8592 ZFS channel programs - rollback illumos/illumos-gate@000cce6b6fad4a8b0eecef6e1251f6aca1719c55 https://github.com/illumos/illumos-gate/commit/000cce6b6fad4a8b0eecef6e1251f6aca1719c55 https://www.illumos.org/issues/8592 ZFS channel programs should be able to perform a rollback. This logic will probably look pretty similar to zfs.sync.destroy(). Reviewed by: Chris Williamson Reviewed by: Matthew Ahrens Approved by: Robert Mustacchi Author: Brad Lewis Modified: vendor/illumos/dist/man/man1m/zfs-program.1m Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c Modified: vendor/illumos/dist/man/man1m/zfs-program.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zfs-program.1m Fri Sep 22 08:13:09 2017 (r323911) +++ vendor/illumos/dist/man/man1m/zfs-program.1m Fri Sep 22 08:15:35 2017 (r323912) @@ -8,7 +8,7 @@ .\" http://www.illumos.org/license/CDDL. .\" .\" -.\" Copyright (c) 2016 by Delphix. All Rights Reserved. +.\" Copyright (c) 2016, 2017 by Delphix. All rights reserved. .\" .Dd January 21, 2016 .Dt ZFS-PROGRAM 1M @@ -361,6 +361,17 @@ dataset (string) .Bd -ragged -compact -offset "xxxx" Clone to be promoted. .Ed +.It Em zfs.sync.rollback(filesystem) +Rollback to the previous snapshot for a dataset. +Returns 0 on successful rollback, or a nonzero error code otherwise. +Rollbacks can be performed on filesystems or zvols, but not on snapshots +or mounted datasets. +EBUSY is returned in the case where the filesystem is mounted. +.Pp +filesystem (string) +.Bd -ragged -compact -offset "xxxx" +Filesystem to rollback. +.Ed .El .It Sy zfs.check submodule For each function in the zfs.sync submodule, there is a corresponding zfs.check @@ -380,6 +391,7 @@ The available zfs.check functions are: .Bl -tag -width "xx" .It Em zfs.check.destroy(dataset, [defer=true|false]) .It Em zfs.check.promote(dataset) +.It Em zfs.check.rollback(filesystem) .El .It Sy zfs.list submodule The zfs.list submodule provides functions for iterating over datasets and From owner-svn-src-vendor@freebsd.org Fri Sep 22 08:15:36 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D0B2EE1DD5A; Fri, 22 Sep 2017 08:15:36 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A7DA583A74; Fri, 22 Sep 2017 08:15:36 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8M8FZJi021968; Fri, 22 Sep 2017 08:15:35 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8M8FZ3n021965; Fri, 22 Sep 2017 08:15:35 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201709220815.v8M8FZ3n021965@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Fri, 22 Sep 2017 08:15:35 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323912 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/man/man1m X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/man/man1m X-SVN-Commit-Revision: 323912 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Sep 2017 08:15:36 -0000 Author: avg Date: Fri Sep 22 08:15:35 2017 New Revision: 323912 URL: https://svnweb.freebsd.org/changeset/base/323912 Log: 8592 ZFS channel programs - rollback illumos/illumos-gate@000cce6b6fad4a8b0eecef6e1251f6aca1719c55 https://github.com/illumos/illumos-gate/commit/000cce6b6fad4a8b0eecef6e1251f6aca1719c55 https://www.illumos.org/issues/8592 ZFS channel programs should be able to perform a rollback. This logic will probably look pretty similar to zfs.sync.destroy(). Reviewed by: Chris Williamson Reviewed by: Matthew Ahrens Approved by: Robert Mustacchi Author: Brad Lewis Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/man/man1m/zfs-program.1m Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c Fri Sep 22 08:13:09 2017 (r323911) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c Fri Sep 22 08:15:35 2017 (r323912) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2011, 2016 by Delphix. All rights reserved. + * Copyright (c) 2011, 2017 by Delphix. All rights reserved. * Copyright (c) 2014, Joyent, Inc. All rights reserved. * Copyright (c) 2014 RackTop Systems. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. @@ -2452,14 +2452,7 @@ dsl_dataset_handoff_check(dsl_dataset_t *ds, void *own return (0); } -typedef struct dsl_dataset_rollback_arg { - const char *ddra_fsname; - const char *ddra_tosnap; - void *ddra_owner; - nvlist_t *ddra_result; -} dsl_dataset_rollback_arg_t; - -static int +int dsl_dataset_rollback_check(void *arg, dmu_tx_t *tx) { dsl_dataset_rollback_arg_t *ddra = arg; @@ -2567,7 +2560,7 @@ dsl_dataset_rollback_check(void *arg, dmu_tx_t *tx) return (0); } -static void +void dsl_dataset_rollback_sync(void *arg, dmu_tx_t *tx) { dsl_dataset_rollback_arg_t *ddra = arg; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h Fri Sep 22 08:13:09 2017 (r323911) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h Fri Sep 22 08:15:35 2017 (r323912) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2011, 2015 by Delphix. All rights reserved. + * Copyright (c) 2011, 2017 by Delphix. All rights reserved. * Copyright (c) 2013, Joyent, Inc. All rights reserved. * Copyright (c) 2013 Steven Hartland. All rights reserved. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. @@ -237,6 +237,13 @@ typedef struct dsl_dataset_promote_arg { cred_t *cr; } dsl_dataset_promote_arg_t; +typedef struct dsl_dataset_rollback_arg { + const char *ddra_fsname; + const char *ddra_tosnap; + void *ddra_owner; + nvlist_t *ddra_result; +} dsl_dataset_rollback_arg_t; + /* * The max length of a temporary tag prefix is the number of hex digits * required to express UINT64_MAX plus one for the hyphen. @@ -375,6 +382,9 @@ void dsl_dataset_set_refreservation_sync_impl(dsl_data void dsl_dataset_zapify(dsl_dataset_t *ds, dmu_tx_t *tx); boolean_t dsl_dataset_is_zapified(dsl_dataset_t *ds); boolean_t dsl_dataset_has_resume_receive_state(dsl_dataset_t *ds); + +int dsl_dataset_rollback_check(void *arg, dmu_tx_t *tx); +void dsl_dataset_rollback_sync(void *arg, dmu_tx_t *tx); int dsl_dataset_rollback(const char *fsname, const char *tosnap, void *owner, nvlist_t *result); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c Fri Sep 22 08:13:09 2017 (r323911) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c Fri Sep 22 08:15:35 2017 (r323912) @@ -14,7 +14,7 @@ */ /* - * Copyright (c) 2016 by Delphix. All rights reserved. + * Copyright (c) 2016, 2017 by Delphix. All rights reserved. */ #include "lua.h" @@ -177,6 +177,37 @@ zcp_synctask_promote(lua_State *state, boolean_t sync, return (err); } +static int zcp_synctask_rollback(lua_State *, boolean_t, nvlist_t *err_details); +static zcp_synctask_info_t zcp_synctask_rollback_info = { + .name = "rollback", + .func = zcp_synctask_rollback, + .space_check = ZFS_SPACE_CHECK_RESERVED, + .blocks_modified = 1, + .pargs = { + {.za_name = "filesystem", .za_lua_type = LUA_TSTRING}, + {NULL, NULL} + }, + .kwargs = { + {NULL, NULL} + } +}; + +static int +zcp_synctask_rollback(lua_State *state, boolean_t sync, nvlist_t *err_details) +{ + int err; + const char *dsname = lua_tostring(state, 1); + dsl_dataset_rollback_arg_t ddra = { 0 }; + + ddra.ddra_fsname = dsname; + ddra.ddra_result = err_details; + + err = zcp_sync_task(state, dsl_dataset_rollback_check, + dsl_dataset_rollback_sync, &ddra, sync, dsname); + + return (err); +} + void zcp_synctask_wrapper_cleanup(void *arg) { @@ -247,6 +278,7 @@ zcp_load_synctask_lib(lua_State *state, boolean_t sync zcp_synctask_info_t *zcp_synctask_funcs[] = { &zcp_synctask_destroy_info, &zcp_synctask_promote_info, + &zcp_synctask_rollback_info, NULL }; From owner-svn-src-vendor@freebsd.org Fri Sep 22 08:18:07 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 66D5FE1DED9; Fri, 22 Sep 2017 08:18:07 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2631383E85; Fri, 22 Sep 2017 08:18:07 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8M8I6bP022121; Fri, 22 Sep 2017 08:18:06 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8M8I5OK022114; Fri, 22 Sep 2017 08:18:05 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201709220818.v8M8I5OK022114@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Fri, 22 Sep 2017 08:18:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323913 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/lib/libzfs_core/common vendor/illumos/dist/man/man1m X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/lib/libzfs_core/common vendor/illumos/dist/man/man1m X-SVN-Commit-Revision: 323913 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Sep 2017 08:18:07 -0000 Author: avg Date: Fri Sep 22 08:18:05 2017 New Revision: 323913 URL: https://svnweb.freebsd.org/changeset/base/323913 Log: 8600 ZFS channel programs - snapshot illumos/illumos-gate@2840dce1a029098fb784afd951d5f98089f850d8 https://github.com/illumos/illumos-gate/commit/2840dce1a029098fb784afd951d5f98089f850d8 https://www.illumos.org/issues/8600 ZFS channel programs should be able to create snapshots. In addition to the base snapshot functionality, this will likely entail adding extra logic to handle edge cases which were formerly not possible, such as creating then destroying a snapshot in the same transaction sync. Reviewed by: Matthew Ahrens Reviewed by: John Kennedy Reviewed by: Brad Lewis Approved by: Robert Mustacchi Author: Chris Williamson Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zcp.h vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_global.c vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c vendor/illumos/dist/man/man1m/zfs-program.1m Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c Fri Sep 22 08:15:35 2017 (r323912) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c Fri Sep 22 08:18:05 2017 (r323913) @@ -1081,13 +1081,6 @@ dsl_dataset_snapshot_reserve_space(dsl_dataset_t *ds, return (0); } -typedef struct dsl_dataset_snapshot_arg { - nvlist_t *ddsa_snaps; - nvlist_t *ddsa_props; - nvlist_t *ddsa_errors; - cred_t *ddsa_cr; -} dsl_dataset_snapshot_arg_t; - int dsl_dataset_snapshot_check_impl(dsl_dataset_t *ds, const char *snapname, dmu_tx_t *tx, boolean_t recv, uint64_t cnt, cred_t *cr) @@ -1147,7 +1140,7 @@ dsl_dataset_snapshot_check_impl(dsl_dataset_t *ds, con return (0); } -static int +int dsl_dataset_snapshot_check(void *arg, dmu_tx_t *tx) { dsl_dataset_snapshot_arg_t *ddsa = arg; @@ -1422,7 +1415,7 @@ dsl_dataset_snapshot_sync_impl(dsl_dataset_t *ds, cons spa_history_log_internal_ds(ds->ds_prev, "snapshot", tx, ""); } -static void +void dsl_dataset_snapshot_sync(void *arg, dmu_tx_t *tx) { dsl_dataset_snapshot_arg_t *ddsa = arg; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h Fri Sep 22 08:15:35 2017 (r323912) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h Fri Sep 22 08:18:05 2017 (r323913) @@ -244,6 +244,13 @@ typedef struct dsl_dataset_rollback_arg { nvlist_t *ddra_result; } dsl_dataset_rollback_arg_t; +typedef struct dsl_dataset_snapshot_arg { + nvlist_t *ddsa_snaps; + nvlist_t *ddsa_props; + nvlist_t *ddsa_errors; + cred_t *ddsa_cr; +} dsl_dataset_snapshot_arg_t; + /* * The max length of a temporary tag prefix is the number of hex digits * required to express UINT64_MAX plus one for the hyphen. @@ -276,6 +283,8 @@ uint64_t dsl_dataset_create_sync(dsl_dir_t *pds, const dsl_dataset_t *origin, uint64_t flags, cred_t *, dmu_tx_t *); uint64_t dsl_dataset_create_sync_dd(dsl_dir_t *dd, dsl_dataset_t *origin, uint64_t flags, dmu_tx_t *tx); +void dsl_dataset_snapshot_sync(void *arg, dmu_tx_t *tx); +int dsl_dataset_snapshot_check(void *arg, dmu_tx_t *tx); int dsl_dataset_snapshot(nvlist_t *snaps, nvlist_t *props, nvlist_t *errors); void dsl_dataset_promote_sync(void *arg, dmu_tx_t *tx); int dsl_dataset_promote_check(void *arg, dmu_tx_t *tx); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zcp.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zcp.h Fri Sep 22 08:15:35 2017 (r323912) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zcp.h Fri Sep 22 08:18:05 2017 (r323913) @@ -14,7 +14,7 @@ */ /* - * Copyright (c) 2016 by Delphix. All rights reserved. + * Copyright (c) 2016, 2017 by Delphix. All rights reserved. */ #ifndef _SYS_ZCP_H @@ -136,8 +136,6 @@ typedef struct zcp_lib_info { const zcp_arg_t pargs[4]; const zcp_arg_t kwargs[2]; } zcp_lib_info_t; - -int zcp_nvlist_to_lua(lua_State *, nvlist_t *, char *, int); #ifdef __cplusplus } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c Fri Sep 22 08:15:35 2017 (r323912) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c Fri Sep 22 08:18:05 2017 (r323913) @@ -14,7 +14,7 @@ */ /* - * Copyright (c) 2016 by Delphix. All rights reserved. + * Copyright (c) 2016, 2017 by Delphix. All rights reserved. */ /* @@ -102,10 +102,15 @@ #include #include +#define ZCP_NVLIST_MAX_DEPTH 20 + uint64_t zfs_lua_check_instrlimit_interval = 100; uint64_t zfs_lua_max_instrlimit = ZCP_MAX_INSTRLIMIT; uint64_t zfs_lua_max_memlimit = ZCP_MAX_MEMLIMIT; +/* + * Forward declarations for mutually recursive functions + */ static int zcp_nvpair_value_to_lua(lua_State *, nvpair_t *, char *, int); static int zcp_lua_to_nvlist_impl(lua_State *, int, nvlist_t *, const char *, int); @@ -212,8 +217,6 @@ zcp_cleanup(lua_State *state) zcp_clear_cleanup(state); } } - -#define ZCP_NVLIST_MAX_DEPTH 20 /* * Convert the lua table at the given index on the Lua stack to an nvlist Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_global.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_global.c Fri Sep 22 08:15:35 2017 (r323912) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_global.c Fri Sep 22 08:18:05 2017 (r323913) @@ -14,7 +14,7 @@ */ /* - * Copyright (c) 2016 by Delphix. All rights reserved. + * Copyright (c) 2016, 2017 by Delphix. All rights reserved. */ #include @@ -62,7 +62,12 @@ static const zcp_errno_global_t errno_globals[] = { {"EPIPE", EPIPE}, {"EDOM", EDOM}, {"ERANGE", ERANGE}, + {"EDEADLK", EDEADLK}, + {"ENOLCK", ENOLCK}, + {"ECANCELED", ECANCELED}, + {"ENOTSUP", ENOTSUP}, {"EDQUOT", EDQUOT}, + {"ENAMETOOLONG", ENAMETOOLONG}, {NULL, NULL} }; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c Fri Sep 22 08:15:35 2017 (r323912) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c Fri Sep 22 08:18:05 2017 (r323913) @@ -39,10 +39,10 @@ typedef int (zcp_synctask_func_t)(lua_State *, boolean typedef struct zcp_synctask_info { const char *name; zcp_synctask_func_t *func; - zfs_space_check_t space_check; - int blocks_modified; const zcp_arg_t pargs[4]; const zcp_arg_t kwargs[2]; + zfs_space_check_t space_check; + int blocks_modified; } zcp_synctask_info_t; /* @@ -91,8 +91,6 @@ static int zcp_synctask_destroy(lua_State *, boolean_t static zcp_synctask_info_t zcp_synctask_destroy_info = { .name = "destroy", .func = zcp_synctask_destroy, - .space_check = ZFS_SPACE_CHECK_NONE, - .blocks_modified = 0, .pargs = { {.za_name = "filesystem | snapshot", .za_lua_type = LUA_TSTRING}, {NULL, NULL} @@ -100,7 +98,9 @@ static zcp_synctask_info_t zcp_synctask_destroy_info = .kwargs = { {.za_name = "defer", .za_lua_type = LUA_TBOOLEAN}, {NULL, NULL} - } + }, + .space_check = ZFS_SPACE_CHECK_NONE, + .blocks_modified = 0 }; /* ARGSUSED */ @@ -140,19 +140,19 @@ zcp_synctask_destroy(lua_State *state, boolean_t sync, return (err); } -static int zcp_synctask_promote(lua_State *, boolean_t, nvlist_t *err_details); +static int zcp_synctask_promote(lua_State *, boolean_t, nvlist_t *); static zcp_synctask_info_t zcp_synctask_promote_info = { .name = "promote", .func = zcp_synctask_promote, - .space_check = ZFS_SPACE_CHECK_RESERVED, - .blocks_modified = 3, .pargs = { {.za_name = "clone", .za_lua_type = LUA_TSTRING}, {NULL, NULL} }, .kwargs = { {NULL, NULL} - } + }, + .space_check = ZFS_SPACE_CHECK_RESERVED, + .blocks_modified = 3 }; static int @@ -208,6 +208,58 @@ zcp_synctask_rollback(lua_State *state, boolean_t sync return (err); } +static int zcp_synctask_snapshot(lua_State *, boolean_t, nvlist_t *); +static zcp_synctask_info_t zcp_synctask_snapshot_info = { + .name = "snapshot", + .func = zcp_synctask_snapshot, + .pargs = { + {.za_name = "filesystem@snapname | volume@snapname", + .za_lua_type = LUA_TSTRING}, + {NULL, NULL} + }, + .kwargs = { + {NULL, NULL} + }, + .space_check = ZFS_SPACE_CHECK_NORMAL, + .blocks_modified = 3 +}; + +/* ARGSUSED */ +static int +zcp_synctask_snapshot(lua_State *state, boolean_t sync, nvlist_t *err_details) +{ + int err; + dsl_dataset_snapshot_arg_t ddsa = { 0 }; + const char *dsname = lua_tostring(state, 1); + zcp_run_info_t *ri = zcp_run_info(state); + + /* + * We only allow for a single snapshot rather than a list, so the + * error list output is unnecessary. + */ + ddsa.ddsa_errors = NULL; + ddsa.ddsa_props = NULL; + ddsa.ddsa_cr = ri->zri_cred; + ddsa.ddsa_snaps = fnvlist_alloc(); + fnvlist_add_boolean(ddsa.ddsa_snaps, dsname); + + /* + * On old pools, the ZIL must not be active when a snapshot is created, + * but we can't suspend the ZIL because we're already in syncing + * context. + */ + if (spa_version(ri->zri_pool->dp_spa) < SPA_VERSION_FAST_SNAP) { + return (ENOTSUP); + } + + err = zcp_sync_task(state, dsl_dataset_snapshot_check, + dsl_dataset_snapshot_sync, &ddsa, sync, dsname); + + fnvlist_free(ddsa.ddsa_snaps); + + return (err); +} + void zcp_synctask_wrapper_cleanup(void *arg) { @@ -279,6 +331,7 @@ zcp_load_synctask_lib(lua_State *state, boolean_t sync &zcp_synctask_destroy_info, &zcp_synctask_promote_info, &zcp_synctask_rollback_info, + &zcp_synctask_snapshot_info, NULL }; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Fri Sep 22 08:15:35 2017 (r323912) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Fri Sep 22 08:18:05 2017 (r323913) @@ -26,7 +26,7 @@ * Copyright 2015, OmniTI Computer Consulting, Inc. All rights reserved. * Copyright 2015 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2014, 2016 Joyent, Inc. All rights reserved. - * Copyright (c) 2011, 2016 by Delphix. All rights reserved. + * Copyright (c) 2011, 2017 by Delphix. All rights reserved. * Copyright (c) 2013 by Saso Kiselkov. All rights reserved. * Copyright (c) 2013 Steven Hartland. All rights reserved. * Copyright (c) 2014 Integros [integros.com] @@ -3623,7 +3623,7 @@ zfs_ioc_channel_program(const char *poolname, nvlist_t if (instrlimit == 0 || instrlimit > zfs_lua_max_instrlimit) return (EINVAL); - if (memlimit == 0 || memlimit > ZCP_MAX_MEMLIMIT) + if (memlimit == 0 || memlimit > zfs_lua_max_memlimit) return (EINVAL); return (zcp_eval(poolname, program, instrlimit, memlimit, From owner-svn-src-vendor@freebsd.org Fri Sep 22 08:18:07 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 974DBE1DEDD; Fri, 22 Sep 2017 08:18:07 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6E03F83E86; Fri, 22 Sep 2017 08:18:07 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8M8I6RL022128; Fri, 22 Sep 2017 08:18:06 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8M8I6lc022126; Fri, 22 Sep 2017 08:18:06 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201709220818.v8M8I6lc022126@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Fri, 22 Sep 2017 08:18:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323913 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/lib/libzfs_core/common vendor/illumos/dist/man/man1m X-SVN-Group: vendor X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/lib/libzfs_core/common vendor/illumos/dist/man/man1m X-SVN-Commit-Revision: 323913 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Sep 2017 08:18:07 -0000 Author: avg Date: Fri Sep 22 08:18:05 2017 New Revision: 323913 URL: https://svnweb.freebsd.org/changeset/base/323913 Log: 8600 ZFS channel programs - snapshot illumos/illumos-gate@2840dce1a029098fb784afd951d5f98089f850d8 https://github.com/illumos/illumos-gate/commit/2840dce1a029098fb784afd951d5f98089f850d8 https://www.illumos.org/issues/8600 ZFS channel programs should be able to create snapshots. In addition to the base snapshot functionality, this will likely entail adding extra logic to handle edge cases which were formerly not possible, such as creating then destroying a snapshot in the same transaction sync. Reviewed by: Matthew Ahrens Reviewed by: John Kennedy Reviewed by: Brad Lewis Approved by: Robert Mustacchi Author: Chris Williamson Modified: vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c vendor/illumos/dist/man/man1m/zfs-program.1m Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zcp.h vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_global.c vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Modified: vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c ============================================================================== --- vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c Fri Sep 22 08:15:35 2017 (r323912) +++ vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c Fri Sep 22 08:18:05 2017 (r323913) @@ -20,7 +20,7 @@ */ /* - * Copyright (c) 2012, 2016 by Delphix. All rights reserved. + * Copyright (c) 2012, 2017 by Delphix. All rights reserved. * Copyright (c) 2013 Steven Hartland. All rights reserved. * Copyright (c) 2014 Integros [integros.com] * Copyright 2017 RackTop Systems. @@ -142,7 +142,12 @@ lzc_ioctl(zfs_ioc_t ioc, const char *name, if (resultp != NULL) { *resultp = NULL; - zc.zc_nvlist_dst_size = MAX(size * 2, 128 * 1024); + if (ioc == ZFS_IOC_CHANNEL_PROGRAM) { + zc.zc_nvlist_dst_size = fnvlist_lookup_uint64(source, + ZCP_ARG_MEMLIMIT); + } else { + zc.zc_nvlist_dst_size = MAX(size * 2, 128 * 1024); + } zc.zc_nvlist_dst = (uint64_t)(uintptr_t) malloc(zc.zc_nvlist_dst_size); if (zc.zc_nvlist_dst == NULL) { @@ -156,7 +161,7 @@ lzc_ioctl(zfs_ioc_t ioc, const char *name, * If ioctl exited with ENOMEM, we retry the ioctl after * increasing the size of the destination nvlist. * - * Channel programs that exit with ENOMEM probably ran over the + * Channel programs that exit with ENOMEM ran over the * lua memory sandbox; they should not be retried. */ if (errno == ENOMEM && resultp != NULL && Modified: vendor/illumos/dist/man/man1m/zfs-program.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zfs-program.1m Fri Sep 22 08:15:35 2017 (r323912) +++ vendor/illumos/dist/man/man1m/zfs-program.1m Fri Sep 22 08:18:05 2017 (r323913) @@ -372,6 +372,18 @@ filesystem (string) .Bd -ragged -compact -offset "xxxx" Filesystem to rollback. .Ed +.It Em zfs.sync.snapshot(dataset) +Create a snapshot of a filesystem. +Returns 0 if the snapshot was successfully created, +and a nonzero error code otherwise. +.Pp +Note: Taking a snapshot will fail on any pool older than legacy version 27. +To enable taking snapshots from ZCP scripts, the pool must be upgraded. +.Pp +dataset (string) +.Bd -ragged -compact -offset "xxxx" +Name of snapshot to create. +.Ed .El .It Sy zfs.check submodule For each function in the zfs.sync submodule, there is a corresponding zfs.check @@ -392,6 +404,7 @@ The available zfs.check functions are: .It Em zfs.check.destroy(dataset, [defer=true|false]) .It Em zfs.check.promote(dataset) .It Em zfs.check.rollback(filesystem) +.It Em zfs.check.snapshot(dataset) .El .It Sy zfs.list submodule The zfs.list submodule provides functions for iterating over datasets and From owner-svn-src-vendor@freebsd.org Fri Sep 22 08:18:50 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60F02E1E024; Fri, 22 Sep 2017 08:18:50 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2F171189; Fri, 22 Sep 2017 08:18:50 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8M8InTH022200; Fri, 22 Sep 2017 08:18:49 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8M8InBl022199; Fri, 22 Sep 2017 08:18:49 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201709220818.v8M8InBl022199@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Fri, 22 Sep 2017 08:18:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323914 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 323914 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Sep 2017 08:18:50 -0000 Author: avg Date: Fri Sep 22 08:18:49 2017 New Revision: 323914 URL: https://svnweb.freebsd.org/changeset/base/323914 Log: 8661 remove "zil-cw2" dtrace probe illumos/illumos-gate@bd9d3f904625846bdc61af8897a1072029c7aeb7 https://github.com/illumos/illumos-gate/commit/bd9d3f904625846bdc61af8897a1072029c7aeb7 https://www.illumos.org/issues/8661 The "zil-cw1" dtrace probe was previously removed in 8558, and the "zil-cw2" probe should have been removed in that patch as well. Unfortunately, the "zil- cw2" was not removed in 8558, so this bug is to track it's removal. Reviewed by: Serapheim Dimitropoulos Reviewed by: Matthew Ahrens Reviewed by: Igor Kozhukhov Approved by: Robert Mustacchi Author: Prakash Surya Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c Fri Sep 22 08:18:05 2017 (r323913) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c Fri Sep 22 08:18:49 2017 (r323914) @@ -1995,7 +1995,6 @@ zil_process_commit_list(zilog_t *zilog) list_remove(&zilog->zl_itx_commit_list, itx); zil_itx_destroy(itx); } - DTRACE_PROBE1(zil__cw2, zilog_t *, zilog); if (lwb == NULL) { /* From owner-svn-src-vendor@freebsd.org Fri Sep 22 08:23:26 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B9CCE1E499; Fri, 22 Sep 2017 08:23:26 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D9EFFBFE; Fri, 22 Sep 2017 08:23:25 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8M8NP2B026427; Fri, 22 Sep 2017 08:23:25 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8M8NORB026425; Fri, 22 Sep 2017 08:23:24 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201709220823.v8M8NORB026425@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Fri, 22 Sep 2017 08:23:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323917 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 323917 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Sep 2017 08:23:26 -0000 Author: avg Date: Fri Sep 22 08:23:24 2017 New Revision: 323917 URL: https://svnweb.freebsd.org/changeset/base/323917 Log: 8648 Fix range locking in ZIL commit codepath illumos/illumos-gate@42b14111721da2ebd5159e7b45012a3eb0e3384c https://github.com/illumos/illumos-gate/commit/42b14111721da2ebd5159e7b45012a3eb0e3384c https://www.illumos.org/issues/8648 I'm opening this bug to track integration of the following ZFS on Linux commit into illumos: commit f763c3d1df569a8d6b60bcb5e95cf07aa7a189e6 Author: LOLi Date: Mon Aug 21 17:59:48 2017 +0200 Fix range locking in ZIL commit codepath Since OpenZFS 7578 (1b7c1e5) if we have a ZVOL with logbias=throughput we will force WR_INDIRECT itxs in zvol_log_write() setting itx->itx_lr offset and length to the offset and length of the BIO from zvol_write()->zvol_log_write(): these offset and length are later used to take a range lock in zillog->zl_get_data function: zvol_get_data(). Now suppose we have a ZVOL with blocksize=8K and push 4K writes to offset 0: we will only be range-locking 0-4096. This means the ASSERTion we make in dbuf_unoverride() is no longer valid because now dmu_sync() is called from zilog's get_data functions holding a partial lock on the dbuf. Fix this by taking a range lock on the whole block in zvol_get_data(). Reviewed-by: Chunwei Chen Reviewed-by: Brian Behlendorf Signed-off-by: loli10K Reviewed by: Igor Kozhukhov Reviewed by: Matt Ahrens Reviewed by: Andriy Gapon Reviewed by: Alexander Motin Approved by: Robert Mustacchi Author: LOLi Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c Fri Sep 22 08:21:35 2017 (r323916) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c Fri Sep 22 08:23:24 2017 (r323917) @@ -1101,7 +1101,7 @@ zfs_get_data(void *arg, lr_write_t *lr, char *buf, str } else { /* indirect write */ /* * Have to lock the whole block to ensure when it's - * written out and it's checksum is being calculated + * written out and its checksum is being calculated * that no one can change the data. We need to re-check * blocksize after we get the lock in case it's changed! */ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c Fri Sep 22 08:21:35 2017 (r323916) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c Fri Sep 22 08:23:24 2017 (r323917) @@ -1003,7 +1003,6 @@ zvol_get_data(void *arg, lr_write_t *lr, char *buf, st zgd = kmem_zalloc(sizeof (zgd_t), KM_SLEEP); zgd->zgd_lwb = lwb; - zgd->zgd_rl = zfs_range_lock(&zv->zv_znode, offset, size, RL_READER); /* * Write records come in two flavors: immediate and indirect. @@ -1012,12 +1011,22 @@ zvol_get_data(void *arg, lr_write_t *lr, char *buf, st * sync the data and get a pointer to it (indirect) so that * we don't have to write the data twice. */ - if (buf != NULL) { /* immediate write */ + if (buf != NULL) { /* immediate write */ + zgd->zgd_rl = zfs_range_lock(&zv->zv_znode, offset, size, + RL_READER); error = dmu_read(os, object, offset, size, buf, DMU_READ_NO_PREFETCH); - } else { + } else { /* indirect write */ + /* + * Have to lock the whole block to ensure when it's written out + * and its checksum is being calculated that no one can change + * the data. Contrarily to zfs_get_data we need not re-check + * blocksize after we get the lock because it cannot be changed. + */ size = zv->zv_volblocksize; offset = P2ALIGN(offset, size); + zgd->zgd_rl = zfs_range_lock(&zv->zv_znode, offset, size, + RL_READER); error = dmu_buf_hold(os, object, offset, zgd, &db, DMU_READ_NO_PREFETCH); if (error == 0) {