Date: Mon, 2 May 2016 05:46:48 +0000 (UTC) From: Xin LI <delphij@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r298920 - in stable/10: contrib/file contrib/file/doc contrib/file/magic contrib/file/magic/Magdir contrib/file/python contrib/file/src lib/libmagic Message-ID: <201605020546.u425kmHi061290@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: delphij Date: Mon May 2 05:46:47 2016 New Revision: 298920 URL: https://svnweb.freebsd.org/changeset/base/298920 Log: MFC r298192: MFV r298178: Update file to 5.26. Relnotes: yes Added: stable/10/contrib/file/magic/Magdir/bioinformatics - copied unchanged from r298192, head/contrib/file/magic/Magdir/bioinformatics stable/10/contrib/file/magic/Magdir/coff - copied unchanged from r298192, head/contrib/file/magic/Magdir/coff stable/10/contrib/file/magic/Magdir/der - copied unchanged from r298192, head/contrib/file/magic/Magdir/der stable/10/contrib/file/magic/Magdir/finger - copied unchanged from r298192, head/contrib/file/magic/Magdir/finger stable/10/contrib/file/magic/Magdir/flif - copied unchanged from r298192, head/contrib/file/magic/Magdir/flif stable/10/contrib/file/magic/Magdir/microfocus - copied unchanged from r298192, head/contrib/file/magic/Magdir/microfocus stable/10/contrib/file/magic/Magdir/polyml - copied unchanged from r298192, head/contrib/file/magic/Magdir/polyml stable/10/contrib/file/magic/Magdir/vacuum-cleaner - copied unchanged from r298192, head/contrib/file/magic/Magdir/vacuum-cleaner stable/10/contrib/file/src/der.c - copied unchanged from r298192, head/contrib/file/src/der.c stable/10/contrib/file/src/der.h - copied unchanged from r298192, head/contrib/file/src/der.h stable/10/contrib/file/src/dprintf.c - copied unchanged from r298192, head/contrib/file/src/dprintf.c Modified: stable/10/contrib/file/ChangeLog stable/10/contrib/file/README stable/10/contrib/file/config.h.in stable/10/contrib/file/configure stable/10/contrib/file/configure.ac stable/10/contrib/file/doc/file.man stable/10/contrib/file/doc/libmagic.man stable/10/contrib/file/magic/Magdir/android stable/10/contrib/file/magic/Magdir/animation stable/10/contrib/file/magic/Magdir/apple stable/10/contrib/file/magic/Magdir/archive stable/10/contrib/file/magic/Magdir/audio stable/10/contrib/file/magic/Magdir/c-lang stable/10/contrib/file/magic/Magdir/cafebabe stable/10/contrib/file/magic/Magdir/commands stable/10/contrib/file/magic/Magdir/compress stable/10/contrib/file/magic/Magdir/console stable/10/contrib/file/magic/Magdir/database stable/10/contrib/file/magic/Magdir/filesystems stable/10/contrib/file/magic/Magdir/fonts stable/10/contrib/file/magic/Magdir/fortran stable/10/contrib/file/magic/Magdir/hitachi-sh stable/10/contrib/file/magic/Magdir/images stable/10/contrib/file/magic/Magdir/intel stable/10/contrib/file/magic/Magdir/java stable/10/contrib/file/magic/Magdir/lisp stable/10/contrib/file/magic/Magdir/mach stable/10/contrib/file/magic/Magdir/macintosh stable/10/contrib/file/magic/Magdir/misctools stable/10/contrib/file/magic/Magdir/modem stable/10/contrib/file/magic/Magdir/msdos stable/10/contrib/file/magic/Magdir/msvc stable/10/contrib/file/magic/Magdir/msx stable/10/contrib/file/magic/Magdir/netbsd stable/10/contrib/file/magic/Magdir/psdbms stable/10/contrib/file/magic/Magdir/python stable/10/contrib/file/magic/Magdir/sendmail stable/10/contrib/file/magic/Magdir/sgml stable/10/contrib/file/magic/Magdir/sinclair stable/10/contrib/file/magic/Magdir/sql stable/10/contrib/file/magic/Magdir/terminfo stable/10/contrib/file/magic/Magdir/windows stable/10/contrib/file/magic/Magdir/wordprocessors stable/10/contrib/file/magic/Makefile.am stable/10/contrib/file/magic/Makefile.in stable/10/contrib/file/python/README stable/10/contrib/file/python/magic.py stable/10/contrib/file/python/setup.py stable/10/contrib/file/src/Makefile.am stable/10/contrib/file/src/Makefile.in stable/10/contrib/file/src/apprentice.c stable/10/contrib/file/src/ascmagic.c stable/10/contrib/file/src/compress.c stable/10/contrib/file/src/file.c stable/10/contrib/file/src/file.h stable/10/contrib/file/src/file_opts.h stable/10/contrib/file/src/fmtcheck.c stable/10/contrib/file/src/funcs.c stable/10/contrib/file/src/magic.c stable/10/contrib/file/src/magic.h stable/10/contrib/file/src/print.c stable/10/contrib/file/src/readcdf.c stable/10/contrib/file/src/readelf.c stable/10/contrib/file/src/readelf.h stable/10/contrib/file/src/softmagic.c stable/10/lib/libmagic/Makefile stable/10/lib/libmagic/config.h Directory Properties: stable/10/ (props changed) Modified: stable/10/contrib/file/ChangeLog ============================================================================== --- stable/10/contrib/file/ChangeLog Mon May 2 05:43:46 2016 (r298919) +++ stable/10/contrib/file/ChangeLog Mon May 2 05:46:47 2016 (r298920) @@ -1,3 +1,49 @@ +2016-04-16 18:34 Christos Zoulas <christos@zoulas.com> + + * release 5.25 + +2016-03-31 13:50 Christos Zoulas <christos@zoulas.com> + + * make the number of bytes read from files configurable. + +2016-03-21 13:40 Christos Zoulas <christos@zoulas.com> + + * Add bounds checks for DER code (discovered by Thomas Jarosch) + * Change indirect recursion limit to indirect use count and + bump from 15 to 50 to prevent abuse. + +2016-03-13 20:39 Christos Zoulas <christos@zoulas.com> + + * Add -00 which prints filename\0description\0 + +2016-03-01 13:28 Christos Zoulas <christos@zoulas.com> + + * Fix ID3 indirect parsing + +2016-01-19 10:18 Christos Zoulas <christos@zoulas.com> + + * add DER parsing capability + +2015-11-13 10:35 Christos Zoulas <christos@zoulas.com> + + * provide dprintf(3) for the OS's that don't have it. + +2015-11-11 16:25 Christos Zoulas <christos@zoulas.com> + + * redo the compression code report decompression errors + +2015-11-10 23:25 Christos Zoulas <christos@zoulas.com> + + * REG_STARTEND code is not working as expected, delete it. + +2015-11-09 16:05 Christos Zoulas <christos@zoulas.com> + + * Add zlib support if we have it. + +2015-11-05 11:22 Christos Zoulas <christos@zoulas.com> + + * PR/492: compression forking was broken with magic_buffer. + 2015-09-16 9:50 Christos Zoulas <christos@zoulas.com> * release 5.25 Modified: stable/10/contrib/file/README ============================================================================== --- stable/10/contrib/file/README Mon May 2 05:43:46 2016 (r298919) +++ stable/10/contrib/file/README Mon May 2 05:46:47 2016 (r298920) @@ -1,6 +1,6 @@ ## README for file(1) Command ## - @(#) $File: README,v 1.49 2015/01/02 20:23:04 christos Exp $ + @(#) $File: README,v 1.50 2016/04/16 22:40:54 christos Exp $ Mailing List: file@mx.gw.com Mailing List archives: http://mx.gw.com/pipermail/file/ @@ -67,17 +67,41 @@ in magic(5) format please, to the mainta COPYING - read this first. README - read this second (you are currently reading this file). INSTALL - read on how to install +src/localtime_r.c +src/magic.c +src/magic.h +src/mygetopt.h +src/newtest2.c +src/newtest3.c +src/pread.c +src/print.c +src/readcdf.c +src/readelf.c +src/readelf.h +src/regex.c +src/regex2.c +src/softmagic.c +src/strcasestr.c +src/strlcat.c +src/strlcpy.c +src/strndup.c +src/tar.h +src/teststrchr.c +src/vasprintf.c +src/x.c src/apprentice.c - parses /etc/magic to learn magic -src/asctime_r.c - replacement for OS's that don't have it. src/apptype.c - used for OS/2 specific application type magic -src/asprintf.c - replacement for OS's that don't have it. src/ascmagic.c - third & last set of tests, based on hardwired assumptions. src/asctime_r.c - replacement for OS's that don't have it. src/asprintf.c - replacement for OS's that don't have it. +src/asctime_r.c - replacement for OS's that don't have it. +src/asprintf.c - replacement for OS's that don't have it. src/cdf.[ch] - parser for Microsoft Compound Document Files src/cdf_time.c - time converter for CDF. src/compress.c - handles decompressing files to look inside. src/ctime_r.c - replacement for OS's that don't have it. +src/der.[ch] - parser for Distinguished Encoding Rules +src/dprintf.c - replacement for OS's that don't have it. src/elfclass.h - common code for elf 32/64. src/encoding.c - handles unicode encodings src/file.c - the main program @@ -88,10 +112,13 @@ src/fsmagic.c - first set of tests the p src/funcs.c - utilility functions src/getline.c - replacement for OS's that don't have it. src/getopt_long.c - replacement for OS's that don't have it. -src/is_tar.c, tar.h - knows about tarchives (courtesy John Gilmore). -src/names.h - header file for ascmagic.c +src/gmtime_r.c - replacement for OS's that don't have it. +src/is_tar.c, tar.h - knows about Tape ARchive format (courtesy John Gilmore). +src/localtime_r.c - replacement for OS's that don't have it. src/magic.h.in - source file for magic.h +src/mygetopt.h - replacement for OS's that don't have it. src/magic.c - the libmagic api +src/names.h - header file for ascmagic.c src/pread.c - replacement for OS's that don't have it. src/print.c - print results, errors, warnings. src/readcdf.c - CDF wrapper. Modified: stable/10/contrib/file/config.h.in ============================================================================== --- stable/10/contrib/file/config.h.in Mon May 2 05:43:46 2016 (r298919) +++ stable/10/contrib/file/config.h.in Mon May 2 05:46:47 2016 (r298920) @@ -32,6 +32,9 @@ /* Define to 1 if you have the <dlfcn.h> header file. */ #undef HAVE_DLFCN_H +/* Define to 1 if you have the `dprintf' function. */ +#undef HAVE_DPRINTF + /* Define to 1 if you have the <err.h> header file. */ #undef HAVE_ERR_H Modified: stable/10/contrib/file/configure ============================================================================== --- stable/10/contrib/file/configure Mon May 2 05:43:46 2016 (r298919) +++ stable/10/contrib/file/configure Mon May 2 05:46:47 2016 (r298920) @@ -1,6 +1,6 @@ #! /bin/sh # Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.69 for file 5.25. +# Generated by GNU Autoconf 2.69 for file 5.26. # # Report bugs to <christos@astron.com>. # @@ -590,8 +590,8 @@ MAKEFLAGS= # Identity of this package. PACKAGE_NAME='file' PACKAGE_TARNAME='file' -PACKAGE_VERSION='5.25' -PACKAGE_STRING='file 5.25' +PACKAGE_VERSION='5.26' +PACKAGE_STRING='file 5.26' PACKAGE_BUGREPORT='christos@astron.com' PACKAGE_URL='' @@ -1327,7 +1327,7 @@ if test "$ac_init_help" = "long"; then # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF -\`configure' configures file 5.25 to adapt to many kinds of systems. +\`configure' configures file 5.26 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... @@ -1397,7 +1397,7 @@ fi if test -n "$ac_init_help"; then case $ac_init_help in - short | recursive ) echo "Configuration of file 5.25:";; + short | recursive ) echo "Configuration of file 5.26:";; esac cat <<\_ACEOF @@ -1507,7 +1507,7 @@ fi test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF -file configure 5.25 +file configure 5.26 generated by GNU Autoconf 2.69 Copyright (C) 2012 Free Software Foundation, Inc. @@ -2163,7 +2163,7 @@ cat >config.log <<_ACEOF This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. -It was created by file $as_me 5.25, which was +It was created by file $as_me 5.26, which was generated by GNU Autoconf 2.69. Invocation command line was $ $0 $@ @@ -3029,7 +3029,7 @@ fi # Define the identity of the package. PACKAGE='file' - VERSION='5.25' + VERSION='5.26' cat >>confdefs.h <<_ACEOF @@ -14385,6 +14385,19 @@ esac fi +ac_fn_c_check_func "$LINENO" "dprintf" "ac_cv_func_dprintf" +if test "x$ac_cv_func_dprintf" = xyes; then : + $as_echo "#define HAVE_DPRINTF 1" >>confdefs.h + +else + case " $LIBOBJS " in + *" dprintf.$ac_objext "* ) ;; + *) LIBOBJS="$LIBOBJS dprintf.$ac_objext" + ;; +esac + +fi + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for gzopen in -lz" >&5 @@ -15036,7 +15049,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_wri # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" -This file was extended by file $as_me 5.25, which was +This file was extended by file $as_me 5.26, which was generated by GNU Autoconf 2.69. Invocation command line was CONFIG_FILES = $CONFIG_FILES @@ -15102,7 +15115,7 @@ _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ -file config.status 5.25 +file config.status 5.26 configured by $0, generated by GNU Autoconf 2.69, with options \\"\$ac_cs_config\\" Modified: stable/10/contrib/file/configure.ac ============================================================================== --- stable/10/contrib/file/configure.ac Mon May 2 05:43:46 2016 (r298919) +++ stable/10/contrib/file/configure.ac Mon May 2 05:46:47 2016 (r298920) @@ -1,5 +1,5 @@ dnl Process this file with autoconf to produce a configure script. -AC_INIT([file],[5.25],[christos@astron.com]) +AC_INIT([file],[5.26],[christos@astron.com]) AM_INIT_AUTOMAKE([subdir-objects foreign]) m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) @@ -145,7 +145,7 @@ dnl Checks for functions AC_CHECK_FUNCS(strerror strndup strtoul mkstemp mkostemp utimes utime wcwidth strtof newlocale uselocale freelocale setlocale) dnl Provide implementation of some required functions if necessary -AC_REPLACE_FUNCS(getopt_long asprintf vasprintf strlcpy strlcat getline ctime_r asctime_r localtime_r gmtime_r pread strcasestr fmtcheck) +AC_REPLACE_FUNCS(getopt_long asprintf vasprintf strlcpy strlcat getline ctime_r asctime_r localtime_r gmtime_r pread strcasestr fmtcheck dprintf) dnl Checks for libraries AC_CHECK_LIB(z,gzopen) Modified: stable/10/contrib/file/doc/file.man ============================================================================== --- stable/10/contrib/file/doc/file.man Mon May 2 05:43:46 2016 (r298919) +++ stable/10/contrib/file/doc/file.man Mon May 2 05:46:47 2016 (r298920) @@ -1,5 +1,5 @@ -.\" $File: file.man,v 1.118 2015/09/11 17:24:09 christos Exp $ -.Dd September 11, 2015 +.\" $File: file.man,v 1.120 2016/03/31 17:51:12 christos Exp $ +.Dd March 13, 2016 .Dt FILE __CSECTION__ .Os .Sh NAME @@ -317,6 +317,7 @@ Set various parameter limits. .It Li elf_phnum Ta 128 Ta max ELF program sections processed .It Li elf_shnum Ta 32768 Ta max ELF sections processed .It Li regex Ta 8192 Ta length limit for regex searches +.It Li bytes Ta 1048576 Ta max number of bytes to read from file .El .It Fl r , Fl Fl raw Don't translate unprintable characters to \eooo. @@ -358,6 +359,11 @@ Nice to .Xr cut 1 the output. This does not affect the separator, which is still printed. +.Pp +If this option is repeated more than once, then +.Nm +prints just the filename followed by a NUL followed by the description +(or ERROR: text) followed by a second NUL for each entry. .It Fl -help Print a help message and exit. .El Modified: stable/10/contrib/file/doc/libmagic.man ============================================================================== --- stable/10/contrib/file/doc/libmagic.man Mon May 2 05:43:46 2016 (r298919) +++ stable/10/contrib/file/doc/libmagic.man Mon May 2 05:46:47 2016 (r298920) @@ -1,4 +1,4 @@ -.\" $File: libmagic.man,v 1.38 2015/09/11 17:24:09 christos Exp $ +.\" $File: libmagic.man,v 1.40 2016/03/31 17:51:12 christos Exp $ .\" .\" Copyright (c) Christos Zoulas 2003. .\" All Rights Reserved. @@ -225,7 +225,7 @@ It returns 0 on success and \-1 on failu .Pp The .Fn magic_compile -function can be used to compile the the colon +function can be used to compile the colon separated list of database files passed in as .Ar filename , or @@ -251,7 +251,7 @@ for the default database. .Pp The .Fn magic_load -function must be used to load the the colon +function must be used to load the colon separated list of database files passed in as .Ar filename , or @@ -282,7 +282,7 @@ The .Fn magic_getparam and .Fn magic_setparam -allow getting and setting various limits related to the the magic +allow getting and setting various limits related to the magic library. .Bl -column "MAGIC_PARAM_ELF_PHNUM_MAX" "size_t" "Default" -offset indent .It Sy "Parameter" Ta Sy "Type" Ta Sy "Default" @@ -292,6 +292,7 @@ library. .It Li MAGIC_PARAM_ELF_PHNUM_MAX Ta size_t Ta 128 .It Li MAGIC_PARAM_ELF_SHNUM_MAX Ta size_t Ta 32768 .It Li MAGIC_PARAM_REGEX_MAX Ta size_t Ta 8192 +.It Li MAGIC_PARAM_BYTES_MAX Ta size_t Ta 1048576 .El .Pp The Modified: stable/10/contrib/file/magic/Magdir/android ============================================================================== --- stable/10/contrib/file/magic/Magdir/android Mon May 2 05:43:46 2016 (r298919) +++ stable/10/contrib/file/magic/Magdir/android Mon May 2 05:46:47 2016 (r298920) @@ -1,6 +1,6 @@ #------------------------------------------------------------ -# $File: android,v 1.8 2015/03/19 18:04:37 christos Exp $ +# $File: android,v 1.9 2016/01/11 21:19:18 christos Exp $ # Various android related magic entries #------------------------------------------------------------ @@ -128,7 +128,7 @@ # partition size in blocks ? #>>>>0x22 ulelong x \b*%d -# Android bootimg format +# Android sparse img format # From https://android.googlesource.com/\ # platform/system/core/+/master/libsparse/sparse_format.h 0 lelong 0xed26ff3a Android sparse image Modified: stable/10/contrib/file/magic/Magdir/animation ============================================================================== --- stable/10/contrib/file/magic/Magdir/animation Mon May 2 05:43:46 2016 (r298919) +++ stable/10/contrib/file/magic/Magdir/animation Mon May 2 05:46:47 2016 (r298920) @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: animation,v 1.56 2014/10/23 23:12:51 christos Exp $ +# $File: animation,v 1.57 2015/11/29 22:11:07 christos Exp $ # animation: file(1) magic for animation/movie formats # # animation formats @@ -76,6 +76,8 @@ >8 string da2b \b, DMB MAF, ext da2a, with 3GPP timed text, DID, TVA, REL, IPMP >8 string da3a \b, DMB MAF aud with HE-AAC aud, JPG/PNG/MNG images >8 string da3b \b, DMB MAF, ext da3a w/ BIFS, 3GPP, DID, TVA, REL, IPMP +>8 string dash \b, MPEG v4 system, Dynamic Adaptive Streaming over HTTP +!:mime video/mp4 >8 string dmb1 \b, DMB MAF supporting all the components defined in the spec >8 string dmpf \b, Digital Media Project >8 string drc1 \b, Dirac (wavelet compression), encap in ISO base media (MP4) Modified: stable/10/contrib/file/magic/Magdir/apple ============================================================================== --- stable/10/contrib/file/magic/Magdir/apple Mon May 2 05:43:46 2016 (r298919) +++ stable/10/contrib/file/magic/Magdir/apple Mon May 2 05:46:47 2016 (r298920) @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: apple,v 1.31 2015/08/29 07:10:35 christos Exp $ +# $File: apple,v 1.32 2015/12/04 20:40:10 christos Exp $ # apple: file(1) magic for Apple file formats # 0 search/1/t FiLeStArTfIlEsTaRt binscii (apple ][) text @@ -65,18 +65,48 @@ # Eric Fischer <enf@pobox.com> # AppleWorks word processor: -# -# This matches the standard tab stops for an AppleWorks file, but if -# a file has a tab stop set in the first four columns this will fail. -# +# URL: https://en.wikipedia.org/wiki/AppleWorks +# Reference: http://www.gno.org/pub/apple2/doc/apple/filetypes/ftn.1a.xxxx +# Update: Joerg Jenderek +# NOTE: # The "O" is really the magic number, but that's so common that it's # necessary to check the tab stops that follow it to avoid false positives. - -4 string O==== AppleWorks word processor data ->85 byte&0x01 >0 \b, zoomed ->90 byte&0x01 >0 \b, paginated ->92 byte&0x01 >0 \b, with mail merge -#>91 byte x \b, left margin %d +# and/or look for unused bits of booleans bytes like zoom, paginated, mail merge +# the newer AppleWorks is from claris with extension CWK +4 string O +# test for unused bits of zoom- , paginated-boolean bytes +>84 ubequad ^0x00Fe00000000Fe00 +# look for tabstop definitions "=" no tab, "|" no tab +# "<" left tab,"^" center tab,">" right tab, "." decimal tab, +# unofficial "!" other , "\x8a" other +# official only if SFMinVers is nonzero +>>5 regex/s [=.<>|!^\x8a]{79} AppleWorks Word Processor +# AppleWorks Word Processor File (Apple II) +# ./apple (version 5.25) labeled the entry as "AppleWorks word processor data" +# application/x-appleworks is mime type for claris version with cwk extension +!:mime application/x-appleworks3 +# http://home.earthlink.net/~hughhood/appleiiworksenvoy/ +# ('p' + 1-byte ProDOS File Type + 2-byte ProDOS Aux Type') +# $70 $1A $F8 $FF is this the apple type ? +#:apple pdospøÿ +!:ext awp +# minimum version needed to read this files. SFMinVers (0 , 30~3.0 ) +>>>183 ubyte 30 3.0 +>>>183 ubyte !30 +>>>>183 ubyte !0 0x%x +# usual tabstop start sequence "=====<" +>>>5 string x \b, tabstop ruler "%6.6s" +# tabstop ruler +#>>>5 string >\0 \b, tabstops "%-79s" +# zoom switch +>>>85 byte&0x01 >0 \b, zoomed +# whether paginated +>>>90 byte&0x01 >0 \b, paginated +# contains any mail-merge commands +>>>92 byte&0x01 >0 \b, with mail merge +# left margin in 1/10 inches ( normally 0 or 10 ) +>>>91 ubyte >0 +>>>>91 ubyte x \b, %d/10 inch left margin # AppleWorks database: # Modified: stable/10/contrib/file/magic/Magdir/archive ============================================================================== --- stable/10/contrib/file/magic/Magdir/archive Mon May 2 05:43:46 2016 (r298919) +++ stable/10/contrib/file/magic/Magdir/archive Mon May 2 05:46:47 2016 (r298920) @@ -1,5 +1,5 @@ #------------------------------------------------------------------------------ -# $File: archive,v 1.91 2015/09/16 13:49:33 christos Exp $ +# $File: archive,v 1.102 2016/01/11 20:59:24 christos Exp $ # archive: file(1) magic for archive formats (see also "msdos" for self- # extracting compressed archives) # @@ -246,7 +246,15 @@ # BA # TODO: idarc says "bytes 0-2 == bytes 3-5" # TTComp -0 string \0\6 TTComp archive data +# URL: http://fileformats.archiveteam.org/wiki/TTComp_archive +# Update: Joerg Jenderek +# GRR: line below is too general as it matches also Panorama database "TCDB 2003-10 demo.pan", others +0 string \0\6 +# look for first keyword of Panorama database *.pan +>12 search/261 DESIGN +# skip keyword with low entropy +>12 default x TTComp archive, binary, 4K dictionary +# (version 5.25) labeled the above entry as "TTComp archive data" # ESP, could this conflict with Easy Software Products' (e.g.ESP ghostscript) documentation? 0 string ESP ESP archive data # ZPack @@ -544,55 +552,212 @@ >>0x36 string >\0 fstype %.8s # LHARC/LHA archiver (Greg Roelofs, newt@uchicago.edu) -2 string -lh0- LHarc 1.x/ARX archive data [lh0] -!:mime application/x-lharc -2 string -lh1- LHarc 1.x/ARX archive data [lh1] -!:mime application/x-lharc -2 string -lz4- LHarc 1.x archive data [lz4] -!:mime application/x-lharc -2 string -lz5- LHarc 1.x archive data [lz5] -!:mime application/x-lharc +# Update: Joerg Jenderek +# URL: https://en.wikipedia.org/wiki/LHA_(file_format) +# Reference: http://web.archive.org/web/20021005080911/http://www.osirusoft.com/joejared/lzhformat.html +# +# check and display information of lharc (LHa,PMarc) file +0 name lharc-file +# check 1st character of method id like -lz4- -lh5- or -pm2- +>2 string - +# check 5th character of method id +>>6 string - +# check header level 0 1 2 3 +>>>20 ubyte <4 +# check 2nd, 3th and 4th character of method id +>>>>3 regex \^(lh[0-9a-ex]|lz[s2-8]|pm[012]|pc1) \b +!:mime application/x-lzh-compressed +# creator type "LHA " +!:apple ????LHA +# display archive type name like "LHa/LZS archive data" or "LArc archive" +>>>>>2 string -lz \b +!:ext lzs +# already known -lzs- -lz4- -lz5- with old names +>>>>>>2 string -lzs LHa/LZS archive data +>>>>>>3 regex \^lz[45] LHarc 1.x archive data +# missing -lz?- with wikipedia names +>>>>>>3 regex \^lz[2378] LArc archive +# display archive type name like "LHa (2.x) archive data" +>>>>>2 string -lh \b +# already known -lh0- -lh1- -lh2- -lh3- -lh4- -lh5- -lh6- -lh7- -lhd- variants with old names +>>>>>>3 regex \^lh[01] LHarc 1.x/ARX archive data +# LHice archiver use ".ICE" as name extension instead usual one ".lzh" +# FOOBAR archiver use ".foo" as name extension instead usual one +# "Florain Orjanov's and Olga Bachetska's ARchiver" not found at the moment +>>>>>>>2 string -lh1 \b +!:ext lha/lzh/ice +>>>>>>3 regex \^lh[23d] LHa 2.x? archive data +>>>>>>3 regex \^lh[7] LHa (2.x)/LHark archive data +>>>>>>3 regex \^lh[456] LHa (2.x) archive data +>>>>>>>2 string -lh5 \b +# https://en.wikipedia.org/wiki/BIOS +# Some mainboard BIOS like Award use LHa compression. So archives with unusal extension are found like +# bios.rom , kd7_v14.bin, 1010.004, ... +!:ext lha/lzh/rom/bin +# missing -lh?- variants (Joe Jared) +>>>>>>3 regex \^lh[89a-ce] LHa (Joe Jared) archive +# UNLHA32 2.67a +>>>>>>2 string -lhx LHa (UNLHA32) archive +# lha archives with standard file name extensions ".lha" ".lzh" +>>>>>>3 regex !\^(lh1|lh5) \b +!:ext lha/lzh +# this should not happen if all -lh variants are described +>>>>>>2 default x LHa (unknown) archive +#!:ext lha +# PMarc +>>>>>3 regex \^pm[012] PMarc archive data +!:ext pma +# append method id without leading and trailing minus character +>>>>>3 string x [%3.3s] +>>>>>>0 use lharc-header +# +# check and display information of lharc header +0 name lharc-header +# header size 0x4 , 0x1b-0x61 +>0 ubyte x +# compressed data size != compressed file size +#>7 ulelong x \b, data size %d +# attribute: 0x2~?? 0x10~symlink|target 0x20~normal +#>19 ubyte x \b, 19_0x%x +# level identifier 0 1 2 3 +#>20 ubyte x \b, level %d +# time stamp +#>15 ubelong x DATE 0x%8.8x +# OS ID for level 1 +>20 ubyte 1 +# 0x20 types find for *.rom files +>>(21.b+24) ubyte <0x21 \b, 0x%x OS +# ascii type like M for MSDOS +>>(21.b+24) ubyte >0x20 \b, '%c' OS +# OS ID for level 2 +>20 ubyte 2 +#>>23 ubyte x \b, OS ID 0x%x +>>23 ubyte <0x21 \b, 0x%x OS +>>23 ubyte >0x20 \b, '%c' OS +# filename only for level 0 and 1 +>20 ubyte <2 +# length of filename +>>21 ubyte >0 \b, with +# filename +>>>21 pstring x "%s" +# +#2 string -lh0- LHarc 1.x/ARX archive data [lh0] +#!:mime application/x-lharc +2 string -lh0- +>0 use lharc-file +#2 string -lh1- LHarc 1.x/ARX archive data [lh1] +#!:mime application/x-lharc +2 string -lh1- +>0 use lharc-file +# NEW -lz2- ... -lz8- +2 string -lz2- +>0 use lharc-file +2 string -lz3- +>0 use lharc-file +2 string -lz4- +>0 use lharc-file +2 string -lz5- +>0 use lharc-file +2 string -lz7- +>0 use lharc-file +2 string -lz8- +>0 use lharc-file # [never seen any but the last; -lh4- reported in comp.compression:] -2 string -lzs- LHa/LZS archive data [lzs] -!:mime application/x-lha -2 string -lh\40- LHa 2.x? archive data [lh ] -!:mime application/x-lha -2 string -lhd- LHa 2.x? archive data [lhd] -!:mime application/x-lha -2 string -lh2- LHa 2.x? archive data [lh2] -!:mime application/x-lha -2 string -lh3- LHa 2.x? archive data [lh3] -!:mime application/x-lha -2 string -lh4- LHa (2.x) archive data [lh4] -!:mime application/x-lha -2 string -lh5- LHa (2.x) archive data [lh5] -!:mime application/x-lha -2 string -lh6- LHa (2.x) archive data [lh6] -!:mime application/x-lha -2 string -lh7- LHa (2.x)/LHark archive data [lh7] -!:mime application/x-lha ->20 byte x - header level %d +#2 string -lzs- LHa/LZS archive data [lzs] +2 string -lzs- +>0 use lharc-file +# According to wikipedia and others such a version does not exist +#2 string -lh\40- LHa 2.x? archive data [lh ] +#2 string -lhd- LHa 2.x? archive data [lhd] +2 string -lhd- +>0 use lharc-file +#2 string -lh2- LHa 2.x? archive data [lh2] +2 string -lh2- +>0 use lharc-file +#2 string -lh3- LHa 2.x? archive data [lh3] +2 string -lh3- +>0 use lharc-file +#2 string -lh4- LHa (2.x) archive data [lh4] +2 string -lh4- +>0 use lharc-file +#2 string -lh5- LHa (2.x) archive data [lh5] +2 string -lh5- +>0 use lharc-file +#2 string -lh6- LHa (2.x) archive data [lh6] +2 string -lh6- +>0 use lharc-file +#2 string -lh7- LHa (2.x)/LHark archive data [lh7] +2 string -lh7- +# !:mime application/x-lha +# >20 byte x - header level %d +>0 use lharc-file +# NEW -lh8- ... -lhe- , -lhx- +2 string -lh8- +>0 use lharc-file +2 string -lh9- +>0 use lharc-file +2 string -lha- +>0 use lharc-file +2 string -lhb- +>0 use lharc-file +2 string -lhc- +>0 use lharc-file +2 string -lhe- +>0 use lharc-file +2 string -lhx- +>0 use lharc-file # taken from idarc [JW] 2 string -lZ PUT archive data -2 string -lz LZS archive data +# already done by LHarc magics +# this should never happen if all sub types of LZS archive are identified +#2 string -lz LZS archive data 2 string -sw1- Swag archive data -# RAR archiver (Greg Roelofs, newt@uchicago.edu) -0 string Rar! RAR archive data, +0 name rar-file-header +>24 byte 15 \b, v1.5 +>24 byte 20 \b, v2.0 +>24 byte 29 \b, v4 +>15 byte 0 \b, os: MS-DOS +>15 byte 1 \b, os: OS/2 +>15 byte 2 \b, os: Win32 +>15 byte 3 \b, os: Unix +>15 byte 4 \b, os: Mac OS +>15 byte 5 \b, os: BeOS + +0 name rar-archive-header +>3 leshort&0x1ff >0 \b, flags: +>>3 leshort &0x01 ArchiveVolume +>>3 leshort &0x02 Commented +>>3 leshort &0x04 Locked +>>3 leshort &0x10 NewVolumeNaming +>>3 leshort &0x08 Solid +>>3 leshort &0x20 Authenticated +>>3 leshort &0x40 RecoveryRecordPresent +>>3 leshort &0x80 EncryptedBlockHeader +>>3 leshort &0x100 FirstVolume + +# RAR (Roshal Archive) archive +0 string Rar!\x1a\7\0 RAR archive data +!:mime application/x-rar +!:ext rar/cbr +# file header +>(0xc.l+9) byte 0x74 +>>(0xc.l+7) use rar-file-header +# subblock seems to share information with file header +>(0xc.l+9) byte 0x7a +>>(0xc.l+7) use rar-file-header +>9 byte 0x73 +>>7 use rar-archive-header + +0 string Rar!\x1a\7\1\0 RAR archive data, v5 +!:mime application/x-rar +!:ext rar + +# Very old RAR archive +# http://jasonblanks.com/wp-includes/images/papers/KnowyourarchiveRAR.pdf +0 string RE\x7e\x5e RAR archive data (<v1.5) !:mime application/x-rar ->44 byte x v%0x, ->10 byte >0 flags: ->>10 byte &0x01 Archive volume, ->>10 byte &0x02 Commented, ->>10 byte &0x04 Locked, ->>10 byte &0x08 Solid, ->>10 byte &0x20 Authenticated, ->35 byte 0 os: MS-DOS ->35 byte 1 os: OS/2 ->35 byte 2 os: Win32 ->35 byte 3 os: Unix -# some old version? idarc says: -0 string RE\x7e\x5e RAR archive data +!:ext rar/cbr # SQUISH archiver (Greg Roelofs, newt@uchicago.edu) 0 string SQSH squished archive data (Acorn RISCOS) @@ -604,9 +769,12 @@ # PKZIP multi-volume archive 0 string PK\x07\x08PK\x03\x04 Zip multi-volume archive data, at least PKZIP v2.50 to extract !:mime application/zip +!:ext zip/cbz # Zip archives (Greg Roelofs, c/o zip-bugs@wkuvx1.wku.edu) 0 string PK\005\006 Zip archive data (empty) +!:mime application/zip +!:ext zip/cbz 0 string PK\003\004 # Specialised zip formats which start with a member named 'mimetype' @@ -764,12 +932,24 @@ 0 string \0\ \ \ \ \ \ \ \ \ \ \ \0\0 LBR archive data # # PMA (CP/M derivative of LHA) +# Update: Joerg Jenderek +# URL: https://en.wikipedia.org/wiki/LHA_(file_format) # -2 string -pm0- PMarc archive data [pm0] -2 string -pm1- PMarc archive data [pm1] -2 string -pm2- PMarc archive data [pm2] +#2 string -pm0- PMarc archive data [pm0] +2 string -pm0- +>0 use lharc-file +#2 string -pm1- PMarc archive data [pm1] +2 string -pm1- +>0 use lharc-file +#2 string -pm2- PMarc archive data [pm2] +2 string -pm2- +>0 use lharc-file 2 string -pms- PMarc SFX archive (CP/M, DOS) +#!:mime application/x-foobar-exec +!:ext com 5 string -pc1- PopCom compressed executable (CP/M) +#!:mime application/x- +#!:ext com # From Rafael Laboissiere <rafael@laboissiere.net> # The Project Revision Control System (see @@ -802,6 +982,9 @@ # Felix von Leitner <felix-file@fefe.de> 0 string d8:announce BitTorrent file !:mime application/x-bittorrent +# Durval Menezes, <jmgthbfile at durval dot com> +0 string d13:announce-list BitTorrent file +!:mime application/x-bittorrent # Atari MSA archive - Teemu Hukkanen <tjhukkan@iki.fi> 0 beshort 0x0e0f Atari MSA archive data @@ -889,19 +1072,16 @@ # From "Nelson A. de Oliveira" <naoliv@gmail.com> 0 string MPQ\032 MoPaQ (MPQ) archive -# From: Dirk Jagdmann <doj@cubic.org> -# xar archive format: http://code.google.com/p/xar/ -0 string xar! xar archive ->6 beshort x - version %d - # From: "Nelson A. de Oliveira" <naoliv@gmail.com> # .kgb 0 string KGB_arch KGB Archiver file >10 string x with compression level %.1s # xar (eXtensible ARchiver) archive +# xar archive format: http://code.google.com/p/xar/ # From: "David Remahl" <dremahl@apple.com> 0 string xar! xar archive +!:mime application/x-xar #>4 beshort x header size %d >6 beshort x version %d, #>8 quad x compressed TOC: %d, @@ -975,3 +1155,9 @@ >0xE08 search/7776 \x55\xAA >>&-512 indirect x \b; contains +# Google Chrome extensions +# https://developer.chrome.com/extensions/crx +# https://developer.chrome.com/extensions/hosting +0 string Cr24 Google Chrome extension +!:mime application/x-chrome-extension +>4 ulong x \b, version %u Modified: stable/10/contrib/file/magic/Magdir/audio ============================================================================== --- stable/10/contrib/file/magic/Magdir/audio Mon May 2 05:43:46 2016 (r298919) +++ stable/10/contrib/file/magic/Magdir/audio Mon May 2 05:46:47 2016 (r298920) @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: audio,v 1.73 2015/03/15 23:21:42 christos Exp $ +# $File: audio,v 1.75 2016/02/08 17:30:11 christos Exp $ # audio: file(1) magic for sound formats (see also "iff") # # Jan Nicolai Langfeldt (janl@ifi.uio.no), Dan Quinlan (quinlan@yggdrasil.com), @@ -585,7 +585,7 @@ 0 string SC68\ Music-file\ /\ (c)\ (BeN)jami sc68 Atari ST music # musepak support From: "Jiri Pejchal" <jiri.pejchal@gmail.com> -0 string MP+ Musepack audio +0 string MP+ Musepack audio (MP+) !:mime audio/x-musepack >3 byte 255 \b, SV pre8 >3 byte&0xF 0x6 \b, SV 6 @@ -619,6 +619,9 @@ >>27 byte 114 \b, Beta 1.14 >>27 byte 115 \b, Alpha 1.15 +0 string MPCK Musepack audio (MPCK) +!:mime audio/x-musepack + # IMY # from http://filext.com/detaillist.php?extdetail=IMY # http://cellphones.about.com/od/cellularfaqs/f/rf_imelody.htm @@ -715,3 +718,41 @@ 0 string ZBOT >4 byte 0xc5 GVOX Encore music, version < 5.0 +# Summary: Garmin Voice Processing Module (WAVE audios) +# From: Joerg Jenderek +# URL: http://www.garmin.com/ +# Reference: http://turboccc.wikispaces.com/share/view/28622555 +# NOTE: there exist 2 other Garmin VPM formats +0 string AUDIMG +# skip text files starting with string "AUDIMG" +>13 ubyte <13 Garmin Voice Processing Module +!:mime audio/x-vpm-wav-garmin +!:ext vpm +# 3 bytes indicating the voice version (200,220) +>>6 string x \b, version %3.3s +# day of release (01-31) +>>12 ubyte x \b, %.2d +# month of release (01-12) +>>13 ubyte x \b.%.2d +# year of release (like 2006, 2007, 2008) +>>14 uleshort x \b.%.4d +# hour of release (0-23) +>>11 ubyte x %.2d +# minute of release (0-59) +>>10 ubyte x \b:%.2d +# second of release (0-59) +>>9 ubyte x \b:%.2d +# if you select a language like german on your garmin device +# you can only select voice modules with correponding language byte ID like 1 +>>18 ubyte x \b, language ID %d +# pointer to 1st audio WAV sample +>>16 uleshort >0 +>>>(16.s) ulelong >0 \b, at offset 0x%x +# WAV length +>>>>(16.s+4) ulelong >0 %d Bytes +# look for magic +>>>>>(&-8.l) string RIFF +# determine type by ./riff +>>>>>>&-4 indirect x \b +# 2 - ~ 131 WAV samples following same way + Copied: stable/10/contrib/file/magic/Magdir/bioinformatics (from r298192, head/contrib/file/magic/Magdir/bioinformatics) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/10/contrib/file/magic/Magdir/bioinformatics Mon May 2 05:46:47 2016 (r298920, copy of r298192, head/contrib/file/magic/Magdir/bioinformatics) @@ -0,0 +1,178 @@ + +#------------------------------------------------------------------------------ +# $File: bioinformatics,v 1.2 2016/02/14 15:53:53 christos Exp $ +# bioinfomatics: file(1) magic for Bioinfomatics file formats + +############################################################################### +# BGZF (Blocked GNU Zip Format) - gzip compatible, but also indexable +# used by SAMtools bgzip/tabix (http://samtools.sourceforge.net/tabix.shtml) +############################################################################### +0 string \037\213 +>3 byte &0x04 +>>12 string BC +>>>14 leshort &0x02 Blocked GNU Zip Format (BGZF; gzip compatible) +>>>>16 leshort x \b, block length %d +!:mime application/x-gzip + + +############################################################################### +# Tabix index file +# used by SAMtools bgzip/tabix (http://samtools.sourceforge.net/tabix.shtml) +############################################################################### +0 string TBI\1 SAMtools TBI (Tabix index format) +>0x04 lelong =1 \b, with %d reference sequence +>0x04 lelong >1 \b, with %d reference sequences +>0x08 lelong &0x10000 \b, using half-closed-half-open coordinates (BED style) +>0x08 lelong ^0x10000 +>>0x08 lelong =0 \b, using closed and one based coordinates (GFF style) +>>0x08 lelong =1 \b, using SAM format +>>0x08 lelong =2 \b, using VCF format +>0x0c lelong x \b, sequence name column: %d +>0x10 lelong x \b, region start column: %d +>0x08 lelong =0 +>>0x14 lelong x \b, region end column: %d +>0x18 byte x \b, comment character: %c +>0x1c lelong x \b, skip line count: %d + + +############################################################################### +# BAM (Binary Sequence Alignment/Map format) +# used by SAMtools (http://samtools.sourceforge.net/SAM1.pdf) +# data is normally present only within compressed BGZF blocks (CDATA), so use file -z to examine it +############################################################################### +0 string BAM\1 SAMtools BAM (Binary Sequence Alignment/Map) +>0x04 lelong >0 +>>&0x00 regex =^[@]HD\t.*VN: \b, with SAM header +>>>&0 regex =[0-9.]+ \b version %s +>>&(0x04) lelong >0 \b, with %d reference sequences + + +############################################################################### +# BAI (BAM indexing format) +# used by SAMtools (http://samtools.sourceforge.net/SAM1.pdf) +############################################################################### +0 string BAI\1 SAMtools BAI (BAM indexing format) +>0x04 lelong >0 \b, with %d reference sequences + + +############################################################################### +# CRAM (Binary Sequence Alignment/Map format) +############################################################################### +0 string CRAM CRAM +>0x04 byte >-1 version %d. +>0x05 byte >-1 \b%d +>0x06 string >\0 (identified as %s) + + +############################################################################### +# BCF (Binary Call Format), version 1 +# used by SAMtools & VCFtools (http://vcftools.sourceforge.net/bcf.pdf) +# data is normally present only within compressed BGZF blocks (CDATA), so use file -z to examine it +############################################################################### +0 string BCF\4 +# length of seqnm data in bytes is positive +>&0x00 lelong >0 +# length of smpl data in bytes is positive +>>&(&-0x04) lelong >0 SAMtools BCF (Binary Call Format) +# length of meta in bytes +>>>&(&-0x04) lelong >0 +# have meta text string +>>>>&0x00 search ##samtoolsVersion= +>>>>>&0x00 string x \b, generated by SAMtools version %s + + +############################################################################### +# BCF (Binary Call Format), version 2.1 +# used by SAMtools (http://samtools.github.io/hts-specs/BCFv2_qref.pdf) +# data is normally present only within compressed BGZF blocks (CDATA), so use file -z to examine it +############################################################################### +0 string BCF\2\1 Binary Call Format (BCF) version 2.1 +# length of header text +>&0x00 lelong >0 +# have header string +>>&0x00 search ##samtoolsVersion= +>>>&0x00 string x \b, generated by SAMtools version %s + + +############################################################################### +# BCF (Binary Call Format), version 2.2 +# used by SAMtools (http://samtools.github.io/hts-specs/BCFv2_qref.pdf) +# data is normally present only within compressed BGZF blocks (CDATA), so use file -z to examine it +############################################################################### +0 string BCF\2\2 Binary Call Format (BCF) version 2.2 +# length of header text +>&0x00 lelong >0 +# have header string +>>&0x00 search ##samtoolsVersion= +>>>&0x00 string x \b, generated by SAMtools version %s + +############################################################################### +# VCF (Variant Call Format) +# used by VCFtools (http://vcftools.sourceforge.net/) +############################################################################### +0 search ##fileformat=VCFv Variant Call Format (VCF) +>&0 string x \b version %s + +############################################################################### +# FASTQ +# used by MAQ (http://maq.sourceforge.net/fastq.shtml) +############################################################################### +# XXX Broken? +# @<seqname> +#0 regex =^@[A-Za-z0-9_.:-]+\?\n +# <seq> +#>&1 regex =^[A-Za-z\n.~]++ +# +[<seqname>] *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201605020546.u425kmHi061290>