From owner-svn-src-all@FreeBSD.ORG Mon Apr 14 23:52:02 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 641E3E1A; Mon, 14 Apr 2014 23:52:02 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D09B1BEF; Mon, 14 Apr 2014 23:52:02 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s3ENq2q8007903; Mon, 14 Apr 2014 23:52:02 GMT (envelope-from jmmv@svn.freebsd.org) Received: (from jmmv@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s3ENpvm4007855; Mon, 14 Apr 2014 23:51:57 GMT (envelope-from jmmv@svn.freebsd.org) Message-Id: <201404142351.s3ENpvm4007855@svn.freebsd.org> From: Julio Merino Date: Mon, 14 Apr 2014 23:51:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r264483 - in stable/10: bin/date/tests bin/mv/tests bin/pax/tests bin/sh/tests bin/test/tests lib/atf/libatf-c++/tests lib/atf/libatf-c++/tests/detail lib/atf/libatf-c/tests lib/atf/lib... X-SVN-Group: stable-10 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Apr 2014 23:52:02 -0000 Author: jmmv Date: Mon Apr 14 23:51:57 2014 New Revision: 264483 URL: http://svnweb.freebsd.org/changeset/base/264483 Log: MFC refactoring of the *.test.mk files. - r263161 Make bsd.test.mk the only public mk fragment for the building of tests. - r263172 Move FreeBSD Test Suite-specific code to a suite.test.mk file. - r263204 Add some documentation for bsd.test.mk. - r263217 Document support for TAP-compliant Perl test programs. This is "make tinderbox" clean. Added: stable/10/share/mk/suite.test.mk - copied unchanged from r263172, head/share/mk/suite.test.mk Modified: stable/10/bin/date/tests/Makefile stable/10/bin/mv/tests/Makefile stable/10/bin/pax/tests/Makefile stable/10/bin/sh/tests/Makefile stable/10/bin/test/tests/Makefile stable/10/lib/atf/libatf-c++/tests/Makefile stable/10/lib/atf/libatf-c++/tests/detail/Makefile stable/10/lib/atf/libatf-c/tests/Makefile stable/10/lib/atf/libatf-c/tests/detail/Makefile stable/10/lib/atf/tests/test-programs/Makefile stable/10/lib/libcrypt/tests/Makefile stable/10/libexec/atf/atf-check/tests/Makefile stable/10/share/examples/tests/tests/atf/Makefile stable/10/share/examples/tests/tests/plain/Makefile stable/10/share/mk/Makefile stable/10/share/mk/atf.test.mk stable/10/share/mk/bsd.README stable/10/share/mk/bsd.test.mk stable/10/share/mk/plain.test.mk stable/10/share/mk/tap.test.mk stable/10/tests/sys/kern/Makefile stable/10/usr.bin/atf/atf-sh/tests/Makefile Directory Properties: stable/10/ (props changed) Modified: stable/10/bin/date/tests/Makefile ============================================================================== --- stable/10/bin/date/tests/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/bin/date/tests/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -6,4 +6,4 @@ TESTSDIR= ${TESTSBASE}/bin/date TAP_TESTS_SH= legacy_test -.include +.include Modified: stable/10/bin/mv/tests/Makefile ============================================================================== --- stable/10/bin/mv/tests/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/bin/mv/tests/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -6,4 +6,4 @@ TESTSDIR= ${TESTSBASE}/bin/mv TAP_TESTS_SH= legacy_test -.include +.include Modified: stable/10/bin/pax/tests/Makefile ============================================================================== --- stable/10/bin/pax/tests/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/bin/pax/tests/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -6,4 +6,4 @@ TESTSDIR= ${TESTSBASE}/bin/pax TAP_TESTS_PERL= legacy_test -.include +.include Modified: stable/10/bin/sh/tests/Makefile ============================================================================== --- stable/10/bin/sh/tests/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/bin/sh/tests/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -15,4 +15,4 @@ TEST_METADATA.legacy_test+= required_use SUBDIR+= builtins errors execution expansion parameters parser set-e -.include +.include Modified: stable/10/bin/test/tests/Makefile ============================================================================== --- stable/10/bin/test/tests/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/bin/test/tests/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -12,4 +12,4 @@ TAP_TESTS_SH= legacy_test # requested. See https://code.google.com/p/kyua/issues/detail?id=6 TEST_METADATA.legacy_test+= required_user="unprivileged" -.include +.include Modified: stable/10/lib/atf/libatf-c++/tests/Makefile ============================================================================== --- stable/10/lib/atf/libatf-c++/tests/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/lib/atf/libatf-c++/tests/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -28,4 +28,4 @@ SRCS.${_T}= ${_T}.cpp test_helpers.cpp ATF_TESTS_SH= pkg_config_test -.include +.include Modified: stable/10/lib/atf/libatf-c++/tests/detail/Makefile ============================================================================== --- stable/10/lib/atf/libatf-c++/tests/detail/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/lib/atf/libatf-c++/tests/detail/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -29,4 +29,4 @@ BINDIR.${p}= ${TESTSDIR} version_helper.o: atf-version .include "../../../common.mk" -.include +.include Modified: stable/10/lib/atf/libatf-c/tests/Makefile ============================================================================== --- stable/10/lib/atf/libatf-c/tests/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/lib/atf/libatf-c/tests/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -35,4 +35,4 @@ SRCS.${_T}= ${_T}.c test_helpers.c ATF_TESTS_SH= pkg_config_test -.include +.include Modified: stable/10/lib/atf/libatf-c/tests/detail/Makefile ============================================================================== --- stable/10/lib/atf/libatf-c/tests/detail/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/lib/atf/libatf-c/tests/detail/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -31,4 +31,4 @@ BINDIR.${p}= ${TESTSDIR} version_helper.o: atf-version .include "../../../common.mk" -.include +.include Modified: stable/10/lib/atf/tests/test-programs/Makefile ============================================================================== --- stable/10/lib/atf/tests/test-programs/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/lib/atf/tests/test-programs/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -21,4 +21,4 @@ ATF_TESTS_SH+= ${_T} ATF_TESTS_SH_SRC_${_T}= common.sh ${_T}.sh .endfor -.include +.include Modified: stable/10/lib/libcrypt/tests/Makefile ============================================================================== --- stable/10/lib/libcrypt/tests/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/lib/libcrypt/tests/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -9,4 +9,4 @@ ATF_TESTS_C= crypt_tests CFLAGS+= -I${.CURDIR:H} LDADD+= -L${.OBJDIR:H} -lcrypt -.include +.include Modified: stable/10/libexec/atf/atf-check/tests/Makefile ============================================================================== --- stable/10/libexec/atf/atf-check/tests/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/libexec/atf/atf-check/tests/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -9,4 +9,4 @@ ATF= ${.CURDIR:H:H:H:H}/contrib/atf ATF_TESTS_SH= atf-check_test -.include +.include Modified: stable/10/share/examples/tests/tests/atf/Makefile ============================================================================== --- stable/10/share/examples/tests/tests/atf/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/share/examples/tests/tests/atf/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -27,4 +27,4 @@ ATF_TESTS_SH= cp_test # definitions from above. KYUAFILE= yes -.include +.include Modified: stable/10/share/examples/tests/tests/plain/Makefile ============================================================================== --- stable/10/share/examples/tests/tests/plain/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/share/examples/tests/tests/plain/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -27,4 +27,4 @@ PLAIN_TESTS_SH= cp_test # definitions from above. KYUAFILE= yes -.include +.include Modified: stable/10/share/mk/Makefile ============================================================================== --- stable/10/share/mk/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/share/mk/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -46,6 +46,7 @@ FILESDIR= ${BINDIR}/mk .if ${MK_TESTS} != "no" FILES+= atf.test.mk FILES+= plain.test.mk +FILES+= suite.test.mk FILES+= tap.test.mk .endif Modified: stable/10/share/mk/atf.test.mk ============================================================================== --- stable/10/share/mk/atf.test.mk Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/share/mk/atf.test.mk Mon Apr 14 23:51:57 2014 (r264483) @@ -1,9 +1,13 @@ # $FreeBSD$ # +# You must include bsd.test.mk instead of this file from your Makefile. +# # Logic to build and install ATF test programs; i.e. test programs linked # against the ATF libraries. -.include +.if !target(____) +.error atf.test.mk cannot be included directly. +.endif # List of C, C++ and shell test programs to build. # @@ -164,5 +168,3 @@ realtest: .PHONY .endif .endif - -.include Modified: stable/10/share/mk/bsd.README ============================================================================== --- stable/10/share/mk/bsd.README Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/share/mk/bsd.README Mon Apr 14 23:51:57 2014 (r264483) @@ -39,6 +39,7 @@ bsd.prog.mk - building programs from so bsd.snmpmod.mk - building modules for the SNMP daemon bsnmpd bsd.subdir.mk - targets for building subdirectories bsd.sys.mk - common settings used for building FreeBSD sources +bsd.test.mk - building test programs from source files sys.mk - default rules for all makes This file does not document bsd.port*.mk. They are documented in ports(7). @@ -365,3 +366,97 @@ It has rules for building profiled objec built by default. Libraries are ranlib'd before installation. + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= + +The include file handles building one or more test programs +intended to be used in the FreeBSD Test Suite under /usr/tests/. + +It has seven targets: + + all: + build the test programs. + clean: + remove the test programs and any object files. + cleandir: + remove all of the files removed by the target clean, as + well as .depend and tags. + depend: + make the dependencies for the source files, and store + them in the file .depend. + install: + install the test programs and their data files; if the + Makefile does not itself define the target install, the + targets beforeinstall and afterinstall may also be used + to cause actions immediately before and after the + install target is executed. + lint: + run lint on the source files. + tags: + create a tags file for the source files. + test: + runs the test programs from the object directory; if the + Makefile does not itself define the target test, the + targets beforetest and aftertest may also be used to + cause actions immediately before and after the test + target is executed. + +It sets/uses the following variables, among many others: + +TESTDIR Path to the installed tests. Must be a subdirectory of + TESTSBASE and the subpath should match the relative + location of the tests within the src tree. + +KYUAFILE If 'auto' (the default), generate a Kyuafile out of the + test programs defined in the Makefile. If 'yes', then a + manually-crafted Kyuafile must be supplied with the + sources. If 'no', no Kyuafile is installed (useful for + subdirectories providing helper programs or data files + only). + +ATF_TESTS_C The names of the ATF C test programs to build. + +ATF_TESTS_CXX The names of the ATF C++ test programs to build. + +ATF_TESTS_SH The names of the ATF sh test programs to build. + +PLAIN_TESTS_C The names of the plain (legacy) programs to build. + +PLAIN_TESTS_CXX The names of the plain (legacy) test programs to build. + +PLAIN_TESTS_SH The names of the plain (legacy) test programs to build. + +TAP_PERL_INTERPRETER + Path to the Perl interpreter to be used for + TAP-compliant test programs that are written in Perl. + Refer to TAP_TESTS_PERL for details. + +TAP_TESTS_C The names of the TAP-compliant C test programs to build. + +TAP_TESTS_CXX The names of the TAP-compliant C++ test programs to + build. + +TAP_TESTS_PERL The names of the TAP-compliant Perl test programs to + build. The corresponding source files should end with + the .pl extension; the test program is marked as + requiring Perl; and TAP_PERL_INTERPRETER is used in the + built scripts as the interpreter of choice. + +TAP_TESTS_SH The names of the TAP-compliant sh test programs to + build. + +TESTS_SUBDIRS List of subdirectories containing tests into which to + recurse. Differs from SUBDIR in that these directories + get registered into the automatically-generated + Kyuafile (if any). + +NOT_FOR_TEST_SUITE + If defined, none of the built test programs get + installed under /usr/tests/ and no Kyuafile is + automatically generated. Should not be used within the + FreeBSD source tree but is provided for the benefit of + third-parties. + +The actual building of the test programs is performed by . +Please see the documentation above for this other file for additional +details on the behavior of . Modified: stable/10/share/mk/bsd.test.mk ============================================================================== --- stable/10/share/mk/bsd.test.mk Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/share/mk/bsd.test.mk Mon Apr 14 23:51:57 2014 (r264483) @@ -2,23 +2,13 @@ # # Generic build infrastructure for test programs. # -# The code in this file is independent of the implementation of the test -# programs being built; this file just provides generic infrastructure for the -# build and the definition of various helper variables and targets. -# -# Makefiles should never include this file directly. Instead, they should -# include one of the various *.test.mk depending on the specific test programs -# being built. +# This is the only public file that should be included by Makefiles when +# tests are to be built. All other *.test.mk files are internal and not +# to be included directly. .include -# Directory in which to install tests defined by the current Makefile. -# Makefiles have to override this to point to a subdirectory of TESTSBASE. -TESTSDIR?= . - -# Name of the test suite these tests belong to. Should rarely be changed for -# Makefiles built into the FreeBSD src tree. -TESTSUITE?= FreeBSD +____: # List of subdirectories containing tests into which to recurse. This has the # same semantics as SUBDIR at build-time. However, the directories listed here @@ -29,31 +19,10 @@ TESTSUITE?= FreeBSD # SUBDIR for directories that may contain helper binaries and/or data files. TESTS_SUBDIRS?= -# Knob to control the handling of the Kyuafile for this Makefile. -# -# If 'yes', a Kyuafile exists in the source tree and is installed into -# TESTSDIR. -# -# If 'auto', a Kyuafile is automatically generated based on the list of test -# programs built by the Makefile and is installed into TESTSDIR. This is the -# default and is sufficient in the majority of the cases. -# -# If 'no', no Kyuafile is installed. -KYUAFILE?= auto - -# Per-test program interface definition. -# -# The name provided here must match one of the interface names supported by -# Kyua as this is later encoded in the Kyuafile test program definitions. -#TEST_INTERFACE.= interface-name - -# Per-test program metadata properties as a list of key/value pairs. -# -# All the variables for a particular program are appended to the program's -# definition in the Kyuafile. This feature can be used to avoid having to -# explicitly supply a Kyuafile in the source directory, allowing the caller -# Makefile to rely on the KYUAFILE=auto behavior defined here. -#TEST_METADATA.+= key="value" +# If defined, indicates that the tests built by the Makefile are not part of +# the FreeBSD Test Suite. The implication of this is that the tests won't be +# installed under /usr/tests/ and that Kyua won't be able to run them. +#NOT_FOR_TEST_SUITE= # List of variables to pass to the tests at run-time via the environment. TESTS_ENV?= @@ -67,17 +36,14 @@ TESTS_ENV+= PATH=${TESTS_PATH:tW:C/ +/:/ TESTS_LD_LIBRARY_PATH+= ${DESTDIR}/lib ${DESTDIR}/usr/lib TESTS_ENV+= LD_LIBRARY_PATH=${TESTS_LD_LIBRARY_PATH:tW:C/ +/:/g} -# List of all tests being built. This variable is internal should not be -# defined by the Makefile. The various *.test.mk modules extend this variable -# as needed. -_TESTS?= - -# Path to the prefix of the installed Kyua CLI, if any. -# -# If kyua is installed from ports, we automatically define a realtest target -# below to run the tests using this tool. The tools are searched for in the -# hierarchy specified by this variable. -KYUA_PREFIX?= /usr/local +# List of all tests being built. The various *.test.mk modules extend this +# variable as needed. +_TESTS= + +# Pull in the definitions of all supported test interfaces. +.include +.include +.include .if !empty(TESTS_SUBDIRS) SUBDIR+= ${TESTS_SUBDIRS} @@ -93,82 +59,9 @@ WITHOUT_MAN=yes PROG_VARS+= BINDIR PROGS_TARGETS+= install -.if ${KYUAFILE:tl} == "yes" -FILES+= Kyuafile -FILESDIR_Kyuafile= ${TESTSDIR} - -CLEANFILES+= Kyuafile.auto Kyuafile.auto.tmp -.elif ${KYUAFILE:tl} == "auto" -FILES+= Kyuafile.auto -FILESDIR_Kyuafile.auto= ${TESTSDIR} -FILESNAME_Kyuafile.auto= Kyuafile - -CLEANFILES+= Kyuafile.auto Kyuafile.auto.tmp - -.NOPATH: Kyuafile.auto -Kyuafile.auto: Makefile - @{ \ - echo '-- Automatically generated by bsd.test.mk.'; \ - echo; \ - echo 'syntax(2)'; \ - echo; \ - echo 'test_suite("${TESTSUITE}")'; \ - echo; \ - } >Kyuafile.auto.tmp -.for _T in ${_TESTS} - @echo '${TEST_INTERFACE.${_T}}_test_program{name="${_T}"${TEST_METADATA.${_T}:C/$/,/:tW:C/^/, /W:C/,$//W}}' \ - >>Kyuafile.auto.tmp -.endfor -.for _T in ${TESTS_SUBDIRS:N.WAIT} - @echo "include(\"${_T}/Kyuafile\")" >>Kyuafile.auto.tmp -.endfor - @mv Kyuafile.auto.tmp Kyuafile.auto -.endif - -KYUA?= ${KYUA_PREFIX}/bin/kyua -.if exists(${KYUA}) -# Definition of the "make test" target and supporting variables. -# -# This target, by necessity, can only work for native builds (i.e. a FreeBSD -# host building a release for the same system). The target runs Kyua, which is -# not in the toolchain, and the tests execute code built for the target host. -# -# Due to the dependencies of the binaries built by the source tree and how they -# are used by tests, it is highly possible for a execution of "make test" to -# report bogus results unless the new binaries are put in place. -realtest: .PHONY - @echo "*** WARNING: make test is experimental" - @echo "***" - @echo "*** Using this test does not preclude you from running the tests" - @echo "*** installed in ${TESTSBASE}. This test run may raise false" - @echo "*** positives and/or false negatives." - @echo - @set -e; \ - ${KYUA} test -k ${DESTDIR}${TESTSDIR}/Kyuafile; \ - result=0; \ - echo; \ - echo "*** Once again, note that "make test" is unsupported."; \ - test $${result} -eq 0 -.endif - -beforetest: .PHONY -.if defined(TESTSDIR) -.if ${TESTSDIR} == ${TESTSBASE} -# Forbid running from ${TESTSBASE}. It can cause false positives/negatives and -# it does not cover all the tests (e.g. it misses testing software in external). - @echo "*** Sorry, you cannot use make test from src/tests. Install the" - @echo "*** tests into their final location and run them from ${TESTSBASE}" - @false -.else - @echo "*** Using this test does not preclude you from running the tests" - @echo "*** installed in ${TESTSBASE}. This test run may raise false" - @echo "*** positives and/or false negatives." -.endif -.else - @echo "*** No TESTSDIR defined; nothing to do." - @false +.if !defined(NOT_FOR_TEST_SUITE) +.include .endif - @echo .if !target(realtest) realtest: .PHONY Modified: stable/10/share/mk/plain.test.mk ============================================================================== --- stable/10/share/mk/plain.test.mk Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/share/mk/plain.test.mk Mon Apr 14 23:51:57 2014 (r264483) @@ -1,10 +1,14 @@ # $FreeBSD$ # +# You must include bsd.test.mk instead of this file from your Makefile. +# # Logic to build and install plain test programs. A plain test programs it not # supposed to use any specific testing framework: all it does is run some code # and report the test's pass or fail status via a 0 or 1 exit code. -.include +.if !target(____) +.error plain.test.mk cannot be included directly. +.endif # List of C, C++ and shell test programs to build. # @@ -58,5 +62,3 @@ ${_T}: ${PLAIN_TESTS_SH_SRC_${_T}} mv ${.TARGET}.tmp ${.TARGET} .endfor .endif - -.include Copied: stable/10/share/mk/suite.test.mk (from r263172, head/share/mk/suite.test.mk) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/10/share/mk/suite.test.mk Mon Apr 14 23:51:57 2014 (r264483, copy of r263172, head/share/mk/suite.test.mk) @@ -0,0 +1,127 @@ +# $FreeBSD$ +# +# You must include bsd.test.mk instead of this file from your Makefile. +# +# Internal glue for the build of /usr/tests/. + +.if !target(____) +.error suite.test.mk cannot be included directly. +.endif + +# Directory in which to install tests defined by the current Makefile. +# Makefiles have to override this to point to a subdirectory of TESTSBASE. +TESTSDIR?= . + +# Name of the test suite these tests belong to. Should rarely be changed for +# Makefiles built into the FreeBSD src tree. +TESTSUITE?= FreeBSD + +# Knob to control the handling of the Kyuafile for this Makefile. +# +# If 'yes', a Kyuafile exists in the source tree and is installed into +# TESTSDIR. +# +# If 'auto', a Kyuafile is automatically generated based on the list of test +# programs built by the Makefile and is installed into TESTSDIR. This is the +# default and is sufficient in the majority of the cases. +# +# If 'no', no Kyuafile is installed. +KYUAFILE?= auto + +# Per-test program interface definition. +# +# The name provided here must match one of the interface names supported by +# Kyua as this is later encoded in the Kyuafile test program definitions. +#TEST_INTERFACE.= interface-name + +# Per-test program metadata properties as a list of key/value pairs. +# +# All the variables for a particular program are appended to the program's +# definition in the Kyuafile. This feature can be used to avoid having to +# explicitly supply a Kyuafile in the source directory, allowing the caller +# Makefile to rely on the KYUAFILE=auto behavior defined here. +#TEST_METADATA.+= key="value" + +# Path to the prefix of the installed Kyua CLI, if any. +# +# If kyua is installed from ports, we automatically define a realtest target +# below to run the tests using this tool. The tools are searched for in the +# hierarchy specified by this variable. +KYUA_PREFIX?= /usr/local + +.if ${KYUAFILE:tl} == "yes" +FILES+= Kyuafile +FILESDIR_Kyuafile= ${TESTSDIR} + +CLEANFILES+= Kyuafile.auto Kyuafile.auto.tmp +.elif ${KYUAFILE:tl} == "auto" +FILES+= Kyuafile.auto +FILESDIR_Kyuafile.auto= ${TESTSDIR} +FILESNAME_Kyuafile.auto= Kyuafile + +CLEANFILES+= Kyuafile.auto Kyuafile.auto.tmp + +.NOPATH: Kyuafile.auto +Kyuafile.auto: Makefile + @{ \ + echo '-- Automatically generated by bsd.test.mk.'; \ + echo; \ + echo 'syntax(2)'; \ + echo; \ + echo 'test_suite("${TESTSUITE}")'; \ + echo; \ + } >Kyuafile.auto.tmp +.for _T in ${_TESTS} + @echo '${TEST_INTERFACE.${_T}}_test_program{name="${_T}"${TEST_METADATA.${_T}:C/$/,/:tW:C/^/, /W:C/,$//W}}' \ + >>Kyuafile.auto.tmp +.endfor +.for _T in ${TESTS_SUBDIRS:N.WAIT} + @echo "include(\"${_T}/Kyuafile\")" >>Kyuafile.auto.tmp +.endfor + @mv Kyuafile.auto.tmp Kyuafile.auto +.endif + +KYUA?= ${KYUA_PREFIX}/bin/kyua +.if exists(${KYUA}) +# Definition of the "make test" target and supporting variables. +# +# This target, by necessity, can only work for native builds (i.e. a FreeBSD +# host building a release for the same system). The target runs Kyua, which is +# not in the toolchain, and the tests execute code built for the target host. +# +# Due to the dependencies of the binaries built by the source tree and how they +# are used by tests, it is highly possible for a execution of "make test" to +# report bogus results unless the new binaries are put in place. +realtest: .PHONY + @echo "*** WARNING: make test is experimental" + @echo "***" + @echo "*** Using this test does not preclude you from running the tests" + @echo "*** installed in ${TESTSBASE}. This test run may raise false" + @echo "*** positives and/or false negatives." + @echo + @set -e; \ + ${KYUA} test -k ${DESTDIR}${TESTSDIR}/Kyuafile; \ + result=0; \ + echo; \ + echo "*** Once again, note that "make test" is unsupported."; \ + test $${result} -eq 0 +.endif + +beforetest: .PHONY +.if defined(TESTSDIR) +.if ${TESTSDIR} == ${TESTSBASE} +# Forbid running from ${TESTSBASE}. It can cause false positives/negatives and +# it does not cover all the tests (e.g. it misses testing software in external). + @echo "*** Sorry, you cannot use make test from src/tests. Install the" + @echo "*** tests into their final location and run them from ${TESTSBASE}" + @false +.else + @echo "*** Using this test does not preclude you from running the tests" + @echo "*** installed in ${TESTSBASE}. This test run may raise false" + @echo "*** positives and/or false negatives." +.endif +.else + @echo "*** No TESTSDIR defined; nothing to do." + @false +.endif + @echo Modified: stable/10/share/mk/tap.test.mk ============================================================================== --- stable/10/share/mk/tap.test.mk Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/share/mk/tap.test.mk Mon Apr 14 23:51:57 2014 (r264483) @@ -1,12 +1,16 @@ # $FreeBSD$ # +# You must include bsd.test.mk instead of this file from your Makefile. +# # Logic to build and install TAP-compliant test programs. # # This is provided to support existing tests in the FreeBSD source tree # (particularly those coming from tools/regression/) that comply with the # Test Anything Protocol. It should not be used for new tests. -.include +.if !target(____) +.error tap.test.mk cannot be included directly. +.endif # List of C, C++ and shell test programs to build. # @@ -87,5 +91,3 @@ ${_T}: ${TAP_TESTS_SH_SRC_${_T}} mv ${.TARGET}.tmp ${.TARGET} .endfor .endif - -.include Modified: stable/10/tests/sys/kern/Makefile ============================================================================== --- stable/10/tests/sys/kern/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/tests/sys/kern/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -9,4 +9,4 @@ LDADD.unix_seqpacket_test+= -lpthread WARNS?= 5 -.include +.include Modified: stable/10/usr.bin/atf/atf-sh/tests/Makefile ============================================================================== --- stable/10/usr.bin/atf/atf-sh/tests/Makefile Mon Apr 14 22:49:37 2014 (r264482) +++ stable/10/usr.bin/atf/atf-sh/tests/Makefile Mon Apr 14 23:51:57 2014 (r264483) @@ -23,4 +23,4 @@ misc_helpers: misc_helpers.sh chmod +x ${.TARGET}.tmp mv ${.TARGET}.tmp ${.TARGET} -.include +.include