From owner-freebsd-current@FreeBSD.ORG Tue Sep 22 11:36:47 2009 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 089951065672 for ; Tue, 22 Sep 2009 11:36:47 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (muon.cran.org.uk [66.246.138.153]) by mx1.freebsd.org (Postfix) with ESMTP id C6B2A8FC12 for ; Tue, 22 Sep 2009 11:36:46 +0000 (UTC) Received: from unknown (87-194-158-129.bethere.co.uk [87.194.158.129]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id 7E48E81C5; Tue, 22 Sep 2009 11:36:45 +0000 (UTC) Date: Tue, 22 Sep 2009 12:36:47 +0100 From: Bruce Cran To: "Robert N. M. Watson" Message-ID: <20090922123647.000078e5@unknown> In-Reply-To: References: X-Mailer: Claws Mail 3.7.1 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: "Arno J. Klaassen" , current@freebsd.org Subject: Re: Call for regression and performance testing - 8.0 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2009 11:36:47 -0000 On Tue, 22 Sep 2009 11:30:27 +0100 "Robert N. M. Watson" wrote: > > On 13 Aug 2009, at 21:55, Arno J. Klaassen wrote: > > > Robert Watson writes: > > > >>> [ ... ] > >> Functional testing > >> > >> We actually have a sizable regression test suite in src/tools/ > >> regression -- > > > > is anyone aware of some rudimentary efforts to set up an overall > > "make && ./runtests" environment? > > Resurfacing after organising conferences, and discovered I had > missed this e-mail. I'm not aware of any such efforts, and it would > be really good to do. Making it easy to do "runtests" would vastly > improve the ease of testing during the release process. Right now the > output of the tests is fairly inconsistent, and this is probably to a > large extent my fault. It would be nice to have a "test template" > program that people can start with when writing new regression tests > -- I know I would find that valuable. There's already a limited template available: src/tools/regression/README specifies the format to use, and many of the regression tests do use it. You can run all the tests which use the specified format by running "prove -r" (after installing devel/p5-Test-Harness), but there are currently quite a few failures. Some tests don't use the template for good reason - for example src/tools/regression/redzone9 depends on the DEBUG_REDZONE kernel option to be present. -- Bruce Cran