From owner-freebsd-current@FreeBSD.ORG Mon Sep 28 01:50:51 2009 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AB0BB106566C for ; Mon, 28 Sep 2009 01:50:51 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (muon.cran.org.uk [66.246.138.153]) by mx1.freebsd.org (Postfix) with ESMTP id 844CF8FC13 for ; Mon, 28 Sep 2009 01:50:51 +0000 (UTC) Received: from unknown (87-194-158-129.bethere.co.uk [87.194.158.129]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id 9988D81D0 for ; Mon, 28 Sep 2009 01:50:50 +0000 (UTC) Date: Mon, 28 Sep 2009 02:49:37 +0100 From: Bruce Cran To: current@freebsd.org Message-ID: <20090928024937.00000d1b@unknown> X-Mailer: Claws Mail 3.7.2cvs15 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Subject: Fw: Call for regression and performance testing - 8.0 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Sep 2009 01:50:51 -0000 [forwarding to the list] Begin forwarded message: Date: Mon, 28 Sep 2009 01:32:36 +0000 From: Bruce Cran To: Robert Watson Cc: Bruce Cran , "Arno J. Klaassen" , current@freebsd.org Subject: Re: Call for regression and performance testing - 8.0 On Tue, Sep 22, 2009 at 01:01:34PM +0100, Robert Watson wrote: > On Tue, 22 Sep 2009, Bruce Cran wrote: > >>> Resurfacing after organising conferences, and discovered I had >>> missed this e-mail. I'm not aware of any such efforts, and it would >>> be really good to do. Making it easy to do "runtests" would vastly >>> improve the ease of testing during the release process. Right now >>> the output of the tests is fairly inconsistent, and this is >>> probably to a large extent my fault. It would be nice to have a >>> "test template" program that people can start with when writing new >>> regression tests -- I know I would find that valuable. >> >> There's already a limited template available: >> src/tools/regression/README specifies the format to use, and many of >> the regression tests do use it. You can run all the tests which use >> the specified format by running "prove -r" (after installing >> devel/p5-Test-Harness), but there are currently quite a few failures. >> >> Some tests don't use the template for good reason - for example >> src/tools/regression/redzone9 depends on the DEBUG_REDZONE kernel >> option to be present. > > However, others don't do it for a good reason -- perhaps mostly my > tests? :-) > > If people want to convert them over that would be excellent. I've updated the PR kern/138465 with a patch to convert the sockets/sendfile test to produce TAP output. It also fixes a bug where it called sendfile with a length of 0 and didn't expect to receive the whole buffer. Are there any examples of producing TAP output in a client/server application, and avoiding bailing when an error occurs? It seems quite a few tests use fork, and any output in the child process isn't seen by 'prove' so any errors need marshalled back to the main thread to print the "not ok" message. Should they be converted to use pthreads? -- Bruce Cran