From owner-freebsd-stable Fri Oct 18 2:42:56 2002 Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4DED437B401 for ; Fri, 18 Oct 2002 02:42:55 -0700 (PDT) Received: from clever.eusc.inter.net (clever.eusc.inter.net [213.73.101.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3ED4443E77 for ; Fri, 18 Oct 2002 02:42:54 -0700 (PDT) (envelope-from plexus@snafu.de) Received: from cerberus.publity.de ([213.61.128.135] helo=snafu.de) by clever.eusc.inter.net with asmtp (Exim 3.36 #4) id 182Ted-00073M-00; Fri, 18 Oct 2002 11:42:52 +0200 Message-ID: <3DAFD77F.7080902@snafu.de> Date: Fri, 18 Oct 2002 11:42:23 +0200 From: "Fischer, Oliver" User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.1) Gecko/20021015 X-Accept-Language: de, en-us, en MIME-Version: 1.0 To: Chris BeHanna Cc: FreeBSD-Stable Subject: Re: freebsd test matrix References: <20021018034039.F1275-100000@topperwein.dyndns.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Chris BeHanna wrote: > Ideally, each developer will have a set of unit tests for what > (s)he is working on, that could, with some massaging, be plugged into > a larger harness to do nightly regression testing (perhaps via > tinderbox). The trouble is keeping the tests up-to-date. Invariably, So the question is, who is writting the tests. In the company I am working in, each developer writes its own unittests ans has to run the whole testsuite every day. In the company of a friend of mine, the developers writes their own tests and the external QA writes tests too. The QA tests the public behavior the developers concentrate on the internals of the system. Here we have one unbreakable rule. No release if a single test fails. > what the tests are testing ends up changing underneath them, leading > to spurious failures in the test report. Ok this is true. But at least you will see that something is 'broken' from the view of the tests. I thing it is very important to have a system to collect the reports and see which test fails where. Then you can decide if the test is right, it is outdated or simply a dependency.... Bye Oliver To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message