Date: Fri, 18 Oct 2002 13:50:07 -0700 (PDT) From: Mike Hoskins <mike@adept.org> To: "Fischer, Oliver" <plexus@snafu.de> Cc: Chris BeHanna <behanna@zbzoom.net>, FreeBSD-Stable <stable@freebsd.org> Subject: Re: freebsd test matrix Message-ID: <20021018134020.D8827-100000@fubar.adept.org> In-Reply-To: <3DAFD77F.7080902@snafu.de>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 18 Oct 2002, Fischer, Oliver wrote: > So the question is, who is writting the tests. In the company I am > working in, each developer writes its own unittests ans has to run the > whole testsuite every day. In the company of a friend of mine, the > developers writes their own tests and the external QA writes tests too. > The QA tests the public behavior the developers concentrate on the > internals of the system. Internally we try to follow most of the principles outlined in the "continuous integration" URL that was posted earlier. Clearly, this process indicates developers should write test cases for their code... Unfortuneately, the last thing an opensource project needs is more work for the developers. :) > > what the tests are testing ends up changing underneath them, leading > > to spurious failures in the test report. > Ok this is true. But at least you will see that something is 'broken' > from the view of the tests. I thing it is very important to have a > system to collect the reports and see which test fails where. Then you > can decide if the test is right, it is outdated or simply a dependency.... If a standardized means of tying the harness to the underlying tests is developed, developers update tests when they update code (as the URL outlines, development is essentially writing code to pass tests) and this isn't too much of an issue. Of course it would be nice to someday have a fully automated, robust test/stress setup for builds... That will involve a lot of work from build/QA people and developers. So, in the meantime, what can we do to provide a resource that is immediately available, and serves to gather initiative for future projects? Would a site that simply let people post hardware specs, kernel configs, dmesg output, debugging info, etc. be useful? I think maintaining a searchable, user-supported database of build results would be a useful first step toward something much larger. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20021018134020.D8827-100000>