From owner-freebsd-stable Wed Oct 16 21:53:55 2002 Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2ABC237B401 for ; Wed, 16 Oct 2002 21:53:54 -0700 (PDT) Received: from topperwein.dyndns.org (acs-24-154-51-184.zoominternet.net [24.154.51.184]) by mx1.FreeBSD.org (Postfix) with ESMTP id 57B0343EAA for ; Wed, 16 Oct 2002 21:53:53 -0700 (PDT) (envelope-from behanna@zbzoom.net) Received: from [192.168.168.10] ([192.168.168.10]) by topperwein.dyndns.org (8.12.6/8.12.5) with ESMTP id g9H4rq3x001007 for ; Thu, 17 Oct 2002 00:53:52 -0400 (EDT) (envelope-from behanna@zbzoom.net) Date: Thu, 17 Oct 2002 00:53:47 -0400 (EDT) From: Chris BeHanna Reply-To: Chris BeHanna To: FreeBSD-Stable Subject: Re: freebsd test matrix In-Reply-To: <20021017005815.GA27918@panix.com> Message-ID: <20021017005151.T771-100000@topperwein.dyndns.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Oct 2002, David Kleiner wrote: > On Wed, Oct 16, 2002 at 11:03:21PM +0200, Fischer, Oliver wrote: > > Mike Hoskins wrote: > > >Hmm, I wonder if something similar is used or was developed for internal > > >QA? Anyone? It seems like there should at least be a "best practice" for > > >testing systems... How do we currently make a -RELEASE with confidence? > > >Of course you can't verify a given release builds on every platform, but > > >is there an automated means of verifying system compoents work and > > >interoperate properly on a given test system or set of systems? > > > > http://martinfowler.com/articles/continuousIntegration.html > > In a commercial world, there is a lot of pre-integration QA testing > going on - question is, where do you start? What test harness you > want to use? Which test suites? Are you interested in standards > (posix and such), ABI, stress testing, library testing, fs testing, > interoperability and so on? All of the above, in bite-sized, doable chunks. I'm currently doing QA on a (very) large software project, and all of those things are important. Some of the testing uses existing industry test suites and benchmarking tools, and some of it (much of it) is custom. Being able to compile, install, and boot is just the tip of the iceberg. -- Chris BeHanna http://www.pennasoft.com Principal Consultant PennaSoft Corporation chris@pennasoft.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message