From owner-freebsd-stable Wed Oct 16 13:32:56 2002 Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6AAA937B401 for ; Wed, 16 Oct 2002 13:32:53 -0700 (PDT) Received: from fubar.adept.org (fubar.adept.org [63.147.172.249]) by mx1.FreeBSD.org (Postfix) with ESMTP id 42EC543E6A for ; Wed, 16 Oct 2002 13:32:53 -0700 (PDT) (envelope-from mike@adept.org) Received: by fubar.adept.org (Postfix, from userid 1001) id A541D15247; Wed, 16 Oct 2002 13:32:42 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by fubar.adept.org (Postfix) with ESMTP id A301B15226; Wed, 16 Oct 2002 13:32:42 -0700 (PDT) Date: Wed, 16 Oct 2002 13:32:42 -0700 (PDT) From: Mike Hoskins To: "Fischer, Oliver" Cc: stable@freebsd.org Subject: Re: freebsd test matrix In-Reply-To: <3DADC827.1000100@snafu.de> Message-ID: <20021016131517.C4295-100000@fubar.adept.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Oct 2002, Fischer, Oliver wrote: > For an operating system it isn't enough only to check if the sources > compile. You would like to know if all the tools which are close to the > hardware work properly. Or you would like to know if the machine can > boot. All this issues aren't addressed by tinderbox. I agree. Although, if a given system can build and install both world and a kernel, and boot without issues... Just knowing that can "verify" many components of the system (gcc, awk, etc. are used in a build). While not verifying all (most/boundary) cases as a test harness would attempt to do, it's better than nothing. :) > Indeed you could have suites with tests for tools as the c compiler, awk > and so on and so on. Hmm, I wonder if something similar is used or was developed for internal QA? Anyone? It seems like there should at least be a "best practice" for testing systems... How do we currently make a -RELEASE with confidence? Of course you can't verify a given release builds on every platform, but is there an automated means of verifying system compoents work and interoperate properly on a given test system or set of systems? > But if are interested in seeting up such a beast (tinderbox) for freebsd > then drop me a line. I would like to take part in this. I am interested in contributing in anyway I can. If people would use this, I can host it. FreeBSD 4.7-STABLE #3: Mon Oct 14 18:40:49 PDT 2002 CPU: Pentium III/Pentium III Xeon/Celeron (1258.22-MHz 686-class CPU) real memory = 2147418112 (2097088K bytes) Currently doing very little - quite the waste. ;) So I have a box, bandwidth, and a volunteer... We need to agree on the best way to test (I.e. get the same output collected in the same way from all submitters), and create/find intuitive submission tools as you pointed out. I'm reading up on tinderbox now. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message