From owner-freebsd-current Sun May 11 14:21:08 1997 Return-Path: Received: (from root@localhost) by hub.freebsd.org (8.8.5/8.8.5) id OAA13306 for current-outgoing; Sun, 11 May 1997 14:21:08 -0700 (PDT) Received: from bitbox.follo.net (bitbox.follo.net [194.198.43.36]) by hub.freebsd.org (8.8.5/8.8.5) with ESMTP id OAA13301 for ; Sun, 11 May 1997 14:21:05 -0700 (PDT) Received: (from eivind@localhost) by bitbox.follo.net (8.7.6/8.7.3) id XAA09282; Sun, 11 May 1997 23:19:22 +0200 (MET DST) Date: Sun, 11 May 1997 23:19:22 +0200 (MET DST) Message-Id: <199705112119.XAA09282@bitbox.follo.net> From: Eivind Eklund To: Terry Lambert CC: perhaps@yes.no, current@FreeBSD.ORG In-reply-to: Terry Lambert's message of Sun, 11 May 1997 11:57:24 -0700 (MST) Subject: Re: Regression tests (was Re: A 3.0-current SNAP building machine has been found!) References: <199705111709.TAA06018@bitbox.follo.net> <199705111857.LAA05834@phaeton.artisoft.com> Sender: owner-current@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > > > > I was actually against using "voting"; I prefer forcing developers to > > > verify that compiles work and the resulting code does not trivially > > > fail prior to it being committed. That way the tree would always work; > > > the best you can get out of "voting" is "provided with high confidence > > > that it might work". > > > > Regression tests. We really should have this integrated into the > > build system. > > This is scary, in that it starts requiring developers to have "big iron". > > The POSIX tests (for example) take a while to run. I don't believe everybody should run the full regression tests each time. Full regression tests are expensive. However, if you e.g. do a modification in dump/restore, one can run the regression tests for dump/restore. You'd be much more certain of not having broken anything than you are today; a full 'make regress' could be done on need (whenever something seems to break, or if you believe there is a chance you've broken something else.) We've already got a limited form of this - 'make world' - but explicitly supporting regression would likely make it better. > > Anyway; being able to do a 'make regress' on the entire FreeBSD > > source tree and know all new code and a lot of old code got tested > > would be a _good_ feeling. > > Yes, very much so! I would limit its application, though; if the > CVS went "split tree with snapshot", then the process could be > automated, but it's probably too much overhead otherwise. 8-(. You're thinking of disk space requirements? I wouldn't worry too much. The regression suite take time to write, and I consider it unlikely that it will grow faster than price of disk space drop :-) For large test files, we could solve part of the problem by making them available through ftp, with an automated download system a la ports. However, as I said, I don't believe I've got The Solution to the problem; I just feel that a good solution to this would be a very nice addition to the build structure. Eivind.