Date: Sat, 11 Dec 2004 14:28:11 +0000 From: Nik Clayton <nik@freebsd.org> To: Giorgos Keramidas <keramida@ceid.upatras.gr> Cc: Nik Clayton <nik@freebsd.org> Subject: Re: Testing the build infrastructure Message-ID: <20041211142811.GB34046@clan.nothing-going-on.org> In-Reply-To: <20041210035853.GA38277@gothmog.gr> References: <20041207221810.GC79404@clan.nothing-going-on.org> <20041210035853.GA38277@gothmog.gr>
next in thread | previous in thread | raw e-mail | index | archive | help
--2B/JsCI69OhZNC5r
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Fri, Dec 10, 2004 at 05:58:53AM +0200, Giorgos Keramidas wrote:
> On 2004-12-07 22:18, Nik Clayton <nik@freebsd.org> wrote:
> > Guys,
> > Given the comments that have been bandied about recently relating to
> > breakage of various parts of the build infrastructure, and functionality
> > that used to work no longer working, I'd like to make some concrete
> > progress towards fixing things. [...]
> >
> > Thoughts?
>=20
> That looks like a good plan :)
>=20
> Any details on how the tests should be written? Can we (re)use the
> style of the tests of src/ at all?
Ideally that's exactly what we'll do. I'm a big fan of prove(1), which
will run tests and generate a report of which succeeded, which failed,
and so on. The output format that tests need to generate is listed in
src/tools/regression/README.
It's easy to write tests in Perl and /bin/sh, and I'm working on a C
implementation. I strongly suspect that most of our tests will be
written in /bin/sh -- a lot of them will be:
* Did this file get created?
* Does the output match the canonical output which we've stored in
/this/file?
* Was the exit code the one we expected?
For example, I don't (because I've not looked in to it in any detail)
understand exactly what's broken in the obj/ building case. But with
tests that describe how things are supposed to work I don't need to.
All I need to do (prior to making a commit) is run the tests. As long
as they all pass everything's fine.
The only extra work (after this is in place) is when we make
infrastructure changes. There need to be corresponding tests in place.
So, when I committed the .sgml -> .xml conversion code, if I'd committed
a test for it as well, we'd know exactly when it broke, and why.
N
--=20
FreeBSD: The Power to Serve http://www.freebsd.org/ (__)
FreeBSD Documentation Project http://www.freebsd.org/docproj/ \\\'',)
\/ \=
^
--- 15B8 3FFC DDB4 34B0 AA5F 94B7 93A8 0764 2C37 E375 --- .\._/=
_)
--2B/JsCI69OhZNC5r
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (FreeBSD)
iD8DBQFBuwP6k6gHZCw343URAmZXAJ9QL2PBVcx307zPx0XfXJBt7eOBNgCghgYv
r3KjNAY9p+XETC9ob5DtUsc=
=9MXw
-----END PGP SIGNATURE-----
--2B/JsCI69OhZNC5r--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20041211142811.GB34046>