Date: Tue, 29 Mar 2005 19:42:37 +0100 From: Peter Sewell <Peter.Sewell@cl.cam.ac.uk> To: Brooks Davis <brooks@one-eyed-alien.net> Cc: freebsd-net@freebsd.org Subject: Re: Rigorous specification for TCP, UDP, and Sockets Message-ID: <E1DGLfi-0001Xt-00@mta1.cl.cam.ac.uk> In-Reply-To: Message from Brooks Davis <brooks@one-eyed-alien.net> <20050329172338.GA5995@odin.ac.hmc.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
>> >Are you plannning to share the tools as >> >well? That's what I'd be most interested in seeing, basically the >> >ability to turn your conformance tests into regression tests. >> >> It's unclear at the moment - we'd certainly like to make that >> regression testing more routine, but the current tools are >> >non-trivial >> to drive. We'll be looking to see how much interest there is, and >> also how accessible the spec is, before going further. > >In what way are they hard to drive? Are they difficult to set up, or >difficult to run? both, I'm afraid - there's quite a complex infrastructure, both for generating tests and for running the checker over them (in parallel on a server farm), then interpreting the results of the checker is non-trivial. One needs to iterate the test generation/checking/spec-fixing loop many times. >If they are difficult to set up, you might take a >look at using EmuLab (www.emulab.net) to build images with configured >tests so OSes just need to be updated to test a change. Interesting - I didn't know about that (though I don't think it helps right now). >> What kind of regression testing is in use now? > >None for the most part. People test some limited things, but I don't >think anyone does the kind if rigorous testing we should be doing. ah :-( cheers, Peter
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E1DGLfi-0001Xt-00>