Date: Mon, 11 Feb 2013 12:43:17 +0100 From: Erik Cederstrand <erik@cederstrand.dk> To: Erich Dollansky <erichsfreebsdlist@alogt.com> Cc: freebsd-current@freebsd.org, Steve Kargl <sgk@troutmask.apl.washington.edu> Subject: Re: 7+ days of dogfood Message-ID: <88F312FE-E783-4C24-8964-BBDE4DE6653E@cederstrand.dk> In-Reply-To: <20130211063858.0375a6ed@X220.ovitrap.com> References: <20130210000723.GA73630@troutmask.apl.washington.edu> <20130210083010.7df5b997@X220.ovitrap.com> <256C7A7B-DFA7-4C0F-B389-AB10E0DA42D0@cederstrand.dk> <20130211063858.0375a6ed@X220.ovitrap.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Erich, Den 11/02/2013 kl. 00.38 skrev Erich Dollansky = <erichsfreebsdlist@alogt.com>: > On Sun, 10 Feb 2013 15:57:01 +0100 > Erik Cederstrand <erik@cederstrand.dk> wrote: >=20 >> And as long as there is no automatic can taster doing quality >> assurance of the produced cans, then foul cans will go unnoticed >> until a dog pukes all over the carpet :-) >>=20 > Isn't this the idea of HEAD? It's certainly not the idea of HEAD that everyone should experience the = same bugs, compile errors, runtime errors and even have old bugs pop up = again repeatedly. It may be the consequence of running HEAD, but = certainly not the idea. >> For this to change, we really need to catch up on years of neglect in >> e.g. src/tools/regression/. I really applaud the people doing the >> thankless job of changing this. >>=20 > I do not believe that this all can be automated. I'm not saying that testing is all-or-nothing. OS testing is not easy, = and many tests are impractical or expensive if they require real = hardware in complicated setups. How do you reliably test an IEEE 802.11s = mesh implementation? Or scheduling on huge servers that are too = expensive to purchase? I think that is one of the reasons that FreeBSD = has not caught up on automated testing and continuous integration. But = regression tests are useful even though they don't give 100% code = coverage. Currently, you can't even "make test" in src/tools/regression/ = and run the tests that are there. Apart from the compile-tests done by = the tinderboxes, I'm not aware of any coordinated effort to = systematically do runtime or even performance testing of FreeBSD. Thanks, Erik=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?88F312FE-E783-4C24-8964-BBDE4DE6653E>