Date: Wed, 5 Nov 2014 18:45:48 -0500 From: Julio Merino <jmmv@freebsd.org> To: Craig Rodrigues <rodrigc@freebsd.org> Cc: "freebsd-testing@freebsd.org" <freebsd-testing@freebsd.org> Subject: Re: Kyua/ATF as a test framework discussion Message-ID: <CAFY7cWCHq4RM5qw0vsPDVjZj65=m6YGhwS5sx-hFCBtwAAiMqg@mail.gmail.com> In-Reply-To: <CAG=rPVeQbv0kiw-f1NO1LbQ4hmYnuPfVQbkWJYnx1eMgS2W1XA@mail.gmail.com> References: <C3091631-0E35-40F4-BEF0-12AB68EF6B97@gmail.com> <CAG=rPVf3YhTc3GmZmjjRAEdJ9HC9nF7QA%2BEjHjEr_XO=dJAOfg@mail.gmail.com> <CAFY7cWBusbyXNK0Sq-LRa1X5qieMxY4SsNaADwpGa5p4hg%2B5Zw@mail.gmail.com> <CAG=rPVeQbv0kiw-f1NO1LbQ4hmYnuPfVQbkWJYnx1eMgS2W1XA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Nov 5, 2014 at 5:02 PM, Craig Rodrigues <rodrigc@freebsd.org> wrote: > > > On Wed, Nov 5, 2014 at 1:41 PM, Julio Merino <jmmv@freebsd.org> wrote: >> >> On Tue, Nov 4, 2014 at 2:40 PM, Craig Rodrigues <rodrigc@freebsd.org> >> wrote: >> > Having hooks in the test framework to do setup/teardown like in other >> > unit >> > test frameworks would be useful. >> >> Please tell me of a test framework that has setup/teardown >> functionality outside of the test program. I would like to see how >> they handle it. > > http://nose.readthedocs.org/en/latest/writing_tests.html#test-functions > https://docs.python.org/3/library/unittest.html#unittest.TestCase.setUp > http://cunit.sourceforge.net/doc/managing_tests.html > http://junit.sourceforge.net/junit3.8.1/javadoc/junit/framework/TestCase.html#method_summary All of these examples point at *in-program* setup/teardown methods, not what you were describing a while ago with hooks in the Kyuafile. Adding per-test case setup/teardown methods to ATF would be easy, but mostly pointless except for maybe in atf-c++ (but I don't think anyone cares about atf-c++). Adding per-test program setup/teardown methods to ATF is interesting, but difficult due to the semantics of the interface of the test programs. Also, the resulting API would be quite hard to handle and understand from a programmer's perspective... See my other mail in this thread. Basically, I believe the current ATF interface is not worth fixing to support these cases and we should look at other existing, unit-testing libraries to see if they fit our needs. Kyua should be the glue to bring them together. In fact, you should be able to use all the libraries you mentioned with Kyua already (or at least that's the goal); and, if not, that's a bug that we should fix.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFY7cWCHq4RM5qw0vsPDVjZj65=m6YGhwS5sx-hFCBtwAAiMqg>