Date: Wed, 27 Nov 2013 08:56:25 -0500 From: Julio Merino <julio@meroh.net> To: kyua-discuss@googlegroups.com Cc: "freebsd-testing@freebsd.org" <freebsd-testing@freebsd.org> Subject: Re: Python unittest backend for Kyua Message-ID: <CADyfeQU64LnutLBOZv1n-Wet_byMFWxSAyh-S-LaAFebyuq1_g@mail.gmail.com> In-Reply-To: <CAOtMX2g35ybAzFHriVuQqYMGq5Snm0%2BEcUWNhQgEr%2Bqx1xYpcA@mail.gmail.com> References: <CAOtMX2g35ybAzFHriVuQqYMGq5Snm0%2BEcUWNhQgEr%2Bqx1xYpcA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Nov 26, 2013 at 11:47 PM, Alan Somers <asomers@freebsd.org> wrote: > I wrote a Kyua backend for Python programs that use the unittest > module from the standard library. It's incomplete (no man page, no > tests, no test case isolation, and no tested version of python other > than 2.7), but I'm posting it to discuss the approach. Do you think > it's a good start? > > Rather than use unittest's CLI, I wrote the backend to dynamically > load the file under test and interrogate the test using unittest as a > library. This allowed more intimate access. For example, the > unittest CLI doesn't have a "list" command, but the backend can still > list tests. > > Does it look good so far? I like the idea a lot (and I had not considered it earlier). The code is a good start, but see below. One concern I have is regarding how you will implement isolation features. The more I think about them, the more I think that they are not specific to the ATF interface at all: i.e. all tests should be executed with the same level of isolation. Now, in your case, it seems you'd have to reimplement these features in Python -- which means that there will be two different implementations for the same thing and this can easily lead to inconsistencies. (atf had 3 versions of this in the very beginning, one for each supported language, and it wasn't pretty!) But having a pure Python binary has its benefits, if only for simplicity. The alternatives would be to either 1) hook the isolation C code into a Python module and call it from your tester, or 2) write the tester in C using all existing code (including CLI parsing for a consistent interface, isolation features, etc.) and just call into the Python interpreter to list tests or run one of them. What do you think? -- Julio Merino / @jmmv
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CADyfeQU64LnutLBOZv1n-Wet_byMFWxSAyh-S-LaAFebyuq1_g>