Date: Mon, 2 Mar 2015 17:07:32 -0700 From: Alan Somers <asomers@freebsd.org> To: Julio Merino <jmmv@meroh.net> Cc: "freebsd-testing@freebsd.org" <freebsd-testing@freebsd.org> Subject: Re: Python unittest backend for Kyua Message-ID: <CAOtMX2hvfpskz2aTy5kwS3ewXPgNLeWeoS3AcSG4EUSK2WTPpA@mail.gmail.com> In-Reply-To: <C288DCCA-6031-4C03-8163-8826FBCE12A7@meroh.net> References: <CAOtMX2g35ybAzFHriVuQqYMGq5Snm0%2BEcUWNhQgEr%2Bqx1xYpcA@mail.gmail.com> <20150214215750.GA5065@rodrigc-laptop1> <CAOtMX2jFi89wt1fC5OAAGZhEDHKpLZza=tAXU7WM5y-pwL4Yqw@mail.gmail.com> <C288DCCA-6031-4C03-8163-8826FBCE12A7@meroh.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Mar 2, 2015 at 4:42 PM, Julio Merino <jmmv@meroh.net> wrote: > >> On Feb 15, 2015, at 00:16, Alan Somers <asomers@freebsd.org> wrote: >> >> Not much further than when I made that post, and no. In fact, I >> hardly use Python at all. Professionally, my company is all about >> Ruby, and I would be very interested in a Ruby test/unit tester for >> Kyua. However, Ruby's test/unit's implementation is much messier than >> Python's unittest, so I thought that writing a unittest tester would >> be good practice for writing a test/unit tester. Indeed it was. I >> found unittest very easy to interface to; it hardly took me any time >> to write that tester. But in the end I stopped working on it because >> jmmv and I disagreed about test isolation. Jmmv's primary interest is >> in writing system-level tests. He basically sees a unittest tester as >> an alternative to atf-sh. But my primary interest is in tying >> together separate components' test suites and get a consistent view of >> all results. So naturally jmmv wants the same level of isolation as >> atf-sh provides, but I want the same level of isolation as unittest >> provides. In fact, atf-sh style isolation is bad for my use case, >> because it can cause tests that were originally written for unittest >> to fail. It also drastically increases runtime because the Python >> interpreter must be restarted between each test. > > As a matter of fact, I have changed my thoughts on this. I would like Kyua to better support "unittest"-style test programs because that's what pretty much all testing libraries implement and what other people know... and well, this model is just much faster and equally useful for the vast majority of the cases. Well, maybe I should dust off my old branch and get back to work. But I don't know how my python tester would work with your executor branch. If I understand correctly, the executor branch dispenses with the *-tester binaries and calls the test programs directly from the kyua executable, correct? The python tester relied on the tester being a separate binary. The tester was actually implemented in Python, which made it very easy for the tester to interact with individual test cases. How would I accomplish that on the executor branch? -Alan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2hvfpskz2aTy5kwS3ewXPgNLeWeoS3AcSG4EUSK2WTPpA>