Date: Sun, 24 Nov 2013 20:54:50 -0500 From: Julio Merino <julio@meroh.net> To: Garrett Cooper <yaneurabeya@gmail.com> Cc: freebsd-testing@freebsd.org Subject: Re: Adding new "test interfaces" to work with Kyua? Message-ID: <CADyfeQU4hDoYXt%2BW%2BjJqvA7X56aD09qcM7K_DXv7GsxrA=o-5g@mail.gmail.com> In-Reply-To: <751B8D9F-9A1B-46B5-80F8-CB9E44F98BFC@gmail.com> References: <82E5F5E2-EA2C-46D2-9068-C419A834686C@gmail.com> <CADyfeQUaX4J1ycPhc=79P1fnfQh8E-UO349OzSL50KCJZtbOFw@mail.gmail.com> <751B8D9F-9A1B-46B5-80F8-CB9E44F98BFC@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Nov 24, 2013 at 5:34 PM, Garrett Cooper <yaneurabeya@gmail.com> wro= te: > On Nov 24, 2013, at 2:29 PM, Julio Merino <julio@meroh.net> wrote: > >> On Sun, Nov 24, 2013 at 5:09 PM, Garrett Cooper <yaneurabeya@gmail.com> = wrote: >>> Trying to reconcile my diffs and get things sane again, I realiz= e that due to refactoring (and especially after I installed devel/kyua) tha= t prove.test.mk no longer functions for me off my branch. Are there directi= ons for how I can program in a new test interface so I can integrate prove = testing into Kyua? >> >> What kind of interface do prove-based test programs have? Are they >> binaries/scripts that you can just run and assume that a 0 exit code >> means passed and 1 means failed? If yes, you'd just funnel them >> through the 'plain' interface and they should just work as a first >> cut. (I believe this approach works for most test programs out >> there.) >> >> But there are some details here: >> >> https://code.google.com/p/kyua/wiki/TestersDesign >> >> and more in the kyua-tester manual page and its references. > > It=92s a bit more complex than that. nose and prove both implement collec= tion and execution test features, meaning that it goes and figures out what= to execute (for nose it=92s a variety of parameters =97 filename, mode, et= c =97 that determine whether or not it=92s testable; for prove it=92s all f= iles with a .t extension), then executes it. I need to read about that a bit more then. Is there a web site for these tools? Their names are so generic that it's hard to find stuff. That said, I have started to take a look at the existing .t tests... and the majority of them seem to be just wrappers that execute a shell script. In these cases, I think we should just drop the .t and use the plain interface to execute the existing shell script. By pure coincidence, I had been working today on "porting" a couple of these tests; expect a patch for the proof of concept soon. > I=92d like to take the error messages and make them more meaningful thoug= h, because it doesn=92t help if something fails and I need to diagnose why = it failed, but lost the info I need to debug the problem... As long as these print the error messages to either stdout or stderr, they will be captured by Kyua and stored into the reports. --=20 Julio Merino / @jmmv
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CADyfeQU4hDoYXt%2BW%2BjJqvA7X56aD09qcM7K_DXv7GsxrA=o-5g>