Date: Wed, 9 Apr 2014 16:18:21 -0700 From: Garrett Cooper <yaneurabeya@gmail.com> To: "Kilner, Peter" <Peter.Kilner@emc.com> Cc: "freebsd-testing@freebsd.org" <freebsd-testing@freebsd.org> Subject: Re: ATF Test Cases Message-ID: <CAGHfRMAF5_s4=A_OucgrGMmKKSj323S8DptrKPOPTV8UHVOZjw@mail.gmail.com> In-Reply-To: <DB777D43-945C-4C0D-A3A8-66DD5C9B214E@isilon.com> References: <DB777D43-945C-4C0D-A3A8-66DD5C9B214E@isilon.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi again Peter! On Wed, Mar 26, 2014 at 3:26 PM, Kilner, Peter <Peter.Kilner@emc.com> wrote= : > Hello, > > I hope I am posting this question to the correct list. > > Is there a way to have a single ATF test case report multiple pass/fail r= esults? Currently I have seen that one can include a number of test condit= ions in a single test case (for example many aft_checks). However the test= case will only report one pass/fail result for that case. > > I would like to build a test case that will iterate though many configura= tions and would like to see a pass/fail for each config. However this is d= ifficult to implement with multiply test cases because of the for loops tha= t I am using. > > Any suggestions would be appreciated. As others have suggested, your approach could be tweaked a bit to iterate over all of the different configurations you are trying to test. In general I would create either a generic library or test driver with common code and have a common validation method, then build off that with additional validation methods if needed. I do that all the time. I think that Julio's suggestion with dynamically eval'ing tests is spot on as well. There's an example here of how to do that (that we might want to extend to a more generic form and push into Kyua or FreeBSD's test infrastructure if it's ok): https://github.com/yaneurabeya/freebsd/blob/master/bin/sh/tests/functional_= test.sh . Important: this code is proof of concept -- please do _NOT_ copy-paste it if at all possible. Thanks, -Garrett
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAGHfRMAF5_s4=A_OucgrGMmKKSj323S8DptrKPOPTV8UHVOZjw>