Date: Tue, 31 Dec 2013 09:10:42 -0800 From: Garrett Cooper <yaneurabeya@gmail.com> To: Julio Merino <julio@meroh.net> Cc: "freebsd-testing@freebsd.org" <freebsd-testing@freebsd.org>, "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Re: First cut of the continuous testing infrastructure Message-ID: <7B0C24C8-8F93-4658-8D77-09B7C4F26805@gmail.com> In-Reply-To: <CADyfeQWfzfdLodxPFGYWtCB_gRvcbKbf2QSK_arSDB0yrJ7XYw@mail.gmail.com> References: <CADyfeQWfzfdLodxPFGYWtCB_gRvcbKbf2QSK_arSDB0yrJ7XYw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Dec 31, 2013, at 5:22, Julio Merino <julio@meroh.net> wrote: > > Dear all, > > With all the work that has gone into the tree to add the barebones of > a test suite in FreeBSD, and with the generous availability of 3 > machines for testing purposes, I am pleased to announce the very first > cut of the testing infrastructure! > > Before spoiling the fun, let me make clear that these pages are /not/ > final by any means. There is still a ton of work to be done to, at > least: 1) make the reports more amenable and usable; 2) extend the > infrastructure to cover more releases and architectures; 3) grow the > test suite to make it a useful tool; 4) fix existing breakage; and 5) > notify developers when things break (possibly even pinpointing at > faulty commits!). > > Having said that, point your browser at: > > http://kyua1.nyi.freebsd.org/ > > and have fun! > > You may read some more on this work on the project page: > > http://wiki.freebsd.org/TestSuite > > and on my blog post introducing the FreeBSD test suite: > > http://julipedia.meroh.net/2013/12/introducing-freebsd-test-suite.html > > Hoping you find this useful in its current form... let the feedback flow! > > Enjoy your entry into 2014 :-) Well done! The only thing I see that could be improved in the HTML format is: 1. The total number of tests should be the last row. 2. Non-engineers (and some engineers like me) like percentages, in addition to raw numbers. Helps reduce panic when someone goes "oh my gosh! We have 18 tests failing!!!" Cheers and have a happy new year :)! -Garrett PS some of the ATF tests (for 0.17 at least) won't work unless you have the right configuration file [from share/?], but if you have that configuration file, it will cause regular tests to bomb because of recursion with /bin/sh . Granted, that was with ATF, not Kyua driving the tests...
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7B0C24C8-8F93-4658-8D77-09B7C4F26805>