Date: Wed, 3 Jun 2015 15:51:11 -0400 From: Ed Maste <emaste@freebsd.org> To: Craig Rodrigues <rodrigc@freebsd.org> Cc: "freebsd-testing@freebsd.org" <freebsd-testing@freebsd.org> Subject: Re: Incorporating the Capsicum test suite Message-ID: <CAPyFy2BDQeZK=4bBcB4TFpoBhaUOOvA=O53f_eDHq7XJhX%2BunQ@mail.gmail.com> In-Reply-To: <CAG=rPVd19tkuy=Rh48KYH%2B224RXsgQNRooUhruOfsXEGM1EwGg@mail.gmail.com> References: <CAPyFy2AiPZKPRmX4u4MGQygvP0PaXMx4Ay9a9O8V1pteULHO%2Bg@mail.gmail.com> <CAG=rPVd19tkuy=Rh48KYH%2B224RXsgQNRooUhruOfsXEGM1EwGg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 3 June 2015 at 11:30, Craig Rodrigues <rodrigc@freebsd.org> wrote: > On Tue, Jun 2, 2015 at 1:40 PM, Ed Maste <emaste@freebsd.org> wrote: >> >> I have a couple of questions. First, do we have an existing pattern >> for connecting googletest tests to Kyua? > > No we do not. Googletest looks like it is basically trying to do > the same thing as ATF or CUnit. Yes, the intent is the same. My desire to integrate it with Kyua is so that we can add this test coverage without needing the user to run additional commands, parse different output formats, etc. We could integrate the existing tests as-is into Jenkins fairly easily, but I also want us to move to a model where developers consistently run the test suite prior to committing a change. > Another option would be to modify kyua, so that it can parse the output of > googletest > natively. This would be quite nice. I haven't found extending kyua to be > very > easy....it requires good knowledge of C++ and Lua. Also, whoever > contributes > code to kyua must sign a Contributor License Agreement (CLA) with Google. > That's something to keep in mind. It seems to me that this is the best way to go, with connecting the existing tests as-is into Jenkins as my second choice. FreeBSD developers who wish to make changes to Kyua and the Google CLA is an issue we could fork it and make changes in our own version. It's not ideal, but in practice if we don't expect large, conflicting changes from upstream it should be a workable approach. > Another option would be to rewrite the Capsicum tests in ATF. > I don't know how many tests there are, but this might be doable. > ATF and Googletest look similar. I think this is feasible, and may make sense for the Capsicum tests. However, the idea extends beyond the Capsicum tests. Clang/LLVM (and LLDB) have a large test suite that I'd like incorporate into our process, and other 3rd party software does as well. It's not feasible to try to convert all of these to ATF. > If you wanted to import the Capsicum tests into FreeBSD as they are right > now, then it looks > like you would need to import googletest into FreeBSD as well. googletest > is BSD licensed, > so that is not a problem, but we already have ATF in the tree, so I don't > know how > useful it would be to have multiple test frameworks in the base system. The parts of googletest actually required for the capsicum tests are fairly small and self-contained (even though the entire googletest package is included in the capsicum-test repo). Assuming capsicum-test gets stripped it down to just what's required I might just consider the embedded googletest part of capsicum-test, rather than importing it as a standalone project and having capsicum-test use it.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAPyFy2BDQeZK=4bBcB4TFpoBhaUOOvA=O53f_eDHq7XJhX%2BunQ>