Date: Wed, 3 Jun 2015 12:19:02 -0700 From: Garrett Cooper <yaneurabeya@gmail.com> To: Craig Rodrigues <rodrigc@FreeBSD.org> Cc: Ed Maste <emaste@freebsd.org>, "freebsd-testing@freebsd.org" <freebsd-testing@freebsd.org> Subject: Re: Incorporating the Capsicum test suite Message-ID: <0582DC06-4021-443D-B693-93F848654441@gmail.com> In-Reply-To: <CAG=rPVd19tkuy=Rh48KYH%2B224RXsgQNRooUhruOfsXEGM1EwGg@mail.gmail.com> References: <CAPyFy2AiPZKPRmX4u4MGQygvP0PaXMx4Ay9a9O8V1pteULHO%2Bg@mail.gmail.com> <CAG=rPVd19tkuy=Rh48KYH%2B224RXsgQNRooUhruOfsXEGM1EwGg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Jun 3, 2015, at 08:30, Craig Rodrigues <rodrigc@FreeBSD.org> wrote: >=20 >> On Tue, Jun 2, 2015 at 1:40 PM, Ed Maste <emaste@freebsd.org> wrote: >>=20 >>=20 >> I have a couple of questions. First, do we have an existing pattern >> for connecting googletest tests to Kyua? >=20 > No we do not. Googletest looks like it is basically trying to do > the same thing as ATF or CUnit. >=20 > Googletest has a way to generate JUnit XML test output: > https://code.google.com/p/googletest/wiki/AdvancedGuide#Generating_an_XML_= Report >=20 > The easiest thing to do would be to configure the Capsicum tests to > generate this XML test output. This can be imported natively into > Jenkins, which can then parse the test report. > This would require no changes to the Capsicum tests. >=20 > If you want to integrate googletest with kyua, then things get tricky. > On the googletest page: https://code.google.com/p/googletest/ > they list a third party plugin which can take googletest and generate TAP > output. > I don't know how well that works. It would be nice if that functionality > was > integrated in googletest natively. >=20 > Another option would be to modify kyua, so that it can parse the output of= > googletest > natively. This would be quite nice. I haven't found extending kyua to be= > very > easy....it requires good knowledge of C++ and Lua. Also, whoever > contributes > code to kyua must sign a Contributor License Agreement (CLA) with Google. > That's something to keep in mind. >=20 > Another option would be to rewrite the Capsicum tests in ATF. > I don't know how many tests there are, but this might be doable. > ATF and Googletest look similar. >=20 > If you wanted to import the Capsicum tests into FreeBSD as they are right > now, then it looks > like you would need to import googletest into FreeBSD as well. googletest= > is BSD licensed, > so that is not a problem, but we already have ATF in the tree, so I don't > know how > useful it would be to have multiple test frameworks in the base system. Hi Ed, I'll take a look at the test infrastructure and see if I can write a sim= ple shim for ATF to interact with it. I agree with a lot of what Craig said. The Google corporate CLA has impe= ded engineers in EMC / Isilon and some other groups from contributing back t= o the ATF / Kyua projects ;( (this is why I've just been filing bugs against= the projects and have been focusing my efforts on integrating tools/regress= ion tests into Kyua), so anything that needs to be done development wise on t= hese projects will need to be done internal to the FreeBSD project for the t= ime being. Thanks! -NGie=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0582DC06-4021-443D-B693-93F848654441>