Skip site navigation (1)Skip section navigation (2) 
Date:      Fri, 28 Nov 2014 10:36:35 -0800
From:      Brian Gardner <openjdk@getsnappy.com>
To:        Craig Rodrigues <rodrigc@FreeBSD.org>
Cc:        "freebsd-testing@freebsd.org" <freebsd-testing@freebsd.org>
Subject:   Re: Call for Help: openjdk8 tests under Continuous Integration
Message-ID:  <8396AB70-4C1F-420A-9BC6-7B832EF91427@getsnappy.com>
In-Reply-To: <177951AE-71AD-4107-85A8-F8089D164514@getsnappy.com>
References:  <CAG=rPVdL32k6yADFv3GrF=1zMHXu-m0NgXMdWEWudvni6fU-hA@mail.gmail.com> <3B1C0F30-6F75-4095-9347-9053F7F2A8A2@getsnappy.com> <CAG=rPVeFwfrZVU3e_6r9zFt4miR0-SKGa17XoBgLyN0Bph-ePw@mail.gmail.com> <5249371E-629C-4026-A220-60764A8B70C4@getsnappy.com> <CAG=rPVcmdZKdvY9gtyh3qFLYBraJCCjcTHB6fbHSf4ZyajMJ5Q@mail.gmail.com> <177951AE-71AD-4107-85A8-F8089D164514@getsnappy.com>
next in thread | previous in thread | raw e-mail | index | archive | help 
Hi Craig,
I=92ve have some scripts setup.  The jtreg produces an xml file for each =
jtreg test and a browsable html structure.=0D  The process is made up a =
two scripts that aren=92t very robust at the moment.  The first script =
starts up poudriere jail with the testport -I openjdk8 options, copies =
the second script and jtreg into the jail, executes it inside the jail, =
then shuts down the jail.  The second script just executes jtreg for =
each subsystem within openjdk that has tests with the appropriate =
arguments.

I might need to modify the scripts depending on how we integrate them =
into Jenkins.  Is there somewhere I should commit these too?  What are =
the next steps to integrating them?


Below is an example jtreg xml file

<?xml version=3D"1.0" encoding=3D"US-ASCII" ?>
<testsuite errors=3D"0" failures=3D"0" tests=3D"1" hostname=3D"localhost" =
name=3D"vm/verifier/VerifyStackForExceptionHandlers.java" time=3D"0.392" =
timestamp=3D"2014-11-331T23:55:32+0000">
    <properties>
        <property name=3D"author" value=3D"Keith McGuigan" />
        <property name=3D"keywords" value=3D"bug6547378" />
        <property name=3D"run" value=3D"ASSUMED_ACTION main =
VerifyStackForExceptionHandlers
" />
        <property name=3D"source" =
value=3D"VerifyStackForExceptionHandlers.java" />
        <property name=3D"title" value=3D"Verify that methods with =
max_stack=3D=3D0 don&apos;t have exception handlers" />
        <property name=3D"description" =
value=3D"file:/wrkdirs/usr/ports/java/openjdk8/work/openjdk/jdk/test/vm/ve=
rifier/VerifyStackForExceptionHandlers.java" />
        <property name=3D"elapsed" value=3D"392 0:00:00.392" />
        <property name=3D"end" value=3D"Thu Nov 27 23:55:32 GMT 2014" />
        <property name=3D"environment" value=3D"regtest" />
        <property name=3D"execStatus" value=3D"Passed. Execution =
successful" />
        <property name=3D"hostname" value=3D"localhost" />
        <property name=3D"javatestOS" value=3D"FreeBSD 10.0-RELEASE =
(amd64)" />
        <property name=3D"javatestVersion" value=3D"4.4.1" />
        <property name=3D"jtregVersion" value=3D"jtreg 4.1 dev b10" />
        <property name=3D"script" =
value=3D"com.sun.javatest.regtest.RegressionScript " />
        <property name=3D"sections" value=3D"script_messages build =
compile main" />
        <property name=3D"start" value=3D"Thu Nov 27 23:55:32 GMT 2014" =
/>
        <property name=3D"test" =
value=3D"vm/verifier/VerifyStackForExceptionHandlers.java" />
        <property name=3D"user.name" value=3D"root" />
        <property name=3D"work" =
value=3D"/wrkdirs/usr/ports/java/openjdk8/work/jtreg-work/jdk/vm/verifier"=
 />
    </properties>
    <testcase =
classname=3D"vm/verifier/VerifyStackForExceptionHandlers.java" =
name=3D"VerifyStackForExceptionHandlers" time=3D"0.392" >
    </testcase>
    <system-out>PASS
    </system-out>
    <system-err>
JavaTest Message:  Test complete.

    </system-err>
</testsuite>


> On Nov 25, 2014, at 12:23 PM, Brian Gardner <openjdk@getsnappy.com> =
wrote:
>=20
> Hi Craig,
> I=92ll continue the investigation, I=92ve been following the openjdk8 =
posts from Kit.  I initially ported openjdk8 and was running jtreg tests =
as part of the process.  If I recall correctly the output was HTML but I =
think this is configurable, worse case I probably can convert it into =
supported xml format.  I=92m thinking we can bake running tests into the =
openjdk8 port Makefile, and use poudriere testport and pass in a script =
that will upload the results into Jenkins.  That should make it really =
easy to run and maintain this on all supported platforms. =20
>=20
> Brian
>=20
>> On Nov 25, 2014, at 12:11 PM, Craig Rodrigues <rodrigc@FreeBSD.org =
<mailto:rodrigc@FreeBSD.org>> wrote:
>>=20
>> Hi,
>>=20
>> Thanks, Brian!
>>=20
>> Well, first of all, we need someone to figure out how to run the =
jtreg tests under FreeBSD,
>> and make that process reproducible.  Would you be willing to do the =
initial investigation for this?
>> If you look at Kip Macy's postings on the freebsd-java list in =
October, you can see that he
>> managed to get the tests to run, so hopefully Kip can answer any =
questions about what he did to get
>> the tests to run.
>>=20
>> I'm not familiar with jtreg.  What ports are you thinking that we =
need to build with poudriere in order to run it?
>>=20
>> Le me try to answer your questions:
>>=20
>> 1)  Let's test on amd64 first, just to iron out all the issues.  Once =
we get that working, we can add more platforms, such as i386.
>> 2)  For an initial effort, maybe we can schedule the tests to run =
once a week, via the Jenkins job scheduler.  We can see if
>>      it makes sense to run the tests more or less frequently after =
that.
>> 3)  I don't know what format the jtreg test output produces.  If it =
can produce JUnit XML output, that would be ideal because
>>      it can be imported directly into Jenkins.  If it does not do =
that, then even if we make the text output of the test logs available =
online,
>>      that is a good start.
>>=20
>> --
>> Craig
>>=20
>> On Sun, Nov 23, 2014 at 9:15 PM, Brian Gardner <openjdk@getsnappy.com =
<mailto:openjdk@getsnappy.com>> wrote:
>> Hello,
>> I=92d like to help.  Sounds like those scripts would be fairly easy =
to write.  Would it be a good idea to try and decouple building of the =
ports (using poudriere) from the actual running of the jtreg tests.  In =
this plan you=92d have a separate job that builds and maintains an =
up-to-date repository of  packages for the supported platforms (10.0 =
amd64, 9.3 i386), then as part of running tests you=92d run pkg to =
update the packages as the first step.
>>=20
>> I have some questions:
>> 1) What platforms will be tested?
>> 2) How often will tests run / What will trigger test runs?
>> 3) How will results of the jtreg tests be presented on Jenkins?
>>=20
>> Brian Gardner
>>=20
>>=20
>> In Response To:
>>=20
>> Hi,
>>=20
>> Recently Kip Macy tried to get the openjdk8 tests working
>> under FreeBSD.  He struggled a bit:
>>=20
>> =
https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010928.html =
<https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010928.html=
>
>> =
https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010929.html =
<https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010929.html=
>
>> =
https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010930.html =
<https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010930.html=
>
>>=20
>> I think it would be good to get these tests running under FreeBSD
>> continous integration.  These tests stress things like threading, =
memory
>> management, networking, and virtual memory.  Running these tests
>> would go a long way to improving Java support under FreeBSD,
>> because it would give a better idea of what can be fixed in FreeBSD =
itself.
>>=20
>> I am a bit overstretched at the moment to take this on.
>>=20
>>=20
>> I need scripts that can do the following:
>>=20
>> (1)  Download the necessary packages, openjdk8, jtreg, etc.
>> (2)  Apply necessary patches to packages in (1)
>> (3)  Build the packages in (1) after patches applied
>> (4)  Run the tests
>>=20
>> I would also like:
>> (A)  Scripts should be committed to =
https://github.com/freebsd/freebsd-ci =
<https://github.com/freebsd/freebsd-ci>;
>> (B)  Script should run under FreeBSD
>> (C)  Scripts should be easy to run under a Jenkins job at
>>        https://jenkins.freebsd.org <https://jenkins.freebsd.org/>;
>> (D)  Followup status and discussion should happen on
>>        the freebsd-testing at freebsd.org =
<https://lists.freebsd.org/mailman/listinfo/freebsd-testing>; mailing =
list
>>=20
>> Is anyone interested in helping out with this?  Starting with some
>> of the things that Kip did would be a good place to start.
>>=20
>> Thanks.
>> --
>> Craig
>>=20
>>=20
>>=20
>>> On Nov 23, 2014, at 8:49 PM, Craig Rodrigues <rodrigc@FreeBSD.org =
<mailto:rodrigc@FreeBSD.org>> wrote:
>>>=20
>>> Hi,
>>>=20
>>> Read this:
>>> =
https://lists.freebsd.org/pipermail/freebsd-testing/2014-November/000668.h=
tml =
<https://lists.freebsd.org/pipermail/freebsd-testing/2014-November/000668.=
html>
>>>=20
>>> and reply to that posting on freebsd-testing@freebsd.org =
<mailto:freebsd-testing@freebsd.org> (subscribe to that list if you
>>> are not on it) with the items that you think that you can
>>> complete.
>>>=20
>>> --
>>> Craig
>>>=20
>>> On Sun, Nov 23, 2014 at 8:28 PM, Brian Gardner =
<openjdk@getsnappy.com <mailto:openjdk@getsnappy.com>> wrote:
>>> Hi Craig,
>>> I would like to help.  Let me know what I can do.
>>>=20
>>> Brian Gardner
>>>=20
>>> > On Nov 22, 2014, at 4:40 PM, Craig Rodrigues <rodrigc@FreeBSD.org =
<mailto:rodrigc@FreeBSD.org>> wrote:
>>> >
>>> > FYI,
>>> >
>>> > =
https://lists.freebsd.org/pipermail/freebsd-testing/2014-November/000668.h=
tml =
<https://lists.freebsd.org/pipermail/freebsd-testing/2014-November/000668.=
html>
>>> >
>>> > Please send followups to freebsd-testing@freebsd.org =
<mailto:freebsd-testing@freebsd.org>.
>>> >
>>> > --
>>> > Craig
>>> > _______________________________________________
>>> > freebsd-java@freebsd.org <mailto:freebsd-java@freebsd.org> mailing =
list
>>> > http://lists.freebsd.org/mailman/listinfo/freebsd-java =
<http://lists.freebsd.org/mailman/listinfo/freebsd-java>;
>>> > To unsubscribe, send any mail to =
"freebsd-java-unsubscribe@freebsd.org =
<mailto:freebsd-java-unsubscribe@freebsd.org>"
>>>=20
>>>=20
>>=20
>>=20
>=20

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8396AB70-4C1F-420A-9BC6-7B832EF91427>