From owner-freebsd-testing@FreeBSD.ORG Fri Nov 28 18:36:53 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F34F9E1E; Fri, 28 Nov 2014 18:36:52 +0000 (UTC) Received: from mailA.getsnappy.com (mailA.getsnappy.com [72.29.186.40]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CEAAB238; Fri, 28 Nov 2014 18:36:52 +0000 (UTC) Received: from [192.168.0.2] (138-229-191-115.dhcp.snlo.ca.charter.com [138.229.191.115] (may be forged)) (authenticated bits=0) by mailA.getsnappy.com (8.14.9/8.14.7) with ESMTP id sASIabei065397 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 28 Nov 2014 10:36:40 -0800 (PST) (envelope-from openjdk@getsnappy.com) X-Authentication-Warning: mailA.getsnappy.com: Host 138-229-191-115.dhcp.snlo.ca.charter.com [138.229.191.115] (may be forged) claimed to be [192.168.0.2] Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: Call for Help: openjdk8 tests under Continuous Integration From: Brian Gardner In-Reply-To: <177951AE-71AD-4107-85A8-F8089D164514@getsnappy.com> Date: Fri, 28 Nov 2014 10:36:35 -0800 Message-Id: <8396AB70-4C1F-420A-9BC6-7B832EF91427@getsnappy.com> References: <3B1C0F30-6F75-4095-9347-9053F7F2A8A2@getsnappy.com> <5249371E-629C-4026-A220-60764A8B70C4@getsnappy.com> <177951AE-71AD-4107-85A8-F8089D164514@getsnappy.com> To: Craig Rodrigues X-Mailer: Apple Mail (2.1993) Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Nov 2014 18:36:53 -0000 Hi Craig, I=92ve have some scripts setup. The jtreg produces an xml file for each = jtreg test and a browsable html structure.=0D The process is made up a = two scripts that aren=92t very robust at the moment. The first script = starts up poudriere jail with the testport -I openjdk8 options, copies = the second script and jtreg into the jail, executes it inside the jail, = then shuts down the jail. The second script just executes jtreg for = each subsystem within openjdk that has tests with the appropriate = arguments. I might need to modify the scripts depending on how we integrate them = into Jenkins. Is there somewhere I should commit these too? What are = the next steps to integrating them? Below is an example jtreg xml file PASS JavaTest Message: Test complete. > On Nov 25, 2014, at 12:23 PM, Brian Gardner = wrote: >=20 > Hi Craig, > I=92ll continue the investigation, I=92ve been following the openjdk8 = posts from Kit. I initially ported openjdk8 and was running jtreg tests = as part of the process. If I recall correctly the output was HTML but I = think this is configurable, worse case I probably can convert it into = supported xml format. I=92m thinking we can bake running tests into the = openjdk8 port Makefile, and use poudriere testport and pass in a script = that will upload the results into Jenkins. That should make it really = easy to run and maintain this on all supported platforms. =20 >=20 > Brian >=20 >> On Nov 25, 2014, at 12:11 PM, Craig Rodrigues > wrote: >>=20 >> Hi, >>=20 >> Thanks, Brian! >>=20 >> Well, first of all, we need someone to figure out how to run the = jtreg tests under FreeBSD, >> and make that process reproducible. Would you be willing to do the = initial investigation for this? >> If you look at Kip Macy's postings on the freebsd-java list in = October, you can see that he >> managed to get the tests to run, so hopefully Kip can answer any = questions about what he did to get >> the tests to run. >>=20 >> I'm not familiar with jtreg. What ports are you thinking that we = need to build with poudriere in order to run it? >>=20 >> Le me try to answer your questions: >>=20 >> 1) Let's test on amd64 first, just to iron out all the issues. Once = we get that working, we can add more platforms, such as i386. >> 2) For an initial effort, maybe we can schedule the tests to run = once a week, via the Jenkins job scheduler. We can see if >> it makes sense to run the tests more or less frequently after = that. >> 3) I don't know what format the jtreg test output produces. If it = can produce JUnit XML output, that would be ideal because >> it can be imported directly into Jenkins. If it does not do = that, then even if we make the text output of the test logs available = online, >> that is a good start. >>=20 >> -- >> Craig >>=20 >> On Sun, Nov 23, 2014 at 9:15 PM, Brian Gardner > wrote: >> Hello, >> I=92d like to help. Sounds like those scripts would be fairly easy = to write. Would it be a good idea to try and decouple building of the = ports (using poudriere) from the actual running of the jtreg tests. In = this plan you=92d have a separate job that builds and maintains an = up-to-date repository of packages for the supported platforms (10.0 = amd64, 9.3 i386), then as part of running tests you=92d run pkg to = update the packages as the first step. >>=20 >> I have some questions: >> 1) What platforms will be tested? >> 2) How often will tests run / What will trigger test runs? >> 3) How will results of the jtreg tests be presented on Jenkins? >>=20 >> Brian Gardner >>=20 >>=20 >> In Response To: >>=20 >> Hi, >>=20 >> Recently Kip Macy tried to get the openjdk8 tests working >> under FreeBSD. He struggled a bit: >>=20 >> = https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010928.html = >> = https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010929.html = >> = https://lists.freebsd.org/pipermail/freebsd-java/2014-October/010930.html = >>=20 >> I think it would be good to get these tests running under FreeBSD >> continous integration. These tests stress things like threading, = memory >> management, networking, and virtual memory. Running these tests >> would go a long way to improving Java support under FreeBSD, >> because it would give a better idea of what can be fixed in FreeBSD = itself. >>=20 >> I am a bit overstretched at the moment to take this on. >>=20 >>=20 >> I need scripts that can do the following: >>=20 >> (1) Download the necessary packages, openjdk8, jtreg, etc. >> (2) Apply necessary patches to packages in (1) >> (3) Build the packages in (1) after patches applied >> (4) Run the tests >>=20 >> I would also like: >> (A) Scripts should be committed to = https://github.com/freebsd/freebsd-ci = >> (B) Script should run under FreeBSD >> (C) Scripts should be easy to run under a Jenkins job at >> https://jenkins.freebsd.org >> (D) Followup status and discussion should happen on >> the freebsd-testing at freebsd.org = mailing = list >>=20 >> Is anyone interested in helping out with this? Starting with some >> of the things that Kip did would be a good place to start. >>=20 >> Thanks. >> -- >> Craig >>=20 >>=20 >>=20 >>> On Nov 23, 2014, at 8:49 PM, Craig Rodrigues > wrote: >>>=20 >>> Hi, >>>=20 >>> Read this: >>> = https://lists.freebsd.org/pipermail/freebsd-testing/2014-November/000668.h= tml = >>>=20 >>> and reply to that posting on freebsd-testing@freebsd.org = (subscribe to that list if you >>> are not on it) with the items that you think that you can >>> complete. >>>=20 >>> -- >>> Craig >>>=20 >>> On Sun, Nov 23, 2014 at 8:28 PM, Brian Gardner = > wrote: >>> Hi Craig, >>> I would like to help. Let me know what I can do. >>>=20 >>> Brian Gardner >>>=20 >>> > On Nov 22, 2014, at 4:40 PM, Craig Rodrigues > wrote: >>> > >>> > FYI, >>> > >>> > = https://lists.freebsd.org/pipermail/freebsd-testing/2014-November/000668.h= tml = >>> > >>> > Please send followups to freebsd-testing@freebsd.org = . >>> > >>> > -- >>> > Craig >>> > _______________________________________________ >>> > freebsd-java@freebsd.org mailing = list >>> > http://lists.freebsd.org/mailman/listinfo/freebsd-java = >>> > To unsubscribe, send any mail to = "freebsd-java-unsubscribe@freebsd.org = " >>>=20 >>>=20 >>=20 >>=20 >=20