From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 20:34:59 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43424DCF for ; Sun, 2 Mar 2014 20:34:59 +0000 (UTC) Received: from mail-qa0-f52.google.com (mail-qa0-f52.google.com [209.85.216.52]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0123D162F for ; Sun, 2 Mar 2014 20:34:58 +0000 (UTC) Received: by mail-qa0-f52.google.com with SMTP id m5so2710511qaj.39 for ; Sun, 02 Mar 2014 12:34:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=5D2J0qXNQaKtjQi91gXO8jym55Nj09NnGJdE4r0qfGg=; b=lzc6ezjeEnv0Wqj2jf162BTE5Dw79FIljYpJOkV61xHuoEX3I/mlLkRvFjA7HPsB7J TAYHHmyhJw29RC2KVeLsbXcbAsTGDQJcrslb6r/mtoPRxRluhy5s9MZ9/oCbR3uZ97n5 8jSQXBLat1jJ18XKJGV7aBvxYwLVmBiBfBC4R3jEm1cChtbxY87upbCkUtp3w5+eSHys sOMAD7pNfomY+j5dQ+2m+POoUHEcqz26lsy4f8dbtm7+nuSiw4R4MvE9VNI8zdi2RyFf FAkA/eAwJ57ppGwDCGanm3EX3xB+kSdAgZ3Mpe0wTMEy0BnXGUqVWWf7pMpsB8mbzzbw 8lhQ== X-Gm-Message-State: ALoCoQnT0gJywF4WXPl+KHc/3SrmP4L34fRwtvT+zETuOuZvumifuP0nqpw4+HXcB/gsTwHxnA/w X-Received: by 10.140.97.137 with SMTP id m9mr3660517qge.95.1393792013553; Sun, 02 Mar 2014 12:26:53 -0800 (PST) MIME-Version: 1.0 Received: by 10.96.83.102 with HTTP; Sun, 2 Mar 2014 12:26:33 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: References: From: Julio Merino Date: Sun, 2 Mar 2014 15:26:33 -0500 Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? To: kyua-discuss@googlegroups.com Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 20:34:59 -0000 On Thu, Feb 27, 2014 at 3:31 PM, Craig Rodrigues wrote: > Hi, > > In this thread on the freebsd-testing mailing list > I asked if it would be a good idea to extend > Kyua so that it could generate test result output > in JUnit XML test result format: > > http://lists.freebsd.org/pipermail/freebsd-testing/2014-February/000199.html > > JUnit XML is parsed natively by frameworks like Jenkins and > Atlassian Bamboo. > > The response on that thread seemed favorable. > > Tyler Croy mentioned that he could help integrate and test things, > but needs help on the C++ side of things. > > I'd like to get going on this and see if we can get a proof of concept > to demo at the May 15 Continuous Testing working group that I am > leading at the BSD Devsummit in Ottawa: (As I mentioned in the other thread) this is relatively easy to do and we have enough time to get it going by May 15th. The path of least resistance to get this out for now is: 1. Reintroduce support for --output=format:path to 'report' so that 'report' can output more than one format. The existence of 'report-html' as a separate command is a pretty ugly hack. 2. Split out the console-specific formatting from cmd_report.cpp into a separate module to clearly expose what is part of the formatting and what is not. Adding JUnit XML support should not require changes to other code. 3. Add a new module with the JUnit XML formatting code. I have finished with 1 (locally, not pushed out yet) and started working on 2, which is slightly trickier. I can also do 3 but not until after AsiaBSDCon 2014 (that is, mid-March). I suppose that with this done, integrating the output into Jenkins should be trivial. (Longer-term, I am pondering the option of converting all the report-generation code to Lua with extensive support for hooks. This would allow you to tweak the output format without having to change Kyua internals.)