From owner-freebsd-testing@FreeBSD.ORG Mon Mar 3 21:27:18 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C267795A; Mon, 3 Mar 2014 21:27:18 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1B45F2AA; Mon, 3 Mar 2014 21:27:17 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id mc6so4922488lab.8 for ; Mon, 03 Mar 2014 13:27:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=BTn9QwkE+rpe9AJw6i7OqdWnfoW3OYNADi2xjiDe690=; b=HU+4XdpQQU16P9WFcx/wPHkuWgZcZsW9vDkcld4iPQY+ixCD/ec2+/hsMiT1IrTEoU mdcGd3bP5+PXbqpqpSB/w1MqAv/j/o/s0baVxOjykDgYqP0PBzxwq8yzPEuYXDBQhY5a cYm56g7nbWFXxci+utC29prHQ/nuOMdJ2d/PUPxPhL/CEfxAhbwQI25N2yb8WY3e/jRw f6xoTNMNBE/7u+CXEePza0vMdDbjzS4wSdmV3d2df+h1nS+KuqsOs0cmBEMpzRRR9u3G 51eFUNWb3PINi1fEGd23eo6hiX0zHtbj9LxfPvCrieFRwSQPA3sCndy0ZkWcIZeFPlU3 Kt3Q== MIME-Version: 1.0 X-Received: by 10.112.51.65 with SMTP id i1mr442815lbo.80.1393882036143; Mon, 03 Mar 2014 13:27:16 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Mon, 3 Mar 2014 13:27:16 -0800 (PST) In-Reply-To: References: Date: Mon, 3 Mar 2014 13:27:16 -0800 X-Google-Sender-Auth: yBfuIp6DgOBsRPtLSTVBD9kP12g Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? From: Craig Rodrigues To: kyua-discuss@googlegroups.com Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 21:27:18 -0000 On Sun, Mar 2, 2014 at 12:26 PM, Julio Merino wrote: > > (As I mentioned in the other thread) this is relatively easy to do and > we have enough time to get it going by May 15th. > > The path of least resistance to get this out for now is: > > 1. Reintroduce support for --output=format:path to 'report' so that > 'report' can output more than one format. The existence of > 'report-html' as a separate command is a pretty ugly hack. > > 2. Split out the console-specific formatting from cmd_report.cpp into > a separate module to clearly expose what is part of the formatting and > what is not. Adding JUnit XML support should not require changes to > other code. > > 3. Add a new module with the JUnit XML formatting code. > I think that these are all fine goals. I might be able to help with 3. > > (Longer-term, I am pondering the option of converting all the > report-generation code to Lua with extensive support for hooks. This > would allow you to tweak the output format without having to change > Kyua internals.) > That's something to think about. After a few days of research I found two other test result output formats: (1) TestNG, which is a successor to JUnit: http://testng.org/doc/index.html , TestNG DTD: http://testng.org/testng-1.0.dtd.php (2) Test Anything Protocol, http://testanything.org/ I don't know how widely these formats are used, but people *are* using them. The easier we make it to customize Kyua's output formats, the easier it will be to get more people to use Kyua. -- Craig