From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 20:47:02 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E501EF12 for ; Sun, 2 Mar 2014 20:47:02 +0000 (UTC) Received: from mail-qa0-f46.google.com (mail-qa0-f46.google.com [209.85.216.46]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9F27A1720 for ; Sun, 2 Mar 2014 20:47:01 +0000 (UTC) Received: by mail-qa0-f46.google.com with SMTP id i13so2768143qae.33 for ; Sun, 02 Mar 2014 12:47:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=pby86351a54jMXmAsyHgmFC6ZJuXNCvMGlRAZq0TwDA=; b=Pt477uRRMlWfYnz8rPwVyBTAZ1gbrlgGiZtpMGCPhYImyIb6r4Eb28geN2+1WaL0K6 QFUPbzTRne9qBQ/CUTzqkCS5ti4qLhAPK2HBZAgt7/Our6bUncl2CdOK7p5iiJO/Pc1b Ctcs7kDv+8S2pJWZ2Os8BJuJuyDuSJVIEt1aEqxw8Odj9E44r48uUfAYdWVYW6Yfv7Lw 3AnuJ0KPuI861tJVgdHz9jjeIKqA3tvHHhMX8YUAh6h1OQtcMBkQxG7O9dluTVx93YVm MOkhmDiGHGeuX1rFTOWreSHGlop3ByeDiq97pIyPvEajF5GmCiLd5pYh9XLNhUTpWVif kbgQ== X-Gm-Message-State: ALoCoQkW9ZeC0kDBN2Txvr5m1VmGhfn0k5g6Tmp/y6XXG/9OHdvm4J3aIz9u5jFlDNQtAPh/t8IR X-Received: by 10.140.39.50 with SMTP id u47mr18337408qgu.46.1393793221272; Sun, 02 Mar 2014 12:47:01 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Sun, 2 Mar 2014 12:46:41 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: <20140225033526.GN85115@kiwi> References: <20140225033526.GN85115@kiwi> From: Julio Merino Date: Sun, 2 Mar 2014 15:46:41 -0500 X-Google-Sender-Auth: azjH0LJ4fveJkeQNScYjQN4gwio Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? To: "R. Tyler Croy" Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , jenkins-admin@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 20:47:03 -0000 On Mon, Feb 24, 2014 at 10:35 PM, R. Tyler Croy wrote: >> 2) I don't see any "expected failure" results in the links that you >> provided. That's an absolute requirement for Kyua. Treating expected >> failures like ordinary failures will greatly reduce the utility of >> Kyua, by reducing its ability to spot regressions. > > > JUnit allows for the concept of "skipped" tests, which Jenkins' parsing code > respects. IIRC this is just a "skipped=true" on a , I believe that > would help solve for this case. No, it's not. An expected failure is not the same as skipped. Possibly the best approximation is to consider an expected failure as a pass.