From owner-freebsd-testing@freebsd.org Thu Apr 11 15:09:26 2019 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C8CAC15869FA for ; Thu, 11 Apr 2019 15:09:26 +0000 (UTC) (envelope-from susan.stanziano@xinuos.com) Received: from zhost.uxsglobal.com (zhost.xinuos.com [144.202.236.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7F9846B1DE; Thu, 11 Apr 2019 15:09:25 +0000 (UTC) (envelope-from susan.stanziano@xinuos.com) Received: from zhost.uxsglobal.com (localhost [127.0.0.1]) by zhost.uxsglobal.com (Postfix) with ESMTPS id AD6BA880671; Thu, 11 Apr 2019 11:09:23 -0400 (EDT) Received: from zhost.uxsglobal.com (localhost [127.0.0.1]) by zhost.uxsglobal.com (Postfix) with ESMTPS id 9740C880679; Thu, 11 Apr 2019 11:09:23 -0400 (EDT) DKIM-Filter: OpenDKIM Filter v2.9.2 zhost.uxsglobal.com 9740C880679 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xinuos.com; s=710E410E-A6C0-11E6-9C55-A8275FB0EC07; t=1554995363; bh=+udjCvhqHiWd6D3e1Q4RwcRSyGOjEHSOThEaxrNuK9o=; h=Date:From:To:Message-ID:Subject:MIME-Version:Content-Type; b=JXr3y+m0iBUOkdH7ZD8hMRG8jB8Pm2YebgbeghgNok4whMcb9znCPAOt5UA3L2Rcu AW4AeiM0G6rmr/7wmUA1TZgBT0IQchzVTGylISDlCJOlSxC2O4FOFCuL2kmeHss8jq oA/8pbZ0TkIcMfPyaGuxGW6SzFS1cF/Ge9TAngDU= Received: from zhost.uxsglobalhost.com (localhost [127.0.0.1]) by zhost.uxsglobal.com (Postfix) with ESMTP id 949A2880671; Thu, 11 Apr 2019 11:09:23 -0400 (EDT) Date: Thu, 11 Apr 2019 11:09:23 -0400 (EDT) From: Susan Stanziano To: NGie Cooper , asomers@freebsd.org Cc: freebsd-testing Message-ID: <989313104.2320168.1554995363573.JavaMail.zimbra@xinuos.com> In-Reply-To: <1433014530.2305640.1554952181864.JavaMail.zimbra@xinuos.com> References: <555587672.2279585.1554935416410.JavaMail.zimbra@xinuos.com> <80FAC842-ECCB-438F-BBC4-0995A7C1D7B1@gmail.com> <359307252.2304183.1554948627748.JavaMail.zimbra@xinuos.com> <0BD361D0-3B1F-4B3D-AD65-A85153E745AA@gmail.com> <1433014530.2305640.1554952181864.JavaMail.zimbra@xinuos.com> Subject: Re: Kyua reporting tool question MIME-Version: 1.0 X-Originating-IP: [71.251.34.187] X-Mailer: Zimbra 8.6.0_GA_1153 (ZimbraWebClient - GC73 (Win)/8.6.0_GA_1153) Thread-Topic: Kyua reporting tool question Thread-Index: Q/IQ9Mtot1GK8GfvLTOvvclyKZPlf/PWz7nw X-Rspamd-Queue-Id: 7F9846B1DE X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=xinuos.com header.s=710E410E-A6C0-11E6-9C55-A8275FB0EC07 header.b=JXr3y+m0; spf=pass (mx1.freebsd.org: domain of susan.stanziano@xinuos.com designates 144.202.236.63 as permitted sender) smtp.mailfrom=susan.stanziano@xinuos.com X-Spamd-Result: default: False [-3.05 / 15.00]; HAS_XOIP(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:144.202.236.60/30]; URI_COUNT_ODD(1.00)[1]; RCVD_COUNT_THREE(0.00)[4]; MX_GOOD(-0.01)[cached: mx1-us2.ppe-hosted.com]; DKIM_TRACE(0.00)[xinuos.com:+]; NEURAL_HAM_SHORT(-0.95)[-0.947,0]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:26094, ipnet:144.202.224.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[xinuos.com:s=710E410E-A6C0-11E6-9C55-A8275FB0EC07]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[xinuos.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(-0.59)[asn: 26094(-2.90), country: US(-0.06)] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Apr 2019 15:09:27 -0000 Just to update this, I tried the report tool on a FreeBSD12 VM this morning= =20 with kyua version 0.13_4,3 (a later version than I had before) and it worke= d as expected.=20 I could get just the failed testcases reported with:=20 "kyua report --results-filter=3Dfailed" or even with "kyua report --results= -filter failed"=20 Alan - should I still report this as a bug on the earlier version?=20 Thanks,=20 Susan=20 From: "Susan Stanziano" =20 To: "NGie Cooper" =20 Cc: "freebsd-testing" =20 Sent: Wednesday, April 10, 2019 11:09:41 PM=20 Subject: Re: Kyua reporting tool question=20 kyua-0.13_2,3=20 Name: kyua=20 Version: 0.13_2,3=20 Installed on: Wed Dec 19 08:45:14 2018 EST=20 Origin: devel/kyua=20 Architecture: FreeBSD:10:amd64=20 Prefix: /usr/local=20 Categories: devel=20 Liceses: BSD3CLAUSE=20 Maintainer: jmmv@FreeBSD.org=20 WWW: https://github.com/jmmv/kyua/=20 This is installed on an OpenServer10 (based on FreeBSD10.3) VM.=20 I don't believe that modifications would have been made to kyua in the Xinu= os repository,=20 but I will ask about that.=20 Tomorrow I will install a FreeBSD12 VM and try running kyua there.=20 Thanks,=20 Susan=20 From: "NGie Cooper" =20 To: "Susan Stanziano" =20 Cc: "freebsd-testing" =20 Sent: Wednesday, April 10, 2019 10:12:26 PM=20 Subject: Re: Kyua reporting tool question=20 This doesn=E2=80=99t make sense=E2=80=A6 Could you share some more details,= e.g., what version of kyua are you using, where did you obtain it from, an= d does the version of kyua you=E2=80=99re using have any local modification= s?=20 Thank you!=20 -Enji=20 On Apr 10, 2019, at 7:10 PM, Susan Stanziano < susan.stanziano@xinuos.com >= wrote:=20 Hi NGie,=20 Thanks for the response. I tried this again with:=20 #kyua report --results-filter=3Dfailed --output=3Dtest-output.xml=20 and I still see "passed" test results in the output file.=20 If this is the correct syntax for creating an output file with only failed = test results,=20 then I'm beginning to wonder if somehow my script is wrong somewhere else a= nd=20 I will spend some time taking it apart step by step.=20 Susan=20 From: "NGie Cooper" < yaneurabeya@gmail.com >=20 To: "Susan Stanziano" < susan.stanziano@xinuos.com >=20 Cc: "freebsd-testing" < freebsd-testing@freebsd.org >=20 Sent: Wednesday, April 10, 2019 6:34:55 PM=20 Subject: Re: Kyua reporting tool question=20 > On Apr 10, 2019, at 3:30 PM, Susan Stanziano < susan.stanziano@xinuos.com= > wrote:=20 >=20 > Hi all,=20 >=20 > A question on the kyua reporting tool:=20 >=20 > kyua report --results-filter=20 >=20 > should not include passed tests if used without an argument.=20 >=20 > But, when I run it this way I get all results including passed.=20 >=20 > And if I run it with an argument (failed), I still get all results:=20 >=20 > kyua report --results-filter failed=20 >=20 > --results-filter types Comma-separated list of the test result types to i= nclude in the=20 > report. The ordering of the values is respected so that you can=20 > determine how you want the list of tests to be shown.=20 >=20 > The valid values are: =E2=80=98broken=E2=80=99, =E2=80=98failed=E2=80=99,= =E2=80=98passed=E2=80=99, =E2=80=98skipped=E2=80=99 and=20 > =E2=80=98xfail=E2=80=99. If the parameter supplied to the option is empty= , filtering=20 > is suppressed and all result types are shown in the report.=20 >=20 > The default value for this flag includes all the test results except=20 > the passed tests. Showing the passed tests by default clutters the=20 > report with too much information, so only abnormal conditions are=20 >=20 > Any insight is appreciated.=20 Hi Susan!=20 What happens if you use `kyua report =E2=80=94results-filter=3Dfailed` (not= e the `=3D`)?=20 -Enji=20