From owner-freebsd-testing@freebsd.org Thu Apr 11 15:12:23 2019 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 53E3A1586A59 for ; Thu, 11 Apr 2019 15:12:23 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com [209.85.167.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 67C446B4AB for ; Thu, 11 Apr 2019 15:12:22 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf1-f50.google.com with SMTP id h5so2079661lfm.1 for ; Thu, 11 Apr 2019 08:12:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=SlJKs+DObaeBA38vR0SbbTWCyy6Se07wz1nracBJhgE=; b=EviVMSRTyBzLs/bVFQUwdvzgfvDWEXLxZMVJyoj9t2VCCDh0Nf+XYeyrpABx7k1bHc D0NgZPPwD6Twud8QqgziKsUCnz+PMlgqkfVYsw9BLZs8yIe8bPpSyvFUBkVGlLyBAskU ahqKidsQyEoHlD7u8/ggeEGUIdfzN/olNL+dAk2j+R23h0DBq3qk8QEgsTmV6iTaPy7Y IRSRXmbT6wmaVd9JTU4UbjMHVd3GPBl9f2xEaBUO71o/5qacqaRqmMSpmM2/FrGuJLuJ lseUFXQC6tpnrk0Gd/2IW36POCp795FRx2eHCUEEZDsYKb5jUZGW2lXFK9b1pjyDGELm etAQ== X-Gm-Message-State: APjAAAUl/9eVGxZA1yxnIfc5XhgIEMdmBYCsDYDGmJSlaWkTGvYFIgFU dtkr3oBwuMmdFroho0PT8dkpimvtlwT5Smq6ETI= X-Google-Smtp-Source: APXvYqxXn1qFGzhVM+vleryL8Nn2mPO7slrldHobg1+j5o/8LQh7Prs2vmKe5mw3/a5+ZuvJSyKXvd18JXwEr7AatpI= X-Received: by 2002:a19:2804:: with SMTP id o4mr12501372lfo.139.1554995535034; Thu, 11 Apr 2019 08:12:15 -0700 (PDT) MIME-Version: 1.0 References: <555587672.2279585.1554935416410.JavaMail.zimbra@xinuos.com> <80FAC842-ECCB-438F-BBC4-0995A7C1D7B1@gmail.com> <359307252.2304183.1554948627748.JavaMail.zimbra@xinuos.com> <0BD361D0-3B1F-4B3D-AD65-A85153E745AA@gmail.com> <1433014530.2305640.1554952181864.JavaMail.zimbra@xinuos.com> <989313104.2320168.1554995363573.JavaMail.zimbra@xinuos.com> In-Reply-To: <989313104.2320168.1554995363573.JavaMail.zimbra@xinuos.com> From: Alan Somers Date: Thu, 11 Apr 2019 09:12:02 -0600 Message-ID: Subject: Re: Kyua reporting tool question To: Susan Stanziano Cc: NGie Cooper , freebsd-testing Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 67C446B4AB X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.50 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-4.34 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-testing@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; MIME_TRACE(0.00)[0:+]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[50.167.85.209.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.97)[-0.973,0]; RCVD_TLS_LAST(0.00)[]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; IP_SCORE(-1.36)[ip: (-0.68), ipnet: 209.85.128.0/17(-3.87), asn: 15169(-2.19), country: US(-0.06)]; FREEMAIL_CC(0.00)[gmail.com] X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Apr 2019 15:12:23 -0000 Probably not, if you can't reproduce the bug on the current version of kyua= . On Thu, Apr 11, 2019 at 9:09 AM Susan Stanziano wrote: > > > Just to update this, I tried the report tool on a FreeBSD12 VM this morni= ng > > with kyua version 0.13_4,3 (a later version than I had before) and it wor= ked as expected. > > I could get just the failed testcases reported with: > > "kyua report --results-filter=3Dfailed" or even with "kyua report --resul= ts-filter failed" > > Alan - should I still report this as a bug on the earlier version? > > Thanks, > Susan > > ________________________________ > From: "Susan Stanziano" > To: "NGie Cooper" > Cc: "freebsd-testing" > Sent: Wednesday, April 10, 2019 11:09:41 PM > Subject: Re: Kyua reporting tool question > > > kyua-0.13_2,3 > Name: kyua > Version: 0.13_2,3 > Installed on: Wed Dec 19 08:45:14 2018 EST > Origin: devel/kyua > Architecture: FreeBSD:10:amd64 > Prefix: /usr/local > Categories: devel > Liceses: BSD3CLAUSE > Maintainer: jmmv@FreeBSD.org > WWW: https://github.com/jmmv/kyua/ > > This is installed on an OpenServer10 (based on FreeBSD10.3) VM. > I don't believe that modifications would have been made to kyua in the Xi= nuos repository, > but I will ask about that. > > Tomorrow I will install a FreeBSD12 VM and try running kyua there. > > Thanks, > Susan > ________________________________ > From: "NGie Cooper" > To: "Susan Stanziano" > Cc: "freebsd-testing" > Sent: Wednesday, April 10, 2019 10:12:26 PM > Subject: Re: Kyua reporting tool question > > This doesn=E2=80=99t make sense=E2=80=A6 Could you share some more detail= s, e.g., what version of kyua are you using, where did you obtain it from, = and does the version of kyua you=E2=80=99re using have any local modificati= ons? > Thank you! > -Enji > > On Apr 10, 2019, at 7:10 PM, Susan Stanziano = wrote: > > Hi NGie, > > Thanks for the response. I tried this again with: > > #kyua report --results-filter=3Dfailed --output=3Dtest-output.xml > > and I still see "passed" test results in the output file. > > If this is the correct syntax for creating an output file with only faile= d test results, > then I'm beginning to wonder if somehow my script is wrong somewhere else= and > I will spend some time taking it apart step by step. > > Susan > > ________________________________ > From: "NGie Cooper" > To: "Susan Stanziano" > Cc: "freebsd-testing" > Sent: Wednesday, April 10, 2019 6:34:55 PM > Subject: Re: Kyua reporting tool question > > > On Apr 10, 2019, at 3:30 PM, Susan Stanziano wrote: > > > > Hi all, > > > > A question on the kyua reporting tool: > > > > kyua report --results-filter > > > > should not include passed tests if used without an argument. > > > > But, when I run it this way I get all results including passed. > > > > And if I run it with an argument (failed), I still get all results: > > > > kyua report --results-filter failed > > > > --results-filter types Comma-separated list of the test result types to= include in the > > report. The ordering of the values is respected so that you ca= n > > determine how you want the list of tests to be shown. > > > > The valid values are: =E2=80=98broken=E2=80=99, =E2=80=98failed= =E2=80=99, =E2=80=98passed=E2=80=99, =E2=80=98skipped=E2=80=99 and > > =E2=80=98xfail=E2=80=99. If the parameter supplied to the opti= on is empty, filtering > > is suppressed and all result types are shown in the report. > > > > The default value for this flag includes all the test results e= xcept > > the passed tests. Showing the passed tests by default clutters= the > > report with too much information, so only abnormal conditions a= re > > > > Any insight is appreciated. > > > Hi Susan! > What happens if you use `kyua report =E2=80=94results-filter=3Dfa= iled` (note the `=3D`)? > -Enji > > >