From owner-freebsd-testing@freebsd.org Thu Apr 11 15:21:21 2019 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A72F01586EC9 for ; Thu, 11 Apr 2019 15:21:21 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 85D8F6B81E; Thu, 11 Apr 2019 15:21:20 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pf1-x42d.google.com with SMTP id 10so3617858pfo.5; Thu, 11 Apr 2019 08:21:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=OuORxd92IoSXC4bqnam+HDtYtHXQXQkrtMaXa4EYmbE=; b=fWgiwm86yHBUSQwnu22AAYp3RQKwLDzbRWHYW6lAn5yaxLQaw+QMcqsj71C1LITk2B 8V5iEGo7NNQA73CcpxVCawADJveqtIM133m5GmzzJhFr0LZNWiYpdUwFoZeDVQwqIULt X3bDEHUAExV3jdAXBcOfH51DcseXe29145WmHZFNCVcLYZWM2BtKCp4Fq12XeH2WanF0 JXm9NPttXF9LpEx/yqzYS7EvGv18WO7ivKjNVsg9Y8r+NecsVROdSLNR7X+A6GCcRAlY iwtPxHy2TQOUcM7eKkfUt3l7hDG5D+seIwU6FUfRZWRAyID1c+0b3EwLR5TvOCaX3GoW sU0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=OuORxd92IoSXC4bqnam+HDtYtHXQXQkrtMaXa4EYmbE=; b=Inq37i5grXpJAe0UEQClbdNihxiqUSU6gI0xT3xnwqFPSgSRV/dFk2OjzoT0JUn2YE gVKa/w/G2x5hPT/pcIZjmSNkCecbb3B5aZnL5liDZXOAgHD52XuEE/S3atUfZoJdnv2h hvxfb+uziHaOxeaDPJR8JHaaNVzTgg9+nngdS0AK24uxUWR1quAcZ1Wu40flvdDP3yVH rkW17ztFjthv1D2+HpEAWUShlR4xeNbEAgQeU0blWDm/eQytCYY4qVzWlD7dJyMUfDoK SzgUh8IjFcDYwASubSRjWljdjU/DNFFdmacM6bZ6ZqyRk047kO8ayBS1hZSSWFDb7lQU Yh3A== X-Gm-Message-State: APjAAAWSr4Y8GJUBoUcaVwoP58qdEaPoakemipVZb+XQYH/YCUx21H+Y HlVUXu+NUbeGXFvrK7rPrLmakW46fzY= X-Google-Smtp-Source: APXvYqwFnWwMslLLzJusIsxt8QqNyOJbWg9cR81XQxvfEtAXNOJGiWpdBUfVFPdXRS33uukvzKK3fA== X-Received: by 2002:a62:5a42:: with SMTP id o63mr51201696pfb.170.1554996079199; Thu, 11 Apr 2019 08:21:19 -0700 (PDT) Received: from [192.168.20.22] (c-73-19-52-228.hsd1.wa.comcast.net. [73.19.52.228]) by smtp.gmail.com with ESMTPSA id k4sm53110954pgk.1.2019.04.11.08.21.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Apr 2019 08:21:18 -0700 (PDT) Mime-Version: 1.0 (1.0) Subject: Re: Kyua reporting tool question From: Enji Cooper X-Mailer: iPhone Mail (16E227) In-Reply-To: <989313104.2320168.1554995363573.JavaMail.zimbra@xinuos.com> Date: Thu, 11 Apr 2019 08:21:17 -0700 Cc: asomers@freebsd.org, freebsd-testing Message-Id: References: <555587672.2279585.1554935416410.JavaMail.zimbra@xinuos.com> <80FAC842-ECCB-438F-BBC4-0995A7C1D7B1@gmail.com> <359307252.2304183.1554948627748.JavaMail.zimbra@xinuos.com> <0BD361D0-3B1F-4B3D-AD65-A85153E745AA@gmail.com> <1433014530.2305640.1554952181864.JavaMail.zimbra@xinuos.com> <989313104.2320168.1554995363573.JavaMail.zimbra@xinuos.com> To: Susan Stanziano X-Rspamd-Queue-Id: 85D8F6B81E X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=fWgiwm86; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of yaneurabeya@gmail.com designates 2607:f8b0:4864:20::42d as permitted sender) smtp.mailfrom=yaneurabeya@gmail.com X-Spamd-Result: default: False [-4.31 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(0.00)[+ip6:2607:f8b0:4000::/36]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; RBL_SEM_IPV6(1.00)[d.2.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.bl.ipv6.spameatingmonkey.net]; DKIM_TRACE(0.00)[gmail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DMARC_POLICY_ALLOW(0.00)[gmail.com,none]; NEURAL_HAM_SHORT(-0.99)[-0.991,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(0.00)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE(-2.81)[ip: (-8.87), ipnet: 2607:f8b0::/32(-2.95), asn: 15169(-2.19), country: US(-0.06)]; BAD_REP_POLICIES(0.10)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[d.2.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Apr 2019 15:21:21 -0000 > On Apr 11, 2019, at 08:09, Susan Stanziano wr= ote: >=20 >=20 > Just to update this, I tried the report tool on a FreeBSD12 VM this mornin= g >=20 > with kyua version 0.13_4,3 (a later version than I had before) and it work= ed as expected. >=20 > I could get just the failed testcases reported with: >=20 > "kyua report --results-filter=3Dfailed" or even with "kyua report --result= s-filter failed"=20 >=20 > Alan - should I still report this as a bug on the earlier version? Hi Susan, Could you please create a bug on bugs.freebsd.org and CC dim@? There are= a number of kyua integration tests that fail on 10.x and 11.x. I wonder if t= he issue is a part of the root cause. Thank you! -Enji > Thanks, > Susan >=20 > From: "Susan Stanziano" > To: "NGie Cooper" > Cc: "freebsd-testing" > Sent: Wednesday, April 10, 2019 11:09:41 PM > Subject: Re: Kyua reporting tool question >=20 >=20 > kyua-0.13_2,3 > Name: kyua > Version: 0.13_2,3 > Installed on: Wed Dec 19 08:45:14 2018 EST > Origin: devel/kyua > Architecture: FreeBSD:10:amd64 > Prefix: /usr/local > Categories: devel > Liceses: BSD3CLAUSE > Maintainer: jmmv@FreeBSD.org > WWW: https://github.com/jmmv/kyua/ >=20 > This is installed on an OpenServer10 (based on FreeBSD10.3) VM. > I don't believe that modifications would have been made to kyua in the Xin= uos repository, > but I will ask about that. >=20 > Tomorrow I will install a FreeBSD12 VM and try running kyua there. >=20 > Thanks, > Susan > From: "NGie Cooper" > To: "Susan Stanziano" > Cc: "freebsd-testing" > Sent: Wednesday, April 10, 2019 10:12:26 PM > Subject: Re: Kyua reporting tool question >=20 > This doesn=E2=80=99t make sense=E2=80=A6 Could you share some more d= etails, e.g., what version of kyua are you using, where did you obtain it fr= om, and does the version of kyua you=E2=80=99re using have any local modific= ations? > Thank you! > -Enji >=20 > On Apr 10, 2019, at 7:10 PM, Susan Stanziano w= rote: >=20 > Hi NGie, >=20 > Thanks for the response. I tried this again with: >=20 > #kyua report --results-filter=3Dfailed --output=3Dtest-output.xml >=20 > and I still see "passed" test results in the output file. >=20 > If this is the correct syntax for creating an output file with only failed= test results, > then I'm beginning to wonder if somehow my script is wrong somewhere else a= nd > I will spend some time taking it apart step by step. >=20 > Susan >=20 > From: "NGie Cooper" > To: "Susan Stanziano" > Cc: "freebsd-testing" > Sent: Wednesday, April 10, 2019 6:34:55 PM > Subject: Re: Kyua reporting tool question >=20 > > On Apr 10, 2019, at 3:30 PM, Susan Stanziano wrote: > >=20 > > Hi all,=20 > >=20 > > A question on the kyua reporting tool:=20 > >=20 > > kyua report --results-filter=20 > >=20 > > should not include passed tests if used without an argument.=20 > >=20 > > But, when I run it this way I get all results including passed.=20 > >=20 > > And if I run it with an argument (failed), I still get all results:=20 > >=20 > > kyua report --results-filter failed=20 > >=20 > > --results-filter types Comma-separated list of the test result types to i= nclude in the > > report. The ordering of the values is respected so that you can= > > determine how you want the list of tests to be shown. > >=20 > > The valid values are: =E2=80=98broken=E2=80=99, =E2=80=98failed=E2= =80=99, =E2=80=98passed=E2=80=99, =E2=80=98skipped=E2=80=99 and > > =E2=80=98xfail=E2=80=99. If the parameter supplied to the optio= n is empty, filtering > > is suppressed and all result types are shown in the report. > >=20 > > The default value for this flag includes all the test results ex= cept > > the passed tests. Showing the passed tests by default clutters t= he > > report with too much information, so only abnormal conditions ar= e=20 > >=20 > > Any insight is appreciated.=20 >=20 >=20 > Hi Susan! > What happens if you use `kyua report =E2=80=94results-filter=3Dfai= led` (note the `=3D`)? > -Enji >=20 >=20