From owner-freebsd-current@FreeBSD.ORG Tue Sep 9 17:29:55 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 118A65F2; Tue, 9 Sep 2014 17:29:55 +0000 (UTC) Received: from mail-pa0-x234.google.com (mail-pa0-x234.google.com [IPv6:2607:f8b0:400e:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C74B1B93; Tue, 9 Sep 2014 17:29:54 +0000 (UTC) Received: by mail-pa0-f52.google.com with SMTP id kq14so2820259pab.11 for ; Tue, 09 Sep 2014 10:29:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=EUHcldLLedBcX4JjxGxFgQB1q8qNKh6TeT0y0TlVS4E=; b=gNrXWUxSX1YvAL65It18X1DvwVJlxr0LppW0DnK1oQyU0dsZ3+gw5Lj5yXyKTcvxxK xd4z6b8G6AR73tFKO++EShn++UNO5U0ssioc9SF5xvtJNNugtwcBrJSUUnM5G5x1u+A3 UaBuJCGN8qO0cHAH/EbQEuDD9OAaE8m5dDbmw/pH533yHMoKp3ROhnGZB6kNOeE6o+81 T65C68C7tlNtrgwtW0i6gyMaB3gm0OgkyEFzJ3Y5a2zq/mVSnmDEYhN/jiQrij5LXuyP bJMlBYyn0VS3vhXupTBjIWS/y/I9ZJqPppni6za88RD6XZpgjyrGP/K9I9uu+Lrx3cmE kC/w== X-Received: by 10.66.155.105 with SMTP id vv9mr58299342pab.61.1410283794288; Tue, 09 Sep 2014 10:29:54 -0700 (PDT) Received: from ?IPv6:2601:8:ab80:7d6:8dc3:2f93:9a63:ed06? ([2601:8:ab80:7d6:8dc3:2f93:9a63:ed06]) by mx.google.com with ESMTPSA id b14sm12355683pdk.2.2014.09.09.10.29.53 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 09 Sep 2014 10:29:53 -0700 (PDT) Content-Type: multipart/signed; boundary="Apple-Mail=_EC0D02CB-CBA7-4703-8A7D-C3B3234E3705"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Need help fixing tests in CURRENT From: yaneurabeya@gmail.com In-Reply-To: Date: Tue, 9 Sep 2014 10:29:52 -0700 Message-Id: References: <540F34B1.7040103@freebsd.org> <0D9BBC19-3139-4E66-B9E3-263ACB1C056B@gmail.com> To: Alan Somers X-Mailer: Apple Mail (2.1878.6) Cc: FreeBSD CURRENT , Allan Jude X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 17:29:55 -0000 --Apple-Mail=_EC0D02CB-CBA7-4703-8A7D-C3B3234E3705 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Sep 9, 2014, at 10:17, Alan Somers wrote: > On Tue, Sep 9, 2014 at 11:13 AM, wrote: >> On Sep 9, 2014, at 10:11, Allan Jude wrote: >>=20 >>> On 2014-09-09 13:05, Craig Rodrigues wrote: >>>> Hi, >>>>=20 >>>> I did a buildworld/installworld of current in a bhyve VM, and then = did the >>>> following. >>>>=20 >>>> pkg install devel/kyua >>>> cd /usr/tests >>>> kyua test >>>> kyua report >>>> kyua report-html >>>> kyua report-junit >>>>=20 >>>> I then published the results in Jenkins: >>>>=20 >>>> The results are here: >>>>=20 >>>> = https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD-tests2/lastCompletedB= uild/testReport/ >>>>=20 >>>> There seem to be 31 test failures. Can folks look into those >>>> and see if the tests need to be fixed >>>>=20 >>>> Right now I ran these steps manually, but hopefully over time >>>> we can automate more of this stuff. >>>=20 >>> Most of the errors (jot, m4, sed, etc) seems to be: >>>=20 >>> kyua-tap-tester: Configuration variables not supported; ignoring >>> 'has.cleanup=3Dfalse' >>> kyua-tap-tester: Configuration variables not supported; ignoring >>> 'unprivileged-user=3Dtests=92 >>=20 >> Not quite=85 >>=20 >>> while everything else about the test was successful. >>=20 >> Hi Craig, >> Taking a stab at the issues, most of the problems are because = of https://github.com/jmmv/kyua/issues/103 . The other issues with atf ( = https://github.com/jmmv/atf/issues/8 ) and kyua ( = https://github.com/jmmv/kyua/issues/109 ) are known (I=92m sure about = ATF, not 100% sure about kyua, but my gut instinct says yes). >> Thanks! >> -Garrett >=20 > There are still a few genuine failures. usr.bin.yacc.yacc_tests.main > is failing for reasons that have nothing to do with ATF or Kyua. So > are bin.pkill.pkill-j_test.main and bin.pkill.pgrep-j_test.main. Ugh=85 the pgrep/pkill -j issue is back ( = https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D191019 )=85 it was = fixed for a few weeks, so I suspect it=92s related to the changes that = kib@ and others have been making to how the proc bits in the kernel are = handled. The yacc issue is new ( = https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD-tests2/lastCompletedB= uild/testReport/usr.bin.yacc/yacc_tests/main/ ). It passes on my = CURRENT/i386 VM, but it=92s a bit out of date (2 weeks old). I=92ve = filed https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D193499 to = track the issue. Thanks! -Garrett $ (cd /usr/tests/usr.bin/yacc/; sudo kyua debug yacc_tests:main) ... ...ok ./yacc/ok_syntax1.tab.c ...ok ./yacc/ok_syntax1.tab.h ** testing ./pure_calc.y ...ok ./yacc/pure_calc.error ...ok ./yacc/pure_calc.output ...ok ./yacc/pure_calc.tab.c ...ok ./yacc/pure_calc.tab.h ** testing ./pure_error.y ...ok ./yacc/pure_error.error ...ok ./yacc/pure_error.output ...ok ./yacc/pure_error.tab.c ...ok ./yacc/pure_error.tab.h ** testing ./quote_calc.y ...ok ./yacc/quote_calc.error ...ok ./yacc/quote_calc.output ...ok ./yacc/quote_calc.tab.c ...ok ./yacc/quote_calc.tab.h ...ok ./yacc/quote_calc-s.error ...ok ./yacc/quote_calc-s.output ...ok ./yacc/quote_calc-s.tab.c ...ok ./yacc/quote_calc-s.tab.h ** testing ./quote_calc2.y ...ok ./yacc/quote_calc2.error ...ok ./yacc/quote_calc2.output ...ok ./yacc/quote_calc2.tab.c ...ok ./yacc/quote_calc2.tab.h ...ok ./yacc/quote_calc2-s.error ...ok ./yacc/quote_calc2-s.output ...ok ./yacc/quote_calc2-s.tab.c ...ok ./yacc/quote_calc2-s.tab.h ** testing ./quote_calc3.y ...ok ./yacc/quote_calc3.error ...ok ./yacc/quote_calc3.output ...ok ./yacc/quote_calc3.tab.c ...ok ./yacc/quote_calc3.tab.h ...ok ./yacc/quote_calc3-s.error ...ok ./yacc/quote_calc3-s.output ...ok ./yacc/quote_calc3-s.tab.c ...ok ./yacc/quote_calc3-s.tab.h ** testing ./quote_calc4.y ...ok ./yacc/quote_calc4.error ...ok ./yacc/quote_calc4.output ...ok ./yacc/quote_calc4.tab.c ...ok ./yacc/quote_calc4.tab.h ...ok ./yacc/quote_calc4-s.error ...ok ./yacc/quote_calc4-s.output ...ok ./yacc/quote_calc4-s.tab.c ...ok ./yacc/quote_calc4-s.tab.h ** testing ./varsyntax_calc1.y ...ok ./yacc/varsyntax_calc1.error ...ok ./yacc/varsyntax_calc1.output ...ok ./yacc/varsyntax_calc1.tab.c ...ok ./yacc/varsyntax_calc1.tab.h yacc_tests:main -> passed $ uname -a FreeBSD isilon-fuji-current.local 11.0-CURRENT FreeBSD 11.0-CURRENT #7 = r270227+e6e86c2(isilon-atf)-dirty: Sat Aug 23 02:26:59 PDT 2014 = ngie@isilon-fuji-current.local:/usr/obj/usr/src/sys/FUJI i386 --Apple-Mail=_EC0D02CB-CBA7-4703-8A7D-C3B3234E3705 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJUDzkQAAoJEMZr5QU6S73eXRQIAIILHF6brN+KVSSjUa+5fqH+ 0TeivLxmouvwrcxhrTrU/Yev2V1NIRe5zjJ7QvvgsVFudFzzJcl/NZE61Vqq9jm9 yJ31XDknBAOubRSVKjAmb6HeSvuHLRbhM7iSpi/IHYRBb5pDp29SlmjjVRC7GnF1 OoMUmVRH+IwDGlQym0U2R/TJ/5GbLp2IJDFcJXNIQ56IsPss6izLNmymxY0tu4M9 mujG/t1e2z+KW/TRRy3gA9SBa/hR7syWw2U+2zujnlPIVmq0QTLvswEyRJG5Njes CXnEjwtL3RRJKnuYHBRxb/Ov7CiDzGFHCa650RIQod+3UCLbhHaPho2wVqRP5rg= =YSg4 -----END PGP SIGNATURE----- --Apple-Mail=_EC0D02CB-CBA7-4703-8A7D-C3B3234E3705--