From owner-freebsd-current@FreeBSD.ORG Tue Sep 9 17:17:19 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BFD4BF7C; Tue, 9 Sep 2014 17:17:19 +0000 (UTC) Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 33685A63; Tue, 9 Sep 2014 17:17:19 +0000 (UTC) Received: by mail-wg0-f44.google.com with SMTP id y10so2949558wgg.3 for ; Tue, 09 Sep 2014 10:17:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=NOIxnB5atevvMq2CqQCWzBRjbbD5T/bVMlN01nTqg+M=; b=OjVuzQtyyjtIwkPXKRtscy6efScvWwjBJTeXRqIbQ+jPKa8X0UMUs4QrHW+/U/LGqA QKj17TOReGB0OGDUzy4nkGgGHNmocpg1IAEzsZUJI7dcCkDuWhjM/PLkK9x/fAz4Ia/n mlfWicJKGvlzvXv7TBDsTH6Vh0iYHbw8rNgOOMW8er4J9aWaTqxpgDQgGe19kSIcSPiY 6CoQHGQUjuOB3JTEG4SSkCTTQdZyX8I01OjJUBS4VJAN27TUoYlpmLCDdBA2AZ5A+UR7 f1Uj8E4c981lrh0XzuOVQXXfzerRxJQ5cw5cQ8yFvRrabSYq4sv0MJje8cjFU+NAcnln qSGg== MIME-Version: 1.0 X-Received: by 10.194.60.240 with SMTP id k16mr6034606wjr.109.1410283037425; Tue, 09 Sep 2014 10:17:17 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.126.1 with HTTP; Tue, 9 Sep 2014 10:17:17 -0700 (PDT) In-Reply-To: <0D9BBC19-3139-4E66-B9E3-263ACB1C056B@gmail.com> References: <540F34B1.7040103@freebsd.org> <0D9BBC19-3139-4E66-B9E3-263ACB1C056B@gmail.com> Date: Tue, 9 Sep 2014 11:17:17 -0600 X-Google-Sender-Auth: lDZCV6lkK9U92wAQI6rP4TK-_AU Message-ID: Subject: Re: Need help fixing tests in CURRENT From: Alan Somers To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD CURRENT , Allan Jude X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Sep 2014 17:17:19 -0000 On Tue, Sep 9, 2014 at 11:13 AM, wrote: > On Sep 9, 2014, at 10:11, Allan Jude wrote: > >> On 2014-09-09 13:05, Craig Rodrigues wrote: >>> Hi, >>> >>> I did a buildworld/installworld of current in a bhyve VM, and then did = the >>> following. >>> >>> pkg install devel/kyua >>> cd /usr/tests >>> kyua test >>> kyua report >>> kyua report-html >>> kyua report-junit >>> >>> I then published the results in Jenkins: >>> >>> The results are here: >>> >>> https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD-tests2/lastComplet= edBuild/testReport/ >>> >>> There seem to be 31 test failures. Can folks look into those >>> and see if the tests need to be fixed >>> >>> Right now I ran these steps manually, but hopefully over time >>> we can automate more of this stuff. >> >> Most of the errors (jot, m4, sed, etc) seems to be: >> >> kyua-tap-tester: Configuration variables not supported; ignoring >> 'has.cleanup=3Dfalse' >> kyua-tap-tester: Configuration variables not supported; ignoring >> 'unprivileged-user=3Dtests=E2=80=99 > > Not quite=E2=80=A6 > >> while everything else about the test was successful. > > Hi Craig, > Taking a stab at the issues, most of the problems are because of = https://github.com/jmmv/kyua/issues/103 . The other issues with atf ( https= ://github.com/jmmv/atf/issues/8 ) and kyua ( https://github.com/jmmv/kyua/i= ssues/109 ) are known (I=E2=80=99m sure about ATF, not 100% sure about kyua= , but my gut instinct says yes). > Thanks! > -Garrett There are still a few genuine failures. usr.bin.yacc.yacc_tests.main is failing for reasons that have nothing to do with ATF or Kyua. So are bin.pkill.pkill-j_test.main and bin.pkill.pgrep-j_test.main.