&os; Test Suite

From owner-svn-doc-head@FreeBSD.ORG Wed Apr 2 10:25:12 2014 Return-Path: Delivered-To: svn-doc-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A7E538C3; Wed, 2 Apr 2014 10:25:12 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 945FC993; Wed, 2 Apr 2014 10:25:12 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s32APCGs012755; Wed, 2 Apr 2014 10:25:12 GMT (envelope-from pgj@svn.freebsd.org) Received: (from pgj@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s32APC40012754; Wed, 2 Apr 2014 10:25:12 GMT (envelope-from pgj@svn.freebsd.org) Message-Id: <201404021025.s32APC40012754@svn.freebsd.org> From: Gabor Pali Date: Wed, 2 Apr 2014 10:25:12 +0000 (UTC) To: doc-committers@freebsd.org, svn-doc-all@freebsd.org, svn-doc-head@freebsd.org Subject: svn commit: r44418 - head/en_US.ISO8859-1/htdocs/news/status X-SVN-Group: doc-head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-doc-head@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SVN commit messages for the doc tree for head List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 10:25:12 -0000 Author: pgj Date: Wed Apr 2 10:25:12 2014 New Revision: 44418 URL: http://svnweb.freebsd.org/changeset/doc/44418 Log: - Add 2014Q1 status report on Kyua Submitted by: jmmv Modified: head/en_US.ISO8859-1/htdocs/news/status/report-2014-01-2014-03.xml Modified: head/en_US.ISO8859-1/htdocs/news/status/report-2014-01-2014-03.xml ============================================================================== --- head/en_US.ISO8859-1/htdocs/news/status/report-2014-01-2014-03.xml Wed Apr 2 09:21:21 2014 (r44417) +++ head/en_US.ISO8859-1/htdocs/news/status/report-2014-01-2014-03.xml Wed Apr 2 10:25:12 2014 (r44418) @@ -18,7 +18,7 @@

Thanks to all the reporters for the excellent work! This report - contains 7 entries and we hope you enjoy reading it.

+ contains 8 entries and we hope you enjoy reading it.

The deadline for submissions covering between April and June 2014 is July 7th, 2014.

@@ -37,6 +37,12 @@ + bin + + Userland Programs + + + ports Ports @@ -445,4 +451,90 @@ Sandvine, Inc + + + &os; Test Suite + + + + + Julio + Merino + + jmmv@FreeBSD.org + + + + + Project page + Testing cluster + Roadmap for the project + AsiaBSDCon 2014 tutorial materials + + + +

The &os; Test Suite project aims to equip &os; with a + comprehensive collection of tests that are easy to run out of + the box and during the development of the system. The test + suite is installed into /usr/tests/ and the + kyua(1) command-line tool (devel/kyua in the + Ports Collection) is used to run them. See the project page for + more details.

+ +

Since the last status report, we have been hard at work + polishing the framework in many different areas. The highlights + are:

+ +

A roadmap for the project has been prepared and published, + see links.
Many tests have been added to the test suite thanks to the + work of various developers and, in particular, a good bunch of + old tests from src/tools/regression/ have been + incorporated into the new test suite. As of this writing, + there are 509 test cases continuously running.
The testing infrastructure in stable/10 branch has + been synced to head. It should now be possible to + seamlessly MFC changes to the stable branch along with their + tests, if any.
The testing cluster, which only issued amd64 + builds, has been extended to perform i386 builds as + well. Additionally, a canary machine has been put in place so + that changes to the cluster configuration can be properly + validated before deployment.
A tutorial on Kyua and the &os; Test Suite was given at + AsiaBSDCon 2014. The tutorial materials are available for + public consumption, please consult the links.
Both Kyua's and ATF's upstream sites have been moved to + GitHub, mostly due to the discontinuation of file downloads in + Google Code.

+ + + + Enable the build of the test suite by default. + + Add alerting for failed or missing test runs from the + testing cluster. + + Add bhyve support to the testing cluster for faster + turnaround times. + + Simplify and improve Kyua HTML reports. In particular, + reports will be coalesced into single HTML files for easier + management and will include more useful details for debugging. + Such details are the revision at which the system was built, + date and duration of the whole run, or list of installed + packages, to mention a few examples. + + Add JUnit XML output to Kyua for better integration with + Jenkins. This work is actively ongoing and should be ready for + prime time at BSDCan 2014. + +