From owner-cvs-src@FreeBSD.ORG Tue Oct 4 10:44:57 2005 Return-Path: X-Original-To: cvs-src@FreeBSD.org Delivered-To: cvs-src@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C8F9516A41F; Tue, 4 Oct 2005 10:44:57 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [204.156.12.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7929243D46; Tue, 4 Oct 2005 10:44:57 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by cyrus.watson.org (Postfix) with ESMTP id 2406C46B80; Tue, 4 Oct 2005 06:44:57 -0400 (EDT) Date: Tue, 4 Oct 2005 11:44:57 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Mike Silbersack In-Reply-To: <20051004032213.R11746@odysseus.silby.com> Message-ID: <20051004114119.G69774@fledge.watson.org> References: <200510040452.j944q0rI020011@gw.catspoiler.org> <20051004032213.R11746@odysseus.silby.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: cvs-src@FreeBSD.org, Don Lewis , src-committers@FreeBSD.org, cvs-all@FreeBSD.org Subject: Re: cvs commit: src/sys/ufs/ffs ffs_alloc.c X-BeenThere: cvs-src@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: CVS commit messages for the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2005 10:44:58 -0000 On Tue, 4 Oct 2005, Mike Silbersack wrote: > I've cc'd rwatson, since he seems to be the regression test king, and > may have some input on this idea. King, eh? :-) There's definitely a niche here for someone who wants to pick up where Nik left off integrating the regression test suite a bit more with a unified run and reporting system. And for someone to take ownership of the system in terms of running it regularly to actually detect regressions, generate reports, and so on. While I've been adding a lot of regression tests, I know relatively little about testing, and so while the tests work, they probably leave quite a bit to be desired. For example, they tend to fail entirely on first detected problem, whereas it might be desirable to run more like configure and test various elements and report on each independently, so that tests aren't skipped when a first failure is hit. Also, I'd love to see some instructions somewhere about how it would be best to format output for a test, and possible a libtest(3) to link against in order to get formatted output routines and automated cleanup routines for when tests end (i.e., please rm -Rf /tmp/testdir.XXXXXX, even if the test process dies). Finally, a number of tests require similar pieces of infrastructure: a UFS file system image, or a configured network interface with a local IP address, etc. Things that could be provided by a framework in a modular way, rather than reproducing them in every regression test. Robert N M Watson