Date: Sun, 11 May 1997 19:09:52 +0200 (MET DST) From: Eivind Eklund <perhaps@yes.no> To: Terry Lambert <terry@lambert.org> Cc: current@FreeBSD.ORG Subject: Regression tests (was Re: A 3.0-current SNAP building machine has been found!) Message-ID: <199705111709.TAA06018@bitbox.follo.net> In-Reply-To: Terry Lambert's message of Sat, 10 May 1997 12:43:05 -0700 (MST) References: <18077.863238129@time.cdrom.com> <199705101943.MAA04327@phaeton.artisoft.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Cc: to announce removed - PLEASE do this for any other followups in the same thread! > > I suppose that Terry will now suggest some sort of voting system and I > > can't even say that it's such a bad idea (just so long as I don't have > > to write the vote collection and tabulation software :-). > > > > Comments? > > I was actually against using "voting"; I prefer forcing developers to > verify that compiles work and the resulting code does not trivially > fail prior to it being committed. That way the tree would always work; > the best you can get out of "voting" is "provided with high confidence > that it might work". Regression tests. We really should have this integrated into the build system. I have been thinking a bit about this, but haven't come up with any system I feel entirely satisfied with yet, and thus haven't brought it up for discussion (Besides, I have a lot of other FreeBSD-related projects not completed yet, and thus feel a bit timid). For my personal projects, I use one of the following systems (depending on the size/complexity of the project): -DTEST - every source file define a main(), and calling this return OK if the test passes or is not present. If the test fails, the program print a message and exit with failure. Advantages: Extremely simple, requires thought about writing testable code Disadvantages: Often too simple, requires thought about writing testable code -DREGRESS - Each object file exports <filename without.c>_regress(), and a new source file is created calling each of these. The auto-generated file is compiled and called. Advantages: Fairly simple, allow dependencies between modules. Still requires thought about testability. Disadvantages: Often too simple. None of the above adapt well to a large established code base. :-( I'd guess we need a combination of something like -DREGRESS, as well as a directory with shell scripts to run to test. These could then do the necessary creation of datafiles and actual testing. They should probably be run in a pre-set order, to allow a later test to depend on a former test and the datafiles created by it. (This might mean that a Makefile for them would be a good idea.) Anyway; being able to do a 'make regress' on the entire FreeBSD source tree and know all new code and a lot of old code got tested would be a _good_ feeling. Eivind.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199705111709.TAA06018>