Date: Thu, 20 Mar 2014 17:13:08 +0000 (UTC) From: Alan Somers <asomers@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r263428 - head/share/man/man7 Message-ID: <201403201713.s2KHD8gx030078@svn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: asomers Date: Thu Mar 20 17:13:07 2014 New Revision: 263428 URL: http://svnweb.freebsd.org/changeset/base/263428 Log: share/man/man7/tests.7 Added a section on test suite configuration, and cleaned up up grammar errors and awkward prose. The config variables were discussed on freebsd-testing. Discussed with: Garrett Cooper, jmmv MFC after: 2 weeks Sponsored by: Spectra Logic Corporation Modified: head/share/man/man7/tests.7 Modified: head/share/man/man7/tests.7 ============================================================================== --- head/share/man/man7/tests.7 Thu Mar 20 17:10:01 2014 (r263427) +++ head/share/man/man7/tests.7 Thu Mar 20 17:13:07 2014 (r263428) @@ -26,7 +26,7 @@ .\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN .\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .\" -.Dd October 19, 2013 +.Dd March 20, 2014 .Dt TESTS 7 .Os .Sh NAME @@ -36,30 +36,29 @@ The .Fx test suite provides a collection of automated tests for two major purposes. -On the one hand, the test suite aids +On one hand, the test suite aids .Em developers -in catching bugs and regressions in the code when they performing modifications -to the source tree. -On the other hand, the test suite allows +to detect bugs and regressions when they modify the source tree. On the other +hand, it allows .Em end users (and, in particular, system administrators) to verify that fresh installations of the .Fx -operating system behave correctly in their hardware platform and also to ensure -that the system does not suffer from regressions during regular system -operation and maintenance. +operating system behave correctly on their hardware platform and also to ensure +that the system does not suffer from regressions during regular operation and +maintenance. .Pp The .Fx -test suite is installed under the +test suite is installed in the .Pa /usr/tests hierarchy. .Pp -This manual page describes how to execute the test suite and how to configure +This manual page describes how to run the test suite and how to configure some of its optional features. .Ss When to run the tests? Before diving into the details of how to run the test suite, here are some -scenarios in which you should be running them: +scenarios in which you should run it: .Bl -bullet -offset indent .It After a fresh installation of @@ -72,8 +71,7 @@ to a different version to ensure that th hardware platform and that the upgrade did not introduce regressions in your configuration. .It -After performing changes to the source tree to catch any bugs and/or regressions -introduced by the modifications. +After modifying the source tree to detect any new bugs and/or regressions. .It Periodically, maybe from a .Xr cron 8 @@ -82,18 +80,18 @@ third-party packages or manual modificat introduce unexpected failures. .El .Ss Running the tests -First of all, you will need to install the -.Sq ports/devel/kyua -package. -.Pp -Use the following command to run the whole test suite: +First, you will need to install the +.Sq devel/kyua +package from +.Xr ports 7 . +Then use the following command to run the whole test suite: .Bd -literal -offset indent $ kyua test -k /usr/tests/Kyuafile .Ed .Pp -The above will go through all test programs in +The above will iterate through all test programs in .Pa /usr/tests -recursively, execute them, store their results and debugging data in Kyua +recursively, execute them, store their results and debugging data in Kyua's database (by default in .Pa ~/.kyua/store.db ) , and print a summary of the results. @@ -111,15 +109,15 @@ utilities: $ kyua test -k /usr/tests/Kyuafile bin/cp usr.bin/cut .Ed .Ss Obtaining reports of the tests execution -Additional information of the results of the execution can be later extracted -from the database by using the various reporting commands of Kyua. -For example, the following would extract a plain-text report of the executed +Additional information about the test results can be retrieved +by using Kyua's various reporting commands. +For example, the following would print a plain-text report of the executed tests and show which ones failed: .Bd -literal -offset indent $ kyua report .Ed .Pp -This other example would generate an HTML report ready to be published on a +This example would generate an HTML report ready to be published on a web server: .Bd -literal -offset indent $ kyua report-html --output ~/public_html/tests @@ -128,30 +126,47 @@ $ kyua report-html --output ~/public_htm For further details on the command-line interface of Kyua, please refer to its manual page .Xr kyua 1 . -.\".Ss Configuring the tests -.\"Some test cases in the -.\".Fx -.\"test suite require the administrator to manually set up some configuration -.\"properties before they can run. -.\"Unless these properties are defined, the tests that require them will be marked -.\"as skipped and thus they will not be really executed. -.\".Pp -.\"Test suites are configured by defining the values to their configuration -.\"variables in -.\".Pa /usr/local/etc/kyua/kyua.conf . -.\"The format of this file is detailed in -.\".Xr kyua.conf 5 . -.\".Pp -.\"The following configuration variables are available in the -.\".Fx -.\"test suite: -.\".Bl -tag -.\".It NONE REGISTERED YET -.\"TBD. +.Ss Configuring the tests +Some test cases in the +.Fx +test suite require manual configuration by the administrator before they can be +run. Unless certain properties are defined, the tests that require them will +be skipped. +.Pp +Test suites are configured by defining their configuration +variables in +.Pa /usr/local/etc/kyua/kyua.conf . +The format of this file is detailed in +.Xr kyua.conf 5 . +.Pp +The following configuration variables are available in the +.Fx +test suite: +.Bl -tag -width "allow_sysctl_side_effects" +.It allow_devfs_side_effects +If defined, enables tests that may destroy and recreate semipermanent device +nodes, like disk devices. Without this variable, tests may still create and +destroy devices nodes that are normally transient, like /dev/tap* and +/dev/pts*, as long as they clean them up afterwards. However, tests that +require this variable have a relaxed cleanup requirement; they must recreate +any devices that they destroyed, but not necessarily with the same devnames. +.It allow_sysctl_side_effects +Enables tests that change globally significant +.Xr sysctl 8 +variables. The tests will undo any changes in their cleanup phases. +.It disks +Must be set to a space delimited list of disk device nodes. Tests that need +destructive access to disks must use these devices. Tests are not required to +preserve any data present on these disks. +.It fibs +Must be set to a space delimited list of FIBs (routing tables). Tests that +need to modify a routing table may use any of these. Tests will cleanup any +new routes that they create. +.El .Ss What to do if something fails? If there is .Em any failure -during the execution of the test suite, please considering reporting it to the +during the execution of the test suite, please consider reporting it to the .Fx developers so that the failure can be analyzed and fixed. To do so, either send a message to the appropriate mailing list or file a @@ -173,7 +188,7 @@ User-specific configuration file for .Xr kyua 1 ; overrides the system file. .It Pa ~/.kyua/store.db -Default database used by Kyua to maintain the data of the executed tests. +Default result database used by Kyua. .It Pa /usr/tests/ Location of the .Fx @@ -184,9 +199,7 @@ Top-level test suite definition file. .Sh SEE ALSO .Xr kyua 1 . .Sh HISTORY -The collection of test programs in -.Pa /usr/tests -first appeared in +This test suite first appeared in .Fx 11.0 . .Pp The
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201403201713.s2KHD8gx030078>