Date: Wed, 14 Nov 2012 01:28:04 -0800 From: Garrett Cooper <yanegomi@gmail.com> To: "freebsd-arch@FreeBSD.org Arch" <freebsd-arch@freebsd.org> Cc: George Neville-Neil <gnn@freebsd.org>, Marcel Moolenaar <marcel@xcllnt.net>, Matthew Fleming <mdf@freebsd.org> Subject: [RFC] test layout/standardization for FreeBSD Message-ID: <CAGH67wSoU08N8=40RE3j0na4B6-VhZFwhMdkw-6CYhoxKKHumg@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
As David O'Brien noted at the BSD Vendor Support, some of the
requirements for standardizing the test process and layout in FreeBSD
aren't clear today. I asked for feedback from some of the stakeholders of
the FreeBSD test automation effort (gnn, marcel, mdf, sjg) I've been
working with, and here is a summary of the agreed upon method for
organizing tests in FreeBSD. Please provide constructive criticism, feel
free to ask questions, etc if any of the points below could be
clarified/improved upon; items 1. and 3. are straightforward, but item 2.
is still up to some debate, in particular item 2.i and 2.iii. The agreed
upon items will be added to the wiki (http://wiki.freebsd.org/TestingFreeBSD
).
Thanks!
-Garrett
1. Userland tests:
- Tests will go into a tests/ subdirectory, e.g. for source directory
lib/libc/, there will be a tests directory lib/libc/tests/; for bin/date,
there will be a tests directory bin/date/tests/.
2. Kernel tests:
i. The user will be able to invoke buildkernel -DWITH_TESTS build all
applicable testcases and install them into ${TESTSBASE}/sys/... .
ii. Generic kernel tests will go into tests/sys/ and will be included
in the standard build process. Example:
sys/tests/capsicum/...
sys/tests/kern/acct/...
sys/tests/kern/read/...
sys/tests/kern/write/...
iii. Driver specific testcases should live close to the source, e.g.
mps(4) testcases should live in sys/dev/mps/tests/, and mps(4) unittest
programs/modules should be compiled/linked as part of the standard module
build process. Moreover the build logic should live under
sys/modules/mps/tests/ [*]. The install path is vendor/developer specific,
but should be ${TESTSBASE}/sys/${DEV}/ in order to match FreeBSD's layout.
This will require another minor piece of build infrastructure (kern.test.mk)
in order to ensure proper versioning, and I will work on drafting up that
next item if it's agreed that this is the correct direction for
standardizing driver testcases.
3. Jumping into tests/ subdirectories should be handled transparently via
bsd.subdir.mk like so:
diff --git a/share/mk/bsd.subdir.mk b/share/mk/bsd.subdir.mk
index 38a9080..a0e9984 100644
--- a/share/mk/bsd.subdir.mk
+++ b/share/mk/bsd.subdir.mk
@@ -43,6 +43,11 @@ distribute:
.endfor
.endif
+.if ${MK_TESTS} != "no" && exists(${.CURDIR}/tests/Makefile)
+SUBDIR+= tests
+SUBDIR_TARGETS+= test
+.endif
+
_SUBDIR: .USE
.if defined(SUBDIR) && !empty(SUBDIR) && !defined(NO_SUBDIR)
@${_+_}set -e; for entry in ${SUBDIR}; do \
This will require adding an additional knob (MK_TESTS) to bsd.own.mk.
[*] How we run and handle driver-specific kernel testcases is outside the
scope of this note as we do not have the full NetBSD testing infrastructure
(RUMP, et all) to commit to a full-scale plan [yet]. This needs to be
worked out sometime in the next 1~2 years and/or should done as a GSoC
project.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAGH67wSoU08N8=40RE3j0na4B6-VhZFwhMdkw-6CYhoxKKHumg>