Date: Wed, 22 Jan 2014 19:06:49 -0500 From: Julio Merino <julio@meroh.net> To: Garrett Cooper <yaneurabeya@gmail.com> Cc: "freebsd-testing@freebsd.org" <freebsd-testing@freebsd.org> Subject: Re: PATCH: add ATF tests in sys Message-ID: <7A53F7F4-349F-480D-802E-F61C9308D009@meroh.net> In-Reply-To: <7051E061-C054-47BF-A5B2-747B9D3E85A1@gmail.com> References: <CAOtMX2iiSH9DvE73_7V=A%2ByqObROmvJu7VRTMG7K74eMtHk4=g@mail.gmail.com> <C2E6F1D5-B409-4F2E-B856-C65473BCC3A0@gmail.com> <CAOtMX2goL4yvGjYZX8Gmk=rx0du5yrsQ_W-Y8XWo0WkCNDjwkw@mail.gmail.com> <7051E061-C054-47BF-A5B2-747B9D3E85A1@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Jan 21, 2014, at 19:23, Garrett Cooper <yaneurabeya@gmail.com> wrote: >>=20 >> On Jan 21, 2014, at 16:02, Alan Somers <asomers@freebsd.org> wrote: >>=20 >>> On Tue, Jan 21, 2014 at 4:56 PM, Garrett Cooper = <yaneurabeya@gmail.com> wrote: >>>=20 >>>> On Jan 21, 2014, at 14:45, Alan Somers <asomers@freebsd.org> wrote: >>>>=20 >>>> I rewrote the unix seqpacket tests in ATF. The hard part is adding >>>> them to the build. The linked patch adds them in the directory >>>> sys/kern/tests and builds them as part of buildworld, not = buildkernel. >>>> They get installed to /usr/tests/sys/kern, but the intermediate >>>> objects are stored in /usr/obj/sys/kern/tests. That means that you >>>> can't have different tests associated with different kernel = configs. >>>> I think that this is desirable, because you wouldn't be able to >>>> install tests for different kernel configs anyway, given our chosen >>>> layout of /usr/tests. >>>>=20 >>>> Please comment on the parts of this patch that deal with Makefiles. >>>> Is this the appropriate way to add sys tests to the build? = Shouldn't >>>> I be building them in buildkernel instead of buildworld? I = couldn't >>>> find a good way to do that. >>>=20 >>> I bypassed that for simplicity and placed the tests in = tests/sys/kern/... etc. Shoehorning things into our overly complex = kernel build system just seems like a really bad idea... Just make it = load as a driver, loadable, and dependent on kern.features to run..? >>> Cheers! >>> -Garrett >>=20 >> I tried to place the tests as closely as possible to the source code >> that they're testing, according to >> = https://wiki.freebsd.org/TestSuite/Structure#Makefiles_for_test_programs >> . I don't feel strongly about it either way, but I think Simon and >> jmmv do. Yes in general, but... see below. >>=20 >> FWIW, the tests run as a userland program, not in kernel mode. >=20 > Fwiw I spent months mulling over various details (this was just one = point) with gnn, marcel, and mfleming. The consensus was that that was a = really bad idea because it wasn't terribly clear that tests covered a = particular area, and some sys/ tests could cover multiple areas = (pjdfstest for instance). >=20 > I mirrored the exact structure in the source tree, but avoided = tainting things in sys/... Because integrating sys/ into buildworld and = starting to mix kernel and world build logic together will irritate a = large number of kernel devs and userland devs. This was the path of = least resistance... I think Garrett has a good point here and, if the build within src/sys/ = is significantly different from the build of userspace tools and has = different restrictions, it is probably a good idea to treat the kernel = tests as "special". After all, these tests are userland programs as you well said and they = are a kind of "integration" tests because there is no way currently to = unit-test kernel code. Such programs will just do "stuff" hoping to = validate kernel code, but there are so many levels of indirection that = they cannot be though of as testing things in very fine-grain detail. (If we had rump-like features as NetBSD, things may be a bit different = though because in that case you really are writing unit tests for very = specific kernel code.) Using src/tests/sys/ and replicating the layout of src/sys/ as much as = possible within it may be a reasonable and easy compromise for now.=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7A53F7F4-349F-480D-802E-F61C9308D009>