Date: Thu, 24 Jul 2014 21:14:42 -0400 From: Julio Merino <jmmv@freebsd.org> To: Garrett Cooper <yaneurabeya@gmail.com> Cc: Garrett Cooper <yanegomi@gmail.com>, "freebsd-testing@freebsd.org" <freebsd-testing@freebsd.org> Subject: Re: Need input on preference on location of 3rd party tests vs FreeBSD tests Message-ID: <CAFY7cWC1H-G-w-Fq_JAHXKUVxVU0P0NyKdA0p8xDyf8DT8QoeA@mail.gmail.com> In-Reply-To: <CAGHfRMAnv-cAZCxBDGtgwr3ZcNTpnzps4XT%2B%2By2sUVz02yjMVA@mail.gmail.com> References: <B1CE2158-B8C1-4D97-AB7A-031ADF55C435@gmail.com> <E5C29F6D-C632-4B7B-B8C8-0816C93DB7B7@gmail.com> <CAOtMX2izFjpC7r7Vxtk68KynAeuFvaDNNbK5nOEGcdFy7vJE5A@mail.gmail.com> <4D9EB4FA-672A-47AC-8F6E-19D2B3FAB3F5@gmail.com> <CAFY7cWBeJuzQStvf7fyCXB957gDB=o87yKv%2Bgt9fAjwLG1QgzA@mail.gmail.com> <CAGHfRMAnv-cAZCxBDGtgwr3ZcNTpnzps4XT%2B%2By2sUVz02yjMVA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Jul 24, 2014 at 4:39 PM, Garrett Cooper <yaneurabeya@gmail.com> wro= te: > On Thu, Jul 24, 2014 at 12:24 PM, Julio Merino <jmmv@freebsd.org> wrote: >> On Fri, Jul 18, 2014 at 5:52 PM, Garrett Cooper <yanegomi@gmail.com> wro= te: >>> On Jul 18, 2014, at 7:45 AM, Alan Somers <asomers@freebsd.org> wrote: >>> >>>> On Fri, Jul 18, 2014 at 1:11 AM, Garrett Cooper <yaneurabeya@gmail.com= > wrote: >>>>> Hi all, >>>>> One of the things that I've done on my fork of FreeBSD is I've impo= rted ATF test suites from NetBSD and I have integrated existing test suites= from freebsd's tools/regression tree into Kyua as well. Due to the size an= d difference in test content/coverage, I pulled lib/libc and lib/msun from = bother sources and integrated them into Kyua. What I did was I put the netb= sd testcases into the tests/ subdirectory and put the FreeBSD test suites i= nto a tests/legacy subdirectory. The goal was that the legacy directory wou= ld eventually be converted over to atf testcases and then could be removed = once the conversion was complete. >>>>> I'm not sure if this scheme makes sense though. Does anyone have a = preference as to whether or not this makes sense? >>>>> Thanks! >>>>> -Garrett >>>> >>>> >>>> I don't understand. What did you put in tests/legacy? >>> >>> The tests from tools/regression. tests/ contains the tests from NetBSD. >> >> I do not think adding tests under src/tests/ is a good idea. We have >> chosen to put tests under the 'tests' subdirectories of the affected >> components and we should stick to this rule except for very specific >> cases (see src/tests/sys/). Creating an artificial barrier between >> tests imported from NetBSD and native tests to FreeBSD is not >> beneficial (e.g. why should users/developers care where the tests came >> from?). > > Sorry if this was a bit misleading in my original email... > > Basically I'm trying to solve the problem of keeping the FreeBSD > testcases isolated from the NetBSD testcases. This only impacted two > components (so far) -- libc and msun. What I did was placed the libc > testcases from NetBSD in lib/libc/tests and the libc testcases from > FreeBSD in lib/libc/tests/legacy (kind of matching the model that we > have for TAP/plain scripts with the "legacy" name, but in name only). Right... but I'm not sure if that's what FreeBSD should be doing. I can see how, in the context of Isilon, this was the best thing to do: keep yourself out of the business of writing tests for the base system where possible and just reuse what NetBSD had already done. However, we are now working towards the test suite that ships with FreeBSD itself, and in this case we really should be fully on-board with maintaining the tests themselves within the project. Compare this to the development of libc itself, for example. My biggest fear is that we will perform the initial import of tests from NetBSD (as a bunch of files littered throughout the tree, with no links to where they came from), then hack on the code to make it work for FreeBSD, then add a bunch of new tests by various users... and then never import updates from NetBSD again. There are plenty of examples of code that was shared across projects and that never suffered further reimports due to the difficulty of it; see the truly obsolete smbfs in NetBSD, the pf in FreeBSD, or the way tmpfs has gone different paths in NetBSD and FreeBSD. This is why I wouldn't introduce any intentional divergence between NetBSD and FreeBSD tests in the tree, in particular to prevent any internal fears about "this code is not ours so better not touch it much". Just import the code and, if we want to pull in new tests from NetBSD, do it on a case-by-case basis. (Note that I don't object to keeping the legacy tests in a subdirectory though as that sounds reasonable. But I feel that this is orthogonal to whether we (re)import tests from NetBSD or we write our shiny new test programs!)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFY7cWC1H-G-w-Fq_JAHXKUVxVU0P0NyKdA0p8xDyf8DT8QoeA>