From owner-freebsd-testing@FreeBSD.ORG Tue Jan 14 19:22:51 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 60C91CA1 for ; Tue, 14 Jan 2014 19:22:51 +0000 (UTC) Received: from mail-we0-x22b.google.com (mail-we0-x22b.google.com [IPv6:2a00:1450:400c:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EC11D16B0 for ; Tue, 14 Jan 2014 19:22:50 +0000 (UTC) Received: by mail-we0-f171.google.com with SMTP id w61so848584wes.30 for ; Tue, 14 Jan 2014 11:22:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=Z+Z/Dor0ltDjT89SnVL9U/Ao9n8zxN9l5v17jAPGAdQ=; b=gnzO3VLLHivsqgHPkrx3K8PT/BPLm1l4SkQpAEgCUVmgXOxPKPLqQvN8CG9qJChHAu 2TojACX4K8rOmGluoCB4TXldEDNd4AkVbCuohiWIoKsflogXugpJ6fJi0cVHqnvzfkgu ty9dccejRYiEM1DUf1+Deqsc5UEPBHuTBelVf9i8rQwt+supIW38i5lW6P3byNAPADCi wbmC4d+I3gVu6C+CK2qg36s7weFkeldC5foVHkpGBaNmsgo5/0MZpVxL1/Sk6FQPuIvQ Xb3BEGrb4EnyxeHj+Bg6MgSKHfkHSv2VYwpHzjb5mzCTrlMwXcfRWJb34iBC6Z1Y4nSR g1fA== MIME-Version: 1.0 X-Received: by 10.194.240.199 with SMTP id wc7mr65056wjc.92.1389727369432; Tue, 14 Jan 2014 11:22:49 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Tue, 14 Jan 2014 11:22:49 -0800 (PST) In-Reply-To: <7D8E1A11-A946-499A-B568-D77D91B49FA6@meroh.net> References: <7D8E1A11-A946-499A-B568-D77D91B49FA6@meroh.net> Date: Tue, 14 Jan 2014 12:22:49 -0700 X-Google-Sender-Auth: OpyyxuHobneaCjizeuWJLbXEEMA Message-ID: Subject: Re: Correct src location for kernel ATF tests From: Alan Somers To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Jan 2014 19:22:51 -0000 On Tue, Jan 14, 2014 at 12:12 PM, Julio Merino wrote: > On Jan 13, 2014, at 22:25, Alan Somers wrote: > >> Where is the correct location within the source tree for ATF tests >> that exercise kernel features? For example, I'm currently trying to >> ATFify tools/regression/sockets/unix_seqpacket_exercise, > > I think it's good practice to _not_ ATFify an existing test at the same t= ime as it's moved. > > I feel it's better to first move the test program "as is" into its new lo= cation, ensure it works, and only then change it to use the ATF libraries. = If done properly (e.g. by maintaining the same binary name pre/post conver= sion), this should cause minimum churn and will allow for easier tracking o= f code changes in the VCS history. In this case, the test doesn't work to begin with. I'm rewriting almost from scratch. But I'll follow your suggestion when I commit to HEAD. I'll move first, then immediately edit to change the contents. > >> but I'm not sure where to put it. > > Ideally tests should live next to the code they exercise, just as we are = doing for user-space applications. This means that, at the very minimum, t= hey should be somewhere within src/sys/. > > Now that's tricky of course. Some thoughts that cross my mind: > > First, I'm not sure about what the implications of putting "user-level" c= ode in src/sys/ are regarding the build infrastructure. I suspect that by = restricting the test code into 'tests' subdirectories and only recursing in= to them with the MK_TESTS knob is good enough, but I don't know for sure at= the moment. > > Second, possibly trickier, is that any kernel tests that currently exist = are integration tests by design: we have no unit-testing framework for the = kernel (such as what rump provides for NetBSD), and as such any test will b= e quite "broad" in scope. This may make it difficult to pinpoint the speci= fic subdirectory in which the test belongs. For your case above, I cannot = tell if any of the src/sys/net*/ subdirectories are relevant and/or specifi= c enough. If they are, then I'd say put the test in one of them using src/= sys/netgraph/tests/unix_seqpacket_exercise to pick an example at random. Sounds good, but the most relevant directory is actually src/sys/kern/, not src/sys/netgraph. > > Lastly, we may need a src/sys/tests/ directory to implement cross-functio= nal tests that don't fit any specific directory, just like we have src/test= s/ for that purpose. Or maybe we can (ab)use the latter instead. We definitely will need some sort of location for cross-functional tests. For example, the STF ZFS tests touch just about everything related to ZFS. But that's a problem for another day. -Alan