From owner-freebsd-testing@FreeBSD.ORG Wed Jan 22 00:23:32 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A24BA679; Wed, 22 Jan 2014 00:23:32 +0000 (UTC) Received: from mail-pd0-x22a.google.com (mail-pd0-x22a.google.com [IPv6:2607:f8b0:400e:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6E0DE18FF; Wed, 22 Jan 2014 00:23:32 +0000 (UTC) Received: by mail-pd0-f170.google.com with SMTP id p10so4613821pdj.15 for ; Tue, 21 Jan 2014 16:23:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=SQChJ83zbhssvW3hfMbbXLSXuytpHLK06/5XVb0gdKQ=; b=klO1yksjfjbQ61HPesfznXyrGj0MDWqM7yGgHMn1iBBSefebTjzgpYnxVsz0pD+dZJ iQymx4dsH6LsWzggVnv/pUv/4nQtOQE0hsrcK3S5sAtl+HHPcaf11w/VvK9TgwnH/STc cGLHBGO2sd0Lvc1/m5IGS1K2qRX+wPVckuZhWQJbF6ieXf/lEAni9xYo7iPBTtPa1E4u yj2TnC9p+/+jAGI0SsGpp2TxvbALIlsMrfpL8BjD3Xy4qWEuwDj0tkx5/3Dvfv2rnwpm bprAD2eo4B1AfEe3YupwSSmXxLN1rTVU0DGHNfD8HnwUdXNPXXS9xyh9jq6SU3bBla1M On1g== X-Received: by 10.68.34.37 with SMTP id w5mr12212513pbi.159.1390350212141; Tue, 21 Jan 2014 16:23:32 -0800 (PST) Received: from [10.0.2.128] ([64.14.143.130]) by mx.google.com with ESMTPSA id tu3sm16129571pbc.40.2014.01.21.16.23.25 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 21 Jan 2014 16:23:31 -0800 (PST) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <7051E061-C054-47BF-A5B2-747B9D3E85A1@gmail.com> X-Mailer: iPhone Mail (11B554a) From: Garrett Cooper Subject: Re: PATCH: add ATF tests in sys Date: Tue, 21 Jan 2014 16:23:25 -0800 To: Alan Somers Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 00:23:32 -0000 > On Jan 21, 2014, at 16:02, Alan Somers wrote: >=20 >> On Tue, Jan 21, 2014 at 4:56 PM, Garrett Cooper w= rote: >>=20 >>> On Jan 21, 2014, at 14:45, Alan Somers wrote: >>>=20 >>> I rewrote the unix seqpacket tests in ATF. The hard part is adding >>> them to the build. The linked patch adds them in the directory >>> sys/kern/tests and builds them as part of buildworld, not buildkernel. >>> They get installed to /usr/tests/sys/kern, but the intermediate >>> objects are stored in /usr/obj/sys/kern/tests. That means that you >>> can't have different tests associated with different kernel configs. >>> I think that this is desirable, because you wouldn't be able to >>> install tests for different kernel configs anyway, given our chosen >>> layout of /usr/tests. >>>=20 >>> Please comment on the parts of this patch that deal with Makefiles. >>> Is this the appropriate way to add sys tests to the build? Shouldn't >>> I be building them in buildkernel instead of buildworld? I couldn't >>> find a good way to do that. >>=20 >> I bypassed that for simplicity and placed the tests in tests/sys/kern/...= etc. Shoehorning things into our overly complex kernel build system just se= ems like a really bad idea... Just make it load as a driver, loadable, and d= ependent on kern.features to run..? >> Cheers! >> -Garrett >=20 > I tried to place the tests as closely as possible to the source code > that they're testing, according to > https://wiki.freebsd.org/TestSuite/Structure#Makefiles_for_test_programs > . I don't feel strongly about it either way, but I think Simon and > jmmv do. >=20 > FWIW, the tests run as a userland program, not in kernel mode. Fwiw I spent months mulling over various details (this was just one point) w= ith gnn, marcel, and mfleming. The consensus was that that was a really bad i= dea because it wasn't terribly clear that tests covered a particular area, a= nd some sys/ tests could cover multiple areas (pjdfstest for instance). I mirrored the exact structure in the source tree, but avoided tainting thin= gs in sys/... Because integrating sys/ into buildworld and starting to mix k= ernel and world build logic together will irritate a large number of kernel d= evs and userland devs. This was the path of least resistance... HTH, -Garrett=