From owner-freebsd-arch@FreeBSD.ORG Tue Oct 2 00:02:12 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3FE6C1065674; Tue, 2 Oct 2012 00:02:12 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from exprod7og110.obsmtp.com (exprod7og110.obsmtp.com [64.18.2.173]) by mx1.freebsd.org (Postfix) with ESMTP id 978518FC0C; Tue, 2 Oct 2012 00:02:10 +0000 (UTC) Received: from P-EMHUB03-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob110.postini.com ([64.18.6.12]) with SMTP ID DSNKUGovAWxyWzC4IQvD1sZjrFmUuVrQOwO3@postini.com; Mon, 01 Oct 2012 17:02:11 PDT Received: from magenta.juniper.net (172.17.27.123) by P-EMHUB03-HQ.jnpr.net (172.24.192.33) with Microsoft SMTP Server (TLS) id 8.3.213.0; Mon, 1 Oct 2012 17:00:32 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id q9200Uh40887; Mon, 1 Oct 2012 17:00:31 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 54CEE58093; Mon, 1 Oct 2012 17:00:30 -0700 (PDT) To: Garrett Cooper In-Reply-To: References: <20121001223100.E7D0D58093@chaos.jnpr.net> Comments: In-reply-to: Garrett Cooper message dated "Mon, 01 Oct 2012 16:15:16 -0700." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Mon, 1 Oct 2012 17:00:30 -0700 Message-ID: <20121002000030.54CEE58093@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain Cc: freebsd-hackers@FreeBSD.org, freebsd-arch@FreeBSD.org, sjg@juniper.net Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 00:02:12 -0000 >> Not to mention the fact that bsd.prog.mk goes from being relatively >> simple, to unspeakably hard to read, and all for rather limited = >return. This btw I think is the more important issue. I was looking at bsd.prog.mk in netbsd the other day. It has no business being that complex. >Getting back to the idea at hand, the enhancement goal was to get the = >testcases to live [and optionally be built/installed] with the source = >code to avoid bitrot; I know this isn't the current NetBSD design, but = >this is what was requested be done by gnn, marcel, and mdf, and I agree. = I agree, that's what we do too. >It order to do this, I need to be able to build multiple programs so = >things are as transparent. On the flipside, bsd.prog[s].mk can live on = We put the test cases in a subdir of the lib/prog This has multiple benefits, and eliminates any impact on the normal build of said libs/progs. Also a number of our teams find it necessary to create mock classes etc to adequately test their libs. Keeping all this in a tests/ subdir makes it easy, to keep things neat & up to date, and ensure that the tests pass. Trying to do all that in the same dir as the lib would be ugly. >> FWIW we use progs.mk (as bsd.progs.mk) from >> ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz=20 >> It isn't ideal, but it certainly avoids a lot of churn and complexity >> for what is essentially a corner case. > >This requires pulling bmake into FreeBSD proper in order for things to = >function the last I checked as bsd.prog.mk depends upon bmake directives = This is already happening ;-) >Ideally however, I would like doing this compared to running a custom = >build infrastructure, but (as you probably know) this requires = >rototilling the current FreeBSD build system to a large degree = Actually building FreeBSD-current using bmake requires only modest changes. >> We have an atf.test.mk which leverages that. >> It also handles automatically running the tests if building for the >> host. This allows us to fail the build if any unit-tests do not pass. > >Hmm=85 that wasn't really the end-goal of bsd.test.mk based on my = >reading, I know, but it is a very useful thing to be able to do. You can add knobs to controll such things of course. >bsd.test.mk in NetBSD isn't completely test-agnostic, but it isn't = >entirely ATF centric either. I'm trying not to stray too far from NetBSD = >in this area because there really isn't any reason to do so. FWIW I'd = Sure - we imported ATF directly from NetBSD to minimize the churn and not had to change much at all. bsd.test.mk was a point worth diverging on though. >like to see other test frameworks (lua, unittest//nose, etc) just snap = >into bsd.test.mk as easily as possible as it would make some groups = Yes, but making one test.mk handle potentially lots of frameworks will get rather ugly quite quickly. Better to add per-framework .mk files, and perhaps have them include a bsd.test.mk which only handles high level logic rather than the details of the frameworks. That's laregly why we didn't use bsd.test.mk --sjg