Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 1 Oct 2012 16:15:16 -0700
From:      Garrett Cooper <yanegomi@gmail.com>
To:        "Simon J. Gerraty" <sjg@juniper.net>
Cc:        freebsd-hackers@FreeBSD.org, freebsd-arch@FreeBSD.org
Subject:   Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
Message-ID:  <FDE5A28C-297A-4476-B06E-638049E20C6C@gmail.com>
In-Reply-To: <20121001223100.E7D0D58093@chaos.jnpr.net>
References:  <CAGH67wRkOmy7rWLkxXnT2155PuSQpwOMyu7dTAKeO1WW2dju7g@mail.gmail.com> <CDA41F27-73C1-47CF-B84D-2627B1F7E7D8@xcllnt.net> <20121001223100.E7D0D58093@chaos.jnpr.net>

next in thread | previous in thread | raw e-mail | index | archive | help
Hi Simon!

On Oct 1, 2012, at 3:31 PM, Simon J. Gerraty wrote:

> Hi Garrett,
>=20
>>> From: Garrett Cooper <yanegomi@gmail.com>
>>> Subject: [CFT/RFC]: refactor bsd.prog.mk to understand multiple =3D
>> programs instead of a singular program
>>> Date: September 2, 2012 11:01:09 PM PDT
>>> To: freebsd-hackers@freebsd.org
>>> Cc: "freebsd-arch@FreeBSD.org Arch" <freebsd-arch@freebsd.org>
>>> =3D20
>>> Hello,
>>>   I've been a bit busy working on porting over ATF from NetBSD, and
>=20
> Thanks, we've been using ATF in Junos for a while and glad to see it=20=

> being imported to FreeBSD.

Thanks (and I appreciate the help marcel@ has given by taking the first =
step in importing it).

>>> one of the pieces that's currently not available in FreeBSD that's
>>> available in NetBSD is the ability to understand and compile =
multiple
>>> programs. In order to do this I had to refactor bsd.prog.mk (a lot).
>=20
> A change like this to bsd.prog.mk can have considerable fallout.
> Eg. any makefile that tweaks OBJS is suddenly out of luck.

Well=85 any Makefiles attempting to be "clever" (e.g. crunchgen) are out =
of luck and would need to change. Most [include] Makefiles will see =
virtually no change.

> Not to mention the fact that bsd.prog.mk goes from being relatively
> simple, to unspeakably hard to read, and all for rather limited =
return.=20

I sort of tried to match bsd.prog.mk in NetBSD in the beginning, but =
things diverged from there=85 I'm sure things could be made more =
readable so any comments would be much appreciated :)!

> Apart from ATF, is there any huge demand for building multiple progs =
in
> the same directory?

Well, I noticed that there are a couple ugly messes in the gnu/... =
directory that attempt to work around the fact that bsd.prog.mk doesn't =
support more than one program by being tricky and emulating bsd.prog.mk =
instead of solving the generic problem (and once I got over that =
compatibility issue I stopped tracking this class of consumer). Most =
consumers don't care (they split up programs into different directories =
or use hardlink hacks/basename detection to differentiate one program =
functionally from another).

Getting back to the idea at hand, the enhancement goal was to get the =
testcases to live [and optionally be built/installed] with the source =
code to avoid bitrot; I know this isn't the current NetBSD design, but =
this is what was requested be done by gnn, marcel, and mdf, and I agree. =
It order to do this, I need to be able to build multiple programs so =
things are as transparent. On the flipside, bsd.prog[s].mk can live on =
its own, be pulled in automatically by bsd.test.mk, and that would be =
it. This encourages code duplication though and bugs can persist in =
either Makefile, when I'd rather work out all the kinks in whatever =
succeeds the legacy bsd.prog.mk file.

> FWIW we use progs.mk (as bsd.progs.mk) from
> ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz=20
> It isn't ideal, but it certainly avoids a lot of churn and complexity
> for what is essentially a corner case.

This requires pulling bmake into FreeBSD proper in order for things to =
function the last I checked as bsd.prog.mk depends upon bmake directives =
in order to function properly 100% of the time (and there are external =
dependencies and assumptions one has to deal with when using =
bsd.prog.mk, etc from NetBSD that made that a bit of a no-go without =
refactoring/pulling in bits from NetBSD). I could be wrong though, so =
please let me know if I am :).

Ideally however, I would like doing this compared to running a custom =
build infrastructure, but (as you probably know) this requires =
rototilling the current FreeBSD build system to a large degree =
(definitely not impossible=85 just needs time and effort).

> We have an atf.test.mk which leverages that.
> It also handles automatically running the tests if building for the
> host. This allows us to fail the build if any unit-tests do not pass.

Hmm=85 that wasn't really the end-goal of bsd.test.mk based on my =
reading, but I'm sure Julio would be interested in it. I need to do =
adjusting in order to allow automatically executing testcases compatible =
to the host architecture, but that isn't hard to do (no more than  a =
couple hours work).

> Note, atf.test.mk is loosely derrived from NetBSD's bsd.tests.mk, but
> named for what it is (ATF specific tests) since we wanted the
> flexibility to have more than one test framework.

bsd.test.mk in NetBSD isn't completely test-agnostic, but it isn't =
entirely ATF centric either. I'm trying not to stray too far from NetBSD =
in this area because there really isn't any reason to do so. FWIW I'd =
like to see other test frameworks (lua, unittest//nose, etc) just snap =
into bsd.test.mk as easily as possible as it would make some groups =
lives easier, but that requires a bit more thought for another day.

Thanks for the feedback!
-Garrett=



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?FDE5A28C-297A-4476-B06E-638049E20C6C>