From owner-freebsd-arch@FreeBSD.ORG Wed Apr 2 17:59:15 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AD782963 for ; Wed, 2 Apr 2014 17:59:15 +0000 (UTC) Received: from mail-pb0-f45.google.com (mail-pb0-f45.google.com [209.85.160.45]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7B9C31B7 for ; Wed, 2 Apr 2014 17:59:15 +0000 (UTC) Received: by mail-pb0-f45.google.com with SMTP id uo5so537331pbc.32 for ; Wed, 02 Apr 2014 10:59:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=bHe4ySkWdhn6IfeLYh2+Qo4Hva1XRp11+6zWgUdXR2M=; b=YK8l2I1s9hXLenZGvFCkmeszypFOdrJpD3SoiE+9G2+1ylQFKO/mMOJ4r3WkvkgPOq FdpXjQH+Vz3z9AFV+wfMAYY5VnV93wy6uHFIVxYRnQ2vAQ4dhm0l5ZpIU2/bMuVzfhM5 bycgwpFpBX2L8Bimx2J6LXgGQvzoHEPA2e4pWT8WdgQhPKcr8wY9wB4F3y4yQk6LKMq2 Xei0tDUn14OzOh3iRDCfF4AM/H59C5qgXaGPria5tKYD81h3Y9YDYxFRWzxEfzJeH+WC o5WOjyd4WIlByKw+b0WkSiKQ44oTTOuQx1BVugs3wmshVP9lpbk/6ArMjTFxiVEC+gN3 jRyw== X-Gm-Message-State: ALoCoQmxlQ2QRAn1ZrO9IsAGCYfrRLgzyhxva0J8mq0WL+MLbLtvRyPTJHMTGdThRC2H+v2Cw4Y1 X-Received: by 10.68.201.97 with SMTP id jz1mr1743710pbc.26.1396461554425; Wed, 02 Apr 2014 10:59:14 -0700 (PDT) Received: from [10.64.24.154] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id yi3sm5828593pbb.51.2014.04.02.10.59.12 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 02 Apr 2014 10:59:13 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: make WITH[OUT]_* more useful? From: Warner Losh In-Reply-To: <20140401195249.8752258097@chaos.jnpr.net> Date: Wed, 2 Apr 2014 11:59:10 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <2349B38E-F6F2-4911-92F2-3D62FCBD73DA@bsdimp.com> References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> <20140401195249.8752258097@chaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.1874) Cc: Baptiste Daroussin , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 17:59:15 -0000 Hi Simon, thanks for the good dialog, and the patient explanation of things. Sorry = I=92ve taken a bit to reply... On Apr 1, 2014, at 1:52 PM, Simon J. Gerraty wrote: > On Tue, 1 Apr 2014 12:57:57 -0600, Warner Losh writes: >>> .if !defined(WITHOUT_CTF) && !defined(NO_CTF) >>> WITH_CTF=3D3Dyes >>> .endif >>> =3D20 >>> else you get errors during build world. >>=20 >> That's a bug I plan on fixing.=20 >=20 > Agreed, and cool. I almost have this bug fixed. There=92s one instance where it isn=92t = that helps illustrate the point you are making. >>> That's not so much a problem for existing options, but makes it >>> impractical to leverage the mechanism for things you might want to = set >>> during sys.mk - like whether to use meta mode or auto objdir = creation. >>=20 >> I=3D92ll have to think about this point. It is a good point, but = I=3D92m =3D >> unsure how >> the proposed changes would fix that. Perhaps you could explain that a = =3D >> bit. >=20 > Sure. For the sake of argument assume I can put something like this = in > local.sys.mk: >=20 > OPTIONS_DEFAULT_NO+=3D \ > META_MODE >=20 > OPTIONS_DEFAULT_YES+=3D \ > AUTO_OBJ >=20 > .include "options.mk" >=20 > .if ${MK_AUTO_OBJ} =3D=3D "yes" > .include "auto.obj.mk" > .endif > .if ${MK_META_MODE} =3D=3D "yes" > .include "meta.sys.mk" > .endif >=20 > and both >=20 > MK_META_MODE=3D{yes,no} > MK_AUTO_OBJ=3D{yes,no} >=20 > which are needed during sys.mk can be influenced by user's = -DWITH[OUT_* OK. A bit of a contrived example, but I suppose if I understood the meta = mode a bit better I might think differently :) I=92m a bit hesitant, though, to have this affect sys.mk because that = affects all users of make, not just /usr/src. >> Setting defaults in make.conf, and then overriding them is tough = because >> bmake doesn't have a tracking system to know where in the food-chain >> different variables are set. However, this is also a good point, but = I >> must be being dense to see how your proposal would fix this. >=20 > It "fixes" it by not throwing an error when both WITH_FOO and = WITHOUT_FOO > are defined. Instead, you decree that one wins (eg WITHOUT_FOO > overrides WITH_FOO). In some cases, you can declare a winner. In other cases that might be = harder to do. With almost all options defaulting to YES, having WITHOUT win = makes good sense. When there=92s more of a mix, I=92m less sure. > You could also address the conflict by giving preference to command = line. > ${.MAKEFLAGS:MWITH*} will give you any WITH_* and WITHOUT_* from = command > line. =20 >=20 > There might be a rare case where an error is desired on conflict, but > if you can sensibly resolve it that is better. >=20 >>> I have a number of knobs to be set during sys.mk >>> AUTO_OBJ automatically create objdirs >>> META_MODE use meta mode >>=20 >> objdirs set in sys.mk?=20 >=20 > yes. >=20 >> I thought that was done bad.obj.mk since it isn't part of the global = system. >=20 > Traditionally done in bsd.obj.mk - but that requires a separate > invocation of make. Right, but can=92t that be done automatically w/o that extra invocation? > The junos build for example has auto-created objdirs for 10+ years. > The projects/bmake build does the same. > It is a good way to ensure consistent/reliable results and avoid > probelms when users forget to 'make obj'. >=20 > When you have to support 2000+ developers you want to be able to > make it hard for them to shoot themselves in the foot. >=20 > If making objdirs automatically, you really want to do it early, since > the result influences how make then behaves. True, all good stuff. Just got caught up quibbling over its need to be in sys.mk vs enhancing bsd.obj.mk to do this... >> META_MODE might belong there. >>=20 >>> setting MK_* is fine, but it is nice if you allow the user to = interact >>> using WITH[OUT]_*, and for that it is best if sys.mk can safely =3D >> include >>> something like options.mk >>=20 >> Not sure how creating a new file solves the bsd.own.mk problem, apart >> perhaps from some name space pollution differences. >=20 > An options.mk which does nothing except process the list of > options provided to it, is always safe to include even by sys.mk. > Whereas bsd.own.mk does other stuff which makes it unsuitable to = include=20 > eg. during the early stages of buildworld. > bsd.own.mk can of course set its own list of options and then include > the same options.mk a simple matter of refactoring to allow re-use. Back to the NO_CTF stuff I talked about above: I totally get this. If = you look at bsd.own.mk it is in 3 sections (plus a stray line at the end that really = belongs in the first section. # section 1 - stuff about ownership # section 2 - stuff to do options # section 3 - setting defaults based on options It is this last section that=92s a problem to CTF specifically. = Normally, we have a paradigm in the tree where we do .include .if ${MK_FOO} !=3D =93no=94 CFLAGS+=3D-DSUPPORT_FOO .endif There=92s one place in the tree that wants to turn off CTF. This is = mostly fixed by just setting MK_CTF=3Dno in that makefile after we include bsd.own.mk. I = say mostly fixed because we wind up with a NULL command where we really want a @: command, though the former I believe is harmless but verbose. = Except one could unset WITH_CTF (which doesn=92t completely work, it still = shows up as defined) and set WITHOUT_CTF before bsd.own.mk and it would work, modulo this bug. This can really only be fixed by making bsd.own.mk look more like # section 1 -same .include # section 3 - same with bsd.options.mk looking a lot like section 2 of bsd.own.mk. Also, = we=92d have to include bsd.options.mk at the head of the Makefile rather than = bsd.own.mk where we do now. This sounds a lot like what you=92re trying to describe, though = placement of bsd.options.mk may be different than I described. The only reason we need it where I suggested it is compatibility with the past. Though we = may be able to get away with it in sys.mk, I=92m hesitant to place it in = there because that=92s global to everything, including ports, etc. Plus, I think it is = too early, due to meta MK_ variables, that I talk about below. >> Yea, back to the point I don=3D92t understand: how is your new file = going =3D >> to be >> materially different than the current mechanism. I=3D92m OK with = change, =3D >> but I >> need to understand how the change is going to make things better. >=20 > Separating the mechanism of option processing from the definition of > options makes re-use easier - see above, sorry I don't know how to = make > it clearer. >=20 > The other aspects you have at least partially addressed already, by > allowing MK_* to be set independently. > Though: >=20 > .if defined(MK_${var}) > .if ${.MAKE.LEVEL} =3D=3D 0 > .error MK_${var} can't be set by a user. > .endif >=20 > would seem to negate that. Why can a makefile at level 0 not set = MK_*? Well, the notion now is that if you want to test MK_* variables, you = need to include bsd.own.mk first. The notion I was going for with the above is = that you=92d have to include bsd.own.mk first, then set the MK_* variables in your = Makefile, so the tests won=92t be hit. But there=92s a problem even if we take the approach above. Section 2 in = bsd.own.mk is actually two separate sections. One that sets the MK_* variables = based on WITH_ or WITHOUT_ and then a second section that cascades the MK_ = variables to other MK_ variables (like MK_CRYPT=3D=3Dno turning of OpenSSL, = Kerberos, etc). If you wanted to set one of those variables in your Makefile, you=92d have = a chicken and egg problem. If you set it before bsd.options.mk, then you=92d get = the cascade effects but hit the warning. If you set it after, you dodge the warning, but = don=92t get the cascade. This problem suggests, perhaps, that the test be deleted. > The outstanding question is dealing with conflict when both WITH_FOO = and > WITHOUT_FOO are defined. True. That=92s a tougher problem than you might think on first blush, as = we=92ve been talking about. For now, I=92d suggest WITHOUT wins, and we see how = far that gets us and what problems come up. Anything else seems like it = would get uber complicated quite quickly as we play whack-a-mole with = different edge use cases. Warner