From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 29 08:05:40 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D790106566C; Fri, 29 Apr 2011 08:05:40 +0000 (UTC) (envelope-from Hartmut.Brandt@dlr.de) Received: from mailhost.dlr.de (mailhost.dlr.de [129.247.252.32]) by mx1.freebsd.org (Postfix) with ESMTP id EB2788FC13; Fri, 29 Apr 2011 08:05:39 +0000 (UTC) Received: from DLREXHUB02.intra.dlr.de (172.21.152.140) by mailhost.dlr.de (172.21.163.100) with Microsoft SMTP Server (TLS) id 14.1.270.2; Fri, 29 Apr 2011 10:05:11 +0200 Received: from beagle.kn.op.dlr.de (129.247.178.136) by smtp.dlr.de (172.21.152.151) with Microsoft SMTP Server (TLS) id 14.1.270.2; Fri, 29 Apr 2011 10:05:17 +0200 Date: Fri, 29 Apr 2011 10:05:18 +0200 From: Hartmut Brandt X-X-Sender: brandt_h@beagle.kn.op.dlr.de To: Roman Divacky In-Reply-To: <20110429070928.GA83618@freebsd.org> Message-ID: <20110429095947.J64684@beagle.kn.op.dlr.de> References: <20110427193946.GA41659@freebsd.org> <20110428174523.I61666@beagle.kn.op.dlr.de> <20110428173613.GA31077@freebsd.org> <20110428203709.M62691@beagle.kn.op.dlr.de> <20110429070928.GA83618@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" X-Originating-IP: [129.247.178.136] Cc: hackers@freebsd.org Subject: Re: make question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Apr 2011 08:05:40 -0000 On Fri, 29 Apr 2011, Roman Divacky wrote: RD>On Thu, Apr 28, 2011 at 08:50:27PM +0200, Hartmut Brandt wrote: RD>> On Thu, 28 Apr 2011, Roman Divacky wrote: RD>> RD>> RD>On Thu, Apr 28, 2011 at 05:52:58PM +0200, Hartmut Brandt wrote: RD>> RD>> Hi Roman, RD>> RD>> RD>> RD>> On Wed, 27 Apr 2011, Roman Divacky wrote: RD>> RD>> RD>> RD>> RD>You seem to have messed with bsd make so I have a question for you :) RD>> RD>> RD>> RD>> Yeah, that was some time ago ... RD>> RD>> RD>> RD>> RD>When a job is about to be executed in JobStart() a pipe is created with RD>> RD>> RD>its ends connected to job->inPipe/job->outPipe. When the job is actually RD>> RD>> RD>created in JobExec() the ps.out is set to job->outPipe so that in RD>> RD>> RD>JobDoOutput() we can read from that pipe and basically just parse the output RD>> RD>> RD>for shell->noPrint and leaving it out from the output. This is meant (I think) RD>> RD>> RD>for supressing the "filter" thing. Ie. that if we do some @command the RD>> RD>> RD>restoration of setting of quiet mode is filtered out. RD>> RD>> RD> RD>> RD>> RD> RD>> RD>> RD>In -B mode we do it differently, as we invoke one shell per command we don't RD>> RD>> RD>have to insert quiet/verbose commands and thus avoid all the piping/parsing RD>> RD>> RD>dance. RD>> RD>> RD> RD>> RD>> RD>So my question is - why don't we invoke one shell per command by default RD>> RD>> RD>and avoid the piping/parsing? Is this because of performance? I think that RD>> RD>> RD>the piping/parsing of the output can have worse impact than invoking a shell RD>> RD>> RD>for every command. Especially given that most targets consists of just one RD>> RD>> RD>command. RD>> RD>> RD>> RD>> The answer is in /usr/share/doc/psd/12.make. This is so one can write RD>> RD>> something like RD>> RD>> RD>> RD>> debug: RD>> RD>> DEBUG_FLAGS=-g RD>> RD>> for i in $(SUBDIR); do RD>> RD>> $(MAKE) -C $$i all RD>> RD>> done RD>> RD>> RD>> RD>> instead of: RD>> RD>> RD>> RD>> debug: RD>> RD>> DEBUG_FLAGS=-g \ RD>> RD>> for i in $(SUBDIR); do \ RD>> RD>> $(MAKE) -C $$i all ; \ RD>> RD>> done RD>> RD>> RD>> RD>> -B means 'backward compatible' and does what the original v7 make did: one RD>> RD>> shell per command. This means you don't have to write the backslashes and RD>> RD>> the shell variable will be seen in the sub-makes and programs. RD>> RD>> RD>> RD>> I think we can change this, because it would break makefiles that assume RD>> RD>> that the entire script is given to the shell in one piece. RD>> RD> RD>> RD>I think you answered the question why we parse the target. But I asked why RD>> RD>we parse the output from it. RD>> RD>> My intention was to say why we use one shell for all commands for a given RD>> rule. If we'd use one shell per line the above would not work, because the RD>> first shell would see just the environment variable assignment (which RD>> would be completly useless). The next shell would see a partial 'for' RD>> statement and complain, and would not have the environment variable and so RD>> on. So this is not so much about parsing, but about execution. RD>> RD>> I suppose that the tricky point is with @-lines in the middle of a RD>> multi-line script. RD> RD>Unless I am reading the code wrong the "one shell per command" is the default RD>mode. RD> RD>see in main.c: RD> RD> /* RD> * Be compatible if user did not specify -j and did not explicitly RD> * turned compatibility on RD> */ RD> if (!compatMake && !forceJobs) RD> compatMake = TRUE; RD> RD>You have to specify -j to turn off the compat mode. Wow. This breakage was introduced in our make in 1996 by an import from NetBSD (I did not check why they introduced it). I fail to see the logic in this handling of the -B option (if the user doesn't provide it, I (make) do it, unless the user said -j). But, the good side is, we can probably change the behavior of the non-compat mode, because the probability of there beeing makefiles that rely on it is now rather low. In any case I recommend to check what is the performance implication of executing a lot more shells in -j mode. I expect this to be in the noise level, but a check would not hurt... harti