From owner-svn-src-head@FreeBSD.ORG Sun Oct 21 07:20:54 2012 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 51B2660C; Sun, 21 Oct 2012 07:20:54 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail03.syd.optusnet.com.au (mail03.syd.optusnet.com.au [211.29.132.184]) by mx1.freebsd.org (Postfix) with ESMTP id 6B1B88FC0A; Sun, 21 Oct 2012 07:20:52 +0000 (UTC) Received: from c122-106-175-26.carlnfd1.nsw.optusnet.com.au (c122-106-175-26.carlnfd1.nsw.optusnet.com.au [122.106.175.26]) by mail03.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id q9L7KfXt024145 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 21 Oct 2012 18:20:42 +1100 Date: Sun, 21 Oct 2012 18:20:41 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: "Simon J. Gerraty" Subject: Re: svn commit: r241298 - in head: . gnu/usr.bin/cc/cc_int gnu/usr.bin/cc/include kerberos5 kerberos5/tools/asn1_compile kerberos5/tools/slc lib/clang/include share/mk tools/build/make_check usr.sbin/c... In-Reply-To: <20121020181822.A6C6C58094@chaos.jnpr.net> Message-ID: <20121021172847.N1824@besplex.bde.org> References: <201210062001.q96K16Or030755@svn.freebsd.org> <20121007161440.C7605@besplex.bde.org> <81CB9A3A-4BFF-41B2-A1F9-3721A40F6260@xcllnt.net> <20121008165228.F7997@besplex.bde.org> <965E2363-6866-4A10-A7B1-C428B3C78925@xcllnt.net> <20121020205008.C1781@besplex.bde.org> <20121020181822.A6C6C58094@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-Cloudmark-Score: 0 X-Optus-Cloudmark-Analysis: v=2.0 cv=Up07rJMB c=1 sm=1 a=LFdMy-NRfKoA:10 a=kj9zAlcOel0A:10 a=PO7r1zJSAAAA:8 a=JzwRw_2MAAAA:8 a=Qs8lTao0p00A:10 a=cZds6CABXkSaZCioNQMA:9 a=CjuIK1q_8ugA:10 a=bxQHXO5Py4tHmhUgaywp5w==:117 Cc: "svn-src-head@FreeBSD.org" , "svn-src-all@FreeBSD.org" , "src-committers@FreeBSD.org" , Bruce Evans , Marcel Moolenaar X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Oct 2012 07:20:54 -0000 On Sat, 20 Oct 2012, Simon J. Gerraty wrote: > On Sat, 20 Oct 2012 21:13:36 +1100 (EST), Bruce Evans writes: >> Back to the old thread. Followup to the new commit thread. > > I seem to have missed a bit of the conversation, so sorry if some of > this has been done to death already. > > First some background: > > I believe the reason for the original change (which was wrong btw, > I had stared at that diff many times and saw it as what I expected it to > be - which I think Marcel has now committed), > is that .ORDER does not work in jobs mode in the presence of explicit > dependencies (see below). > > Since we auto collect dependencies during the build, we've learned to > avoid .ORDER, it is often responsible for introducing cycles in the > dependency graph. That seems to be a bug in the dependency collection. .ORDER expresses a requirement that is similar to a dependency requirement, and any dependency that conflicts with that is a bug. In the yacc directives it expresses the requirement more clearly than a fake dependency -- '.ORDER: a b' was replaced by 'b: a' (or perhaps the reverse, since the order can be either), but there is no real dependency, only a sequential requirement. Buggy auto-generation of depencies would cause essentially the same cycle if it genrated 'a: b'. But old make didn't detect cycles due to .ORDER. > Given the makefile: > > --------------------8<-------------------- > .MAIN: all > > ALPHA=a b c d e f g h i j k l m n o p q r s t u v w x y z > > SECS?=1 > > ${ALPHA}: > @echo "`date +%s` $@..."; sleep ${SECS} > > .NOPATH: ${ALPHA} > .PHONY: ${ALPHA} > > ALL=r a l p h i s g i t > > .ifdef ORDER > x!= echo .ORDER: r a l p h >&2; echo > .ORDER: r a l p h > .endif > > all: ${ALL} > > .ifdef DEPS_A > x!= echo a: p e >&2; echo > a: p e > .endif > --------------------8<-------------------- > > The following is using FreeBSD make from 10.x (though it doesn't make > any difference): > > First compat mode: > > $ fmake -r -f /home/sjg/make-tests/order > 1350755164 r... > 1350755165 a... > 1350755166 l... > 1350755168 p... > 1350755169 h... > 1350755170 i... > 1350755171 s... > 1350755172 g... > 1350755173 t... > $ > > Looks as you would expect, as does jobs mode: > > $ fmake -r -f /home/sjg/make-tests/order -j3 > 1350755240 r... > 1350755240 l... > 1350755240 a... > 1350755241 p... > 1350755241 h... > 1350755241 i... > 1350755242 s... > 1350755242 g... > 1350755242 t... > $ > > Ie. the order is no longer as listed, but we haven't introduce .ORDER yet. > When we throw .ORDER in: > > $ fmake -r -f /home/sjg/make-tests/order -j3 -DORDER > .ORDER: r a l p h > 1350755270 s... > 1350755270 i... > 1350755270 r... > 1350755271 g... > 1350755271 t... > 1350755271 a... > 1350755272 l... > 1350755273 p... > 1350755275 h... > $ > > This is again what we should expect, the order is not as listed but > the sequence r a l p h is correctly ordered. > > However, as soon as we thrown in an explicit dependency - not even one > that would conflict: > > $ fmake -r -f /home/sjg/make-tests/order -j3 -DORDER -DDEPS_A > .ORDER: r a l p h > a: p e > 1350755349 r... > 1350755350 i... > 1350755350 s... > 1350755351 g... > 1350755351 t... > 1350755351 e... > $ > > That is simply incorrect. The dependency conflicts with the ordering requirement, so it should be about make like a cycle does. It says that p must be built before a, but the ordering requirement says that a must be built before p. > Even if we go back to compat mode: > > $ fmake -r -f /home/sjg/make-tests/order -DORDER -DDEPS_A > .ORDER: r a l p h > a: p e > 1350755881 r... > 1350755882 p... > 1350755883 e... > 1350755884 a... > 1350755885 l... > 1350755887 h... > 1350755888 i... > 1350755889 s... > 1350755890 g... > 1350755891 t... > $ > > This is not correct either, but you can see that the dependency takes > precedence over .ORDER. The opposite precedence would break things faster, but there is no correct choice. Everthing works unsurprisingly with the dependency "p: a" that is consistent with the order. > The results with bmake are more spectaclular since it detects that it is > lost a spews its complaints: > > $ bmake -r -f /home/sjg/make-tests/order -DORDER -DDEPS_A -j3 > .ORDER: r a l p h > a: p e > --- r --- > 1350756420 r... > `p' was not built (made 1, flags 2009, type b030001)! > `p' has .ORDER dependency against l (made 0, flags 9, type b030001) > `e' was not built (made 0, flags 2009, type b030001)! > `l' was not built (made 0, flags 2009, type b030001)! > `l' has .ORDER dependency against a (made 1, flags 3019, type > b030001) > `h' was not built (made 0, flags 2009, type b030001)! > `h' has .ORDER dependency against p (made 1, flags 2009, type > b030001) > `i' was not built (made 0, flags 2009, type b030001)! > `s' was not built (made 0, flags 2009, type b030001)! > `g' was not built (made 0, flags 2009, type b030001)! > `t' was not built (made 0, flags 2009, type b030001)! > $ > > This is actually handy since you can check build logs for that sort of > thing and know you have an issue. It should detect the problem as an inconsistent makefile earlier (not its problem). The requirement for building yacc files would be even more clearly expressed by a .SEQUENTIAL directive than a .ORDER one, since we don't care about the precise order. Then there would be no pseudo- dependency and make would have to choose an order permitted by the real dependencies. make already has the .NOTPARALLEL directive, but it applies to the whole makefile. > The current logic does (for the interesting case - assuming yacc -d etc) > > parser.h: parser.y > yacc etc > > parser.c: parser.h > touch ${.TARGET} > > the only downside of this, is that if parser.c is removed > but parser.h is left alone, and empty parser.c will be produced. > This is actually a safe failure mode (build will fail), for what is > essentially a corner case. > > In all other respects the above is preferable because it works reliably > in jobs mode and it avoids any risk of running yacc multiple times to do > the same thing, and will not conflict with any auto collected > dependencies (assuming parser.c includes parser.h) Any dependency of parser.c on parser.h is a bug, whether it is in the makefile auto-collected. parser.c can't be compiled until parser.h is created, but this isn't a dependency of parser.c on parser.h; it is a dependency of the object file on both. Was the original bug just this buggy dependency? The .ORDER statement gives the opposite pseudo-dependency, so the bug might be detected accidentally as a pseudo-cycle. Bruce