From owner-svn-src-head@FreeBSD.ORG Fri Jul 5 16:08:13 2013 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id E6E67BD5; Fri, 5 Jul 2013 16:08:13 +0000 (UTC) (envelope-from tijl@freebsd.org) Received: from mailrelay011.isp.belgacom.be (mailrelay011.isp.belgacom.be [195.238.6.178]) by mx1.freebsd.org (Postfix) with ESMTP id EFD0815D5; Fri, 5 Jul 2013 16:08:12 +0000 (UTC) X-Belgacom-Dynamic: yes X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AlUGAMPu1lFR8Z+L/2dsb2JhbABagwkywQeBABd0giMBAQUnLyMQCxQECRYPCQMCAQIBJx4GDQEHAQGIDwi5BI9aEQeDbQOQCYEthzyQHIMTOg Received: from 139.159-241-81.adsl-dyn.isp.belgacom.be (HELO kalimero.tijl.coosemans.org) ([81.241.159.139]) by relay.skynet.be with ESMTP; 05 Jul 2013 18:06:43 +0200 Received: from kalimero.tijl.coosemans.org (kalimero.tijl.coosemans.org [127.0.0.1]) by kalimero.tijl.coosemans.org (8.14.7/8.14.7) with ESMTP id r65G6gV7003640; Fri, 5 Jul 2013 18:06:42 +0200 (CEST) (envelope-from tijl@FreeBSD.org) Message-ID: <51D6EF0D.3050506@FreeBSD.org> Date: Fri, 05 Jul 2013 18:06:37 +0200 From: Tijl Coosemans User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:17.0) Gecko/20130701 Thunderbird/17.0.7 MIME-Version: 1.0 To: "Simon J. Gerraty" Subject: Re: svn commit: r251422 - in head: contrib/bmake usr.bin/bmake References: <201306051612.r55GCpPG050941@svn.freebsd.org> <51C8967F.2060905@FreeBSD.org> In-Reply-To: <51C8967F.2060905@FreeBSD.org> X-Enigmail-Version: 1.5.1 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="----enig2VVTBMJRSHSAICIPQPDED" Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jul 2013 16:08:14 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) ------enig2VVTBMJRSHSAICIPQPDED Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: quoted-printable On 2013-06-24 20:57, Tijl Coosemans wrote: > On 2013-06-05 18:12, Simon J. Gerraty wrote: >> Author: sjg >> Date: Wed Jun 5 16:12:50 2013 >> New Revision: 251422 >> URL: http://svnweb.freebsd.org/changeset/base/251422 >> >> Log: >> Update to bmake-20130604 to fix file descriptor leak. >> >> Modified: head/contrib/bmake/job.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D >> --- head/contrib/bmake/job.c Wed Jun 5 15:52:24 2013 (r251421) >> +++ head/contrib/bmake/job.c Wed Jun 5 16:12:50 2013 (r251422) >> @@ -1,4 +1,4 @@ >> -/* $NetBSD: job.c,v 1.172 2013/03/05 22:01:43 christos Exp $ */ >> +/* $NetBSD: job.c,v 1.173 2013/06/05 03:59:43 sjg Exp $ */ >> =20 >> /* >> * Copyright (c) 1988, 1989, 1990 The Regents of the University of Ca= lifornia. >> @@ -70,14 +70,14 @@ >> */ >> =20 >> #ifndef MAKE_NATIVE >> -static char rcsid[] =3D "$NetBSD: job.c,v 1.172 2013/03/05 22:01:43 c= hristos Exp $"; >> +static char rcsid[] =3D "$NetBSD: job.c,v 1.173 2013/06/05 03:59:43 s= jg Exp $"; >> #else >> #include >> #ifndef lint >> #if 0 >> static char sccsid[] =3D "@(#)job.c 8.2 (Berkeley) 3/19/94"; >> #else >> -__RCSID("$NetBSD: job.c,v 1.172 2013/03/05 22:01:43 christos Exp $");= >> +__RCSID("$NetBSD: job.c,v 1.173 2013/06/05 03:59:43 sjg Exp $"); >> #endif >> #endif /* not lint */ >> #endif >> @@ -414,6 +414,15 @@ JobCreatePipe(Job *job, int minfd) >> if (pipe(job->jobPipe) =3D=3D -1) >> Punt("Cannot create pipe: %s", strerror(errno)); >> =20 >> + for (i =3D 0; i < 2; i++) { >> + /* Avoid using low numbered fds */ >> + fd =3D fcntl(job->jobPipe[i], F_DUPFD, minfd); >> + if (fd !=3D -1) { >> + close(job->jobPipe[i]); >> + job->jobPipe[i] =3D fd; >> + } >> + } >> + =20 >> /* Set close-on-exec flag for both */ >> (void)fcntl(job->jobPipe[0], F_SETFD, 1); >> (void)fcntl(job->jobPipe[1], F_SETFD, 1); >=20 > I've been noticing that bmake doesn't run parallel jobs as like fmake. > I've attached a Makefile that I think shows what's going wrong. >=20 > If you run "make -j4" it outputs the following: >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- all --- > -j 4 -i -J 15,16 > 4 > -j 4 -i > 4 > --- sub_2 --- > -j 4 -i -J 15,16 > 4 > -j 4 -i > 4 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D >=20 > Bmake outputs the target name in -j mode (e.g. "--- all ---"), but > there's no "--- sub_1 ---" and "--- sub_3 ---" which suggests -j isn't > working there. The -J flag also doesn't appear in .MAKEFLAGS in those > targets. >=20 > I suspect the descriptors for the job server have to remain open so > submakes can pick them up. At least, when I comment out the two fcntl > calls above (and two more below), I do get the output I expect: >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- all --- > -j 4 -J 15,16 > 4 > --- sub_1 --- > -j 4 -J 15,16 > 4 > --- sub_2 --- > -j 4 -J 15,16 > 4 > --- sub_3 --- > -j 4 -J 15,16 > 4 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D >=20 >> @@ -426,15 +435,6 @@ JobCreatePipe(Job *job, int minfd) >> */ >> fcntl(job->jobPipe[0], F_SETFL,=20 >> fcntl(job->jobPipe[0], F_GETFL, 0) | O_NONBLOCK); >> - >> - for (i =3D 0; i < 2; i++) { >> - /* Avoid using low numbered fds */ >> - fd =3D fcntl(job->jobPipe[i], F_DUPFD, minfd); >> - if (fd !=3D -1) { >> - close(job->jobPipe[i]); >> - job->jobPipe[i] =3D fd; >> - } >> - } >> } >> =20 >> /*- >> @@ -2828,6 +2828,8 @@ Job_ServerStart(int max_tokens, int jp_0 >> /* Pipe passed in from parent */ >> tokenWaitJob.inPipe =3D jp_0; >> tokenWaitJob.outPipe =3D jp_1; >> + (void)fcntl(jp_0, F_SETFD, 1); >> + (void)fcntl(jp_1, F_SETFD, 1); >=20 > These two fcntl calls have to be commented out too. When those four lines are commented out buildworld with four jobs runs 10% faster. Now that I've taken a closer look at the code it seems bmake requires targets that run make to be marked with .MAKE. Several targets in Makefile and Makefile.inc1 already have this, but some don't, such as bootstrap-tools. Can this be added there? ------enig2VVTBMJRSHSAICIPQPDED Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iF4EAREIAAYFAlHW7xIACgkQfoCS2CCgtitdMwD/eBS2UgwGm2AEexZWdQaExWHO GLcYSeiesB3OeHmlbokA/jce+G1c32CWRUq3oHeT4pS1myBWROR182OkX8iiMBP1 =3O1T -----END PGP SIGNATURE----- ------enig2VVTBMJRSHSAICIPQPDED--