Date: Wed, 7 Jan 2015 10:51:01 -0700 (MST) From: sson <sson@FreeBSD.org> To: freebsd-python@freebsd.org Subject: Re: python waf bypasses _MAKE_JOBS number Message-ID: <1420653060968-5979111.post@n5.nabble.com> In-Reply-To: <54AD5BB9.2060701@ignoranthack.me> References: <54AC74E0.4050404@ignoranthack.me> <20150107072132.GA1050@medusa.sysfault.org> <54AD5BB9.2060701@ignoranthack.me>
next in thread | previous in thread | raw e-mail | index | archive | help
Sean Bruno-6 wrote > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > On 01/07/15 00:21, Marcus von Appen wrote: >> On, Wed Jan 07, 2015, Sean Bruno wrote: >> >>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 >>> >>> Hey, so ... because qemu has a bug in it, we're trying to enforce >>> no SMP behaviour in builds. >>> >>> Turns out that python waf bypasses all of ports logic and probes >>> for the number of cpus and does its own thing. We noted this in >>> our builds as they locked up when using qemu due to a bug. >>> >>> Can this behaviour be investigated (configure behaviour) by some >>> python knowledgeable folks? >> >> Can you point us to the waf build logic for the qemu port? Looking >> at emulators/qemu, I do not see anything of relevance. >> >> Cheers Marcus >> > > Should have been a bit clearer, its qemu-bsd-user (via > qemu-user-static) that I'm talking about. > > I've added Stacey to the email to clarify a bit more. > > The behavior we are talking about isn't specific to qemu at all, its > just the configure script will probe the number of cpus and ignore my > override. > > sean > _______________________________________________ > freebsd-python@ > mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-python > To unsubscribe, send any mail to " > freebsd-python-unsubscribe@ > " Hi all: The problem we are seeing is actually described in pretty good detail in an old a bug report: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=160717 It is hard to follow what was actually changed that fixed this problem on FreeBSD/amd64 but it seems the threading code for FreeBSD/{arm, mips, mips64} may not be as mature as the amd64 port. In any case, if the "--jobs=1" flag is used then the ports will build without hanging. Python waf assumes, however, that the default jobs number should be the number of cores available on the system when the "--jobs=#" (or "-j#") is not explicitly given rather than assuming "--jobs=1". It seems, however, that many (if not all) the ports that use python waf make the assumption that it defaults to "--jobs=1". So, in summary: (1) It seems that described bug above wasn't fixed for all the FreeBSD arch's. It is unclear was the cause of the problem was and what was changed. If it is know what was changed it may be possible to figure out what needs to be changed to fix this problem in the other FreeBSD ports. (2) It seems to be assumed by many of the port maintainers that python waf defaults to "--jobs=1" when the argument is not given. This is problematic for poudriere, for example, given that it limits the MAKE_JOBS_NUMBER to 1 to better control the load on the build server. Of course, if #2 was fixed then it would solve two problems (for us anyway). :) Best Regards, -stacey. -- View this message in context: http://freebsd.1045724.n5.nabble.com/python-waf-bypasses-MAKE-JOBS-number-tp5978938p5979111.html Sent from the freebsd-python mailing list archive at Nabble.com.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1420653060968-5979111.post>