Date: Fri, 09 Jan 2015 18:33:34 +0100 From: Marcus von Appen <mva@freebsd.org> To: "sson [via FreeBSD]" <ml-node+s1045724n5979111h21@n5.nabble.com> Cc: sbruno@freebsd.org, python@freebsd.org Subject: Re: python waf bypasses _MAKE_JOBS number Message-ID: <20150109183334.Horde.DDLUOEFT2TrMFUcP63h_9Q1@webmail.df.eu> In-Reply-To: <1420653060968-5979111.post@n5.nabble.com> References: <54AC74E0.4050404@ignoranthack.me> <20150107072132.GA1050@medusa.sysfault.org> <54AD5BB9.2060701@ignoranthack.me> <1420653060968-5979111.post@n5.nabble.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Quoting "sson [via FreeBSD]" <ml-node+s1045724n5979111h21@n5.nabble.com>: > Sean Bruno-6 wrote >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA512 >> >> On 01/07/15 00:21, Marcus von Appen wrote: >>> On, Wed Jan 07, 2015, Sean Bruno wrote: >>> >>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 >>>> >>>> Hey, so ... because qemu has a bug in it, we're trying to enforce >>>> no SMP behaviour in builds. >>>> >>>> Turns out that python waf bypasses all of ports logic and probes >>>> for the number of cpus and does its own thing. We noted this in >>>> our builds as they locked up when using qemu due to a bug. >>>> >>>> Can this behaviour be investigated (configure behaviour) by some >>>> python knowledgeable folks? >>> >>> Can you point us to the waf build logic for the qemu port? Looking >>> at emulators/qemu, I do not see anything of relevance. >>> >>> Cheers Marcus >>> >> >> Should have been a bit clearer, its qemu-bsd-user (via >> qemu-user-static) that I'm talking about. >> >> I've added Stacey to the email to clarify a bit more. >> >> The behavior we are talking about isn't specific to qemu at all, its >> just the configure script will probe the number of cpus and ignore my >> override. >> >> sean >> _______________________________________________ > >> freebsd-python@ > >> mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-python >> To unsubscribe, send any mail to " > >> freebsd-python-unsubscribe@ > >> " > > Hi all: > > The problem we are seeing is actually described in pretty good detail in an > old a bug report: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=160717 > > It is hard to follow what was actually changed that fixed this problem on > FreeBSD/amd64 but it seems the threading code for FreeBSD/{arm, mips, > mips64} may not be as mature as the amd64 port. In any case, if the So this is not a general issue, but a platform-specific one? > "--jobs=1" flag is used then the ports will build without hanging. Python > waf assumes, however, that the default jobs number should be the number of > cores available on the system when the "--jobs=#" (or "-j#") is not > explicitly given rather than assuming "--jobs=1". It seems, however, that > many (if not all) the ports that use python waf make the assumption that it > defaults to "--jobs=1". So, in summary: So, the waf build system does things, it should not do, but instead respect our wish for _MAKE_JOBS - but only on the platforms, on which it currently breaks? > (1) It seems that described bug above wasn't fixed for all the FreeBSD > arch's. It is unclear was the cause of the problem was and what was > changed. If it is know what was changed it may be possible to figure out > what needs to be changed to fix this problem in the other FreeBSD ports. > > (2) It seems to be assumed by many of the port maintainers that python waf > defaults to "--jobs=1" when the argument is not given. This is problematic > for poudriere, for example, given that it limits the MAKE_JOBS_NUMBER to 1 > to better control the load on the build server. I doubt this. Most port maintainers more likely assume: if the port builds and runs on $platform (amd64 and i386 for the majority), everything is good. > Of course, if #2 was fixed then it would solve two problems (for us anyway). > :) Point me to the correct script/location and I'll look through the code. Right now I do not understand enough of the qemu build infrastructure to find my way around it. Especially since I do not find the typical "waf" scripts within the extracted upstream sources. Cheers Marcus
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150109183334.Horde.DDLUOEFT2TrMFUcP63h_9Q1>