Date: Sun, 18 Aug 2019 14:00:23 +0200 From: Niclas Zeising <zeising@freebsd.org> To: Mark Millard <marklmi@yahoo.com> Cc: freebsd-ports@freebsd.org Subject: Re: FYI: x11/xscreensaver appears to have a build-race Message-ID: <0683efc0-f9b9-93b1-8277-297cb648220c@freebsd.org> In-Reply-To: <E558C32A-99EF-4BE7-A3E1-DC44C12448DA@yahoo.com> References: <6EAA8CE2-3521-42D9-9B6D-30C40CE442B4@yahoo.com> <40596e0e-d455-6b34-9564-e5d0c9c1ab95@freebsd.org> <BDF3EF0C-1616-4795-8B98-DE6305E35A2A@yahoo.com> <ed9946a1-978f-921b-a17b-7c8b47e28a4b@freebsd.org> <B5C40D5D-A558-4325-95AC-11A027A280CD@yahoo.com> <E36205E2-5732-4155-B5A3-CDB5024EA01C@yahoo.com> <E558C32A-99EF-4BE7-A3E1-DC44C12448DA@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2019-08-18 11:16, Mark Millard wrote: > [I missed an important word: "always" in front > of "completed".] > > On 2019-Aug-18, at 02:05, Mark Millard <marklmi at yahoo.com> wrote: > > >> >> On 2019-Aug-18, at 01:27, Mark Millard <marklmi at yahoo.com> wrote: >> >> >> >>> On 2019-Aug-18, at 01:03, Niclas Zeising <zeising at freebsd.org> wrote: >>> >>>> On 2019-08-18 09:48, Mark Millard wrote: >>>>> On 2019-Aug-18, at 00:34, Niclas Zeising <zeising at freebsd.org> wrote: >>>>>> On 2019-08-18 09:07, Mark Millard via freebsd-ports wrote: >>>>>>> I ran two separate devel/poudriere-devel amd64->aarch64 >>>>>>> cross builds on the same system (head -r 351178 based) >>>>>>> with the same /usr/ports/ tree (ports head -r509171), >>>>>>> building the same 97 ports each, mostly overlapping >>>>>>> in time, and one got: >>>>>>> gmake[2]: *** [Makefile:37: gen/apple_png.h] Error 2 >>>>>>> while building x11/xscreensaver but the other >>>>>>> completed building x11/xscreensaver just fine. >>>>>>> There was also a armv7-targeting poudriere-devel ports >>>>>>> update running, also mostly overlapping in time. With >>>>>>> 28 FreeBSD CPUs (under Hyper-V), the load average was >>>>>>> frequently over 90. (The armv7 x11/xscreensaver build >>>>>>> also completed fine.) >>>>>>> All 3 poudriere's had ALLOW_MAKE_JOBS=yes and all 3 set >>>>>>> up to have "Building 97 packages using 28 builders". >>>>>>> devel/llvm90 was one of the ports being built. All 3 >>>>>>> poudriere's had native-xtools in use. The context has >>>>>>> ECC DRAM, 96 GiBytes worth. >>>>>> >>>>>> I'm just about to update x11/xscreensaver, but through multiple poudriere runs I haven't noticed any issues. I have a much smaller machine to do builds on though. >>>>>> It would be interesting to see more context of the error as well. >>>>> Here you go: >>>> >>>> Thank you >>>> >>>>> The overall build is still in progress. So I've not had >>>>> a chance to run poudriere again to have it retry just >>>>> the 3 ports (1 failed and 2 skipped). (Presumes the rest >>>>> build okay.) >>>>> A race suggests that MAKE_JOBS_UNSAFE=yes may well be appropriate. >>>> >>>> I'll look into it. However, this is the only report I have of this issue, so it's not very common, and setting MAKE_JOBS_UNSAFE would slow down the build quite a lot. >>>> >>>> The error is a little strange, since it looks like it is when generating apple_png.h. I wonder which file it is that it complains about not being there... >>> >>> From what I can tell: >>> >>> ../../utils/bin2c apple.png gen/apple_png.h >>> >>> is supposed to produce gen/apple_png.h from the binary >>> file apple.png . Presuming that is true, the: >>> >>> gmake[2]: *** [Makefile:37: gen/apple_png.h] Error 2 >>> >>> indicates that the Makefile tried to reference >>> gen/apple_png.h before ../../utils/bin2c had gotten >>> around to creating gen/apple_png.h (the race). >> >> I looked up with "the Makefile" would be for apple_png.h >> and it was: >> >> /wrkdirs/usr/ports/x11/xscreensaver/work/xscreensaver-5.42/hacks/Makefile >> >> via its lines: >> >> bsod.o: images/gen/apple_png.h >> >> and: >> >> bsod: bsod.o $(HACK_OBJS) $(GRAB) $(APPLE2) $(PNG) >> $(CC_HACK) -o $@ $@.o $(HACK_OBJS) $(GRAB) $(APPLE2) $(PNG) $(PNG_LIBS) $(THRL) >> >> and: >> >> # Make sure the images have been packaged. This is the first one hit: >> images/gen/som_png.h: >> cd $(srcdir)/images && $(MAKE) >> images/gen/6x10font_png.h: >> cd $(srcdir)/images && $(MAKE) >> >> >> The presumption that images/gen/som_png.h and images/gen/6x10font_png.h >> will have their actions (including sub-makes) completed before other > > I should have typed: "always completed" above, not just "completed". > >> things such as "bsod.o: images/gen/apple_png.h" are attempted ( without >> MAKE_JOBS_UNSAFE=yes ) is false. This is the source of the race that I >> find in looking around. >> >> This kind of structure needs MAKE_JOBS_UNSAFE=yes for reliability. >> Otherwise a different structure is needed for reliability. >> >> >>> Similar points go for gen/atm_png.h gen/atari_png.h and >>> gen/earth_png.h . >>> >>> >>> The original 3 builds completed so I started poudriere again >>> for the failing one, without the machine being otherwise busy. >>> The result was: >>> >>> [00:00:36] [01] [00:00:00] Building x11/xscreensaver | xscreensaver-5.42 >>> [00:04:25] [01] [00:03:49] Finished x11/xscreensaver | xscreensaver-5.42: Success >>> >>> This too suggests a race condition. >>> >>> If the FreeBSD port-build servers report this sort of failure >>> at some point, you may then want to change things to avoid >>> the failures on those servers so that dependent ports are not >>> skipped. >> You are right. I'll add MAKE_JOBS_UNSAFE to the port's Makefile. Regards -- Niclas Zeising
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0683efc0-f9b9-93b1-8277-297cb648220c>