Date: Sun, 18 Aug 2019 02:16:48 -0700 From: Mark Millard <marklmi@yahoo.com> To: Niclas Zeising <zeising@freebsd.org> Cc: freebsd-ports@freebsd.org Subject: Re: FYI: x11/xscreensaver appears to have a build-race Message-ID: <E558C32A-99EF-4BE7-A3E1-DC44C12448DA@yahoo.com> In-Reply-To: <E36205E2-5732-4155-B5A3-CDB5024EA01C@yahoo.com> References: <6EAA8CE2-3521-42D9-9B6D-30C40CE442B4@yahoo.com> <40596e0e-d455-6b34-9564-e5d0c9c1ab95@freebsd.org> <BDF3EF0C-1616-4795-8B98-DE6305E35A2A@yahoo.com> <ed9946a1-978f-921b-a17b-7c8b47e28a4b@freebsd.org> <B5C40D5D-A558-4325-95AC-11A027A280CD@yahoo.com> <E36205E2-5732-4155-B5A3-CDB5024EA01C@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[I missed an important word: "always" in front of "completed".] On 2019-Aug-18, at 02:05, Mark Millard <marklmi at yahoo.com> wrote: >=20 > On 2019-Aug-18, at 01:27, Mark Millard <marklmi at yahoo.com> wrote: >=20 >=20 >=20 >> On 2019-Aug-18, at 01:03, Niclas Zeising <zeising at freebsd.org> = wrote: >>=20 >>> On 2019-08-18 09:48, Mark Millard wrote: >>>> On 2019-Aug-18, at 00:34, Niclas Zeising <zeising at freebsd.org> = wrote: >>>>> On 2019-08-18 09:07, Mark Millard via freebsd-ports wrote: >>>>>> I ran two separate devel/poudriere-devel amd64->aarch64 >>>>>> cross builds on the same system (head -r 351178 based) >>>>>> with the same /usr/ports/ tree (ports head -r509171), >>>>>> building the same 97 ports each, mostly overlapping >>>>>> in time, and one got: >>>>>> gmake[2]: *** [Makefile:37: gen/apple_png.h] Error 2 >>>>>> while building x11/xscreensaver but the other >>>>>> completed building x11/xscreensaver just fine. >>>>>> There was also a armv7-targeting poudriere-devel ports >>>>>> update running, also mostly overlapping in time. With >>>>>> 28 FreeBSD CPUs (under Hyper-V), the load average was >>>>>> frequently over 90. (The armv7 x11/xscreensaver build >>>>>> also completed fine.) >>>>>> All 3 poudriere's had ALLOW_MAKE_JOBS=3Dyes and all 3 set >>>>>> up to have "Building 97 packages using 28 builders". >>>>>> devel/llvm90 was one of the ports being built. All 3 >>>>>> poudriere's had native-xtools in use. The context has >>>>>> ECC DRAM, 96 GiBytes worth. >>>>>=20 >>>>> I'm just about to update x11/xscreensaver, but through multiple = poudriere runs I haven't noticed any issues. I have a much smaller = machine to do builds on though. >>>>> It would be interesting to see more context of the error as well. >>>> Here you go: >>>=20 >>> Thank you >>>=20 >>>> The overall build is still in progress. So I've not had >>>> a chance to run poudriere again to have it retry just >>>> the 3 ports (1 failed and 2 skipped). (Presumes the rest >>>> build okay.) >>>> A race suggests that MAKE_JOBS_UNSAFE=3Dyes may well be = appropriate. >>>=20 >>> I'll look into it. However, this is the only report I have of this = issue, so it's not very common, and setting MAKE_JOBS_UNSAFE would slow = down the build quite a lot. >>>=20 >>> The error is a little strange, since it looks like it is when = generating apple_png.h. I wonder which file it is that it complains = about not being there... >>=20 >> =46rom what I can tell: >>=20 >> ../../utils/bin2c apple.png gen/apple_png.h >>=20 >> is supposed to produce gen/apple_png.h from the binary >> file apple.png . Presuming that is true, the: >>=20 >> gmake[2]: *** [Makefile:37: gen/apple_png.h] Error 2 >>=20 >> indicates that the Makefile tried to reference >> gen/apple_png.h before ../../utils/bin2c had gotten >> around to creating gen/apple_png.h (the race). >=20 > I looked up with "the Makefile" would be for apple_png.h > and it was: >=20 > = /wrkdirs/usr/ports/x11/xscreensaver/work/xscreensaver-5.42/hacks/Makefile >=20 > via its lines: >=20 > bsod.o: images/gen/apple_png.h >=20 > and: >=20 > bsod: bsod.o $(HACK_OBJS) $(GRAB) $(APPLE2) $(PNG) > $(CC_HACK) -o $@ $@.o $(HACK_OBJS) $(GRAB) $(APPLE2) $(PNG) = $(PNG_LIBS) $(THRL) >=20 > and: >=20 > # Make sure the images have been packaged. This is the first one hit: > images/gen/som_png.h: > cd $(srcdir)/images && $(MAKE) > images/gen/6x10font_png.h: > cd $(srcdir)/images && $(MAKE) >=20 >=20 > The presumption that images/gen/som_png.h and = images/gen/6x10font_png.h > will have their actions (including sub-makes) completed before other I should have typed: "always completed" above, not just "completed". > things such as "bsod.o: images/gen/apple_png.h" are attempted ( = without > MAKE_JOBS_UNSAFE=3Dyes ) is false. This is the source of the race that = I > find in looking around. >=20 > This kind of structure needs MAKE_JOBS_UNSAFE=3Dyes for reliability. > Otherwise a different structure is needed for reliability. >=20 >=20 >> Similar points go for gen/atm_png.h gen/atari_png.h and >> gen/earth_png.h . >>=20 >>=20 >> The original 3 builds completed so I started poudriere again >> for the failing one, without the machine being otherwise busy. >> The result was: >>=20 >> [00:00:36] [01] [00:00:00] Building x11/xscreensaver | = xscreensaver-5.42 >> [00:04:25] [01] [00:03:49] Finished x11/xscreensaver | = xscreensaver-5.42: Success >>=20 >> This too suggests a race condition. >>=20 >> If the FreeBSD port-build servers report this sort of failure >> at some point, you may then want to change things to avoid >> the failures on those servers so that dependent ports are not >> skipped. >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E558C32A-99EF-4BE7-A3E1-DC44C12448DA>