From owner-freebsd-ports@freebsd.org Sun Aug 18 12:00:28 2019 Return-Path: Delivered-To: freebsd-ports@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7CC22C5426 for ; Sun, 18 Aug 2019 12:00:28 +0000 (UTC) (envelope-from zeising@freebsd.org) Received: from mail.daemonic.se (mail.daemonic.se [176.58.89.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46BFxS0xTDz4Sfc for ; Sun, 18 Aug 2019 12:00:27 +0000 (UTC) (envelope-from zeising@freebsd.org) Received: from cid.daemonic.se (localhost [IPv6:::1]) by mail.daemonic.se (Postfix) with ESMTP id 46BFxQ2Dp3z3l4f; Sun, 18 Aug 2019 12:00:26 +0000 (UTC) X-Virus-Scanned: amavisd-new at daemonic.se Received: from mail.daemonic.se ([IPv6:::1]) (using TLS with cipher ECDHE-RSA-AES128-GCM-SHA256) by cid.daemonic.se (mailscanner.daemonic.se [IPv6:::1]) (amavisd-new, port 10587) with ESMTPS id fco8mxYfHK0X; Sun, 18 Aug 2019 12:00:25 +0000 (UTC) Received: from vivi.daemonic.se (vivi.daemonic.se [IPv6:2001:470:dca9:2::4]) by mail.daemonic.se (Postfix) with ESMTPSA id 46BFxP2zjSz3c7W; Sun, 18 Aug 2019 12:00:25 +0000 (UTC) Subject: Re: FYI: x11/xscreensaver appears to have a build-race To: Mark Millard Cc: freebsd-ports@freebsd.org References: <6EAA8CE2-3521-42D9-9B6D-30C40CE442B4@yahoo.com> <40596e0e-d455-6b34-9564-e5d0c9c1ab95@freebsd.org> From: Niclas Zeising Message-ID: <0683efc0-f9b9-93b1-8277-297cb648220c@freebsd.org> Date: Sun, 18 Aug 2019 14:00:23 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46BFxS0xTDz4Sfc X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; ASN(0.00)[asn:36236, ipnet:176.58.89.0/24, country:US] X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2019 12:00:28 -0000 On 2019-08-18 11:16, Mark Millard wrote: > [I missed an important word: "always" in front > of "completed".] > > On 2019-Aug-18, at 02:05, Mark Millard wrote: > > >> >> On 2019-Aug-18, at 01:27, Mark Millard wrote: >> >> >> >>> On 2019-Aug-18, at 01:03, Niclas Zeising wrote: >>> >>>> On 2019-08-18 09:48, Mark Millard wrote: >>>>> On 2019-Aug-18, at 00:34, Niclas Zeising wrote: >>>>>> On 2019-08-18 09:07, Mark Millard via freebsd-ports wrote: >>>>>>> I ran two separate devel/poudriere-devel amd64->aarch64 >>>>>>> cross builds on the same system (head -r 351178 based) >>>>>>> with the same /usr/ports/ tree (ports head -r509171), >>>>>>> building the same 97 ports each, mostly overlapping >>>>>>> in time, and one got: >>>>>>> gmake[2]: *** [Makefile:37: gen/apple_png.h] Error 2 >>>>>>> while building x11/xscreensaver but the other >>>>>>> completed building x11/xscreensaver just fine. >>>>>>> There was also a armv7-targeting poudriere-devel ports >>>>>>> update running, also mostly overlapping in time. With >>>>>>> 28 FreeBSD CPUs (under Hyper-V), the load average was >>>>>>> frequently over 90. (The armv7 x11/xscreensaver build >>>>>>> also completed fine.) >>>>>>> All 3 poudriere's had ALLOW_MAKE_JOBS=yes and all 3 set >>>>>>> up to have "Building 97 packages using 28 builders". >>>>>>> devel/llvm90 was one of the ports being built. All 3 >>>>>>> poudriere's had native-xtools in use. The context has >>>>>>> ECC DRAM, 96 GiBytes worth. >>>>>> >>>>>> I'm just about to update x11/xscreensaver, but through multiple poudriere runs I haven't noticed any issues. I have a much smaller machine to do builds on though. >>>>>> It would be interesting to see more context of the error as well. >>>>> Here you go: >>>> >>>> Thank you >>>> >>>>> The overall build is still in progress. So I've not had >>>>> a chance to run poudriere again to have it retry just >>>>> the 3 ports (1 failed and 2 skipped). (Presumes the rest >>>>> build okay.) >>>>> A race suggests that MAKE_JOBS_UNSAFE=yes may well be appropriate. >>>> >>>> I'll look into it. However, this is the only report I have of this issue, so it's not very common, and setting MAKE_JOBS_UNSAFE would slow down the build quite a lot. >>>> >>>> The error is a little strange, since it looks like it is when generating apple_png.h. I wonder which file it is that it complains about not being there... >>> >>> From what I can tell: >>> >>> ../../utils/bin2c apple.png gen/apple_png.h >>> >>> is supposed to produce gen/apple_png.h from the binary >>> file apple.png . Presuming that is true, the: >>> >>> gmake[2]: *** [Makefile:37: gen/apple_png.h] Error 2 >>> >>> indicates that the Makefile tried to reference >>> gen/apple_png.h before ../../utils/bin2c had gotten >>> around to creating gen/apple_png.h (the race). >> >> I looked up with "the Makefile" would be for apple_png.h >> and it was: >> >> /wrkdirs/usr/ports/x11/xscreensaver/work/xscreensaver-5.42/hacks/Makefile >> >> via its lines: >> >> bsod.o: images/gen/apple_png.h >> >> and: >> >> bsod: bsod.o $(HACK_OBJS) $(GRAB) $(APPLE2) $(PNG) >> $(CC_HACK) -o $@ $@.o $(HACK_OBJS) $(GRAB) $(APPLE2) $(PNG) $(PNG_LIBS) $(THRL) >> >> and: >> >> # Make sure the images have been packaged. This is the first one hit: >> images/gen/som_png.h: >> cd $(srcdir)/images && $(MAKE) >> images/gen/6x10font_png.h: >> cd $(srcdir)/images && $(MAKE) >> >> >> The presumption that images/gen/som_png.h and images/gen/6x10font_png.h >> will have their actions (including sub-makes) completed before other > > I should have typed: "always completed" above, not just "completed". > >> things such as "bsod.o: images/gen/apple_png.h" are attempted ( without >> MAKE_JOBS_UNSAFE=yes ) is false. This is the source of the race that I >> find in looking around. >> >> This kind of structure needs MAKE_JOBS_UNSAFE=yes for reliability. >> Otherwise a different structure is needed for reliability. >> >> >>> Similar points go for gen/atm_png.h gen/atari_png.h and >>> gen/earth_png.h . >>> >>> >>> The original 3 builds completed so I started poudriere again >>> for the failing one, without the machine being otherwise busy. >>> The result was: >>> >>> [00:00:36] [01] [00:00:00] Building x11/xscreensaver | xscreensaver-5.42 >>> [00:04:25] [01] [00:03:49] Finished x11/xscreensaver | xscreensaver-5.42: Success >>> >>> This too suggests a race condition. >>> >>> If the FreeBSD port-build servers report this sort of failure >>> at some point, you may then want to change things to avoid >>> the failures on those servers so that dependent ports are not >>> skipped. >> You are right. I'll add MAKE_JOBS_UNSAFE to the port's Makefile. Regards -- Niclas Zeising