From owner-freebsd-ports@freebsd.org Sun Aug 18 09:16:56 2019 Return-Path: Delivered-To: freebsd-ports@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F2E4EC1D47 for ; Sun, 18 Aug 2019 09:16:56 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-56.consmr.mail.ne1.yahoo.com (sonic308-56.consmr.mail.ne1.yahoo.com [66.163.187.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46BBJl5n4hz4KZt for ; Sun, 18 Aug 2019 09:16:55 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: U00HO7YVM1nccMirjtuCdwO44OmBQU9Nx7WDWfn4pM9f5nT0TIHt.4Zbi3MIM6f 8_9pk19vVibldjvyY9qBNX53ed5ZV_C3aYjWtf6v1innzgygixgYh7AHfsdbDk34xL2Vkm_IFNum JSU6pcXH_iGQ5QFtgkRM7.fLgukI.RFW280kjk5f.Wet2dFvCZA4DZMAgvjiaikJumTPeHTK4feV bW0WwyAW5P10IGzy0hwXBkGGQc6WMb7O2ha.uOgoPYqHAnwTmVknRdU7yKDG9yr51NaXHvpDU_lD DlElKhXYMD7QdWQ1d4dvSeZXc93R.ZTr1wsQ962BgMxrRHGdngbJWfgTyxFL1qllJE9lO7AvozCD 021Q0rJ3PRYC5hSJ8wnaUBN8uFZgM3k1gGTfKtNi2Xr6y19LWhev92F0zk1S.bhy8gTgeA2yWDok wVINRIsfiTZH9yNJV4GAwO2NPU0vSN1MW9xhVaEei1kpwiP.jy1liaizUhc7mTrV.bkgv2jWFDut OFU_2mfzq3btI8XXUIwgaOe_jZaQEr6912kiJ_wtrIrnjPvbL2fGdtpR2v9wgq4EAvtF0IG8u3hu RKLeqiBi1tWK2aViKdiR25DJjTcrfgu5r2klYDuIoaq23tbaTKYptB9Hfe0.CSnCmQFXz7rNYte9 co.gC51FGdg9LIuUccNvKfDw2XScUKIaWvd1n7wIRQkbgG_.NcIAUUPer0vOzpYk5w_9UuHuUFbT nc9.s65xTYMsC26Bq8wpXXxdQDiRJpgsMjNJr6mwh58dYpm9p78zuwjRqmmhXlgC3xkZfMnAfVMC pFtjfA4Qwx7SkUjnN5tLd2YlcgnQFc1Z1_UCCckW50Me1p9zGkD4..DkIwIp_SpbujCXrDEp9Yhw e2xBPreBjUcNsD5YwsiBFg6o902Bgv7zp7ouBI3XZwODOSQ9TXl3cuVFuCdo5g_p2His0M4gEdkz lYStw24rdTrVJUOTFX3nduXR6nAR9TiUvb67hsjblT8ezNIOzqDJzh5gxljL1tk..VLWGZ06LOzJ zwNNN4uS7blzWbYtfSRjBmiKovEt7tFKkOYtycSU7EWOjSHFGHMZOJ0x9eEJHLhEYQv52pXwWe6g LCM_beQhVumZjwtJGMEBMnWCjmzazOo_3AT52AWmeD9n2rg9kxdFqjYVUfXYmx4fY9Ys2qAkCaYM tbKCI9XO04HLSOFBvAYKS_TJxtgBBBMFkakrfruTPlU0VoCT1TOJAuqYPJBHFDK8Vo1WQxEDOPIg fgvg1O6TDmHZpqhOZb0F0 Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.ne1.yahoo.com with HTTP; Sun, 18 Aug 2019 09:16:54 +0000 Received: by smtp418.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID a157f6d3e913e9592c38c05ed1e24817; Sun, 18 Aug 2019 09:16:51 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: FYI: x11/xscreensaver appears to have a build-race From: Mark Millard In-Reply-To: Date: Sun, 18 Aug 2019 02:16:48 -0700 Cc: freebsd-ports@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <6EAA8CE2-3521-42D9-9B6D-30C40CE442B4@yahoo.com> <40596e0e-d455-6b34-9564-e5d0c9c1ab95@freebsd.org> To: Niclas Zeising X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 46BBJl5n4hz4KZt X-Spamd-Bar: - X-Spamd-Result: default: False [-1.48 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.72)[-0.717,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.64)[0.645,0]; NEURAL_HAM_LONG(-0.91)[-0.912,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(0.00)[ip: (3.18), ipnet: 66.163.184.0/21(1.28), asn: 36646(1.02), country: US(-0.05)]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[31.187.163.66.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2019 09:16:57 -0000 [I missed an important word: "always" in front of "completed".] On 2019-Aug-18, at 02:05, Mark Millard wrote: >=20 > On 2019-Aug-18, at 01:27, Mark Millard wrote: >=20 >=20 >=20 >> On 2019-Aug-18, at 01:03, Niclas Zeising = wrote: >>=20 >>> On 2019-08-18 09:48, Mark Millard wrote: >>>> On 2019-Aug-18, at 00:34, Niclas Zeising = wrote: >>>>> On 2019-08-18 09:07, Mark Millard via freebsd-ports wrote: >>>>>> I ran two separate devel/poudriere-devel amd64->aarch64 >>>>>> cross builds on the same system (head -r 351178 based) >>>>>> with the same /usr/ports/ tree (ports head -r509171), >>>>>> building the same 97 ports each, mostly overlapping >>>>>> in time, and one got: >>>>>> gmake[2]: *** [Makefile:37: gen/apple_png.h] Error 2 >>>>>> while building x11/xscreensaver but the other >>>>>> completed building x11/xscreensaver just fine. >>>>>> There was also a armv7-targeting poudriere-devel ports >>>>>> update running, also mostly overlapping in time. With >>>>>> 28 FreeBSD CPUs (under Hyper-V), the load average was >>>>>> frequently over 90. (The armv7 x11/xscreensaver build >>>>>> also completed fine.) >>>>>> All 3 poudriere's had ALLOW_MAKE_JOBS=3Dyes and all 3 set >>>>>> up to have "Building 97 packages using 28 builders". >>>>>> devel/llvm90 was one of the ports being built. All 3 >>>>>> poudriere's had native-xtools in use. The context has >>>>>> ECC DRAM, 96 GiBytes worth. >>>>>=20 >>>>> I'm just about to update x11/xscreensaver, but through multiple = poudriere runs I haven't noticed any issues. I have a much smaller = machine to do builds on though. >>>>> It would be interesting to see more context of the error as well. >>>> Here you go: >>>=20 >>> Thank you >>>=20 >>>> The overall build is still in progress. So I've not had >>>> a chance to run poudriere again to have it retry just >>>> the 3 ports (1 failed and 2 skipped). (Presumes the rest >>>> build okay.) >>>> A race suggests that MAKE_JOBS_UNSAFE=3Dyes may well be = appropriate. >>>=20 >>> I'll look into it. However, this is the only report I have of this = issue, so it's not very common, and setting MAKE_JOBS_UNSAFE would slow = down the build quite a lot. >>>=20 >>> The error is a little strange, since it looks like it is when = generating apple_png.h. I wonder which file it is that it complains = about not being there... >>=20 >> =46rom what I can tell: >>=20 >> ../../utils/bin2c apple.png gen/apple_png.h >>=20 >> is supposed to produce gen/apple_png.h from the binary >> file apple.png . Presuming that is true, the: >>=20 >> gmake[2]: *** [Makefile:37: gen/apple_png.h] Error 2 >>=20 >> indicates that the Makefile tried to reference >> gen/apple_png.h before ../../utils/bin2c had gotten >> around to creating gen/apple_png.h (the race). >=20 > I looked up with "the Makefile" would be for apple_png.h > and it was: >=20 > = /wrkdirs/usr/ports/x11/xscreensaver/work/xscreensaver-5.42/hacks/Makefile >=20 > via its lines: >=20 > bsod.o: images/gen/apple_png.h >=20 > and: >=20 > bsod: bsod.o $(HACK_OBJS) $(GRAB) $(APPLE2) $(PNG) > $(CC_HACK) -o $@ $@.o $(HACK_OBJS) $(GRAB) $(APPLE2) $(PNG) = $(PNG_LIBS) $(THRL) >=20 > and: >=20 > # Make sure the images have been packaged. This is the first one hit: > images/gen/som_png.h: > cd $(srcdir)/images && $(MAKE) > images/gen/6x10font_png.h: > cd $(srcdir)/images && $(MAKE) >=20 >=20 > The presumption that images/gen/som_png.h and = images/gen/6x10font_png.h > will have their actions (including sub-makes) completed before other I should have typed: "always completed" above, not just "completed". > things such as "bsod.o: images/gen/apple_png.h" are attempted ( = without > MAKE_JOBS_UNSAFE=3Dyes ) is false. This is the source of the race that = I > find in looking around. >=20 > This kind of structure needs MAKE_JOBS_UNSAFE=3Dyes for reliability. > Otherwise a different structure is needed for reliability. >=20 >=20 >> Similar points go for gen/atm_png.h gen/atari_png.h and >> gen/earth_png.h . >>=20 >>=20 >> The original 3 builds completed so I started poudriere again >> for the failing one, without the machine being otherwise busy. >> The result was: >>=20 >> [00:00:36] [01] [00:00:00] Building x11/xscreensaver | = xscreensaver-5.42 >> [00:04:25] [01] [00:03:49] Finished x11/xscreensaver | = xscreensaver-5.42: Success >>=20 >> This too suggests a race condition. >>=20 >> If the FreeBSD port-build servers report this sort of failure >> at some point, you may then want to change things to avoid >> the failures on those servers so that dependent ports are not >> skipped. >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)