From owner-freebsd-ports@freebsd.org Sun Aug 18 09:05:18 2019 Return-Path: Delivered-To: freebsd-ports@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EE258C17C3 for ; Sun, 18 Aug 2019 09:05:18 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-21.consmr.mail.gq1.yahoo.com (sonic314-21.consmr.mail.gq1.yahoo.com [98.137.69.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46BB3J50DRz4K1f for ; Sun, 18 Aug 2019 09:05:16 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: cK2I8_IVM1nZyfvNlzPwqHVur4LvGotvWPUsQNiERuzWtO3.avFWKIv0_LuiUq5 nMKH.Vg_GOZl.4D7IKBTErQyKshLN8w_kiAx.Yha6RNOOfIanf7f1xNzwdy.7LqYXa2uPWoLWR4F cVADstAnGt7xcUdzxX8ztdTYALlD2mMp6OSZ6cfTSoUeZL7iScVTQGw56vGXWB0MDiXGbPFQEmDA BRTpZCY_0vv2kzlSPd4sie.SwnLp.5S7.YO_fdHMhsvfm4cYVQF3X1IH5d4YEbaby8sOuePFjCGD DmmH_igS_nf2KjIMP19J4GLXXuSbAI.HOaKDsf9j09SOsY9_UHc9UkN0_ca_SuqLCEcrQzU3lYlL reiwXEmB6FI_XfNkVKLaAa.1v85xlDVP4T.2xdK.xuMKu1_W0_Sa2bXkNRjD4iSEhmhwymRV4nrs Ty6EvTnLZOwKQDPnjzlCIY7CSZUoG9iXaN6V5ezV2uEq_BA2tzSgy8ULTBKsITXZgujCanFZBUfN 2jaIMYxzX3.QZLsHiQZ789HXPPaCrz4I5RSU5243jZk3uakW3ki0nNsjuL_tIM4Tsu6R40cE5dMr RlgD4h96s_x9FW.k_eX2e4ZF1tVI5PT3I0Bm.mu2yvZeKjHleKmFNX2QZ0PTc8kstZs87QuK0cZ6 DnD65vtwfLt.FKGI3P2HA5frMxZfmG.ynSkulXGcZPT5eKyJo2zuBAu4mBCWOgSErqmmJKesavXY LTrbI0HXulPqtSHl7UaaFwhyL7xMPELpFmKQHAiUJadbzq36nDXSSY3hWFQjaeWCUoh0.jng5zKL TCJ.Qac2jUfKyMJPPq5mf4RnykSTpPV7Dprvy1Tx4fqrxu1xR0NYrqdq13mSNhJMDE3EGHllFDXZ Slb7SVfkFZkoKFVEnL7xw8j6BcTfEuEjxNd2jjrDk5Lkkk2zPnm8iJycf9B5PRdkDMOSuVUoa5JC GTRvjz4UUKWrTfwC1Di7l5LfPbD.JWiOAkl4Ww2e4Q10FG.MG_6LQBVokCd0NY91FxuU2HfMjd8A GYVIHLGDfCuxEKYVrsjQRtnDBGNRCoQSD4fylK8oNgaWfa8awipYRhrrfDan6nYr50c5e2t2ZMBV ZDpkMscJ1bIuOg3rnTfJK9k2nMSxzrN3rhbnGS2VEpNBHPn3uiQ5S14CZgDeY1ylm8XgKcTmdjLC cnc7wMn.ogRPeIXgGUXn61LK5jwT.WzdCBwdIG9GY_M2wFzoIy0i5dSm1ijK68jr33CSxeQ7WHQv Q_N3sRTiphkmBy6.a Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Sun, 18 Aug 2019 09:05:14 +0000 Received: by smtp422.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID f5ad12736c261e18edf83e5a044005ab; Sun, 18 Aug 2019 09:05:10 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: FYI: x11/xscreensaver appears to have a build-race From: Mark Millard In-Reply-To: Date: Sun, 18 Aug 2019 02:05:10 -0700 Cc: freebsd-ports@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <6EAA8CE2-3521-42D9-9B6D-30C40CE442B4@yahoo.com> <40596e0e-d455-6b34-9564-e5d0c9c1ab95@freebsd.org> To: Niclas Zeising X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 46BB3J50DRz4K1f X-Spamd-Bar: / X-Spamd-Result: default: False [-0.85 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.35)[-0.350,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.78)[0.784,0]; NEURAL_HAM_LONG(-0.79)[-0.789,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(0.00)[ip: (6.81), ipnet: 98.137.64.0/21(0.91), asn: 36647(0.73), country: US(-0.05)]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[84.69.137.98.list.dnswl.org : 127.0.5.0]; RBL_BLOCKLISTDE_FAIL(0.00)[84.69.137.98.bl.blocklist.de:server fail]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2019 09:05:19 -0000 On 2019-Aug-18, at 01:27, Mark Millard wrote: > On 2019-Aug-18, at 01:03, Niclas Zeising = wrote: >=20 >> On 2019-08-18 09:48, Mark Millard wrote: >>> On 2019-Aug-18, at 00:34, Niclas Zeising = wrote: >>>> On 2019-08-18 09:07, Mark Millard via freebsd-ports wrote: >>>>> I ran two separate devel/poudriere-devel amd64->aarch64 >>>>> cross builds on the same system (head -r 351178 based) >>>>> with the same /usr/ports/ tree (ports head -r509171), >>>>> building the same 97 ports each, mostly overlapping >>>>> in time, and one got: >>>>> gmake[2]: *** [Makefile:37: gen/apple_png.h] Error 2 >>>>> while building x11/xscreensaver but the other >>>>> completed building x11/xscreensaver just fine. >>>>> There was also a armv7-targeting poudriere-devel ports >>>>> update running, also mostly overlapping in time. With >>>>> 28 FreeBSD CPUs (under Hyper-V), the load average was >>>>> frequently over 90. (The armv7 x11/xscreensaver build >>>>> also completed fine.) >>>>> All 3 poudriere's had ALLOW_MAKE_JOBS=3Dyes and all 3 set >>>>> up to have "Building 97 packages using 28 builders". >>>>> devel/llvm90 was one of the ports being built. All 3 >>>>> poudriere's had native-xtools in use. The context has >>>>> ECC DRAM, 96 GiBytes worth. >>>>=20 >>>> I'm just about to update x11/xscreensaver, but through multiple = poudriere runs I haven't noticed any issues. I have a much smaller = machine to do builds on though. >>>> It would be interesting to see more context of the error as well. >>> Here you go: >>=20 >> Thank you >>=20 >>> The overall build is still in progress. So I've not had >>> a chance to run poudriere again to have it retry just >>> the 3 ports (1 failed and 2 skipped). (Presumes the rest >>> build okay.) >>> A race suggests that MAKE_JOBS_UNSAFE=3Dyes may well be appropriate. >>=20 >> I'll look into it. However, this is the only report I have of this = issue, so it's not very common, and setting MAKE_JOBS_UNSAFE would slow = down the build quite a lot. >>=20 >> The error is a little strange, since it looks like it is when = generating apple_png.h. I wonder which file it is that it complains = about not being there... >=20 > =46rom what I can tell: >=20 > ../../utils/bin2c apple.png gen/apple_png.h >=20 > is supposed to produce gen/apple_png.h from the binary > file apple.png . Presuming that is true, the: >=20 > gmake[2]: *** [Makefile:37: gen/apple_png.h] Error 2 >=20 > indicates that the Makefile tried to reference > gen/apple_png.h before ../../utils/bin2c had gotten > around to creating gen/apple_png.h (the race). I looked up with "the Makefile" would be for apple_png.h and it was: = /wrkdirs/usr/ports/x11/xscreensaver/work/xscreensaver-5.42/hacks/Makefile via its lines: bsod.o: images/gen/apple_png.h and: bsod: bsod.o $(HACK_OBJS) $(GRAB) $(APPLE2) $(PNG) $(CC_HACK) -o $@ $@.o $(HACK_OBJS) $(GRAB) $(APPLE2) $(PNG) = $(PNG_LIBS) $(THRL) and: # Make sure the images have been packaged. This is the first one hit: images/gen/som_png.h: cd $(srcdir)/images && $(MAKE) images/gen/6x10font_png.h: cd $(srcdir)/images && $(MAKE) The presumption that images/gen/som_png.h and images/gen/6x10font_png.h will have their actions (including sub-makes) completed before other things such as "bsod.o: images/gen/apple_png.h" are attempted ( without MAKE_JOBS_UNSAFE=3Dyes ) is false. This is the source of the race that I find in looking around. This kind of structure needs MAKE_JOBS_UNSAFE=3Dyes for reliability. Otherwise a different structure is needed for reliability. > Similar points go for gen/atm_png.h gen/atari_png.h and > gen/earth_png.h . >=20 >=20 > The original 3 builds completed so I started poudriere again > for the failing one, without the machine being otherwise busy. > The result was: >=20 > [00:00:36] [01] [00:00:00] Building x11/xscreensaver | = xscreensaver-5.42 > [00:04:25] [01] [00:03:49] Finished x11/xscreensaver | = xscreensaver-5.42: Success >=20 > This too suggests a race condition. >=20 > If the FreeBSD port-build servers report this sort of failure > at some point, you may then want to change things to avoid > the failures on those servers so that dependent ports are not > skipped. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)