From owner-freebsd-stable@freebsd.org Tue May 4 04:27:17 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6617E625716 for ; Tue, 4 May 2021 04:27:17 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-22.consmr.mail.gq1.yahoo.com (sonic309-22.consmr.mail.gq1.yahoo.com [98.137.65.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZ6J42tryz4b0T for ; Tue, 4 May 2021 04:27:16 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1620102434; bh=5vn3CYtZvFsTpF5Obu35isw8a6/iG1ZCO6qVLijwEqU=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=paJQRbzYGTyzQCn10x0pQvjOPAT2lRQ85Iflx1Rr25diXcHcKMbpB36KLT4yi103ozz8tIRXbfuSgO7g11DkDogGQpAYh/4wNiUBfZuKZFBPRNXGycqbKo2/VulwlkcyzaZZbWAdgSwuFMK48EgWtcl1t3ra+u0NmjV5f1EQp9Qts44BvxxLfQOSMLubzDi2AmND5Xq5G56HowMzpTp2P1rhsHUFotI23RpiI4O7oU01bNGO2E4QPlme2XF/2+l2Itni7lZ/WljxrqaCk7T3L+DNfuzHN8K2wDw2rw7JRj54IoFSzL4mzJyAHTthPwCHNEWoTzGU9s0F72POxs9H0A== X-YMail-OSG: 4CVIMHkVM1m1Ebnvf8orEQVhXRBTik6nAZl6w9qKXUypiBEuDTXRwDOqnlpxlzB F88wS67sqkcBwP8LDjcoRvy1syyg0A821jOuzQ9zBf1115BNbJSNj2VV5GiOKPX8oHf7_uc4MB.f gWSVgF.BYNW55k4GQTJeuveFX7pIkoUTfVBFsIALf4qZch2EhHF62ZslTsLUXTM8.OKEvFWSqauX tEvKso1W6nf3iiKWPgJ5chqmc2AJN1OX.gULbwNcJjrthCI7pH7VWW8UH_BiwFCFpgluhHiQTQGl jI2OylLV26hh1SRjG09Se49IJdAMtJbHKif3Nm_DvTBxYoMDXGTvX_PECyw6PGzlxRAgtCjA.uLi e2.XhAvzSbi4JnBn1RrFf3_XtTVdMZ9aaVqdOQtRaeXQEcO2FtLdEBnoQ3syWH3wqBo9W9wNT5tk 4.MNzQTRtZBrLIs8dr0Mv.zIPNY3AH200EbZQJfk2bHTaKdKWqKfFo.wnnTuEzGiO_C8m2FxWu60 eT_DPszAPmy1.n_ZFCAnVW_elasMYuZh.bapmioONjWKaPvqJEPKH2r_xJSBnVQ6pOdbriAILCCE scuuTjw_SCz7SVaG64kfqXe245g.Eg1YPme0pw0OUG4n5jX4nFcDH8GKt1p.kPjRaeJyDVx.9x.F KVphon6hVk._lyk2hXW5jo3KHBcD08kCT3fQKyBEH3xQpvvPNdq5c40GNmJzuYZ8E5UBZWBQljK5 MnCctirF7PKq7MadurQOQUL5FuVgLv5iZMpkonlN.oKT_Doux0umADJDwSaVu321vp.W1dcIZYMC AOaKo89BfExl0I2SKNZdzw7yakX4RdJCj.sThfgru0Zu6QCZRAvLbyAq4SVb4tlPWa5gI35OfpER PobYHUxt4T23YMs4p6Bpkv6A7pSDjK_1T438nK5QCo86nRL8zm7B0_kiLcfEs_C3ye8kmD6_4Y75 MZxj5x8Dzqmi5QosYM4gHvjdHK2MIdKzkyAuAyu78Zu.zL.6h.QcliQJXl8qQ7YPyiaNsyZskzxO .uoPGENP3CQ64pWuEfASfYGqDgHs6y0N.VH0XCUcTA1f21Cu.GaAnfnfkOlpaj9G8WgnDCXE59Uw 5kjJq8DvErNihFxU.1E7qTiPTfgHOHQoQne4SM09Q9gONIXxeTUeu6ZXgFALgKI4uqSU.FlbcP8K VbO7p.U8wjw39V76Fzdo_y10xf3nP7Kdrhg1vPCNPcHrzjqWrWvxD4z6Gbf1LdeCWxg6bJT.ZQam ovi2ANbn5mPek9aIJMjkae6ysjRJdENdNYamFVVZJBMF1TNVQK9Keh3krpJZBMEcAQfOnRVEqv4V 4mLTBatFVDxWaCMFEQn2f7bD9or_yLmBwaORk7r4_pVvEPPCE9EhMXfDYDgT66yvhcY2R12Cl7Wn ud0huD1_BMuJYC5RGfQhx_J1DIahCxD9QPC4bzhaYj5VnbgR5SlrZ4.spI2WDV96eWHF.mN7orw9 pMf27qlg6pzU7lCvxRKCAMtUNDetEjhIq2UBQHY4fXDC_Nqf14mOb2k5M.P21obVd0BQh7DOW8vv e8V2wCTxe26JYoTmbSvx4KdRNhCOBKPHPi_vGxGkJvM9lYWx7Whj56u0FUombeIJPQ8GnsOdfxNY gqej6lHzOLFk1u9fShw0kSXVar6A80fk3hJgnG6IDFgesG6UMDo5Wkw_Gk.ajnEj3tv_CIvtXEff JpHVx.5LbxdA2Duj_0wWHlFOrax8MwY4aj1m9TCQAKsyshavtm3k_hGoOitaJO2dBV2iLYqhdK2T 0pu.U0HwygFk.zYjhUPA6x5DrpBlB0EJUEbtlEeFtLmLCmhE3aS_CZyw9jFQDDfJUZ4Jh1Wtpppq a7cpM0Id3IyI3ezw.2_mDElcWBkbV.X7buL3DuvwUP7EZTkRJIzKTGSsq72NS4x59yg.X2zloJlx ua8I.s8cO9QmVR2U.Pr1G1XW0s5dpelsC.fZuJPSv2AKvbb2RF.fnv7iqoSF9qqZ8s4i4u4mw_JJ Qb6UBtU1Vuflm0p7AELMU9oYYuy_gD2mU2gKorZrK07aQwm4IN44bgokPEbVQ48jPYDxTmXlGYY4 yW8CGTSTyruGXzOZ_eSrEOU8gdzGLGKZr2MAdVCd_3i72GW2r.mswQii_MFsYRif3BliohJhB0qr UyX1h7vfglft2yslNWM97zvPj.1QnQ5TZJReqnqqPGu1YjVVLFfi7H4to3YydOvxm9VKhOPMelT6 xihUvRH0Cmp2mgS6mBsAedMW1RhlY08e5ZAguBMSSh2nEQ9DdmQQOt0W.QgVdKpPnAnBlao.lrZJ XgdYed5jO0KZi8mMaAPG1aee56k.26_SPHGr0thL3FdrayHx7VUxrYLZfa48t0VhOV.IWjUNpQei HiCPuoN8gWDYIcxtEu..1wA2LsQw0mjTommJxFk9QOwW8D_62uoSLHi__zxV6V3lXQ3Fo16GW2tS gW9Pn5rmAA5FEmf8Gm6d2QUEAQspyLZ1A_dAmSYMWRlE48sgpIbtTKBm6Amsy_MuLqTfFMMMU_Wo JGrCI0kvy8y5vfRRizJ39beWgUeospZ3BFzpkti4uqYgCO5maIw2WAPhPL.c46mDu8WXNK4UaNiv c.LI9W9KgtZiLt8URRsjVV0KMUnBJ11or7Exa9jZMMClBrX_qKrvyY4DTlMefdpC3bCRQVrAZhHO HCNMK2C9hD_EUsjCkN.8Hi0JvBrQybBeFtBrsoK6dQA-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Tue, 4 May 2021 04:27:14 +0000 Received: by kubenode508.mail-prod1.omega.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID ab1cae79be7857d625e2d1658bf2680a; Tue, 04 May 2021 04:27:09 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files? From: Mark Millard In-Reply-To: <91F820A1-8940-4246-A20A-E62685F50079@yahoo.com> Date: Mon, 3 May 2021 21:27:07 -0700 Cc: FreeBSD-STABLE Mailing List , freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: References: <35482701-95A3-48B2-9A8E-B7E0092119B1.ref@yahoo.com> <35482701-95A3-48B2-9A8E-B7E0092119B1@yahoo.com> <43F20589-A7C7-42FF-9020-09CEE037D1CD@yahoo.com> <91F820A1-8940-4246-A20A-E62685F50079@yahoo.com> To: Ed Maste X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FZ6J42tryz4b0T X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.65.148:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; SUBJECT_ENDS_QUESTION(1.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.65.148:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.148:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.148:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 04:27:17 -0000 On 2021-May-3, at 19:26, Mark Millard wrote: > On 2021-May-3, at 10:51, Mark Millard wrote: >=20 >> On 2021-May-3, at 07:47, Ed Maste wrote: >>=20 >>> On Thu, 29 Apr 2021 at 02:50, Mark Millard via freebsd-current >>> wrote: >>>>=20 >>>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping and = /usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping differ >>>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping6 and = /usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping6 differ >>>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/usr/bin/ntpq and = /usr/obj/DESTDIRs/13_0R-CA7-poud/usr/bin/ntpq differ... >>>=20 >>> This is unexpected. Unfortunately I haven't looked at = reproducibility >>> in a while, and my work was all on x86. This could be a regression = or >>> a longstanding issue with arm64. >>>=20 >>> If you install the diffoscope package (py37-diffoscope) and run it = on >>> the two directories / files it should give a more convenient view of >>> the differences. (Or, if you can make a tarball of the differing = files >>> I can take a look.) >>=20 >> I no longer have the same content in those directory >> trees: newer rebuild and the same buildworld used to >> installworld to both places, instead of 2 different >> buildworld's. I'm also unsure how reproducible getting >> differences was. >>=20 >> I can eventually do experiments to test multiple separate >> buildworld's and installworld's, but the machine is busy >> building ports and the llvm builds involved means it >> will be some time before I'd switch activities. And the >> buildworld's involve llvm builds as well and take notable >> time themselves. So my next comparison will not be any >> time soon. >>=20 >> I'll let you know if I manage to generate another example, >> this time being sure to keep the data. If I try multiple >> times without finding any differences, I'll eventually >> decide "enough is enough" and let you know. >=20 > I've still got a long ways to go to do the first > actual comparison of builds. >=20 > But I'll note that I've built and stalled py37-diffoscope > (new to me). A basic quick test showed that it reports: >=20 > W: diffoscope.main: Fuzzy-matching is currently disabled as the "tlsh" = module is unavailable. >=20 > As I'm not familiar with the tool, you might need to send > notes about how you want me to use the tool to get the > output that you would want. (And, so, I get to learn . . .) I've tried another experiment (* in the path matches "28" and "30"): # diffoscope /.zfs/snapshot/2021-04-*-01:40:48-0/bin/sh $<3/>2021-05-03 21:08:48 W: diffoscope.main: Fuzzy-matching is currently = disabled as the "tlsh" module is unavailable. $<3/>Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", line = 745, in main sys.exit(run_diffoscope(parsed_args)) File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", line = 677, in run_diffoscope difference =3D load_diff_from_path(path1) File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", = line 31, in load_diff_from_path return load_diff(codecs.getreader("utf-8")(fp), path) File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", = line 35, in load_diff return JSONReaderV1().load(fp, path) File = "/usr/local/lib/python3.7/site-packages/diffoscope/readers/json.py", = line 33, in load raw =3D json.load(fp) File "/usr/local/lib/python3.7/json/__init__.py", line 293, in load return loads(fp.read(), File "/usr/local/lib/python3.7/codecs.py", line 504, in read newchars, decodedbytes =3D self.decode(data, self.errors) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position 18: = invalid start byte The two older snapshots of a Boot Environment have bin/sh files that compare equal. But every program I tried the above sort of thing against on got the same UnicodeDecodeError result from diffoscope, byte value and position matching. These snapshots have more than an installworld in them and so are messy to compare overall. But the installworld (and installkernel) content show similar differences to what I reported before as far as example files with differences go. But this is aarch64, not armv7. It will still be notable time before I have simple installworld tree's to compare. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)