From nobody Sun Apr 6 23:43:45 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZW87N408bz5sDHF for ; Sun, 06 Apr 2025 23:44:00 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-25.consmr.mail.gq1.yahoo.com (sonic304-25.consmr.mail.gq1.yahoo.com [98.137.68.206]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZW87L4KKhz3WTm for ; Sun, 06 Apr 2025 23:43:58 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=SZohvuLS; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.206 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1743983036; bh=7mmTPt0nWRZxUldiR6tW53WPOv4jxoDuXWXL/f1MESU=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=SZohvuLS/07dGBd7V5zRGRHDAs6IZMeLC5yNYgGJ12YtkXj0F7MIQCp66jYZ9C9t8R5PWM9ksOu4bG21Q/u7UWx8eSYye5ifL6gB3YLdzXyNW9f8pdzyZfd8BbY0WD/mVjWuYBobxIQEVpz8vkompbh+hUfHCOt3lycbU7KwIh+FhmhmyRFaC64C2QcNNPOXRgojGgzb+OHhzwpu58ip7UtVlX5aKiybrIHinmoalIg+U3Lt8lXF1P4UCI1cV2iKIb7Y9ALzUGLTCQaL69V+YDAzBALgzFh9BpSRRZlzLL9z1sJbpgskCVUqmw/JeNxvBFkkOIBEn+NwyjopwShwpA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1743983036; bh=sCUCUDYb21D94Bk9BT00Vfle7II7N4Xne3fjtyMqRab=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=e9sHHTMMBBhRosbezRKj9kmg6TU+B2xxbjNsL4j3bkrmyVaEhhIra2vTygIxAP784gW1lt4PItW4jIkaM1qE2Es02KOpg2C1+r0c+vP0/qmcjSHfax9Xb2Ta7JtiDww0Q4ecLDAqXz3OB+ZE/S10ZCfWz4M3izeRf8IUe4405kj/qfOJPaQu+hvp7iKB3VfvQ2w5vNRD06EtYS8uRLBqkG9jdC95siJKPgQ55LOARhHYis4TVP2oM3RBXOdEOurP4Fl6fo6jAE0VZtJqR6enfQZQdJPuIJxdC+laJrfckOvdb+T5yx5imOCu4NBqYxN7GQqsCWyKVBqU73V7OnSc4g== X-YMail-OSG: ppYbDf0VM1mRduO26HP756eARv8O_scoAUr0GqrzTtqY2kNK1RNR7LYnmArdop6 gVk8pDWk3K7DWs98F0EcNZJqQR77TkjU6HwTIPgP.t0LUCMcgB.tFdCorKYDLX303UNGntkVpsjI iyCs_L27h0YDvck5lPbYgTOPxjUTFHoJw09WMnKkWQQnK4RrH_wBgasJKAo6a2Auuka9Cfz2_t_g LCwJH4fEuME3PQWBN8QQSG497JI4Mx3SYwVbQPtLUbql9S3C7rf7TehZpFUj0lYkTdt6ni3Co2_a 8voEo49NKOBhU2AIE58f35Ldk0iNrZSL9fS9Vy1SZMvfJmPW1i8DFwviBOXjYtLdHEfDid8pktDn k3LNMetn21cIwvUadABR4AV9p67bd60W1OTVbLs60LLLVWgzvqxBlwCqdkzzXoVI7nu5oofb__lY GNjuGKjvSXgvZVhWrq0TBtPnsGb31piUtdLw3T_k2UYbMHz2PCa9CNHh.bkGYIRiDmtgMApFN1LA 3JVfwFlGB8clU5kRGc4ySO9rE41ofl1fgK11bNhOuqZPFnccGshTW3v1AZLfAzQLJ2pLBVt6ywpd AZ6QWADcNR0uNPpMAa03lWu5Tux2ccpVklYd9Zes8jKke7HkFx3cEAtVjF2pfavqtUR0OPfLTYkL dKSbQhBGs9Pe4XMJ.8gFA4mtitAqrr_pu7tJ_EUzE_MCDTFRphNdx0y6pMQKtI4G_X8eZmjl5Iuj QHC.1kcR70_XCfq7iV.SAfDwjMPIrZVQrknfAHmwlMMUb0910LxGawew2Ie0CQ0pWP069wBIcvmA rKM3I2P2vbOO.23QMSzzz6DvMAX_vIdlotxR.O8QpNZjrRS.i.Bed0g9.CNzyZMc3j_OoVLfgqdZ lo4JlYAJJrfz2zsnLOjic_IrIXxHkiV_DMxh0LFW6FcGaNa2IKrtUDw6LdTJverlvKM0qQuhF.mA AV4xkIEo7Y_TrFaVUTNozmCwMa9uVRmywFkalliRR5SVbXXFvuQzqdf.kJWgFNeLgehb.kPaznVE .lE.uTONnNbuTZiq_.2.uyL8rPr08ItzwEkrCXi7hQylZgqWH7mnXGD.wHWILvQIbOs6dondsMuV UxZUxnOtDL9YKcoWi783Sa463aKTk.opdNxheemCE6kggRuUxqVzpcbBE5xxCZNkoSrVYfP2D7aX wpLo66ZdenTy3yLheuZFuYCHv9eTjeGF_VADIDjjzoGR3PolMIrk.1WfJcqdkOqxh63Nn1yzf2Qa q9PokF3CVaEP.S0GDDXH0IXQauDtJ3iKeXBi3d0woCm25ESEbt0MSyrBzFtJaftPJ0htf23v8ciE zHOt1RBb0k3U5WnpxHk7QHQddTZZw0el9_Q38y_bPM5OlAJ6bYuBYBurT1ZSDiO9BiMTSAy5zMxC qG_IF6oY_b_.QR_QtwCYRn0kxYq7LEPyJPy9tDGjMF0Ce0f28gd_or3JHKr4M_4rtUjUSn9BGARW D6WH8h6wapaEs7bYTzG5PIYFK3j9WG5x8fIdHkj2OG6QcEUEzIt.ENMNplOx_F1vTGsEXICOvz2a BQEZq6lepqGaxX2qMrah_ywoEx_hsglDeeB778Big_9qO0JBwK8UiNpNnwUvaxrDzvGtBKU2kNkq GXPhm0nrZ.bj1ygw_nCOYHlvofqorwGxmbV7RaiZsibvXmybBq6Zj3tHZEXseIdoTG9jQLT4rRB8 9.ficBGUF6.f1JtFBUrsL21gekLiyRLOBkDMorzoid9Z0a.7tGnn.EhKN.J5cNTWFl5i4x0kohAN lGntriT6FoxxLDwOqKXM0HLLmFOXX0XDg9s_uMeeFZ5cLfTER5sY2MRE0n8LWc4BdxNNq31_ACA2 mgEE.WtUiFU3Ov3oIgTDVQE3V7wpwy1BUmMaLTSYBoUF6BMnK394T8DOTm.OVPM1XQkfoC49RrJF SiYAZHj7wcr6RGO4XSkNdnyRyHZKztv_zq0eem.wkQvHe0W8N3lfZnXBXMNHZMZdjtio7y_6E7fL .GpJhQJn6q4BP.c1q9UuiiivhhzJctQm546ljjt01ItzOeeQNfMgSAZ8eyHrdWbkxWYVrVIvvCtd .k91TBvBughbcE6i8dPYXaUGwKZpHngW.CRkB0OliHTWuNiMMeeRzAVsd4E7j2DYfX0lXkJJJf9p 7d.Mw8HAZS9IsFCUl2h.ZONG.Dkd7BITRLPktX9RIMm.uJLqesxdH4b9c.UEofij1gKDpMSLjZFa Sk5KNAx_9_2b8JbjbivURQIeFccN5w1r90873aOJJvIonkPp4X_DhFhYft0yv8KrvLNTEpJqQPoy 1eMneM6FJRg-- X-Sonic-MF: X-Sonic-ID: 020607ef-ff6b-4c49-a9ba-503b3ad70260 Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Sun, 6 Apr 2025 23:43:56 +0000 Received: by hermes--production-gq1-5c477bf655-fdl68 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID a9fd625be398ac396a972a1c91a4dd14; Sun, 06 Apr 2025 23:43:56 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.500.181.1.5\)) Subject: Re: UPDATE: pkg 2.1.0 looks to be making official bulk builds of packages take much longer [it is more complicated then that for aarch64] From: Mark Millard In-Reply-To: Date: Sun, 6 Apr 2025 16:43:45 -0700 Cc: FreeBSD Current , FreeBSD Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: <366E27FD-FA5B-4BF8-B6C4-6C495DB289C5@yahoo.com> References: <8E2FBAD3-EF6F-4D99-A340-21F8FD19AE0F@yahoo.com> <84FBBAF8-025E-4B9D-9797-51735567A8DB@yahoo.com> To: Baptiste Daroussin , Konstantin Belousov , Warner Losh X-Mailer: Apple Mail (2.3826.500.181.1.5) X-Spamd-Result: default: False [-4.49 / 15.00]; RBL_SENDERSCORE_REPUT_9(-1.00)[98.137.68.206:from]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-0.997]; NEURAL_HAM_SHORT(-0.99)[-0.993]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.206:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.206:from]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_FIVE(0.00)[5] X-Rspamd-Queue-Id: 4ZW87L4KKhz3WTm X-Spamd-Bar: ---- [For the most part the prior history of my notes is not important so they are mostly omitted this time.] It looks like my notes about official bulk package builds taking longer may be from observations of more than one distinct issue, one leading to a very rough factor of 2 and the other not leading to anything like such. I'd reported that main-arm64-default was taking very roughly 2x as long to do its official bulk -a -c (from scratch) style build. This is what got me to classify the time increase as rather significant. It was the first thing that I'd noticed that started my looking around and biased my interpretation in later looking. The recent information for main-arm64-default was: > Just for reference for official main-arm64-default bulk -c -a (from > scratch) builds: >=20 >=20 > p25bf3a3260c7_s680d34896c3 queued 36447 > and has built 15523 and has 19479 remaining, 134:23:16 so far > (will have built up to 15523+19479 =3D=3D 35002 when done, if it = finishes) >=20 > So: 12 to 13 days (around 300 hrs) as an estimate. >=20 >=20 > The prior longest main-arm64-default official build that completed: > p02dd5021d6f9_s717adecbbb5 queued 36466 > and had built 34853 and had 0 remaining, 163:20:35 >=20 > So: 6.8 days or so. >=20 > Overall: very roughly doubling the overall time when the "for > real" context does not apply. It turns out that the above may well be essentially independent of the pkg 2.1.0 related time changes. I did a local test of building my usual aarch64 ports from scratch prior to updating the ports tree to have pkg 2.1.0 instead on 2.0.6 . I then updated to use an updated ports tree that has pkg 2.1.0 and did a test for that context. I got nothing remotely like a factor of 2: pkg 2.0.6 based /usr/ports/ : [01:04:57] [release-aarch64-default] [2025-04-06_12h23m09s] [committing] = Time: 01:04:46 Queued: 264 Inspected: 0 Ignored: 0 Built: 264 Failed: 0 = Skipped: 0 Fetched: 0 Remaining: 0 vs. pkg 2.1.0 based /usr/ports-alt/ : [01:07:28] [release-aarch64-alt] [2025-04-06_14h16m21s] [committing] = Time: 01:07:27 Queued: 265 Inspected: 0 Ignored: 0 Built: 265 Failed: 0 = Skipped: 0 Fetched: 0 Remaining: 0 So: 2 min 31 sec or so difference for overall somewhat over an hour, i.e., 151 sec or so. That is under 1 sec per package built. At 1 sec or so per package for more like 34853 packages, that would be more like 9.68 hrs or so added overall. (Something I should have noticed and reported earlier.) That may well be more like what is going on for pkg 2.1.0 related extra time. The test was in an apple silicon M4 MAX context under Parallels on macOS. My build configuration is more biased to allowing high load averages than the official builds are as well. Also: Host OSVERSION: 1500034 Jail OSVERSION: 1402000 So: Host was before 1500035 (which may not be significant). Also, it is a NO-DEBUG based personal build of the kernel that had been booted. That build was based on use of -mcpu=3Dcortex-a76 . The booted world was of an official PkgBase install. The poudriere jail was also via an official PkgBase build rather than a personal build: # poudriere jail -l JAILNAME VERSION OSVERSION ARCH METHOD = TIMESTAMP PATH release-aarch64 14.2-RELEASE-p1 aarch64 pkgbase = 2025-03-12 21:11:39 /usr/local/poudriere/jails/release-aarch64 . . . (Note: "poudriere jail -j release-aarch64 -u" does not seem to update the "-p1" part of VERSION.) For aarch64, only main-arm64-default ( p25bf3a3260c7_s680d34896c3 ) has started a from scratch build. The closest aarch64 alternative is the large build: 134arm64-quarterly ( 359bbf7fc5af ) that queued 28018 and has built 12870 and has 13677 remaining: 92:24:15 That suggests 180+ hours overall for less than a from-scratch build. That is more than the prior largest time for a completed 134arm64-quarterly build: 125:58:32 --and by far more than 10 hours. 134arm64-quarterly ( 2cbed7722168 ) that queued 36335 and built 34903 and had 0 remaining: 125:58:32 So there being an aarch64 timing issue is not limited to debug kernel builds, although the factor may be somewhat smaller than 2 for a non-debug kernel and world being used. =3D=3D=3D Mark Millard marklmi at yahoo.com