From nobody Sun Apr 6 23:55:41 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZW8PF66fVz5sFQ1 for ; Sun, 06 Apr 2025 23:56:01 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic315-55.consmr.mail.gq1.yahoo.com (sonic315-55.consmr.mail.gq1.yahoo.com [98.137.65.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZW8PD0vswz3bpy for ; Sun, 06 Apr 2025 23:56:00 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=uGx6coEK; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.31 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1743983753; bh=PvG+ZCDC1hy2qevq5ik9WX3E3jjR0gWGuj5JNjlccH8=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=uGx6coEKa6UjT08oKRUxi/b5AnCSffhu1wRBgdltt+QMfWjerUL1pOY5PM+rnTz6Wx1g6mS6hP/Shhna0vVMjoSaYoiNnOnrOQxc6bISaXrhT0rHFGmHQBUWeRaGl42tus92UxmFRM7zSjrypxt20jXi3EL2IVw5WVumo9xi7NVPG2vfHcquqNoEzZQhU+xDGyuLmw7Es3640w+Brr5TH4ffxAuxPwyAoM9OAq5ExqsbDg9PHobGfsCpZl8eRIiMwkZyt2ya6bOmb5ghzDUO/kGwrJ+FmFTsogUfG8ezAL/N86jWU6Fs96WEYMvkFB+eaEJq66uUBNp64TZ1XkLI0Q== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1743983753; bh=3gAggIK+GW2LUuKgFnUYxGWm5e9ykI+nPmlit0colyE=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=LiZFOoq/kyhOHXy9+vViZbjnY9w3Fgf8kjExWpp1Wuu8DhFQy//vwxfCJFgNewltXPhyp7NF/SyhIYczE6zVIr4p9VMqFvUcgecm5F6t5Xqgd6NJCAQ3qv5rJxqgYnybZh5qqGZQ8scbPemNTnDR58va67tzJ4aUOESsxlPx5zd2brSzjOYMLtPIDxDDz3OBhV3uPqceEcJy81SevsvqZiuSkVvE9nmxgLKkBFckiRSbWJ9hBUm2iTjZFtUquZcg3vGO27t4JlfbfdC2svDBowjK0CXX/1+SNd98wZg3Hd0Zg25HRoqpEwO5EF1XHNCJPFSvSaouvPTpm7i/HeUZSw== X-YMail-OSG: 39gFZ_cVM1mzevcsigaUgXLtJVFScGd6SAIzYnvtEmwPJoUlHZimavqwFY2NGTA ZaYgOrTj2OA3gtzsCYNYKX6uxqTAWYPmrUZk9Ahr5D5B3tOLmkkW1dwEPX6T.5EAOwStVzDn5wlE 2pGDHsq_Amw7S3QSB4dQPmNwzfT7TmdTEsKBZrTyCRzing8ARMkhHyLZ4O.iT5o5q.w6dhTUvEUk 4IbaB_prJ_no3Mq_48ihR7Xk05DmBE7MTf444s07KGeZA8x01h86yfw4dCcrUP_sBqoJtH5ondEe QURGNj2UDiVvhUf1N0I_HM5AdkDfj5NAk6oKFicc87QByB1UksfUFLwHbo5rPvTxJ4TKDri5GRCM BDXxsvpVBHP7JmTYgRUmDEKLBOIGy_2QmrNDk_8x6loWP5wJlu3XaUiReK_rKKiKaUz4a9UyhdHI LFGdY8uHebMMwALBMW_KhJdf6pc8pfzaFPWYgExmXVfH.Ox7t9WHnKbwZj0Q4df3SRftIzwPamtr 1jlEnMSaKFnF4LseuLT1qAL89OsZbH1kwr5HhHhCiiftbOTvrsKgF.t3krE00n0Q4r1Y1oULyuON U_pn.e0M2kFAQn.Ps8ewtGkGywDOlpSnv7ufAP63wuOs.pTp9ghOKDRA.WWAXLI6jFbyC0LeEmJ_ GwiPQ_H_7CJbuLQ9rYXoVmeNoB0lDNl.faY0hMTYg2h1n.2Qkxbd77KDCmXrxv4lSmG8oQqgMX7M vdjawfQWcn3FyMBBgIrmf6n6i3n5ma6XIAkmQanUwXdFnSTHTP3SxvmKWr5q_Q7R054ZcjdrcCdr EXgva4zCs1kc2ub6MNgAYTNqOVGDyE06zJdj7z0pra4JDPngZQ4AcDgtMotKQlSq.KOw5M0cGvbG pQdU9.JpJoYETOWsAbCXroG3OoAZ99m0EZAyyfEHEo6IjiF7Nh8.uirBpH0CK0QoVaQfkUBsZYpU m3PqjXrQxXkJrkDdUGpR5hnbOU4I5Y2TchpD.gMNy75e1OEjFvZ2xyv9lsl.hkDECJWXZEbTR8c4 SRSgv1hK6geEUm65wiXSbU1wWnm47Y6cIX_KeFXgIk54.ePgsjcJeyLv6wEMY5_uw7JrvRXUDe9D QKC5trwfL4VNJJKPAGsjYi9rIUTQvxkARm8PzwWELv9RQYRY0QNYAd_GlslZ52_LtoNURrr.JfUm kGEeRDuU9zFt6xFaErARnG2caQqg3ePCOuFpV.AXbhJVqaG41WXc90LJ3WckvykR.YmIfWqkMOO0 raQm6aMvd4FZBayOGvsPRCBXwW_nT9MJOrMT7EtdzHUCLCldkk6n8VSPYOhUu9e6w_TijgHWma.H SU3GyEE5_o2uOUatv_08jw7bqMquKF2ci5LSJ89xDNHZ1sm4v2QMgvR4hPP9rd7oH9Ff1kPTNqHm C9GczuSyqvzNr0UogT6x7F4jpeNyhTkTtSoogAU.bHCl_ZMjNcb3IJpHjZ02lGzCOXr8aXbJe4ws XGfm7lIMtwiiUw5R4xm_qVYDseiXvGjtTV4oJJkMYojo_mc2QAxXS5FRU6IG0muyicopLiBKV_yJ C6Tw4XF9LXeWSv9nMiJpsH0F2SvEsKhWE5tYRdQvD1Hwjhb1LCSrPsAlAyWmQEOt1nfjtCkq2huK _5UHlzKXFQTj1GIipeWwMzVyAG.zgkxRF5qW5QFe.EqTGoXDI3X0K8kWpDnc5wejwKHPrsL62vyY VHtUiTgXxu9EBsP0y6x.XuudZqa6sbBNTMTS4IIfCBf65Zgoyl2pBR.oSZwignj5rJLc9_zDaurn VrgLMQXdMVRLYTGA_MMIoJxoK_m5dowHXG_31Q9QXJBJlJftKXtrNx7oXX0fJ.QVhLVcvV_Sp4oQ 8ZVazxsIO9h_4zobOh26aoIu64ESoBfiAN9s.Z8OSh8_uXgVB9XBYgqjxQ.mdJMH3PLkfIA6Cb0r ojWHUkGJSLnQSg.nA64yNdUKWW5SwWYSerWhwt5b0asST6dgJDRxisml6idLAY.s2T0_uQgNFlZ7 fVgqTLptH6Cd.v_AQRT4Z8WX7PBrBNsRqXnlfqZMOXDG1uVd1_KwT1.AqvpoBQZB4sWpqiSORYeO IfRe2Ytv.GAoHx_.CJYID5EdtLwHnu1jEbgOeY7PItR3Vy7TSHStxukcQnC3LMYP4w50DiFh.WDK Gsmi49gExW_X7lOIL0vCSGoWM.b36qqRo3HFdShLzbhh0D1fGnSoyJp1aJ4n2sSBZZCXhNLu19uD he1RTiOKG_RCXqllZSzO2uW5RfFY7HTIWfQ3mjSVsIwslHK7O7jk6zAT7rsG4DIB3zwc_RrpBjB_ F6Yg0vqkxD8UjwQ-- X-Sonic-MF: X-Sonic-ID: ea7718d5-335a-4add-8549-f05ef3358b01 Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.gq1.yahoo.com with HTTP; Sun, 6 Apr 2025 23:55:53 +0000 Received: by hermes--production-gq1-5c477bf655-7nzz9 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 0d39677a63a2aa28c23f3dd3e028958d; Sun, 06 Apr 2025 23:55:51 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.500.181.1.5\)) Subject: Re: UPDATE: pkg 2.1.0 looks to be making official bulk builds of packages take much longer [it is more complicated then that for aarch64] From: Mark Millard In-Reply-To: <366E27FD-FA5B-4BF8-B6C4-6C495DB289C5@yahoo.com> Date: Sun, 6 Apr 2025 16:55:41 -0700 Cc: FreeBSD Current , FreeBSD Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: <5B20A794-DA42-4F46-A80C-9A262C0382B5@yahoo.com> References: <8E2FBAD3-EF6F-4D99-A340-21F8FD19AE0F@yahoo.com> <84FBBAF8-025E-4B9D-9797-51735567A8DB@yahoo.com> <366E27FD-FA5B-4BF8-B6C4-6C495DB289C5@yahoo.com> To: Baptiste Daroussin , Konstantin Belousov , Warner Losh X-Mailer: Apple Mail (2.3826.500.181.1.5) X-Spamd-Result: default: False [-4.49 / 15.00]; RBL_SENDERSCORE_REPUT_9(-1.00)[98.137.65.31:from]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-0.995]; NEURAL_HAM_SHORT(-0.99)[-0.993]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.31:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.31:from]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_FIVE(0.00)[5] X-Rspamd-Queue-Id: 4ZW8PD0vswz3bpy X-Spamd-Bar: ---- [Correcting a dumb mistaken reference.] On Apr 6, 2025, at 16:43, Mark Millard wrote: > [For the most part the prior history of my notes is not > important so they are mostly omitted this time.] >=20 > It looks like my notes about official bulk package builds > taking longer may be from observations of more than one > distinct issue, one leading to a very rough factor of 2 > and the other not leading to anything like such. >=20 > I'd reported that main-arm64-default was taking very roughly > 2x as long to do its official bulk -a -c (from scratch) > style build. This is what got me to classify the time > increase as rather significant. It was the first thing that > I'd noticed that started my looking around and biased my > interpretation in later looking. >=20 > The recent information for main-arm64-default was: >=20 >> Just for reference for official main-arm64-default bulk -c -a (from >> scratch) builds: >>=20 >>=20 >> p25bf3a3260c7_s680d34896c3 queued 36447 >> and has built 15523 and has 19479 remaining, 134:23:16 so far >> (will have built up to 15523+19479 =3D=3D 35002 when done, if it = finishes) >>=20 >> So: 12 to 13 days (around 300 hrs) as an estimate. >>=20 >>=20 >> The prior longest main-arm64-default official build that completed: >> p02dd5021d6f9_s717adecbbb5 queued 36466 >> and had built 34853 and had 0 remaining, 163:20:35 >>=20 >> So: 6.8 days or so. >>=20 >> Overall: very roughly doubling the overall time when the "for >> real" context does not apply. >=20 > It turns out that the above may well be essentially independent > of the pkg 2.1.0 related time changes. I did a local test of > building my usual aarch64 ports from scratch prior to updating > the ports tree to have pkg 2.1.0 instead on 2.0.6 . I then > updated to use an updated ports tree that has pkg 2.1.0 and > did a test for that context. >=20 > I got nothing remotely like a factor of 2: >=20 > pkg 2.0.6 based /usr/ports/ : > [01:04:57] [release-aarch64-default] [2025-04-06_12h23m09s] = [committing] Time: 01:04:46 > Queued: 264 Inspected: 0 Ignored: 0 Built: 264 Failed: 0 = Skipped: 0 Fetched: 0 Remaining: 0 >=20 > vs. >=20 > pkg 2.1.0 based /usr/ports-alt/ : > [01:07:28] [release-aarch64-alt] [2025-04-06_14h16m21s] [committing] = Time: 01:07:27 > Queued: 265 Inspected: 0 Ignored: 0 Built: 265 Failed: 0 = Skipped: 0 Fetched: 0 Remaining: 0 >=20 >=20 > So: 2 min 31 sec or so difference for overall somewhat over an > hour, i.e., 151 sec or so. That is under 1 sec per package > built. >=20 > At 1 sec or so per package for more like 34853 packages, > that would be more like 9.68 hrs or so added overall. > (Something I should have noticed and reported earlier.) > That may well be more like what is going on for pkg 2.1.0 > related extra time. >=20 > The test was in an apple silicon M4 MAX context under Parallels > on macOS. My build configuration is more biased to allowing > high load averages than the official builds are as well. Also: >=20 > Host OSVERSION: 1500034 > Jail OSVERSION: 1402000 >=20 > So: Host was before 1500035 (which may not be significant). > Also, it is a NO-DEBUG based personal build of the kernel that > had been booted. That build was based on use of > -mcpu=3Dcortex-a76 . The booted world was of an official PkgBase > install. The poudriere jail was also via an official PkgBase > build rather than a personal build: >=20 > # poudriere jail -l > JAILNAME VERSION OSVERSION ARCH METHOD = TIMESTAMP PATH > release-aarch64 14.2-RELEASE-p1 aarch64 pkgbase = 2025-03-12 21:11:39 /usr/local/poudriere/jails/release-aarch64 > . . . >=20 > (Note: "poudriere jail -j release-aarch64 -u" does not seem > to update the "-p1" part of VERSION.) >=20 >=20 > For aarch64, only main-arm64-default ( p25bf3a3260c7_s680d34896c3 ) > has started a from scratch build. The closest aarch64 alternative > is the large build: >=20 > 134arm64-quarterly ( 359bbf7fc5af ) that queued 28018 > and has built 12870 and has 13677 remaining: 92:24:15 >=20 > That suggests 180+ hours overall for less than a > from-scratch build. That is more than the prior > largest time for a completed 134arm64-quarterly > build: 125:58:32 --and by far more than 10 hours. >=20 > 134arm64-quarterly ( 2cbed7722168 ) that queued 36335 > and built 34903 and had 0 remaining: 125:58:32 >=20 > So there being an aarch64 timing issue is not limited to > debug kernel builds, Actually, 134arm64-quarterly also has Host 1500035 : Host OSVERSION: 1500035 Jail OSVERSION: 1304000 and I've no clue if the 1500035 is a debug version vs. not. > although the factor may be somewhat > smaller than 2 for a non-debug kernel and world being > used. =3D=3D=3D Mark Millard marklmi at yahoo.com