From nobody Mon Apr 14 23:43:57 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Zc3m05JqPz5sVBC for ; Mon, 14 Apr 2025 23:44:16 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-25.consmr.mail.gq1.yahoo.com (sonic303-25.consmr.mail.gq1.yahoo.com [98.137.64.206]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Zc3m00t6Pz3F4l for ; Mon, 14 Apr 2025 23:44:16 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=QgYsK0bR; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.206 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1744674250; bh=7mDQxwoAEi90LNsR5iuzQy/cIjqo5n6GgbT2QTIPNBE=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=QgYsK0bRaymtKeIo9V+E5pe9rvXC2oR7dQ9ogVY5bmsWsQ5DhZvL2Rs+NPeRaBmsCH6ccbPy+8jNYx3hh8NAPaRzjUTkgr28z4buisL7mpHApdMAM8i6+cSXW2MKzvj7Wy9o3mmGhk21FtDtnjohNLHH5lqGDRA7NdoCUtljSK/xNO8UBKuzcR8kqUabxS4dDKJxeXa/Si5FxNOGz2fZw53XQThVBTphXro44iIczOJm93tzeXcTi338HiXbQW2HR+vELPFigfoY+s+3LtwQ7lUogURBlSlaH0j/2Fq/r6cGYa7Ur+QH/6WpEjCDPh3wqMYMmJw+o2H9hdDwMJvYrA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1744674250; bh=e1ckDVOayS6HYXgGDruQn8/1P3QgZRucItavwgR6BM5=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=OE0PWgEKReQsjiWE4237uB2srh8X+BUrVqn285gE3lGOa14vcp5OrJlX+LLuul9LDIbC4dwEfG13XvlrQD+d9Ds2JbTlDcnfyfk+mrWCYMDdC5PE7wUwbIiOAjgu6DNNx77frMpBG/fhxgW0nRx1HWccagna4aYCZ7FoIuGOtUiNfFVPRQ+DgWg3qPfi69sVwqqM69KyXUrrgvLoxLL4hjiKEN8pGHQmKYBYjuIkSH07AbgyU4wwA3OC5+sRm6mmSuBYwIFDncG1K+Je2NcByUbAqFI2SuCHoYrf9zmc7N+FYEKQPyxHYzx2P07DBFwiD7StRIRUYTOiCqgRvWm9Bw== X-YMail-OSG: xY0jMogVM1nzyQGQV8S0gdxHK8jogRMoWi4Zaakt1NPbmDi3pZ70fRilbLcPg.G tRl0tqiLudSht8HntZ5N.wV4G2hqfDXH1sbfmo91GCIECXV_16JvGpddwdhrxY70aDqid8JztHuu azvDEJ2Tt.GTg3ZirNVYC7UB.RH8gZdTxwMT3LuM_EbMPIyykP402pbSY7KZG7qYhoaxrbbTklvr z51LLmsG746xFms4.SK7NRbXWMoXjP5d7z7dyUXQXxdf1W48bVOZiIDIohqB1zZmKz0A4kM6pHM7 yF7c86syjMWafXEcFX2i4IK0IY4lIxnPrPiuxaQnkqS7vhBpmpTzggzynUQMFZR7KbGxShE7Mf5n jhQPBWL_xFJirWIJY5aYOl_hOJBcM_Ti2ZSn_9.O8wxL_b6LCXzS.ECnuUBQAf92Nro2oFWxmyQb uwF_MkYAGrygb2akvUrqzH0RHN1.ba3t0wKEChl34BWK2nwpwI8_iuWgzuKh2R2SQR1HLaeQPD_q lqkoIEijCZczhANR11BOLi8Lkelf3PqpD01OHbV0aPxBo.kiHWT9kYo7oYvVFc6fEu3U01Bt7ZI1 p4CsVKRabnquf.PvGspjueUfaPjbAdgQ4cDwOMVDZpkx.X8fDVUEDRs9n_E.5qJ2..Lfrjn0idPD jmpZwlfsiRQpqJ7WuDrVW_QK695U9Udf.zA2uKw1A93t65ovbzBJ5VFli8Fif_f6.FzKKeq.Kbk1 _SNByBVxShH8rw9DwQQeEyTsyEUfTO1gJ6YSfB5X9j_T00zo0hPb7huDmb.vZCv1.qb.AXvKKSpH c1ddyQXIaya3RvHRn32cz1hLdLum8nmbhMGMqo6YM.g9ywJG1P.rbfnGqcH636k7_DaDcGp4wVjp uu5eL4XichjJmyTKIAfBWyuUwkCNnDM6ZrqghLM9B4a4XCa36PYDuPu_vprUvPtmDYY_6ROAXXus lfXPQtEycs2kz0m47naSg9MAIfbtjkLiMuHS4sQTeE0wIRMSuQF.0Jo4WV_gDU2PEHXS_TtIWt7a kOuAtDiCqlUOak_e_aaeDe0pvUnoD2A3cc4y7a26f7HJoeA8Kp5dtU5z9ZCsLvZ2W9enbOho4Rvg w3M87SzWXqvVqiUdrHtLQf4elL_z.kp1W_nOMpk92Q81S2.qvhHM_Lj7RdRgYrLFF69oUSZ8wUZG 3021ntbu54i5bkXhkpbRroa9bcFPMjBaWw6YzUQQku3vW249ZJlzUoJxVoIGaDXdCD7DTF4iU04P 0RhrxjleXsiNkyOsZ9OwRD6gp6Mr1I_SsMI9yFsYwGZdD9dRIsnCaz0PUD4rbIeces_SO_pCMgia An4X0UMEhuMH_4Nj67RwMa1Df0prBsh2N21tXkAQTdNRB4wHiWYOE_ehTc14smneIttvIhy8ud7C E2Tuazl_TcBMFQfpCnf8HTelbsqRPvsGkS2ZVyRoM40mFj.YlPkNblLrb_tmYJ__6UeJzBVawje9 Eo8X48z0_lab4h3bEAS0Om2LMaPnjHvQiZphTM9b2ym9ZCtaftNfbJvoZMw7sevP9RJY8V4.qsXb Edwdj5m.YxMYYlHESFfgih5oPsPC4YPFiF7CCf8wtatAa.MImTnQKbGGhsk6srOvgnuB1bTX0_dn jOGZW0PF5iPM09J8PkKGL8ZqqmHBSRSy5OHlcjp7TOIqDAbLO3RdMUyG9Hl0CAWIaTGwx422rgki e4GHaPkMA_uVTCWRmUzQpqldc.YMLaL.MsSUFiS6w6r.iLOP5RD7N.3DVTH_DBrajryAjmX8HmsZ 4X6qBLMVsBY3Hq2tCem865u6.qRNLLO5l0SlyZ6VVUrqHTuZH9wnrkIGXeW_cSFNGU_wXqZ2SIMa JwdO2jrzVmfcW_gr1S0kAhVkwPBCG8cpVq_4._sT7JKhNKC_7.klzid9pAQIlLgeYpa0C5UkF2JB c8.qKdLH7hrlBseYMW.3GGYnXvYy2eAes9k5ia3UdOTHs8I3FK4z3Kz8sVjNl.yUDs.zvB_mVmuT uzp0oQgkJtvMl_MHVYFr44ZjpUfsTljZs2OR4EnAwubD3R77RwH6HczzWSt5qbOxB.wrcauXEioo 1SSWG.NGkYK.8WxJLEiT7FNaCWVcav3Lg6mESW1UVt9e3ohnUCofhUyUSUX4c5GwOLf3iYElF7Om JoUcts0DTfuVeA3bcrk3_sFQgNTvcxunJrRZjDvdRlPNmWsNK1kH6XbiU8cs6EIuQu1UIgYRVqcR QCJeeLUUqWxBv.vYXuA35B9DDHzrJXEgDcp6Rkgb7HzJa2xmZcOeVCFjeFNceiUTroE0wgQhTID_ vqOw- X-Sonic-MF: X-Sonic-ID: 02f733f9-498d-43fc-b8f9-4e9d20a00ced Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Mon, 14 Apr 2025 23:44:10 +0000 Received: by hermes--production-gq1-74d64bb7d7-dp9cd (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ad230736f6e251cf689cf2f3679cb732; Mon, 14 Apr 2025 23:44:08 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.500.181.1.5\)) Subject: Re: UPDATE: pkg 2.1.0 looks to be making official bulk builds of packages take much longer [reproducible examples notes] From: Mark Millard In-Reply-To: <6C67E39F-3634-4AF6-95EC-46159E7391E5@yahoo.com> Date: Mon, 14 Apr 2025 16:43:57 -0700 Cc: Gleb Popov <6yearold@gmail.com>, FreeBSD Current , FreeBSD Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: <8308705D-5155-48B2-ADFC-2BCF32F7D55C@yahoo.com> References: <8E2FBAD3-EF6F-4D99-A340-21F8FD19AE0F@yahoo.com> <84FBBAF8-025E-4B9D-9797-51735567A8DB@yahoo.com> <366E27FD-FA5B-4BF8-B6C4-6C495DB289C5@yahoo.com> <7ziazrj7szuqhov3oppjbh3jyu3f2p2owntv4oxprelrdjzc6u@hkuf5szf3zwy> <8B070D1D-0524-4DA4-A5C2-EF2CF98C5E15@yahoo.com> <7A1322FA-A118-4F87-9D96-DE8B05E09424@yahoo.com> <6C67E39F-3634-4AF6-95EC-46159E7391E5@yahoo.com> To: Baptiste Daroussin X-Mailer: Apple Mail (2.3826.500.181.1.5) X-Spamd-Result: default: False [-0.84 / 15.00]; RBL_SENDERSCORE_REPUT_9(-1.00)[98.137.64.206:from]; NEURAL_SPAM_LONG(0.99)[0.994]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; NEURAL_HAM_MEDIUM(-0.42)[-0.421]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_SHORT(0.09)[0.086]; FREEMAIL_FROM(0.00)[yahoo.com]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; TO_DN_ALL(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.206:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.206:from]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] X-Rspamd-Queue-Id: 4Zc3m00t6Pz3F4l X-Spamd-Bar: / On Apr 11, 2025, at 23:55, Mark Millard wrote: > On Apr 11, 2025, at 19:28, Mark Millard wrote: >>=20 >>> . . . >>>=20 >> . . . >>=20 >=20 > . . . >=20 >>=20 >>=20 >> Back to the originally intended content . . . >>=20 >>=20 >> On Apr 11, 2025, at 14:04, Mark Millard wrote: >>>=20 >>> On Apr 11, 2025, at 11:39, Mark Millard wrote: >>>=20 >>>> On Apr 7, 2025, at 08:14, Baptiste Daroussin = wrote: >>>>=20 >>>>> . . . >>>>> the problem we have is the >>>>> performance changes depending on what is happening in parallel on = the machines. >>>>=20 >>>> In separate list messages I've provided multiple examples >>>> of the time-taking issue that do not depend on what is >>>> running in parallel on the machines, no parallel builds >>>> involved. >>>>=20 >>>> Part of the issue is that there are thousands of examples of >>>> "small build-step time" packages for which the build-depends, >>>> lib-depends, run-depends combination, takes notable time, >>>> given that the total time contribution across those thousands >>>> of package builds is notable overall. >>>>=20 >>>> As stands, mostly it is the early part of "bulk -c -a" avoids >>>> the issue via building packages that have no or few >>>> dependencies. Later "small build-step time" packages tend to >>>> have various dependencies, greatly changing the time scale >>>> for their builds. Few builds are of "large build-step >>>> time" packages (relative to there being 30000+ packages). That=20 >>>> has implications for there being 30000+ packages to build for >>>> "bulk -c -a" or other builds with large numbers of packages >>>> to try to build. >>>>=20 >>>>> which makes the performance issues invisible on local poudriere if = you want to >>>>> test it on port A or port B, >>>>=20 >>>> I've provided counter examples to that that only involve the >>>> one builder, after the prerequisites have already been built >>>> (same or prior bulk run). >>>>=20 >>>>> if we want to reduce the performance penalty we >>>>> need to be able to make a reproducible case which can then be = profiled, to know >>>>> where to optimize if needed. >>>>=20 >>>> I've provided examples of such . . . >>>> (time intervals shown are from the aarch64 >>>> Windows Dev Kit 2023 with just the one >>>> builder active) >>>>=20 >>>> www/rt50 >>>> build-depends: 00:00:27->00:08:46 >>=20 >> More detailed comparison/contrast of non-parallel builds: >>=20 >> A pkg 2.0.6 vintage of ports tree on Windows Dev Kit 2023: >>=20 >> [00:01:11] [01] [00:00:00] Building www/rt50 | rt50-5.0.7 >> [00:01:14] [01] [00:00:03] Status www/rt50 | rt50-5.0.7: = check-sanity >> [00:01:14] [01] [00:00:03] Status www/rt50 | rt50-5.0.7: = pkg-depends >> [00:01:15] [01] [00:00:04] Status www/rt50 | rt50-5.0.7: = fetch-depends >> [00:01:15] [01] [00:00:04] Status www/rt50 | rt50-5.0.7: fetch >> [00:01:15] [01] [00:00:04] Status www/rt50 | rt50-5.0.7: checksum >> [00:01:15] [01] [00:00:04] Status www/rt50 | rt50-5.0.7: = extract-depends >> [00:01:15] [01] [00:00:04] Status www/rt50 | rt50-5.0.7: extract >> [00:01:16] [01] [00:00:05] Status www/rt50 | rt50-5.0.7: = patch-depends >> [00:01:16] [01] [00:00:05] Status www/rt50 | rt50-5.0.7: patch >> [00:01:16] [01] [00:00:05] Status www/rt50 | rt50-5.0.7: = build-depends >> [00:01:24] [01] [00:00:13] Status www/rt50 | rt50-5.0.7: = lib-depends >> [00:01:24] [01] [00:00:13] Status www/rt50 | rt50-5.0.7: = configure >> [00:01:26] [01] [00:00:15] Status www/rt50 | rt50-5.0.7: build >> [00:01:26] [01] [00:00:15] Status www/rt50 | rt50-5.0.7: = run-depends >> [00:01:26] [01] [00:00:15] Status www/rt50 | rt50-5.0.7: stage >> [00:01:29] [01] [00:00:18] Status www/rt50 | rt50-5.0.7: package >> [00:01:50] [01] [00:00:39] Finished www/rt50 | rt50-5.0.7: Success >>=20 >> A pkg 2.1.0 vintage of ports tree on Windows Dev Kit 2023: >>=20 >> [00:03:04] [06] [00:00:00] Building www/rt50 | rt50-5.0.7 >> [00:03:06] [06] [00:00:02] Status www/rt50 | rt50-5.0.7: = check-sanity >> [00:03:06] [06] [00:00:02] Status www/rt50 | rt50-5.0.7: = pkg-depends >> [00:03:07] [06] [00:00:03] Status www/rt50 | rt50-5.0.7: = fetch-depends >> [00:03:07] [06] [00:00:03] Status www/rt50 | rt50-5.0.7: fetch >> [00:03:07] [06] [00:00:03] Status www/rt50 | rt50-5.0.7: checksum >> [00:03:07] [06] [00:00:03] Status www/rt50 | rt50-5.0.7: = extract-depends >> [00:03:07] [06] [00:00:03] Status www/rt50 | rt50-5.0.7: extract >> [00:03:07] [06] [00:00:03] Status www/rt50 | rt50-5.0.7: = patch-depends >> [00:03:08] [06] [00:00:04] Status www/rt50 | rt50-5.0.7: patch >> [00:03:08] [06] [00:00:04] Status www/rt50 | rt50-5.0.7: = build-depends >> [00:16:26] [06] [00:13:22] Status www/rt50 | rt50-5.0.7: = lib-depends >> [00:16:26] [06] [00:13:22] Status www/rt50 | rt50-5.0.7: = configure >> [00:16:27] [06] [00:13:23] Status www/rt50 | rt50-5.0.7: build >> [00:16:27] [06] [00:13:23] Status www/rt50 | rt50-5.0.7: = run-depends >> [00:16:28] [06] [00:13:24] Status www/rt50 | rt50-5.0.7: stage >> [00:16:30] [06] [00:13:26] Status www/rt50 | rt50-5.0.7: package >> [00:17:03] [06] [00:13:59] Finished www/rt50 | rt50-5.0.7: Success >>=20 >> (That I got the 00:13:22 is interesting, given the prior >> 00:08:46. May be the A78C cores were used instead of the >> X1C cores? May be that there were no builds, just Inspecting >> activity for the prerequisites. Did I not match the USE_TMPFS >> settings? I expect that the general structural conclusions >> are not invalidated.) >>=20 >>>> devel/py-inline-snapshot@py311 >>>> build-depends: 00:00:01->00:00:55 >>>> run-depends: 00:00:56->00:01:47 >>=20 >> A pkg 2.0.6 vintage of ports tree on Windows Dev Kit 2023: >>=20 >> [00:00:54] [04] [00:00:00] Building devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1 >> [00:00:54] [04] [00:00:00] Allowing MAKE_JOBS for = devel/py-inline-snapshot@py311 | py311-inline-snapshot-0.18.1 >> [00:00:59] [04] [00:00:05] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: check-sanity >> [00:00:59] [04] [00:00:05] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: pkg-depends >> [00:00:59] [04] [00:00:05] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: fetch-depends >> [00:00:59] [04] [00:00:05] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: fetch >> [00:01:00] [04] [00:00:06] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: checksum >> [00:01:00] [04] [00:00:06] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: extract-depends >> [00:01:00] [04] [00:00:06] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: extract >> [00:01:00] [04] [00:00:06] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: patch-depends >> [00:01:00] [04] [00:00:06] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: patch >> [00:01:00] [04] [00:00:06] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: build-depends >> [00:01:01] [04] [00:00:07] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: lib-depends >> [00:01:01] [04] [00:00:07] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: configure >> [00:01:01] [04] [00:00:07] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: build >> [00:01:02] [04] [00:00:08] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: run-depends >> [00:01:03] [04] [00:00:09] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: stage >> [00:01:03] [04] [00:00:09] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: package >> [00:01:04] [04] [00:00:10] Finished devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.18.1: Success >>=20 >> A pkg 2.1.0 vintage of ports tree on Windows Dev Kit 2023: >>=20 >> [00:02:46] [02] [00:00:00] Building devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8 >> [00:02:46] [02] [00:00:00] Allowing MAKE_JOBS for = devel/py-inline-snapshot@py311 | py311-inline-snapshot-0.20.8 >> [00:02:47] [02] [00:00:01] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: check-sanity >> [00:02:47] [02] [00:00:01] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: pkg-depends >> [00:02:47] [02] [00:00:01] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: fetch-depends >> [00:02:47] [02] [00:00:01] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: fetch >> [00:02:47] [02] [00:00:01] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: checksum >> [00:02:47] [02] [00:00:01] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: extract-depends >> [00:02:47] [02] [00:00:01] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: extract >> [00:02:47] [02] [00:00:01] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: patch-depends >> [00:02:47] [02] [00:00:01] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: patch >> [00:02:48] [02] [00:00:02] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: build-depends >> [00:03:59] [02] [00:01:13] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: lib-depends >> [00:03:59] [02] [00:01:13] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: configure >> [00:03:59] [02] [00:01:13] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: build >> [00:04:00] [02] [00:01:14] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: run-depends >> [00:05:27] [02] [00:02:41] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: stage >> [00:05:28] [02] [00:02:42] Status devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: package >> [00:05:28] [02] [00:02:42] Finished devel/py-inline-snapshot@py311 = | py311-inline-snapshot-0.20.8: Success >>=20 >> (Again longer 2.1.0 times vs. previous 2.1.0 times.) >>=20 >>>>=20 >>>> mail/mailest@nox >>>> build-depends: 00:00:01->00:00:28 >>>> run-depends: 00:00:30->00:00:59 >>=20 >> A pkg 2.0.6 vintage of ports tree on Windows Dev Kit 2023: >>=20 >> [00:00:58] [01] [00:00:00] Building mail/mailest@nox | = mailest-emacs_nox-0.9.24_21 >> [00:00:59] [01] [00:00:01] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: check-sanity >> [00:00:59] [01] [00:00:01] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: pkg-depends >> [00:00:59] [01] [00:00:01] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: fetch-depends >> [00:00:59] [01] [00:00:01] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: fetch >> [00:00:59] [01] [00:00:01] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: checksum >> [00:01:00] [01] [00:00:02] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: extract-depends >> [00:01:00] [01] [00:00:02] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: extract >> [00:01:00] [01] [00:00:02] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: patch-depends >> [00:01:00] [01] [00:00:02] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: patch >> [00:01:00] [01] [00:00:02] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: build-depends >> [00:01:00] [01] [00:00:02] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: lib-depends >> [00:01:00] [01] [00:00:02] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: configure >> [00:01:00] [01] [00:00:02] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: build >> [00:01:03] [01] [00:00:05] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: run-depends >> [00:01:08] [01] [00:00:10] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: stage >> [00:01:09] [01] [00:00:11] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: package >> [00:01:09] [01] [00:00:11] Finished mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: Success >>=20 >> A pkg 2.1.0 vintage of ports tree on Windows Dev Kit 2023: >>=20 >> [00:02:50] [01] [00:00:00] Building mail/mailest@nox | = mailest-emacs_nox-0.9.24_21 >> [00:02:51] [01] [00:00:01] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: check-sanity >> [00:02:51] [01] [00:00:01] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: pkg-depends >> [00:02:51] [01] [00:00:01] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: fetch-depends >> [00:02:51] [01] [00:00:01] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: fetch >> [00:02:51] [01] [00:00:01] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: checksum >> [00:02:51] [01] [00:00:01] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: extract-depends >> [00:02:51] [01] [00:00:01] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: extract >> [00:02:51] [01] [00:00:01] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: patch-depends >> [00:02:51] [01] [00:00:01] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: patch >> [00:02:52] [01] [00:00:02] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: build-depends >> [00:02:52] [01] [00:00:02] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: lib-depends >> [00:03:31] [01] [00:00:41] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: configure >> [00:03:31] [01] [00:00:41] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: build >> [00:03:32] [01] [00:00:42] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: run-depends >> [00:04:08] [01] [00:01:18] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: stage >> [00:04:08] [01] [00:01:18] Status mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: package >> [00:04:09] [01] [00:01:19] Finished mail/mailest@nox | = mailest-emacs_nox-0.9.24_21: Success >>=20 >> (Again longer 2.1.0 times vs. previous 2.1.0 times.) >>=20 >>>>=20 >>>> devel/dwarves >>>> build-depends: 00:00:05->00:02:23 >>>> lib-depends: 00:02:23->00:02:42 >>=20 >> A pkg 2.0.6 vintage of ports tree on Windows Dev Kit 2023: >>=20 >> [00:00:56] [07] [00:00:00] Building devel/dwarves | dwarves-1.19_3 >> [00:01:01] [07] [00:00:05] Status devel/dwarves | dwarves-1.19_3: = check-sanity >> [00:01:01] [07] [00:00:05] Status devel/dwarves | dwarves-1.19_3: = pkg-depends >> [00:01:01] [07] [00:00:05] Status devel/dwarves | dwarves-1.19_3: = fetch-depends >> [00:01:01] [07] [00:00:05] Status devel/dwarves | dwarves-1.19_3: = fetch >> [00:01:01] [07] [00:00:05] Status devel/dwarves | dwarves-1.19_3: = checksum >> [00:01:01] [07] [00:00:05] Status devel/dwarves | dwarves-1.19_3: = extract-depends >> [00:01:01] [07] [00:00:05] Status devel/dwarves | dwarves-1.19_3: = extract >> [00:01:02] [07] [00:00:06] Status devel/dwarves | dwarves-1.19_3: = patch-depends >> [00:01:02] [07] [00:00:06] Status devel/dwarves | dwarves-1.19_3: = patch >> [00:01:02] [07] [00:00:06] Status devel/dwarves | dwarves-1.19_3: = build-depends >> [00:01:07] [07] [00:00:11] Status devel/dwarves | dwarves-1.19_3: = lib-depends >> [00:01:08] [07] [00:00:12] Status devel/dwarves | dwarves-1.19_3: = configure >> [00:01:08] [07] [00:00:12] Status devel/dwarves | dwarves-1.19_3: = build >> [00:01:13] [07] [00:00:17] Status devel/dwarves | dwarves-1.19_3: = run-depends >> [00:01:13] [07] [00:00:17] Status devel/dwarves | dwarves-1.19_3: = stage >> [00:01:13] [07] [00:00:17] Status devel/dwarves | dwarves-1.19_3: = package >> [00:01:14] [07] [00:00:18] Finished devel/dwarves | dwarves-1.19_3: = Success >>=20 >> A pkg 2.1.0 vintage of ports tree on Windows Dev Kit 2023: >>=20 >> [00:02:54] [05] [00:00:00] Building devel/dwarves | dwarves-1.19_3 >> [00:02:58] [05] [00:00:04] Status devel/dwarves | dwarves-1.19_3: = check-sanity >> [00:02:58] [05] [00:00:04] Status devel/dwarves | dwarves-1.19_3: = pkg-depends >> [00:02:59] [05] [00:00:05] Status devel/dwarves | dwarves-1.19_3: = fetch-depends >> [00:02:59] [05] [00:00:05] Status devel/dwarves | dwarves-1.19_3: = fetch >> [00:02:59] [05] [00:00:05] Status devel/dwarves | dwarves-1.19_3: = checksum >> [00:02:59] [05] [00:00:05] Status devel/dwarves | dwarves-1.19_3: = extract-depends >> [00:02:59] [05] [00:00:05] Status devel/dwarves | dwarves-1.19_3: = extract >> [00:02:59] [05] [00:00:05] Status devel/dwarves | dwarves-1.19_3: = patch-depends >> [00:02:59] [05] [00:00:05] Status devel/dwarves | dwarves-1.19_3: = patch >> [00:02:59] [05] [00:00:05] Status devel/dwarves | dwarves-1.19_3: = build-depends >> [00:05:33] [05] [00:02:39] Status devel/dwarves | dwarves-1.19_3: = lib-depends >> [00:06:07] [05] [00:03:13] Status devel/dwarves | dwarves-1.19_3: = configure >> [00:06:07] [05] [00:03:13] Status devel/dwarves | dwarves-1.19_3: = build >> [00:06:12] [05] [00:03:18] Status devel/dwarves | dwarves-1.19_3: = run-depends >> [00:06:12] [05] [00:03:18] Status devel/dwarves | dwarves-1.19_3: = stage >> [00:06:12] [05] [00:03:18] Status devel/dwarves | dwarves-1.19_3: = package >> [00:06:12] [05] [00:03:18] Finished devel/dwarves | dwarves-1.19_3: = Success >>=20 >> (Again longer 2.1.0 times vs. previous 2.1.0 times.) >>=20 >>> net-mgmt/fastnetmon >>> build-depends: 00:00:03->00:00:42 >>> lib-depends: 00:00:42->00:01:29 >>=20 >> A pkg 2.0.6 vintage of ports tree on Windows Dev Kit 2023: >>=20 >> [00:01:00] [02] [00:00:00] Building net-mgmt/fastnetmon | = fastnetmon-1.2.8 >> [00:01:00] [02] [00:00:00] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: check-sanity >> [00:01:00] [02] [00:00:00] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: pkg-depends >> [00:01:01] [02] [00:00:01] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: fetch-depends >> [00:01:01] [02] [00:00:01] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: fetch >> [00:01:01] [02] [00:00:01] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: checksum >> [00:01:01] [02] [00:00:01] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: extract-depends >> [00:01:01] [02] [00:00:01] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: extract >> [00:01:01] [02] [00:00:01] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: patch-depends >> [00:01:01] [02] [00:00:01] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: patch >> [00:01:01] [02] [00:00:01] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: build-depends >> [00:01:03] [02] [00:00:03] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: lib-depends >> [00:01:07] [02] [00:00:07] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: configure >> [00:01:10] [02] [00:00:10] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: build >> [00:03:15] [02] [00:02:15] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: run-depends >> [00:03:15] [02] [00:02:15] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: stage >> [00:03:15] [02] [00:02:15] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: package >> [00:03:18] [02] [00:02:18] Finished net-mgmt/fastnetmon | = fastnetmon-1.2.8: Success >>=20 >> A pkg 2.1.0 vintage of ports tree on Windows Dev Kit 2023: >>=20 >> [00:02:54] [06] [00:00:00] Building net-mgmt/fastnetmon | = fastnetmon-1.2.8 >> [00:02:55] [06] [00:00:01] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: check-sanity >> [00:02:55] [06] [00:00:01] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: pkg-depends >> [00:02:56] [06] [00:00:02] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: fetch-depends >> [00:02:56] [06] [00:00:02] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: fetch >> [00:02:56] [06] [00:00:02] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: checksum >> [00:02:56] [06] [00:00:02] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: extract-depends >> [00:02:56] [06] [00:00:02] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: extract >> [00:02:56] [06] [00:00:02] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: patch-depends >> [00:02:56] [06] [00:00:02] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: patch >> [00:02:56] [06] [00:00:02] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: build-depends >> [00:04:10] [06] [00:01:16] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: lib-depends >> [00:05:41] [06] [00:02:47] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: configure >> [00:05:44] [06] [00:02:50] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: build >> [00:07:43] [06] [00:04:49] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: run-depends >> [00:07:43] [06] [00:04:49] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: stage >> [00:07:44] [06] [00:04:50] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: package >> [00:07:46] [06] [00:04:52] Finished net-mgmt/fastnetmon | = fastnetmon-1.2.8: Success >>=20 >> (Again longer 2.1.0 times vs. previous 2.1.0 times.) >>=20 >>> (See later below.) >>>=20 >>>> The timings are from the column next to >>>> the Building/Status/Finished column from >>>> using bulk -v , not from the column for >>>> the overall bulk run. >>>>=20 >>>>> I have tried to reproduce each individual case which happen in the = ports tree >>>>> and I am not able to reproduce them, so impossible to know where = to look at >>>>> exactly. >>>>=20 >>>> Try some of the examples that I've provided? >>>>=20 >>>> There are more examples that I could check >>>> and report non-parallel timings on if you >>>> want. I just picked to report on only a few >>>> initially. >>>>=20 >>>> An example that you might want is my >>>> providing more examples of lib-depends >>>> with non-parallel timings. >>>=20 >>> I took a quick look and quickly ran into: >>> (aarch64 Windows Dev Kit 2023 no-parallel-builders >>> timing again, after having built the prerequisites >>> that had not previously been built) >>>=20 >>> [00:11:37] [01] [00:00:00] Building net-mgmt/fastnetmon | = fastnetmon-1.2.8 >>> [00:11:39] [01] [00:00:02] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: check-sanity >>> [00:11:39] [01] [00:00:02] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: pkg-depends >>> [00:11:40] [01] [00:00:03] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: fetch-depends >>> [00:11:40] [01] [00:00:03] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: fetch >>> [00:11:40] [01] [00:00:03] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: checksum >>> [00:11:40] [01] [00:00:03] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: extract-depends >>> [00:11:40] [01] [00:00:03] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: extract >>> [00:11:40] [01] [00:00:03] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: patch-depends >>> [00:11:40] [01] [00:00:03] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: patch >>> [00:11:40] [01] [00:00:03] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: build-depends >>> [00:12:19] [01] [00:00:42] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: lib-depends >>> [00:13:06] [01] [00:01:29] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: configure >>> [00:13:09] [01] [00:01:32] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: build >>> [00:14:20] [01] [00:02:43] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: run-depends >>> [00:14:20] [01] [00:02:43] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: stage >>> [00:14:20] [01] [00:02:43] Status net-mgmt/fastnetmon | = fastnetmon-1.2.8: package >>> [00:14:22] [01] [00:02:45] Finished net-mgmt/fastnetmon | = fastnetmon-1.2.8: Success >>>=20 >>> (I still have thousands of packages that have not built >>> in the bulk -v -a build activity. The M4 MAX is in use >>> for that.) >>>=20 >>>>> I know what is new and what causes the performance penalty, but = not >>>>> which part is causing the super extra penalty on the cluster. >>>>=20 >>>> Various examples reproduce the timing issues >>>> outside the cluster and without the parallel >>>> builds. These results are from the M4 MAX context for pkg 2.1.0 use. I finished a "bulk -a" sequence (without having kldloaded linux = support). So I can now do the likes of: # poudriere bulk -jrelease-aarch64 -v -p alt -C www/gitlab@ee without having to build the prerequisites. No parallel builds involved. For that specific example I'll do it once before rebooting and once after, checking on caching effects. FYI: [00:05:28] Building 285 packages using up to 12 builders (But the prerequisites just get: "Inspecting . . .: determining shlib requirements", no actual builds.) I'll not repeat that part below. I'll also note that "Creating pkg repository" after such a build seems to not be an incremental activity for the small number of packages that change (1 here): (before reboot case) [00:29:41] Creating pkg repository Creating repository in /tmp/packages: 100% Packing files for repository: 100% [01:34:32] Committing packages to repository: = /usr/local/poudriere/data/packages/release-aarch64-alt/.real_1744660767 = via .latest symlink (after reboot case) [00:28:08] Creating pkg repository Creating repository in /tmp/packages: 100% Packing files for repository: 100% [01:29:53] Committing packages to repository: = /usr/local/poudriere/data/packages/release-aarch64-alt/.real_1744668230 = via .latest symlink (I'm not so sure if the pkg-static threads for my context mostly end up waiting for each other, the "gstat -spod" L(q) generally showing 16..22, sometimes more. biord and getblk commonly show in top, with at most 1 CPU? showing.) The 1hr+ extra makes experimenting more time consuming. I'm glad it is the faster M4 MAX as the context. It also means that I'm unlikely to try such on the Windows Dev Kit 2023 where the time could be much longer. Before reboot (but after bulk -a): [00:05:44] [01] [00:00:00] Building www/gitlab@ee | gitlab-ee-17.10.3 [00:05:46] [01] [00:00:02] Status www/gitlab@ee | gitlab-ee-17.10.3: = check-sanity [00:05:46] [01] [00:00:02] Status www/gitlab@ee | gitlab-ee-17.10.3: = pkg-depends [00:05:46] [01] [00:00:02] Status www/gitlab@ee | gitlab-ee-17.10.3: = fetch-depends [00:05:46] [01] [00:00:02] Status www/gitlab@ee | gitlab-ee-17.10.3: = fetch [00:05:56] [01] [00:00:12] Status www/gitlab@ee | gitlab-ee-17.10.3: = checksum [00:05:56] [01] [00:00:12] Status www/gitlab@ee | gitlab-ee-17.10.3: = extract-depends [00:08:25] [01] [00:02:41] Status www/gitlab@ee | gitlab-ee-17.10.3: = extract [00:08:34] [01] [00:02:50] Status www/gitlab@ee | gitlab-ee-17.10.3: = patch-depends [00:08:34] [01] [00:02:50] Status www/gitlab@ee | gitlab-ee-17.10.3: = patch [00:08:34] [01] [00:02:50] Status www/gitlab@ee | gitlab-ee-17.10.3: = build-depends [00:27:47] [01] [00:22:03] Status www/gitlab@ee | gitlab-ee-17.10.3: = lib-depends [00:27:47] [01] [00:22:03] Status www/gitlab@ee | gitlab-ee-17.10.3: = configure [00:27:48] [01] [00:22:04] Status www/gitlab@ee | gitlab-ee-17.10.3: = build [00:27:48] [01] [00:22:04] Status www/gitlab@ee | gitlab-ee-17.10.3: = run-depends [00:27:48] [01] [00:22:04] Status www/gitlab@ee | gitlab-ee-17.10.3: = stage [00:27:53] [01] [00:22:09] Status www/gitlab@ee | gitlab-ee-17.10.3: = package [00:29:40] [01] [00:23:56] Finished www/gitlab@ee | gitlab-ee-17.10.3: = Success So, somewhat over 19 min build-depends -> lib-depends. After reboot: [00:05:58] [01] [00:00:00] Building www/gitlab@ee | gitlab-ee-17.10.3 [00:05:59] [01] [00:00:01] Status www/gitlab@ee | gitlab-ee-17.10.3: = check-sanity [00:05:59] [01] [00:00:01] Status www/gitlab@ee | gitlab-ee-17.10.3: = pkg-depends [00:05:59] [01] [00:00:01] Status www/gitlab@ee | gitlab-ee-17.10.3: = fetch-depends [00:05:59] [01] [00:00:01] Status www/gitlab@ee | gitlab-ee-17.10.3: = fetch [00:06:06] [01] [00:00:08] Status www/gitlab@ee | gitlab-ee-17.10.3: = checksum [00:06:06] [01] [00:00:08] Status www/gitlab@ee | gitlab-ee-17.10.3: = extract-depends [00:09:37] [01] [00:03:39] Status www/gitlab@ee | gitlab-ee-17.10.3: = extract [00:09:46] [01] [00:03:48] Status www/gitlab@ee | gitlab-ee-17.10.3: = patch-depends [00:09:46] [01] [00:03:48] Status www/gitlab@ee | gitlab-ee-17.10.3: = patch [00:09:46] [01] [00:03:48] Status www/gitlab@ee | gitlab-ee-17.10.3: = build-depends [00:26:31] [01] [00:20:33] Status www/gitlab@ee | gitlab-ee-17.10.3: = lib-depends [00:26:31] [01] [00:20:33] Status www/gitlab@ee | gitlab-ee-17.10.3: = configure [00:26:31] [01] [00:20:33] Status www/gitlab@ee | gitlab-ee-17.10.3: = build [00:26:31] [01] [00:20:33] Status www/gitlab@ee | gitlab-ee-17.10.3: = run-depends [00:26:32] [01] [00:20:34] Status www/gitlab@ee | gitlab-ee-17.10.3: = stage [00:26:37] [01] [00:20:39] Status www/gitlab@ee | gitlab-ee-17.10.3: = package [00:28:07] [01] [00:22:09] Finished www/gitlab@ee | gitlab-ee-17.10.3: = Success So, somewhat over 16 min build-depends -> lib-depends. So, say, around 18 min for both before reboot and after it. Reproducible for general timescale. During the earlier "bulk -a" www/gitlab@ee got: [1D:19:47:18] [07] [00:00:00] Building www/gitlab@ee | = gitlab-ee-17.10.3 [1D:19:47:19] [07] [00:00:01] Status www/gitlab@ee | = gitlab-ee-17.10.3: check-sanity [1D:19:47:19] [07] [00:00:01] Status www/gitlab@ee | = gitlab-ee-17.10.3: pkg-depends [1D:19:47:20] [07] [00:00:02] Status www/gitlab@ee | = gitlab-ee-17.10.3: fetch-depends [1D:19:47:20] [07] [00:00:02] Status www/gitlab@ee | = gitlab-ee-17.10.3: fetch [1D:19:47:25] [07] [00:00:07] Status www/gitlab@ee | = gitlab-ee-17.10.3: checksum [1D:19:47:25] [07] [00:00:07] Status www/gitlab@ee | = gitlab-ee-17.10.3: extract-depends [1D:19:48:19] [07] [00:01:01] Status www/gitlab@ee | = gitlab-ee-17.10.3: extract [1D:19:48:50] [07] [00:01:32] Status www/gitlab@ee | = gitlab-ee-17.10.3: patch-depends [1D:19:48:50] [07] [00:01:32] Status www/gitlab@ee | = gitlab-ee-17.10.3: patch [1D:19:48:50] [07] [00:01:32] Status www/gitlab@ee | = gitlab-ee-17.10.3: build-depends [1D:21:11:47] [07] [01:24:29] Status www/gitlab@ee | = gitlab-ee-17.10.3: lib-depends [1D:21:11:47] [07] [01:24:29] Status www/gitlab@ee | = gitlab-ee-17.10.3: configure [1D:21:11:48] [07] [01:24:30] Status www/gitlab@ee | = gitlab-ee-17.10.3: build [1D:21:11:48] [07] [01:24:30] Status www/gitlab@ee | = gitlab-ee-17.10.3: run-depends [1D:21:11:50] [07] [01:24:32] Status www/gitlab@ee | = gitlab-ee-17.10.3: stage [1D:21:11:57] [07] [01:24:39] Status www/gitlab@ee | = gitlab-ee-17.10.3: package [1D:21:16:11] [07] [01:28:53] Finished www/gitlab@ee | = gitlab-ee-17.10.3: Success where the load averages were near the FreeBSD cpu count over the time frame. So, it appears that competing for I/O bandwidth with the other builders makes the difference of: around 18 min vs. around 83 min for build-depends -> lib-depends . =3D=3D=3D Mark Millard marklmi at yahoo.com