From nobody Mon Apr 7 16:44:42 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZWZnX3YZVz5rZDg for ; Mon, 07 Apr 2025 16:45:04 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-55.consmr.mail.gq1.yahoo.com (sonic308-55.consmr.mail.gq1.yahoo.com [98.137.68.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZWZnX1JkHz4767 for ; Mon, 07 Apr 2025 16:45:04 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1744044297; bh=CDbTS01qpohfnfM8Q4kphETdEvhrt1bD6wQEXRI17P0=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=giUlaReTFKAX+UJ1fQ1Uoglzi7SgRNGimRfUsQgL1E+Nsf+053oQiWSzs7hC/aeLEudk4Mutz2/PkAPEFU6yv/JAi0MfOuhWWRuuuZRbd9uwh2FcV7iODxg+O0N1/mCwcEX5vtbSSt+Gacww/W0LQGhpGcAriYrqK1Dt6srC9JYRURzR7Vj3u82ms/W4SgHDhcJMoW40sEmhLXYE+cBBJ48KBEY85X4GT8v+dKHUsW/S6yJIfiRQGXnw9V1YEYDF+bUwkqpOmeZxwNQN/joIh5bhrTEcteU1eWFqLG4czWXQKKovq9Vr5mFl9S6w1XM3TzqLsiG8FR8LHTnld43F4w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1744044297; bh=albLPdcu2GLeNrzxrAo8PCYdqFhVBuH+fM6A2NB7hAn=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=DVfgQD0Kc9b965BgSsGqddqCCS1vQZxM0f68nl6Fq7Nm3Hrs7KxswVdjFXz0JLncKmr5Dc4PAtdNS/SULnZkq9D1sHjMWjH9sDXqjyxH7lZo4fh2Tk5dF9UZqIyRYiHK1N3MV4Kg6KUKVBQtgrcKbvt8gNhV/Ufd0Bn2BaIEO01yT8EwyMdCX0XIilgtFayn55l9Hkf4F7H0kYnX9PUMWzmaRJSelPSN8wF/RmSbnY2QIiLLXZ9kQfvpF12YrLW2XOBFjY0KfolQ8WYyx1bjFxb3s+xie1GrNfvfTZCL3zG2TbXiHdDJuRD2E6A68gkJvoY1mkHRFBGu1mAFI31IxA== X-YMail-OSG: AncgcqAVM1mm8EQprGXg.H1FlYGAbSur85JAQLhrcqkUv9z6g9u_F.55Wm0VxS8 IdTFW6nSDGIlAkkuBR7yze9m58zgSGpMV2Lj5JFyNRo9cyRz4izTkgOigCG60.oqrvZjm2YcCzlV ggyBjSn6lIKg7PHqL2TYbx0WXU0brlOYazUWZDbayW96YrAWxkVGECgIdhD40Jf3BNnk_GPVtcJk 9GJdKYMXSNuZ46lA4yDBWfiml2PO9rgnx.G5TNVUeng.lLFOvNwb6Sux7DrhhJTTINoLbENaj6VD gHFVeEr_1ZHNb1LqY2vTqxaVGqVUxfBh.dri8Z46eOQVMHXjiCHN.XK6DOTK.GQwtkLfll_SBUPB DNtgZt5.lZzC9NfigcFoOZXTEq0qeqEbWD1PY_mrOfQoPcviwaTSfhcToNMnIRmdDNWUMNCduWeW O.neT4zykf58u8nYd_vI5kny0CXZfEM0mg7HwyfWCy89UtA1xlbC1Ag2NjXbBdOXELsCbYshZF7y omi0LiB7xRbb4cxkR5r.xueFCF6.MKy.itthl6T_JWtug2fQmCD8RpjfWVIDums.hZCgka15SjCh wGNk8WGzsCNTilEqOsvkrFubg7fmyoUUGxpLtj6ll6POG6pMV2SXxKlcvlslg_s2WKfj4WIGfsYU 9xxBKd.7YobZ__DIH.2L9wn5W96WRjcaFPqxBc3gRV3peSMVf3tjUKfsTLpSrhKweUH_7WcmzzP9 Vr1jczGL7M_QdB5Bf5SXmRQ.fXCdOchrY6azAR8MNKQlWvdvUmxQSGsz.oLYApqVxPM4d8ZJKeo9 qkomQUOoO_bELxs5x5KdZ4hjvMXfVXlkDmitI6uUKc0HFU8tXhtfTvAdTdj4KPlxPmTBz33WTWCO eWeUJCySj.daurkIGeCs8VCnnitToT8u8L_7vyWc.jggoMs0lRP4ukwn0vGJqgug27T5ZKVOsNeb YYeSRhUxDmm6ZesGFEs5Sf2L.jIligDrstp3QROvlk56vAEZg4DHOF0QoHyiDYo1m2CSRA7QXXTl cEb3LUeUFkdv_FtBcxldtnWyQXRm7Q0GQaz_iWYOezOqecLVbO7D8tSrqcR2MZ0395JJH7ki9ys4 TF6RiZKnFeXwjBK_5M0R8O1V2A0nO4GInDR4MltzWQW3TBJmMrOaeSMT1x__U1oYodSnkRohxV8O XyIEFW9cMnCI2eq8Sno3L5.hP9P_bJoWauq1h2VHEfUze_hYEtVw7dn6QJnMvVap2ap1pOeiSogl 8VjLneQazafzHY7x34IY9eNkbyBSxr3GgsCE7l5rGYsyo_E9Ft5Ak2TuNdIPJIK8EfBwH6snASMk OwfOAOzllKUuPVFJlAFUZDzR1VhA.sW8LYEArH9nRVa6_g6KdFpmgMbVtYN.s1OkotREc33bkO_O saiKS1AXu03ioLQcjN00cN0QM8f61auPyEMJAZijS0.alixc.kIaM3RQkZyOTCyFkcaafe_vDXwP .ajX5jZvnBCfME..CFQK0cATcqf37qv4vg29aP4eXfzRh3oaYal1EICgmG2_za5b4DREaMV_iA_C usb1WjkBW0VCVz.xq2Qtr_jINkXis0f1v3jtRJ.c9laMd8C3t0LyJ6nxxfLBgZuBFsoj9xqQDKbX HYO96ze0sE7LkZ.5VhjENjoP3NZRWm_0E_H3VtV87IjVmOmxjoSemob9GU3hO9_MJbA6zt14dz6F BxKM7V550MC61u_Nwj5QeYb1bYAL_Ctq2sRZZFKewMhSq4zlsqtiqp5rwGfzv1ct2nDCqhEhMB3e GgQ8o2TfLVMenymhi0jsy4S06vPNsJOl00rdyyUlZr1Caq7TvD5YkUTibA5tPhYL6X1L1pE7skZm QsrwWXPkve8JyHxr0DMKW8tt0TrMSIWIhNBdhQLtH9qVnsqT_.aC5ixNuK9B_4b63b3.qKqLU9SF RSRv0g3bYywPfUEkq.O.A5MvrCIVsl2vlB2AxF5VitF2eJkGkPG4fFLVHqo5EymttqmRB4t.Kziz rLvpDbPmAuuTGb01cMe0y1046yjdqrEO5DxRHL7P2wRG4BoV4l3Z90kMBxJPvlS7I_HsQBpFUMwX Rer0sG3oomy4k0jOfyZ7E9CsZc.YAAbPqaYP2YP3ElHNU4nqA7YjxBBRMbvu5TVPT.WfQQNn7EfT gr8lNjvcUwQRbWRpbgoL7OctVGdkEupQD9cb4hsL4Rck6eIUrtj3M35UKFBBwpoY.LZZBCA0F3Nr Whvt0Ft.4aAfA2HiKs4_iIusAtNZLzrKOSI6Q3awxbxDIPWBowe5Lt1oYTR7Mdue_3MClbY6Qn.V xCdw- X-Sonic-MF: X-Sonic-ID: 19697dfd-c176-4c57-85d5-742e5c25fcb1 Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.gq1.yahoo.com with HTTP; Mon, 7 Apr 2025 16:44:57 +0000 Received: by hermes--production-gq1-5c477bf655-fdl68 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 9e7bccd7b090af4b235397afeb50e73c; Mon, 07 Apr 2025 16:44:53 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.500.181.1.5\)) Subject: Re: UPDATE: pkg 2.1.0 looks to be making official bulk builds of packages take much longer [a specific package with large time factor] From: Mark Millard In-Reply-To: <7ziazrj7szuqhov3oppjbh3jyu3f2p2owntv4oxprelrdjzc6u@hkuf5szf3zwy> Date: Mon, 7 Apr 2025 09:44:42 -0700 Cc: Gleb Popov <6yearold@gmail.com>, FreeBSD Current , FreeBSD Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: <0414EBFB-B63A-4738-ADB0-38B6CF3725DA@yahoo.com> References: <8E2FBAD3-EF6F-4D99-A340-21F8FD19AE0F@yahoo.com> <84FBBAF8-025E-4B9D-9797-51735567A8DB@yahoo.com> <366E27FD-FA5B-4BF8-B6C4-6C495DB289C5@yahoo.com> <7ziazrj7szuqhov3oppjbh3jyu3f2p2owntv4oxprelrdjzc6u@hkuf5szf3zwy> To: Baptiste Daroussin X-Mailer: Apple Mail (2.3826.500.181.1.5) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4ZWZnX1JkHz4767 X-Spamd-Bar: ---- On Apr 7, 2025, at 08:14, Baptiste Daroussin wrote: > On Mon 07 Apr 08:07, Mark Millard wrote: >> . . . >=20 > Listing like this is clearly not any useful, the problem we have is = the > performance changes depending on what is happening in parallel on the = machines. I've been exploring looking for an example that reproduces the timing issue via commands like: # poudriere bulk -jrelease-aarch64 -v -p default -c www/gitlab@ee vs. # poudriere bulk -jrelease-aarch64 -v -p alt -c www/gitlab@ee so that prior builds are not involved in creating such a context. Also, when www/gitlab@ee itself is building, no other builder will be active. I've started such a build based on a pkg 2.0.6 /usr/ports/ context and will try one based on a pkg 2.1.0 /usr/ports-alt/ context. I'm trying www/gitlab@ee because, on beefy17, it went from: 00:09:01 (pre pkg 2.1.0 example) to: 05:35:01 (pkg 23.1.0 example) (so somewhat over 37 times longer) and when I looked it up it has a huge number of dependencies: # pkg rquery -U -r FreeBSD "%#d : %n %o" www/gitlab@ee 298 : gitlab-ee www/gitlab The factor of 37 is large enough to be unlikely to have only load averages on beefy17 as a major contributor. Given the evidence about the count of dependencies, I will see. what I get. The test environment is a Apple Silicon M4 MAX system with FreeBSD running under Parallels in macOS. [00:00:07] Building 943 packages using up to 14 builders OOPS (via checking ampere2 logs): Looks like aarch64 might end up blocked for a rubygem-redis-actionpack-rails70 "Phase: stage" failure. I may have to set up a amd64 context for such experiments. > which makes the performance issues invisible on local poudriere if you = want to > test it on port A or port B, if we want to reduce the performance = penalty we > need to be able to make a reproducible case which can then be = profiled, to know > where to optimize if needed. >=20 > I have tried to reproduce each individual case which happen in the = ports tree > and I am not able to reproduce them, so impossible to know where to = look at > exactly. I'm hoping to supply reproducible steps. > I know what is new and what causes the performance penalty, but not > which part is causing the super extra penalty on the cluster. =3D=3D=3D Mark Millard marklmi at yahoo.com