Date: Mon, 7 Apr 2025 09:44:42 -0700 From: Mark Millard <marklmi@yahoo.com> To: Baptiste Daroussin <bapt@FreeBSD.org> Cc: Gleb Popov <6yearold@gmail.com>, FreeBSD Current <freebsd-current@freebsd.org>, FreeBSD Mailing List <freebsd-ports@freebsd.org> Subject: Re: UPDATE: pkg 2.1.0 looks to be making official bulk builds of packages take much longer [a specific package with large time factor] Message-ID: <0414EBFB-B63A-4738-ADB0-38B6CF3725DA@yahoo.com> In-Reply-To: <7ziazrj7szuqhov3oppjbh3jyu3f2p2owntv4oxprelrdjzc6u@hkuf5szf3zwy> References: <8E2FBAD3-EF6F-4D99-A340-21F8FD19AE0F@yahoo.com> <FA9C7841-6D61-4285-AC6D-46328CBC56D2@yahoo.com> <C7BBEC76-228F-4F9E-A5F9-86E9F53372FD@yahoo.com> <D3C2F75D-838B-47E9-B125-71104A3C16EA@yahoo.com> <CD66A7B2-422B-40FE-BB17-145032DBA46F@FreeBSD.org> <84FBBAF8-025E-4B9D-9797-51735567A8DB@yahoo.com> <B295D1CB-E47D-4359-A79A-986213E7AFCE@yahoo.com> <366E27FD-FA5B-4BF8-B6C4-6C495DB289C5@yahoo.com> <CALH631mm8cdL1A9TaumnENk9OGCm=PEfMDH6uSvjKaUDuzg8Hw@mail.gmail.com> <F82E31E6-D9C1-461A-B3C8-4DB5D491C000@yahoo.com> <7ziazrj7szuqhov3oppjbh3jyu3f2p2owntv4oxprelrdjzc6u@hkuf5szf3zwy>
index | next in thread | previous in thread | raw e-mail
On Apr 7, 2025, at 08:14, Baptiste Daroussin <bapt@FreeBSD.org> wrote: > On Mon 07 Apr 08:07, Mark Millard wrote: >> . . . > > Listing like this is clearly not any useful, the problem we have is the > performance changes depending on what is happening in parallel on the machines. I've been exploring looking for an example that reproduces the timing issue via commands like: # poudriere bulk -jrelease-aarch64 -v -p default -c www/gitlab@ee vs. # poudriere bulk -jrelease-aarch64 -v -p alt -c www/gitlab@ee so that prior builds are not involved in creating such a context. Also, when www/gitlab@ee itself is building, no other builder will be active. I've started such a build based on a pkg 2.0.6 /usr/ports/ context and will try one based on a pkg 2.1.0 /usr/ports-alt/ context. I'm trying www/gitlab@ee because, on beefy17, it went from: 00:09:01 (pre pkg 2.1.0 example) to: 05:35:01 (pkg 23.1.0 example) (so somewhat over 37 times longer) and when I looked it up it has a huge number of dependencies: # pkg rquery -U -r FreeBSD "%#d : %n %o" www/gitlab@ee 298 : gitlab-ee www/gitlab The factor of 37 is large enough to be unlikely to have only load averages on beefy17 as a major contributor. Given the evidence about the count of dependencies, I will see. what I get. The test environment is a Apple Silicon M4 MAX system with FreeBSD running under Parallels in macOS. [00:00:07] Building 943 packages using up to 14 builders OOPS (via checking ampere2 logs): Looks like aarch64 might end up blocked for a rubygem-redis-actionpack-rails70 "Phase: stage" failure. I may have to set up a amd64 context for such experiments. > which makes the performance issues invisible on local poudriere if you want to > test it on port A or port B, if we want to reduce the performance penalty we > need to be able to make a reproducible case which can then be profiled, to know > where to optimize if needed. > > I have tried to reproduce each individual case which happen in the ports tree > and I am not able to reproduce them, so impossible to know where to look at > exactly. I'm hoping to supply reproducible steps. > I know what is new and what causes the performance penalty, but not > which part is causing the super extra penalty on the cluster. === Mark Millard marklmi at yahoo.comhome | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0414EBFB-B63A-4738-ADB0-38B6CF3725DA>