From nobody Mon Feb 24 11:37:17 2025 X-Original-To: freebsd-pkg@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Z1dy44Y1Bz5nvcN for ; Mon, 24 Feb 2025 11:37:32 +0000 (UTC) (envelope-from diego@bsd.com.br) Received: from mail-yb1-xb29.google.com (mail-yb1-xb29.google.com [IPv6:2607:f8b0:4864:20::b29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Z1dy23Q4Sz3jb7 for ; Mon, 24 Feb 2025 11:37:30 +0000 (UTC) (envelope-from diego@bsd.com.br) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsd.com.br header.s=capeta header.b=VhPGI6K+; dmarc=none; spf=pass (mx1.freebsd.org: domain of diego@bsd.com.br designates 2607:f8b0:4864:20::b29 as permitted sender) smtp.mailfrom=diego@bsd.com.br Received: by mail-yb1-xb29.google.com with SMTP id 3f1490d57ef6-e5dd164ee34so3775653276.2 for ; Mon, 24 Feb 2025 03:37:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsd.com.br; s=capeta; t=1740397049; x=1741001849; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=xejfiB6Gq/YEIqHNLiFlODn1kiL9Nj+fw8BajpUwxDY=; b=VhPGI6K+hpU+oirKDIK625QVNfAvbkR8IX1jlSJEpHDY4h5qsySF3Cfqua/Q6+B0uf 1c62DqIQeAIA7ubCGT62eawTwT2SwWyZU4K73SPjV8VuZ0xzwHIfNh3xXR7lz7cY6C7e 50x5bKDSy3e2yjl5paqMC31xCMztnhBj7Ni2Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740397049; x=1741001849; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=xejfiB6Gq/YEIqHNLiFlODn1kiL9Nj+fw8BajpUwxDY=; b=R+Hrax+CLikmlCHaahSMs2aPhQNuhEBBogynyLPoa+d67ThwKYgEwWv0Z6gyc+06gS cyx00tvMs3gRoDvYDfJVlZjqnvyrrgk/L2KLkg4CRSFM9fUD+9RTAJ8qJghqN9oG0pdU SSQ6vMJ4a+RUiBP5nMQskcwfRXDiy+3dtbu7C7+S1AVFcUqsEKtExGmHUw4XikRuhrE/ D4aJI1Pm3NFE8kVXuOvmfzZImSHUNaPsqf9j7GjGjOgza/swULYJUXhsJ++2Og1Zzo4J 0IXuz2cSqBobirhqiLXVaUh368CekbO5nlkShKm3ySsQOioU1NpPvlR51xYe//negJ+d f3pA== X-Gm-Message-State: AOJu0YyDA6vxvIxNWlKgBYZi04JBvXuiu6eiN3THB7w3upN9n8mAGvFS L5BfUvzySxhr/S0pSnIWLnIXKXbWetL8B2nDf/1w+TJQuGw+jci3HDynIygm1DxYgVqvLATayfn b3CmViTWi4OP3wNZAWgS8zW5hRlrQaic7Lr8O X-Gm-Gg: ASbGncu+e5GyodXQuw8nLX9mkMLrfboYqRSS/TpLTLdPF4d5B3rm6QhgfE6Lvv4DGsC AIH4UB6JfTfGrgwOWwFrbLC3omP+L6a6rXIJh8Rel9RR053nYo8Q5ZjW+zOvcKJFDHVCa1fag1I 2b1VJwdNcD X-Google-Smtp-Source: AGHT+IEs6rNvpNY/TB2xcjNkvtAo3MEja0F8JoU/W4cs3U4bTeI3i4ynnJqVPcGfXd8jcchCCIrRuUpFgNd9F+9FAks= X-Received: by 2002:a05:6902:a09:b0:e5e:6d6:8eeb with SMTP id 3f1490d57ef6-e5e8afcd8fbmr8752805276.12.1740397048910; Mon, 24 Feb 2025 03:37:28 -0800 (PST) List-Id: Binary package management and package tools discussion List-Archive: https://lists.freebsd.org/archives/freebsd-pkg List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-pkg@FreeBSD.org MIME-Version: 1.0 References: <566D4C18-A850-4394-A717-4E61FF0EA129.ref@yahoo.com> <566D4C18-A850-4394-A717-4E61FF0EA129@yahoo.com> In-Reply-To: From: Diego Linke Date: Mon, 24 Feb 2025 08:37:17 -0300 X-Gm-Features: AWEUYZkLXkB6WrHqTRA9Ej2oTBUBsh9D_p6ICL0s-OHByFjSR_ZrItdSfAn4zZY Message-ID: Subject: Re: arm64 pkg building is taking longer To: Mark Millard Cc: FreeBSD-pkg@freebsd.org Content-Type: multipart/alternative; boundary="000000000000de76ed062ee1c3bd" X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; R_DKIM_ALLOW(-0.20)[bsd.com.br:s=capeta]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[diego]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::b29:from]; RCVD_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[FreeBSD-pkg@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[yahoo.com]; MISSING_XM_UA(0.00)[]; RCVD_TLS_LAST(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-pkg@freebsd.org]; DMARC_NA(0.00)[bsd.com.br]; DKIM_TRACE(0.00)[bsd.com.br:+] X-Rspamd-Queue-Id: 4Z1dy23Q4Sz3jb7 X-Spamd-Bar: --- --000000000000de76ed062ee1c3bd Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Mark, good morning! The bind920 package, in the quarterly branch, is still unavailable for ARM64. We already have a new version of bind920 committed 9.20.6. Mat committed the Security fixes on Feb 3rd. I already fixed it by building it myself, but I am thinking about the broad community and the increase of the adoption of ARM64 architecture. Considering the challenges that you described, do you think more ampere servers would help? If so, maybe we should work with The FreeBSD Foundations to try to get these resources for this project as part of some sponsorship. Thanks, Diego Linke On Mon, Feb 10, 2025 at 7:22=E2=80=AFPM Diego Linke wrot= e: > Hi Mark, > > Thank you very much for the detailed explanation. > > I appreciate the insights into the differences in build resources and the > constraints affecting ARM64. I also appreciate that security updates are > already prioritized within the available resources. > > Regards, > > Diego Linke > > > On Mon, Feb 10, 2025 at 6:48=E2=80=AFPM Mark Millard = wrote: > >> Diego Linke wrote on >> Date: Mon, 10 Feb 2025 12:07:19 UTC : >> >> > I noticed that ARM64 packages take a few days longer to build and beco= me >> > available on the official FreeBSD servers compared to AMD64 and i386. >> >> The amd64 systems are generally faster and there are >> a lot more of them used as package builders. >> >> There are only 3 aarch64 build systems: ampere[1-3]. >> >> By contrast, there are 10 amd64 build systems, 3 being >> fairly modern and vastly faster than the ampere*'s >> (far more hardware threads, for example). >> >> ampere1 cycles through building and distributing: >> 141arm64-quarterly >> 141releng-armv7-quarterly >> 1341arm64-quarterly >> 134releng-armv7-quarterly >> >> amd64 systems do not build that many variations on the >> same machine: in fact each normally only builds one >> variation, no waiting for other cycle members to >> finish. >> >> As each takes longer, the next time it gets back to the >> same type of build, it is likely that far more packages >> that need to be built (compared to the analogous amd64 >> context). It is not as extreme for quarterly as it is >> for default (a.k.a. latest). >> >> ampere3 is similar (default here is a.k.a. latest): >> 141arm64-default >> 141releng-armv7-default >> 1341arm64-default >> 134releng-armv7-default >> >> ampere3 likely builds the most per poudriere bulk run >> compared to ampere1 above, taking the largest to get >> back to the next build of the same member of the cycle. >> >> ampere2 has only 2 cycle members (as stands, main is 15.0): >> main-arm64-default >> main-armv7-default >> >> So that makes 10 separate variations overall, spread >> over the 3 ampere*'s. >> >> amd64/i386 has 10 separate variations as well, but >> 1 per amd64 system that does the type build. >> 141amd64-quarterly , 141amd64-default , and >> 141i386-default are what get the fastest 3 builder >> machines. >> >> > For example, last Monday, Mat committed the Bind 9.2 (dns/bind920) >> update >> > to 9.20.5 in the quarterly branch, which has two security fixes. This >> > update was available on amd64 and i386 2 days later. It's still not >> > available (after 7 days) for ARM64. >> >> Expected sort of result, given the resources available to put >> to use. >> >> > Could we prioritize building packages with security updates, especiall= y >> for >> > the quarterly branch? >> >> Already done: The more and bigger "default" builds do not >> complete for the machine time on ampere1. Mixed on the >> same machine the "default" builds would further delay the >> quarterly builds. >> >> > Is anyone aware of any initiatives to improve this >> > process? >> >> Unless the aarch64/armv7 system resources are considered >> as part of the "process": it is not basically a process >> problem. (I'm not intending to imply that "no optimization >> is possible", just that such is not likely to lead to >> a large change of scale for how long things take in >> order to reach similar times to amd64 now takes.) >> >> > PS: I=E2=80=99m aware that I can set up my own package build infrastru= cture >> using >> > Poudriere and am considering it as a fallback option. However, I=E2=80= =99d like >> to >> > explore whether we can improve the process for everyone. >> >> That last note repeats here: it is not basically a process >> problem for what is the major constraint. >> >> >> >> =3D=3D=3D >> Mark Millard >> marklmi at yahoo.com >> >> --000000000000de76ed062ee1c3bd Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Mark, good morning!

The bind= 920 package, in the quarterly branch, is still unavailable for ARM64. We al= ready have a new version of bind920 committed=C2=A09.20.6.
Mat co= mmitted=C2=A0the Security fixes on Feb 3rd. I already fixed it by building = it myself, but I am thinking about the broad community and the increase of = the adoption of ARM64 architecture.

Considering th= e challenges that you described, do you think more ampere servers would hel= p? If so, maybe we should work with The FreeBSD Foundations to try to get t= hese resources for this project as part of some sponsorship.=C2=A0

Thanks,

Diego Linke


On Mon, Feb 10, 2025 at 7:22=E2=80=AFPM= Diego Linke <diego@bsd.com.br&g= t; wrote:
Hi Mark,

Thank you very much for t= he detailed explanation.=C2=A0

I appreciate the in= sights into the differences in build resources and the constraints affectin= g ARM64. I also appreciate that security updates are already prioritized wi= thin the available resources.=C2=A0

Regards,

Diego Linke


On Mon, Feb 10, 2025 at 6:48=E2=80=AFPM Mark Millard <marklmi@yahoo.com>= wrote:
Diego Li= nke <diego_at_bsd.com.br> wrote on
Date: Mon, 10 Feb 2025 12:07:19 UTC :

> I noticed that ARM64 packages take a few days longer to build and beco= me
> available on the official FreeBSD servers compared to AMD64 and i386.<= br>
The amd64 systems are generally faster and there are
a lot more of them used as package builders.

There are only 3 aarch64 build systems: ampere[1-3].

By contrast, there are 10 amd64 build systems, 3 being
fairly modern and vastly faster than the ampere*'s
(far more hardware threads, for example).

ampere1 cycles through building and distributing:
141arm64-quarterly
141releng-armv7-quarterly
1341arm64-quarterly
134releng-armv7-quarterly

amd64 systems do not build that many variations on the
same machine: in fact each normally only builds one
variation, no waiting for other cycle members to
finish.

As each takes longer, the next time it gets back to the
same type of build, it is likely that far more packages
that need to be built (compared to the analogous amd64
context). It is not as extreme for quarterly as it is
for default (a.k.a. latest).

ampere3 is similar (default here is a.k.a. latest):
141arm64-default
141releng-armv7-default
1341arm64-default
134releng-armv7-default

ampere3 likely builds the most per poudriere bulk run
compared to ampere1 above, taking the largest to get
back to the next build of the same member of the cycle.

ampere2 has only 2 cycle members (as stands, main is 15.0):
main-arm64-default
main-armv7-default

So that makes 10 separate variations overall, spread
over the 3 ampere*'s.

amd64/i386 has 10 separate variations as well, but
1 per amd64 system that does the type build.
141amd64-quarterly , 141amd64-default , and
141i386-default are what get the fastest 3 builder
machines.

> For example, last Monday, Mat committed the Bind 9.2 (dns/bind920) upd= ate
> to 9.20.5 in the quarterly branch, which has two security fixes. This<= br> > update was available on amd64 and i386 2 days later. It's still no= t
> available (after 7 days) for ARM64.

Expected sort of result, given the resources available to put
to use.

> Could we prioritize building packages with security updates, especiall= y for
> the quarterly branch?

Already done: The more and bigger "default" builds do not
complete for the machine time on ampere1. Mixed on the
same machine the "default" builds would further delay the
quarterly builds.

> Is anyone aware of any initiatives to improve this
> process?

Unless the aarch64/armv7 system resources are considered
as part of the "process": it is not basically a process
problem. (I'm not intending to imply that "no optimization
is possible", just that such is not likely to lead to
a large change of scale for how long things take in
order to reach similar times to amd64 now takes.)

> PS: I=E2=80=99m aware that I can set up my own package build infrastru= cture using
> Poudriere and am considering it as a fallback option. However, I=E2=80= =99d like to
> explore whether we can improve the process for everyone.

That last note repeats here: it is not basically a process
problem for what is the major constraint.



=3D=3D=3D
Mark Millard
marklmi at yahoo.com

--000000000000de76ed062ee1c3bd--