From nobody Sun Mar 1 01:57:43 2026 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4fNlZl2C2vz6SmwX for ; Sun, 01 Mar 2026 01:58:07 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-yw1-x1129.google.com (mail-yw1-x1129.google.com [IPv6:2607:f8b0:4864:20::1129]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4fNlZj4KY0z3qtv for ; Sun, 01 Mar 2026 01:58:05 +0000 (UTC) (envelope-from kob6558@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=OBhrsmdb; dmarc=pass (policy=none) header.from=gmail.com; arc=pass ("google.com:s=arc-20240605:i=1"); spf=pass (mx1.freebsd.org: domain of kob6558@gmail.com designates 2607:f8b0:4864:20::1129 as permitted sender) smtp.mailfrom=kob6558@gmail.com Received: by mail-yw1-x1129.google.com with SMTP id 00721157ae682-793fdbb8d3aso39261327b3.3 for ; Sat, 28 Feb 2026 17:58:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1772330280; cv=none; d=google.com; s=arc-20240605; b=G7lEpJAvNUB8vMQStNUor0ulM7qFh+/vBHwp5wFFInypJXxiJFWsFuXBA3mQX9H/sD IZRa67IF6w2CAAIJR6ZCMNilRzoXWmCdisFQ/UI16o55QjmpIs2JSTC8xrYWKjRuT176 oYTW1hRhRJpruHabnfPX9OvXUIx1w5A+aFRVtylvDyuCFN8eSsAB7AsVUnJkUlPRwCnD B9TPzALs10Rj7zAB9rtbVxcaE7xWA1YZ0utxfLTc8Mhp2p3WIaaUbrvRaVCBilzcMcWj hXw84li6W4rPsioOJvDQAGHdimfBdaf+VWNdEwB/mxLpPYMRUjtg8gpUchgunQASIUdP z0Fg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=zWFkz9EalbZPkk1PzH97eCP+7nu9jKt3ZzxUSJQoHxc=; fh=Xm+2y58Xeg/86S/wqA3YUSPH4zyCoVBn9p5W2Fgxa8g=; b=dqglRzcBXlRzEKy/e7D8BEPucn+z68Eu/w9gpbJ9mtfd44qVWDM+y8DnbWA8i2Zx5d 6KvMSeAfziacuv28BLXOKRJYsfWPQW4xBWszqppunGfaW9aXvHJqSqOVV36r82waLJNF UC4xPvmT9JNA+2uBGdpZbJk32toVmSay3GwyMPJjjivpt44HkYUgk/DbFqU6Zu77qaFY jLOjA3sgpErfLXEhEUfiKySQ6vOAGb87yTs3IeOCqQ6XPZu0Y08p9IzflXeWGIQiSFqK sdkdHQymQdJG8RPYMKmkSq0GqENPlt8lsQE5T+6p2SkES7TuDkaUOoTOmVbckMTCG6It tPCQ==; darn=freebsd.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1772330280; x=1772935080; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=zWFkz9EalbZPkk1PzH97eCP+7nu9jKt3ZzxUSJQoHxc=; b=OBhrsmdbnjTdAHW2wZOFR0l7h71qpLDPZ6fiy19jP4cA/9yuVOYCWLTxeckVbHILxa 66aChv160rkkNvl/webR/J96CIrNAZeUGfP0WRScrGdom17VowBELpi7qDBQD+I+J29F y39hgdF8wp61D0x1tTdUS6OSS6qWSnoKuvzzdI0DtrTWqdY7cWrAjKRATbyKzvYYm1wo h9x7I7y4i/k8DZLNMAlmcluq6otKUn0Su2NBUiyYiNDgebUgaQTjdzQHAOpxkQ36uuku QpGa+R1Vb/QU878TeBa4ZC7mp+h4+TJ0RZikX4ENSgCA7xlOuBWdXBrM7fWoDo3RH7Nz nxsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772330280; x=1772935080; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=zWFkz9EalbZPkk1PzH97eCP+7nu9jKt3ZzxUSJQoHxc=; b=nLjfUNAn7BVs+pv3DwkvqVsHh8umHeLSezK/KqXlcgqKRcI2/THXhnLqkmXIU7umUY pC8BsVwxI/XFcXd8JT/S9TX1Pr2l4p05uwSxhU6CwfxIpiaVSbAtG8YDNhANDBr8BYV4 lyNuFvoVqGuOdQpQXhd4mtBif3+D6ReNJ6p2i8CGP0Fhpm4ZRIVuzxNpL4GQr5+sC4lC Ez17aUKq3neBI7jkXuKQxuoX3fi34teC/dtWxN+JbsZeXLTdLW8t/KqmazTHmpqHPtK9 e05jscrxOYIrfhNS/qY0HDtoq8gUodWZqq3wjeNw5E15oZlGMxNV3DYQiyAEroPUnOy8 f5fQ== X-Gm-Message-State: AOJu0YyD0/iYrWhjsybtIND4rwjBVUhoXlCjy0SXBCJsKVQYOZAhO3xu KbRRuhzzxcaQZcR2LMFCGGn0+PRNOdPHasm09KjGpXff9DvrSqfZaqs4Rkm2XpVlpEbsi1h0e/D QeHUq7axL2vIXwOqUuHSJujwZHxpy5l4= X-Gm-Gg: ATEYQzzyQxcRz3qp4GHObh96IKStJbbTIkqJ4CVLqH5xlxTP7ARbfZkTSkeVLl2EIuM 94gRAT8T15cv5tFqbJb0YeW3Jrh4z3igVJPxqjZLZB1AlXEyAA2c8G7sEZ3MvZ+XWpiDRXGehTk TizU3ouwlOg4ucrI6L2L98GL5x8+vWRJJlRsMMlaVQBaf7ucHQsIMb2VDoInXXvDq5ZU4qpa4p9 QlWO5pVqZzWDZx9EDwL9Ylhey8xlby2X0rfV8G3+dV0ncN//1O/bdBSRIng2DbOmnJ0Mn9S1PL/ YZSUYE9+50gi9bqPnBlDC8fPbhsgiCdzuft2FRnoWKG2Zep10wLO+cNLNh6wfRpJ7P3m X-Received: by 2002:a05:690e:42dc:b0:64c:aab1:89fb with SMTP id 956f58d0204a3-64cc20cbca6mr4681569d50.32.1772330279799; Sat, 28 Feb 2026 17:57:59 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: <1895ba62-b069-4e28-a910-6c666703ee8e@gmail.com> In-Reply-To: From: Kevin Oberman Date: Sat, 28 Feb 2026 17:57:43 -0800 X-Gm-Features: AaiRm50xH-DUCVt1hb1R6bCJXI3k_HioVO58sYj9k54zctPxx5Tt3mqY9qnNCV0 Message-ID: Subject: Re: 504 gateway time-outs (was: FYI: The elapsed time shown for beefy24 [main-amd64] and beefy23 [150amd64-default] stopped over 2hrs ago) To: Graham Perrin Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Content-Type: multipart/alternative; boundary="000000000000c08eda064beccca4" X-Spamd-Result: default: False [-4.69 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; ARC_ALLOW(-1.00)[google.com:s=arc-20240605:i=1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.992]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FORGED_SENDER(0.30)[rkoberman@gmail.com,kob6558@gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEMAIL_FROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; RCVD_COUNT_ONE(0.00)[1]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[rkoberman@gmail.com,kob6558@gmail.com]; RCPT_COUNT_THREE(0.00)[3]; MID_RHS_MATCH_FROMTLD(0.00)[]; MISSING_XM_UA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1129:from] X-Rspamd-Queue-Id: 4fNlZj4KY0z3qtv X-Spamd-Bar: ---- --000000000000c08eda064beccca4 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Feb 28, 2026 at 5:48=E2=80=AFPM Kevin Oberman = wrote: > On Sat, Feb 28, 2026 at 3:33=E2=80=AFAM Graham Perrin > wrote: > >> On 28/02/2026 10:32, Mark Millard wrote: >> > =E2=80=A6 >> > >> > The following got "504 Gateway Time-out" when I tried them: >> > >> > < >> https://pkg-status.freebsd.org/beefy24/build.html?mastername=3Dmain-amd6= 4-default&build=3Dpdf4f957ea181_s178d0b5b8d >> > >> > >> > < >> https://pkg-status.freebsd.org/beefy23/build.html?mastername=3D150amd64-= default&build=3Ddf4f957ea181 >> > >> >> >> Both somewhat slow to load, however they do load for me. >> >> Thanks > > > beefy22 is showing hte same issues. Something is tying up the builders > with builds hanging in "build-depends" and "lib-depends" for very long > periods of time, as long as over 4 hours) with high load averages. This h= as > been going on for months and seems to be getting worse. Chromium builds > which used to take around 20 hours are now running around 40 and I really > don't think that this is just code bloat. At times, about half of the > active builds are in some state other than "build"; mostly one of the > depends states which should never take long as all dependencies are built > before a build is started. > > After some time passes (often hours) things clear up. Dependencies are > suddenly loaded in a few minutes or less. Note that things do continue to > complete, but the 10 minute averages are in single digits and frequently = go > to 0. During this time, my connection to beefy22 will timeout and sometim= es > I can't establish a connection. > > I don't have any special access to the machines... just via pkg-status, s= o > I'm fairly limited in any analysis. > Just before I started my last message, all was well, but I last saw were a dozen builds in one of the "depends" states and "impulse" at 74 when I lost my connection. I can reconnect to the beefy22 status page, but it has not updated for several minutes. --=20 Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 --000000000000c08eda064beccca4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Sat, Feb 28, 2026 at 5:48=E2= =80=AFPM Kevin Oberman <rkoberman= @gmail.com> wrote:
On Sat, Feb 28, 2026 at 3:33=E2=80=AFAM Graham Perrin <grahamperrin@gmail.com= > wrote:
On 28/02/2026 10:32, Mark Millard wrote:
> =E2=80=A6
>
> The following got "504 Gateway Time-out" when I tried them:<= br> >
> <https://pkg-status.freebsd.org/beefy24/build.ht= ml?mastername=3Dmain-amd64-default&build=3Dpdf4f957ea181_s178d0b5b8d>
>
> <https://pkg-status.freebsd.org/beefy23/build.html?mastername= =3D150amd64-default&build=3Ddf4f957ea181>


Both somewhat slow to load, however they do load for me.

Thanks
=C2=A0
beefy22 is showing hte same issues. Someth= ing is tying up the builders with builds hanging in "build-depends&quo= t; and "lib-depends" for very long periods of time, as long as ov= er 4 hours) with high load averages. This has been going on for months and = seems to be getting worse. Chromium builds which used to take around 20 hou= rs are now running around 40 and I really don't think=C2=A0that this is= just code bloat. At times, about half of the active builds=C2=A0are in som= e state=C2=A0other than "build"; mostly one of the depends states= which should never take long as all dependencies are built before a build = is started.

A= fter some time passes (often hours) things clear up. Dependencies are sudde= nly loaded in a few minutes or less. Note that things do continue to comple= te, but the 10 minute averages are in single digits and frequently go to 0.= During this time, my connection to beefy22 will timeout and sometimes I ca= n't establish a connection.

I don't have any special access to the machines... j= ust via pkg-status, so I'm fairly limited in any analysis.
<= /div>

Just before I started my l= ast message,=C2=A0 all was well, but I last=C2=A0saw were a dozen builds in= one of the=C2=A0"depends" states and "impulse" at 74 w= hen I lost my connection. I can reconnect to the beefy22 status page, but i= t has not updated for several=C2=A0minutes.
--
Kevin Oberman, Part time kid herder and retired Network EngineerE-mail: rkoberma= n@gmail.com
PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB= 39EF1B055683
--000000000000c08eda064beccca4--