Date: Sun, 8 Aug 2021 15:53:25 -0700 From: Mark Millard via freebsd-toolchain <freebsd-toolchain@freebsd.org> To: linimon@portsmon.org Cc: FreeBSD Toolchain <freebsd-toolchain@freebsd.org> Subject: Re: FYI: Over 2 hours spent (so far) in /usr/local/share/poudriere/processonelog.sh executing bzgrep commands for www/chromium failure Message-ID: <74DE7FBF-31E4-4EAB-B923-1AA3A150DA6F@yahoo.com> In-Reply-To: <E29E175F-4809-4B38-9F70-FCDB97802DB5@yahoo.com> References: <00FAF070-71DE-40B6-8D63-C1E74FFCD08C.ref@yahoo.com> <00FAF070-71DE-40B6-8D63-C1E74FFCD08C@yahoo.com> <270299435.875261.1628285088620@privateemail.com> <E29E175F-4809-4B38-9F70-FCDB97802DB5@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2021-Aug-6, at 17:39, Mark Millard via freebsd-toolchain =
<freebsd-toolchain at freebsd.org> wrote:
> On 2021-Aug-6, at 14:24, linimon@portsmon.org linimon@portsmon.org =
<linimon at portsmon.org> wrote:
>>=20
>>> On 08/04/2021 4:43 PM Mark Millard via freebsd-toolchain =
<freebsd-toolchain at freebsd.org> wrote:
>>> it has spent over 2 hours (so far) doing:
>>>=20
>>> /usr/local/share/poudriere/processonelog.sh activity ( =
/usr/bin/bzgrep 's ) of/for:
>>=20
>> fwiw processonelog.sh is not part of the toolchain per se. It is a =
script that is part
>> of poudriere.
>=20
> poudriere clearly is not part of the buildworld buildkernel
> support.
>=20
> Since poudriere is used as infrastructure for building
> ports into packages, including use on one or more
> aarch64 systems for such, I took a guess that toolchain
> was more appropriate than ports for what list to use.
> Sorry if I guessed wrong.
>=20
>> I have maintained it in various incarnations over the years.
>=20
>> One of the things that I have tried to do is minimize the time the =
script takes in a run
>> over the errorlogs for an entire build. These generally complete in =
around a minute or so.
>=20
> Until attempting www/chromium builds, I'd never noticed
> the time frame but in this case I was monitoring with top,
> as well as waiting for pouriere's next output after it
> reported having saved the wrkdir.
>=20
> In the end it was about 2 hr 50 min. Part of this is
> likely just the size of a log file from a www/chromium
> build attempt that gets near the end.
>=20
> The actual log file is not compressed and can be looked at
> with just less or more or the like. Such is how I normally
> take a preliminary look at build failures.
>=20
>> You will need to figure out how big your logfile is, and compare it =
to the size of a
>> successful logfile, to figure out what is going on.
>>=20
>=20
> Unfortunately the system involved is down/not-accessible
> for about a day or two. So I'll not be able to check
> details for a while.
>=20
> The www/chromium build is huge and my original report
> lists the size of the log file in the "ls -LTld" output:
> 153683812 . So, between 146 MiByte and 147 MiByte.
>=20
> A successful build's log file would be larger. The build
> only made it into the 94% range for what ninja reports
> about steps executed. (Unfortunately, rebuilds use
> different alternative orderings that satisfy the partial
> ordering criteria. So build to build varies. I've also
> seen 77% for the same failure on the same file.)
>=20
> For reference: the (root) file system is on an Optane
> 480 GiByte in the PCIe 3 slot and is a ZFS file system.
> The system has 64 GiByte of RAM. The swap partitioning
> is on the same media.
>=20
> For reference:
> Using USE_TMPFS=3D"yes" lead to the tmpfs for the wrkdir
> involved being over 99000 MiByte (so somewhat under
> 100 GiByte). The system configuration was sufficient to
> not run out of swap/paging space, 64 GiByte of RAM + 120
> GiByte of swap-space. I've since adjusted to just have
> USE_TMPFS=3D"data" now that I know of an example of such a
> huge wrkdir. (lang/rust ends up with a small wrkdir
> peak size, by comparison to www/chromium anyway.)
>=20
>=20
> Until I again get access to the system, that is about
> all I can report.
Here is a "time -l" for one of the bzgrep's that is involved,
the first:
# time -l bzgrep -qE "(Error: mtree file ./etc/mtree/BSD.local.dist. is =
missing|error in pkg_delete|filesystem was touched prior to .make =
install|list of extra files and directories|listof files present before =
this port was installed|list of filesystem changes from before and =
after|Error: Files or directories left over|Error: Filesystem touched =
during build)" =
/usr/local/poudriere/data/logs/bulk/main-CA72-default/2021-08-03_22h54m48s=
/logs/chromium-91.0.4472.114.log
806.63 real 806.63 user 0.19 sys
2476 maximum resident set size
20 average shared memory size
8 average unshared data size
128 average unshared stack size
496 page reclaims
0 page faults
0 swaps
0 block input operations
0 block output operations
37521 messages sent
4691 messages received
0 signals received
7138 voluntary context switches
8536 involuntary context switches
Using grep directly is not much different
for the times:
# time -l grep -qE "(Error: mtree file ./etc/mtree/BSD.local.dist. is =
missing|error in pkg_delete|filesystem was touched prior to .make =
install|list of extra files and directories|list of files present before =
this port was installed|list of filesystem changes from before and =
after|Error: Files or directories left over|Error: Filesystem touched =
during build)" =
/usr/local/poudriere/data/logs/bulk/main-CA72-default/2021-08-03_22h54m48s=
/logs/chromium-91.0.4472.114.log
804.02 real 803.86 user 0.18 sys
2476 maximum resident set size
19 average shared memory size
7 average unshared data size
127 average unshared stack size
158 page reclaims
0 page faults
0 swaps
579 block input operations
0 block output operations
0 messages sent
0 messages received
0 signals received
2432 voluntary context switches
9089 involuntary context switches
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?74DE7FBF-31E4-4EAB-B923-1AA3A150DA6F>