Date: Fri, 6 Aug 2021 17:39:17 -0700 From: Mark Millard via freebsd-toolchain <freebsd-toolchain@freebsd.org> To: "linimon@portsmon.org linimon@portsmon.org" <linimon@portsmon.org> Cc: FreeBSD Toolchain <freebsd-toolchain@freebsd.org> Subject: Re: FYI: Over 2 hours spent (so far) in /usr/local/share/poudriere/processonelog.sh executing bzgrep commands for www/chromium failure Message-ID: <E29E175F-4809-4B38-9F70-FCDB97802DB5@yahoo.com> In-Reply-To: <270299435.875261.1628285088620@privateemail.com> References: <00FAF070-71DE-40B6-8D63-C1E74FFCD08C.ref@yahoo.com> <00FAF070-71DE-40B6-8D63-C1E74FFCD08C@yahoo.com> <270299435.875261.1628285088620@privateemail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2021-Aug-6, at 14:24, linimon@portsmon.org linimon@portsmon.org = <linimon@portsmon.org> wrote: >=20 >> On 08/04/2021 4:43 PM Mark Millard via freebsd-toolchain = <freebsd-toolchain@freebsd.org> wrote: >> it has spent over 2 hours (so far) doing: >>=20 >> /usr/local/share/poudriere/processonelog.sh activity ( = /usr/bin/bzgrep 's ) of/for: >=20 > fwiw processonelog.sh is not part of the toolchain per se. It is a = script that is part > of poudriere. poudriere clearly is not part of the buildworld buildkernel support. Since poudriere is used as infrastructure for building ports into packages, including use on one or more aarch64 systems for such, I took a guess that toolchain was more appropriate than ports for what list to use. Sorry if I guessed wrong. > I have maintained it in various incarnations over the years. > One of the things that I have tried to do is minimize the time the = script takes in a run > over the errorlogs for an entire build. These generally complete in = around a minute or so. Until attempting www/chromium builds, I'd never noticed the time frame but in this case I was monitoring with top, as well as waiting for pouriere's next output after it reported having saved the wrkdir. In the end it was about 2 hr 50 min. Part of this is likely just the size of a log file from a www/chromium build attempt that gets near the end. The actual log file is not compressed and can be looked at with just less or more or the like. Such is how I normally take a preliminary look at build failures. > You will need to figure out how big your logfile is, and compare it to = the size of a > successful logfile, to figure out what is going on. >=20 Unfortunately the system involved is down/not-accessible for about a day or two. So I'll not be able to check details for a while. The www/chromium build is huge and my original report lists the size of the log file in the "ls -LTld" output: 153683812 . So, between 146 MiByte and 147 MiByte. A successful build's log file would be larger. The build only made it into the 94% range for what ninja reports about steps executed. (Unfortunately, rebuilds use different alternative orderings that satisfy the partial ordering criteria. So build to build varies. I've also seen 77% for the same failure on the same file.) For reference: the (root) file system is on an Optane 480 GiByte in the PCIe 3 slot and is a ZFS file system. The system has 64 GiByte of RAM. The swap partitioning is on the same media. For reference: Using USE_TMPFS=3D"yes" lead to the tmpfs for the wrkdir involved being over 99000 MiByte (so somewhat under 100 GiByte). The system configuration was sufficient to not run out of swap/paging space, 64 GiByte of RAM + 120 GiByte of swap-space. I've since adjusted to just have USE_TMPFS=3D"data" now that I know of an example of such a huge wrkdir. (lang/rust ends up with a small wrkdir peak size, by comparison to www/chromium anyway.) Until I again get access to the system, that is about all I can report. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E29E175F-4809-4B38-9F70-FCDB97802DB5>