Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 16 Aug 2023 16:30:46 +0900
From:      Tomoaki AOKI <junchoon@dec.sakura.ne.jp>
To:        freebsd-current@freebsd.org
Subject:   Re: www/chromium will not build on a host w/ 8 CPU and 16G mem
Message-ID:  <20230816163046.081be1181df386fae55065bc@dec.sakura.ne.jp>
In-Reply-To: <2227F902-847E-4E50-B48A-B012CE51D96D@yahoo.com>
References:  <2227F902-847E-4E50-B48A-B012CE51D96D.ref@yahoo.com> <2227F902-847E-4E50-B48A-B012CE51D96D@yahoo.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 15 Aug 2023 23:19:37 -0700
Mark Millard <marklmi@yahoo.com> wrote:

> Matthias Apitz <guru_at_unixarea.de> wrote on
> Date: Wed, 16 Aug 2023 04:31:38 UTC :
> 
> > I have built ~2200 ports successful on my build server, which is a
> > Dell R210 with 8x 3.30GHz CPU and 15.8 GB memory:
> > 
> > Aug 11 19:03:21 jet kernel: CPU: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz (3292.74-MHz K8-class CPU)
> > Aug 11 19:03:21 jet kernel: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
> > Aug 11 19:03:21 jet kernel: avail memory = 16582250496 (15814 MB)
> > 
> > I have set swap to 4GB + 10GB + 10GB:
> > 
> > # swapctl -lh
> > Device: Bytes Used:
> > /dev/da0p3 4.0G 1.5G
> > /dev/md9 10G 1.5G
> > /dev/md10 10G 1.5G
> 
> Are those /dev/md* vnode backed? If not, what type are they?
> 
> FYI: On 2017-Feb-13, at 7:20 PM, Konstantin Belousov
> <kostikbel at gmail.com> wrote on the freebsd-arm list:
> 
> QUOTE
> . . .
> 
> swapfile write requires the write request to come through the filesystem
> write path, which might require the filesystem to allocate more memory
> and read some data. E.g. it is known that any ZFS write request
> allocates memory, and that write request on large UFS file might require
> allocating and reading an indirect block buffer to find the block number
> of the written block, if the indirect block was not yet read.
> 
> As result, swapfile swapping is more prone to the trivial and unavoidable
> deadlocks where the pagedaemon thread, which produces free memory, needs
> more free memory to make a progress.  Swap write on the raw partition over
> simple partitioning scheme directly over HBA are usually safe, while e.g.
> zfs over geli over umass is the worst construction.
> END QUOTE
> 
> Use of swap partitions is to be recommended to minimize the chance of
> paging related deadlocks.
> 
> You have not reported on how much swap was in use shortly before the
> "was killed: a thread waited too long to allocate a page" notice.
> (After the notice is too late of a time frame to be of interest.)
> 
> > and poudriere does use ZFS.
> > 
> > Building the last outstanding port www/chromium in single job mode
> > fails reproducible with a kernel message:
> > 
> > Aug 16 06:14:04 jet kernel: pid 48725 (ninja), jid 3, uid 65534, was killed:
> > a thread waited too long to allocate a page
> 
> You have not been explicit about how you have managed
> RAM+SWAP tradeoffs in /usr/local/etc/poudriere.conf
> settings.
> 
> In /usr/local/etc/poudriere.conf : what are you using
> as the USE_TMPFS value:
> 
> # Use tmpfs(5)
> # This can be a space-separated list of options:
> # wrkdir    - Use tmpfs(5) for port building WRKDIRPREFIX
> # data      - Use tmpfs(5) for poudriere cache/temp build data
> # localbase - Use tmpfs(5) for LOCALBASE (installing ports for packaging/testing)
> # all       - Run the entire build in memory, including builder jails.
> # yes       - Enables tmpfs(5) for wrkdir and data
> # no        - Disable use of tmpfs(5)
> 
> Only 2 of the options keep the tmpfs use small:
> 
> data
> no
> 
> For example, building rust can use 10 GiBytes+ of just tmpfs
> space that competes for RAM+SWAP.
> 
> The last personal I helped that was getting "a thread waited
> too long to allocate a page" it turned out that changing the
> USE_TMPFS= assignment to one of the 2 options eliminated the
> issue. (No guarnatee of such here.) [There were 2 USE_TMPFS=
> assignments, the 2nd "winning" --but the first had been
> intended.]
> 
> Are you defining ALLOW_MAKE_JOBS= ? If yes, are you using
> /usr/local/etc/poudriere.d/make.conf (or the like) to assign
> MAKE_JOBS_NUMBER= (or the like)? If yes, to what? Last I knew
> the official port -> package builders use MAKE_JOBS_NUMBER=2
> for their activity with ALLOW_MAKE_JOBS in use.
> 
> A similar point goes for if you are assigning
> ALLOW_MAKE_JOBS_PACKAGES= . Are you?
> 
> These can contribute to RAM+SWAP usage and higher load
> averages.
> 
> > What could I do in the port's options or kernel values?
> 
> I've not actually gone in either of those directions above.
> But nothing at this point says if I've happened to cover
> your case vs. not.
> 
> ===
> Mark Millard
> marklmi at yahoo.com

Just a FYI:
www/chromium built fine (stable/13, though) with poudriere.
RAM is 32GB and 64GB single dedicated swap partition on SSD.
Using ports-mgmt/poudriere-devel.
No ccache used.

In /usr/local/etc/poudriere.conf,
USE_TMPFS=yes
ALLOW_MAKE_JOBS=yes

The line below is kept commented out.
#TMPFS_LIMIT=8


On main (booted from different physical drive on the same PC), I don't
use poudriere[-devel], but www/chromium (previous version) built fine
using ports-mgmt/pkg_replace. The size of /tmp (swap-backed tmpfs) is
not limited (means at maximum of 64GB).

-- 
Tomoaki AOKI    <junchoon@dec.sakura.ne.jp>



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20230816163046.081be1181df386fae55065bc>