Date: Wed, 25 Oct 2023 11:32:05 +0100 From: void <void@f-m.fm> To: ports@freebsd.org Subject: Re: Rust build huge memory consumption Message-ID: <ZTjupT9KKpgNnFxu@int21h> In-Reply-To: <09a4bfb8-ba61-4e05-b646-973104337f77@groumpf.org> References: <09a4bfb8-ba61-4e05-b646-973104337f77@groumpf.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi,
On Wed, Oct 25, 2023 at 08:38:54AM +0200, Xavier Humbert wrote:
>Hi,
>
>When building rust, all memory is eaten, leading to OOM killing almost
>everything, eventually failing, and leading to an unstable system.
>
>Is there a way to tell the compiler to avoid this behaviour ?
Are you building just from the ports tree or are you using poudriere?
if the former, try unsetting LTO in /etc/make.conf and if you have a
MAKE_JOBS statement in there, comment it out.
Try installing devel/sccache before your build. Ensure that
ccache is installed too. pay attention to the pkg-message after
install.
If you're using poudriere, the following settings are relevant:
1. make sure you have ports-mgmt/sccache-overlay installed.
2. in the make.conf for the poudriere jail, have these set:
SCCACHE_DIR=/var/cache/ccache/sccache
OVERLAYS+=/usr/local/share/sccache/overlay
WITH_CCACHE_BUILD=
CCACHE_DIR=/var/cache/ccache
3. also make sure devel/ccache-static (it *has* to be the -static variant)
is installed.
To use the sscache overlay, invoke the poudriere build with -O sccache
after installing as per the instructions.
On a rpi4b/8GB (therefore relatively resource constrained) running 14-stable
and poudriere-devel, it has the following settings (relevant to this discussion)
in poudriere.conf
USE_TMPFS=all
TMPFS_BLACKLIST="rust* llvm* libreo* firef* tex* ungoogled-ch* chromi* gcc* openjdk* electron* iridium"
MAX_FILES=4096
PARALLEL_JOBS=1
PREPARE_PARALLEL_JOBS=2
The following tunables are set in /etc/sysctl.conf
# filesystem
vm.pageout_oom_seq=120
vm.pfault_oom_attempts=-1
vm.pageout_update_period=0
vfs.zfs.arc.min=536870912
vfs.zfs.arc.max=1073741824
The rpi is clocked to 2.1GHz. It took 9hrs 3 mins to build rust-1.73.0
good luck,
--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ZTjupT9KKpgNnFxu>