Date: Sat, 16 Jul 2022 21:52:55 +0000 From: bugzilla-noreply@freebsd.org To: toolchain@FreeBSD.org Subject: [Bug 265254] lang/gcc11: build gets stuck Message-ID: <bug-265254-29464-Zrrocjar57@https.bugs.freebsd.org/bugzilla/> In-Reply-To: <bug-265254-29464@https.bugs.freebsd.org/bugzilla/> References: <bug-265254-29464@https.bugs.freebsd.org/bugzilla/>
next in thread | previous in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D265254 Mark Millard <marklmi26-fbsd@yahoo.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |marklmi26-fbsd@yahoo.com --- Comment #4 from Mark Millard <marklmi26-fbsd@yahoo.com> --- It was suggested "I suggest you to disable LTO_BOOTSTRAP" and I'd 2nd that. But this note goes in a different direction, not so limited to lang/gcc* despite it being a driving example. I see 7 separate /usr/local/bin/ld commands running at once, each being an example of LTO processing. But I generalize below to principles beyond just lang/gcc* . lang/gcc* are far from being the only resource intensive port builds. The FreeBSD build servers avoid this for port building by using not only ALLOW_MAKE_JOBS=3Dyes in /usr/local/etc/poudriere.conf but also in the *make.conf that it uses in/for poudriere: # Build ALLOW_MAKE_JOBS_PACKAGES with 2 jobs MAKE_JOBS_NUMBER=3D2 That makes the maximum make jobs 2*(number of builders). Without the MAKE_JOBS_NUMBER (or analogous) but using ALLOW_MAKE_JOBS=3Dyes the number of make jobs is: (number of freebsd cpus)*(number of builders) And if the number of builders is set to the number freebsd cpus, the number of make jobs can the square of the freebsd cpu count. FreeBSD build servers are configured to avoid such. In fact they commonly configure for having: 2*(number of builders) < (number of freebsd cpus) You can configure your build context in such ways as well. It would make these sorts of issues less likely. FreeBSD is unlikely to make things such that you do not need to deal with such configuration of your environment. I'll note that . . . However, various ports have steps that create more processes and/or use multi-threaded processes beyond this, outside of the poudriere's control and even outside make's control. There is no global control on the load average that will/can occur during port builds, even if one sets up only one builder and avoids ALLOW_MAKE_JOBS=3Dyes . This is part of why the FreeBSD build servers use MAKE_JOBS_NUMBER=3D2 and: 2*(number of builders) < (number of freebsd cpus) There are also questions relative to memory use, such as if you are using USE_TMPFS=3Ddata or USE_TMPFS=3Dno vs. some other value. All the other values lead to various examples of huge amounts of memory-space use by tmpfs for some ports. (There is a poudriere mechanism for making only specific ports avoid tmpfs use but I do not use it.) Personally I use USE_TMPFS=3Ddata and ALLOW_MAKE_JOBS=3Dyes and as many builders as freebsd cpus (for example, 32 or 16), no use of the likes of MAKE_JOBS_NUMBER. But I configured everything for high load average activity, including having a mean of 4 GiBytes of RAM per freebsd cpu on the machines used for such builds. I also have SwapSpace =3D=3D 3.6*RAM or so and fairly fast storage space media (including for swapping/paging to/from swap space). No spinning rust. My builds have not produced examples of "this kills the machine" for how I've configured things. On very rare occasions I test doing "bulk -a -c". I also have some system tailoring: in /boot/loader.conf : vm.pageout_oom_seq=3D120 in /etc/sysctl.conf : vm.swap_enabled=3D0 in /etc/sysctl.conf : vm.swap_idle_enabled=3D0 I'll not get into why here. I do some experimentation with an environment with a mean of only 2GiBytes of RAM per FreeBSD cpu, but only 4 such cpus. This has not had problems yet for my poudriere use. --=20 You are receiving this mail because: You are the assignee for the bug.=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-265254-29464-Zrrocjar57>