Date: Tue, 8 Jan 2019 01:53:52 -0800 From: Mark Millard <marklmi@yahoo.com> To: ticso@cicely.de, Victor <vdemart@gmail.com> Cc: freebsd-arm <freebsd-arm@freebsd.org> Subject: Re: make buildworld error while compiling 12.0-releng for RPI2 Message-ID: <67DDC863-63C5-451F-ACFC-336A3304EF4D@yahoo.com> In-Reply-To: <20190107214116.GB30472@cicely7.cicely.de> References: <897D5769-58A0-4349-9FA4-1C4551F3A5A6@gmail.com> <E1ggUXq-00CNCk-As@smtp.hs-karlsruhe.de> <290CC356-180C-405C-B1D1-1CD5F96BA905@gmail.com> <20190107214116.GB30472@cicely7.cicely.de>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2019-Jan-7, at 13:41, Bernd Walter <ticso at cicely7.cicely.de> =
wrote:
> On Mon, Jan 07, 2019 at 02:59:22PM +0100, Victor wrote:
>> Thank you, Ralf.
>> As a matter of fact I issued=20
>>=20
>> make buildword
>>=20
>> NOT=20
>>=20
>> make -j4 buildworld
>>=20
>> The compilation problem is still there.
>=20
> Anyways - this is an out of memory problem.
> dmesg will show you why the process was killed.
> You have to add more swap space.
> Unfortunately compilers get more and more CPU and memory hungry.
> These days I use memory sticks capable of high random load as =
temporarys
> swap space or NFS space - later mostly for obj and src, not as swap so =
much.
> For me SanDisk Extreme Go 3.1 and Ultra USB 3.0 worked great.
> That said, I havn't build FreeBSD12 on any ARM myself yet and this is
> only a general experience.
>=20
I'm just adding some notes about interpreting the console/dmesg =
messages,
as one of them is misleading (a misnomer) --plus other related notes.
There was a prolonged list exchange and investigation of buildworld
on rpi2's and rpi3's during 2018, ending in 2018-Sept. The below was
learned during that.
If one is really out of swap space there should be messages like:
Aug 5 17:54:01 sentinel kernel: swap_pager_getswapspace(32): failed
But if one does not see such messages yet does see messages like:
Oct 30 05:15:31 xindi kernel: pid 68935 (rustc), uid 65534, was killed:=20=
out of swap space
then one is likely not actually out of swap space and adding swap space
will likely not prevent the process kills.
Unfortunately, the wording of that last message is a misnomer for what
drives the kills: it is actually driven by being unable to gain more
free memory but FreeBSD will not swap-out processes that stay runnable
(or are running), only ones that are waiting. Even a single process
that stays runnable and keeps lots of RAM in the active category can
lead to kills when swap is unused or little used. So the kill-behavior
is very workload dependent.
If only the later type of message is showing up, having something like:
# more /etc/sysctl.conf=20
vm.pageout_oom_seq=3D120
with a figure bigger than the default 12, possibly even something like
1024 or 2048 or even more may delay the kills long enough to avoid the
kills because the processes in question then completes first. (The units
are not time but a count, but larger counts take more time before the
process kills start.)
By contrast, if you are seeing the swap_pager_getswapspace messages, you
have to add swap space. There is kern.maxswzone figure that is =
associated
with swap size and with possible warning pairs when swap space is added,
for example:
warning: total configured swap (524288 pages) exceeds maximum =
recommended amount (405460 pages).
warning: increase kern.maxswzone or reduce amount of swap.
(An rpi2 V1.1 (armv7) and an rpi3 (aarch64) with the same amount of RAM
get rather different figures at the right.)
There was in stable/12 -r341716 :
MFC r341375 :
Allow to create swap zone larger than v_page_count / 2.
and in stable/11 -r341718 :
MFC r341375 :
Allow to create swap zone larger than v_page_count / 2.
(So release/12.0.0 or release/11.2.0 do not allow such an
increase.)
There is is a kernel memory tradeoff structure to increase in
kern.maxswzone being larger as I understand. Quoting "man 8 loader"
(but the "eight times" is system/architecture specific and will
likely be different):
kern.maxswzone
Limits the amount of KVM to be used to hold swap =
metadata,
which directly governs the maximum amount of swap the
system can support, at the rate of approximately 200 =
MB of
swap space per 1 MB of metadata. This value is =
specified
in bytes of KVA space. If no value is provided, the =
system
allocates enough memory to handle an amount of swap =
that
corresponds to eight times the amount of physical =
memory
present in the system.
Note that swap metadata can be fragmented, which means =
that
the system can run out of space before it reaches the
theoretical limit. Therefore, care should be taken to =
not
configure more swap than approximately half of the
theoretical maximum.
Running out of space for swap metadata can leave the =
system
in an unrecoverable state. Therefore, you should only
change this parameter if you need to greatly extend =
the KVM
reservation for other resources such as the buffer =
cache or
kern.ipc.nmbclusters. Modifies kernel option
VM_SWZONE_SIZE_MAX.
I also use (not just on small arm hardware):
LDFLAGS.lld+=3D -Wl,--no-threads
in my equivalents of /etc/make.conf . This avoids lld creating N+2 =
threads,
N being the the cpu count as FreeBSD counts cpus. This also seems to cut
down on memory use during links.
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?67DDC863-63C5-451F-ACFC-336A3304EF4D>