From nobody Mon Oct 31 02:47:30 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4N0yHk5gwMz4gM7n for ; Mon, 31 Oct 2022 02:47:46 +0000 (UTC) (envelope-from archimedes.gaviola@gmail.com) Received: from mail-yb1-xb31.google.com (mail-yb1-xb31.google.com [IPv6:2607:f8b0:4864:20::b31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4N0yHj5XDnz3WrV for ; Mon, 31 Oct 2022 02:47:45 +0000 (UTC) (envelope-from archimedes.gaviola@gmail.com) Received: by mail-yb1-xb31.google.com with SMTP id t186so12271979yba.12 for ; Sun, 30 Oct 2022 19:47:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=p7l8Efk32oBX1iIXeoXC2IWvtD4FZPnN75TdimjE25E=; b=f7wlfwu4Z8vc3sKehG3QjA1FdriZAvyDFZzdZr4ncpLyl68BmLM1HX773T38nSZj5f 85E4uc9U0iCE31xW4KUiwIu0nwjbgAuX5lWc9HSaszLPkk+z9oaEJaTa/ks38ZQT1n2A XEWk138YUeayuabZf3TA+TzR6BZpMxFyVFJ/PBjEGj25MfdpwUOhJL6lyFoE5IDvPn9G lEDEDZ1XGjshrKIHTMS3XGNKYrr58Rb+24LzizORRVqHssfOoHhxHFUzAw3IJr3p40yj 1D4kKFpERi0nsJy9ogRd66YfHeEzX/5eVpsND+3nDPklyCCQjDfpuUCNbxPrbXcGbfJN Dxmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=p7l8Efk32oBX1iIXeoXC2IWvtD4FZPnN75TdimjE25E=; b=BDGeE1ECGrs6YvZXzn9934KVel7RV0pcOFG2TrZINgKzHr8zh912+NdjIZncBisq5q Xv2uMWSqumLvLPZIiSlbdXbGLuHBYaqFnKUP1pHPqRLdIJ1cnO+BZrTpwmco2KemGWGi +Q+cZmgaCkD1bPYhXwSIFB0xw3xsFtIxZOKpxFsyUJqBjnTUhkXHpuK9IFv6FcYBp3YT PIxLg5nzJyVnRSon49Heelh30/nNsJEK3x3J++oc3u2xQYE2tK9i4Y4RjX4obJx5zy7+ ytQ8uOVDxMvFHelPing0Iov5qQ7rUuRIPaH39px59A4+/tQ31qRN5Medu7its/XGr+X+ hvUQ== X-Gm-Message-State: ACrzQf2s4F+n/S4fyzjePCTJ4bIaWAjK7+/RKX68o9hf5XuMsflAwbS6 MeMA7DGnu/zdbKzr4Pual1cmFtO0lMCotMMvuEuPpIQaFBA= X-Google-Smtp-Source: AMsMyM5t1VJXVCIdGoo+xr1zRw8W+PeXalNN+E2//zsh5gWiqO/nldTyxhWCycCXygWz8maEIVvxJMKnWvg1Q6QAKwU= X-Received: by 2002:a25:6412:0:b0:6cc:6507:bfae with SMTP id y18-20020a256412000000b006cc6507bfaemr7008539ybb.537.1667184464366; Sun, 30 Oct 2022 19:47:44 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <84376CC9-B991-4BF3-AF5F-0AA09CB28339.ref@yahoo.com> <84376CC9-B991-4BF3-AF5F-0AA09CB28339@yahoo.com> In-Reply-To: <84376CC9-B991-4BF3-AF5F-0AA09CB28339@yahoo.com> From: Archimedes Gaviola Date: Mon, 31 Oct 2022 10:47:30 +0800 Message-ID: Subject: Re: 14.0-CURRENT failed to reclaim memory error in RPi 3B build To: Mark Millard Cc: freebsd-current Content-Type: multipart/alternative; boundary="000000000000c615c905ec4ba1e6" X-Rspamd-Queue-Id: 4N0yHj5XDnz3WrV X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=f7wlfwu4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of archimedes.gaviola@gmail.com designates 2607:f8b0:4864:20::b31 as permitted sender) smtp.mailfrom=archimedes.gaviola@gmail.com X-Spamd-Result: default: False [-3.55 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-0.997]; NEURAL_HAM_MEDIUM(-0.55)[-0.551]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::b31:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_ALL(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N --000000000000c615c905ec4ba1e6 Content-Type: text/plain; charset="UTF-8" On Mon, Oct 31, 2022 at 1:29 AM Mark Millard wrote: > Archimedes Gaviola wrote on > Date: Sun, 30 Oct 2022 13:41:52 UTC : > > > I am building a kernel and world in 14.0-CURRENT > > > https://download.freebsd.org/ftp/snapshots/arm64/aarch64/ISO-IMAGES/14.0/FreeBSD-14.0-CURRENT-arm64-aarch64-RPI-20221027-769b884e2e2-258837.img.xz > > with Raspberry Pi 3B (ARM kernel config file and in default system > > configurations) and compilation breaks due to "failed to reclaim memory" > > error as found in the dmesg. > > > > pid 91224 (llvm-tblgen), jid 0, uid 0, was killed: failed to reclaim > memory > > pid 91131 (make), jid 0, uid 0, was killed: failed to reclaim memory > > > > Here's the set of the build commands I invoked. > > > > root@generic# cd /usr/src ; make KERNCONF=ARM TARGET_ARCH=aarch64 > > buildkernel buildworld installkernel installworld distribution > > DESTDIR=/home/freebsd/rpi3b > > > > . . . > > > > Any thoughts? As I don't have any idea about VM pageout. > Hi Mark, > > Multiple configuration things from what I use: > > I use a swap partition (not a swap file!) to give the system > someplace to put copies of inactive memory pages (paging): > > # swapinfo > Device 1K-blocks Used Avail Capacity > /dev/gpt/Rock64swp2 3670016 0 3670016 0% > Oh I see, there's no swap partition in the default installation. root@generic:~ # swapinfo Device 1K-blocks Used Avail Capacity root@generic:~ # top -S last pid: 92429; load averages: 0.00, 0.00, 0.00 up 1+12:49:11 23:41:10 52 processes: 2 running, 48 sleeping, 2 waiting CPU: 0.0% user, 0.0% nice, 0.6% system, 0.0% interrupt, 99.4% idle Mem: 2000K Active, 626M Inact, 223M Wired, 97M Buf, 20M Free Let me try to create a swap partition. Let me mount a spare USB flash drive for swap as during the installation all my microSD card storage was allocated in the root filesystem with growfs. > > where gpart show -p lists it as (a gpt context, not MBR): > > 534528 7340032 da0p2 freebsd-swap (3.5G) > > and gpart show -pl lists it as: > > 534528 7340032 da0p2 Rock64swp2 (3.5G) > Okay noted on GPT not MBR method with gpart. By the way, what's the proper allocation size of swap in FreeBSD? This RPi 3B has 1GB of RAM (~947 MB), do I need to set twice the capacity of this physical RAM? > > (Note: swap file usage is subject to deadlock conditions > avoided by use of swap partitions.) > This is noted. > > I use a serial console & ssh session only context to avoid > having sizable competition for RAM. > > I avoid using tmpfs because it competes for RAM use. > > I use the likes of ( in, say, /boot/loader/conf ): > > # > # Delay when persistent low free RAM leads to > # Out Of Memory killing of processes: > vm.pageout_oom_seq=120 > > This delays potential "killed: failed to reclaim memory" kills, > possibly long enough to reach a state where sufficient memory is > reclaimed. > Alright this is well noted too. > > I'll note that the status "killed: failed to reclaim memory" does > not require that swap be used much at all. Sustained low free RAM > from just one process that always stays runnable and has a > sufficiently large active set of pages can be sufficient to end up > with such kills. Having swap allows for inactive pages to get out > of the way, which can help. > > I use the likes of ( in, say, /etc/ssyctl.conf ): > > # > # Together this pair avoids swapping out the process kernel stacks. > # This avoids processes for interacting with the system from being > # hung-up. > vm.swap_enabled=0 > vm.swap_idle_enabled=0 > > This allows paging to the swap space but disallows moving > kernel thread stacks to the swap space. Otherwise the > processes used to interact with the RPi3 can become > non-runnable, preventing such interactions. > Okay this too is well noted. > > I have NVMe or SSD based USB media, not microsd cards nor > spinning rust. (I use just bootcode.bin and timeout files > on microsd media for the RPi3B. Even the rest of the RPi* > firmware is on the USB media, as well as u-boot.bin .) > > My usage of such a configuration struture for building > software (world, kernel, ports) applies to all the > systems I do such with, including ones with a lot more > resources, including a lot more RAM. > Thanks for these inputs, noted on these things! I haven't tried NVMe and SSD media in my RPi 3B. So, they are far more superior as compared to microSD cards when it comes to building software? Thanks and best regards, Archimedes --000000000000c615c905ec4ba1e6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Mon, Oct 31, 2022 at 1:29 AM Mark = Millard <marklmi@yahoo.com> = wrote:
Archimede= s Gaviola <archimedes.gaviola_at_gmail.com>wrote on<= br> Date: Sun, 30 Oct 2022 13:41:52 UTC :

> I am building a kernel and world in 14.0-CURRENT
> https://download.freebsd= .org/ftp/snapshots/arm64/aarch64/ISO-IMAGES/14.0/FreeBSD-14.0-CURRENT-arm64= -aarch64-RPI-20221027-769b884e2e2-258837.img.xz
> with Raspberry Pi 3B=C2=A0 (ARM kernel config file and in default syst= em
> configurations) and compilation breaks due to "failed to reclaim = memory"
> error as found in the dmesg.
>
> pid 91224 (llvm-tblgen), jid 0, uid 0, was killed: failed to reclaim m= emory
> pid 91131 (make), jid 0, uid 0, was killed: failed to reclaim memory >
> Here's the set of the build commands I invoked.
>
> root@generic# cd /usr/src ; make KERNCONF=3DARM TARGET_ARCH=3Daarch64<= br> > buildkernel buildworld installkernel installworld distribution
> DESTDIR=3D/home/freebsd/rpi3b
>
> . . .
>
> Any thoughts? As I don't have any idea about VM pageout.

Hi Mark,
=C2=A0

Multiple configuration things from what I use:

I use a swap partition (not a swap file!) to give the system
someplace to put copies of inactive memory pages (paging):

# swapinfo
Device=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1K-blocks=C2=A0 =C2=A0 =C2=A0Used= =C2=A0 =C2=A0 Avail Capacity
/dev/gpt/Rock64swp2=C2=A0 =C2=A03670016=C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 = 3670016=C2=A0 =C2=A0 =C2=A00%

Oh I see,= there's no swap partition in the default installation.

<= /div>
root@generic:~ # swapinfo
Device =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A01K-blocks =C2=A0 =C2=A0 Used =C2=A0 =C2=A0Avail Capacity

root@= generic:~ # top -S
last pid: 92429; =C2=A0load averages: =C2=A00.00, =C2= =A00.00, =C2=A00.00 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 u= p 1+12:49:11 =C2=A023:41:10
52 processes: =C2=A02 running, 48 sleeping, = 2 waiting
CPU: =C2=A00.0% user, =C2=A00.0% nice, =C2=A00.6% system, =C2= =A00.0% interrupt, 99.4% idle
Mem: 2000K Active, 626M Inact, 223M Wired,= 97M Buf, 20M Free

Let me try to create a swap par= tition. Let me mount a spare USB flash drive for swap as during the install= ation all my microSD card storage was allocated in the root filesystem with= growfs.=C2=A0
=C2=A0

where gpart show -p lists it as (a gpt context, not MBR):

=C2=A0 =C2=A0 =C2=A0 534528=C2=A0 =C2=A0 =C2=A07340032=C2=A0 da0p2=C2=A0 fr= eebsd-swap=C2=A0 (3.5G)

and gpart show -pl lists it as:

=C2=A0 =C2=A0 =C2=A0 534528=C2=A0 =C2=A0 =C2=A07340032=C2=A0 da0p2=C2=A0 Ro= ck64swp2=C2=A0 (3.5G)

Okay noted on GPT= not MBR method with gpart. By the way, what's the proper allocation si= ze of swap in FreeBSD? This RPi 3B has 1GB of RAM (~947 MB), do I need to s= et twice the capacity of this physical RAM?
=C2=A0
<= blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l= eft:1px solid rgb(204,204,204);padding-left:1ex">
(Note: swap file usage is subject to deadlock conditions
avoided by use of swap partitions.)

Thi= s is noted.
=C2=A0

I use a serial console & ssh session only context to avoid
having sizable competition for RAM.

I avoid using tmpfs because it competes for RAM use.

I use the likes of ( in, say, /boot/loader/conf ):

#
# Delay when persistent low free RAM leads to
# Out Of Memory killing of processes:
vm.pageout_oom_seq=3D120

This delays potential "killed: failed to reclaim memory" kills, possibly long enough to reach a state where sufficient memory is
reclaimed.

Alright this is well noted t= oo.
=C2=A0

I'll note that the status "killed: failed to reclaim memory" = does
not require that swap be used much at all. Sustained low free RAM
from just one process that always stays runnable and has a
sufficiently large active set of pages can be sufficient to end up
with such kills. Having swap allows for inactive pages to get out
of the way, which can help.

I use the likes of ( in, say, /etc/ssyctl.conf ):

#
# Together this pair avoids swapping out the process kernel stacks.
# This avoids processes for interacting with the system from being
# hung-up.
vm.swap_enabled=3D0
vm.swap_idle_enabled=3D0

This allows paging to the swap space but disallows moving
kernel thread stacks to the swap space. Otherwise the
processes used to interact with the RPi3 can become
non-runnable, preventing such interactions.

=
Okay this too is well noted.
=C2=A0

I have NVMe or SSD based USB media, not microsd cards nor
spinning rust. (I use just bootcode.bin and timeout files
on microsd media for the RPi3B. Even the rest of the RPi*
firmware is on the USB media, as well as u-boot.bin .)

My usage of such a configuration struture for building
software (world, kernel, ports) applies to all the
systems I do such with, including ones with a lot more
resources, including a lot more RAM.

Th= anks for these inputs, noted on these things! I haven't tried NVMe and = SSD media in my RPi 3B. So, they are far more superior as compared to micro= SD cards when it comes to building software?

Thanks and best regards,
Archimedes
--000000000000c615c905ec4ba1e6--