Date: Wed, 14 Dec 2022 00:41:13 -0800 From: Mark Millard <marklmi@yahoo.com> To: jhs@berklix.com, freebsd-arm <freebsd-arm@freebsd.org> Subject: Re: armv6/armv7/aarch64 vs. FreeBSD images (retitled from an arch reply, moving to freebsd-arm list) Message-ID: <E9A113DC-95B8-44B3-AC27-96C9E542087E@yahoo.com> In-Reply-To: <D9D1D72C-79FB-4F86-831B-53A6CA014CB0@yahoo.com> References: <D9D1D72C-79FB-4F86-831B-53A6CA014CB0@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Dec 13, 2022, at 22:45, Mark Millard <marklmi@yahoo.com> wrote: > Julian H. Stacey <jhs_at_berklix.com> wrote on > Date: Wed, 14 Dec 2022 02:22:10 UTC : > >> . . . > >> I previously had some kind of a 3B, & now have a "Pi 3 Model B+", >> but have not got beyond booting images, starting to customise /etc/ >> & then it crashes yet again, needing yet another dd, & repeat, >> Being certain one is using the right image would be nice. > > This does not sound normal at all. I expect something more > specific to your context is involved. > >> My notes: http://www.berklix.org/~jhs/pi/#images > > I'll look at the notes separately, possibly not tonight. > Note: I presume use of an aarch64 FreeBSD, not armv7 or armv6. Otherwise some things would be different below, such as via tighter constraints on swap space sizes for 1 GiByte of RAM. Quoting http://www.berklix.org/~jhs/pi/#images : QUOTE pid 17 (sh), jid @, uid 8, was killed: failed to reclain menory END QUOTE This is the FreeBSD kernel complaining about the configuration not well matching the RPi3B+ workload. In essence, it was unable to achieve it targeted minimum amount of free RAM in the sort of time frame (really: effort) it is configured for. Depending on what you do, the FreeBSD defaults do not work well for 1 GiByte of RAM. Swap space alone is insufficient because FreeBSD does not swap out processes that stay runnable. Just one process that stays runnable using a working set that is as large as the fits RAM for overall operation will lead to such "failed to reclaim memory" kills. But, if you are getting this, you will almost certainly need a non-trivial swap space anyway. I have a starting point to recommend, configuring some settings. As I've no detailed clue for your context, I'll just provide the general description. A) I recommend a swap space something like shown in the below (from gpart show output): => 40 1953525088 da0 GPT (932G) 40 532480 1 efi (260M) 532520 2008 - free - (1.0M) 534528 7340032 2 freebsd-swap (3.5G) . . . 67643392 1740636160 5 freebsd-ufs (830G) 1808279552 145245576 - free - (69G) This size (3.5 GiBytes or so) is somewhat below were FreeBSD starts to complain about potential mistuning from a large swap space, given the 1 GiByte of RAM. (I boot the same boot media on a variety of machines and have other swap partitions to match up with RAM sizes. But I omitted showing them.) It is easy to have things like buildworld or building ports end up with individual processes that are temporarily bigger than the 1 GiByte RAM. Getting multiple cores going can also lead to not fitting and needing to page. I'll note that I normally use USB3 NVMe media that also works with USB2 ports. My alternate is USB3 SSD media that works with USB2 ports. I avoid spinning rust and microsd cards. This limits what I can usefully comment on for some aspects of configuration related to the alternatives. B) /boot/loader.conf content: # # Delay when persistent low free RAM leads to # Out Of Memory killing of processes: vm.pageout_oom_seq=120 # # For plunty of swap/paging space (will not # run out), avoid pageout delays leading to # Out Of Memory killing of processes: vm.pfault_oom_attempts=-1 # # For possibly insufficient swap/paging space # (might run out), increase the pageout delay # that leads to Out Of Memory killing of # processes (showing defaults at the time): #vm.pfault_oom_attempts= 3 #vm.pfault_oom_wait= 10 # (The multiplication is the total but there # are other potential tradoffs in the factors # multiplied, even for nearly the same total.) If use of vm.pfault_oom_attempts=-1 is going to be inappropriate, I do not have background with figuring out a good combination of settings for vm.pfault_oom_attempts and vm.pfault_oom_wait . I'll note that vm.pageout_oom_seq is not a time --more like how many insufficient tries to reclaim RAM happen in sequence before an OOM kill is started (effort). 120 is 10 times the default. While nothing disables such criteria, larger figures can be used if needed. (I've never had to but others have.) C) /etc/sysctl.conf content: # # Together this pair avoids swapping out the process kernel stacks. # This avoids one way for processes for interacting with the system # from ending up being hung-up. vm.swap_enabled=0 vm.swap_idle_enabled=0 D) I strictly avoid having tmpfs complete for RAM in this kind of context. tmpfs use just makes avoiding "failed to reclaim memory" more difficult to avoid. (As various folks have run into despite having vastly more RAM than an RPi3B+.) So my /usr/local/etc/poudriere.conf has: USE_TMPFS=no There are examples, like building rust, where anything but "no" or "data" leads to huge 10 GiByte+ tmpfs spaces for poudriere's build activity. Not a good match to an RPi3B+ . That is it for the recommendations of a starting point configuration. With such measures, I've been able to have poudriere with -j4 but also using ALLOW_MAKE_JOBS= without using the likes of MAKE_JOBS_NUMBER limiting it. (So the load average could be around 16 a fair amount of the time but still not get "failed to reclaim memory" kills.) Note: I'm not claiming above that -j4 is the best setting to use from, say, a elapsed time point of view for my poudriere bulk activity. I'll not volunteer anyone specific, but there are folks that have reported using RPi3B+'s on the lists, mostly via asking some question related to system operation. So you might be able to ask for information from anyone that really has an RPi3B+ in use, possibly describing the context that gives rise to the question(s). === Mark Millard marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E9A113DC-95B8-44B3-AC27-96C9E542087E>