Date: Tue, 8 Nov 2022 14:50:19 +0800 From: Archimedes Gaviola <archimedes.gaviola@gmail.com> To: Mark Millard <marklmi@yahoo.com> Cc: freebsd-current <freebsd-current@freebsd.org> Subject: Re: 14.0-CURRENT failed to reclaim memory error in RPi 3B build Message-ID: <CAJFbk7GY6apxRaCY0nJwmiaYXX0EY5LOVqiphOMNuJhpRdmXJg@mail.gmail.com> In-Reply-To: <FE38E458-0694-460C-B103-817BCC1A9B6D@yahoo.com> References: <84376CC9-B991-4BF3-AF5F-0AA09CB28339.ref@yahoo.com> <84376CC9-B991-4BF3-AF5F-0AA09CB28339@yahoo.com> <CAJFbk7FfYPSe3eF00HgDdebW70HKp5zKR0JaChTVniUDPG2qxQ@mail.gmail.com> <CA350C16-3604-4D88-9C14-040A45F6F125@yahoo.com> <CAJFbk7Hxvr9gs7GnniWtJ-QEH4yjYbB9S-vKVLjipa8v5VHa%2Bw@mail.gmail.com> <CAJFbk7GQhvgQS-23jFfFf=ershgGNKByHR8ug08Df8merwkh6Q@mail.gmail.com> <A11AA52B-BDB0-43C7-BF85-3C252B276F17@yahoo.com> <CAJFbk7HnFTdzANtdERqvgX30hcHmufAZmrNvbfEWORkUJJ7_3w@mail.gmail.com> <CAJFbk7HJ1WA5Qc0LNEZpKgv78yiM0w7ex=gjgpjjTf3chhHhiQ@mail.gmail.com> <FE38E458-0694-460C-B103-817BCC1A9B6D@yahoo.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] On Tue, Nov 8, 2022 at 11:25 AM Mark Millard <marklmi@yahoo.com> wrote: > On Nov 7, 2022, at 18:40, Archimedes Gaviola <archimedes.gaviola@gmail.com> > wrote: > > > . . . > > > > Hi Mark, > > > > With this set of build commands now, > > > > # cd /usr/src; make -j3 KERNCONF=ARM TARGET_ARCH=aarch64 buildworld > kernel-toolchain buildkernel installworld installkernel distribution > DESTDIR=/home/freebsd/rpi3b > > > > in RPi 3B, I encountered the other OOM error which is the 'thread waited > too long to allocate a page'. This occurred from every build I conducted. > Though the first error on 'failed to reclaim memory' was never experienced > again. Below are the error logs. > > ... > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 256929, size: 4096 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 3628, size: 4096 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255839, size: 40960 > > pid 46153 (c++), jid 0, uid 0, was killed: a thread waited too long to > allocate a page > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255857, size: 28672 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 3634, size: 8192 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 256037, size: 4096 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255320, size: 8192 > > > > Perhaps some further tweaks are needed in the system so I set aside my > RPi 3B temporarily and switched over to my RPi 4B using the same microSD > card and USB flash drive (3.5 GB swap partition device) and the build > completed successfully. It took around 30 hours to complete. This RPi 4B > has 2GB RAM capacity while the RPi 3B has 1GB. From here, I'll continue > looking further for system tunables in RPi 3B which has lesser RAM capacity. > Hi Mark, > Given that you have added enough swap/paging > space to avoid needing more: > > # > # For plunty of swap/paging space (will not > # run out), avoid pageout delays leading to > # Out Of Memory killing of processes: > vm.pfault_oom_attempts=-1 > > With the above setting, if you did run out of > swap/paging space and needed more, deadlocks > would be possible as I understand. The above > disables getting that type of OOM kill > completely but, effectively, a deadlock is > sort of a form of less-controlled kill. > Okay, confirmed the existing value is 3. root@generic:~ # sysctl vm.pfault_oom_attempts vm.pfault_oom_attempts: 3 and is writable as tested. root@generic:~ # sysctl vm.pfault_oom_attempts=-1 vm.pfault_oom_attempts: 3 -> -1 > There is an alternative, but I've no clue how to > find what values to set for any specific context. > I just know the names and default values (as of > when I last checked such defaults): > > # > # For possibly insufficient swap/paging space > # (might run out), increase the pageout delay > # that leads to Out Of Memory killing of > # processes (showing defaults at the time): > #vm.pfault_oom_attempts= 3 > #vm.pfault_oom_wait= 10 > # (The multiplication is the total but there > # are other potential tradoffs in the factors > # multiplied, even for nearly the same total.) > > (Yes, one of those names is the same as was set > to -1 in the earlier suggestion above. -1 > disables making attempts and just waits as long > as it takes. That makes vm.pfault_oom_wait > irrelevant in that kind of context.) > > As for where the settings can be placed . . . > > # sysctl -T vm.pfault_oom_attempts > vm.pfault_oom_attempts: -1 > > # sysctl -T vm.pfault_oom_wait > vm.pfault_oom_wait: 10 > > (So /boot/loader.conf is appropriate: loader tunables.) > Okay noted this. > > # sysctl -W vm.pfault_oom_attempts > vm.pfault_oom_attempts: -1 > > # sysctl -W vm.pfault_oom_wait > vm.pfault_oom_wait: 10 > Checking these values... root@generic:~ # sysctl -T vm.pfault_oom_attempts vm.pfault_oom_attempts: -1 root@generic:~ # sysctl -T vm.pfault_oom_wait vm.pfault_oom_wait: 10 > (So /etc/sysctl.conf or the like is an alternative: > Also writable.) > Okay this is noted. Thanks and best regards, Archimedes [-- Attachment #2 --] <div dir="ltr"><div dir="ltr"><br></div><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Nov 8, 2022 at 11:25 AM Mark Millard <<a href="mailto:marklmi@yahoo.com" target="_blank">marklmi@yahoo.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Nov 7, 2022, at 18:40, Archimedes Gaviola <<a href="mailto:archimedes.gaviola@gmail.com" target="_blank">archimedes.gaviola@gmail.com</a>> wrote:<br> <br> > . . .<br> > <br> > Hi Mark,<br> > <br> > With this set of build commands now,<br> > <br> > # cd /usr/src; make -j3 KERNCONF=ARM TARGET_ARCH=aarch64 buildworld kernel-toolchain buildkernel installworld installkernel distribution DESTDIR=/home/freebsd/rpi3b<br> > <br> > in RPi 3B, I encountered the other OOM error which is the 'thread waited too long to allocate a page'. This occurred from every build I conducted. Though the first error on 'failed to reclaim memory' was never experienced again. Below are the error logs.<br> > ...<br> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 256929, size: 4096<br> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 3628, size: 4096<br> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255839, size: 40960<br> > pid 46153 (c++), jid 0, uid 0, was killed: a thread waited too long to allocate a page<br> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255857, size: 28672<br> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 3634, size: 8192<br> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 256037, size: 4096<br> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255320, size: 8192<br> > <br> > Perhaps some further tweaks are needed in the system so I set aside my RPi 3B temporarily and switched over to my RPi 4B using the same microSD card and USB flash drive (3.5 GB swap partition device) and the build completed successfully. It took around 30 hours to complete. This RPi 4B has 2GB RAM capacity while the RPi 3B has 1GB. From here, I'll continue looking further for system tunables in RPi 3B which has lesser RAM capacity.<br></blockquote><div><br></div><div>Hi Mark,</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <br> Given that you have added enough swap/paging<br> space to avoid needing more:<br> <br> #<br> # For plunty of swap/paging space (will not<br> # run out), avoid pageout delays leading to<br> # Out Of Memory killing of processes:<br> vm.pfault_oom_attempts=-1<br> <br> With the above setting, if you did run out of<br> swap/paging space and needed more, deadlocks<br> would be possible as I understand. The above<br> disables getting that type of OOM kill<br> completely but, effectively, a deadlock is<br> sort of a form of less-controlled kill.<br></blockquote><div><br></div><div>Okay, confirmed the existing value is 3.<br></div><div><br></div><div>root@generic:~ # sysctl vm.pfault_oom_attempts<br>vm.pfault_oom_attempts: 3</div><div><br></div><div>and is writable as tested.</div><div><br></div><div>root@generic:~ # sysctl vm.pfault_oom_attempts=-1<br>vm.pfault_oom_attempts: 3 -> -1</div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> There is an alternative, but I've no clue how to<br> find what values to set for any specific context.<br> I just know the names and default values (as of<br> when I last checked such defaults):<br> <br> #<br> # For possibly insufficient swap/paging space<br> # (might run out), increase the pageout delay<br> # that leads to Out Of Memory killing of<br> # processes (showing defaults at the time):<br> #vm.pfault_oom_attempts= 3<br> #vm.pfault_oom_wait= 10<br> # (The multiplication is the total but there<br> # are other potential tradoffs in the factors<br> # multiplied, even for nearly the same total.)<br> <br> (Yes, one of those names is the same as was set<br> to -1 in the earlier suggestion above. -1<br> disables making attempts and just waits as long<br> as it takes. That makes vm.pfault_oom_wait<br> irrelevant in that kind of context.)<br> <br> As for where the settings can be placed . . .<br> <br> # sysctl -T vm.pfault_oom_attempts<br> vm.pfault_oom_attempts: -1<br> <br> # sysctl -T vm.pfault_oom_wait<br> vm.pfault_oom_wait: 10<br> <br> (So /boot/loader.conf is appropriate: loader tunables.)<br></blockquote><div><br></div><div>Okay noted this.</div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <br> # sysctl -W vm.pfault_oom_attempts<br> vm.pfault_oom_attempts: -1<br> <br> # sysctl -W vm.pfault_oom_wait<br> vm.pfault_oom_wait: 10<br></blockquote><div><br></div><div>Checking these values...</div><div><br></div><div>root@generic:~ # sysctl -T vm.pfault_oom_attempts<br>vm.pfault_oom_attempts: -1<br><br>root@generic:~ # sysctl -T vm.pfault_oom_wait<br>vm.pfault_oom_wait: 10</div><div><br></div><div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <br> (So /etc/sysctl.conf or the like is an alternative:<br> Also writable.)<br></blockquote><div><br></div><div>Okay this is noted.</div></div><div class="gmail_quote"><br></div><div class="gmail_quote">Thanks and best regards,</div><div class="gmail_quote">Archimedes<br></div></div> </div>help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJFbk7GY6apxRaCY0nJwmiaYXX0EY5LOVqiphOMNuJhpRdmXJg>