Date: Tue, 8 Nov 2022 14:50:19 +0800 From: Archimedes Gaviola <archimedes.gaviola@gmail.com> To: Mark Millard <marklmi@yahoo.com> Cc: freebsd-current <freebsd-current@freebsd.org> Subject: Re: 14.0-CURRENT failed to reclaim memory error in RPi 3B build Message-ID: <CAJFbk7GY6apxRaCY0nJwmiaYXX0EY5LOVqiphOMNuJhpRdmXJg@mail.gmail.com> In-Reply-To: <FE38E458-0694-460C-B103-817BCC1A9B6D@yahoo.com> References: <84376CC9-B991-4BF3-AF5F-0AA09CB28339.ref@yahoo.com> <84376CC9-B991-4BF3-AF5F-0AA09CB28339@yahoo.com> <CAJFbk7FfYPSe3eF00HgDdebW70HKp5zKR0JaChTVniUDPG2qxQ@mail.gmail.com> <CA350C16-3604-4D88-9C14-040A45F6F125@yahoo.com> <CAJFbk7Hxvr9gs7GnniWtJ-QEH4yjYbB9S-vKVLjipa8v5VHa%2Bw@mail.gmail.com> <CAJFbk7GQhvgQS-23jFfFf=ershgGNKByHR8ug08Df8merwkh6Q@mail.gmail.com> <A11AA52B-BDB0-43C7-BF85-3C252B276F17@yahoo.com> <CAJFbk7HnFTdzANtdERqvgX30hcHmufAZmrNvbfEWORkUJJ7_3w@mail.gmail.com> <CAJFbk7HJ1WA5Qc0LNEZpKgv78yiM0w7ex=gjgpjjTf3chhHhiQ@mail.gmail.com> <FE38E458-0694-460C-B103-817BCC1A9B6D@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--0000000000009f850205eceff5e7 Content-Type: text/plain; charset="UTF-8" On Tue, Nov 8, 2022 at 11:25 AM Mark Millard <marklmi@yahoo.com> wrote: > On Nov 7, 2022, at 18:40, Archimedes Gaviola <archimedes.gaviola@gmail.com> > wrote: > > > . . . > > > > Hi Mark, > > > > With this set of build commands now, > > > > # cd /usr/src; make -j3 KERNCONF=ARM TARGET_ARCH=aarch64 buildworld > kernel-toolchain buildkernel installworld installkernel distribution > DESTDIR=/home/freebsd/rpi3b > > > > in RPi 3B, I encountered the other OOM error which is the 'thread waited > too long to allocate a page'. This occurred from every build I conducted. > Though the first error on 'failed to reclaim memory' was never experienced > again. Below are the error logs. > > ... > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 256929, size: 4096 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 3628, size: 4096 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255839, size: 40960 > > pid 46153 (c++), jid 0, uid 0, was killed: a thread waited too long to > allocate a page > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255857, size: 28672 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 3634, size: 8192 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 256037, size: 4096 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255320, size: 8192 > > > > Perhaps some further tweaks are needed in the system so I set aside my > RPi 3B temporarily and switched over to my RPi 4B using the same microSD > card and USB flash drive (3.5 GB swap partition device) and the build > completed successfully. It took around 30 hours to complete. This RPi 4B > has 2GB RAM capacity while the RPi 3B has 1GB. From here, I'll continue > looking further for system tunables in RPi 3B which has lesser RAM capacity. > Hi Mark, > Given that you have added enough swap/paging > space to avoid needing more: > > # > # For plunty of swap/paging space (will not > # run out), avoid pageout delays leading to > # Out Of Memory killing of processes: > vm.pfault_oom_attempts=-1 > > With the above setting, if you did run out of > swap/paging space and needed more, deadlocks > would be possible as I understand. The above > disables getting that type of OOM kill > completely but, effectively, a deadlock is > sort of a form of less-controlled kill. > Okay, confirmed the existing value is 3. root@generic:~ # sysctl vm.pfault_oom_attempts vm.pfault_oom_attempts: 3 and is writable as tested. root@generic:~ # sysctl vm.pfault_oom_attempts=-1 vm.pfault_oom_attempts: 3 -> -1 > There is an alternative, but I've no clue how to > find what values to set for any specific context. > I just know the names and default values (as of > when I last checked such defaults): > > # > # For possibly insufficient swap/paging space > # (might run out), increase the pageout delay > # that leads to Out Of Memory killing of > # processes (showing defaults at the time): > #vm.pfault_oom_attempts= 3 > #vm.pfault_oom_wait= 10 > # (The multiplication is the total but there > # are other potential tradoffs in the factors > # multiplied, even for nearly the same total.) > > (Yes, one of those names is the same as was set > to -1 in the earlier suggestion above. -1 > disables making attempts and just waits as long > as it takes. That makes vm.pfault_oom_wait > irrelevant in that kind of context.) > > As for where the settings can be placed . . . > > # sysctl -T vm.pfault_oom_attempts > vm.pfault_oom_attempts: -1 > > # sysctl -T vm.pfault_oom_wait > vm.pfault_oom_wait: 10 > > (So /boot/loader.conf is appropriate: loader tunables.) > Okay noted this. > > # sysctl -W vm.pfault_oom_attempts > vm.pfault_oom_attempts: -1 > > # sysctl -W vm.pfault_oom_wait > vm.pfault_oom_wait: 10 > Checking these values... root@generic:~ # sysctl -T vm.pfault_oom_attempts vm.pfault_oom_attempts: -1 root@generic:~ # sysctl -T vm.pfault_oom_wait vm.pfault_oom_wait: 10 > (So /etc/sysctl.conf or the like is an alternative: > Also writable.) > Okay this is noted. Thanks and best regards, Archimedes --0000000000009f850205eceff5e7 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><div dir=3D"ltr"><div dir=3D"lt= r"><br></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail= _attr">On Tue, Nov 8, 2022 at 11:25 AM Mark Millard <<a href=3D"mailto:m= arklmi@yahoo.com" target=3D"_blank">marklmi@yahoo.com</a>> wrote:<br></d= iv><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bord= er-left:1px solid rgb(204,204,204);padding-left:1ex">On Nov 7, 2022, at 18:= 40, Archimedes Gaviola <<a href=3D"mailto:archimedes.gaviola@gmail.com" = target=3D"_blank">archimedes.gaviola@gmail.com</a>> wrote:<br> <br> > . . .<br> > <br> > Hi Mark,<br> > <br> > With this set of build commands now,<br> > <br> > # cd /usr/src; make -j3 KERNCONF=3DARM TARGET_ARCH=3Daarch64 buildworl= d kernel-toolchain buildkernel installworld installkernel distribution DEST= DIR=3D/home/freebsd/rpi3b<br> > <br> > in RPi 3B, I encountered the other OOM error which is the 'thread = waited too long to allocate a page'. This occurred from every build I c= onducted. Though the first error on 'failed to reclaim memory' was = never experienced again. Below are the error logs.<br> > ...<br> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 256929, size: 40= 96<br> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 3628, size: 4096= <br> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255839, size: 40= 960<br> > pid 46153 (c++), jid 0, uid 0, was killed: a thread waited too long to= allocate a page<br> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255857, size: 28= 672<br> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 3634, size: 8192= <br> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 256037, size: 40= 96<br> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255320, size: 81= 92<br> > <br> > Perhaps some further tweaks are needed in the system so I set aside my= RPi 3B temporarily and switched over to my RPi 4B using the same microSD c= ard and USB flash drive (3.5 GB swap partition device) and the build comple= ted successfully. It took around 30 hours to complete. This RPi 4B has 2GB = RAM capacity while the RPi 3B has 1GB. From here, I'll continue looking= further for system tunables in RPi 3B which has lesser RAM capacity.<br></= blockquote><div><br></div><div>Hi Mark,</div><div><br></div><blockquote cla= ss=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid = rgb(204,204,204);padding-left:1ex"> <br> Given that you have added enough swap/paging<br> space to avoid needing more:<br> <br> #<br> # For plunty of swap/paging space (will not<br> # run out), avoid pageout delays leading to<br> # Out Of Memory killing of processes:<br> vm.pfault_oom_attempts=3D-1<br> <br> With the above setting, if you did run out of<br> swap/paging space and needed more, deadlocks<br> would be possible as I understand. The above<br> disables getting that type of OOM kill<br> completely but, effectively, a deadlock is<br> sort of a form of less-controlled kill.<br></blockquote><div><br></div><div= >Okay, confirmed the existing value is 3.<br></div><div><br></div><div>root= @generic:~ # sysctl vm.pfault_oom_attempts<br>vm.pfault_oom_attempts: 3</di= v><div><br></div><div>and is writable as tested.</div><div><br></div><div>r= oot@generic:~ # sysctl vm.pfault_oom_attempts=3D-1<br>vm.pfault_oom_attempt= s: 3 -> -1</div><div>=C2=A0<br></div><blockquote class=3D"gmail_quote" s= tyle=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);pad= ding-left:1ex"> There is an alternative, but I've no clue how to<br> find what values to set for any specific context.<br> I just know the names and default values (as of<br> when I last checked such defaults):<br> <br> #<br> # For possibly insufficient swap/paging space<br> # (might run out), increase the pageout delay<br> # that leads to Out Of Memory killing of<br> # processes (showing defaults at the time):<br> #vm.pfault_oom_attempts=3D 3<br> #vm.pfault_oom_wait=3D 10<br> # (The multiplication is the total but there<br> # are other potential tradoffs in the factors<br> # multiplied, even for nearly the same total.)<br> <br> (Yes, one of those names is the same as was set<br> to -1 in the earlier suggestion above. -1<br> disables making attempts and just waits as long<br> as it takes. That makes vm.pfault_oom_wait<br> irrelevant in that kind of context.)<br> <br> As for where the settings can be placed . . .<br> <br> # sysctl -T vm.pfault_oom_attempts<br> vm.pfault_oom_attempts: -1<br> <br> # sysctl -T vm.pfault_oom_wait<br> vm.pfault_oom_wait: 10<br> <br> (So /boot/loader.conf is appropriate: loader tunables.)<br></blockquote><di= v><br></div><div>Okay noted this.</div><div>=C2=A0<br></div><blockquote cla= ss=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid = rgb(204,204,204);padding-left:1ex"> <br> # sysctl -W vm.pfault_oom_attempts<br> vm.pfault_oom_attempts: -1<br> <br> # sysctl -W vm.pfault_oom_wait<br> vm.pfault_oom_wait: 10<br></blockquote><div><br></div><div>Checking these v= alues...</div><div><br></div><div>root@generic:~ # sysctl -T vm.pfault_oom_= attempts<br>vm.pfault_oom_attempts: -1<br><br>root@generic:~ # sysctl -T vm= .pfault_oom_wait<br>vm.pfault_oom_wait: 10</div><div><br></div><div></div><= blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l= eft:1px solid rgb(204,204,204);padding-left:1ex"> <br> (So /etc/sysctl.conf or the like is an alternative:<br> Also writable.)<br></blockquote><div><br></div><div>Okay this is noted.</di= v></div><div class=3D"gmail_quote"><br></div><div class=3D"gmail_quote">Tha= nks and best regards,</div><div class=3D"gmail_quote">Archimedes<br></div><= /div> </div> --0000000000009f850205eceff5e7--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJFbk7GY6apxRaCY0nJwmiaYXX0EY5LOVqiphOMNuJhpRdmXJg>