Date: Tue, 8 Nov 2022 10:40:09 +0800 From: Archimedes Gaviola <archimedes.gaviola@gmail.com> To: Mark Millard <marklmi@yahoo.com> Cc: freebsd-current <freebsd-current@freebsd.org> Subject: Re: 14.0-CURRENT failed to reclaim memory error in RPi 3B build Message-ID: <CAJFbk7HJ1WA5Qc0LNEZpKgv78yiM0w7ex=gjgpjjTf3chhHhiQ@mail.gmail.com> In-Reply-To: <CAJFbk7HnFTdzANtdERqvgX30hcHmufAZmrNvbfEWORkUJJ7_3w@mail.gmail.com> References: <84376CC9-B991-4BF3-AF5F-0AA09CB28339.ref@yahoo.com> <84376CC9-B991-4BF3-AF5F-0AA09CB28339@yahoo.com> <CAJFbk7FfYPSe3eF00HgDdebW70HKp5zKR0JaChTVniUDPG2qxQ@mail.gmail.com> <CA350C16-3604-4D88-9C14-040A45F6F125@yahoo.com> <CAJFbk7Hxvr9gs7GnniWtJ-QEH4yjYbB9S-vKVLjipa8v5VHa%2Bw@mail.gmail.com> <CAJFbk7GQhvgQS-23jFfFf=ershgGNKByHR8ug08Df8merwkh6Q@mail.gmail.com> <A11AA52B-BDB0-43C7-BF85-3C252B276F17@yahoo.com> <CAJFbk7HnFTdzANtdERqvgX30hcHmufAZmrNvbfEWORkUJJ7_3w@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000ee420f05ecec76a2 Content-Type: text/plain; charset="UTF-8" On Sat, Nov 5, 2022 at 5:28 PM Archimedes Gaviola < archimedes.gaviola@gmail.com> wrote: > > > > On Thu, Nov 3, 2022 at 7:52 AM Mark Millard <marklmi@yahoo.com> wrote: > >> On 2022-Nov-2, at 14:09, Archimedes Gaviola <archimedes.gaviola@gmail.com> >> wrote: >> >> > On Mon, Oct 31, 2022 at 1:47 PM Archimedes Gaviola < >> archimedes.gaviola@gmail.com> wrote: >> > >> > . . . >> > >> > . . . >> > >> > >> > Hi Mark, >> > >> > Just an update, as kernel and world compilation is ongoing with my >> RPi3B system (with swap partition) is doing so far, so good. It already >> surpassed the tough part that breaks the compilation process here. >> > ... >> > >> > llvm-tblgen -gen-asm-matcher -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenAsmMatcher.inc.d -o RISCVGenAsmMatcher.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> > llvm-tblgen -gen-asm-writer -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenAsmWriter.inc.d -o RISCVGenAsmWriter.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> > llvm-tblgen -gen-callingconv -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenCallingConv.inc.d -o RISCVGenCallingConv.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> > llvm-tblgen -gen-compress-inst-emitter -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenCompressInstEmitter.inc.d -o RISCVGenCompressInstEmitter.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> > llvm-tblgen -gen-dag-isel -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenDAGISel.inc.d -o RISCVGenDAGISel.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> > llvm-tblgen -gen-disassembler -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenDisassemblerTables.inc.d -o RISCVGenDisassemblerTables.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> > llvm-tblgen -gen-global-isel -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenGlobalISel.inc.d -o RISCVGenGlobalISel.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> > llvm-tblgen -gen-instr-info -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenInstrInfo.inc.d -o RISCVGenInstrInfo.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> > llvm-tblgen -gen-emitter -I /usr/src/contrib/llvm-project/llvm/include >> -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenMCCodeEmitter.inc.d -o RISCVGenMCCodeEmitter.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> > llvm-tblgen -gen-pseudo-lowering -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenMCPseudoLowering.inc.d -o RISCVGenMCPseudoLowering.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> > llvm-tblgen -gen-register-bank -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenRegisterBank.inc.d -o RISCVGenRegisterBank.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> > llvm-tblgen -gen-register-info -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenRegisterInfo.inc.d -o RISCVGenRegisterInfo.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> > llvm-tblgen -gen-searchable-tables -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenSearchableTables.inc.d -o RISCVGenSearchableTables.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> > llvm-tblgen -gen-subtarget -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenSubtargetInfo.inc.d -o RISCVGenSubtargetInfo.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> > llvm-tblgen -gen-searchable-tables -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenSystemOperands.inc.d -o RISCVGenSystemOperands.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> > >> > Any thoughts why this part is quite a challenge when it comes to memory >> usage? The other architectures do not possess such behavior... just curious. >> > > Hi Mark, > > Sorry for the late response, I got fully occupied at work for the past few > days due to deliverables. Thanks for your feedback and further inputs! > > >> I've not done any monitoring of buildworld buildkernel build >> activity (RAM use, memory space use, swap partition use over >> time) on RPi3B class hardware in a very long time. >> > > It's alright, it so happened that I just observed that behavior on that > particular part as it requires more memory than other architectures while > compiling. The additional 3.5G swap partition really helps on that part > that's why I was so happy that the compilation continued and never broke. > Your input of having 3.5G swap allocation is very effective. > > Even on systems that I have monitored in more recent times, >> what I usually monitor tends to be builds with -jN (such as >> -j4 fora 4-hardware-thread system). (I once did have an >> example of -j3 taking less time than -j4 on a RPi4B. >> > > Wow, this is interesting this -jN. Let me explore this as well. I usually > build kernel the old way but recently since I have to include building the > world then I need to use the new way. > > >> Basically, the memory subsystem can be saturated without all >> the cores being in use. The extra interference made things >> take longer.) >> > > Oh I see so it's the reason. > > >> You had listed that you were using the likes of: >> >> # cd /usr/src ; make KERNCONF=ARM TARGET_ARCH=aarch64 \ >> buildkernel buildworld installkernel installworld distribution \ >> DESTDIR=/home/freebsd/rpi3b >> >> I'll note that the standard order of the first 2 is: >> >> buildworld buildkernel >> >> This is because buildworld builds some software that >> buildkernel does not build for itself but does use. >> > > Okay this is noted, thanks for clarifying and correcting me, I really > appreciate it. I'll reflect on the proper build sequence for much > efficiency. > > >> There is a kernel-toolchain target for avoiding the >> need to do a full buildworld just to buildkernel , so: >> >> kernel-toolchain buildkernel >> >> is an expected sequence. >> > > Okay I'll take note of this too. > > >> I do not know how long a from-scratch buildworld >> buildkernel without a -jN takes on a RPi3B these >> days. If I remember right, for -jN with 1<N, I last >> saw claims about such they were somewhere in the >> range 36hr..48hr. > > > There's an ongoing build at the moment, it's already taking 41 hours since > I started it. I took another build when I came back home from the office. > > >> But I'm unsure of the specific N >> that was in use. Nor do I know the storage media >> type(s) involved, for example. I do not remember >> any reports for -j1. >> > > I'll try this with RPi 3B. The current build that I have will be my > baseline. > > Use of the likes of: vm.pageout_oom_seq=120 >> was essential to such -jN usage on a RPi3B as N >> gets larger. Of course, swap partition use for >> paging was also essential. >> > > Wow, that's great! I have this vm.pageout_oom_seq=120 configured in my > system now based on your previous inputs. > > Likewise use of: >> >> vm.swap_enabled=0 >> vm.swap_idle_enabled=0 >> >> can be important to not losing communication >> with the RPi3B. Those last 2 are not tunable >> but are writable: >> >> # sysctl -aT | grep swap_ >> # sysctl -aW | grep swap_ >> vm.swap_enabled: 0 >> . . . >> vm.swap_idle_enabled: 0 >> . . . >> >> (This means that they have fewer places where >> assignments can be made. For example, the >> loader can not make the assignments.) >> >> By contrast, vm.pageout_oom_seq is both >> writable and tunable: >> >> # sysctl -aW | grep oom >> . . . >> vm.pageout_oom_seq: 120 >> . . . >> >> # sysctl -aT | grep oom >> . . . >> vm.pageout_oom_seq: 120 >> . . . >> >> (So even the loader can make such assignments.) >> > > Yes, I have these two sysctl parameters configured in the system. Thanks > for the details and further inputs. > > >> I'll note that I've no interest in using arm hardware >> to build for other types of hardware. So I normally >> have the targeting of support for building for other >> architectures disabled when I build on aarch64 or >> armv7. (Basically, a less complete clang/clang++ >> related toolchain ends up being built.) >> > > Ah okay, so you mean to say that you disable these other architectures by > declaring and accomplishing it in the /etc/src.conf? > > I'll provide an update here once the build is done knowing how long it > takes to finish. > Hi Mark, With this set of build commands now, # cd /usr/src; make -j3 KERNCONF=ARM TARGET_ARCH=aarch64 buildworld kernel-toolchain buildkernel installworld installkernel distribution DESTDIR=/home/freebsd/rpi3b in RPi 3B, I encountered the other OOM error which is the 'thread waited too long to allocate a page'. This occurred from every build I conducted. Though the first error on 'failed to reclaim memory' was never experienced again. Below are the error logs. ... swap_pager: indefinite wait buffer: bufobj: 0, blkno: 256929, size: 4096 swap_pager: indefinite wait buffer: bufobj: 0, blkno: 3628, size: 4096 swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255839, size: 40960 pid 46153 (c++), jid 0, uid 0, was killed: a thread waited too long to allocate a page swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255857, size: 28672 swap_pager: indefinite wait buffer: bufobj: 0, blkno: 3634, size: 8192 swap_pager: indefinite wait buffer: bufobj: 0, blkno: 256037, size: 4096 swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255320, size: 8192 Perhaps some further tweaks are needed in the system so I set aside my RPi 3B temporarily and switched over to my RPi 4B using the same microSD card and USB flash drive (3.5 GB swap partition device) and the build completed successfully. It took around 30 hours to complete. This RPi 4B has 2GB RAM capacity while the RPi 3B has 1GB. From here, I'll continue looking further for system tunables in RPi 3B which has lesser RAM capacity. Thanks and best regards, Archimedes --000000000000ee420f05ecec76a2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Sat, Nov 5, 2022 at 5:28 PM Archim= edes Gaviola <<a href=3D"mailto:archimedes.gaviola@gmail.com">archimedes= .gaviola@gmail.com</a>> wrote:<br></div><blockquote class=3D"gmail_quote= " style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);= padding-left:1ex"><div dir=3D"ltr"><div dir=3D"ltr"><br></div><div dir=3D"l= tr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote"><div dir=3D"l= tr" class=3D"gmail_attr">On Thu, Nov 3, 2022 at 7:52 AM Mark Millard <<a= href=3D"mailto:marklmi@yahoo.com" target=3D"_blank">marklmi@yahoo.com</a>&= gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0= px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 20= 22-Nov-2, at 14:09, Archimedes Gaviola <<a href=3D"mailto:archimedes.gav= iola@gmail.com" target=3D"_blank">archimedes.gaviola@gmail.com</a>> wrot= e:<br> <br> > On Mon, Oct 31, 2022 at 1:47 PM Archimedes Gaviola <<a href=3D"mail= to:archimedes.gaviola@gmail.com" target=3D"_blank">archimedes.gaviola@gmail= .com</a>> wrote:<br> > <br> > . . .<br> > <br> > . . .<br> > <br> > <br> > Hi Mark,<br> > <br> > Just an update, as kernel and world compilation is ongoing with my RPi= 3B system (with swap partition) is doing so far, so good. It already surpas= sed the tough part that breaks the compilation process here.<br> > ...<br> > <br> > llvm-tblgen -gen-asm-matcher=C2=A0 -I /usr/src/contrib/llvm-project/ll= vm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV=C2=A0 -d = RISCVGenAsmMatcher.inc.d -o RISCVGenAsmMatcher.inc=C2=A0 /usr/src/contrib/l= lvm-project/llvm/lib/Target/RISCV/RISCV.td<br> > llvm-tblgen -gen-asm-writer=C2=A0 -I /usr/src/contrib/llvm-project/llv= m/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV=C2=A0 -d R= ISCVGenAsmWriter.inc.d -o RISCVGenAsmWriter.inc=C2=A0 /usr/src/contrib/llvm= -project/llvm/lib/Target/RISCV/RISCV.td<br> > llvm-tblgen -gen-callingconv=C2=A0 -I /usr/src/contrib/llvm-project/ll= vm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV=C2=A0 -d = RISCVGenCallingConv.inc.d -o RISCVGenCallingConv.inc=C2=A0 /usr/src/contrib= /llvm-project/llvm/lib/Target/RISCV/RISCV.td<br> > llvm-tblgen -gen-compress-inst-emitter=C2=A0 -I /usr/src/contrib/llvm-= project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV= =C2=A0 -d RISCVGenCompressInstEmitter.inc.d -o RISCVGenCompressInstEmitter.= inc=C2=A0 /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td<br> > llvm-tblgen -gen-dag-isel=C2=A0 -I /usr/src/contrib/llvm-project/llvm/= include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV=C2=A0 -d RIS= CVGenDAGISel.inc.d -o RISCVGenDAGISel.inc=C2=A0 /usr/src/contrib/llvm-proje= ct/llvm/lib/Target/RISCV/RISCV.td<br> > llvm-tblgen -gen-disassembler=C2=A0 -I /usr/src/contrib/llvm-project/l= lvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV=C2=A0 -d= RISCVGenDisassemblerTables.inc.d -o RISCVGenDisassemblerTables.inc=C2=A0 /= usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td<br> > llvm-tblgen -gen-global-isel=C2=A0 -I /usr/src/contrib/llvm-project/ll= vm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV=C2=A0 -d = RISCVGenGlobalISel.inc.d -o RISCVGenGlobalISel.inc=C2=A0 /usr/src/contrib/l= lvm-project/llvm/lib/Target/RISCV/RISCV.td<br> > llvm-tblgen -gen-instr-info=C2=A0 -I /usr/src/contrib/llvm-project/llv= m/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV=C2=A0 -d R= ISCVGenInstrInfo.inc.d -o RISCVGenInstrInfo.inc=C2=A0 /usr/src/contrib/llvm= -project/llvm/lib/Target/RISCV/RISCV.td<br> > llvm-tblgen -gen-emitter=C2=A0 -I /usr/src/contrib/llvm-project/llvm/i= nclude -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV=C2=A0 -d RISC= VGenMCCodeEmitter.inc.d -o RISCVGenMCCodeEmitter.inc=C2=A0 /usr/src/contrib= /llvm-project/llvm/lib/Target/RISCV/RISCV.td<br> > llvm-tblgen -gen-pseudo-lowering=C2=A0 -I /usr/src/contrib/llvm-projec= t/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV=C2=A0= -d RISCVGenMCPseudoLowering.inc.d -o RISCVGenMCPseudoLowering.inc=C2=A0 /u= sr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td<br> > llvm-tblgen -gen-register-bank=C2=A0 -I /usr/src/contrib/llvm-project/= llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV=C2=A0 -= d RISCVGenRegisterBank.inc.d -o RISCVGenRegisterBank.inc=C2=A0 /usr/src/con= trib/llvm-project/llvm/lib/Target/RISCV/RISCV.td<br> > llvm-tblgen -gen-register-info=C2=A0 -I /usr/src/contrib/llvm-project/= llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV=C2=A0 -= d RISCVGenRegisterInfo.inc.d -o RISCVGenRegisterInfo.inc=C2=A0 /usr/src/con= trib/llvm-project/llvm/lib/Target/RISCV/RISCV.td<br> > llvm-tblgen -gen-searchable-tables=C2=A0 -I /usr/src/contrib/llvm-proj= ect/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV=C2= =A0 -d RISCVGenSearchableTables.inc.d -o RISCVGenSearchableTables.inc=C2=A0= /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td<br> > llvm-tblgen -gen-subtarget=C2=A0 -I /usr/src/contrib/llvm-project/llvm= /include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV=C2=A0 -d RI= SCVGenSubtargetInfo.inc.d -o RISCVGenSubtargetInfo.inc=C2=A0 /usr/src/contr= ib/llvm-project/llvm/lib/Target/RISCV/RISCV.td<br> > llvm-tblgen -gen-searchable-tables=C2=A0 -I /usr/src/contrib/llvm-proj= ect/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV=C2= =A0 -d RISCVGenSystemOperands.inc.d -o RISCVGenSystemOperands.inc=C2=A0 /us= r/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td<br> > <br> > Any thoughts why this part is quite a challenge when it comes to memor= y usage? The other architectures do not possess such behavior... just curio= us.<br></blockquote><div><br></div><div>Hi Mark,</div><div><br></div><div>S= orry for the late response, I got fully occupied at work for the past few d= ays due to deliverables. Thanks for your feedback and further inputs!</div>= <div>=C2=A0<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px = 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> I've not done any monitoring of buildworld buildkernel build<br> activity (RAM use, memory space use, swap partition use over<br> time) on RPi3B class hardware in a very long time.<br></blockquote><div><br= ></div><div>It's alright, it so happened that I just observed that beha= vior on that particular part as it requires more memory than other architec= tures while compiling. The additional 3.5G swap partition really helps on t= hat part that's why I was so happy that the compilation continued and n= ever broke. Your input of having 3.5G swap allocation is very effective.</d= iv><div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px= 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> Even on systems that I have monitored in more recent times,<br> what I usually monitor tends to be builds with -jN (such as<br> -j4 fora 4-hardware-thread system). (I once did have an<br> example of -j3 taking less time than -j4 on a RPi4B.<br></blockquote><div><= br></div><div>Wow, this is interesting this -jN. Let me explore this as wel= l. I usually build kernel the old way but recently since I have to include = building the world then I need to use the new way. <br></div><div>=C2=A0<br= ></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;= border-left:1px solid rgb(204,204,204);padding-left:1ex"> Basically, the memory subsystem can be saturated without all<br> the cores being in use. The extra interference made things<br> take longer.)<br></blockquote><div><br></div><div>Oh I see so it's the = reason.</div><div>=C2=A0<br></div><blockquote class=3D"gmail_quote" style= =3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding= -left:1ex"> You had listed that you were using the likes of:<br> <br> # cd /usr/src ; make KERNCONF=3DARM TARGET_ARCH=3Daarch64 \<br> buildkernel buildworld installkernel installworld distribution \<br> DESTDIR=3D/home/freebsd/rpi3b<br> <br> I'll note that the standard order of the first 2 is:<br> <br> buildworld buildkernel<br> <br> This is because buildworld builds some software that<br> buildkernel does not build for itself but does use.<br></blockquote><div><b= r></div><div>Okay this is noted, thanks for clarifying and correcting me, I= really appreciate it. I'll reflect on the proper build sequence for mu= ch efficiency.<br></div><div>=C2=A0<br></div><blockquote class=3D"gmail_quo= te" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204= );padding-left:1ex"> There is a kernel-toolchain target for avoiding the<br> need to do a full buildworld just to buildkernel , so:<br> <br> kernel-toolchain buildkernel<br> <br> is an expected sequence.<br></blockquote><div><br></div><div>Okay I'll = take note of this too.</div><div>=C2=A0<br></div><blockquote class=3D"gmail= _quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204= ,204);padding-left:1ex"> I do not know how long a from-scratch buildworld<br> buildkernel without a -jN takes on a RPi3B these<br> days. If I remember right, for -jN with 1<N, I last<br> saw claims about such they were somewhere in the<br> range 36hr..48hr.</blockquote><div><br></div><div>There's an ongoing bu= ild at the moment, it's already taking 41 hours since I started it. I t= ook another build when I came back home from the office.<br></div><div>=C2= =A0<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px = 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">But I'm = unsure of the specific N<br> that was in use. Nor do I know the storage media<br> type(s) involved, for example. I do not remember<br> any reports for -j1.<br></blockquote><div><br></div><div>I'll try this = with RPi 3B. The current build that I have will be my baseline.</div><div><= br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8e= x;border-left:1px solid rgb(204,204,204);padding-left:1ex"> Use of the likes of: vm.pageout_oom_seq=3D120<br> was essential to such -jN usage on a RPi3B as N<br> gets larger. Of course, swap partition use for<br> paging was also essential.<br></blockquote><div><br></div><div>Wow, that= 9;s great! I have this=20 vm.pageout_oom_seq=3D120 configured in my system now based on your previous= inputs.<br></div><div><br></div><blockquote class=3D"gmail_quote" style=3D= "margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-le= ft:1ex"> Likewise use of:<br> <br> vm.swap_enabled=3D0<br> vm.swap_idle_enabled=3D0<br> <br> can be important to not losing communication<br> with the RPi3B. Those last 2 are not tunable<br> but are writable:<br> <br> # sysctl -aT | grep swap_<br> # sysctl -aW | grep swap_<br> vm.swap_enabled: 0<br> . . .<br> vm.swap_idle_enabled: 0<br> . . .<br> <br> (This means that they have fewer places where<br> assignments can be made. For example, the<br> loader can not make the assignments.)<br> <br> By contrast, vm.pageout_oom_seq is both<br> writable and tunable:<br> <br> # sysctl -aW | grep oom<br> . . .<br> vm.pageout_oom_seq: 120<br> . . .<br> <br> # sysctl -aT | grep oom<br> . . .<br> vm.pageout_oom_seq: 120<br> . . .<br> <br> (So even the loader can make such assignments.)<br></blockquote><div><br></= div><div>Yes, I have these two sysctl parameters configured in the system. = Thanks for the details and further inputs.<br></div><div>=C2=A0 <br></div><= blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l= eft:1px solid rgb(204,204,204);padding-left:1ex"> I'll note that I've no interest in using arm hardware<br> to build for other types of hardware. So I normally<br> have the targeting of support for building for other<br> architectures disabled when I build on aarch64 or<br> armv7. (Basically, a less complete clang/clang++<br> related toolchain ends up being built.)<br></blockquote><div><br></div><div= >Ah okay, so you mean to say that you disable these other architectures by = declaring and accomplishing it in the /etc/src.conf?</div><div><br></div><d= iv>I'll provide an update here once the build is done knowing how long = it takes to finish.<br></div></div></div></div></blockquote><div><br></div>= </div><div class=3D"gmail_quote">Hi Mark,</div><div class=3D"gmail_quote"><= br></div><div class=3D"gmail_quote">With this set of build commands now,</d= iv><div class=3D"gmail_quote"><br></div><div class=3D"gmail_quote"># cd /us= r/src; make -j3 KERNCONF=3DARM TARGET_ARCH=3Daarch64 buildworld kernel-tool= chain buildkernel installworld installkernel distribution DESTDIR=3D/home/f= reebsd/rpi3b</div><div class=3D"gmail_quote"><br></div><div class=3D"gmail_= quote">in RPi 3B, I encountered the other OOM error which is the 'threa= d waited too long to allocate a page'. This occurred from every build I= conducted. Though the first error on 'failed to reclaim memory' wa= s never experienced again. Below are the error logs.<br></div><div class=3D= "gmail_quote">...<br></div><div class=3D"gmail_quote">swap_pager: indefinit= e wait buffer: bufobj: 0, blkno: 256929, size: 4096<br>swap_pager: indefini= te wait buffer: bufobj: 0, blkno: 3628, size: 4096<br>swap_pager: indefinit= e wait buffer: bufobj: 0, blkno: 255839, size: 40960<br>pid 46153 (c++), ji= d 0, uid 0, was killed: a thread waited too long to allocate a page<br>swap= _pager: indefinite wait buffer: bufobj: 0, blkno: 255857, size: 28672<br>sw= ap_pager: indefinite wait buffer: bufobj: 0, blkno: 3634, size: 8192<br>swa= p_pager: indefinite wait buffer: bufobj: 0, blkno: 256037, size: 4096<br>sw= ap_pager: indefinite wait buffer: bufobj: 0, blkno: 255320, size: 8192</div= ><div class=3D"gmail_quote"><br></div><div class=3D"gmail_quote">Perhaps so= me further tweaks are needed in the system so I set aside my RPi 3B tempora= rily and switched over to my RPi 4B using the same microSD card and USB fla= sh drive (3.5 GB swap partition device) and the build completed successfull= y. It took around 30 hours to complete. This RPi 4B has 2GB RAM capacity wh= ile the RPi 3B has 1GB. From here, I'll continue looking further for sy= stem tunables in RPi 3B which has lesser RAM capacity.</div><div class=3D"g= mail_quote"><br></div><div class=3D"gmail_quote">Thanks and best regards,</= div><div class=3D"gmail_quote">Archimedes <br></div><div class=3D"gmail_quo= te"><br></div><div class=3D"gmail_quote"><br></div><div class=3D"gmail_quot= e"><br></div></div> --000000000000ee420f05ecec76a2--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJFbk7HJ1WA5Qc0LNEZpKgv78yiM0w7ex=gjgpjjTf3chhHhiQ>