Date: Wed, 2 Nov 2022 21:46:47 +0000 From: Nuno Teixeira <eduardo@freebsd.org> To: Archimedes Gaviola <archimedes.gaviola@gmail.com> Cc: Mark Millard <marklmi@yahoo.com>, freebsd-current <freebsd-current@freebsd.org> Subject: Re: 14.0-CURRENT failed to reclaim memory error in RPi 3B build Message-ID: <CAFDf7U%2B96eNagXnn-EihMmG62UJHiY=X6U8CJEWFZX52U9ZYVQ@mail.gmail.com> In-Reply-To: <CAFDf7ULZa3q0AwEwuPU5ar26uKJXkwVbRQDMCRq4AE5HBt5tOg@mail.gmail.com> References: <84376CC9-B991-4BF3-AF5F-0AA09CB28339.ref@yahoo.com> <84376CC9-B991-4BF3-AF5F-0AA09CB28339@yahoo.com> <CAJFbk7FfYPSe3eF00HgDdebW70HKp5zKR0JaChTVniUDPG2qxQ@mail.gmail.com> <CA350C16-3604-4D88-9C14-040A45F6F125@yahoo.com> <CAJFbk7Hxvr9gs7GnniWtJ-QEH4yjYbB9S-vKVLjipa8v5VHa%2Bw@mail.gmail.com> <CAJFbk7GQhvgQS-23jFfFf=ershgGNKByHR8ug08Df8merwkh6Q@mail.gmail.com> <CAFDf7ULZa3q0AwEwuPU5ar26uKJXkwVbRQDMCRq4AE5HBt5tOg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000bab44505ec83c7a4 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sorry, wrong post! Nuno Teixeira <eduardo@freebsd.org> escreveu no dia quarta, 2/11/2022 =C3= =A0(s) 21:44: > Hello, > > From > https://lists.freebsd.org/archives/freebsd-ports/2022-August/002476.html > > --- > With both FLANG and MLIR: > (...) > [13:49:55] [01] [13:49:17] Finished devel/llvm13 | llvm13-13.0.1_3: Succe= ss > > load averages: . . . MaxObs: 6.43, 5.91, 5.77 > (Note: spanned overnight so the nightly cron job was > spanned.) > > Note: Given that SWAP was used, I report more > Max(imum)Obs(erved) figures for this case than > I've been reporting for other tests: > > 5696Mi MaxObsActive > 1775Mi MaxObsSwapUsed > 7374Mi MaxObs(Act+Lndry+SwapUsed) > 9333Mi MaxObs(Act+Wir+Lndry+SwapUsed) > > Reminder: MaximumOfASum <=3D TheSumOfTheMaximums > Note: The various Maximums need not be from the same time. > > > By contrast . . . > > No FLANG, no MLIR: > > (...) > [11:07:48] [01] [08:58:53] Finished devel/llvm13 | llvm13-13.0.1_3: Succe= ss > > load averages: . . . MaxObs: 5.31, 4.94, 4.79 > > 1479Mi MaxObs(Act+Lndry+SwapUsed) > > So, vastly less RAM+SWAP space use. Somewhat under > 5 hours less build time (about 9hr vs. somewhat under 14hr). > --- > > Archimedes Gaviola <archimedes.gaviola@gmail.com> escreveu no dia quarta, > 2/11/2022 =C3=A0(s) 21:10: > >> >> >> On Mon, Oct 31, 2022 at 1:47 PM Archimedes Gaviola < >> archimedes.gaviola@gmail.com> wrote: >> >>> >>> > Okay noted on GPT not MBR method with gpart. >>> >>> >>>> I did not happen to have a MBR example around. So I could >>>> only show GPT. The note was more to avoid confusion than >>>> anything, since the two are not equivalent for how they >>>> work. >>>> >>> >>> Okay, this is noted. >>> >>> >>>> >>>> > By the way, what's the proper allocation size of swap in FreeBSD? >>>> >>>> FreeBSD has a waring that it produces indicating possible mistuning >>>> when you potentially have too much. An example is: >>>> >>>> warning: total configured swap (2097152 pages) exceeds maximum >>>> recommended amount (916632 pages). >>>> warning: increase kern.maxswzone or reduce amount of swap. >>>> >>>> The numbers are dependent on the amount of RAM present and >>>> other details. >>>> >>>> My understanding is that increasing kern.maxswzone has tradeoffs. >>>> I avoid getting the message because I do not understand the >>>> tradeoffs or how to manage the tradeoffs or even how to identify >>>> an instance of hitting such a tradeoff. >>>> >>> >>> Basically the warning messages you've shared are the messages I >>> encountered with my older FreeBSD system running on MIPS32 at the time = I >>> allocated a swap partition because of the higher allocation size I've m= ade. >>> So what I did is gradually adjust the swap size until such warnings >>> disappear. I did not go through the details as most likely it requires = a >>> deeper knowledge on this area. That's why this experience illuminated m= e >>> again with my RPi 3B ARM system on the proper allocation size. But yes, >>> below you have the allocation size. >>> >>> >>>> >>>> For aarch64 I've been about to have swap of about 3.4 to 3.5 or >>>> so times the amount of RAM without getting the warnings. That >>>> is why 3.5G in my RPi3B example. (So RAM+SWAP approx.=3D 4.5*RAM.) >>>> (armv7 only allows more like 1.8 times the RAM before getting >>>> the warning.) >>>> >>> >>> Okay this is noted. I'll take the 3.5G size as this is based on your >>> actual experience. >>> >>> >>>> >>>> I avoid even getting too close to the warning as there seems to >>>> be some build-to-build variability in what fits vs. not. This >>>> avoids having to frequently adjust the size. >>>> >>>> >>> I, too, need to avoid such warnings as much as possible with this RPi 3= B >>> configuration. >>> >>> >>>> Going from the other side, how much RAM+SWAP will your activities >>>> use? To avoid accurately figuring out such, you may just want to >>>> have near the 3.4 to 3.5 times RAM. (There have been times when >>>> clang had memory use oddities that required more than normal for >>>> a time, for example.) >>>> >>> >>> I'll just follow the size you have and let me observe how it goes. >>> >>> >>>> >>>> > This RPi 3B has 1GB of RAM (~947 MB), do I need to set twice the >>>> capacity of this physical RAM? >>>> >>>> Ultimately your choice. How much parallel activity you >>>> want to attempt likely contributes. If you build ports, >>>> you might do so in a way that uses more RAM+SWAP than >>>> system builds do, for example. >>>> >>> >>> Okay this is noted. For now, building the kernel and world is my goal, >>> no ports yet. >>> >>> >>>> >>>> > (Note: swap file usage is subject to deadlock conditions >>>> > avoided by use of swap partitions.) >>>> > >>>> > This is noted. >>>> > >>>> > >>>> > I use a serial console & ssh session only context to avoid >>>> > having sizable competition for RAM. >>>> > >>>> > I avoid using tmpfs because it competes for RAM use. >>>> > >>>> > I use the likes of ( in, say, /boot/loader/conf ): >>>> > >>>> > # >>>> > # Delay when persistent low free RAM leads to >>>> > # Out Of Memory killing of processes: >>>> > vm.pageout_oom_seq=3D120 >>>> > >>>> > This delays potential "killed: failed to reclaim memory" kills, >>>> > possibly long enough to reach a state where sufficient memory is >>>> > reclaimed. >>>> > >>>> > Alright this is well noted too. >>>> >>>> There is tuning related to "a thread waited too long to >>>> allocate a page" that happens because of paging I/O >>>> characteristics. But but I've not hit that type of >>>> error. >>>> >>>> I'll also note that the "out of swap space" case is a >>>> misnomer in that it is one or two of 2 internal data >>>> structures that is out of space, not necessarily the >>>> swap space on the media. Again, I've not ever hit that >>>> type of error. I'm not aware of tuning for this case. >>>> >>> >>> Okay, noted as well on this info. Let me just try the 3.5G swap >>> allocation. I will post another thread if I ever encounter these types = of >>> errors. >>> >>> >>>> >>>> > I'll note that the status "killed: failed to reclaim memory" does >>>> > not require that swap be used much at all. Sustained low free RAM >>>> > from just one process that always stays runnable and has a >>>> > sufficiently large active set of pages can be sufficient to end up >>>> > with such kills. Having swap allows for inactive pages to get out >>>> > of the way, which can help. >>>> > >>>> > I use the likes of ( in, say, /etc/ssyctl.conf ): >>>> > >>>> > # >>>> > # Together this pair avoids swapping out the process kernel stacks. >>>> > # This avoids processes for interacting with the system from being >>>> > # hung-up. >>>> > vm.swap_enabled=3D0 >>>> > vm.swap_idle_enabled=3D0 >>>> > >>>> > This allows paging to the swap space but disallows moving >>>> > kernel thread stacks to the swap space. Otherwise the >>>> > processes used to interact with the RPi3 can become >>>> > non-runnable, preventing such interactions. >>>> > >>>> > Okay this too is well noted. >>>> > >>>> > >>>> > I have NVMe or SSD based USB media, not microsd cards nor >>>> > spinning rust. (I use just bootcode.bin and timeout files >>>> > on microsd media for the RPi3B. Even the rest of the RPi* >>>> > firmware is on the USB media, as well as u-boot.bin .) >>>> >>>> This may contribute to why I've never gotten a "a thread >>>> waited too long to allocate a page" on any system. (Some >>>> systems, while bootable via USB3 media I have, also have >>>> have even faster internal media that is normally used.) >>>> >>> >>> Alright so there's significance. >>> >>> >>>> >>>> > My usage of such a configuration struture for building >>>> > software (world, kernel, ports) applies to all the >>>> > systems I do such with, including ones with a lot more >>>> > resources, including a lot more RAM. >>>> > >>>> > Thanks for these inputs, noted on these things! I haven't tried NVMe >>>> and SSD media in my RPi 3B. So, they are far more superior as compared= to >>>> microSD cards when it comes to building software? >>>> >>>> My understanding is that microsd card media is fairly >>>> generally not as good for such contexts: slower, fails >>>> sooner, etc. >>>> >>> >>> I'll take note of this one as I may encounter those attributes along th= e >>> course of building software. It's something that I need to explore and = do >>> some research ahead. >>> >>> >>>> >>>> I happen to boot multiple types of machines from the >>>> same media so I use USB3 media that is compatible with >>>> USB2 use, a single such USB3 device not needing a >>>> powered hub for use on the likes of an RPi3B. (Lots >>>> of USB3 media around would require external power for >>>> USB2 or an RPi3B use.) I need a powered hub for 2 or >>>> more such media on a RPi3B. >>>> >>> >>> Okay, that's right. In my experience, inserting some devices tends to >>> reset the 4 USB ports' power, thus to prevent such behavior needs a >>> self-powered hub. >>> >>> >> Hi Mark, >> >> Just an update, as kernel and world compilation is ongoing with my RPi3B >> system (with swap partition) is doing so far, so good. It already surpas= sed >> the tough part that breaks the compilation process here. >> ... >> >> llvm-tblgen -gen-asm-matcher -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenAsmMatcher.inc.d -o RISCVGenAsmMatcher.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> llvm-tblgen -gen-asm-writer -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenAsmWriter.inc.d -o RISCVGenAsmWriter.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> llvm-tblgen -gen-callingconv -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenCallingConv.inc.d -o RISCVGenCallingConv.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> llvm-tblgen -gen-compress-inst-emitter -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenCompressInstEmitter.inc.d -o RISCVGenCompressInstEmitter.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> llvm-tblgen -gen-dag-isel -I /usr/src/contrib/llvm-project/llvm/include >> -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenDAGISel.inc.d -o RISCVGenDAGISel.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> llvm-tblgen -gen-disassembler -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenDisassemblerTables.inc.d -o RISCVGenDisassemblerTables.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> llvm-tblgen -gen-global-isel -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenGlobalISel.inc.d -o RISCVGenGlobalISel.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> llvm-tblgen -gen-instr-info -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenInstrInfo.inc.d -o RISCVGenInstrInfo.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> llvm-tblgen -gen-emitter -I /usr/src/contrib/llvm-project/llvm/include >> -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenMCCodeEmitter.inc.d -o RISCVGenMCCodeEmitter.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> llvm-tblgen -gen-pseudo-lowering -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenMCPseudoLowering.inc.d -o RISCVGenMCPseudoLowering.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> llvm-tblgen -gen-register-bank -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenRegisterBank.inc.d -o RISCVGenRegisterBank.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> llvm-tblgen -gen-register-info -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenRegisterInfo.inc.d -o RISCVGenRegisterInfo.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> llvm-tblgen -gen-searchable-tables -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenSearchableTables.inc.d -o RISCVGenSearchableTables.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> llvm-tblgen -gen-subtarget -I /usr/src/contrib/llvm-project/llvm/includ= e >> -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenSubtargetInfo.inc.d -o RISCVGenSubtargetInfo.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> llvm-tblgen -gen-searchable-tables -I >> /usr/src/contrib/llvm-project/llvm/include -I >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV -d >> RISCVGenSystemOperands.inc.d -o RISCVGenSystemOperands.inc >> /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td >> >> Any thoughts why this part is quite a challenge when it comes to memory >> usage? The other architectures do not possess such behavior... just curi= ous. >> >> Thanks and best regards, >> Archimedes >> >> >> > > > -- > Nuno Teixeira > FreeBSD Committer (ports) > --=20 Nuno Teixeira FreeBSD Committer (ports) --000000000000bab44505ec83c7a4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Sorry, wrong post!<br></div><br><div class=3D"gmail_quote"= ><div dir=3D"ltr" class=3D"gmail_attr">Nuno Teixeira <<a href=3D"mailto:= eduardo@freebsd.org">eduardo@freebsd.org</a>> escreveu no dia quarta, 2/= 11/2022 =C3=A0(s) 21:44:<br></div><blockquote class=3D"gmail_quote" style= =3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding= -left:1ex"><div dir=3D"ltr"><div>Hello,</div><div><br></div><div>From <a hr= ef=3D"https://lists.freebsd.org/archives/freebsd-ports/2022-August/002476.h= tml" target=3D"_blank">https://lists.freebsd.org/archives/freebsd-ports/202= 2-August/002476.html</a></div><div><br></div><div>---<br></div><div>With bo= th FLANG and MLIR:<span></span></div><div><span></span></div><span> </span><span>(...)<br></span> [13:49:55] [01] [13:49:17] Finished devel/llvm13 | llvm13-13.0.1_3: Success= <br> <br> load averages:=C2=A0 =C2=A0. . . MaxObs:=C2=A0 =C2=A06.43,=C2=A0 =C2=A05.91= ,=C2=A0 =C2=A05.77<br> (Note: spanned overnight so the nightly cron job was<br> spanned.)<br> <br> Note: Given that SWAP was used, I report more<br> Max(imum)Obs(erved) figures for this case than<br> I've been reporting for other tests:<br> <br> 5696Mi MaxObsActive<br> 1775Mi MaxObsSwapUsed<br> 7374Mi MaxObs(Act+Lndry+SwapUsed)<br> 9333Mi MaxObs(Act+Wir+Lndry+SwapUsed)<br> <br> Reminder: MaximumOfASum <=3D TheSumOfTheMaximums<br> Note: The various Maximums need not be from the same time.<br> <br> <br> By contrast . . .<br> <br><div> No FLANG, no MLIR:<span><br></span></div><div><span><br></span></div><div><= span>(...)<br></span></div><div><span></span></div><span> </span>[11:07:48] [01] [08:58:53] Finished devel/llvm13 | llvm13-13.0.1_3: = Success<br> <br> load averages:=C2=A0 =C2=A0. . . MaxObs:=C2=A0 =C2=A05.31,=C2=A0 =C2=A04.94= ,=C2=A0 =C2=A04.79<br> <br> 1479Mi MaxObs(Act+Lndry+SwapUsed)<br> <br> So, vastly less RAM+SWAP space use. Somewhat under<br><div> 5 hours less build time (about 9hr vs. somewhat under 14hr).</div><div>---<= br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gma= il_attr">Archimedes Gaviola <<a href=3D"mailto:archimedes.gaviola@gmail.= com" target=3D"_blank">archimedes.gaviola@gmail.com</a>> escreveu no dia= quarta, 2/11/2022 =C3=A0(s) 21:10:<br></div><blockquote class=3D"gmail_quo= te" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204= );padding-left:1ex"><div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div cl= ass=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Mon, Oct 31, 2= 022 at 1:47 PM Archimedes Gaviola <<a href=3D"mailto:archimedes.gaviola@= gmail.com" target=3D"_blank">archimedes.gaviola@gmail.com</a>> wrote:<br= ></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;= border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><= div dir=3D"ltr"><br></div><div>> Okay noted on GPT not MBR method with g= part.</div><div><br></div><div class=3D"gmail_quote"><blockquote class=3D"g= mail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204= ,204,204);padding-left:1ex"> <br> I did not happen to have a MBR example around. So I could<br> only show GPT. The note was more to avoid confusion than<br> anything, since the two are not equivalent for how they<br> work.<br></blockquote><div><br></div><div>Okay, this is noted.</div><div>= =C2=A0<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0= px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <br> > By the way, what's the proper allocation size of swap in FreeBSD?<= br> <br> FreeBSD has a waring that it produces indicating possible mistuning<br> when you potentially have too much. An example is:<br> <br> warning: total configured swap (2097152 pages) exceeds maximum recommended = amount (916632 pages).<br> warning: increase kern.maxswzone or reduce amount of swap.<br> <br> The numbers are dependent on the amount of RAM present and<br> other details.<br> <br> My understanding is that increasing kern.maxswzone has tradeoffs.<br> I avoid getting the message because I do not understand the<br> tradeoffs or how to manage the tradeoffs or even how to identify<br> an instance of hitting such a tradeoff.<br></blockquote><div><br></div><div= >Basically the warning messages you've shared are the messages I encoun= tered with my older FreeBSD system running on MIPS32 at the time I allocate= d a swap partition because of the higher allocation size I've made. So = what I did is gradually adjust the swap size until such warnings disappear.= I did not go through the details as most likely it requires a deeper knowl= edge on this area. That's why this experience illuminated me again with= my RPi 3B ARM system on the proper allocation size. But yes, below you hav= e the allocation size. <br></div><div>=C2=A0<br></div><blockquote class=3D"= gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(20= 4,204,204);padding-left:1ex"> <br> For aarch64 I've been about to have swap of about 3.4 to 3.5 or<br> so times the amount of RAM without getting the warnings. That<br> is why 3.5G in my RPi3B example. (So RAM+SWAP approx.=3D 4.5*RAM.)<br> (armv7 only allows more like 1.8 times the RAM before getting<br> the warning.)<br></blockquote><div><br></div><div>Okay this is noted. I'= ;ll take the 3.5G size as this is based on your actual experience.</div><di= v>=C2=A0<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px= 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <br> I avoid even getting too close to the warning as there seems to<br> be some build-to-build variability in what fits vs. not. This<br> avoids having to frequently adjust the size.<br> <br></blockquote><div><br></div><div>I, too, need to avoid such warnings as= much as possible with this RPi 3B configuration.</div><div>=C2=A0<br></div= ><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border= -left:1px solid rgb(204,204,204);padding-left:1ex"> Going from the other side, how much RAM+SWAP will your activities<br> use? To avoid accurately figuring out such, you may just want to<br> have near the 3.4 to 3.5 times RAM. (There have been times when<br> clang had memory use oddities that required more than normal for<br> a time, for example.)<br></blockquote><div><br></div><div>I'll just fol= low the size you have and let me observe how it goes.</div><div>=C2=A0<br><= /div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bo= rder-left:1px solid rgb(204,204,204);padding-left:1ex"> <br> > This RPi 3B has 1GB of RAM (~947 MB), do I need to set twice the capac= ity of this physical RAM?<br> <br> Ultimately your choice. How much parallel activity you<br> want to attempt likely contributes. If you build ports,<br> you might do so in a way that uses more RAM+SWAP than<br> system builds do, for example.<br></blockquote><div><br></div><div>Okay thi= s is noted. For now, building the kernel and world is my goal, no ports yet= .</div><div>=C2=A0=C2=A0 <br></div><blockquote class=3D"gmail_quote" style= =3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding= -left:1ex"> <br> > (Note: swap file usage is subject to deadlock conditions<br> > avoided by use of swap partitions.)<br> > <br> > This is noted.<br> >=C2=A0 <br> > <br> > I use a serial console & ssh session only context to avoid<br> > having sizable competition for RAM.<br> > <br> > I avoid using tmpfs because it competes for RAM use.<br> > <br> > I use the likes of ( in, say, /boot/loader/conf ):<br> > <br> > #<br> > # Delay when persistent low free RAM leads to<br> > # Out Of Memory killing of processes:<br> > vm.pageout_oom_seq=3D120<br> > <br> > This delays potential "killed: failed to reclaim memory" kil= ls,<br> > possibly long enough to reach a state where sufficient memory is<br> > reclaimed.<br> > <br> > Alright this is well noted too.<br> <br> There is tuning related to "a thread waited too long to<br> allocate a page" that happens because of paging I/O<br> characteristics. But but I've not hit that type of<br> error.<br> <br> I'll also note that the "out of swap space" case is a<br> misnomer in that it is one or two of 2 internal data<br> structures that is out of space, not necessarily the<br> swap space on the media. Again, I've not ever hit that<br> type of error. I'm not aware of tuning for this case.<br></blockquote><= div><br></div><div>Okay, noted as well on this info. Let me just try the 3.= 5G swap allocation. I will post another thread if I ever encounter these ty= pes of errors.<br></div><div>=C2=A0<br></div><blockquote class=3D"gmail_quo= te" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204= );padding-left:1ex"> <br> > I'll note that the status "killed: failed to reclaim memory&q= uot; does<br> > not require that swap be used much at all. Sustained low free RAM<br> > from just one process that always stays runnable and has a<br> > sufficiently large active set of pages can be sufficient to end up<br> > with such kills. Having swap allows for inactive pages to get out<br> > of the way, which can help.<br> > <br> > I use the likes of ( in, say, /etc/ssyctl.conf ):<br> > <br> > #<br> > # Together this pair avoids swapping out the process kernel stacks.<br= > > # This avoids processes for interacting with the system from being<br> > # hung-up.<br> > vm.swap_enabled=3D0<br> > vm.swap_idle_enabled=3D0<br> > <br> > This allows paging to the swap space but disallows moving<br> > kernel thread stacks to the swap space. Otherwise the<br> > processes used to interact with the RPi3 can become<br> > non-runnable, preventing such interactions.<br> > <br> > Okay this too is well noted.<br> >=C2=A0 <br> > <br> > I have NVMe or SSD based USB media, not microsd cards nor<br> > spinning rust. (I use just bootcode.bin and timeout files<br> > on microsd media for the RPi3B. Even the rest of the RPi*<br> > firmware is on the USB media, as well as u-boot.bin .)<br> <br> This may contribute to why I've never gotten a "a thread<br> waited too long to allocate a page" on any system. (Some<br> systems, while bootable via USB3 media I have, also have<br> have even faster internal media that is normally used.)<br></blockquote><di= v><br></div><div>Alright so there's significance.</div><div>=C2=A0<br><= /div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bo= rder-left:1px solid rgb(204,204,204);padding-left:1ex"> <br> > My usage of such a configuration struture for building<br> > software (world, kernel, ports) applies to all the<br> > systems I do such with, including ones with a lot more<br> > resources, including a lot more RAM.<br> > <br> > Thanks for these inputs, noted on these things! I haven't tried NV= Me and SSD media in my RPi 3B. So, they are far more superior as compared t= o microSD cards when it comes to building software?<br> <br> My understanding is that microsd card media is fairly<br> generally not as good for such contexts: slower, fails<br> sooner, etc.<br></blockquote><div><br></div><div>I'll take note of this= one as I may encounter those attributes along the course of building softw= are. It's something that I need to explore and do some research ahead.<= br></div><div>=C2=A0<br></div><blockquote class=3D"gmail_quote" style=3D"ma= rgin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:= 1ex"> <br> I happen to boot multiple types of machines from the<br> same media so I use USB3 media that is compatible with<br> USB2 use, a single such USB3 device not needing a<br> powered hub for use on the likes of an RPi3B. (Lots<br> of USB3 media around would require external power for<br> USB2 or an RPi3B use.) I need a powered hub for 2 or<br> more such media on a RPi3B.<br></blockquote><div><br></div>Okay, that's= right.=C2=A0 In my experience, inserting some devices tends to reset the 4= USB ports' power, thus to prevent such behavior needs a self-powered h= ub.</div><div class=3D"gmail_quote"><br></div></div></blockquote><div><br><= /div><div>Hi Mark,</div><div><br></div><div>Just an update, as kernel and w= orld compilation is ongoing with my RPi3B system (with swap partition) is d= oing so far, so good. It already surpassed the tough part that breaks the c= ompilation process here.<br></div><div></div><div></div><div>...<br></div><= div><br></div><div>llvm-tblgen -gen-asm-matcher =C2=A0-I /usr/src/contrib/l= lvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/R= ISCV =C2=A0-d RISCVGenAsmMatcher.inc.d -o RISCVGenAsmMatcher.inc =C2=A0/usr= /src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td<br>llvm-tblgen -ge= n-asm-writer =C2=A0-I /usr/src/contrib/llvm-project/llvm/include -I /usr/sr= c/contrib/llvm-project/llvm/lib/Target/RISCV =C2=A0-d RISCVGenAsmWriter.inc= .d -o RISCVGenAsmWriter.inc =C2=A0/usr/src/contrib/llvm-project/llvm/lib/Ta= rget/RISCV/RISCV.td<br>llvm-tblgen -gen-callingconv =C2=A0-I /usr/src/contr= ib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Targ= et/RISCV =C2=A0-d RISCVGenCallingConv.inc.d -o RISCVGenCallingConv.inc =C2= =A0/usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td<br>llvm-tbl= gen -gen-compress-inst-emitter =C2=A0-I /usr/src/contrib/llvm-project/llvm/= include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV =C2=A0-d RIS= CVGenCompressInstEmitter.inc.d -o RISCVGenCompressInstEmitter.inc =C2=A0/us= r/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td<br>llvm-tblgen -g= en-dag-isel =C2=A0-I /usr/src/contrib/llvm-project/llvm/include -I /usr/src= /contrib/llvm-project/llvm/lib/Target/RISCV =C2=A0-d RISCVGenDAGISel.inc.d = -o RISCVGenDAGISel.inc =C2=A0/usr/src/contrib/llvm-project/llvm/lib/Target/= RISCV/RISCV.td<br>llvm-tblgen -gen-disassembler =C2=A0-I /usr/src/contrib/l= lvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/R= ISCV =C2=A0-d RISCVGenDisassemblerTables.inc.d -o RISCVGenDisassemblerTable= s.inc =C2=A0/usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td<br= >llvm-tblgen -gen-global-isel =C2=A0-I /usr/src/contrib/llvm-project/llvm/i= nclude -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV =C2=A0-d RISC= VGenGlobalISel.inc.d -o RISCVGenGlobalISel.inc =C2=A0/usr/src/contrib/llvm-= project/llvm/lib/Target/RISCV/RISCV.td<br>llvm-tblgen -gen-instr-info =C2= =A0-I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-p= roject/llvm/lib/Target/RISCV =C2=A0-d RISCVGenInstrInfo.inc.d -o RISCVGenIn= strInfo.inc =C2=A0/usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV= .td<br>llvm-tblgen -gen-emitter =C2=A0-I /usr/src/contrib/llvm-project/llvm= /include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV =C2=A0-d RI= SCVGenMCCodeEmitter.inc.d -o RISCVGenMCCodeEmitter.inc =C2=A0/usr/src/contr= ib/llvm-project/llvm/lib/Target/RISCV/RISCV.td<br>llvm-tblgen -gen-pseudo-l= owering =C2=A0-I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/con= trib/llvm-project/llvm/lib/Target/RISCV =C2=A0-d RISCVGenMCPseudoLowering.i= nc.d -o RISCVGenMCPseudoLowering.inc =C2=A0/usr/src/contrib/llvm-project/ll= vm/lib/Target/RISCV/RISCV.td<br>llvm-tblgen -gen-register-bank =C2=A0-I /us= r/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/ll= vm/lib/Target/RISCV =C2=A0-d RISCVGenRegisterBank.inc.d -o RISCVGenRegister= Bank.inc =C2=A0/usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td= <br>llvm-tblgen -gen-register-info =C2=A0-I /usr/src/contrib/llvm-project/l= lvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV =C2=A0-d= RISCVGenRegisterInfo.inc.d -o RISCVGenRegisterInfo.inc =C2=A0/usr/src/cont= rib/llvm-project/llvm/lib/Target/RISCV/RISCV.td<br>llvm-tblgen -gen-searcha= ble-tables =C2=A0-I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/= contrib/llvm-project/llvm/lib/Target/RISCV =C2=A0-d RISCVGenSearchableTable= s.inc.d -o RISCVGenSearchableTables.inc =C2=A0/usr/src/contrib/llvm-project= /llvm/lib/Target/RISCV/RISCV.td<br>llvm-tblgen -gen-subtarget =C2=A0-I /usr= /src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llv= m/lib/Target/RISCV =C2=A0-d RISCVGenSubtargetInfo.inc.d -o RISCVGenSubtarge= tInfo.inc =C2=A0/usr/src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.t= d<br>llvm-tblgen -gen-searchable-tables =C2=A0-I /usr/src/contrib/llvm-proj= ect/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/RISCV =C2= =A0-d RISCVGenSystemOperands.inc.d -o RISCVGenSystemOperands.inc =C2=A0/usr= /src/contrib/llvm-project/llvm/lib/Target/RISCV/RISCV.td</div><div><br></di= v><div>Any thoughts why this part is quite a challenge when it comes to mem= ory usage? The other architectures do not possess such behavior... just cur= ious.</div><div><br></div><div>Thanks and best regards,</div><div>Archimede= s<br></div><div><br></div><div>=C2=A0</div></div></div> </blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr"><div dir= =3D"ltr"><span style=3D"color:rgb(102,102,102)">Nuno Teixeira<br>FreeBSD Co= mmitter (ports)</span></div></div> </blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"g= mail_signature"><div dir=3D"ltr"><span style=3D"color:rgb(102,102,102)">Nun= o Teixeira<br>FreeBSD Committer (ports)</span></div></div> --000000000000bab44505ec83c7a4--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFDf7U%2B96eNagXnn-EihMmG62UJHiY=X6U8CJEWFZX52U9ZYVQ>