Date: Wed, 24 May 2023 21:04:43 -0700 From: Mark Millard <marklmi@yahoo.com> To: bob prohaska <fbsd@www.zefox.net> Cc: freebsd-arm <freebsd-arm@freebsd.org> Subject: Re: RPi3B -j4 vs. -j3 main [so: 14] from-scratch buildworld times for my context; buildkernel too; swap space usage and such Message-ID: <5104F70F-C9C7-4EB6-8764-00C14EF68049@yahoo.com> In-Reply-To: <ZG6/HpJjRbEWvvgB@www.zefox.net> References: <ZGD/8CXbIFtCuiFF@www.zefox.net> <7758885B-3115-47F0-A453-1C010313D4B8@yahoo.com> <ZGF1mOJXTarXIAwZ@www.zefox.net> <A1CFA2DF-1D38-45E2-8A82-05E94FC29C90@yahoo.com> <ZGT6I9%2BlYSJfdp6G@www.zefox.net> <03F330A1-35E4-40D4-B9C6-407041BBEC58@yahoo.com> <D0F9EB42-2D70-4510-860B-26E23127EDE0@yahoo.com> <7AA87E52-AB3D-48E4-B62A-EE73EBB23003@yahoo.com> <3C1F1156-F0CA-4958-B2D1-A28B2F2D36F5@yahoo.com> <BBF6BA4E-A3B4-4BE0-8548-8DCA4D370076@yahoo.com> <ZG6/HpJjRbEWvvgB@www.zefox.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On May 24, 2023, at 18:51, bob prohaska <fbsd@www.zefox.net> wrote: > On Wed, May 24, 2023 at 05:46:11PM -0700, Mark Millard wrote: >> RPi3B -j4 vs. -j3 buildworld times for my context: >>=20 >> World built in 120764 seconds, ncpu: 4, make -j4 [So a little under = 33 hr 35 min] >> World built in 115635 seconds, ncpu: 4, make -j3 [So a little under = 32 hr 10 min] >> [A delta of a little under 1hr 30min] >>=20 >> So: -j4 buildworld spent more time waiting for its trashing of >> the swap space than time it gained from having use of a 4th >> core. The trashing is mostly during building of libllvm, libclang, >> and liblldb. The RPi3B RAM subsystem can limit the gain from >> having more cores active as well. >>=20 >>=20 >> By contrast . . . >>=20 >> RPi3B -j4 vs. -j3 buildkernel times for my context: >>=20 >> Kernel(s) GENERIC-NODBG-CA53 built in 7836 seconds, ncpu: 4, make = -j4 [So a little under 2 hr 15 min] >> Kernel(s) GENERIC-NODBG-CA53 built in 8723 seconds, ncpu: 4, make = -j3 [So a little under 2 hr 30 min] >> [A delta of a little under 15 min] >>=20 >> So: -j4 buildkernel spent less time waiting for its trashing of >> the swap space than time it gained from having use of a 4th >> core. (Not much thrashing occurred.) >>=20 >>=20 >> And mem/swap usage info for buildworld+buildkernel . . . >>=20 >> Overall -j4 vs -j3 buildworld buildkernel info for my context: >>=20 >> -j4 Mem: . . ., 677688Ki MaxObsActive, 249652Ki MaxObsWired, 950032Ki = MaxObs(Act+Wir+Lndry) >> -j3 Mem: . . ., 683416Ki MaxObsActive, 315140Ki MaxObsWired, 927424Ki = MaxObs(Act+Wir+Lndry) >>=20 >> -j4 Swap: . . ., 1495Mi MaxObsUsed, 2117Mi = MaxObs(Act+Lndry+SwapUsed), 2358Mi MaxObs(Act+Wir+Lndry+SwapUsed) >> -j3 Swap: . . ., 1178Mi MaxObsUsed, 1811Mi = MaxObs(Act+Lndry+SwapUsed), 2049Mi MaxObs(Act+Wir+Lndry+SwapUsed) >>=20 >>=20 >>=20 >> FYI for the context: >> make[1]: "/usr/main-src/Makefile.inc1" line 326: SYSTEM_COMPILER: = Determined that CC=3Dcc matches the source tree. Not bootstrapping a = cross-compiler. >> make[1]: "/usr/main-src/Makefile.inc1" line 331: SYSTEM_LINKER: = Determined that LD=3Dld matches the source tree. Not bootstrapping a = cross-linker. >>=20 >>=20 >> Notes: >>=20 >> Incremental buildworld's would depend on how much rebuilding of >> libllvm, libclang, and liblldb would happen to occur. >>=20 >> A system with 2 GiBytes of RAM would have far less trashing of >> the swap space. A system with 4 GiBytes of RAM would not thrash >> the swap space. The closest comparison I could make with 4 >> GiBytes of RAM would be the Rock64 doing a from-scratch build. >> It is also cortex-a53 based. As I remember, its RAM subsystem >> does not limit multiple cores as easily/much. I've no access to >> an analogous 2 GiByte context. >>=20 >=20 > Would a DRAM-backed USB "drive" used only for swap help any? I don't > think it's practical, but I'm curious in principle. Long ago I think > folks actually made hardware consisting of dynamic RAM coupled to a > disk interface, probably SCSI, to get around physical RAM limits > on older computers. It's kinda silly for a Pi, and expensive. (Notes presumes well implemented. Notes ignore uses like crash dumps that may need to have data survive power loss.) Compared to spinning rust? Sure, especially for latency contributions to taking time. Compared to the NVMe USB3 drives I'm using? I expect so. But the RPi3B limits the data rate when it is actually transferring data. DRAM would not change that. Compared to a 960 GB U2 Optane used via a USB3 adapter? I'm not sure at what point the USB2 interface ends up being the primary bottleneck, such that detecting differences in various low latency media becomes difficult. But, what capacity? Cost vs an aarch64 with the capacity in its RAM for capacities like 4 GiByte or 8 GiByte or . . . ? Having a 4 GiByte+ aarch64 system is probably more effective. Of course, there are ports that are bigger builds than a FreeBSD buildworld buildkernel. So the problem moves around. But the extra capacity helps all cases. FYI: The -j4 Rock64 buildworld buildkernel is in progress. The Rock64 is set to 1200 MHz for its clock. (The 1296 MHz setting never provided a reliable context for what I have access to.) FYI: I've added a "(sample) mean so far of %idle" to my set of top hacks. Things like thrashing the swap space would drive up the mean compared to not having to wait for the I/O. (Waiting for swap space I/O counts as idle time.) Thus, if the sample mean is much greater than 25% in a 4 core context that is thrashing for -j4, it suggests that trying -j3 might cut the overall time. Similarly for some other %idle figures and -jN numbers. (I start the top after the build starts and use figures from during the build, not a significant time after.) =3D=3D=3D Mark Millard marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5104F70F-C9C7-4EB6-8764-00C14EF68049>