Date: Fri, 8 Sep 2023 17:43:48 -0700 From: Mark Millard <marklmi@yahoo.com> To: bob prohaska <fbsd@www.zefox.net> Cc: freebsd-arm@freebsd.org Subject: Re: High swap use building Kyuafile on Pi3 Message-ID: <3815402A-CAC0-4299-BDC8-153E9C71FCDC@yahoo.com> In-Reply-To: <ZPuxUMtbQLxWAhce@www.zefox.net> References: <ZPtIUumSvbAwicNe@www.zefox.net> <5DD3836C-E6F1-4EB5-8F11-3C342A5915DF@yahoo.com> <B2B938D2-10BA-4E48-ADB9-C192F72DD6BC@yahoo.com> <ZPuxUMtbQLxWAhce@www.zefox.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sep 8, 2023, at 16:42, bob prohaska <fbsd@www.zefox.net> wrote: > On Fri, Sep 08, 2023 at 01:32:06PM -0700, Mark Millard wrote: >> On Sep 8, 2023, at 10:58, Mark Millard <marklmi@yahoo.com> wrote: >>=20 >>> On Sep 8, 2023, at 09:14, bob prohaska <fbsd@www.zefox.net> wrote: >>>=20 >>>> While building a -current world on Pi3 using -DWITH_META_MODE it = appears that >>>> swap use is quite heavy (~2GB) well after clang finishes compiling.=20= >>>>=20 >>>> The tail of the build log shows=20 >>>> Building = /usr/obj/usr/src/arm64.aarch64/lib/googletest/tests/gmock_main/Kyuafile >>>> as the last entry, suggesting something in tests is the cause. >>>>=20 >>>> The machine reports >>>> FreeBSD pelorus.zefox.org 15.0-CURRENT FreeBSD 15.0-CURRENT aarch64 = 1500000 #49 main-n265134-4a9cd9fc22d7: Mon Sep 4 10:08:30 PDT 2023 = bob@pelorus.zefox.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64 >>>>=20 >>>> The build command is >>>> make -j3 -DWITH_META_MODE buildworld > buildworld.log >>>=20 >>> So up to 3 builders can be active at the same time. >>> You seem to have described only 1 builder's activity. >>>=20 >>> Was it the only active builder? If other builders were >>> active at the time you also need to check on what they >>> were doing. The ~2GB is the total across all activity, >>> including the (up to) 3 builders. >>>=20 >>> A command that would show the active builders would be: >>>=20 >>> # poudriere status -b >=20 > I'm lost at this point. No poudriere use is involved, Gack, I substituted the wrong context. Sorry. No poudriere details that I referenced apply. > it's > simply a -j3 buildworld in the "building everything" phase. However, buildworld and buildkernel with -jN do parallel build activities (for example, parallel compiles/links), with up to N at a time. Do you know how many and which commands it was as the time: 1, 2, or 3 active for a sustained, overlapping time (if more than 1)? > Normally swap use peaks while building clang and then > diminishes markedly in the building everything stage. I'll remind of history when a google test build step used to prevent your builds until they changed the optimization level down to avoid the memory-space resource use. LLVM related build activity is likely to have the major duration for notable memory-space use. But it need not be the only example of notable memory space use overall. I'll also note that the early LLVM activity is library code and later the actual compiler, linker, and llbd are built (using the library code to do so). These activites are likely of shorter duration than the library activity, but that need not mean much about the memory-space usage peak. > Previously, by then a -j3 build isn't swap-bound. Monitoring for what is running during the ~2BG swapspace use would be appropriate. top sorting in some appropriate order may give a clue, for example. More detail about what would seem to be needed. > Buildworld was still running, with three jobs, two of which > were over 1GB each in total size, though the RES numbers=20 > totaled only about 700 MB IIRC.=20 What were each of the 3 jobs doing over the time frame leading to and spanning the ~2GB? (I assume USE_TMPFS=3Dno and other avoidance of having tmpfs competing for RAM, for example.) RPi3B variant: so 1 GiByte of RAM or so. 2 GiByte of swapspace or so used. (1+2) GiByte of RAM+SWAP or so (based on the little detail I have). RAM is actually not all available, so on the low side overall. The kernel and other processes use RAM too. RES only tells you (incomplete) information related to the RAM part of RAM+SWAP. Note that, say there was only the 3 jobs and the kernel: 3 GiByte/4 is about 750 MiByte RAM+SWAP for a mean. You indicate two possibly 1 GB jobs, so that might total to 512 MiByte more than the mean scaled to 2 jobs. The figures do not suggest vastly less than ~2GB of swap space use for the whole system. It still suggests the original reporting over focused on one job, apparently one with the command already completed. A more overall span likely is required evidence, possibly across time leading to and during the ~2GB as well. =3D=3D=3D Mark Millard marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3815402A-CAC0-4299-BDC8-153E9C71FCDC>