Date: Sun, 5 Jun 2022 15:55:21 -0700 From: Mark Millard <marklmi@yahoo.com> To: David Cross <david@crossfamilyweb.com> Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: What can I learn about data that is staying paged out? (There is a more specific poudriere bulk related context given.) Message-ID: <50163894-DCEB-4033-B555-698523E4D100@yahoo.com> In-Reply-To: <27DD20B8-E19B-444B-BAE7-A09CD5390D2F@crossfamilyweb.com> References: <A9C9AC24-62EC-43C1-B713-F2012CD1FD5B@yahoo.com> <27DD20B8-E19B-444B-BAE7-A09CD5390D2F@crossfamilyweb.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2022-Jun-5, at 15:45, David Cross <david@crossfamilyweb.com> wrote:
> On Jun 5, 2022, at 6:06 PM, Mark Millard <marklmi@yahoo.com> wrote:
>>
>> On 2022-Jun-5, at 12:42, Mark Millard <marklmi@yahoo.com> wrote:
>>
>>> I have a poudriere bulk -a -c going on a 8 Gibyte
>>> aarch64 system. top has been showing an occasionally
>>> increasing swap usage but never any sizable decreases.
>>> Over 5800 ports have built so far. The context is UFS
>>> only. The system is running a non-debug build of main.
>>>
>>> Part of the context is ( in /etc/sysctl.conf ):
>>>
>>> vm.swap_enabled=0
>>> vm.swap_idle_enabled=0
>>>
>>> Also ( in /usr/local/etc/poudriere.conf ):
>>>
>>> USE_TMPFS="data"
>>>
>>> poudriere's TMPFS reports normally total under 128
>>> KiBytes across the 4 builders.
>>>
>>> For reference, example figures . . .
>>>
>>> A top variant shows:
>>>
>>> Swap: 30720Mi Total, 306816Ki Used
>>>
>>> vmstat -s shows:
>>>
>>> 78152 swap pager pages paged out
>>>
>>> Note: (78152*4096)/1024 == 312608Ki
>>>
>>> So nearly all of the "swap pager pages paged out"
>>> pages are still sitting out in the used swap/paging
>>> space. Thus, the usage is not held by user processes
>>> or is held via very long running processes or is
>>> not directly tied to user processes --or some mix.
>>>
>>> The variant of top reports never having observed
>>> more than: 6658Mi MaxObs(Act+Wir+Lndry).
>>> ("MaxObs" is short for "Maximum Observed".)
>>> Such high usage is for a bounded time, long past
>>> at this point. (Until some combination of port
>>> builds ends up active that uses such.)
>>>
>>> So I'm curious:
>>>
>>> What can I learn about the data that is staying
>>> paged out (and is gradually growing)? How can I
>>> learn it?
>>>
>>>
>>> Other notes:
>>>
>>> The poudriere jail being built is:
>>>
>>> # poudriere jail -jmain-CA7-bulk_a -i
>>> Jail name: main-CA7-bulk_a
>>> Jail version: 14.0-CURRENT
>>> Jail arch: arm.armv7
>>> Jail method: null
>>> Jail mount: /usr/obj/DESTDIRs/main-CA7-poud-bulk_a
>>> Jail fs:
>>> Jail updated: 2022-05-23 02:21:24
>>> Jail pkgbase: disabled
>>>
>>> (Just in case the armv7 jail usage or the null method
>>> or such is important to the issue.)
>>
>> Hmm. systat -swap reports a toal for the Devices/Paths Used
>> that is somewhat less than the total for what reports for the
>> Pid . . . Total figures (not the Pid Swap figures!):
>>
>> # systat -swap
>> /0 /1 /2 /3 /4 /5 /6 /7 /8 /9 /10
>> Load Average ||||||||
>>
>> Device/Path Size Used |0% /10 /20 /30 /40 / 60\ 70\ 80\ 90\ 100|
>> gpt/CA72USBswp14 14G 150M
>> gpt/CA72USBswp16 16G 150M
>> Total 30G 300M
>>
>> Pid Username Command Swap/Total Per-Process Per-System
>> 1453 root nfsd 1M / 15M 9% 0%
>> 1451 root mountd 1M / 15M 7% 0%
>> 1481 root sshd 912K / 20M 4% 0%
>> 1406 root ntpd 740K / 27M 2% 0%
>> 1513 root login 724K / 14M 5% 0%
>> 1514 root sh 656K / 13M 4% 0%
>> 342 _dhcp dhclient 516K / 13M 3% 0%
>> 1363 root rpcbind 448K / 13M 3% 0%
>> 1454 root nfsd 400K / 12M 3% 0%
>> 341 root dhclient 380K / 13M 2% 0%
>> 1341 root syslogd 324K / 12M 2% 0%
>> 1505 root getty 292K / 12M 2% 0%
>> 1510 root getty 292K / 12M 2% 0%
>> 1511 root getty 292K / 12M 2% 0%
>> 1512 root getty 292K / 12M 2% 0%
>> 1509 root getty 292K / 12M 2% 0%
>> 1508 root getty 292K / 12M 2% 0%
>> 1507 root getty 292K / 12M 2% 0%
>> 1506 root getty 288K / 12M 2% 0%
>> 1135 root devd 272K / 11M 2% 0%
>> 338 root dhclient 264K / 13M 2% 0%
>> 1 root init 244K / 11M 2% 0%
>> 1486 root cron 188K / 13M 1% 0%
>>
>> I'm, Still looking for a clear indication of what
>> most of the 300 MiBytes or so of swap/paging space
>> is in use for.
>>
>> ===
>> Mark Millard
>> marklmi at yahoo.com
>
> My guess is that this is swapped out buffer cache (files).
>
Thanks for the idea. Know how I could find an approximation
to the amount of paged out buffer cache to see about how much
of the ~300 MiBytes it might explain?
Mark
===
Mark Millard
marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?50163894-DCEB-4033-B555-698523E4D100>