Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 11 Jun 2022 15:42:24 -0700
From:      Mark Millard <marklmi@yahoo.com>
To:        FreeBSD Hackers <freebsd-hackers@freebsd.org>
Cc:        Daniel Ebdrup Jensen <debdrup@freebsd.org>, David Cross <david@crossfamilyweb.com>
Subject:   Re: What can I learn about data that is staying paged out? (There is a more specific poudriere bulk related context given.)
Message-ID:  <840FA150-5623-4BE7-A289-015E7C14436C@yahoo.com>
In-Reply-To: <6EA27152-3355-4356-B246-A083F31452F2@yahoo.com>
References:  <C337A09C-D546-46FC-A166-DBB3237D1AFC@yahoo.com> <A9C9AC24-62EC-43C1-B713-F2012CD1FD5B@yahoo.com> <573B8B0C-5209-459D-98AD-EE92DDA4DF83@yahoo.com> <6EA27152-3355-4356-B246-A083F31452F2@yahoo.com>

next in thread | previous in thread | raw e-mail | index | archive | help
I got evidence finally. An example is the sequence:

# swapoff /dev/gpt/CA72USBswp14 /dev/gpt/CA72USBswp16
# swapon /dev/gpt/CA72USBswp14 /dev/gpt/CA72USBswp16

(Wait for a time letting some "staying paged out" space
accumulate.)

# du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/
319	/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/

(Wait for a time letting some more "staying paged out" space
accumulate.)

# du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/
307	/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/

(The size decreases seem to roughly track the sustained Swap
Used increases. [There is  some other paging activity but it
is not a lot.])

# swapoff /dev/gpt/CA72USBswp14 /dev/gpt/CA72USBswp16

# du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/
379	/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/

(Apparently, du -xsm does not count the paged out
pages for tmpfs but the pages show up when forced
back into RAM.)

# swapon /dev/gpt/CA72USBswp14 /dev/gpt/CA72USBswp16

# du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/
379	/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/

(I expect that when I see the swap usage grow in a staying
paged out manor, I'll find that this -xsm figure will also
have decreased on the retry in the new context.)


The primary space in question is actually in a subdirectory:

Taking a quick estimate of the file count in . . ./ref/.p/var/cache/
I get:

# ls -Tla =
/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ | =
wc
 82146  821452 7942485

So something like 82144 files ( ignoring . and .. ).

# ls -Tla =
/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ | =
more
total 301234
drwxr-xr-x  2 root  wheel  5257152 May 28 02:31:26 2022 .
drwxr-xr-x  4 root  wheel      128 May 28 01:56:42 2022 ..
-rw-r--r--  1 root  wheel        2 May 28 02:26:40 2022 =
builder_active%01
-rw-r--r--  1 root  wheel        2 May 28 02:31:24 2022 =
builder_active%02
-rw-r--r--  1 root  wheel        2 May 28 02:31:26 2022 =
builder_active%03
-rw-r--r--  1 root  wheel        2 May 28 02:31:26 2022 =
builder_active%04
-rw-r--r--  1 root  wheel       18 May 28 01:56:51 2022 =
originspec-pkgname%accessibility_accerciser
-rw-r--r--  1 root  wheel       19 May 28 01:56:51 2022 =
originspec-pkgname%accessibility_at-spi2-atk
-rw-r--r--  1 root  wheel       20 May 28 01:56:51 2022 =
originspec-pkgname%accessibility_at-spi2-core
-rw-r--r--  1 root  wheel       11 May 28 01:56:51 2022 =
originspec-pkgname%accessibility_atk
-rw-r--r--  1 root  wheel       13 May 28 01:56:51 2022 =
originspec-pkgname%accessibility_atkmm
-rw-r--r--  1 root  wheel       17 May 28 01:56:51 2022 =
originspec-pkgname%accessibility_caribou
. . .
-rw-r--r--  1 root  wheel       17 May 28 01:56:55 2022 =
pkgname-originspec%zutils-1.11
-rw-r--r--  1 root  wheel       15 May 28 02:03:32 2022 =
pkgname-originspec%zxfer-1.1.7
-rw-r--r--  1 root  wheel       14 May 28 02:03:00 2022 =
pkgname-originspec%zxid-1.42_1
-rw-r--r--  1 root  wheel       19 May 28 02:04:03 2022 =
pkgname-originspec%zxing-cpp-1.3.0
-rw-r--r--  1 root  wheel       12 May 28 01:59:41 2022 =
pkgname-originspec%zydis-3.1.0
-rw-r--r--  1 root  wheel       18 May 28 01:57:13 2022 =
pkgname-originspec%zynaddsubfx-3.0.6,2
-rw-r--r--  1 root  wheel        9 May 28 02:02:07 2022 =
pkgname-originspec%zyre-2.0.1
-rw-r--r--  1 root  wheel       14 May 28 01:59:41 2022 =
pkgname-originspec%zziplib-0.13.72_1
-rw-r--r--  1 root  wheel       14 May 28 02:03:00 2022 =
pkgname-originspec%zzuf-0.13_1
-rw-r--r--  1 root  wheel       13 May 28 01:56:45 2022 =
ports_metadata%top_git_hash
-rw-r--r--  1 root  wheel        4 May 28 01:56:49 2022 =
ports_metadata%top_unclean

Each file seems to have a 4096 Byte page used, making for
something around 320 MiBytes. Much of it not being heavily
used, some of it is gradually paged out to make room for
other things.

# du -xsm =
/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/
326	=
/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/

(Still no reported swap usage.)

For reference:

# du -xsAm =
/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/
46	=
/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/


=3D=3D=3D
Mark Millard
marklmi at yahoo.com




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?840FA150-5623-4BE7-A289-015E7C14436C>