Date: Sat, 11 Jun 2022 15:03:26 -0700 From: Mark Millard <marklmi@yahoo.com> To: Bryan Drewery <bdrewery@FreeBSD.org>, FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: (Retitled!) /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p (example) USE_TMPFS="data" tmpfs usage for poudriere Message-ID: <C0A22976-2372-4A98-A676-186FF12F5C2A@yahoo.com> In-Reply-To: <69FDC72E-F1DB-449F-A9C1-25827895E6B3@yahoo.com> References: <4997AB05-8CD4-4A2C-AAB3-34F6DB2CE325.ref@yahoo.com> <4997AB05-8CD4-4A2C-AAB3-34F6DB2CE325@yahoo.com> <82ffa6bb-5826-62b6-2b82-a735b183272f@grosbein.net> <2F1A9EE0-FB9E-4BBF-AF76-4E5AD372FCC1@yahoo.com> <69FDC72E-F1DB-449F-A9C1-25827895E6B3@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2022-Jun-11, at 13:50, Mark Millard <marklmi@yahoo.com> wrote: > [I've deleted prior parts of the exchange.] >=20 > I finally have a hypothesis with some evidence about what > primarily contributes to the 400 or so "1M-blocks Used" in: >=20 > # df -mi /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ > Filesystem 1M-blocks Used Avail Capacity iused ifree %iused Mounted = on > tmpfs 1024 400 623 39% 103659 3828501 3% = /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p >=20 > --where the poudriere-devel is using USE_TMPFS=3D"data" for a > "bulk -a -c" (that has been running for a little over 2 weeks > now and is past 1/3 done). >=20 > Taking a quick estimate of the file count in . . ./ref/.p/var/cache/ > I get: >=20 > # ls -Tla = /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ | = wc > 82146 821452 7942485 >=20 > Taking 82146*4096 I get: 336,470,016 > Taking 336,470,016 / 1024 / 1024 I get around: 320 Mi >=20 > By contrast: >=20 > # du -xsAm = /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ > 46 = /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ >=20 > But: >=20 > # du -xsm = /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ > 295 = /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ >=20 > and: >=20 > # du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ > 319 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ I see that a little relevant material you have not been able to see because of what I omitted from earlier messages --messages not sent to you at the time. In my earlier messages I'd shown: # du -xsAm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p 101 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p # du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p 68 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p so -xsm resulsts were smaller than -xsAm results and way smaller than the "df -m" results. The 101-ish -xsAm result for .p/ has stayed fairly consistent. It appears that the swapoff activity mentioned below changed the -xsm results by forcing tmpfs material back into RAM. This ends up explaining the original 403 MiBytes vs. 101 MiBytes question that was mentioned in the original message subject. I mention this as it would appear to be important to trying to replicate or otherwise test the issue: Context matters for interpreting "du -xsm" results for tmpfs from what I can tell. > This is all after having done a swapoff for the 2 swap partitions > and a swap on for both and then waiting a while with the "bulk -a -c" > still running throughout and after. (So under 82 MiBytes is in > use in swap space at this point.) The above seems to be what lead to my "du -xsm" result being fairly close to the "df -m" result. > poudriere's tmpfs usage reporting does not seem to cover this > area's tmpfs usage. But, at least for "bulk -a -c" kinds of > activity, it ends up being the majority of the tmpfs usage for > USE_TMPFS=3D"data" types of configuration. >=20 > For reference: >=20 > # ls -Tla = /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ | = more > total 301234 > drwxr-xr-x 2 root wheel 5257152 May 28 02:31:26 2022 . > drwxr-xr-x 4 root wheel 128 May 28 01:56:42 2022 .. > -rw-r--r-- 1 root wheel 2 May 28 02:26:40 2022 = builder_active%01 > -rw-r--r-- 1 root wheel 2 May 28 02:31:24 2022 = builder_active%02 > -rw-r--r-- 1 root wheel 2 May 28 02:31:26 2022 = builder_active%03 > -rw-r--r-- 1 root wheel 2 May 28 02:31:26 2022 = builder_active%04 > -rw-r--r-- 1 root wheel 18 May 28 01:56:51 2022 = originspec-pkgname%accessibility_accerciser > -rw-r--r-- 1 root wheel 19 May 28 01:56:51 2022 = originspec-pkgname%accessibility_at-spi2-atk > -rw-r--r-- 1 root wheel 20 May 28 01:56:51 2022 = originspec-pkgname%accessibility_at-spi2-core > -rw-r--r-- 1 root wheel 11 May 28 01:56:51 2022 = originspec-pkgname%accessibility_atk > -rw-r--r-- 1 root wheel 13 May 28 01:56:51 2022 = originspec-pkgname%accessibility_atkmm > -rw-r--r-- 1 root wheel 17 May 28 01:56:51 2022 = originspec-pkgname%accessibility_caribou > . . . > -rw-r--r-- 1 root wheel 17 May 28 01:56:55 2022 = pkgname-originspec%zutils-1.11 > -rw-r--r-- 1 root wheel 15 May 28 02:03:32 2022 = pkgname-originspec%zxfer-1.1.7 > -rw-r--r-- 1 root wheel 14 May 28 02:03:00 2022 = pkgname-originspec%zxid-1.42_1 > -rw-r--r-- 1 root wheel 19 May 28 02:04:03 2022 = pkgname-originspec%zxing-cpp-1.3.0 > -rw-r--r-- 1 root wheel 12 May 28 01:59:41 2022 = pkgname-originspec%zydis-3.1.0 > -rw-r--r-- 1 root wheel 18 May 28 01:57:13 2022 = pkgname-originspec%zynaddsubfx-3.0.6,2 > -rw-r--r-- 1 root wheel 9 May 28 02:02:07 2022 = pkgname-originspec%zyre-2.0.1 > -rw-r--r-- 1 root wheel 14 May 28 01:59:41 2022 = pkgname-originspec%zziplib-0.13.72_1 > -rw-r--r-- 1 root wheel 14 May 28 02:03:00 2022 = pkgname-originspec%zzuf-0.13_1 > -rw-r--r-- 1 root wheel 13 May 28 01:56:45 2022 = ports_metadata%top_git_hash > -rw-r--r-- 1 root wheel 4 May 28 01:56:49 2022 = ports_metadata%top_unclean =3D=3D=3D Mark Millard marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C0A22976-2372-4A98-A676-186FF12F5C2A>