Date: Tue, 23 Aug 2022 20:48:35 +0200 From: porsolic <porsolic@gmail.com> To: freebsd-hackers@freebsd.org Subject: Too much RAM used for wired memory Message-ID: <CAHmzOhiyGpye%2Bco6bdPj1LyY9gCwGRGh-7woD6sqGQ2OR06X8w@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
--000000000000a8049a05e6ed03c5
Content-Type: text/plain; charset="UTF-8"
My PC uses too much wired memory after few days.
After two weeks of uptime, wired memory usage is more than 50% of my RAM
(17G Wired, 32 GB of physical RAM)
PC is used with ZFS, 1 VM (bhyve, 1 CPU, 256 MB of RAM, passthru PCIe LAN
card, pinned to 1 phy CPU, priority/nice is -10), Xorg, multiple Firefox
instances (one used for playing videos), work in shell (zsh, tmux, vim,
...), GUI filemanager, GUI PDF reader and sometimes with KiCAD and/or WINE.
Even after stopping VM, all programs, Xorg, tmux, dbus, avahi, moused,
cups, other daemons, exporting non-root ZFS pool, going to single user mode
- wired memory still doesn't decrease
man vmstat shows:
-z Report on memory used by the kernel zone allocator, uma(9), by zone.
Is that the same as wired memory?
How can I see what exactly is using that much of wired memory?
What metrics should I use to pinpoint issue? vmstat? sysctl counters?
# uname -srpKU
FreeBSD 13.1-RELEASE amd64 1301000 1301000
# top -nSz | head -6
last pid: 1884; load averages: 2.57, 2.96, 2.91 up 15+05:21:55
15:00:00
191 processes: 5 running, 184 sleeping, 1 zombie, 1 waiting
CPU: 5.1% user, 0.0% nice, 4.2% system, 0.1% interrupt, 90.7% idle
Mem: 6136M Active, 3218M Inact, 1986M Laundry, 17G Wired, 96K Buf, 719M Free
ARC: 1513M Total, 548M MFU, 419M MRU, 11M Anon, 17M Header, 517M Other
464M Compressed, 633M Uncompressed, 1.36:1 Ratio
# vmstat -s | grep wired
4354572 pages wired down
65537 virtual user pages wired down
# vmstat -m | (head -n1 && sort -hk3 | tail -5)
Type InUse MemUse Requests Size(s)
vtbuf 24 4560K 70 4096
linux 52226 17342K 2088029157
16,32,64,128,256,384,512,1024,2048,4096,8192,16384,32768,65536
amdvi 16924 67689K 16924 32,4096
solaris 888982 303053K 5679401089
16,32,64,128,256,384,512,1024,2048,4096,8192,16384,32768,65536
drm_kms 18446744073709504758 18014398509470267K 14 64
# vmstat -z | sort -t , -k3 | (head -n1 && tail -5)
ITEM SIZE LIMIT USED FREE REQ FAILSLEEP
XDOMAIN
malloc-384: 384, 0, 368738, 1982,1477234413, 0, 0,
0
dnode_t: 816, 0, 383683, 192,235818160, 0, 0,
0
dmu_buf_impl_t: 296, 0, 386372, 27938,569079198, 0, 0,
0
vm pgcache: 4096, 0, 646128, 1726,2444497080,66010,
0, 0
vm pgcache: 4096, 0, 6560096, 4312,6998441782,53699,
0, 0
# zfs-stats -MA
------------------------------------------------------------------------
ZFS Subsystem Report Tue Aug 23 15:00:00 2022
------------------------------------------------------------------------
System Memory:
20.73% 5.99 GiB Active, 10.87% 3.14 GiB Inact
57.45% 16.61 GiB Wired, 0.00% 0 Bytes Cache
2.40% 709.61 MiB Free, 8.56% 2.47 GiB Gap
Real Installed: 32.00 GiB
Real Available: 92.97% 29.75 GiB
Real Managed: 97.19% 28.92 GiB
Logical Total: 32.00 GiB
Logical Used: 88.01% 28.16 GiB
Logical Free: 11.99% 3.84 GiB
Kernel Memory: 16.00 EiB
Data: 100.00% 16.00 EiB
Text: 0.00% 49.03 MiB
Kernel Memory Map: 4.00 GiB
Size: 63.11% 2.52 GiB
Free: 36.89% 1.48 GiB
------------------------------------------------------------------------
ARC Summary: (HEALTHY)
Memory Throttle Count: 0
ARC Misc:
Deleted: 39.87 m
Mutex Misses: 7.69 m
Evict Skips: 3.07 b
ARC Size: 36.95% 1.48 GiB
Target Size: (Adaptive) 28.12% 1.12 GiB
Min Size (Hard Limit): 23.24% 952.04 MiB
Max Size (High Water): 4:1 4.00 GiB
Compressed Data Size: 464.17 MiB
Decompressed Data Size: 632.91 MiB
Compression Factor: 1.36
ARC Size Breakdown:
Recently Used Cache Size: 55.47% 839.54 MiB
Frequently Used Cache Size: 44.53% 673.85 MiB
ARC Hash Breakdown:
Elements Max: 448.86 k
Elements Current: 12.85% 57.67 k
Collisions: 2.04 m
Chain Max: 5
Chains: 408
------------------------------------------------------------------------
--000000000000a8049a05e6ed03c5
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">My PC uses too much wired memory after few days.<br>After =
two weeks of uptime, wired memory usage is more than 50% of my RAM (17G Wir=
ed, 32 GB of physical RAM)<br>PC is used with ZFS, 1 VM (bhyve, 1 CPU, 256 =
MB of RAM, passthru PCIe LAN card, pinned to 1 phy CPU, priority/nice is -1=
0), Xorg, multiple Firefox instances (one used for playing videos), work in=
shell (zsh, tmux, vim, ...), GUI filemanager, GUI PDF reader and sometimes=
with KiCAD and/or WINE.<br><br>Even after stopping VM, all programs, Xorg,=
tmux, dbus, avahi, moused, cups, other daemons, exporting non-root ZFS poo=
l, going to single user mode - wired memory still doesn't decrease<br><=
br>man vmstat shows:<br>-z =C2=A0 =C2=A0 =C2=A0Report on memory used by the=
kernel zone allocator, uma(9), by zone.<br><br>Is that the same as wired m=
emory?<br><br>How can I see what exactly is using that much of wired memory=
?<br>What metrics should I use to pinpoint issue? vmstat? sysctl counters?<=
br><br># uname -srpKU<br>FreeBSD 13.1-RELEASE amd64 1301000 1301000<br><br>=
# top -nSz | head -6<br>last pid: =C2=A01884; =C2=A0load averages: =C2=A02.=
57, =C2=A02.96, =C2=A02.91 =C2=A0up 15+05:21:55 =C2=A0 =C2=A015:00:00<br>19=
1 processes: 5 running, 184 sleeping, 1 zombie, 1 waiting<br>CPU: =C2=A05.1=
% user, =C2=A00.0% nice, =C2=A04.2% system, =C2=A00.1% interrupt, 90.7% idl=
e<br>Mem: 6136M Active, 3218M Inact, 1986M Laundry, 17G Wired, 96K Buf, 719=
M Free<br>ARC: 1513M Total, 548M MFU, 419M MRU, 11M Anon, 17M Header, 517M =
Other<br>=C2=A0 =C2=A0 =C2=A0464M Compressed, 633M Uncompressed, 1.36:1 Rat=
io<br><br># vmstat -s | grep wired<br>=C2=A0 4354572 pages wired down<br>=
=C2=A0 =C2=A0 65537 virtual user pages wired down<br><br># vmstat -m | (hea=
d -n1 && sort -hk3 | tail -5)<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
Type InUse MemUse Requests =C2=A0Size(s)<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 vtb=
uf =C2=A0 =C2=A024 =C2=A04560K =C2=A0 =C2=A0 =C2=A0 70 =C2=A04096<br>=C2=A0=
=C2=A0 =C2=A0 =C2=A0 linux 52226 17342K 2088029157 =C2=A016,32,64,128,256,=
384,512,1024,2048,4096,8192,16384,32768,65536<br>=C2=A0 =C2=A0 =C2=A0 =C2=
=A0 amdvi 16924 67689K =C2=A0 =C2=A016924 =C2=A032,4096<br>=C2=A0 =C2=A0 =
=C2=A0 solaris 888982 303053K 5679401089 =C2=A016,32,64,128,256,384,512,102=
4,2048,4096,8192,16384,32768,65536<br>=C2=A0 =C2=A0 =C2=A0 drm_kms 18446744=
073709504758 18014398509470267K =C2=A0 =C2=A0 =C2=A0 14 =C2=A064<br><br># v=
mstat -z | sort -t , -k3 | (head -n1 && tail -5)<br>ITEM =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 SIZE =C2=A0LIMIT =C2=
=A0 =C2=A0 USED =C2=A0 =C2=A0 FREE =C2=A0 =C2=A0 =C2=A0REQ =C2=A0 =C2=A0 FA=
ILSLEEP XDOMAIN<br>malloc-384: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 38=
4, =C2=A0 =C2=A0 =C2=A00, =C2=A0368738, =C2=A0 =C2=A01982,1477234413, =C2=
=A0 0, =C2=A0 0, =C2=A0 0<br>dnode_t: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0816, =C2=A0 =C2=A0 =C2=A00, =C2=A0383683, =C2=A0 =C2=A0=
192,235818160, =C2=A0 0, =C2=A0 0, =C2=A0 0<br>dmu_buf_impl_t: =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 296, =C2=A0 =C2=A0 =C2=A00, =C2=A0386372, =C2=A0 27938,56=
9079198, =C2=A0 0, =C2=A0 0, =C2=A0 0<br>vm pgcache: =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A04096, =C2=A0 =C2=A0 =C2=A00, =C2=A0646128, =C2=A0 =C2=
=A01726,2444497080,66010, =C2=A0 0, =C2=A0 0<br>vm pgcache: =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A04096, =C2=A0 =C2=A0 =C2=A00, 6560096, =C2=A0 =C2=
=A04312,6998441782,53699, =C2=A0 0, =C2=A0 0<br><br># zfs-stats -MA<br><br>=
------------------------------------------------------------------------<br=
>ZFS Subsystem Report Tue Aug 23 15:00:00 2022<br>----------------------=
--------------------------------------------------<br><br>System Memory:<br=
><br> 20.73% 5.99 GiB Active, 10.87% 3.14 GiB Inact<br> 57.45% 16.61 GiB Wi=
red, 0.00% 0 Bytes Cache<br> 2.40% 709.61 MiB Free, 8.56% 2.47 GiB Gap<br><=
br> Real Installed: 32.00 GiB<br> Real Available: 92.97% 29.75 GiB<br>=
Real Managed: 97.19% 28.92 GiB<br><br> Logical Total: 32.00 GiB<br> L=
ogical Used: 88.01% 28.16 GiB<br> Logical Free: 11.99% 3.84 GiB<br><br>=
Kernel Memory: 16.00 EiB<br> Data: 100.00% 16.00 EiB<br> Text: 0.=
00% 49.03 MiB<br><br>Kernel Memory Map: 4.00 GiB<br> Size: 63.11% 2.5=
2 GiB<br> Free: 36.89% 1.48 GiB<br><br>---------------------------------=
---------------------------------------<br><br>ARC Summary: (HEALTHY)<br> M=
emory Throttle Count: 0<br><br>ARC Misc:<br> Deleted: 39.87 m<br> Mute=
x Misses: 7.69 m<br> Evict Skips: 3.07 b<br><br>ARC Size: 36.95% 1=
.48 GiB<br> Target Size: (Adaptive) 28.12% 1.12 GiB<br> Min Size (Hard Lim=
it): 23.24% 952.04 MiB<br> Max Size (High Water): 4:1 4.00 GiB<br> Compre=
ssed Data Size: 464.17 MiB<br> Decompressed Data Size: 632.91 MiB<br> C=
ompression Factor: 1.36<br><br>ARC Size Breakdown:<br> Recently Used Cach=
e Size: 55.47% 839.54 MiB<br> Frequently Used Cache Size: 44.53% 673.85 MiB=
<br><br>ARC Hash Breakdown:<br> Elements Max: 448.86 k<br> Elements Curr=
ent: 12.85% 57.67 k<br> Collisions: 2.04 m<br> Chain Max: 5<br> Chai=
ns: 408<br><br>--------------------------------------------------------=
----------------<br></div>
--000000000000a8049a05e6ed03c5--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAHmzOhiyGpye%2Bco6bdPj1LyY9gCwGRGh-7woD6sqGQ2OR06X8w>