Date: Thu, 13 Sep 2018 16:14:32 +0200 From: Daniel Nebdal <dnebdal@gmail.com> To: eric@vangyzen.net Cc: Current <freebsd-current@freebsd.org>, markj@freebsd.org Subject: Re: arc_reclaim_thread running hot Message-ID: <CA%2Bt49PLZXfw7VLrHgqCetbrs8hBD3=WyqnD-NdD59rV6n3dwOg@mail.gmail.com> In-Reply-To: <0cdba37f-6b47-0929-fb72-dcca7e70a05a@vangyzen.net> References: <0cdba37f-6b47-0929-fb72-dcca7e70a05a@vangyzen.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 13 Sep 2018 at 15:19, Eric van Gyzen <eric@vangyzen.net> wrote:
>
> This morning, I found the arc_reclaim_thread running hot on my laptop
> running 12.0-ALPHA5 r338572.
>
> vfs.zfs.arc_max="4294967296" <-- 4 GiB
>
> last pid: 13288; load averages: 1.32, 1.26, 1.16
>
> Mem: 456M Active, 3837M Inact, 743M Laundry, 2563M Wired, 167M Free
> ARC: 1131M Total, 304M MFU, 145M MRU, 1344K Anon, 9116K Header, 671M Other
> 89M Compressed, 361M Uncompressed, 4.03:1 Ratio
>
> 22 root -8 - 0 256K CPU2 2 309:20 99.75%
> zfskern{arc_reclaim_thread}
>
> zfs_arc_meta_strategy is still the default of 1.
>
> I sampled the thread's stacks with
>
> for N in `jot 1000`; do procstat -kk 100101; done | grep 100101
>
> and put the results here:
>
> https://people.freebsd.org/~vangyzen/arc_reclaim_thread_stacks.txt
>
> I'm happy to help debug this. Just let me know what you need.
>
> Eric
Just to summarize, these are the top 15 active
(guessing that the leftmost function is "current"):
22 arc_space_return
22 uma_dbg_free
26 uma_dbg_alloc
30 mi_switch
31 uma_zalloc_arg
33 multilist_sublist_lock
34 __mtx_unlock_flags
39 <running>
46 _sx_xlock
48 hdr_full_cons
50 uma_zfree_arg
58 aggsum_add
91 arc_adjust
156 _sx_xunlock
185 arc_evict_state
And the top 15 in total
(that is, no matter where in the call stack):
44 multilist_sublist_lock
44 uma_dbg_free
46 _sx_xlock
47 uma_dbg_alloc
61 arc_space_return
127 aggsum_add
139 hdr_full_cons
156 _sx_xunlock
211 uma_zfree_arg
219 uma_zalloc_arg
833 arc_evict_state
926 arc_adjust
961 arc_reclaim_thread
961 fork_exit
961 fork_trampoline
(I have nothing sensible to add about the actual problem.)
--
Daniel Nebdal
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2Bt49PLZXfw7VLrHgqCetbrs8hBD3=WyqnD-NdD59rV6n3dwOg>