Date: Thu, 7 Jun 2018 19:04:29 +0930 From: Shane Ambler <Shane@ShaneWare.Biz> To: Peter Jeremy <peter@rulingia.com>, freebsd-stable@freebsd.org Subject: Re: ZFS+find(1) wiring all RAM Message-ID: <1da884f4-1442-9bba-02d3-2cedadf97a06@ShaneWare.Biz> In-Reply-To: <20180607063924.GG1697@server.rulingia.com> References: <20180607063924.GG1697@server.rulingia.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 07/06/2018 16:09, Peter Jeremy wrote: > I've noticed that 11-stable/amd64 has been wiring seemingly excessive > amounts of RAM for some time (the problem goes back at least 6 months). > This extends to getting ENOMEM errors from g_io_deliver() and out-of-swap > errors killing processes on a low-memory system. I'm not sure when it > started by it seems to hawe gotten worse between r331535 and r334494. Don't know if this will help you at all -- I have seen excess wired for a few years, since 10.1, I now run 11-stable, my experience has seen the severity varying over time. I first reported this 28/10/2014 related to heavy disk use on a zpool. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194654 The inability to get solid repeatable steps to reproduce have prevented me from chasing this more. My desktop machine originally had 8G and when wired went over 7G there was no choice but to reset. I now have 16G and have a script running constantly to monitor the wired and react when needed, allocating a big chunk of ram normally causes wired to be released, this has kept wired under 10G for several months. I see arc usage sitting at the arc_max setting with wired going more than two times that. My vm.max_wired is at 5G (default) and I think this is a limit that is somehow bypassed, if it could be enforced we might be good. This could be the point to chase in a bug report. > 1) Significant amounts of wired memory are in use but I can't find anything > in "vmstat -mz" that would explain where it's going. I started logging my wired usage and have about 6 months of data, in the following sample, allocating 2G more than free caused wired to drop 4G in about 2 secs, you can also see that arc usage did not change in this time. This leads me to think it may be related to changes made to add support for zfs but I don't think zfs is the one allocating this wired amount. The values I am logging are from - vm.max_wired vm.stats.vm.v_wire_count kstat.zfs.misc.arcstats.size vm.stats.vm.v_free_count 2018/06/06_13:22:53 wired: 9779M max_wired: 5210M ARC: 4.1G Free:3.1G 2018/06/06_13:22:53 wired: 9781M max_wired: 5210M ARC: 4.1G Free:3.0G 2018/06/06_13:22:54 Excess wired detected. Allocating 4.9G to reduce wired. 2018/06/06_13:22:54 wired: 9787M max_wired: 5210M ARC: 4.1G Free:3.0G 2018/06/06_13:22:54 wired: 9796M max_wired: 5210M ARC: 4.1G Free:1.9G 2018/06/06_13:22:55 wired: 9809M max_wired: 5210M ARC: 4.1G Free:389.6M 2018/06/06_13:22:55 wired: 8169M max_wired: 5210M ARC: 4.1G Free:1.2G 2018/06/06_13:22:55 wired: 5917M max_wired: 5210M ARC: 4.1G Free:2.6G 2018/06/06_13:22:55 wired: 5566M max_wired: 5210M ARC: 4.1G Free:2.2G -- Shane Ambler Shane (at) ShaneWare (dot) Biz http://ShaneWare.Biz
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1da884f4-1442-9bba-02d3-2cedadf97a06>