Date: Fri, 27 Mar 2015 11:16:37 +0200 From: Konstantin Belousov <kostikbel@gmail.com> To: J David <j.david.lists@gmail.com> Cc: freebsd-stable <freebsd-stable@freebsd.org>, "freebsd-questions@freebsd.org" <freebsd-questions@freebsd.org> Subject: Re: Significant memory leak in 9.3p10? Message-ID: <20150327091637.GH2379@kib.kiev.ua> In-Reply-To: <CABXB=RTe9d0DD68RCi6JWKH%2BcK%2Ba8McmKmeejTypLhVZRc0t7w@mail.gmail.com> References: <CABXB=RRhynY5FWvw3tHrLFRyitTemavXYLBpev5Mjs_kPqimXA@mail.gmail.com> <20150316232404.GM2379@kib.kiev.ua> <CABXB=RSt0MgEyoJs4o5utTg7oSu0RZ%2B-czeY0k-Ro%2BfRubK3kQ@mail.gmail.com> <CABXB=RTe9d0DD68RCi6JWKH%2BcK%2Ba8McmKmeejTypLhVZRc0t7w@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Mar 26, 2015 at 03:46:05PM -0400, J David wrote: > On Mon, Mar 16, 2015 at 7:52 PM, J David <j.david.lists@gmail.com> wrote: > > On Mon, Mar 16, 2015 at 7:24 PM, Konstantin Belousov > > <kostikbel@gmail.com> wrote: > >> There are a lot of possibilities to create persistent anonymous shared > >> memory objects. Not complete list is tmpfs mounts, swap-backed md disks, > >> sysv shared memory, possibly posix shared memory (I do not remember which > >> implementation is used in stable/9). > > > > If that's the explanation, how could it be > > detected/measured/investigated/resolved/prevented? > > > > Under ordinary circumstances, machines will go run like this for days/weeks: > > > > Mem: 549M Active, 3623M Inact, 567M Wired, 3484K Cache, 827M Buf, 3156M Free > > Swap: 1024M Total, 1024M Free > > > > Then, when this happens, it rapidly degrades from that to so bad that > > processes start getting killed for being out of swap space. > > These FreeBSD machines running out of swap space and dying continues > to be a daily problem causing outages and unscheduled reboots. Is > there really no way to even research what might be causing the > problem? > > (Widening the cross-posting in the hopes of eliciting more help, so > the brief summary of the problem orginally posted to freebsd-stable is > that an unknown actor consumes all the user-space memory in the > system, including swap space, to the point where processes are killed > for being out of swap space, but if every process on the machine is > stopped, very little of the user-space memory in use is freed. > Original message with more details is here: > https://lists.freebsd.org/pipermail/freebsd-stable/2015-March/081986.html > .) > > There are no tmpfs mounts or md disks, so it would have to be one of > the other causes. How can FreeBSD's use of persistent, anonymous > shared memory objects be investigated, measured, or controlled so we > can get a handle on this issue? Start by providing useful information about your system, not a description of the information. E.g., a consistent snapshot of the following: ps auxww swapinfo mount -v mdconfig -lv vmstat -z vmstat -m vmstat -s sysctl -a ipcs -a Collect this data both during the normal run, run while the problem appear but userspace is not killed, and after you killed the processes. Just in case, show kldstat.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150327091637.GH2379>