Date: Fri, 27 Mar 2015 11:16:37 +0200 From: Konstantin Belousov <kostikbel@gmail.com> To: J David <j.david.lists@gmail.com> Cc: freebsd-stable <freebsd-stable@freebsd.org>, "freebsd-questions@freebsd.org" <freebsd-questions@freebsd.org> Subject: Re: Significant memory leak in 9.3p10? Message-ID: <20150327091637.GH2379@kib.kiev.ua> In-Reply-To: <CABXB=RTe9d0DD68RCi6JWKH%2BcK%2Ba8McmKmeejTypLhVZRc0t7w@mail.gmail.com> References: <CABXB=RRhynY5FWvw3tHrLFRyitTemavXYLBpev5Mjs_kPqimXA@mail.gmail.com> <20150316232404.GM2379@kib.kiev.ua> <CABXB=RSt0MgEyoJs4o5utTg7oSu0RZ%2B-czeY0k-Ro%2BfRubK3kQ@mail.gmail.com> <CABXB=RTe9d0DD68RCi6JWKH%2BcK%2Ba8McmKmeejTypLhVZRc0t7w@mail.gmail.com>
index | next in thread | previous in thread | raw e-mail
On Thu, Mar 26, 2015 at 03:46:05PM -0400, J David wrote: > On Mon, Mar 16, 2015 at 7:52 PM, J David <j.david.lists@gmail.com> wrote: > > On Mon, Mar 16, 2015 at 7:24 PM, Konstantin Belousov > > <kostikbel@gmail.com> wrote: > >> There are a lot of possibilities to create persistent anonymous shared > >> memory objects. Not complete list is tmpfs mounts, swap-backed md disks, > >> sysv shared memory, possibly posix shared memory (I do not remember which > >> implementation is used in stable/9). > > > > If that's the explanation, how could it be > > detected/measured/investigated/resolved/prevented? > > > > Under ordinary circumstances, machines will go run like this for days/weeks: > > > > Mem: 549M Active, 3623M Inact, 567M Wired, 3484K Cache, 827M Buf, 3156M Free > > Swap: 1024M Total, 1024M Free > > > > Then, when this happens, it rapidly degrades from that to so bad that > > processes start getting killed for being out of swap space. > > These FreeBSD machines running out of swap space and dying continues > to be a daily problem causing outages and unscheduled reboots. Is > there really no way to even research what might be causing the > problem? > > (Widening the cross-posting in the hopes of eliciting more help, so > the brief summary of the problem orginally posted to freebsd-stable is > that an unknown actor consumes all the user-space memory in the > system, including swap space, to the point where processes are killed > for being out of swap space, but if every process on the machine is > stopped, very little of the user-space memory in use is freed. > Original message with more details is here: > https://lists.freebsd.org/pipermail/freebsd-stable/2015-March/081986.html > .) > > There are no tmpfs mounts or md disks, so it would have to be one of > the other causes. How can FreeBSD's use of persistent, anonymous > shared memory objects be investigated, measured, or controlled so we > can get a handle on this issue? Start by providing useful information about your system, not a description of the information. E.g., a consistent snapshot of the following: ps auxww swapinfo mount -v mdconfig -lv vmstat -z vmstat -m vmstat -s sysctl -a ipcs -a Collect this data both during the normal run, run while the problem appear but userspace is not killed, and after you killed the processes. Just in case, show kldstat.home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150327091637.GH2379>