Date: Sat, 27 Nov 2010 07:26:24 -0800 From: Jeremy Chadwick <freebsd@jdc.parodius.com> To: Ian Smith <smithi@nimnet.asn.au> Cc: freebsd-stable@freebsd.org Subject: Re: memory leak and swapfile Message-ID: <20101127152624.GA2077@icarus.home.lan> In-Reply-To: <20101128001107.L47536@sola.nimnet.asn.au> References: <1DDFC51B19BA4655A2B8AFE4EC536337@jarasc430> <20101127031259.7CD631CC0C@ptavv.es.net> <20101127121743.GA97603@icarus.home.lan> <20101128001107.L47536@sola.nimnet.asn.au>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Nov 28, 2010 at 01:13:37AM +1100, Ian Smith wrote:
> On Sat, 27 Nov 2010, Jeremy Chadwick wrote:
> > On Fri, Nov 26, 2010 at 07:12:59PM -0800, Kevin Oberman wrote:
> > > > From: "Jack Raats" <jack@jarasoft.net>
> > > > Date: Fri, 26 Nov 2010 19:17:05 +0100
> > > > Sender: owner-freebsd-stable@freebsd.org
> > > >
> > > > It looks like that there may be a memory leak of my swap space with one of
> > > > the processes that is running.
> > > > Big question: How can I determine which process is responsible.
> > > >
> > > > Any suggestions?
> > >
> > > ps -aux Look for processes with large values in the VSZ column.
> > >
> > > I'm sure that there are other ways to see this, but that's an easy
> > > one. You can, of course, pipe the output to sort and use the -k 5 -n
> > > options.
> >
> > I believe he should be looking for a process that has a large value in
> > RSS ("RES" in top), not VSZ ("SIZE" in top).
>
> Not necessarily. More noticeable in physical-memory constrained systems
> (try 160MB!), processes that leak memory badly (say mozilla) push leaked
> but quiescent memory out to swap, where it lurks forever, growing with
> every page of eg crappy javascript, while RSS stays more constrained.
>
> % ps aux | sort -rnk5 | head -2
> smithi 11451 0.0 27.3 119008 42276 ?? S 11Nov10 336:47.74 /usr/local/lib/mozilla/mozilla-bin
> root 45236 1.9 7.8 45328 12132 v7 R 15Aug10 3261:37.36 X :0 -auth /home/smithi/.serverau
>
> At least 35MB of moz's VSZ is leaked and will disappear once restarted,
> plus another 15MB or so of VSZ X is hanging onto on mozilla's account.
>
> Small system, small example; big-iron folks miss out on half the fun :)
Here's an example (and not really from a "big-iron" system), though swap
isn't used in this scenario.
# pstat -s
Device 1K-blocks Used Avail Capacity
/dev/ada0s1b 16777216 0 16777216 0%
# top -b | egrep 'COMMAND|mysqld'
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
1199 mysql 10 44 0 801M 271M sigwai 1 16:28 0.00% mysqld
# ps -auxw | grep 1199
mysql 1199 0.0 3.3 820544 277924 ?? S 19Oct10 123:00.88 /usr/local/libexec/mysqld --defaults-extra-file=/storage/mysql/my.c
# procstat -v 1199 | sort -k5 -n | tail -5
1199 0x90600000 0x90a00000 rw- 612 0 1 0 -- df
1199 0x400000 0x8f0000 r-x 1206 1564 2 1 CN vn /usr/local/libexec/mysqld
1199 0x90c00000 0x91400000 rw- 1276 0 2 0 -- df
1199 0x8fe00000 0x90600000 rw- 2044 0 1 0 -- df
1199 0x61600000 0x8fe00000 rw- 60285 0 2 0 -- df
# procstat -v 1199 | sort -k6 -n | tail -5
1199 0x7fffffbff000 0x7fffffc00000 --- 0 0 0 0 -- --
1199 0x7ffffffe0000 0x800000000000 rwx 15 0 1 0 -- df
1199 0x61190000 0x6119c000 r-x 10 14 2 1 CN vn /lib/libgcc_s.so.1
1199 0x60e68000 0x60f3c000 r-x 126 252 2 1 CN vn /usr/lib/libstdc++.so.6
1199 0x400000 0x8f0000 r-x 1206 1564 2 1 CN vn /usr/local/libexec/mysqld
# procstat -v 1199 | awk '/1199/ { res += $5; pres += $6 } END { print res " " pres }'
69267 1830
Now what's important to take into mind is that the counts shown in
columns 5 and 6 are page counts, not actual memory usage. I'm making
the assumption that page size is hw.pagesize even though the system
shows 2MB pages available in hw.pagesizes. So:
69267 * 4096 = 283717632 bytes
1830 * 4096 = 7495680 bytes
This more or less matches what's in RSS/RES (the ~10MByte difference
probably has to do with scaling of bytes -> megabytes).
--
| Jeremy Chadwick jdc@parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP: 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20101127152624.GA2077>