Date: Tue, 1 Aug 2006 15:10:57 +1000 From: Q <qdolan@gmail.com> To: freebsd-hackers@freebsd.org Subject: Req: Help tracking possible kernel memory leak. Message-ID: <F8BD57AF-976A-4DDE-BF9D-9A9F1B0C530C@gmail.com>
next in thread | raw e-mail | index | archive | help
(Resent: Apologies if this is a duplicate)
Hi,
I was wondering if someone could help point me in the direction of
how to go about trying to resolve what I assume to be a memory leak
in FreeBSD 6.x.
I have two database servers, one running 6.0 the other 6.1, both are
running PostgreSQL. They both have 4gig of memory, running a generic
kernel with the following sysctl's tweaked:
kern.ipc.semmni=128
kern.ipc.semmns=512
kern.ipc.semume=100
kern.ipc.semmnu=256
kern.maxdsiz=1073741824
kern.dfldsiz=1073741824
kern.maxssiz=134217728
kern.ipc.shmmax=536870912
kern.ipc.shmall=262144
kern.ipc.shm_use_phys=1
net.inet.udp.maxdgram=63535
They are both used for processing an extremely large amount of data
collected from various sources every 5 minutes and therefore perform
virtually identical workloads. All processing is performed locally,
there is only 1 external connection being made to the database, once
a day to retrieve a small selection of data.
Both machines are showing a constantly growing 'Active' memory usage
in 'top' until they reach a point where the database performance drop
dramatically and disk IO goes through the roof. If the machine is
left to run in this state it appears to eventually just hang (at
least this is what happened to one of the machines).
Most recently one of the servers (running 6.1) had an "Active Memory"
total of 1.6Gig, database performance was significantly worse than
normal, and disk io was dramatically higher. Queries that previously
took a few seconds were taking several minutes.
Using vmstat, ps and top, along with restarting the database I was
unable to find anything that would indicate a user space leak
consuming this 1.6Gig of memory. The only way I found to free this
memory and ultimately restore the database performance was to reboot
the machine. Which resulted in the "Active memory" resetting back to
virtually nothing and proceeded to slowly climb again (after 4 days
one server is up to about 380Mb, and the other server is at about
680Mb after 7 days) and growing at an almost linear rate.
If someone would be so kind as to provide some advice on how to track
down this issue it would be much appreciated.
Having to reboot these machines every 15 days is simply not a viable
option.
--
Seeya...Q
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
_____ / Quinton Dolan - qdolan@gmail.com
__ __/ / / __/ / /
/ __ / _/ / / Gold Coast, QLD, Australia
__/ __/ __/ ____/ / - / Ph: +61 419 729 806
_______ /
_\
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?F8BD57AF-976A-4DDE-BF9D-9A9F1B0C530C>