Date: Thu, 3 Feb 2005 01:24:53 -0800 From: Mark Edwards <mark@antsclimbtree.com> To: J65nko BSD <j65nko@gmail.com> Cc: "'questions@freebsd.org'" <questions@FreeBSD.ORG> Subject: Re: swap getting consumed Message-ID: <08b2b6e756939878281f8659ca178c5b@antsclimbtree.com> In-Reply-To: <19861fba0501231927799975a3@mail.gmail.com> References: <9EB6F666-6D91-11D9-9D03-000393A5ED5E@antsclimbtree.com> <19861fba05012315482bf084a7@mail.gmail.com> <F6D559FD-6DA7-11D9-9D03-000393A5ED5E@antsclimbtree.com> <19861fba0501231927799975a3@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Jan 23, 2005, at 7:27 PM, J65nko BSD wrote: > On Sun, 23 Jan 2005 17:33:35 -0800, Mark Edwards > <mark@antsclimbtree.com> wrote: >> On Jan 23, 2005, at 3:48 PM, J65nko BSD wrote: >> >>> On Sun, 23 Jan 2005 14:53:38 -0800, Mark Edwards >>> <mark@antsclimbtree.com> wrote: >>>> I have posted about this problem a couple of times with not much >>>> response, I'm afraid, but here is a different take on it perhaps. I >>>> have a 4.10p5 running and for roughly the last two months my swap >>>> space >>>> has been getting eaten uncontrollably. The only clue I have is that >>>> it >>>> resets when I restart Apache (1.3.33), and so if I set up a cron job >>>> to >>>> restart Apache every day or hour the problem is contained: >>>> >>>> https://secure.antsclimbtree.com/mrtg/ants.swap-year.png >>>> >>>> Oddly, my physical memory doesn't seem to be generally affected: >>>> >>>> https://secure.antsclimbtree.com/mrtg/ants.ram-year.png >>>> >>>> I have 128MB of physical RAM and 384MB of swap. >>>> >>>> The only clue I have with Apache is a lot of this in >>>> /var/log/messages: >>>> >>>> Jan 21 18:25:06 lilbuddy /kernel: pid 68446 (httpd), uid 80: exited >>>> on >>>> signal 6 >>>> >>>> But I'm not sure what to make of it. The only thing Google turns up >>>> related to those messages are notes about CodeRed virus attacks, but >>>> Apache 1.3.33 is suppose to address that issue. >>>> >>>> Does anyone have any suggestion of how to attack this problem? I'm >>>> totally stumped. >>>> >>> >>> You don't give much background info about the server. Which modules >>> Apache is using, mod_perl, mod_ssl?, which applications it runs, >>> WebMail? >>> >>> Cannot you see in the Apache logs, which pages are being served >>> before >>> the time of the signal 6 messages? >>> Google for "Apache+memory+leak" and include every module or app you >>> are running.. >>> >>> Gather more info what the system is doing exactly. Write a script for >>> cron to run "netstat -m", "sockstat -4", "ps -ax". at regular times >>> and have the output mailed to you. >>> >>> BTW 128 MB memory for a webserver is not much these days ;) >>> >>> =Adriaan= >> >> Thanks for the reply. I am running apache+mod_ssl-1.3.33+2.8.22, with >> mod_php4 and mod_dav. >> >> As far as web apps, I run Squirrelmail, phpbb2, MT-2.661, Mailman, >> analog, webalizer. The weird thing is that I ran all of those except >> phpbb2 for years with no problem. Perhaps it is phpbb2's doing, but I >> can't find a correlation between that and the signal 6 messages, and >> in >> any case phpbb2 hardly gets used. I can watch swap go up and up >> without anyone actually using phpbb2. >> >> Doing Google on Apache+memory+leak as you suggest leads to people >> describing similar situations, where Apache eats more and more swap >> and >> has to be restarted, but I can't find anyone posing a solution to the >> problem. >> >> I will just keep monitoring it I guess, looking for clues. 128MB >> might >> not be much, but it has worked fine for running essentially the same >> set of software for years, and all of a sudden there's a problem. I >> wish I could tie it to some specific update or change, but I can't. >> The closest I can come is turning on softupdates and installing >> phpbb2, >> and neither of those seems to correlate. >> >> Very odd... > > Have you seen this security warning for phpbb? > http://www.auscert.org.au/render.html?it=4653 Just want to close this thread by saying that I seem to have solved the problem, although I'm not 100% sure what did it. I'm pretty sure it was a recompile of mm-1.3.1. I tried this because I noticed that it was the only required component for apache that was last updated in mid-November, right around when this nonsense started. All the other required ports had been updated more recently, so I figured that perhaps something was amiss when that one got compiled in November. Plus, the fact that mm deals with memory management. That appears to have been the culprit. I'm running along with swap moving between the 100MB and 200MB range, with no big spikes, and RAM at a steady 64MB out of 128MB. Performance is steady and back to normal. Hurrah! A 180Mhz Pentium Pro with 128MB RAM is no speed demon, mind you, but it serves web pages, hosts email, and routes pretty damn fine for a machine that was literally in the garbage when I rescued it. PHP is a bit slow, but still totally acceptable, and faster than the 256Kbps outgoing connection. I also doubled my swap from 400MB on one drive to 800MB between two drives. And I stripped down my kernel a bit, removing lots of options that aren't used on this box. Those might have affected things as well, but my hunch is it was mm-1.3.1 that was out of whack. Anyway, all's well that ends well. Thanks!
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?08b2b6e756939878281f8659ca178c5b>