From owner-freebsd-questions@FreeBSD.ORG Sun Apr 25 08:27:24 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 57144106564A for ; Sun, 25 Apr 2010 08:27:24 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-qy0-f181.google.com (mail-qy0-f181.google.com [209.85.221.181]) by mx1.freebsd.org (Postfix) with ESMTP id 0CCE88FC19 for ; Sun, 25 Apr 2010 08:27:23 +0000 (UTC) Received: by qyk11 with SMTP id 11so13494954qyk.13 for ; Sun, 25 Apr 2010 01:27:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=1ppFpu4DjgCC4sdiNA7GWnES9SQZ7LjQ5Q0aZIPE7+0=; b=TjDdaoEYpc9UY7Rc43LYuRKklXEm/BgRzfIcVQLoyr4wj6WHxTZMiyUuuLbZSPsvlJ J2skG6Qz0MqrmIln7zStXeDF6B1/aDWnWa1UwCNGni0YGPV2LRFwDpHqGVc8Sink+iab 4JVRNGBqlV6Z8NlPTeVjOrcfHcz4cWoBIhrIg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=wybjwsB0J4MPyOtA+KydNnKI/+JIVnR884BuV9yaZUSIF0Kecetbr5GwNNC1a4s149 w8Ec7mdVMbZj9fJKyGcNToFS2t+grXQLg3tVJy586G9vC/aBXYaSvNVxZMvmElB+UttW DJLUOGcXLfPtQqLdKMdP+iCRRsmzQLuBWE94g= MIME-Version: 1.0 Received: by 10.229.221.84 with SMTP id ib20mr2666251qcb.93.1272184035065; Sun, 25 Apr 2010 01:27:15 -0700 (PDT) Received: by 10.229.99.67 with HTTP; Sun, 25 Apr 2010 01:27:15 -0700 (PDT) In-Reply-To: <4BD3EAF9.2080203@netmusician.org> References: <4BD394BC.7030501@netmusician.org> <4BD3EAF9.2080203@netmusician.org> Date: Sun, 25 Apr 2010 03:27:15 -0500 Message-ID: From: Adam Vande More To: Joe Auty Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-questions@freebsd.org Subject: Re: Advice for finding a leaky Apache (probably PHP) process X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Apr 2010 08:27:24 -0000 On Sun, Apr 25, 2010 at 2:10 AM, Joe Auty wrote: > Well, I'm fishing. It is also possible that I'm seeing a denial of service > attack or something, but the result is my Apache processes ballooning and > CPU usage for some of my httpd processes going up to around 100%. There are > several PHP apps running on the server, so it is very hard to pinpoint > things to one app, which is part of the problem. > > I can actually see the memory growth, I can sit and watch top and see my > memory consumption balloon until the machine swaps and then just grinds to a > halt. Sometimes it gets so bad that I'm forced to killall -9 httpd just to > bring the machine back to life. > > What are some good techniques for trying to ascertain whether a particular > web app is being exploited for some sort of attack? Since I had to recompile > PHP and all of my PHP extensions is there a possibility that a particular > extension is causing memory consumption to balloon? A long time ago I had an > attack on a very old version of WordPress. I found this via my Apache > server-status page, but it was sort of a pure fluke that I did find this. > Surely there has to be better ways to connect httpd processes to pages that > are being served? > > I wish that the machine was a little more responsive when I get to this > point so that I can ktrace the processes... > well if you're just looking for some general advice, I'll tell you what I do and you take what you like. - I don't use Apache anymore for several reasons. Speed and configuration are just two of them. A couple of the more popular http server alternative are lighttpd and nginx. I prefer nginx. Both very small and very fast compared to apache. I didn't believe the difference others claimed until I ran the benchmarks myself. Not only where they much faster and lighter than apache, they were also more reliable especially under load. - jails are a lightweight method of isolating insecure apps. Get to know them and use them extensively. PHP apps are well known for this type of thing, and if you're going to run them it's very wise to make sure you're safe while doing so. I like to use sysutils/ezjail to create full jails for each php app each with it's own install of an http server and php. The root host would run a reverse proxy of your choosing and direct http requests to the appropriate jail. jails can be assigned cpu sets as well eg if your cpu has 4 cores, a jail can be bound to 1 or more of them. jails also have their own process list so top is not so difficult to decipher(your unresponsive system could be helped by this and you can also be stricter in your php.ini mem limits per jail). This method will use a little more memory, but easily it's the best time-wise from an administration approach that I've found if you have a lot of different types of clients. - use ports-mgmt/portaudit regularly. - for debugging php, if truss and ktrace aren't helping, use valgrind. -- Adam Vande More