From owner-freebsd-stable@FreeBSD.ORG Mon Oct 15 13:49:33 2007 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1962916A420 for ; Mon, 15 Oct 2007 13:49:33 +0000 (UTC) (envelope-from bill@lefebvre.org) Received: from atl04.ws-e.com (vh00.ws-e.com [69.61.31.90]) by mx1.freebsd.org (Postfix) with ESMTP id C617D13C505 for ; Mon, 15 Oct 2007 13:49:32 +0000 (UTC) (envelope-from bill@lefebvre.org) Received: from lilburn.lefebvre.org (ocee.groupsys.com [66.149.10.161]) by atl04.ws-e.com (8.13.8/8.13.8) with ESMTP id l9FDFd91009329 for ; Mon, 15 Oct 2007 09:15:39 -0400 (EDT) (envelope-from bill@lefebvre.org) Received: from [10.88.88.6] (milton.lefebvre.org [10.88.88.6]) by lilburn.lefebvre.org (8.13.8/8.13.8) with ESMTP id l9FDFbHs057015 for ; Mon, 15 Oct 2007 09:15:37 -0400 (EDT) (envelope-from bill@lefebvre.org) Message-ID: <471367F2.7050303@lefebvre.org> Date: Mon, 15 Oct 2007 09:15:30 -0400 From: William LeFebvre User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <008801c80e66$7be49490$0c00a8c0@Artem> In-Reply-To: <008801c80e66$7be49490$0c00a8c0@Artem> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.4 required=4.0 tests=ALL_TRUSTED autolearn=failed version=3.1.6 X-Spam-Checker-Version: SpamAssassin 3.1.6 (2006-10-03) on lilburn.lefebvre.org X-Scanned-By: MIMEDefang 2.57 on 192.168.0.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (atl04.ws-e.com [69.61.31.90]); Mon, 15 Oct 2007 09:15:40 -0400 (EDT) Subject: Re: Question about 'top' values on memory usage X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Oct 2007 13:49:33 -0000 Artem Kuchin wrote: > Hello! > > Maybe someone with deeper knowledge of the internals of FreeBSD > can clean up something for me (any for many others)^ > > Here are lines from my top: > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > 9258 hordelo_ru 1 4 0 40992K 4260K accept 0 0:00 0.00% httpd > 9257 hordelo_ru 1 44 0 40992K 4296K select 1 0:00 0.00% httpd > 9259 hordelo_ru 1 4 0 40992K 4292K select 1 0:00 0.00% httpd > > As you see, 'size' is the same for all processes, while RES varies. > > As i understand, the real memory taken by a process is RES and SIZE include > a bunch of shares .so libs, so, if more httpd's started each will take > only about 4300K more, so, 100 https will take 430000K to run, right? > > Another question is that is httpd uses threads (as provided by FreeBSD) > starting a new thread will or will not copy executable copy and data? > Basically, > will a new thread eat another 4300K or just a little bit for its data? > SIZE is the total amount of virtual memory that a process has allocated. This includes text, data, and stack. It also includes all the stuff that's shared with other processes (mostly through the use of shared libraries). RES is the amount of physical memory in use by the process and will only include that part of a process's virtual memory space which is currently allocated in physical memory. Unfortunately, freebsd does not appear to track the amount of shared virtual memory for each process. It could be obtained by walking through all the pages in a process's vm map, but that would really slow top down. I don't know of any freebsd utility that would give that information for an individual process. But hey, if it's out there somewhere where it is easy to grab, I would be very happy to add it to top. > All this i need to calculate maximum possible number of https i can run > on a box > with certain amount of memory and select proper MPM for Apache. > Somehow, i could not find any practical info on this regarding FreeBSD. That's because the answer isn't as straightforward as you want it to be. There are actually two things you need to worry about: swap space and physical memory. The amount of swap space will place a hard upper bound on the number of httpd processes you can run before sbrk (and thus malloc) start failing. In order to determine that amount you are going to have to see how much extra swap is gobbled up by each new process. But in actuality that swap is only taken when needed, and if a data page remains untouched by the process then it won't need to allocate swap (backing store) for that page. At least that's my limited understanding of how the freebsd VM model works. But I would argue that long before you hit that hard upper limit you will hit a much more serious practical limit in just how much physical memory is adequate for the environment. And that doesn't really limit the number of processes, rather it limits how many of them can be active at a given time. You're not just going to be able to sit down and plug numbers in to a formula and say "voila!". You will have to observe how httpd performs in your particular environment to see how many page faults per second it generates and decide for yourself the point at which X pf/s is too much. Personally, based on my experience, I would be more concerned with the amount of available cpu cycles than memory. In my experience, once you run out of idle time on a web server you have exceeded its capacity to serve pages. In that situation it doesn't matter how many httpd processes there are, the system is still not able to keep up with demand. And that will probably happen before the system starts thrashing from limited memory. Bill LeFebvre