From owner-freebsd-questions@FreeBSD.ORG Wed Apr 26 15:41:23 2006 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 10D6916A402 for ; Wed, 26 Apr 2006 15:41:23 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id 96A6F43D45 for ; Wed, 26 Apr 2006 15:41:22 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id A559C5E3B; Wed, 26 Apr 2006 11:41:21 -0400 (EDT) X-Virus-Scanned: amavisd-new at codefab.com Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PUiHfYcMjjF4; Wed, 26 Apr 2006 11:41:20 -0400 (EDT) Received: from [199.103.21.238] (pan.codefab.com [199.103.21.238]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by pi.codefab.com (Postfix) with ESMTP id AF4225CBF; Wed, 26 Apr 2006 11:41:20 -0400 (EDT) In-Reply-To: <20060426145037.GA6516@dan.emsphone.com> References: <18e02bd30604260709m6a0c33dp972a4f1d0a0e17ea@mail.gmail.com> <20060426145037.GA6516@dan.emsphone.com> Mime-Version: 1.0 (Apple Message framework v749.3) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <7BB18A0B-DF80-45B6-89B3-A7D1819FA71F@mac.com> Content-Transfer-Encoding: 7bit From: Charles Swiger Date: Wed, 26 Apr 2006 11:41:19 -0400 To: Dan Nelson X-Mailer: Apple Mail (2.749.3) Cc: Andrew Pantyukhin , FreeBSD Questions Subject: Re: Huge (100k+) number of processes X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Apr 2006 15:41:23 -0000 On Apr 26, 2006, at 10:50 AM, Dan Nelson wrote: >> The problem is that kern.maxproc=200000 line in /boot/loader.conf >> only raised the real value to 21576. Is that authoritative, or should >> I just try and run 200k processes? I only have 512Mb of RAM, and I >> wonder if it's enough to run 100k+ of, say, /bin/sleep? > > Make sure you have enough RAM to cover those 100k processes. There's > code that limits maxproc to physpages/12 ("Limit maxproc so that kmap > entries cannot be exhausted by processes"). You can try removing that > code, but each process does require a couple of private pages. > Assuming 5 pages, you'll need 2 GB of RAM just for that overhead, > before you take into account any memory your application will > allocate. And then add at least three, maybe four pages-- one for TEXT, one for BSS, one for the heap, and one for the stack) for the most minimal reasonable process, and anything which uses libc is going to run dozens to ~100 pages-- /bin/sleep wants 410K resident, for example. The pure text pages are not duplicated for each process, but even so, you'd need on the order of 20GB or RAM or swap to run 100,000 instances of sleep. FreeBSD scales into the thousands of processes OK, and might do 10K or 20K under extreme circumstances, but that's almost certainly not the right architecture to head towards; a smaller number of persistent server processes along the lines of apache's preforked children or a multithreaded or poll()/select() based concurrent design are going to be better.... -- -Chuck