From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 13 12:20:48 2004 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 43A5016A4CE for ; Fri, 13 Feb 2004 12:20:48 -0800 (PST) Received: from mail.gulfgate-inc.com (mail.gulfgate-inc.com [64.1.98.180]) by mx1.FreeBSD.org (Postfix) with ESMTP id E4BA743D1F for ; Fri, 13 Feb 2004 12:20:47 -0800 (PST) (envelope-from mpf@inodes.us) Received: (qmail 92157 invoked by uid 85); 13 Feb 2004 20:19:06 -0000 Received: from mpf@inodes.us by linkdead.beatlab.org by uid 89 with qmail-scanner-1.15 (uvscan: v4.1.60/v4248. spamassassin: 2.x. Clear:. Processed in 1.042371 secs); 13 Feb 2004 20:19:06 -0000 Received: from unknown (HELO inodes.us) (192.168.0.4) by 192.168.0.40 with SMTP; 13 Feb 2004 20:19:04 -0000 Message-ID: <402D319B.1090809@inodes.us> Date: Fri, 13 Feb 2004 14:20:43 -0600 From: Matt Freitag User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20031013 Thunderbird/0.3 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Bill , freebsd-hackers@freebsd.org References: <20040213001513.M60613@gardrail.com> In-Reply-To: <20040213001513.M60613@gardrail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Obtaining 75k (active) concurrent tcp sessions.. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Feb 2004 20:20:48 -0000 For FreeBSD to support that many concurrent connections some kernel values must be tweaked. Namely, you'll need to set "kern.ipc.nmbclusters=81920" in loader.conf as it's a read-only oid. Another route is to add "options NMBCLUSTERS=81920" into your kernel and compile/install (if it's too high/too low you'll have to compile and install again, much more cumbersome than setting it in loader.conf). Also you'll need to set "kern.maxfilesperproc=81920" if you'll be having the same daemon receive each of those tcp connections, or else you'll run out of fd's fairly quick. I don't think it'll hurt to bump kern.maxfiles either. Now, 81920 might not be the integral value you want, you can obviously make it more or less Personally I've had a mixed experience with that, as I've had kernel panics when setting nmbclusters above that threshold. Your own mileage may vary. Another variable to keep an eye is kern.ipc.somaxconn as this will dictate your connection queue. If there's going to be a flood of connections, the default value will be seriously anemic in this respect. If you run a stateful ipfw firewall on this machine as well, and use keep-alive connections, then be sure your "net.inet.ip.fw.dyn_max" is high enough to allocate dynamic rulesets for them all. Personally when dealing with machines handling connection loads like that, I have a tendancy to turn down "net.inet.tcp.sendspace" so it uses less memory per tcp connection. If you're handling this many connections, turning this up isn't a good idea. The "net.inet.tcp.recvspace" default should be ok for this, though if you really do push >75k connections concurrent, turning it down wouldn't be such a bad idea imho. One last word of advice, "man 7 tuning". -mpf --- Only dimly aware of a certain unease in the air --- Bill wrote: >What steps would I need to take in order to obtain 75,000 concurrent TCP >sessions on a FreeBSD 5.2 system running on the following hardware: > >dual xenon 3ghz 1mb cache processors >2 gigs of memory >two dual port fibre gigabit nic's >1 onboard copper 10/100 nic > >I read a post that was sent to freebsd-hackers, which mentioned an >individual was able to obtain 1.6 million concurrent tcp sessions, so I >assume it's possible. > >My goal is to setup a server, which is capable of accepting at least 75k tcp >connections to perform some firewall stress tests at work. Given that >information on this subject is quite scarce, I thought I'd post this >question and see what type of response I get back. > >Any assistance or suggestions would be greatly appreciated, > >Thanks in advance, > >-=-Bill-=- >_______________________________________________ >freebsd-hackers@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > >