Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 13 Feb 2004 14:20:43 -0600
From:      Matt Freitag <mpf@inodes.us>
To:        Bill <freebsd@gardrail.com>, freebsd-hackers@freebsd.org
Subject:   Re: Obtaining 75k (active) concurrent tcp sessions..
Message-ID:  <402D319B.1090809@inodes.us>
In-Reply-To: <20040213001513.M60613@gardrail.com>
References:  <20040213001513.M60613@gardrail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
For FreeBSD to support that many concurrent connections some kernel 
values must be tweaked. Namely, you'll need to set 
"kern.ipc.nmbclusters=81920" in loader.conf as it's a read-only oid. 
Another route is to add "options   NMBCLUSTERS=81920" into your kernel 
and compile/install (if it's too high/too low you'll have to compile and 
install again, much more cumbersome than setting it in loader.conf).
Also you'll need to set "kern.maxfilesperproc=81920" if you'll be having 
the same daemon receive each of those tcp connections, or else you'll 
run out of fd's fairly quick. I don't think it'll hurt to bump 
kern.maxfiles either.
Now, 81920 might not be the integral value you want, you can obviously 
make it more or less Personally I've had a mixed experience with that, 
as I've had kernel panics when setting nmbclusters above that threshold. 
Your own mileage may vary.
Another variable to keep an eye is kern.ipc.somaxconn as this will 
dictate your connection queue. If there's going to be a flood of 
connections, the default value will be seriously anemic in this respect. 
If you run a stateful ipfw firewall on this machine as well, and use 
keep-alive connections, then be sure your "net.inet.ip.fw.dyn_max" is 
high enough to allocate dynamic rulesets for them all. Personally when 
dealing with machines handling connection loads like that, I have a 
tendancy to turn down "net.inet.tcp.sendspace" so it uses less memory 
per tcp connection. If you're handling this many connections, turning 
this up isn't a good idea. The "net.inet.tcp.recvspace" default should 
be ok for this, though if you really do push >75k connections 
concurrent, turning it down wouldn't be such a bad idea imho. One last 
word of advice, "man 7 tuning".

-mpf

---
Only dimly aware of a certain unease in the air
---

Bill wrote:

>What steps would I need to take in order to obtain 75,000 concurrent TCP 
>sessions on a FreeBSD 5.2 system running on the following hardware:
>
>dual xenon 3ghz 1mb cache processors
>2 gigs of memory
>two dual port fibre gigabit nic's
>1 onboard copper 10/100 nic
>
>I read a post that was sent to freebsd-hackers, which mentioned an 
>individual was able to obtain 1.6 million concurrent tcp sessions, so I 
>assume it's possible.
>
>My goal is to setup a server, which is capable of accepting at least 75k tcp 
>connections to perform some firewall stress tests at work.  Given that 
>information on this subject is quite scarce, I thought I'd post this 
>question and see what type of response I get back.
>
>Any assistance or suggestions would be greatly appreciated,
>
>Thanks in advance,
>
>-=-Bill-=-
>_______________________________________________
>freebsd-hackers@freebsd.org mailing list
>http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
>To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
>
>  
>




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?402D319B.1090809>