Date: Fri, 13 Feb 2004 14:20:43 -0600 From: Matt Freitag <mpf@inodes.us> To: Bill <freebsd@gardrail.com>, freebsd-hackers@freebsd.org Subject: Re: Obtaining 75k (active) concurrent tcp sessions.. Message-ID: <402D319B.1090809@inodes.us> In-Reply-To: <20040213001513.M60613@gardrail.com> References: <20040213001513.M60613@gardrail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
For FreeBSD to support that many concurrent connections some kernel values must be tweaked. Namely, you'll need to set "kern.ipc.nmbclusters=81920" in loader.conf as it's a read-only oid. Another route is to add "options NMBCLUSTERS=81920" into your kernel and compile/install (if it's too high/too low you'll have to compile and install again, much more cumbersome than setting it in loader.conf). Also you'll need to set "kern.maxfilesperproc=81920" if you'll be having the same daemon receive each of those tcp connections, or else you'll run out of fd's fairly quick. I don't think it'll hurt to bump kern.maxfiles either. Now, 81920 might not be the integral value you want, you can obviously make it more or less Personally I've had a mixed experience with that, as I've had kernel panics when setting nmbclusters above that threshold. Your own mileage may vary. Another variable to keep an eye is kern.ipc.somaxconn as this will dictate your connection queue. If there's going to be a flood of connections, the default value will be seriously anemic in this respect. If you run a stateful ipfw firewall on this machine as well, and use keep-alive connections, then be sure your "net.inet.ip.fw.dyn_max" is high enough to allocate dynamic rulesets for them all. Personally when dealing with machines handling connection loads like that, I have a tendancy to turn down "net.inet.tcp.sendspace" so it uses less memory per tcp connection. If you're handling this many connections, turning this up isn't a good idea. The "net.inet.tcp.recvspace" default should be ok for this, though if you really do push >75k connections concurrent, turning it down wouldn't be such a bad idea imho. One last word of advice, "man 7 tuning". -mpf --- Only dimly aware of a certain unease in the air --- Bill wrote: >What steps would I need to take in order to obtain 75,000 concurrent TCP >sessions on a FreeBSD 5.2 system running on the following hardware: > >dual xenon 3ghz 1mb cache processors >2 gigs of memory >two dual port fibre gigabit nic's >1 onboard copper 10/100 nic > >I read a post that was sent to freebsd-hackers, which mentioned an >individual was able to obtain 1.6 million concurrent tcp sessions, so I >assume it's possible. > >My goal is to setup a server, which is capable of accepting at least 75k tcp >connections to perform some firewall stress tests at work. Given that >information on this subject is quite scarce, I thought I'd post this >question and see what type of response I get back. > >Any assistance or suggestions would be greatly appreciated, > >Thanks in advance, > >-=-Bill-=- >_______________________________________________ >freebsd-hackers@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?402D319B.1090809>