Date: Fri, 17 Jan 2003 15:27:02 -0800 From: Terry Lambert <tlambert2@mindspring.com> To: Yury Tarasievich <grog@grsu.by> Cc: freebsd-hackers@freebsd.org Subject: Re: FreeBSD firewall for high profile hosts - waste of time ? Message-ID: <3E289146.1EBBB3B8@mindspring.com> References: <20030116124254.J9642-100000@mail.econolodgetulsa.com> <3E2739D1.5402B7A6@mindspring.com> <3E282DE5.4090209@grsu.by>
next in thread | previous in thread | raw e-mail | index | archive | help
Yury Tarasievich wrote: > Terry Lambert wrote: > >FreeBSD is actually adding pointers and other complexity to its > >stack > > > [...etc.] > > So you are referring to common features of stacks of both 4.* and 5.*, > right? Mostly 5.x for the mbuf metadata modifications; they haven't been back-ported to the 4.x branch, AFAIK. The IPv4 IPSEC problems for no-IPSEC-using connections are 4.x and 5.x both, and so are the timer and the livelock issues. > As far as I understand the matter, this all have to be (and I > guess actually is) provable. Yes. Duke University and Rice University have both done papers in this area (see "Scala server" or "lazy receiver processing" in a search engine; I recommend http://citeseer.nj.nec.com/cs for technical papers). > Now, you were saying that 5.* is even worse, I quote: > > FreeBSD 5.x network performance is really poor, relative to 4.x; I do > > not recommend using FreeBSD 5.x in a production environment. > > Then I wonder *what* were the reasons for not working on matters pointed > out??? The SMP-ification and the use of the interrupt threads model, etc., adds latency to packet processing. Additional latency translates into additional pool retention time, latency in response after the processing finally happens, and reduced reuse rate. The number that was posted by someone who had done benchmarking as -65% for flat network performance, going from 4.x to 5.x. The reality is that this is probably a little exaggerated, if you take into account common real-world operations, instead of benchmarking. One actual good benchmark is "connections per second"; using a combination of soft interrupt coelescing, hard interrupt coelescing, and lazy receiver processing, you get the following: Connections per second: Well tuned stock FreeBSD Well tuned modified FreeBSD ---------------------------- --------------------------- 7,200 32,126 This is on a FreeBSD *without* a SYN cache. Most of the reasons this stuff is not in FreeBSD is NIH (not being the pet research project of a committer), license, the need to "productize" the code from research, etc.. For the complaints about scalability... Linux has a project that they are very proud of, in order to obtain 10,000 simultaneous TCP connections. With respect, I personally achieved 1,600,000 simultaneous TCP connections on a modified FreeBSD box with 4G of RAM. During this process, I found a credentials reference count overflow bug (since fixed in FreeBSD), which occurred on close, after opening more than 32,763 connections in one process. No one else reported this bug, so I have to assume that no one else ever ran FreeBSD up to that number of connections, before. So... the primary reason is that no one is using FreeBSD under the loads necessary to cause the problems to exhibit themselves. You have to have a need in order to be interested in a way of satifying a need. 8-). > You were also saying in same post that: > > > The fixes are mostly > > simple, but for whatever reason, they never make it into FreeBSD > > proper (my theory is that the developement focus is heavily skewed > > to general purpose processing, rather than network processing). > > And throttling the "FlagFeature"??? That sort of thing has to be there, for FreeBSD to be interesting as a reasearch OS, so that additional work occurs on the platform; that's pretty much a given. You wouldn't have someone as well-known as Sam Leffler donating code, if that code was unlikely to get in, since it would be a waste of his time and effort (the same reason some people have left the project, actually). So even if you don't like "feature creep" or "bloat", when it impacts top end performance, top end performance really doesn't matter to most people who are doing the coding (i.e. how many OC3's do you have to your computer? How many would you need to have to saturate even a single gigabit ethernet?). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3E289146.1EBBB3B8>