From owner-freebsd-net Wed Sep 5 16:54:57 2001 Delivered-To: freebsd-net@freebsd.org Received: from silby.com (cb34181-a.mdsn1.wi.home.com [24.14.173.39]) by hub.freebsd.org (Postfix) with ESMTP id DD0EA37B407 for ; Wed, 5 Sep 2001 16:54:53 -0700 (PDT) Received: (qmail 11773 invoked by uid 1000); 5 Sep 2001 23:54:53 -0000 Received: from localhost (sendmail-bs@127.0.0.1) by localhost with SMTP; 5 Sep 2001 23:54:53 -0000 Date: Wed, 5 Sep 2001 18:54:53 -0500 (CDT) From: Mike Silbersack To: Terry Lambert Cc: "Vladimir A. Jakovenko" , Subject: Re: SO_REUSEPORT on unicast UDP sockets In-Reply-To: <3B95EE4A.EF204095@mindspring.com> Message-ID: <20010905183432.Y11391-100000@achilles.silby.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-net@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Wed, 5 Sep 2001, Terry Lambert wrote: > Mike Silbersack wrote: > > The client TIME_WAIT problem is more an issue for port reuse; > for a 2MSL timer in the standard 60 second range, this will > basically limit you to 65535/60, or ~1000 outbound connections > a second per IP address, as a sustained rate, with a total > outstanding count of 65535 * IP_address_count. > > Unless you set SO_REUSEPORT/SO_REUSEADDR. Hm, I wonder if client-side TIME_WAIT recycling should be made the norm even without those options specified. Of course, that's if it's a real problem; I'm not sure that many apps would be affected. Oh, I think there's one other bug with the current code; it can't seem to find free ports if they're surrounded by sockets in the TIME_WAIT state already. I think that was the case, at least... when I ran into these problems myself I decided not to pursue it and just concentrated on how server-side time_wait recycling was working. > > Maybe net or openbsd handle these situations better, I'll have > > to check later. > > I doubt it. Until I did testing on 4.3, no one had really > run over 32,766 open sockets in a production server, since at > that point, the ucred reference count overflowed, which would > result in some strange and very hard to identify crashes, when > closing those connections. True, true. > NetBSD and OpenBSD have some hacks on the server side of the > scaling problem (e.g. they have each implemented a SYN cache, > which is OK as far as it goes, but is really inferior to the > SYN cache and rate halving algorithm code (also against FreeBSD) > out of the Pittsburgh Supercomputing Center. What's the primary difference between the psc syn cache and netbsd one? > One interesting thing about the SYN cache code is that it could > use the tcptmpl allocation until it saw the ACK (or even the > first data, as was suggested by some of the researchers at that > startup in India, a while back, though that's very aggressive). Well, it still could; the pointer's there if you want to associate it with a mbuf. :) Waiting for data to arrive before setting up the socket sounds like it's not so much part of defense against a syn flood as part of reducing kernel -> userland switches. I suspect accept filters + a normal syn cache would achieve the same purpose. > Mostly, you aren't going to see the hashing on both source and > detination IP's and ports -- what you'd see in an L2/L3 switch, > if you were building one -- which would let you reuse the local > pair, so long as it was associated with a different remote pair. > > That's probably the real long term fix, if there is one. > > -- Terry Or we could just rig something up. :) But, since it's probably not a problem in many cases, I'm not going to worry about it for a while. Mike "Silby" Silbersack To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message