From owner-freebsd-hackers Sun Jan 19 12:08:21 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id MAA16711 for hackers-outgoing; Sun, 19 Jan 1997 12:08:21 -0800 (PST) Received: from root.com (implode.root.com [198.145.90.17]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id MAA16697 for ; Sun, 19 Jan 1997 12:08:16 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by root.com (8.7.6/8.6.5) with SMTP id MAA23398; Sun, 19 Jan 1997 12:05:43 -0800 (PST) Message-Id: <199701192005.MAA23398@root.com> X-Authentication-Warning: implode.root.com: Host localhost [127.0.0.1] didn't use HELO protocol To: Luigi Rizzo cc: hackers@freebsd.org Subject: Re: cleaning up TIME_WAIT states (fwd) From: David Greenman Reply-To: dg@root.com Date: Sun, 19 Jan 1997 12:05:43 -0800 Sender: owner-hackers@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >> When BSDI upgraded their stack this past summer to make their Web >> server "faster", they moved all the connections in the TIME_WAIT >> state onto their own queue, to get them out of the tcp_slowtimo() >> function. I'd bet that's the majority of the CPU savings right >> there. (I've always thought that the BSD tcp_{slow,fast}timo() >> functions must be one of the biggest bottlenecks on a busy system.) >> >>> Another technique I've seen thrown around was to keep track of the >>> timeouts using a heap. The idea is that the CPU overhead is mostly >>> from the search times, if you can begin to bound that search time even >>> when the list becomes huge due to all the TIME_WAIT connections, it >>> would help tremendously. I said: > This was an issue prior to when FreeBSD had PCB hashing. It's not a >significant issue now, however. I think the extra overhead in moving the >PCB to the other queue would negate any benefit. Of course I entirely missed that the point above is to reduce the search time for slow timeout processing. I should have actually read the message before responding. :-) We actually have PCBs threaded on two queues right now. A linear queue for everything (for doing linear searches, especially in the timeout case), and a hashed list for hashed PCB lookups. Perhaps TIME_WAIT connections could be moved off the linear queue and onto the tail of a (sorted) TIME_WAIT queue. This would mean that all other places where the linear search occurs would have to be made aware of the other queue, but this case is rare and I don't think the overhead would be significant. So...it's an interesting idea that I'll put on my whiteboard. Thanks! -DG David Greenman Core-team/Principal Architect, The FreeBSD Project