From owner-freebsd-arch@FreeBSD.ORG Fri Oct 22 15:45:19 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from green.homeunix.org (freefall.freebsd.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id D8F7516A4CE; Fri, 22 Oct 2004 15:45:18 +0000 (GMT) Received: from green.homeunix.org (green@localhost [127.0.0.1]) by green.homeunix.org (8.13.1/8.13.1) with ESMTP id i9MFjIFg094553; Fri, 22 Oct 2004 11:45:18 -0400 (EDT) (envelope-from green@green.homeunix.org) Received: (from green@localhost) by green.homeunix.org (8.13.1/8.13.1/Submit) id i9MFjHYG094552; Fri, 22 Oct 2004 11:45:17 -0400 (EDT) (envelope-from green) Date: Fri, 22 Oct 2004 11:45:17 -0400 From: Brian Fundakowski Feldman To: Andre Oppermann Message-ID: <20041022154517.GN1072@green.homeunix.org> References: <4177C8AD.6060706@freebsd.org> <71C3A1EA-238F-11D9-9171-000A95C705DC@chittenden.org> <41780672.6AAC705F@freebsd.org> <417923BF.661D2A6D@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <417923BF.661D2A6D@freebsd.org> User-Agent: Mutt/1.5.6i cc: freebsd-arch@freebsd.org Subject: Re: Removing T/TCP and replacing it with something simpler X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Oct 2004 15:45:19 -0000 On Fri, Oct 22, 2004 at 05:14:07PM +0200, Andre Oppermann wrote: > Sean Chittenden wrote: > > > > >>> However something like T/TCP is certainly useful and I know of one > > >>> special > > >>> purpose application using it (Web Proxy Server/Client for high-delay > > >>> Satellite > > >>> connections). > > >> > > >> Actually, there are two/three programs that I know of that use it. > > >> memcached(1), which found a fantastic decrease in its benchmarks. > > >> Here's an excerpt from the following link: > > >> > > >> http://lists.danga.com/pipermail/memcached/2003-August/000111.html > > > > > > I think you got something wrong here. T/TCP is never ever mentioned > > > in this. Memcached is not using T/TCP as far as I can see. > > > > It's not, but I thought setsockopt(2) w/ TCP_NOPUSH enabled the use of > > T/TCP in that there was no 3WHS performed on a TCP_NOPUSH'ed > > connection. > > No, it is not. T/TCP will only be used if you use sendto(), have T/TCP > globally enabled on the machine and the server supports it too. > > TCP_NOPUSH was introduced together with or some time after T/TCP to > change the behaviour how tcp_output() pushes non-full packets on the > wire. It pretty closely related to the same purpose as TCP_CORK. > > > >> and an internal reverse proxy server/modified apache that I've hacked > > >> together (reduces latency in a tiered request hierarchy a great deal, > > >> on order of the benchmarks from above). > > > > > > What syscall do you use to get to the other side in your reverse proxy? > > > > On the client, sendto()/read(). On the server, setsockopt() + write(). > > Ok, then you are indeed using T/TCP (provided you have enabled it on > both machines). The setsockopt() optimizes packet sending on the server > but otherwise doesn't have anything to do with T/TCP. > > > > I'm not sure if I can follow you here. TCP_CORK deals with the > > > different > > > behaviour of connections with Nagle vs. TCP_NODELAY. TCP_CORK allows > > > to > > > avoid the delays of Nagle by corking (sort of blocking) the sending of > > > packets until you are done with write()ing to the socket. Then the > > > connection is uncorked and all data will be sent in one go even if it > > > doesn't fill an entire packet. Sort of an fsync() for sockets. There > > > are no security implications with TCP_CORK as far as I am aware. > > > > Isn't that what NOPUSH does? Or is it that CORK uses a fully > > established TCP connection, but blocks sending data until the > > connection has been uncorked/flushed? I thought that TCP_CORK had the > > same security implications that NOPUSH does (ie, the lack of a hand > > shake). > > None of it. Neither NOPUSH nor CORK have any security implications. > Those are only with the specification of T/TCP. Blocking the data > is independend of 3WSH. Normally you have Nagle enabled (default) > and when you don't fill an entire packet worth of data it will wait > up to 200ms to send the packet in anticipation of more data from the > socket. This screws the responsiveness of your connection. The first > solution is to turn off Nagle (with TCP_NODELAY) but now you get a > packet for every single write() you do. Fine for telnet and ssh but > not the right thing for a database server. There you don't want the > delay but at the same time you want several successive write()s that > will go in one packet on the wire. Here NOPUSH and CORK come into > play. Why is just tuning the delay a bad solution? -- Brian Fundakowski Feldman \'[ FreeBSD ]''''''''''\ <> green@FreeBSD.org \ The Power to Serve! \ Opinions expressed are my own. \,,,,,,,,,,,,,,,,,,,,,,\