From owner-freebsd-current@FreeBSD.ORG Tue Dec 12 23:03:39 2006 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4367616A4B3 for ; Tue, 12 Dec 2006 23:03:39 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id D988844D01 for ; Tue, 12 Dec 2006 22:29:02 +0000 (GMT) (envelope-from andre@freebsd.org) Received: (qmail 65716 invoked from network); 12 Dec 2006 22:17:51 -0000 Received: from dotat.atdotat.at (HELO [62.48.0.47]) ([62.48.0.47]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 12 Dec 2006 22:17:51 -0000 Message-ID: <457F2D82.6000905@freebsd.org> Date: Tue, 12 Dec 2006 23:30:26 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8b) Gecko/20050217 MIME-Version: 1.0 To: freebsd-current@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Automatic TCP send and receive socket buffer sizing X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2006 23:03:39 -0000 This is a patch adding automatic TCP send and receive socket buffer sizing. Normally the socket buffers are static (either derived from global defaults or set with setsockopt) and do not adapt to real network conditions. Two things happen: a) your socket buffers are too small and you can't reach the full potential of the network between both hosts; b) your socket buffers are too big and you waste a lot of kernel memory for data just sitting around. With automatic TCP send and receive socket buffers we can start with a small buffer and quickly grow it in parallel with the TCP congestion window to match real network conditions. FreeBSD has a default 32K send socket buffer. This supports a maximal transfer rate of only slightly more than 2Mbit/s on a 100ms RTT trans- continental link. Or at 200ms just above 1Mbit/s. With TCP send buffer auto scaling and the default values below it supports 20Mbit/s at 100ms and 10Mbit/s at 200ms. That's an improvement of factor 10, or 1000%. For the receive side it looks slightly better with a default of 64K buffer size. The automatic send buffer sizing patch is currently running on one half of the FTP.FreeBSD.ORG cluster w/o any problems so far. Against this machine with the automatic receive buffer sizing patch I can download at 5.7MBytes per second. Without patch it maxed out at 1.6MBytes per second as the delay bandwidth product became equal to the static socket buffer size without hitting the limits of the physical link between the machines. My test machine is about 35ms from that FTP.FreeBSD.ORG and connected through a moderately loaded 100Mbit Internet link. New sysctl's are: net.inet.tcp.sendbuf_auto=1 (enabled) net.inet.tcp.sendbuf_inc=8192 (8K, step size) net.inet.tcp.sendbuf_max=262144 (256K, growth limit) net.inet.tcp.recvbuf_auto=1 (enabled) net.inet.tcp.recvbuf_inc=16384 (16K, step size) net.inet.tcp.recvbuf_max=262144 (256K, growth limit) The patch is available here (it may apply with some fuzz): http://people.freebsd.org/~andre/tcp_auto_buf-20061212.diff Any tests and test reports are very welcome. -- Andre