From owner-freebsd-current@FreeBSD.ORG Fri Dec 15 00:55:50 2006 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2D6F716A40F for ; Fri, 15 Dec 2006 00:55:50 +0000 (UTC) (envelope-from chrcoluk@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id BF7A143CA5 for ; Fri, 15 Dec 2006 00:53:53 +0000 (GMT) (envelope-from chrcoluk@gmail.com) Received: by nf-out-0910.google.com with SMTP id x37so950856nfc for ; Thu, 14 Dec 2006 16:55:14 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=IgZhTrhEMQgre6lM7qKiq1vg9GPuAWR+cVhlsmYlNsuy3QB4NsOkIVuoY3imkQ0sbcFjjGR+woV2167uDbmlpobQFEMn+EKMFfOZV1XH0Yzs3IP/3AGFyHyqXhE++csi/mk0pSfsA4rA6EvM9kBXpQ93WTQGkcazI73eEglkHl0= Received: by 10.82.138.6 with SMTP id l6mr29568bud.1166144114109; Thu, 14 Dec 2006 16:55:14 -0800 (PST) Received: by 10.82.134.15 with HTTP; Thu, 14 Dec 2006 16:55:13 -0800 (PST) Message-ID: <3aaaa3a0612141655x72c1905cw7fc8f415e08d70b8@mail.gmail.com> Date: Fri, 15 Dec 2006 00:55:13 +0000 From: Chris To: "Andre Oppermann" In-Reply-To: <45812CDE.7000103@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <457F2D82.6000905@freebsd.org> <3aaaa3a0612131706w5ae75edcvadd7958274a1e2e2@mail.gmail.com> <45812CDE.7000103@freebsd.org> Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: Automatic TCP send and receive socket buffer sizing X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Dec 2006 00:55:50 -0000 On 14/12/06, Andre Oppermann wrote: > Chris wrote: > > On 12/12/06, Andre Oppermann wrote: > >> This is a patch adding automatic TCP send and receive socket buffer > >> sizing. > >> Normally the socket buffers are static (either derived from global > >> defaults > >> or set with setsockopt) and do not adapt to real network conditions. Two > >> things happen: a) your socket buffers are too small and you can't > >> reach the > >> full potential of the network between both hosts; b) your socket > >> buffers are > >> too big and you waste a lot of kernel memory for data just sitting > >> around. > >> > >> With automatic TCP send and receive socket buffers we can start with a > >> small > >> buffer and quickly grow it in parallel with the TCP congestion window > >> to match > >> real network conditions. > >> > >> FreeBSD has a default 32K send socket buffer. This supports a maximal > >> transfer rate of only slightly more than 2Mbit/s on a 100ms RTT trans- > >> continental link. Or at 200ms just above 1Mbit/s. With TCP send buffer > >> auto scaling and the default values below it supports 20Mbit/s at 100ms > >> and 10Mbit/s at 200ms. That's an improvement of factor 10, or 1000%. > >> For the receive side it looks slightly better with a default of 64K > >> buffer > >> size. > >> > >> The automatic send buffer sizing patch is currently running on one > >> half of > >> the FTP.FreeBSD.ORG cluster w/o any problems so far. Against this > >> machine > >> with the automatic receive buffer sizing patch I can download at > >> 5.7MBytes > >> per second. Without patch it maxed out at 1.6MBytes per second as the > >> delay > >> bandwidth product became equal to the static socket buffer size > >> without hitting > >> the limits of the physical link between the machines. My test machine > >> is about > >> 35ms from that FTP.FreeBSD.ORG and connected through a moderately > >> loaded 100Mbit > >> Internet link. > >> > >> New sysctl's are: > >> > >> net.inet.tcp.sendbuf_auto=1 (enabled) > >> net.inet.tcp.sendbuf_inc=8192 (8K, step size) > >> net.inet.tcp.sendbuf_max=262144 (256K, growth limit) > >> net.inet.tcp.recvbuf_auto=1 (enabled) > >> net.inet.tcp.recvbuf_inc=16384 (16K, step size) > >> net.inet.tcp.recvbuf_max=262144 (256K, growth limit) > >> > >> The patch is available here (it may apply with some fuzz): > >> > >> http://people.freebsd.org/~andre/tcp_auto_buf-20061212.diff > >> > >> Any tests and test reports are very welcome. > >> > >> -- > >> Andre > > > > Hi does this patch work on 6.x? I used the send patch on 6.x and works > > great please make a 6.x patch thank you and I will happily test. > > No, this patch doesn't work on 6.x. It makes changes to struct tcpcb > to add two additional fields. This requires netstat(1) to be recompiled > and is a ABI change. However I've got a number of requests for 6.x > patch so I may make one anyway. > > -- > Andre > > Please that would be great, the send patch works on 6.x flawlessly and has improved performance enormously it would be a shame to have to wait for 7.x to be production ready before I use it. Thanks Chris