From owner-freebsd-net@FreeBSD.ORG Wed Jun 4 19:59:01 2008 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7DB5E1065676 for ; Wed, 4 Jun 2008 19:59:01 +0000 (UTC) (envelope-from lev@serebryakov.spb.ru) Received: from ftp.translate.ru (ftp.translate.ru [195.131.4.140]) by mx1.freebsd.org (Postfix) with ESMTP id 2D60A8FC2A for ; Wed, 4 Jun 2008 19:59:01 +0000 (UTC) (envelope-from lev@serebryakov.spb.ru) Received: from desktop.home.serebryakov.spb.ru (unknown [89.163.10.141]) (Authenticated sender: lev@serebryakov.spb.ru) by ftp.translate.ru (Postfix) with ESMTPA id 435F213DFAB for ; Wed, 4 Jun 2008 23:42:31 +0400 (MSD) Date: Wed, 4 Jun 2008 23:42:31 +0400 From: Lev Serebryakov X-Mailer: The Bat! (v3.99.3) Professional X-Priority: 3 (Normal) Message-ID: <1761236634.20080604234231@serebryakov.spb.ru> To: net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: samba performance on 1Gig link: how to replace black magic with science? And why TCP windows scaling is not in play? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jun 2008 19:59:01 -0000 Hello, net. I'm setupping file server for WindowsXP/SP3 clients, based on FreeBSD 7.0-Stable nad latest samba port. First of all, Is et these kernel variables: kern.ipc.maxsockbuf=16777216 net.inet.tcp.sendbuf_max=16777216 net.inet.tcp.recvbuf_max=16777216 net.inet.tcp.sendspace=131072 net.inet.tcp.recvspace=131072 All tests are performed with only one client for start. Server and client are linked with 1Gig link, jumbo frames are enabled (9014 bytes) and iperf shows about 900Mbit/s of raw TCP with big windows (about 256Kb for start). Local tests shows 75Mb/s Read/write from/to shared file system with big files. Default configuration of samba gives me only 20Mb/s read and loosy 15Mb/s write. With "socket options = TCP_NODELAY" speed becomes 25/20 Mb/S R/W. Then I start to change SO_RCVBUF/SO_SNDBUF and found (with binary search), that best values are 49152. Yes, this strange values. 65536 is worse, and 32768 is worse too. This magic value gives me about 40MB/s read and 30-35Mb/s write. "Recommended" 8192 gives me 40K/s read (YES, 40 _K_ilobytes per second!) But I don't like this situation. Why? Because, it is black magic, not science. Why 49152? How can this value be found without binary search? What affects this value? Second problem is about TCP windows, which can give more thoughtput on 1Gig link. I've tried to tcpdump traffic between samba and WinXP client, and it shows, that window scaling is turned off. Always. It is enabled on WinXP client, it is enabled on FreeBSD server, it us used by iperf (with great effect), but not by samba! What do I do wrong? -- // Black Lion AKA Lev Serebryakov