From owner-freebsd-questions@FreeBSD.ORG Fri Mar 19 21:14:51 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF99B1065678; Fri, 19 Mar 2010 21:14:51 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from mail-bw0-f216.google.com (mail-bw0-f216.google.com [209.85.218.216]) by mx1.freebsd.org (Postfix) with ESMTP id B63CD8FC1F; Fri, 19 Mar 2010 21:14:50 +0000 (UTC) Received: by bwz8 with SMTP id 8so3431141bwz.3 for ; Fri, 19 Mar 2010 14:14:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=VtYu6IlkRcrTpjamNmL7ifAXdEWEVeuZkVCUKd06v1k=; b=ijz+NKqoXWkZutS6lPlKUEtDKCs4ck+1/p5/XZ3juAhvBSBt3mzFE36YdUJu71Jjlh gITWTtykZyZGq9AIrIZzltARRdx2fqjs/tIoJBG0GSpX0RQ+c2NRlh5J6SK1bwJtea2T hczGm5pIBSq3SvjURZ42gULHnxrReyHTx4SxM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=ANKV6hYVoEMX+Caef8LR7TNcEHYD23tUdMTQfSQq0MyTHKUuUIbZiV0eOYfavHyAlC 7bf19ZG3xyvjbu1H51oGFxuMMrca7n5pnVtd07HERmZ9BHFzoR5ZMYf1ajCTbrzsTCMr UswHies4ahCgS5DAr5uVKhL5ArGO8daWRV7NU= MIME-Version: 1.0 Received: by 10.204.85.73 with SMTP id n9mr1890804bkl.71.1269033288061; Fri, 19 Mar 2010 14:14:48 -0700 (PDT) Date: Fri, 19 Mar 2010 23:14:47 +0200 Message-ID: From: Dan Naumov To: freebsd-net@freebsd.org, freebsd-questions@freebsd.org, FreeBSD-STABLE Mailing List , freebsd-performance@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: Samba read speed performance tuning X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 21:14:51 -0000 On a FreeBSD 8.0-RELEASE/amd64 system with a Supermicro X7SPA-H board using an Intel gigabit nic with the em driver, running on top of a ZFS mirror, I was seeing a strange issue. Local reads and writes to the pool easily saturate the disks with roughly 75mb/s throughput, which is roughly the best these drives can do. However, working with Samba, writes to a share could easily pull off 75mb/s and saturate the disks, but reads off a share were resulting in rather pathetic 18mb/s throughput. I found a threadon the FreeBSD forums (http://forums.freebsd.org/showthread.php?t=9187) and followed the suggested advice. I rebuilt Samba with AIO support, kldloaded the aio module and made the following changes to my smb.conf From: socket options=TCP_NODELAY To: socket options=SO_RCVBUF=131072 SO_SNDBUF=131072 TCP_NODELAY min receivefile size=16384 use sendfile=true aio read size = 16384 aio write size = 16384 aio write behind = true dns proxy = no[/CODE] This showed a very welcome improvement in read speed, I went from 18mb/s to 48mb/s. The write speed remained unchanged and was still saturating the disks. Now I tried the suggested sysctl tunables: atombsd# sysctl net.inet.tcp.delayed_ack=0 net.inet.tcp.delayed_ack: 1 -> 0 atombsd# sysctl net.inet.tcp.path_mtu_discovery=0 net.inet.tcp.path_mtu_discovery: 1 -> 0 atombsd# sysctl net.inet.tcp.recvbuf_inc=524288 net.inet.tcp.recvbuf_inc: 16384 -> 524288 atombsd# sysctl net.inet.tcp.recvbuf_max=16777216 net.inet.tcp.recvbuf_max: 262144 -> 16777216 atombsd# sysctl net.inet.tcp.sendbuf_inc=524288 net.inet.tcp.sendbuf_inc: 8192 -> 524288 atombsd# sysctl net.inet.tcp.sendbuf_max=16777216 net.inet.tcp.sendbuf_max: 262144 -> 16777216 atombsd# sysctl net.inet.tcp.sendspace=65536 net.inet.tcp.sendspace: 32768 -> 65536 atombsd# sysctl net.inet.udp.maxdgram=57344 net.inet.udp.maxdgram: 9216 -> 57344 atombsd# sysctl net.inet.udp.recvspace=65536 net.inet.udp.recvspace: 42080 -> 65536 atombsd# sysctl net.local.stream.recvspace=65536 net.local.stream.recvspace: 8192 -> 65536 atombsd# sysctl net.local.stream.sendspace=65536 net.local.stream.sendspace: 8192 -> 65536 This improved the read speeds a further tiny bit, now I went from 48mb/s to 54mb/s. This is it however, I can't figure out how to increase Samba read speed any further. Any ideas?