From owner-freebsd-stable@FreeBSD.ORG Thu Dec 1 05:35:25 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F32921065678 for ; Thu, 1 Dec 2011 05:35:24 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta12.emeryville.ca.mail.comcast.net (qmta12.emeryville.ca.mail.comcast.net [76.96.27.227]) by mx1.freebsd.org (Postfix) with ESMTP id D8D548FC17 for ; Thu, 1 Dec 2011 05:35:24 +0000 (UTC) Received: from omta08.emeryville.ca.mail.comcast.net ([76.96.30.12]) by qmta12.emeryville.ca.mail.comcast.net with comcast id 3haz1i0010FhH24AChbJBX; Thu, 01 Dec 2011 05:35:18 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta08.emeryville.ca.mail.comcast.net with comcast id 3haS1i00N1t3BNj8UhaSA6; Thu, 01 Dec 2011 05:34:26 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 21362102C1D; Wed, 30 Nov 2011 21:35:23 -0800 (PST) Date: Wed, 30 Nov 2011 21:35:23 -0800 From: Jeremy Chadwick To: John Baldwin Message-ID: <20111201053523.GA66988@icarus.home.lan> References: <201111180310.pAI3ARbZ075115@mippet.ci.com.au> <201111290012.pAT0CdPw070812@mippet.ci.com.au> <201111291036.44620.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201111291036.44620.jhb@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@freebsd.org, Daryl Sayers Subject: Re: Low nfs write throughput X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2011 05:35:25 -0000 On Tue, Nov 29, 2011 at 10:36:44AM -0500, John Baldwin wrote: > On Monday, November 28, 2011 7:12:39 pm Daryl Sayers wrote: > > >>>>> "Bengt" == Bengt Ahlgren writes: > > > > > Daryl Sayers writes: > > >> Can anyone suggest why I am getting poor write performance from my nfs setup. > > >> I have 2 x FreeBSD 8.2-STABLE i386 machines with ASUS P5B-plus mother boards, > > >> 4G mem and Dual core 3g processor using 147G 15k Seagate SAS drives with > > >> onboard Gb network cards connected to an idle network. The results below show > > >> that I get nearly 100Mb/s with a dd over rsh but only 15Mbs using nfs. It > > >> improves if I use async but a smbfs mount still beats it. I am using the same > > >> file, source and destinations for all tests. I have tried alternate Network > > >> cards with no resulting benefit. > > > > > [...] > > > > >> Looking at a systat -v on the destination I see that the nfs test does not > > >> exceed 16KB/t with 100% busy where the other tests reach up to 128KB/t. > > >> For the record I get reads of 22Mb/s without and 77Mb/s with async turned on > > >> for the nfs mount. > > > > > On an UFS filesystem you get NFS writes with the same size as the > > > filesystem blocksize. So an easy way to improve performance is to > > > create a filesystem with larger blocks. I accidentally found this out > > > when I had two NFS exported filesystems from the same box with 16K and > > > 64K blocksizes respectively. > > > > > (Larger blocksize also tremendously improves the performance of UFS > > > snapshots!) > > > > Thanks to all that answered. I did try the 'sysctl -w vfs.nfsrv.async=1' with > > no reportable change in performance. We are using a UFS2 filesystem so the > > zfs command was not required. I did not try the patch as we would like to stay > > as standard as possible but will upgrade if the patch is released in new > > kernel. > > If you can test the patch then it is something I will likely put into the > next release. I have already tested it as far as robustness locally, what > I don't have are good performance tests. It would really be helpful if you > were able to test it. John, We'd like to test this patch[1], but need to know if it needs to be applied to just the system acting as the NFS server, or the NFS clients as well. [1]: http://www.freebsd.org/~jhb/patches/nfs_server_cluster.patch -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |