From owner-freebsd-fs@FreeBSD.ORG  Mon Mar 26 16:32:24 2012
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 0CDA81065672
	for <freebsd-fs@freebsd.org>; Mon, 26 Mar 2012 16:32:24 +0000 (UTC)
	(envelope-from sven@crashme.org)
Received: from celaeno.tauri.mw.lg.virgo.supercluster.net
	(celaeno.tauri.mw.lg.virgo.supercluster.net [213.252.140.33])
	by mx1.freebsd.org (Postfix) with ESMTP id B99E68FC1B
	for <freebsd-fs@freebsd.org>; Mon, 26 Mar 2012 16:32:23 +0000 (UTC)
Received: from miram.persei.mw.lg.virgo.supercluster.net ([213.252.140.37]
	helo=[192.168.20.6])
	by celaeno.tauri.mw.lg.virgo.supercluster.net with esmtps
	(TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.76)
	(envelope-from <sven@crashme.org>) id 1SCCqC-0004TL-TH
	for freebsd-fs@freebsd.org; Mon, 26 Mar 2012 16:32:22 +0000
Message-ID: <4F709A18.50907@crashme.org>
Date: Mon, 26 Mar 2012 18:32:24 +0200
From: Sven Brandenburg <sven@crashme.org>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:11.0) Gecko/20120310 Thunderbird/11.0
MIME-Version: 1.0
To: freebsd-fs@freebsd.org
References: <4F703815.8070809@crashme.org> <jkpgt4$v55$1@dough.gmane.org>
In-Reply-To: <jkpgt4$v55$1@dough.gmane.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: 0.0 (/)
Subject: Re: NFSv3, ZFS, 10GE performance
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 26 Mar 2012 16:32:24 -0000


On 03/26/2012 12:37 PM, Ivan Voras wrote:
> You could try modifying the rsize and wsize NFS options (read
> mount_nfs(8)), they help with UFS.

I tried this a few days ago and fiddling rsize alters performance from 
"ok" to "terrible".
However, you made me revisit this and mount_nfs(8) seems to have a gem 
in its options: readahead.
This did the trick for me and my (long and sequential) reads.
While the manpage says its limited to 0-4, the best results were 
achieved with readahead=8 : 1.1GB/s - which is what I had hoped for.

On a tangent: gnu-dd 1GB/s is 10^9 Bytes/s, not 2^30. Yes, I fell for it 
at first :)
The good news is that there was no fiddling on the NFS server side. 
(Apart from MTU increases, PCI settings and more buffers to get TCP 
performance to full tilt in the first place)

Hopefully, readahead doesn't kill performance for smaller files.. :-)

regards,
Sven