From owner-freebsd-net@FreeBSD.ORG  Sat Oct 13 12:12:02 2007
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@FreeBSD.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 334A516A41A;
	Sat, 13 Oct 2007 12:12:02 +0000 (UTC)
	(envelope-from kris@FreeBSD.org)
Received: from weak.local (hub.freebsd.org [IPv6:2001:4f8:fff6::36])
	by mx1.freebsd.org (Postfix) with ESMTP id 5881F13C45D;
	Sat, 13 Oct 2007 12:12:01 +0000 (UTC)
	(envelope-from kris@FreeBSD.org)
Message-ID: <4710B60E.3020102@FreeBSD.org>
Date: Sat, 13 Oct 2007 14:11:58 +0200
From: Kris Kennaway <kris@FreeBSD.org>
User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728)
MIME-Version: 1.0
To: Jim Rees <rees@freebsd.org>
References: <200710121912.l9CJCLeI085992@repoman.freebsd.org>
	<470FDD06.4090904@FreeBSD.org>
	<20071013012907.GA11606@citi.umich.edu>
In-Reply-To: <20071013012907.GA11606@citi.umich.edu>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: freebsd-net@FreeBSD.org, Mohan Srinivasan <mohans@FreeBSD.org>
Subject: Re: cvs commit: src/sys/nfsclient nfs.h nfs_socket.c nfs_subs.c
	nfsmount.h
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 13 Oct 2007 12:12:02 -0000

Jim Rees wrote:
> Kris Kennaway wrote:
> 
>   Mohan's previous commit that increases the nfs server socket buffer size 
>   is also very important for NFS performance.  Without it I was only 
>   getting 1-2MB/sec throughput over 10Gb ethernet with UDP mounts
> 
> You're trying to use nfs/udp at 10G?  I wouldn't expect that to work.

Well there some corruption issues, yes.  These happen even at 1ge 
though, and seem to be due to packet loss (e.g. one problem is that the 
ID number is 16 bits so if we are doing fragment reassembly with a 
dropped packet it can easily roll over and assemble a corrupted packet 
before the correct one is retransmitted).

Depending on what workload I use I see up to about 300 MB/sec throughput 
before it is limited by the nfsd.

> I've recently been trying to get good performance from nfs at 10Gbps (on
> linux).  The required window is 150MB for a transatlantic link.

Heh, that probably has its own challenges :)

Kris