From owner-freebsd-current@FreeBSD.ORG  Wed Jul 22 23:40:07 2009
Return-Path: <owner-freebsd-current@FreeBSD.ORG>
Delivered-To: freebsd-current@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 22CDE106566B
	for <freebsd-current@freebsd.org>; Wed, 22 Jul 2009 23:40:07 +0000 (UTC)
	(envelope-from dillon@apollo.backplane.com)
Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2])
	by mx1.freebsd.org (Postfix) with ESMTP id EC0BA8FC0C
	for <freebsd-current@freebsd.org>; Wed, 22 Jul 2009 23:40:06 +0000 (UTC)
	(envelope-from dillon@apollo.backplane.com)
Received: from apollo.backplane.com (localhost [127.0.0.1])
	by apollo.backplane.com (8.14.2/8.14.1) with ESMTP id n6MNe4HE013224;
	Wed, 22 Jul 2009 16:40:04 -0700 (PDT)
Received: (from dillon@localhost)
	by apollo.backplane.com (8.14.2/8.13.4/Submit) id n6MNe3K5013221;
	Wed, 22 Jul 2009 16:40:03 -0700 (PDT)
Date: Wed, 22 Jul 2009 16:40:03 -0700 (PDT)
From: Matthew Dillon <dillon@apollo.backplane.com>
Message-Id: <200907222340.n6MNe3K5013221@apollo.backplane.com>
To: Chris Hedley <freebsd-current@chrishedley.com>
References: <alpine.BSF.2.00.0907222232310.3132@teapot.cbhnet>
	<200907222307.n6MN7YhU012788@apollo.backplane.com>
	<alpine.BSF.2.00.0907230010250.3132@teapot.cbhnet>
Cc: freebsd-current@freebsd.org
Subject: Re: Linux NFS ate my bge
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
	<freebsd-current.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
	<mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
	<mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 22 Jul 2009 23:40:07 -0000

    TCP will likely work better, for several reasons, not the least of
    which being that the NFS client does not have to estimate a retransmit
    timeout on a rpc-by-rpc basis.  Such estimations fail utterly in the
    face of a large number of concurrent RPCs because latency winds up being
    governed by the disk backlog on the server.  A UDP mount will wind up
    retransmitting even under completely lossless conditions.

    Another reason TCP tends to work better is that UDP uses IP fragmentation
    and IP fragmentation reassembly is not typically in the critical path.
    The desired NFS filesystem block size is 16K (smaller will typically
    reduce performance), so even a 9000 MTU won't help.

    Also use netstat ... not sure what option, I think -x, to determine the
    actual size of the socket buffer being employed for the connection
    (TCP or UDP).  There are multiple internal caps in the kernel and it
    is often not as big as you might have thought it should be.  You want
    a 256KB socket buffer at a minimum for a GigE network.  Smaller works
    (at least for linear transfers), but you lose a lot of RPC concurrency
    from the client.  Again, something that matters more for a linux client
    vs a FreeBSD client.

						-Matt