From owner-freebsd-performance@FreeBSD.ORG  Wed Feb 20 21:42:49 2008
Return-Path: <owner-freebsd-performance@FreeBSD.ORG>
Delivered-To: freebsd-performance@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 737CE16A404;
	Wed, 20 Feb 2008 21:42:49 +0000 (UTC)
	(envelope-from kris@FreeBSD.org)
Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28])
	by mx1.freebsd.org (Postfix) with ESMTP id AE6A513C459;
	Wed, 20 Feb 2008 21:42:47 +0000 (UTC)
	(envelope-from kris@FreeBSD.org)
Message-ID: <47BC9ED5.1010505@FreeBSD.org>
Date: Wed, 20 Feb 2008 22:42:45 +0100
From: Kris Kennaway <kris@FreeBSD.org>
User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031)
MIME-Version: 1.0
To: Chuck Swiger <cswiger@mac.com>
References: <27dbfc8c0802190243y113d3059yd0c602850a4dbd6b@mail.gmail.com>	<47BB33AD.1050005@FreeBSD.org>	<27dbfc8c0802200323r13f69905l4940d0d5accd1eb1@mail.gmail.com>	<9BCE1D41-EC1A-4FE6-8551-E725DBE5D3A8@mac.com>	<20080220210118.GY99258@elvis.mu.org>
	<B4337B98-B9A2-40A2-9C85-1BCC3F5FB54F@mac.com>
In-Reply-To: <B4337B98-B9A2-40A2-9C85-1BCC3F5FB54F@mac.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: "freebsd-performance@freebsd.org" <freebsd-performance@freebsd.org>,
	Alfred Perlstein <alfred@freebsd.org>,
	Valerio Daelli <valerio.daelli@gmail.com>
Subject: Re: Bad performance of 7.0 nfs client with Solaris nfs server
X-BeenThere: freebsd-performance@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Performance/tuning <freebsd-performance.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-performance>,
	<mailto:freebsd-performance-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-performance>
List-Post: <mailto:freebsd-performance@freebsd.org>
List-Help: <mailto:freebsd-performance-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-performance>,
	<mailto:freebsd-performance-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 20 Feb 2008 21:42:49 -0000

Chuck Swiger wrote:
> On Feb 20, 2008, at 1:01 PM, Alfred Perlstein wrote:
>>> Take a look at the level of packet fragmentation you are encountering;
>>> yes, this is expected and things will work but there is extra latency
>>> added when the IP stack has to reassemble packets before the data can
>>> be delivered.  Try setting the NFS rsize/wsize to 1024 or perhaps 1400
>>> and see whether that improves performance.
>>>
>>> Or, if your switch and NICs support it, see whether you can get Gb
>>> Ethernet jumbo frames working so that you don't have to fragment for
>>> 2K or 4K data packets....
>>
>> TCP mounts do not have this problem.  You can safely use
>> 32k or higher sizes with TCP without fragmentation.
> 
> Oh, sure.  But there is a bit more overhead with TCP transport than 
> UDP-- for local (switched) networks, UDP generally seems to be a 
> win...TCP seems to be a better choice over a VPN or some similar kind of 
> WAN.

Actually this is no longer true.  At modern LAN speeds (e.g. gige) you 
can transmit packets fast enough that two things happen:

1) UDP socket buffer overruns are common, leading to packet loss.

2) the 16-bit sequence numbers wrap *much* faster than the IP fragment 
lifetime (30 seconds).

These combine to cause data corruption when fragmented packets are 
dropped and then reassembled with a later transmission of the same 
sequence number.

TCP mounts should be used whenever possible thesedays (I flipped the 
default mode in 8.0 the other day).

Kris