From owner-freebsd-stable@FreeBSD.ORG Mon Dec 19 09:37:46 2005 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4CD4516A41F for ; Mon, 19 Dec 2005 09:37:46 +0000 (GMT) (envelope-from jd@ugcs.caltech.edu) Received: from lira.ugcs.caltech.edu (lira.ugcs.caltech.edu [131.215.176.118]) by mx1.FreeBSD.org (Postfix) with ESMTP id B4ACE43D5D for ; Mon, 19 Dec 2005 09:37:45 +0000 (GMT) (envelope-from jd@ugcs.caltech.edu) Received: by lira.ugcs.caltech.edu (Postfix, from userid 3640) id E00116B002; Mon, 19 Dec 2005 01:37:44 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by lira.ugcs.caltech.edu (Postfix) with ESMTP id A92225300A; Mon, 19 Dec 2005 01:37:44 -0800 (PST) Date: Mon, 19 Dec 2005 01:37:44 -0800 (PST) From: Jon Dama To: Oliver Brandmueller In-Reply-To: <20051219091441.GQ34429@e-Gitt.NET> Message-ID: References: <20051216143936.GF34429@e-Gitt.NET> <20051216163031.6d5ce939@TP51.local> <20051218124537.GK34429@e-Gitt.NET> <20051218165029.6aa75b7b@TP51.local> <20051219091441.GQ34429@e-Gitt.NET> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-stable@freebsd.org Subject: Re: [SOLVED] Re: NFS UDP mounts on RELENG_6? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2005 09:37:46 -0000 No shame. This is a common problem, and if you think real hard you should start getting quasy about UDP NFS under high load. The problem arises because if part of a UDP packet is lost the entire packet is lost, and as UDP NFS uses fixed packet sizes... this means the system never recovers if frames stop droping perpetually. Its easy to see how this happens for speed mismatches, the switch (vs a router) provides no buffering and therefore frames from the fast source are dropped. Now imagine that you have many clients all writing data to the NFS server at once... Suddently that pipe into the server is like a narrow straw. Ooops. I haven't see any evidence that suggests using NFS with UDP is actually useful. IMO, its a false economy. TCP processing takes on the order of 1uS of CPU time--which is on the order of the frame latency through a single switch! That is to say, nothing, but the behavior of TCP NFS under load (when it counts) is superior. TCP SACK and interrupt aggregation are better ways of squeezing extra performance out of your hardware than simply using UDP... Just my two cents. -Jon On Mon, 19 Dec 2005, Oliver Brandmueller wrote: > Hi. > > On Mon, Dec 19, 2005 at 01:01:34AM -0800, Jon Dama wrote: > > A very critical question here is the network topology. > > > > UDP NFS _cannot_ be used across switches where the ports are operating at > > different speeds--unless the UDP packet size is to be smaller than MTU. > > > > Be sure and verify that every link between the server and the client are > > operating at the same speed. > > *ouch* shame on me. > > I looked at interfaces, links, errors - everything. I found the problem > in a misconfiguration and you just pointed at it: > > The server has not been booted for a few hundred days before upgrading. > >From an old test there was an "mtu 9000" for the NFS interface still in > /etc/rc.conf (while it has been reset after failed tests with other > hardware on that network manually to 1500). > > So: SORRY for making everybody mad here. It was just me being blind. > > Thanx for pointing me at that! > > - Oliver > > -- > | Oliver Brandmueller | Offenbacher Str. 1 | Germany D-14197 Berlin | > | Fon +49-172-3130856 | Fax +49-172-3145027 | WWW: http://the.addict.de/ | > | Ich bin das Internet. Sowahr ich Gott helfe. | > | Eine gewerbliche Nutzung aller enthaltenen Adressen ist nicht gestattet! | >