From owner-freebsd-performance@FreeBSD.ORG Thu Feb 21 05:11:56 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09C0516A402 for ; Thu, 21 Feb 2008 05:11:56 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id E90DB13C448 for ; Thu, 21 Feb 2008 05:11:55 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id C32F81A4D7E; Wed, 20 Feb 2008 21:11:55 -0800 (PST) Date: Wed, 20 Feb 2008 21:11:55 -0800 From: Alfred Perlstein To: Kris Kennaway Message-ID: <20080221051155.GE99258@elvis.mu.org> References: <27dbfc8c0802190243y113d3059yd0c602850a4dbd6b@mail.gmail.com> <47BB33AD.1050005@FreeBSD.org> <27dbfc8c0802200323r13f69905l4940d0d5accd1eb1@mail.gmail.com> <9BCE1D41-EC1A-4FE6-8551-E725DBE5D3A8@mac.com> <20080220210118.GY99258@elvis.mu.org> <47BC9ED5.1010505@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <47BC9ED5.1010505@FreeBSD.org> User-Agent: Mutt/1.4.2.3i Cc: "freebsd-performance@freebsd.org" , Valerio Daelli Subject: Re: Bad performance of 7.0 nfs client with Solaris nfs server X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Feb 2008 05:11:56 -0000 * Kris Kennaway [080220 13:42] wrote: > Chuck Swiger wrote: > >On Feb 20, 2008, at 1:01 PM, Alfred Perlstein wrote: > >>>Take a look at the level of packet fragmentation you are encountering; > >>>yes, this is expected and things will work but there is extra latency > >>>added when the IP stack has to reassemble packets before the data can > >>>be delivered. Try setting the NFS rsize/wsize to 1024 or perhaps 1400 > >>>and see whether that improves performance. > >>> > >>>Or, if your switch and NICs support it, see whether you can get Gb > >>>Ethernet jumbo frames working so that you don't have to fragment for > >>>2K or 4K data packets.... > >> > >>TCP mounts do not have this problem. You can safely use > >>32k or higher sizes with TCP without fragmentation. > > > >Oh, sure. But there is a bit more overhead with TCP transport than > >UDP-- for local (switched) networks, UDP generally seems to be a > >win...TCP seems to be a better choice over a VPN or some similar kind of > >WAN. > > Actually this is no longer true. At modern LAN speeds (e.g. gige) you > can transmit packets fast enough that two things happen: > > 1) UDP socket buffer overruns are common, leading to packet loss. > > 2) the 16-bit sequence numbers wrap *much* faster than the IP fragment > lifetime (30 seconds). > > These combine to cause data corruption when fragmented packets are > dropped and then reassembled with a later transmission of the same > sequence number. > > TCP mounts should be used whenever possible thesedays (I flipped the > default mode in 8.0 the other day). Additionally with smaller read/write sizes you must generate more RPCs, for instance using a tcp read size of 32k will allow the client to generate a single NFS_READ request for that 32k, however a UDP mount at 1.2k will need to generate appoximately 26 requests and the server will have to service that many requests as well. -- - Alfred Perlstein