Date: Wed, 20 Feb 2008 21:11:55 -0800 From: Alfred Perlstein <alfred@freebsd.org> To: Kris Kennaway <kris@FreeBSD.org> Cc: "freebsd-performance@freebsd.org" <freebsd-performance@freebsd.org>, Valerio Daelli <valerio.daelli@gmail.com> Subject: Re: Bad performance of 7.0 nfs client with Solaris nfs server Message-ID: <20080221051155.GE99258@elvis.mu.org> In-Reply-To: <47BC9ED5.1010505@FreeBSD.org> References: <27dbfc8c0802190243y113d3059yd0c602850a4dbd6b@mail.gmail.com> <47BB33AD.1050005@FreeBSD.org> <27dbfc8c0802200323r13f69905l4940d0d5accd1eb1@mail.gmail.com> <9BCE1D41-EC1A-4FE6-8551-E725DBE5D3A8@mac.com> <20080220210118.GY99258@elvis.mu.org> <B4337B98-B9A2-40A2-9C85-1BCC3F5FB54F@mac.com> <47BC9ED5.1010505@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
* Kris Kennaway <kris@FreeBSD.org> [080220 13:42] wrote: > Chuck Swiger wrote: > >On Feb 20, 2008, at 1:01 PM, Alfred Perlstein wrote: > >>>Take a look at the level of packet fragmentation you are encountering; > >>>yes, this is expected and things will work but there is extra latency > >>>added when the IP stack has to reassemble packets before the data can > >>>be delivered. Try setting the NFS rsize/wsize to 1024 or perhaps 1400 > >>>and see whether that improves performance. > >>> > >>>Or, if your switch and NICs support it, see whether you can get Gb > >>>Ethernet jumbo frames working so that you don't have to fragment for > >>>2K or 4K data packets.... > >> > >>TCP mounts do not have this problem. You can safely use > >>32k or higher sizes with TCP without fragmentation. > > > >Oh, sure. But there is a bit more overhead with TCP transport than > >UDP-- for local (switched) networks, UDP generally seems to be a > >win...TCP seems to be a better choice over a VPN or some similar kind of > >WAN. > > Actually this is no longer true. At modern LAN speeds (e.g. gige) you > can transmit packets fast enough that two things happen: > > 1) UDP socket buffer overruns are common, leading to packet loss. > > 2) the 16-bit sequence numbers wrap *much* faster than the IP fragment > lifetime (30 seconds). > > These combine to cause data corruption when fragmented packets are > dropped and then reassembled with a later transmission of the same > sequence number. > > TCP mounts should be used whenever possible thesedays (I flipped the > default mode in 8.0 the other day). Additionally with smaller read/write sizes you must generate more RPCs, for instance using a tcp read size of 32k will allow the client to generate a single NFS_READ request for that 32k, however a UDP mount at 1.2k will need to generate appoximately 26 requests and the server will have to service that many requests as well. -- - Alfred Perlstein
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080221051155.GE99258>