From owner-freebsd-current@freebsd.org Sun Oct 29 15:37:14 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E652FE4287A for ; Sun, 29 Oct 2017 15:37:14 +0000 (UTC) (envelope-from yuripv@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 644C381BE3; Sun, 29 Oct 2017 15:37:14 +0000 (UTC) (envelope-from yuripv@gmx.com) Received: from thor.xvoid.org ([94.233.210.201]) by mail.gmx.com (mrgmx001 [212.227.17.184]) with ESMTPSA (Nemesis) id 0LwrwO-1d6mmN1ccq-016M4X; Sun, 29 Oct 2017 16:31:59 +0100 Subject: Re: NFSv3 issues with latest -current From: Yuri Pankov To: freebsd-current Cc: Stephen Hurd References: <9ceeafa5-cb7f-cb82-db07-de6f28b209e2@gmx.com> <790d22eb-e04d-3bc7-0e79-e01feedd4267@gmx.com> Message-ID: Date: Sun, 29 Oct 2017 18:31:58 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <790d22eb-e04d-3bc7-0e79-e01feedd4267@gmx.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:b6+lD/HcN4YECXcWGzDekCqG2Ol6dJMZaFP093j6DYv/maBUdcG u0yKlqQEbVRKORD/xytqCM9cYcuioie8ekQ7ILYPFD9qk584qFbhkoV0L6A65MGHOU7z8sG zmhLeTB3aSeqK+3DrR86aJ9m+4MOvWCaDF5IM5t37zBvEfXLYiNzu9AmYBgSgeqa4lnpMK7 btuR/E/ZZrDzfoNswsPTg== X-UI-Out-Filterresults: notjunk:1;V01:K0:vBiYfh/GHHg=:1OfD1AKdHtyI0xNcGJMuH6 aLwFFdXWyFh0RMWpPxjyhRqmdC4MqXIio7ApLr4p1KmtYAnb64nX9lWXEtYWPwez34C8dzADk P98yOuYcOMqo2P/D3dMX9OvwPbrS9rDmAb7nmPefwUXVI2JeGOk7MAveNduKshv4nj8aRDGTq AbzbfYgyTr263MpVcw7qeTUrRkwbPOt+BVmHra5qGkgsgZKebefgaCG93cEW1ZgG2cJ7nD65z o4yhwKWppUoFgCXSkKHoKa0HCV9UFAKGFWHdq7QjOCiNB36O+klOl+jUGxyk2VRoQvmwtdc2f 4V44d1JVb19ziKQR9q+ay4spOedtGl1lLAnJ5uJyBQ45m+pnL9GzlHh6j6xmp7YeGNi1XB4Nw Dr9dc5xbvoZs4IhHScAYhAd9rEqwsXuac2Km84U1yhhwFKNjr4aHF4LeIEOrgmntdVdgfF5sh ULknDQFQEeZPNR06esM9U9xloF+GSdaGovPwtvOlF9Ca6cz4B2qFpd/dZmQ0yQNQpp7dCKb4K BCo7alu3dGeia1odmzAY9wq89Pa+B6OPHf+iqbzibY2OumvPfLeXpU+mwMWfKhuBz5ssK1/Rq CjZlO4BnRxaDrSf5bPBTS71gkqq20SC9dK0i9M4ZXAFnTGZUvu6nZebYmGDJDjU61fN7Kk/n1 aILVR0iT5Qdu3t0pA4vjP1tVGW5FzkS2JoDH3H3lHGpvBIf7mixaIFXOtCp+uC3L4d4VUlajA fpx+JuW4CT902yEEtNG0ZZxm5PJWqrzz0xJzbfE9UPdZgb8hSym/wgjLp20bG4INF3/BpX+cW lCifnflUbGG1HlgbPysHqkfxOsXNegwF3ZqMZz2+Iqthb53hp4HoU/3n0Dofmatozlmxze0uo B4WXYvAHLqiY/l49vXIdFbvUJN0Qckq/M+OszNf2dnhpdnR+2hZIAHECdqUqc5xf1oD7wAdiD bQHarsyX2tA== X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Oct 2017 15:37:15 -0000 On Sun, 29 Oct 2017 18:11:41 +0300, Yuri Pankov wrote: > On Sun, 29 Oct 2017 13:13:31 +0000, Rick Macklem wrote: >> Yuri Pankov wrote: >>> All file operations (e.g. copying the file over NFSv3 for me) seem to be >>> stuck running the latest -current (r325100). Reverting just the kernel >>> to r323779 (arbitrary chosen) seems to help. I noticed the "Stale file >>> handle when mounting nfs" message but I don't get the "stale file >>> handle" messages from mount, probably as I'm not running any linux clients. >> These kinds of problems are usually related to your net interface device >> driver or the TCP stack. >> >> A couple of things to try: >> - Disable TSO (look for a sysctl with "tso" in it). >> - Try using mount options rsize=32768,waize=32768 to reduce the I/O >> size. Some device drivers don't handle long chains of mbufs well, >> especially when the size is near 64K. >> (These issues have been fixed in current, but if a bug slips into a net driver >> update or ???) >> - Look at recent changes to the net device driver you are using and try reverting >> those changes if you can do so. >> - Capture packets and look at them in wireshark (which knows NFS) and see >> what is going on the wire. >> >> There hasn't been any recent changes to NFS that should affect NFSv3 mounts >> or to the kernel rpc, so I doubt the NFSv4.1 changes would be involved. > > Thanks for the hints, Rick! > > Indeed, it was one of the changes to sys/dev/e1000, reverting just the > commit made everything look normal again (CC'ing the author). One thing I forgot to mention here, the problem is visible only if the client side has MTU of 1500 configured; when both sides have MTU 9000, everything looks to be normal -- noticed this when my XenServer (having MTU of 1500 on management interface) wasn't able to read the ISO from NFS datastore. > The NIC is: > > igb0@pci0:2:0:0: class=0x020000 card=0x10c915d9 chip=0x10c98086 > rev=0x01 hdr=0x00 > vendor = 'Intel Corporation' > device = '82576 Gigabit Network Connection' > class = network > subclass = ethernet > > Interface configuration (note the MTU): > > igb0: flags=8843 metric 0 mtu 9000 > options=e525bb > ether 00:25:90:72:54:22 > inet6 fe80::225:90ff:fe72:5422%igb0 prefixlen 64 scopeid 0x1 > inet 192.168.1.4 netmask 0xffffff00 broadcast 192.168.1.255 > nd6 options=23 > media: Ethernet autoselect (1000baseT ) > status: active > > And the commit itself: > > commit f81cb8df32ae96299b8bbc2e948c17ad3aab59ca > Author: shurd > Date: Sat Sep 23 01:33:20 2017 +0000 > > Some small packet performance improvements > > If the packet is smaller than MTU, disable the TSO flags. > Move TCP header parsing inside the IS_TSO?() test. > Add a new IFLIB_NEED_ZERO_CSUM flag to indicate the checksums need > to be zeroed before TX. > > Reviewed by: sbruno > Approved by: sbruno (mentor) > Sponsored by: Limelight Networks > Differential Revision: https://reviews.freebsd.org/D12442 > > Notes: > svn path=/head/; revision=323941