From owner-freebsd-stable@freebsd.org Fri Mar 9 15:26:06 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0BAE6F3161E for ; Fri, 9 Mar 2018 15:26:06 +0000 (UTC) (envelope-from Andreas.Nagy@frequentis.com) Received: from mail1.frequentis.com (mail1.frequentis.com [195.20.158.50]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "spamquarantine.frequentis.frq", Issuer "Frequentis Enterprise Issuing CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 678697C223 for ; Fri, 9 Mar 2018 15:26:04 +0000 (UTC) (envelope-from Andreas.Nagy@frequentis.com) X-IronPort-AV: E=Sophos;i="5.47,446,1515452400"; d="scan'208";a="27563328" Received: from vie191nt.frequentis.frq ([172.16.1.191]) by mail1.frequentis.com with ESMTP; 09 Mar 2018 16:25:57 +0100 Received: from vie196nt.frequentis.frq ([172.16.1.196]) by vie191nt.frequentis.frq ([172.16.1.191]) with mapi id 14.03.0382.000; Fri, 9 Mar 2018 16:25:56 +0100 From: NAGY Andreas To: Rick Macklem , "'freebsd-stable@freebsd.org'" Subject: =?iso-8859-1?Q?RE:_NFS_4.1_RECLAIM=5FCOMPLETE_FS=A0failed_error_in_combin?= =?iso-8859-1?Q?ation_with_ESXi_client?= Thread-Topic: =?iso-8859-1?Q?NFS_4.1_RECLAIM=5FCOMPLETE_FS=A0failed_error_in_combinatio?= =?iso-8859-1?Q?n_with_ESXi_client?= Thread-Index: AdOx8zAe5+TceuOWQkax+IhJZhNDgQAnzopHABn27/AAIBzCQgAP8SUgAAmzYWAADy6jKgAZTvpAABQ08bcAKLRnLwArPwd4AAPK4ZAAFJa1tQAZS9wAABCp2AIAIYll8AAB3b2w Date: Fri, 9 Mar 2018 15:25:56 +0000 Message-ID: Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.13.148] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Mar 2018 15:26:06 -0000 >The attached patch changes BindConnectiontoSession to reply NFS4ERR_BAD_SE= SSION when the session doesn't exist. This might trigger recovery after a s= erver reboot. >This patch must be applied after bindconn.patch. Works perfect! ESXi host reconnects to the datastore as soon as the nfsserv= er is available. >I took a quick look at your packet trace and it appears that this client d= oes all write FILESYNC. As such, setting vfs.nfsd.async=3D1 won't have any = affect. >If you apply the attached patch, it should change the FILESYNC->UNSTABLE s= o that vfs.nfsd.async=3D1 will make a difference. Again, doing this does pu= t data at risk when the server crashes. Yes, ESXi writes everything sync. With the patch + vfs.nfsd.async=3D1 (only= for testing) writes are a bit faster.=20 If have not tuned anything on the fs settings so far (just formatted it as = standard UFS), compared to multipath iSCSI with VMFS reads are a little bit= faster, but writes are a bit slower. That's just what I see from a simple ATTO benchmark from within a Windows V= M, have not done any details IOP benchmark,... The RAID controller on this machines does not support IT mode, but I think = I will still use ZFS, but only as filesystem on a single hw raid disk. Must= check what are the best setting for this on the hw raid + zfs for nfs. >This one is a mystery to me. It seemed to be upset that the directory is c= hanging (I assume either the Change or ModifyTime attributes). However, if = entries are being deleted, the directory is changing and, as far as I know,= the Change and ModifyTime attributes are supposed to change. >I might try posting on nfsv4@ietf.org in case somebody involved with this = client reads that list and can explain what this is? Maybe it is really just a bug in the VMware integrated host client browser.= Deleting folders on the mounted datastore in the ESXi shell is no problem = and does also not generate any warnings in the vmkernel.log. >I'll take another look and see if I can guess why it doesn't like "2" as a= reason for not issuing a delegation. (As noted before, I don't think this = is serious, but???) This warnings are still there, but don't seem to have any impact. It looks = like they only appeare when files are created or modified on the datastore = from the datastore browser or from shell, have not seen this warnings when = working in a VM on a virtual disk that is stored on the nfs datastore. >Yes, before I posted that I didn't understand why multiple TCP links would= be faster. >I didn't notice at the time that you mentioned using different subnets and= , as such, links couldn't be trunked below TCP. In your case trunking above= TCP makes sense. Yes, actually I am working on a lab environment/testsys I often get some se= rvers as leftovers from projects and there are also plenty of Cisco 1GB swi= tches, but 10GBs are rare. I already did some tests with iSCSI multipathing= but I prefer NFS and now with this I get also the same speed.=20 As I have never seen a working setup with multiple paths for NFS, so I am r= eally happy that you got this working in such a short time. andi