From owner-freebsd-stable@freebsd.org Thu Mar 8 22:54:21 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B58D5F39128 for ; Thu, 8 Mar 2018 22:54:21 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-QB1-obe.outbound.protection.outlook.com (mail-eopbgr660060.outbound.protection.outlook.com [40.107.66.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4531C75A86 for ; Thu, 8 Mar 2018 22:54:20 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM (52.132.66.153) by YQBPR0101MB0948.CANPRD01.PROD.OUTLOOK.COM (52.132.66.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.567.14; Thu, 8 Mar 2018 22:54:19 +0000 Received: from YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM ([fe80::3531:c817:d6f:9b93]) by YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM ([fe80::3531:c817:d6f:9b93%13]) with mapi id 15.20.0567.012; Thu, 8 Mar 2018 22:54:19 +0000 From: Rick Macklem To: NAGY Andreas , "'freebsd-stable@freebsd.org'" Subject: =?iso-8859-1?Q?Re:_NFS_4.1_RECLAIM=5FCOMPLETE_FS=A0failed_error_in_combin?= =?iso-8859-1?Q?ation_with_ESXi_client?= Thread-Topic: =?iso-8859-1?Q?NFS_4.1_RECLAIM=5FCOMPLETE_FS=A0failed_error_in_combinatio?= =?iso-8859-1?Q?n_with_ESXi_client?= Thread-Index: AdOx8zAe5+TceuOWQkax+IhJZhNDgQAnzopHABn27/AAIBzCQgAP8SUgAAmzYWAADy6jKgAZTvpAABQ08bcAKLRnLwArPwd4AAPK4ZAAFJa1tQAZS9wAABCp2AI= Date: Thu, 8 Mar 2018 22:54:19 +0000 Message-ID: References: , , , , , <2feda1e2-16d5-43b5-98eb-dcc71cc67c6f@frequentis.com> , , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; YQBPR0101MB0948; 7:C1zUu/T1LL87rp+zhwAJZh+Xgxcm9omlfA1agNkPdasNydOEpCrWw9B0WYne8DBGekH+Ko+YulEQ9s2JmJKx6q5I9XNpmzLwz7NjyXaERQx/3kiAq2ccXYFzsccHfaWxkYkAVW2uufrsD/4KfpWSj1nwZYqM+YBcbCHhX6bgKsjOYGFI1/PL+n+OMmJrowCAWfGI6lldg77V2vg93+IX2tIwZr5CQTVE+8otCkoKMHcbV5uxOGFUcMnvIQHqVq1h x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: a9a8cb64-9eba-4ff6-36bc-08d5854786b9 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(8989060)(5600026)(4604075)(3008032)(4534165)(4627221)(201703031133081)(201702281549075)(8990040)(2017052603328)(7153060)(7193020); SRVR:YQBPR0101MB0948; x-ms-traffictypediagnostic: YQBPR0101MB0948: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040518)(2401047)(5005006)(8121501046)(93006095)(93001095)(10201501046)(3231220)(944501244)(52105095)(3002001)(6041306)(20161123558120)(20161123562045)(20161123564045)(20161123560045)(201703131423095)(201702281529075)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011); SRVR:YQBPR0101MB0948; BCL:0; PCL:0; RULEID:; SRVR:YQBPR0101MB0948; x-forefront-prvs: 060503E79B x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(366004)(346002)(39380400002)(39860400002)(376002)(189003)(199004)(229853002)(76176011)(5660300001)(7696005)(110136005)(59450400001)(99286004)(8936002)(5250100002)(33656002)(3660700001)(102836004)(81166006)(81156014)(305945005)(74316002)(186003)(2950100002)(6506007)(105586002)(478600001)(93886005)(316002)(14454004)(106356001)(786003)(26005)(25786009)(2906002)(97736004)(74482002)(55016002)(9686003)(68736007)(2900100001)(3280700002)(53936002)(6246003)(86362001)(6436002)(491001); DIR:OUT; SFP:1101; SCL:1; SRVR:YQBPR0101MB0948; H:YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) x-microsoft-antispam-message-info: DavoK6LrjxmMbEM98DXmCLEvz/lIQ/kHSuzVS2TIP1KIx9rNOBgpnfqfiihD7jxW2IOL+5vnH/2TQmVFM2y3Ya4Mj8ifnJhXNn3lci2Ytaz33As+ORhbGsf4aRhJjbeuev55UBqoMGNWee9UhziEBBhiftGINnm1oIDzBppYCxqp8fBzP9xW3HlmFXA6p0/oe/X3+lKF1UZq2E4gZlTVK4PMVEt6E7gIVKzZae0mQPxJliBxicIN/EofJh9TyIx4tuF/q5VqYdwYMhvXDgxr+RhCknYgOdIHMIxfyrLYDisiVjbiQ+r6xjecUuRwaKUbnUCKv4rWBpCtiG4KJfkQDw== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: a9a8cb64-9eba-4ff6-36bc-08d5854786b9 X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Mar 2018 22:54:19.1781 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB0948 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Mar 2018 22:54:21 -0000 NAGY Andreas wrote: >Thanks you, really great how fast you adapt the source/make patches for th= is. Saw so many >posts were people did not get NFS41 working with ESXi and = FreeBSD and now I have it already >running with your changes. > >I have now compiled the kernel with all 4 patches, and it works now. Ok. Sounds like we are making progress. It also takes someone willing to te= st patches, so thanks for doing so. >Some problems are still left: > >- the "Server returned improper reason for no delegation: 2" warnings are = still in the >vmkernel.log. > 2018-03-08T11:41:20.290Z cpu0:68011 opID=3D488969b0)WARNIN= G: NFS41: >NFS41ValidateDelegation:608: Server returned improper reason for= no delegation: 2 I'll take another look and see if I can guess why it doesn't like "2" as a = reason for not issuing a delegation. (As noted before, I don't think this is serious, but?= ??) >- can't delete a folder with the VMware host client datastore browser: > 2018-03-08T11:34:00.349Z cpu1:67981 opID=3Df5159ce3)WARNIN= G: NFS41: >NFS41FileOpReaddir:4728: Failed to process READDIR result for fh= 0x43046e4cb158: Transient >file system condition, suggest retry [more of these snipped] > 2018-03-08T11:34:00.352Z cpu1:67981 opID=3Df5159ce3)WARNIN= G: UserFile: 2155: >hostd-worker: Directory changing too often to perform r= eaddir operation (11 retries), >returning busy This one is a mystery to me. It seemed to be upset that the directory is ch= anging (I assume either the Change or ModifyTime attributes). However, if entries are= being deleted, the directory is changing and, as far as I know, the Change and Mo= difyTime attributes are supposed to change. I might try posting on nfsv4@ietf.org in case somebody involved with this c= lient reads that list and can explain what this is? >- after a reboot of the FreeBSD machine the ESXi does not restore the NFS = datastore again >with following warning (just disconnecting the links is fi= ne) > 2018-03-08T12:39:44.602Z cpu23:66484)WARNING: NFS41: NFS41= _Bug:2361: BUG - >Invalid BIND_CONN_TO_SESSION error: NFS4ERR_NOTSUPP Hmm. Normally after a server reboot, the clients will try some RPC that sta= rts with a Sequence (the session op) and the server will reply NFS4ERR_BAD_SESSION. This triggers recovery in the client. The BindConnectiontoSession operation is done in an RPC by itself, so there= is no Sequence op to trigger NFS4ERR_BAD_SESSION. Maybe this client expects to see NFS4ERR_BAD_SESSION for the BindConnection= toSession. I'll post a patch that modifies the BindConnectiontoSession to do that. >Actually I have only made some quick benchmarks with ATTO in a Windows VM = which has a >vmdk on the NFS41 datastore which is mounted over two 1GB link= s in different subnets. >Read is nearly the double of just a single connection and write is just a = bit faster. Don't know if >write speed could be improved, actually the shar= e is UFS on a HW raid controller which has >local write speeds about 500MB/= s. Yes, before I posted that I didn't understand why multiple TCP links would = be faster. I didn't notice at the time that you mentioned using different subnets and,= as such, links couldn't be trunked below TCP. In your case trunking above TCP makes = sense. Getting slower write rates than read rates from NFS is normal. Did you try "sysctl vfs.nfsd.async=3D1"? The other thing that might help for UFS is increasing the size of the buffe= r cache. (If this server is mainly an NFS server you could probably make the buffer = cache greater than half of the machine's ram. Note to others, since ZFS doesn't use the buffer cache, the opposite is tr= ue for ZFS.) rick