From owner-dev-commits-src-all@freebsd.org Thu Sep 16 00:32:59 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 59C7767126B; Thu, 16 Sep 2021 00:32:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H8yjR24DNz4QlV; Thu, 16 Sep 2021 00:32:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2787717A6E; Thu, 16 Sep 2021 00:32:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 18G0WxFn063884; Thu, 16 Sep 2021 00:32:59 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 18G0Wxbc063883; Thu, 16 Sep 2021 00:32:59 GMT (envelope-from git) Date: Thu, 16 Sep 2021 00:32:59 GMT Message-Id: <202109160032.18G0Wxbc063883@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Rick Macklem Subject: git: 9ebe4b8c67bf - main - nfscl: Add vfs.nfs.maxalloclen to limit Allocate/Deallocate RPC RTT MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 9ebe4b8c67bf823e62617ba9fbd3c9c1768c8b3b Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2021 00:32:59 -0000 The branch main has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=9ebe4b8c67bf823e62617ba9fbd3c9c1768c8b3b commit 9ebe4b8c67bf823e62617ba9fbd3c9c1768c8b3b Author: Rick Macklem AuthorDate: 2021-09-16 00:29:45 +0000 Commit: Rick Macklem CommitDate: 2021-09-16 00:29:45 +0000 nfscl: Add vfs.nfs.maxalloclen to limit Allocate/Deallocate RPC RTT Unlike Copy, the NFSv4.2 Allocate and Deallocate operations do not allow a reply with partial completion. As such, the only way to limit the time the operation takes to provide a reasonable RPC RTT is to limit the size of the allocation/deallocation in the NFSv4.2 client. This patch adds a sysctl called vfs.nfs.maxalloclen to set the limit on the size of the Allocate operation. There is no way to know how long a server will take to do an allocate operation, but 64Mbytes results in a reasonable RPC RTT for the slow hardware I test on, so that is what the default value for vfs.nfs.maxalloclen is set to. For an 8Gbyte allocation, the elapsed time for doing it in 64Mbyte chunks was the same as the elapsed time taken for a single large allocation operation for a FreeBSD server with a UFS file system. MFC after: 2 weeks --- sys/fs/nfsclient/nfs_clvnops.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/sys/fs/nfsclient/nfs_clvnops.c b/sys/fs/nfsclient/nfs_clvnops.c index bfd77d50bc91..85b5dd9cfbb1 100644 --- a/sys/fs/nfsclient/nfs_clvnops.c +++ b/sys/fs/nfsclient/nfs_clvnops.c @@ -301,6 +301,10 @@ int newnfs_directio_allow_mmap = 1; SYSCTL_INT(_vfs_nfs, OID_AUTO, nfs_directio_allow_mmap, CTLFLAG_RW, &newnfs_directio_allow_mmap, 0, "Enable mmaped IO on file with O_DIRECT opens"); +static uint64_t nfs_maxalloclen = 64 * 1024 * 1024; +SYSCTL_U64(_vfs_nfs, OID_AUTO, maxalloclen, CTLFLAG_RW, + &nfs_maxalloclen, 0, "NFS max allocate/deallocate length"); + #define NFSACCESS_ALL (NFSACCESS_READ | NFSACCESS_MODIFY \ | NFSACCESS_EXTEND | NFSACCESS_EXECUTE \ | NFSACCESS_DELETE | NFSACCESS_LOOKUP) @@ -3639,6 +3643,7 @@ nfs_allocate(struct vop_allocate_args *ap) struct thread *td = curthread; struct nfsvattr nfsva; struct nfsmount *nmp; + off_t alen; int attrflag, error, ret; attrflag = 0; @@ -3652,12 +3657,16 @@ nfs_allocate(struct vop_allocate_args *ap) * file's allocation on the server. */ error = ncl_flush(vp, MNT_WAIT, td, 1, 0); - if (error == 0) - error = nfsrpc_allocate(vp, *ap->a_offset, *ap->a_len, + if (error == 0) { + alen = *ap->a_len; + if ((uint64_t)alen > nfs_maxalloclen) + alen = nfs_maxalloclen; + error = nfsrpc_allocate(vp, *ap->a_offset, alen, &nfsva, &attrflag, td->td_ucred, td, NULL); + } if (error == 0) { - *ap->a_offset += *ap->a_len; - *ap->a_len = 0; + *ap->a_offset += alen; + *ap->a_len -= alen; } else if (error == NFSERR_NOTSUPP) { mtx_lock(&nmp->nm_mtx); nmp->nm_privflag |= NFSMNTP_NOALLOCATE;