From owner-dev-commits-src-all@freebsd.org Sat Jun 26 23:04:44 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3B8FB65B90C; Sat, 26 Jun 2021 23:04:44 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GC8b01G6kz3vs7; Sat, 26 Jun 2021 23:04:44 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0A16E27177; Sat, 26 Jun 2021 23:04:44 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 15QN4hlO069327; Sat, 26 Jun 2021 23:04:43 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 15QN4hCG069326; Sat, 26 Jun 2021 23:04:43 GMT (envelope-from git) Date: Sat, 26 Jun 2021 23:04:43 GMT Message-Id: <202106262304.15QN4hCG069326@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Rick Macklem Subject: git: 6c5dae287ee7 - stable/13 - nfscl: Add a "has acquired a delegation" flag for delegations MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 6c5dae287ee722036240bde5bc0b3a8d5a000836 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jun 2021 23:04:44 -0000 The branch stable/13 has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=6c5dae287ee722036240bde5bc0b3a8d5a000836 commit 6c5dae287ee722036240bde5bc0b3a8d5a000836 Author: Rick Macklem AuthorDate: 2021-06-09 15:00:43 +0000 Commit: Rick Macklem CommitDate: 2021-06-26 23:01:09 +0000 nfscl: Add a "has acquired a delegation" flag for delegations A problem was reported via email, where a large (130000+) accumulation of NFSv4 opens on an NFSv4 mount caused significant lock contention on the mutex used to protect the client mount's open/lock state. Although the root cause for the accumulation of opens was not resolved, it is obvious that the NFSv4 client is not designed to handle 100000+ opens efficiently. For a common case where delegations are not being issued by the NFSv4 server, the code acquires the mutex lock for open/lock state, finds the delegation list empty and just unlocks the mutex and returns. This patch adds an NFS mount point flag that is set when a delegation is issued for the mount. Then the patched code checks for this flag before acquiring the open/lock mutex, avoiding the need to acquire the lock for the case where delegations are not being issued by the NFSv4 server. This change appears to be performance neutral for a small number of opens, but should reduce lock contention for a large number of opens for the common case where server is not issuing delegations. This commit should not affect the high level semantics of delegation handling. (cherry picked from commit 5e5ca4c8fc53d31bf71e182e08c4ccd8290428c7) --- sys/fs/nfsclient/nfs_clstate.c | 44 +++++++++++++++++++++++++++++++++++++++--- sys/fs/nfsclient/nfsmount.h | 1 + 2 files changed, 42 insertions(+), 3 deletions(-) diff --git a/sys/fs/nfsclient/nfs_clstate.c b/sys/fs/nfsclient/nfs_clstate.c index 5e4ac2ae9d4d..0659e77289e9 100644 --- a/sys/fs/nfsclient/nfs_clstate.c +++ b/sys/fs/nfsclient/nfs_clstate.c @@ -438,20 +438,32 @@ nfscl_deleg(mount_t mp, struct nfsclclient *clp, u_int8_t *nfhp, int fhlen, struct ucred *cred, NFSPROC_T *p, struct nfscldeleg **dpp) { struct nfscldeleg *dp = *dpp, *tdp; + struct nfsmount *nmp; + KASSERT(mp != NULL, ("nfscl_deleg: mp NULL")); + nmp = VFSTONFS(mp); /* * First, if we have received a Read delegation for a file on a * read/write file system, just return it, because they aren't * useful, imho. */ - if (mp != NULL && dp != NULL && !NFSMNT_RDONLY(mp) && + if (dp != NULL && !NFSMNT_RDONLY(mp) && (dp->nfsdl_flags & NFSCLDL_READ)) { - (void) nfscl_trydelegreturn(dp, cred, VFSTONFS(mp), p); + nfscl_trydelegreturn(dp, cred, nmp, p); free(dp, M_NFSCLDELEG); *dpp = NULL; return (0); } + /* + * Since a delegation might be added to the mount, + * set NFSMNTP_DELEGISSUED now. If a delegation already + * exagain ists, setting this flag is harmless. + */ + NFSLOCKMNT(nmp); + nmp->nm_privflag |= NFSMNTP_DELEGISSUED; + NFSUNLOCKMNT(nmp); + /* Look for the correct deleg, based upon FH */ NFSLOCKCLSTATE(); tdp = nfscl_finddeleg(clp, nfhp, fhlen); @@ -3356,12 +3368,20 @@ nfscl_delegreturnvp(vnode_t vp, NFSPROC_T *p) struct nfscldeleg *dp; struct ucred *cred; struct nfsnode *np; + struct nfsmount *nmp; + nmp = VFSTONFS(vp->v_mount); + NFSLOCKMNT(nmp); + if ((nmp->nm_privflag & NFSMNTP_DELEGISSUED) == 0) { + NFSUNLOCKMNT(nmp); + return; + } + NFSUNLOCKMNT(nmp); np = VTONFS(vp); cred = newnfs_getcred(); dp = NULL; NFSLOCKCLSTATE(); - clp = VFSTONFS(vp->v_mount)->nm_clp; + clp = nmp->nm_clp; if (clp != NULL) dp = nfscl_finddeleg(clp, np->n_fhp->nfh_fh, np->n_fhp->nfh_len); @@ -4500,6 +4520,12 @@ nfscl_nodeleg(vnode_t vp, int writedeleg) nmp = VFSTONFS(vp->v_mount); if (!NFSHASNFSV4(nmp)) return (1); + NFSLOCKMNT(nmp); + if ((nmp->nm_privflag & NFSMNTP_DELEGISSUED) == 0) { + NFSUNLOCKMNT(nmp); + return (1); + } + NFSUNLOCKMNT(nmp); NFSLOCKCLSTATE(); clp = nfscl_findcl(nmp); if (clp == NULL) { @@ -4856,6 +4882,12 @@ nfscl_delegmodtime(vnode_t vp) nmp = VFSTONFS(vp->v_mount); if (!NFSHASNFSV4(nmp)) return; + NFSLOCKMNT(nmp); + if ((nmp->nm_privflag & NFSMNTP_DELEGISSUED) == 0) { + NFSUNLOCKMNT(nmp); + return; + } + NFSUNLOCKMNT(nmp); NFSLOCKCLSTATE(); clp = nfscl_findcl(nmp); if (clp == NULL) { @@ -4885,6 +4917,12 @@ nfscl_deleggetmodtime(vnode_t vp, struct timespec *mtime) nmp = VFSTONFS(vp->v_mount); if (!NFSHASNFSV4(nmp)) return; + NFSLOCKMNT(nmp); + if ((nmp->nm_privflag & NFSMNTP_DELEGISSUED) == 0) { + NFSUNLOCKMNT(nmp); + return; + } + NFSUNLOCKMNT(nmp); NFSLOCKCLSTATE(); clp = nfscl_findcl(nmp); if (clp == NULL) { diff --git a/sys/fs/nfsclient/nfsmount.h b/sys/fs/nfsclient/nfsmount.h index 57adcd8f2fca..f8ea8c9a1418 100644 --- a/sys/fs/nfsclient/nfsmount.h +++ b/sys/fs/nfsclient/nfsmount.h @@ -115,6 +115,7 @@ struct nfsmount { #define NFSMNTP_NOXATTR 0x00000080 #define NFSMNTP_NOADVISE 0x00000100 #define NFSMNTP_NOALLOCATE 0x00000200 +#define NFSMNTP_DELEGISSUED 0x00000400 /* New mount flags only used by the kernel via nmount(2). */ #define NFSMNT_TLS 0x00000001