From owner-svn-src-all@freebsd.org Tue Sep 15 22:13:22 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 71AD53E2073; Tue, 15 Sep 2020 22:13:22 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Brctp2Nh4z3VtS; Tue, 15 Sep 2020 22:13:22 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 35D88F0AD; Tue, 15 Sep 2020 22:13:22 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 08FMDLk8065413; Tue, 15 Sep 2020 22:13:21 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 08FMDLZv065410; Tue, 15 Sep 2020 22:13:21 GMT (envelope-from kib@FreeBSD.org) Message-Id: <202009152213.08FMDLZv065410@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Tue, 15 Sep 2020 22:13:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r365786 - head/sys/fs/tmpfs X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: head/sys/fs/tmpfs X-SVN-Commit-Revision: 365786 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Sep 2020 22:13:22 -0000 Author: kib Date: Tue Sep 15 22:13:21 2020 New Revision: 365786 URL: https://svnweb.freebsd.org/changeset/base/365786 Log: Microoptimize tmpfs node ref/unref by using atomics. Avoid tmpfs mount and node locks when ref count is greater than zero, which is the case until node is being destroyed by unlink or unmount. Reviewed by: markj Tested by: pho Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D26346 Modified: head/sys/fs/tmpfs/tmpfs.h head/sys/fs/tmpfs/tmpfs_subr.c head/sys/fs/tmpfs/tmpfs_vnops.c Modified: head/sys/fs/tmpfs/tmpfs.h ============================================================================== --- head/sys/fs/tmpfs/tmpfs.h Tue Sep 15 22:06:36 2020 (r365785) +++ head/sys/fs/tmpfs/tmpfs.h Tue Sep 15 22:13:21 2020 (r365786) @@ -228,7 +228,7 @@ struct tmpfs_node { int tn_vpstate; /* (i) */ /* Transient refcounter on this node. */ - u_int tn_refcount; /* (m) + (i) */ + u_int tn_refcount; /* 0<->1 (m) + (i) */ /* misc data field for different tn_type node */ union { @@ -412,7 +412,6 @@ struct tmpfs_dir_cursor { */ void tmpfs_ref_node(struct tmpfs_node *node); -void tmpfs_ref_node_locked(struct tmpfs_node *node); int tmpfs_alloc_node(struct mount *mp, struct tmpfs_mount *, enum vtype, uid_t uid, gid_t gid, mode_t mode, struct tmpfs_node *, const char *, dev_t, struct tmpfs_node **); Modified: head/sys/fs/tmpfs/tmpfs_subr.c ============================================================================== --- head/sys/fs/tmpfs/tmpfs_subr.c Tue Sep 15 22:06:36 2020 (r365785) +++ head/sys/fs/tmpfs/tmpfs_subr.c Tue Sep 15 22:13:21 2020 (r365786) @@ -49,6 +49,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -211,23 +212,17 @@ tmpfs_pages_check_avail(struct tmpfs_mount *tmp, size_ void tmpfs_ref_node(struct tmpfs_node *node) { +#ifdef INVARIANTS + u_int old; - TMPFS_NODE_LOCK(node); - tmpfs_ref_node_locked(node); - TMPFS_NODE_UNLOCK(node); + old = +#endif + refcount_acquire(&node->tn_refcount); +#ifdef INVARIANTS + KASSERT(old > 0, ("node %p zero refcount", node)); +#endif } -void -tmpfs_ref_node_locked(struct tmpfs_node *node) -{ - - TMPFS_NODE_ASSERT_LOCKED(node); - KASSERT(node->tn_refcount > 0, ("node %p zero refcount", node)); - KASSERT(node->tn_refcount < UINT_MAX, ("node %p refcount %u", node, - node->tn_refcount)); - node->tn_refcount++; -} - /* * Allocates a new node of type 'type' inside the 'tmp' mount point, with * its owner set to 'uid', its group to 'gid' and its mode set to 'mode', @@ -370,6 +365,8 @@ tmpfs_alloc_node(struct mount *mp, struct tmpfs_mount void tmpfs_free_node(struct tmpfs_mount *tmp, struct tmpfs_node *node) { + if (refcount_release_if_not_last(&node->tn_refcount)) + return; TMPFS_LOCK(tmp); TMPFS_NODE_LOCK(node); @@ -384,19 +381,19 @@ tmpfs_free_node_locked(struct tmpfs_mount *tmp, struct bool detach) { vm_object_t uobj; + bool last; TMPFS_MP_ASSERT_LOCKED(tmp); TMPFS_NODE_ASSERT_LOCKED(node); - KASSERT(node->tn_refcount > 0, ("node %p refcount zero", node)); - node->tn_refcount--; - if (node->tn_attached && (detach || node->tn_refcount == 0)) { + last = refcount_release(&node->tn_refcount); + if (node->tn_attached && (detach || last)) { MPASS(tmp->tm_nodes_inuse > 0); tmp->tm_nodes_inuse--; LIST_REMOVE(node, tn_entries); node->tn_attached = false; } - if (node->tn_refcount > 0) + if (!last) return (false); #ifdef INVARIANTS @@ -596,7 +593,7 @@ tmpfs_alloc_vp(struct mount *mp, struct tmpfs_node *no error = 0; tm = VFS_TO_TMPFS(mp); TMPFS_NODE_LOCK(node); - tmpfs_ref_node_locked(node); + tmpfs_ref_node(node); loop: TMPFS_NODE_ASSERT_LOCKED(node); if ((vp = node->tn_vnode) != NULL) { Modified: head/sys/fs/tmpfs/tmpfs_vnops.c ============================================================================== --- head/sys/fs/tmpfs/tmpfs_vnops.c Tue Sep 15 22:06:36 2020 (r365785) +++ head/sys/fs/tmpfs/tmpfs_vnops.c Tue Sep 15 22:13:21 2020 (r365786) @@ -1663,7 +1663,7 @@ restart: if (tnp->tn_type != VDIR) continue; TMPFS_NODE_LOCK(tnp); - tmpfs_ref_node_locked(tnp); + tmpfs_ref_node(tnp); /* * tn_vnode cannot be instantiated while we hold the