From owner-svn-src-head@freebsd.org Tue Aug 4 19:54:11 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C6EC537DC90; Tue, 4 Aug 2020 19:54:11 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLlnb4yFsz3Xr9; Tue, 4 Aug 2020 19:54:11 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8D4E4A7D3; Tue, 4 Aug 2020 19:54:11 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 074JsBg3009764; Tue, 4 Aug 2020 19:54:11 GMT (envelope-from mjg@FreeBSD.org) Received: (from mjg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 074JsBEP009762; Tue, 4 Aug 2020 19:54:11 GMT (envelope-from mjg@FreeBSD.org) Message-Id: <202008041954.074JsBEP009762@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mjg set sender to mjg@FreeBSD.org using -f From: Mateusz Guzik Date: Tue, 4 Aug 2020 19:54:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r363856 - in head/sys: kern sys X-SVN-Group: head X-SVN-Commit-Author: mjg X-SVN-Commit-Paths: in head/sys: kern sys X-SVN-Commit-Revision: 363856 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2020 19:54:11 -0000 Author: mjg Date: Tue Aug 4 19:54:10 2020 New Revision: 363856 URL: https://svnweb.freebsd.org/changeset/base/363856 Log: vfs: add vfs_op_thread_enter/exit _crit variants and employ them in the namecache. Eliminates all spurious checks for preemption. Modified: head/sys/kern/vfs_cache.c head/sys/sys/mount.h Modified: head/sys/kern/vfs_cache.c ============================================================================== --- head/sys/kern/vfs_cache.c Tue Aug 4 19:52:52 2020 (r363855) +++ head/sys/kern/vfs_cache.c Tue Aug 4 19:54:10 2020 (r363856) @@ -3484,29 +3484,29 @@ cache_fplookup_climb_mount(struct cache_fpl *fpl) prev_mp = NULL; for (;;) { - if (!vfs_op_thread_enter(mp)) { + if (!vfs_op_thread_enter_crit(mp)) { if (prev_mp != NULL) - vfs_op_thread_exit(prev_mp); + vfs_op_thread_exit_crit(prev_mp); return (cache_fpl_partial(fpl)); } if (prev_mp != NULL) - vfs_op_thread_exit(prev_mp); + vfs_op_thread_exit_crit(prev_mp); if (!vn_seqc_consistent(vp, vp_seqc)) { - vfs_op_thread_exit(mp); + vfs_op_thread_exit_crit(mp); return (cache_fpl_partial(fpl)); } if (!cache_fplookup_mp_supported(mp)) { - vfs_op_thread_exit(mp); + vfs_op_thread_exit_crit(mp); return (cache_fpl_partial(fpl)); } vp = atomic_load_ptr(&mp->mnt_rootvnode); if (vp == NULL || VN_IS_DOOMED(vp)) { - vfs_op_thread_exit(mp); + vfs_op_thread_exit_crit(mp); return (cache_fpl_partial(fpl)); } vp_seqc = vn_seqc_read_any(vp); if (seqc_in_modify(vp_seqc)) { - vfs_op_thread_exit(mp); + vfs_op_thread_exit_crit(mp); return (cache_fpl_partial(fpl)); } prev_mp = mp; @@ -3515,7 +3515,7 @@ cache_fplookup_climb_mount(struct cache_fpl *fpl) break; } - vfs_op_thread_exit(prev_mp); + vfs_op_thread_exit_crit(prev_mp); fpl->tvp = vp; fpl->tvp_seqc = vp_seqc; return (0); Modified: head/sys/sys/mount.h ============================================================================== --- head/sys/sys/mount.h Tue Aug 4 19:52:52 2020 (r363855) +++ head/sys/sys/mount.h Tue Aug 4 19:54:10 2020 (r363856) @@ -1023,23 +1023,36 @@ int vfs_mount_fetch_counter(struct mount *, enum mount *zpcpu_get(mp->mnt_thread_in_ops_pcpu) == 1; \ }) -#define vfs_op_thread_enter(mp) ({ \ - bool _retval = true; \ - critical_enter(); \ +#define vfs_op_thread_enter_crit(mp) ({ \ + bool _retval_crit = true; \ + MPASS(curthread->td_critnest > 0); \ MPASS(!vfs_op_thread_entered(mp)); \ zpcpu_set_protected(mp->mnt_thread_in_ops_pcpu, 1); \ __compiler_membar(); \ if (__predict_false(mp->mnt_vfs_ops > 0)) { \ - vfs_op_thread_exit(mp); \ - _retval = false; \ + vfs_op_thread_exit_crit(mp); \ + _retval_crit = false; \ } \ + _retval_crit; \ +}) + +#define vfs_op_thread_enter(mp) ({ \ + bool _retval; \ + critical_enter(); \ + _retval = vfs_op_thread_enter_crit(mp); \ + if (__predict_false(!_retval)) \ + critical_exit(); \ _retval; \ }) -#define vfs_op_thread_exit(mp) do { \ +#define vfs_op_thread_exit_crit(mp) do { \ MPASS(vfs_op_thread_entered(mp)); \ __compiler_membar(); \ zpcpu_set_protected(mp->mnt_thread_in_ops_pcpu, 0); \ +} while (0) + +#define vfs_op_thread_exit(mp) do { \ + vfs_op_thread_exit_crit(mp); \ critical_exit(); \ } while (0)