From owner-svn-src-head@freebsd.org Thu Jul 16 14:09:19 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2AB5E362E59; Thu, 16 Jul 2020 14:09:19 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B6x2R0S6Rz4Z4Z; Thu, 16 Jul 2020 14:09:19 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E6E4992EF; Thu, 16 Jul 2020 14:09:18 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 06GE9I4h045264; Thu, 16 Jul 2020 14:09:18 GMT (envelope-from mjg@FreeBSD.org) Received: (from mjg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 06GE9Ihc045261; Thu, 16 Jul 2020 14:09:18 GMT (envelope-from mjg@FreeBSD.org) Message-Id: <202007161409.06GE9Ihc045261@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mjg set sender to mjg@FreeBSD.org using -f From: Mateusz Guzik Date: Thu, 16 Jul 2020 14:09:18 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r363249 - in head/sys: kern security/mac X-SVN-Group: head X-SVN-Commit-Author: mjg X-SVN-Commit-Paths: in head/sys: kern security/mac X-SVN-Commit-Revision: 363249 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Jul 2020 14:09:19 -0000 Author: mjg Date: Thu Jul 16 14:09:18 2020 New Revision: 363249 URL: https://svnweb.freebsd.org/changeset/base/363249 Log: vfs: fix vn_poll performance with either MAC or AUDIT The code would unconditionally lock the vnode to audit or call the mac hoook, even if neither want to do anything. Pre-check the state to avoid locking in the common case of nothing to do. Note this code should not be normally executed anyway as vnodes are always return ready. However, poll1/2 from will-it-scale use regular files for benchmarking, presumably to focus on the interface itself as the vnode handler is not supposed to do almost anything. This in particular fixes poll2 which passes 128 fds. $ ./poll2_processes -s 10 before: 134411 after: 271572 Modified: head/sys/kern/vfs_vnops.c head/sys/security/mac/mac_framework.c head/sys/security/mac/mac_framework.h Modified: head/sys/kern/vfs_vnops.c ============================================================================== --- head/sys/kern/vfs_vnops.c Thu Jul 16 14:04:29 2020 (r363248) +++ head/sys/kern/vfs_vnops.c Thu Jul 16 14:09:18 2020 (r363249) @@ -1636,12 +1636,14 @@ vn_poll(struct file *fp, int events, struct ucred *act vp = fp->f_vnode; #if defined(MAC) || defined(AUDIT) - vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); - AUDIT_ARG_VNODE1(vp); - error = mac_vnode_check_poll(active_cred, fp->f_cred, vp); - VOP_UNLOCK(vp); - if (error != 0) - return (error); + if (AUDITING_TD(td) || mac_vnode_check_poll_enabled()) { + vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); + AUDIT_ARG_VNODE1(vp); + error = mac_vnode_check_poll(active_cred, fp->f_cred, vp); + VOP_UNLOCK(vp); + if (error != 0) + return (error); + } #endif error = VOP_POLL(vp, events, fp->f_cred, td); return (error); Modified: head/sys/security/mac/mac_framework.c ============================================================================== --- head/sys/security/mac/mac_framework.c Thu Jul 16 14:04:29 2020 (r363248) +++ head/sys/security/mac/mac_framework.c Thu Jul 16 14:09:18 2020 (r363249) @@ -118,11 +118,18 @@ SYSCTL_UINT(_security_mac, OID_AUTO, version, CTLFLAG_ ""); /* - * Flags for inlined checks. + * Flags for inlined checks. Note this would be best hotpatched at runtime. + * The following is a band-aid. + * + * Use FPFLAG for hooks running in commonly executed paths and FPFLAG_RARE + * for the rest. */ #define FPFLAG(f) \ bool __read_frequently mac_##f##_fp_flag +#define FPFLAG_RARE(f) \ +bool __read_mostly mac_##f##_fp_flag + FPFLAG(priv_check); FPFLAG(priv_grant); FPFLAG(vnode_check_lookup); @@ -131,8 +138,10 @@ FPFLAG(vnode_check_stat); FPFLAG(vnode_check_read); FPFLAG(vnode_check_write); FPFLAG(vnode_check_mmap); +FPFLAG_RARE(vnode_check_poll); #undef FPFLAG +#undef FPFLAG_RARE /* * Labels consist of a indexed set of "slots", which are allocated policies @@ -416,6 +425,8 @@ struct mac_policy_fastpath_elem mac_policy_fastpath_ar .flag = &mac_vnode_check_write_fp_flag }, { .offset = FPO(vnode_check_mmap), .flag = &mac_vnode_check_mmap_fp_flag }, + { .offset = FPO(vnode_check_poll), + .flag = &mac_vnode_check_poll_fp_flag }, }; static void Modified: head/sys/security/mac/mac_framework.h ============================================================================== --- head/sys/security/mac/mac_framework.h Thu Jul 16 14:04:29 2020 (r363248) +++ head/sys/security/mac/mac_framework.h Thu Jul 16 14:09:18 2020 (r363249) @@ -463,10 +463,14 @@ mac_vnode_check_open(struct ucred *cred, struct vnode int mac_vnode_check_mprotect(struct ucred *cred, struct vnode *vp, int prot); + +#define mac_vnode_check_poll_enabled() __predict_false(mac_vnode_check_poll_fp_flag) #ifdef MAC +extern bool mac_vnode_check_poll_fp_flag; int mac_vnode_check_poll(struct ucred *active_cred, struct ucred *file_cred, struct vnode *vp); #else +#define mac_vnode_check_poll_fp_flag 0 static inline int mac_vnode_check_poll(struct ucred *active_cred, struct ucred *file_cred, struct vnode *vp)