From owner-svn-src-head@freebsd.org Mon May 7 22:29:33 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2D357FBEBD0; Mon, 7 May 2018 22:29:33 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CFCB0850FB; Mon, 7 May 2018 22:29:32 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C62FC216FC; Mon, 7 May 2018 22:29:32 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w47MTW7N062493; Mon, 7 May 2018 22:29:32 GMT (envelope-from mjg@FreeBSD.org) Received: (from mjg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w47MTWFd062491; Mon, 7 May 2018 22:29:32 GMT (envelope-from mjg@FreeBSD.org) Message-Id: <201805072229.w47MTWFd062491@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mjg set sender to mjg@FreeBSD.org using -f From: Mateusz Guzik Date: Mon, 7 May 2018 22:29:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r333339 - in head/sys: kern sys X-SVN-Group: head X-SVN-Commit-Author: mjg X-SVN-Commit-Paths: in head/sys: kern sys X-SVN-Commit-Revision: 333339 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 May 2018 22:29:33 -0000 Author: mjg Date: Mon May 7 22:29:32 2018 New Revision: 333339 URL: https://svnweb.freebsd.org/changeset/base/333339 Log: Avoid calls to syscall_thread_enter/exit for statically defined syscalls The entire mechanism is rarely used and is quite not performant due to atomci ops on the syscall table. It also has added overhead for completely unrelated syscalls. Reduce it by avoiding the func calls if possible (which consistutes vast majority of cases). Provides about 3% syscall rate speed up for getuid on Broadwell. Modified: head/sys/kern/kern_syscalls.c head/sys/sys/sysent.h Modified: head/sys/kern/kern_syscalls.c ============================================================================== --- head/sys/kern/kern_syscalls.c Mon May 7 21:42:22 2018 (r333338) +++ head/sys/kern/kern_syscalls.c Mon May 7 22:29:32 2018 (r333339) @@ -80,14 +80,12 @@ syscall_thread_drain(struct sysent *se) } int -syscall_thread_enter(struct thread *td, struct sysent *se) +_syscall_thread_enter(struct thread *td, struct sysent *se) { u_int32_t cnt, oldcnt; do { oldcnt = se->sy_thrcnt; - if ((oldcnt & SY_THR_STATIC) != 0) - return (0); if ((oldcnt & (SY_THR_DRAINING | SY_THR_ABSENT)) != 0) return (ENOSYS); cnt = oldcnt + SY_THR_INCR; @@ -96,14 +94,12 @@ syscall_thread_enter(struct thread *td, struct sysent } void -syscall_thread_exit(struct thread *td, struct sysent *se) +_syscall_thread_exit(struct thread *td, struct sysent *se) { u_int32_t cnt, oldcnt; do { oldcnt = se->sy_thrcnt; - if ((oldcnt & SY_THR_STATIC) != 0) - return; cnt = oldcnt - SY_THR_INCR; } while (atomic_cmpset_rel_32(&se->sy_thrcnt, oldcnt, cnt) == 0); } Modified: head/sys/sys/sysent.h ============================================================================== --- head/sys/sys/sysent.h Mon May 7 21:42:22 2018 (r333338) +++ head/sys/sys/sysent.h Mon May 7 22:29:32 2018 (r333339) @@ -289,8 +289,26 @@ struct nosys_args; int lkmnosys(struct thread *, struct nosys_args *); int lkmressys(struct thread *, struct nosys_args *); -int syscall_thread_enter(struct thread *td, struct sysent *se); -void syscall_thread_exit(struct thread *td, struct sysent *se); +int _syscall_thread_enter(struct thread *td, struct sysent *se); +void _syscall_thread_exit(struct thread *td, struct sysent *se); + +static inline int +syscall_thread_enter(struct thread *td, struct sysent *se) +{ + + if (__predict_true((se->sy_thrcnt & SY_THR_STATIC) != 0)) + return (0); + return (_syscall_thread_enter(td, se)); +} + +static inline void +syscall_thread_exit(struct thread *td, struct sysent *se) +{ + + if (__predict_true((se->sy_thrcnt & SY_THR_STATIC) != 0)) + return; + _syscall_thread_exit(td, se); +} int shared_page_alloc(int size, int align); int shared_page_fill(int size, int align, const void *data);