From owner-svn-src-head@freebsd.org Sun Nov 8 15:55:00 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 444CF464C75; Sun, 8 Nov 2020 15:55:00 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CTdxJ1Lgrz3qLd; Sun, 8 Nov 2020 15:55:00 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2116922590; Sun, 8 Nov 2020 15:55:00 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0A8Ft0Y4041946; Sun, 8 Nov 2020 15:55:00 GMT (envelope-from trasz@FreeBSD.org) Received: (from trasz@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0A8FsxtL041919; Sun, 8 Nov 2020 15:54:59 GMT (envelope-from trasz@FreeBSD.org) Message-Id: <202011081554.0A8FsxtL041919@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: trasz set sender to trasz@FreeBSD.org using -f From: Edward Tomasz Napierala Date: Sun, 8 Nov 2020 15:54:59 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r367488 - in head/sys: kern sys X-SVN-Group: head X-SVN-Commit-Author: trasz X-SVN-Commit-Paths: in head/sys: kern sys X-SVN-Commit-Revision: 367488 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Nov 2020 15:55:00 -0000 Author: trasz Date: Sun Nov 8 15:54:59 2020 New Revision: 367488 URL: https://svnweb.freebsd.org/changeset/base/367488 Log: Move syscall_thread_{enter,exit}() into the slow path. This is only needed for syscalls from unloadable modules. Reviewed by: kib MFC after: 2 weeks Sponsored by: EPSRC Differential Revision: https://reviews.freebsd.org/D26988 Modified: head/sys/kern/kern_syscalls.c head/sys/kern/subr_syscall.c head/sys/sys/sysent.h Modified: head/sys/kern/kern_syscalls.c ============================================================================== --- head/sys/kern/kern_syscalls.c Sun Nov 8 14:08:00 2020 (r367487) +++ head/sys/kern/kern_syscalls.c Sun Nov 8 15:54:59 2020 (r367488) @@ -80,10 +80,13 @@ syscall_thread_drain(struct sysent *se) } int -_syscall_thread_enter(struct thread *td, struct sysent *se) +syscall_thread_enter(struct thread *td, struct sysent *se) { u_int32_t cnt, oldcnt; + KASSERT((se->sy_thrcnt & SY_THR_STATIC) == 0, + ("%s: not a static syscall", __func__)); + do { oldcnt = se->sy_thrcnt; if ((oldcnt & (SY_THR_DRAINING | SY_THR_ABSENT)) != 0) @@ -94,9 +97,12 @@ _syscall_thread_enter(struct thread *td, struct sysent } void -_syscall_thread_exit(struct thread *td, struct sysent *se) +syscall_thread_exit(struct thread *td, struct sysent *se) { u_int32_t cnt, oldcnt; + + KASSERT((se->sy_thrcnt & SY_THR_STATIC) == 0, + ("%s: not a static syscall", __func__)); do { oldcnt = se->sy_thrcnt; Modified: head/sys/kern/subr_syscall.c ============================================================================== --- head/sys/kern/subr_syscall.c Sun Nov 8 14:08:00 2020 (r367487) +++ head/sys/kern/subr_syscall.c Sun Nov 8 15:54:59 2020 (r367488) @@ -61,6 +61,7 @@ syscallenter(struct thread *td) struct syscall_args *sa; struct sysent *se; int error, traced; + bool sy_thr_static; VM_CNT_INC(v_syscall); p = td->td_proc; @@ -128,12 +129,6 @@ syscallenter(struct thread *td) } #endif - error = syscall_thread_enter(td, se); - if (error != 0) { - td->td_errno = error; - goto retval; - } - /* * Fetch fast sigblock value at the time of syscall entry to * handle sleepqueue primitives which might call cursig(). @@ -145,8 +140,19 @@ syscallenter(struct thread *td) KASSERT((td->td_pflags & TDP_NERRNO) == 0, ("%s: TDP_NERRNO set", __func__)); + sy_thr_static = (se->sy_thrcnt & SY_THR_STATIC) != 0; + if (__predict_false(SYSTRACE_ENABLED() || - AUDIT_SYSCALL_ENTER(sa->code, td))) { + AUDIT_SYSCALL_ENTER(sa->code, td) || + !sy_thr_static)) { + if (!sy_thr_static) { + error = syscall_thread_enter(td, se); + if (error != 0) { + td->td_errno = error; + goto retval; + } + } + #ifdef KDTRACE_HOOKS /* Give the syscall:::entry DTrace probe a chance to fire. */ if (__predict_false(se->sy_entry != 0)) @@ -176,6 +182,9 @@ syscallenter(struct thread *td) (*systrace_probe_func)(sa, SYSTRACE_RETURN, error ? -1 : td->td_retval[0]); #endif + + if (!sy_thr_static) + syscall_thread_exit(td, se); } else { error = (se->sy_call)(td, sa->args); /* Save the latest error return value. */ @@ -184,7 +193,6 @@ syscallenter(struct thread *td) else td->td_errno = error; } - syscall_thread_exit(td, se); retval: KTR_STOP4(KTR_SYSC, "syscall", syscallname(p, sa->code), Modified: head/sys/sys/sysent.h ============================================================================== --- head/sys/sys/sysent.h Sun Nov 8 14:08:00 2020 (r367487) +++ head/sys/sys/sysent.h Sun Nov 8 15:54:59 2020 (r367488) @@ -294,26 +294,8 @@ struct nosys_args; int lkmnosys(struct thread *, struct nosys_args *); int lkmressys(struct thread *, struct nosys_args *); -int _syscall_thread_enter(struct thread *td, struct sysent *se); -void _syscall_thread_exit(struct thread *td, struct sysent *se); - -static inline int -syscall_thread_enter(struct thread *td, struct sysent *se) -{ - - if (__predict_true((se->sy_thrcnt & SY_THR_STATIC) != 0)) - return (0); - return (_syscall_thread_enter(td, se)); -} - -static inline void -syscall_thread_exit(struct thread *td, struct sysent *se) -{ - - if (__predict_true((se->sy_thrcnt & SY_THR_STATIC) != 0)) - return; - _syscall_thread_exit(td, se); -} +int syscall_thread_enter(struct thread *td, struct sysent *se); +void syscall_thread_exit(struct thread *td, struct sysent *se); int shared_page_alloc(int size, int align); int shared_page_fill(int size, int align, const void *data);