From owner-svn-src-head@freebsd.org Fri Feb 14 13:14:20 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4C0602346F8; Fri, 14 Feb 2020 13:14:20 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48Jv3c0z7cz4ZZS; Fri, 14 Feb 2020 13:14:20 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1CACD2419D; Fri, 14 Feb 2020 13:14:20 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 01EDEJSd000657; Fri, 14 Feb 2020 13:14:19 GMT (envelope-from mjg@FreeBSD.org) Received: (from mjg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 01EDEJsQ000656; Fri, 14 Feb 2020 13:14:19 GMT (envelope-from mjg@FreeBSD.org) Message-Id: <202002141314.01EDEJsQ000656@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mjg set sender to mjg@FreeBSD.org using -f From: Mateusz Guzik Date: Fri, 14 Feb 2020 13:14:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r357914 - head/sys/amd64/amd64 X-SVN-Group: head X-SVN-Commit-Author: mjg X-SVN-Commit-Paths: head/sys/amd64/amd64 X-SVN-Commit-Revision: 357914 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2020 13:14:20 -0000 Author: mjg Date: Fri Feb 14 13:14:19 2020 New Revision: 357914 URL: https://svnweb.freebsd.org/changeset/base/357914 Log: amd64: only check for error != 0 in the inlined part of l1d flush check this replaces the following near the syscall exit: cmp $0x39,%rax ja 0xffffffff8108f82c movabs $0x200001800060005,%rcx bt %rax,%rcx jae 0xffffffff8108f82c with: test %edi,%edi jne 0xffffffff8091a49c Modified: head/sys/amd64/amd64/trap.c Modified: head/sys/amd64/amd64/trap.c ============================================================================== --- head/sys/amd64/amd64/trap.c Fri Feb 14 13:10:52 2020 (r357913) +++ head/sys/amd64/amd64/trap.c Fri Feb 14 13:14:19 2020 (r357914) @@ -1068,25 +1068,32 @@ flush_l1d_hw(void) wrmsr(MSR_IA32_FLUSH_CMD, IA32_FLUSH_CMD_L1D); } -static void __inline -amd64_syscall_ret_flush_l1d_inline(int error) +static void __noinline +amd64_syscall_ret_flush_l1d_check(int error) { void (*p)(void); - if (error != 0 && error != EEXIST && error != EAGAIN && - error != EXDEV && error != ENOENT && error != ENOTCONN && - error != EINPROGRESS) { - p = syscall_ret_l1d_flush; + if (error != EEXIST && error != EAGAIN && error != EXDEV && + error != ENOENT && error != ENOTCONN && error != EINPROGRESS) { + p = (void *)atomic_load_ptr(&syscall_ret_l1d_flush); if (p != NULL) p(); } } +static void __inline +amd64_syscall_ret_flush_l1d_check_inline(int error) +{ + + if (__predict_false(error != 0)) + amd64_syscall_ret_flush_l1d_check(error); +} + void amd64_syscall_ret_flush_l1d(int error) { - amd64_syscall_ret_flush_l1d_inline(error); + amd64_syscall_ret_flush_l1d_check_inline(error); } void @@ -1190,5 +1197,5 @@ amd64_syscall(struct thread *td, int traced) if (__predict_false(td->td_frame->tf_rip >= VM_MAXUSER_ADDRESS)) set_pcb_flags(td->td_pcb, PCB_FULL_IRET); - amd64_syscall_ret_flush_l1d_inline(td->td_errno); + amd64_syscall_ret_flush_l1d_check_inline(td->td_errno); }