From owner-svn-src-all@freebsd.org Tue Mar 3 15:02:09 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 36CFD252459; Tue, 3 Mar 2020 15:02:09 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48X0bj0jmfz3ylX; Tue, 3 Mar 2020 15:02:09 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 136536FA1; Tue, 3 Mar 2020 15:02:09 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 023F28RW033312; Tue, 3 Mar 2020 15:02:08 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 023F27tF033304; Tue, 3 Mar 2020 15:02:07 GMT (envelope-from kib@FreeBSD.org) Message-Id: <202003031502.023F27tF033304@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Tue, 3 Mar 2020 15:02:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r358579 - in stable/12/sys: amd64/amd64 dev/cpuctl i386/i386 x86/acpica x86/include x86/x86 X-SVN-Group: stable-12 X-SVN-Commit-Author: kib X-SVN-Commit-Paths: in stable/12/sys: amd64/amd64 dev/cpuctl i386/i386 x86/acpica x86/include x86/x86 X-SVN-Commit-Revision: 358579 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2020 15:02:09 -0000 Author: kib Date: Tue Mar 3 15:02:07 2020 New Revision: 358579 URL: https://svnweb.freebsd.org/changeset/base/358579 Log: MFC r358315: Fix IBRS for machines with IBRS_ALL capability. Modified: stable/12/sys/amd64/amd64/initcpu.c stable/12/sys/amd64/amd64/support.S stable/12/sys/dev/cpuctl/cpuctl.c stable/12/sys/i386/i386/support.s stable/12/sys/x86/acpica/acpi_wakeup.c stable/12/sys/x86/include/x86_var.h stable/12/sys/x86/x86/cpu_machdep.c Directory Properties: stable/12/ (props changed) Modified: stable/12/sys/amd64/amd64/initcpu.c ============================================================================== --- stable/12/sys/amd64/amd64/initcpu.c Tue Mar 3 14:58:53 2020 (r358578) +++ stable/12/sys/amd64/amd64/initcpu.c Tue Mar 3 15:02:07 2020 (r358579) @@ -255,7 +255,7 @@ initializecpu(void) wrmsr(MSR_EFER, msr); pg_nx = PG_NX; } - hw_ibrs_recalculate(); + hw_ibrs_recalculate(false); hw_ssb_recalculate(false); amd64_syscall_ret_flush_l1d_recalc(); switch (cpu_vendor_id) { Modified: stable/12/sys/amd64/amd64/support.S ============================================================================== --- stable/12/sys/amd64/amd64/support.S Tue Mar 3 14:58:53 2020 (r358578) +++ stable/12/sys/amd64/amd64/support.S Tue Mar 3 15:02:07 2020 (r358579) @@ -1547,7 +1547,7 @@ handle_ibrs_\l: /* all callers already saved %rax, %rdx, and %rcx */ ENTRY(handle_ibrs_entry) - cmpb $0,hw_ibrs_active(%rip) + cmpb $0,hw_ibrs_ibpb_active(%rip) je 1f movl $MSR_IA32_SPEC_CTRL,%ecx rdmsr Modified: stable/12/sys/dev/cpuctl/cpuctl.c ============================================================================== --- stable/12/sys/dev/cpuctl/cpuctl.c Tue Mar 3 14:58:53 2020 (r358578) +++ stable/12/sys/dev/cpuctl/cpuctl.c Tue Mar 3 15:02:07 2020 (r358579) @@ -538,8 +538,8 @@ cpuctl_do_eval_cpu_features(int cpu, struct thread *td set_cpu(cpu, td); identify_cpu1(); identify_cpu2(); - hw_ibrs_recalculate(); restore_cpu(oldcpu, is_bound, td); + hw_ibrs_recalculate(true); hw_ssb_recalculate(true); #ifdef __amd64__ amd64_syscall_ret_flush_l1d_recalc(); Modified: stable/12/sys/i386/i386/support.s ============================================================================== --- stable/12/sys/i386/i386/support.s Tue Mar 3 14:58:53 2020 (r358578) +++ stable/12/sys/i386/i386/support.s Tue Mar 3 15:02:07 2020 (r358579) @@ -446,7 +446,7 @@ msr_onfault: ret ENTRY(handle_ibrs_entry) - cmpb $0,hw_ibrs_active + cmpb $0,hw_ibrs_ibpb_active je 1f movl $MSR_IA32_SPEC_CTRL,%ecx rdmsr Modified: stable/12/sys/x86/acpica/acpi_wakeup.c ============================================================================== --- stable/12/sys/x86/acpica/acpi_wakeup.c Tue Mar 3 14:58:53 2020 (r358578) +++ stable/12/sys/x86/acpica/acpi_wakeup.c Tue Mar 3 15:02:07 2020 (r358579) @@ -245,7 +245,7 @@ acpi_sleep_machdep(struct acpi_softc *sc, int state) } #endif #ifdef __amd64__ - hw_ibrs_active = 0; + hw_ibrs_ibpb_active = 0; hw_ssb_active = 0; cpu_stdext_feature3 = 0; CPU_FOREACH(i) { Modified: stable/12/sys/x86/include/x86_var.h ============================================================================== --- stable/12/sys/x86/include/x86_var.h Tue Mar 3 14:58:53 2020 (r358578) +++ stable/12/sys/x86/include/x86_var.h Tue Mar 3 15:02:07 2020 (r358579) @@ -84,7 +84,7 @@ extern int use_xsave; extern uint64_t xsave_mask; extern u_int max_apic_id; extern int pti; -extern int hw_ibrs_active; +extern int hw_ibrs_ibpb_active; extern int hw_mds_disable; extern int hw_ssb_active; extern int x86_taa_enable; @@ -141,7 +141,7 @@ int is_physical_memory(vm_paddr_t addr); int isa_nmi(int cd); void handle_ibrs_entry(void); void handle_ibrs_exit(void); -void hw_ibrs_recalculate(void); +void hw_ibrs_recalculate(bool all_cpus); void hw_mds_recalculate(void); void hw_ssb_recalculate(bool all_cpus); void x86_taa_recalculate(void); Modified: stable/12/sys/x86/x86/cpu_machdep.c ============================================================================== --- stable/12/sys/x86/x86/cpu_machdep.c Tue Mar 3 14:58:53 2020 (r358578) +++ stable/12/sys/x86/x86/cpu_machdep.c Tue Mar 3 15:02:07 2020 (r358579) @@ -855,23 +855,27 @@ nmi_handle_intr(u_int type, struct trapframe *frame) nmi_call_kdb(PCPU_GET(cpuid), type, frame); } -int hw_ibrs_active; +static int hw_ibrs_active; +int hw_ibrs_ibpb_active; int hw_ibrs_disable = 1; SYSCTL_INT(_hw, OID_AUTO, ibrs_active, CTLFLAG_RD, &hw_ibrs_active, 0, "Indirect Branch Restricted Speculation active"); void -hw_ibrs_recalculate(void) +hw_ibrs_recalculate(bool for_all_cpus) { if ((cpu_ia32_arch_caps & IA32_ARCH_CAP_IBRS_ALL) != 0) { - x86_msr_op(MSR_IA32_SPEC_CTRL, MSR_OP_LOCAL | - (hw_ibrs_disable ? MSR_OP_ANDNOT : MSR_OP_OR), + x86_msr_op(MSR_IA32_SPEC_CTRL, (for_all_cpus ? + MSR_OP_RENDEZVOUS : MSR_OP_LOCAL) | + (hw_ibrs_disable != 0 ? MSR_OP_ANDNOT : MSR_OP_OR), IA32_SPEC_CTRL_IBRS); - return; + hw_ibrs_active = hw_ibrs_disable == 0; + hw_ibrs_ibpb_active = 0; + } else { + hw_ibrs_active = hw_ibrs_ibpb_active = (cpu_stdext_feature3 & + CPUID_STDEXT3_IBPB) != 0 && !hw_ibrs_disable; } - hw_ibrs_active = (cpu_stdext_feature3 & CPUID_STDEXT3_IBPB) != 0 && - !hw_ibrs_disable; } static int @@ -884,7 +888,7 @@ hw_ibrs_disable_handler(SYSCTL_HANDLER_ARGS) if (error != 0 || req->newptr == NULL) return (error); hw_ibrs_disable = val != 0; - hw_ibrs_recalculate(); + hw_ibrs_recalculate(true); return (0); } SYSCTL_PROC(_hw, OID_AUTO, ibrs_disable, CTLTYPE_INT | CTLFLAG_RWTUN |