From owner-svn-src-all@freebsd.org Sat Nov 16 00:29:30 2019 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 204581B4D4C; Sat, 16 Nov 2019 00:29:30 +0000 (UTC) (envelope-from scottl@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47FGLc4pQWz4Dnl; Sat, 16 Nov 2019 00:29:28 +0000 (UTC) (envelope-from scottl@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0A176210E2; Sat, 16 Nov 2019 00:29:28 +0000 (UTC) (envelope-from scottl@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id xAG0TRJC097354; Sat, 16 Nov 2019 00:29:27 GMT (envelope-from scottl@FreeBSD.org) Received: (from scottl@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id xAG0TQh9097347; Sat, 16 Nov 2019 00:29:26 GMT (envelope-from scottl@FreeBSD.org) Message-Id: <201911160029.xAG0TQh9097347@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: scottl set sender to scottl@FreeBSD.org using -f From: Scott Long Date: Sat, 16 Nov 2019 00:29:26 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r354760 - in stable/12/sys: amd64/amd64 dev/cpuctl kern sys x86/include x86/x86 X-SVN-Group: stable-12 X-SVN-Commit-Author: scottl X-SVN-Commit-Paths: in stable/12/sys: amd64/amd64 dev/cpuctl kern sys x86/include x86/x86 X-SVN-Commit-Revision: 354760 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Nov 2019 00:29:30 -0000 Author: scottl Date: Sat Nov 16 00:29:26 2019 New Revision: 354760 URL: https://svnweb.freebsd.org/changeset/base/354760 Log: MFC r354756: Create a new sysctl tree, machdep.mitigations Sponsored by: Intel Modified: stable/12/sys/amd64/amd64/machdep.c stable/12/sys/dev/cpuctl/cpuctl.c stable/12/sys/kern/kern_mib.c stable/12/sys/sys/sysctl.h stable/12/sys/x86/include/x86_var.h stable/12/sys/x86/x86/cpu_machdep.c Directory Properties: stable/12/ (props changed) Modified: stable/12/sys/amd64/amd64/machdep.c ============================================================================== --- stable/12/sys/amd64/amd64/machdep.c Sat Nov 16 00:26:42 2019 (r354759) +++ stable/12/sys/amd64/amd64/machdep.c Sat Nov 16 00:29:26 2019 (r354760) @@ -1790,6 +1790,7 @@ hammer_time(u_int64_t modulep, u_int64_t physfree) TUNABLE_INT_FETCH("machdep.syscall_ret_l1d_flush", &syscall_ret_l1d_flush_mode); TUNABLE_INT_FETCH("hw.mds_disable", &hw_mds_disable); + TUNABLE_INT_FETCH("hw.tsx_disable", &hw_tsx_disable); finishidentcpu(); /* Final stage of CPU initialization */ initializecpu(); /* Initialize CPU registers */ Modified: stable/12/sys/dev/cpuctl/cpuctl.c ============================================================================== --- stable/12/sys/dev/cpuctl/cpuctl.c Sat Nov 16 00:26:42 2019 (r354759) +++ stable/12/sys/dev/cpuctl/cpuctl.c Sat Nov 16 00:29:26 2019 (r354760) @@ -546,6 +546,7 @@ cpuctl_do_eval_cpu_features(int cpu, struct thread *td pmap_allow_2m_x_ept_recalculate(); #endif hw_mds_recalculate(); + hw_tsx_recalculate(); printcpuinfo(); return (0); } Modified: stable/12/sys/kern/kern_mib.c ============================================================================== --- stable/12/sys/kern/kern_mib.c Sat Nov 16 00:26:42 2019 (r354759) +++ stable/12/sys/kern/kern_mib.c Sat Nov 16 00:29:26 2019 (r354760) @@ -78,6 +78,8 @@ SYSCTL_ROOT_NODE(CTL_HW, hw, CTLFLAG_RW, 0, "hardware"); SYSCTL_ROOT_NODE(CTL_MACHDEP, machdep, CTLFLAG_RW, 0, "machine dependent"); +SYSCTL_NODE(_machdep, OID_AUTO, mitigations, CTLFLAG_RW, 0, + "Machine dependent platform mitigations."); SYSCTL_ROOT_NODE(CTL_USER, user, CTLFLAG_RW, 0, "user-level"); SYSCTL_ROOT_NODE(CTL_P1003_1B, p1003_1b, CTLFLAG_RW, 0, Modified: stable/12/sys/sys/sysctl.h ============================================================================== --- stable/12/sys/sys/sysctl.h Sat Nov 16 00:26:42 2019 (r354759) +++ stable/12/sys/sys/sysctl.h Sat Nov 16 00:29:26 2019 (r354760) @@ -1056,6 +1056,7 @@ SYSCTL_DECL(_hw_bus); SYSCTL_DECL(_hw_bus_devices); SYSCTL_DECL(_hw_bus_info); SYSCTL_DECL(_machdep); +SYSCTL_DECL(_machdep_mitigations); SYSCTL_DECL(_user); SYSCTL_DECL(_compat); SYSCTL_DECL(_regression); Modified: stable/12/sys/x86/include/x86_var.h ============================================================================== --- stable/12/sys/x86/include/x86_var.h Sat Nov 16 00:26:42 2019 (r354759) +++ stable/12/sys/x86/include/x86_var.h Sat Nov 16 00:29:26 2019 (r354760) @@ -87,6 +87,7 @@ extern int pti; extern int hw_ibrs_active; extern int hw_mds_disable; extern int hw_ssb_active; +extern int hw_tsx_disable; struct pcb; struct thread; @@ -143,6 +144,7 @@ void handle_ibrs_exit(void); void hw_ibrs_recalculate(void); void hw_mds_recalculate(void); void hw_ssb_recalculate(bool all_cpus); +void hw_tsx_recalculate(void); void nmi_call_kdb(u_int cpu, u_int type, struct trapframe *frame); void nmi_call_kdb_smp(u_int type, struct trapframe *frame); void nmi_handle_intr(u_int type, struct trapframe *frame); Modified: stable/12/sys/x86/x86/cpu_machdep.c ============================================================================== --- stable/12/sys/x86/x86/cpu_machdep.c Sat Nov 16 00:26:42 2019 (r354759) +++ stable/12/sys/x86/x86/cpu_machdep.c Sat Nov 16 00:29:26 2019 (r354760) @@ -1120,6 +1120,187 @@ SYSCTL_PROC(_hw, OID_AUTO, mds_disable, CTLTYPE_INT | "Microarchitectural Data Sampling Mitigation " "(0 - off, 1 - on VERW, 2 - on SW, 3 - on AUTO"); +int hw_tsx_disable; +int hw_tsx_state; +enum { + TSX_TAA_NONE = 0, + TSX_TAA_DISABLE = 1, + TSX_TAA_VERW = 2, + TSX_TAA_AUTO = 3 +}; + +static void +hw_tsx_set_one(bool enable) +{ + uint64_t v; + + v = rdmsr(MSR_IA32_TSX_CTRL); + if (enable) + v |= (uint64_t)(IA32_TSX_CTRL_RTM_DISABLE | + IA32_TSX_CTRL_TSX_CPUID_CLEAR); + else + v &= ~(uint64_t)(IA32_TSX_CTRL_RTM_DISABLE | + IA32_TSX_CTRL_TSX_CPUID_CLEAR); + + wrmsr(MSR_IA32_TSX_CTRL, v); +} + +static void +hw_tsx_set(bool enable, bool all) +{ + struct thread *td; + int bound_cpu, i, is_bound; + + if (all) { + td = curthread; + thread_lock(td); + is_bound = sched_is_bound(td); + bound_cpu = td->td_oncpu; + CPU_FOREACH(i) { + sched_bind(td, i); + hw_tsx_set_one(enable); + } + if (is_bound) + sched_bind(td, bound_cpu); + else + sched_unbind(td); + thread_unlock(td); + } else + hw_tsx_set_one(enable); + +} + +void +hw_tsx_recalculate(void) +{ + static int tsx_saved_mds_disable = 0; + int tsx_need = 0, tsx_state = 0; + int mds_disable = 0, need_mds_recalc = 0; + + /* Check CPUID.07h.EBX.HLE and RTM for the presence of TSX */ + if ((cpu_stdext_feature & CPUID_STDEXT_HLE) == 0 || + (cpu_stdext_feature & CPUID_STDEXT_RTM) == 0) { + /* TSX is not present */ + hw_tsx_state = 0; + return; + } + + /* Check to see what mitigation options the CPU gives us */ + if (cpu_ia32_arch_caps & IA32_ARCH_CAP_TAA_NO) + tsx_need = TSX_TAA_NONE; + else if (cpu_ia32_arch_caps & IA32_ARCH_CAP_TSX_CTRL) + tsx_need = TSX_TAA_DISABLE; + else { + /* No TSX specific remedies are available. */ + if (hw_tsx_disable == TSX_TAA_DISABLE) { + /* The user asked for the disable option, but + * it's not available. */ + if (bootverbose) + printf("TSX control not available\n"); + return; + } else + tsx_need = TSX_TAA_VERW; + } + + /* Can we automatically take action, or are we being forced? */ + if (hw_tsx_disable == TSX_TAA_AUTO) + tsx_state = tsx_need; + else + tsx_state = hw_tsx_disable; + + /* No state change, nothing to do */ + if (tsx_state == hw_tsx_state) { + if (bootverbose) + printf("No TSX change made\n"); + return; + } + + /* Does the MSR need to be turned on or off? */ + if (tsx_state == TSX_TAA_DISABLE) + hw_tsx_set(1 /* enable */, 1 /* all */); + else if (hw_tsx_state == TSX_TAA_DISABLE) + hw_tsx_set(0 /* disable */, 1 /* all */); + + /* Does MDS need to be set to turn on VERW? */ + if (tsx_state == TSX_TAA_VERW) { + tsx_saved_mds_disable = hw_mds_disable; + mds_disable = hw_mds_disable = 1; + need_mds_recalc = 1; + } else if (hw_tsx_state == TSX_TAA_VERW) { + mds_disable = hw_mds_disable = tsx_saved_mds_disable; + need_mds_recalc = 1; + } + if (need_mds_recalc) { + hw_mds_recalculate(); + if (mds_disable != hw_mds_disable) { + if (bootverbose) + printf("Cannot change MDS state for TSX\n"); + /* Don't update our state */ + return; + } + } + + hw_tsx_state = tsx_state; + return; +} + +static void +hw_tsx_recalculate_boot(void * arg __unused) +{ + + hw_tsx_recalculate(); +} +SYSINIT(tsx_recalc, SI_SUB_SMP, SI_ORDER_ANY, hw_tsx_recalculate_boot, NULL); + +static int +sysctl_tsx_disable_handler(SYSCTL_HANDLER_ARGS) +{ + int error, val; + + val = hw_tsx_disable; + error = sysctl_handle_int(oidp, &val, 0, req); + if (error != 0 || req->newptr == NULL) + return (error); + if (val < 0 || val > 3) + return (EINVAL); + hw_tsx_disable = val; + hw_tsx_recalculate(); + return (0); +} + +SYSCTL_PROC(_hw, OID_AUTO, tsx_disable, CTLTYPE_INT | + CTLFLAG_RWTUN | CTLFLAG_NOFETCH | CTLFLAG_MPSAFE, NULL, 0, + sysctl_tsx_disable_handler, "I", + "TSX Asynchronous Abort Mitigation " + "(0 - off, 1 - disable TSX, 2 - MDS/VERW, 3 - on AUTO"); + +static int +sysctl_hw_tsx_disable_state_handler(SYSCTL_HANDLER_ARGS) +{ + const char *state; + + switch (hw_tsx_state) { + case TSX_TAA_NONE: + state = "inactive"; + break; + case TSX_TAA_DISABLE: + state = "TSX disabled"; + break; + case TSX_TAA_VERW: + state = "MDS/VERW"; + break; + default: + state = "unknown"; + } + + return (SYSCTL_OUT(req, state, strlen(state))); +} + +SYSCTL_PROC(_hw, OID_AUTO, tsx_disable_state, + CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, NULL, 0, + sysctl_hw_tsx_disable_state_handler, "A", + "Transactional Memory Asynchronous Abort Mitigation state"); + /* * Enable and restore kernel text write permissions. * Callers must ensure that disable_wp()/restore_wp() are executed