From owner-svn-src-head@freebsd.org Tue Aug 14 17:29:43 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EA9BD107F3D9; Tue, 14 Aug 2018 17:29:42 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6B29D7CE3E; Tue, 14 Aug 2018 17:29:42 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4C4F11E9C9; Tue, 14 Aug 2018 17:29:42 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w7EHTgRw017110; Tue, 14 Aug 2018 17:29:42 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w7EHTfrd017107; Tue, 14 Aug 2018 17:29:41 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201808141729.w7EHTfrd017107@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Tue, 14 Aug 2018 17:29:41 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337785 - head/sys/amd64/vmm/intel X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: head/sys/amd64/vmm/intel X-SVN-Commit-Revision: 337785 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Aug 2018 17:29:43 -0000 Author: kib Date: Tue Aug 14 17:29:41 2018 New Revision: 337785 URL: https://svnweb.freebsd.org/changeset/base/337785 Log: Provide part of the mitigation for L1TF-VMM. On the guest entry in bhyve, flush L1 data cache, using either L1D flush command MSR if available, or by reading enough uninteresting data to fill whole cache. Flush is automatically enabled on CPUs which do not report RDCL_NO, and can be disabled with the hw.vmm.l1d_flush tunable/kenv. Security: CVE-2018-3646 Reviewed by: emaste. jhb, Tony Luck Sponsored by: The FreeBSD Foundation Modified: head/sys/amd64/vmm/intel/vmx.c head/sys/amd64/vmm/intel/vmx_genassym.c head/sys/amd64/vmm/intel/vmx_support.S Modified: head/sys/amd64/vmm/intel/vmx.c ============================================================================== --- head/sys/amd64/vmm/intel/vmx.c Tue Aug 14 17:29:22 2018 (r337784) +++ head/sys/amd64/vmm/intel/vmx.c Tue Aug 14 17:29:41 2018 (r337785) @@ -188,6 +188,12 @@ static u_int vpid_alloc_failed; SYSCTL_UINT(_hw_vmm_vmx, OID_AUTO, vpid_alloc_failed, CTLFLAG_RD, &vpid_alloc_failed, 0, NULL); +static int guest_l1d_flush; +SYSCTL_INT(_hw_vmm_vmx, OID_AUTO, l1d_flush, CTLFLAG_RD, + &guest_l1d_flush, 0, NULL); + +uint64_t vmx_msr_flush_cmd; + /* * The definitions of SDT probes for VMX. */ @@ -798,6 +804,12 @@ vmx_init(int ipinum) printf("vmx_init: ept initialization failed (%d)\n", error); return (error); } + + guest_l1d_flush = (cpu_ia32_arch_caps & IA32_ARCH_CAP_RDCL_NO) == 0; + TUNABLE_INT_FETCH("hw.vmm.l1d_flush", &guest_l1d_flush); + if (guest_l1d_flush && + (cpu_stdext_feature3 & CPUID_STDEXT3_L1D_FLUSH) != 0) + vmx_msr_flush_cmd = IA32_FLUSH_CMD_L1D; /* * Stash the cr0 and cr4 bits that must be fixed to 0 or 1 Modified: head/sys/amd64/vmm/intel/vmx_genassym.c ============================================================================== --- head/sys/amd64/vmm/intel/vmx_genassym.c Tue Aug 14 17:29:22 2018 (r337784) +++ head/sys/amd64/vmm/intel/vmx_genassym.c Tue Aug 14 17:29:41 2018 (r337785) @@ -38,6 +38,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include "vmx_cpufunc.h" @@ -88,3 +89,6 @@ ASSYM(PM_EPTGEN, offsetof(struct pmap, pm_eptgen)); ASSYM(KERNEL_SS, GSEL(GDATA_SEL, SEL_KPL)); ASSYM(KERNEL_CS, GSEL(GCODE_SEL, SEL_KPL)); + +ASSYM(PAGE_SIZE, PAGE_SIZE); +ASSYM(KERNBASE, KERNBASE); Modified: head/sys/amd64/vmm/intel/vmx_support.S ============================================================================== --- head/sys/amd64/vmm/intel/vmx_support.S Tue Aug 14 17:29:22 2018 (r337784) +++ head/sys/amd64/vmm/intel/vmx_support.S Tue Aug 14 17:29:41 2018 (r337785) @@ -30,6 +30,7 @@ */ #include +#include #include "vmx_assym.h" @@ -175,9 +176,47 @@ ENTRY(vmx_enter_guest) jbe invept_error /* Check invept instruction error */ guest_restore: - cmpl $0, %edx - je do_launch + /* + * Flush L1D cache if requested. Use IA32_FLUSH_CMD MSR if available, + * otherwise load enough of the data from the zero_region to flush + * existing L1D content. + */ +#define L1D_FLUSH_SIZE (64 * 1024) + movl %edx, %r8d + cmpb $0, guest_l1d_flush(%rip) + je after_l1d + movq vmx_msr_flush_cmd(%rip), %rax + testq %rax, %rax + jz 1f + movq %rax, %rdx + shrq $32, %rdx + movl $MSR_IA32_FLUSH_CMD, %ecx + wrmsr + jmp after_l1d +1: movq $KERNBASE, %r9 + movq $-L1D_FLUSH_SIZE, %rcx + /* + * pass 1: Preload TLB. + * Kernel text is mapped using superpages. TLB preload is + * done for the benefit of older CPUs which split 2M page + * into 4k TLB entries. + */ +2: movb L1D_FLUSH_SIZE(%r9, %rcx), %al + addq $PAGE_SIZE, %rcx + jne 2b + xorl %eax, %eax + cpuid + movq $-L1D_FLUSH_SIZE, %rcx + /* pass 2: Read each cache line */ +3: movb L1D_FLUSH_SIZE(%r9, %rcx), %al + addq $64, %rcx + jne 3b + lfence +#undef L1D_FLUSH_SIZE +after_l1d: + cmpl $0, %r8d + je do_launch VMX_GUEST_RESTORE vmresume /*