From owner-svn-src-all@freebsd.org Tue Aug 14 17:51:14 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 49CFC107FCFC; Tue, 14 Aug 2018 17:51:14 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EED7F7E6A0; Tue, 14 Aug 2018 17:51:13 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D10791ED98; Tue, 14 Aug 2018 17:51:13 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w7EHpDZJ029092; Tue, 14 Aug 2018 17:51:13 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w7EHpDM9029089; Tue, 14 Aug 2018 17:51:13 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201808141751.w7EHpDM9029089@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Tue, 14 Aug 2018 17:51:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337794 - stable/11/sys/amd64/vmm/intel X-SVN-Group: stable-11 X-SVN-Commit-Author: kib X-SVN-Commit-Paths: stable/11/sys/amd64/vmm/intel X-SVN-Commit-Revision: 337794 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Aug 2018 17:51:14 -0000 Author: kib Date: Tue Aug 14 17:51:12 2018 New Revision: 337794 URL: https://svnweb.freebsd.org/changeset/base/337794 Log: MFC r337785: Provide part of the mitigation for L1TF-VMM. Security: CVE-2018-3646 Approved by: so (insta-MFC) Modified: stable/11/sys/amd64/vmm/intel/vmx.c stable/11/sys/amd64/vmm/intel/vmx_genassym.c stable/11/sys/amd64/vmm/intel/vmx_support.S Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/amd64/vmm/intel/vmx.c ============================================================================== --- stable/11/sys/amd64/vmm/intel/vmx.c Tue Aug 14 17:49:52 2018 (r337793) +++ stable/11/sys/amd64/vmm/intel/vmx.c Tue Aug 14 17:51:12 2018 (r337794) @@ -185,6 +185,12 @@ static u_int vpid_alloc_failed; SYSCTL_UINT(_hw_vmm_vmx, OID_AUTO, vpid_alloc_failed, CTLFLAG_RD, &vpid_alloc_failed, 0, NULL); +static int guest_l1d_flush; +SYSCTL_INT(_hw_vmm_vmx, OID_AUTO, l1d_flush, CTLFLAG_RD, + &guest_l1d_flush, 0, NULL); + +uint64_t vmx_msr_flush_cmd; + /* * Use the last page below 4GB as the APIC access address. This address is * occupied by the boot firmware so it is guaranteed that it will not conflict @@ -719,6 +725,12 @@ vmx_init(int ipinum) printf("vmx_init: ept initialization failed (%d)\n", error); return (error); } + + guest_l1d_flush = (cpu_ia32_arch_caps & IA32_ARCH_CAP_RDCL_NO) == 0; + TUNABLE_INT_FETCH("hw.vmm.l1d_flush", &guest_l1d_flush); + if (guest_l1d_flush && + (cpu_stdext_feature3 & CPUID_STDEXT3_L1D_FLUSH) != 0) + vmx_msr_flush_cmd = IA32_FLUSH_CMD_L1D; /* * Stash the cr0 and cr4 bits that must be fixed to 0 or 1 Modified: stable/11/sys/amd64/vmm/intel/vmx_genassym.c ============================================================================== --- stable/11/sys/amd64/vmm/intel/vmx_genassym.c Tue Aug 14 17:49:52 2018 (r337793) +++ stable/11/sys/amd64/vmm/intel/vmx_genassym.c Tue Aug 14 17:51:12 2018 (r337794) @@ -36,6 +36,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include "vmx_cpufunc.h" @@ -86,3 +87,6 @@ ASSYM(PM_EPTGEN, offsetof(struct pmap, pm_eptgen)); ASSYM(KERNEL_SS, GSEL(GDATA_SEL, SEL_KPL)); ASSYM(KERNEL_CS, GSEL(GCODE_SEL, SEL_KPL)); + +ASSYM(PAGE_SIZE, PAGE_SIZE); +ASSYM(KERNBASE, KERNBASE); Modified: stable/11/sys/amd64/vmm/intel/vmx_support.S ============================================================================== --- stable/11/sys/amd64/vmm/intel/vmx_support.S Tue Aug 14 17:49:52 2018 (r337793) +++ stable/11/sys/amd64/vmm/intel/vmx_support.S Tue Aug 14 17:51:12 2018 (r337794) @@ -30,6 +30,7 @@ */ #include +#include #include "vmx_assym.h" @@ -175,9 +176,47 @@ ENTRY(vmx_enter_guest) jbe invept_error /* Check invept instruction error */ guest_restore: - cmpl $0, %edx - je do_launch + /* + * Flush L1D cache if requested. Use IA32_FLUSH_CMD MSR if available, + * otherwise load enough of the data from the zero_region to flush + * existing L1D content. + */ +#define L1D_FLUSH_SIZE (64 * 1024) + movl %edx, %r8d + cmpb $0, guest_l1d_flush(%rip) + je after_l1d + movq vmx_msr_flush_cmd(%rip), %rax + testq %rax, %rax + jz 1f + movq %rax, %rdx + shrq $32, %rdx + movl $MSR_IA32_FLUSH_CMD, %ecx + wrmsr + jmp after_l1d +1: movq $KERNBASE, %r9 + movq $-L1D_FLUSH_SIZE, %rcx + /* + * pass 1: Preload TLB. + * Kernel text is mapped using superpages. TLB preload is + * done for the benefit of older CPUs which split 2M page + * into 4k TLB entries. + */ +2: movb L1D_FLUSH_SIZE(%r9, %rcx), %al + addq $PAGE_SIZE, %rcx + jne 2b + xorl %eax, %eax + cpuid + movq $-L1D_FLUSH_SIZE, %rcx + /* pass 2: Read each cache line */ +3: movb L1D_FLUSH_SIZE(%r9, %rcx), %al + addq $64, %rcx + jne 3b + lfence +#undef L1D_FLUSH_SIZE +after_l1d: + cmpl $0, %r8d + je do_launch VMX_GUEST_RESTORE vmresume /*