From owner-svn-src-all@freebsd.org Sun Apr 19 17:19:30 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0A11C27BB34; Sun, 19 Apr 2020 17:19:30 +0000 (UTC) (envelope-from mr@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 494xQT6XFDz4ctJ; Sun, 19 Apr 2020 17:19:29 +0000 (UTC) (envelope-from mr@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DB4CC1442; Sun, 19 Apr 2020 17:19:29 +0000 (UTC) (envelope-from mr@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 03JHJT3E013195; Sun, 19 Apr 2020 17:19:29 GMT (envelope-from mr@FreeBSD.org) Received: (from mr@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 03JHJT86013192; Sun, 19 Apr 2020 17:19:29 GMT (envelope-from mr@FreeBSD.org) Message-Id: <202004191719.03JHJT86013192@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mr set sender to mr@FreeBSD.org using -f From: Michael Reifenberger Date: Sun, 19 Apr 2020 17:19:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r360103 - in stable/12/sys/amd64/vmm: intel io X-SVN-Group: stable-12 X-SVN-Commit-Author: mr X-SVN-Commit-Paths: in stable/12/sys/amd64/vmm: intel io X-SVN-Commit-Revision: 360103 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Apr 2020 17:19:30 -0000 Author: mr Date: Sun Apr 19 17:19:29 2020 New Revision: 360103 URL: https://svnweb.freebsd.org/changeset/base/360103 Log: MFC r358848: Untangle TPR shadowing and APIC virtualization. This speeds up Windows guests tremendously. The patch does: Add a new tuneable 'hw.vmm.vmx.use_tpr_shadowing' to disable TLP shadowing. Also add 'hw.vmm.vmx.cap.tpr_shadowing' to be able to query if TPR shadowing is used. Detach the initialization of TPR shadowing from the initialization of APIC virtualization. APIC virtualization still needs TPR shadowing, but not vice versa. Any CPU that supports APIC virtualization should also support TPR shadowing. When TPR shadowing is used, the APIC page of each vCPU is written to the VMCS_VIRTUAL_APIC field of the VMCS so that the CPU can write directly to the page without intercept. On vm exit, vlapic_update_ppr() is called to update the PPR. Geänderte Pfade: M /head/sys/amd64/vmm/intel/vmx.c M /head/sys/amd64/vmm/io/vlapic.c M /head/sys/amd64/vmm/io/vlapic.h Submitted by: Yamagi Burmeister Reviewed by: grehan Differential Revision: https://reviews.freebsd.org/D22942 Modified: stable/12/sys/amd64/vmm/intel/vmx.c stable/12/sys/amd64/vmm/io/vlapic.c stable/12/sys/amd64/vmm/io/vlapic.h Directory Properties: stable/12/ (props changed) Modified: stable/12/sys/amd64/vmm/intel/vmx.c ============================================================================== --- stable/12/sys/amd64/vmm/intel/vmx.c Sun Apr 19 17:01:21 2020 (r360102) +++ stable/12/sys/amd64/vmm/intel/vmx.c Sun Apr 19 17:19:29 2020 (r360103) @@ -172,6 +172,10 @@ static int cap_invpcid; SYSCTL_INT(_hw_vmm_vmx_cap, OID_AUTO, invpcid, CTLFLAG_RD, &cap_invpcid, 0, "Guests are allowed to use INVPCID"); +static int tpr_shadowing; +SYSCTL_INT(_hw_vmm_vmx_cap, OID_AUTO, tpr_shadowing, CTLFLAG_RD, + &tpr_shadowing, 0, "TPR shadowing support"); + static int virtual_interrupt_delivery; SYSCTL_INT(_hw_vmm_vmx_cap, OID_AUTO, virtual_interrupt_delivery, CTLFLAG_RD, &virtual_interrupt_delivery, 0, "APICv virtual interrupt delivery support"); @@ -627,7 +631,7 @@ vmx_restore(void) static int vmx_init(int ipinum) { - int error, use_tpr_shadow; + int error; uint64_t basic, fixed0, fixed1, feature_control; uint32_t tmp, procbased2_vid_bits; @@ -751,6 +755,24 @@ vmx_init(int ipinum) &tmp) == 0); /* + * Check support for TPR shadow. + */ + error = vmx_set_ctlreg(MSR_VMX_PROCBASED_CTLS, + MSR_VMX_TRUE_PROCBASED_CTLS, PROCBASED_USE_TPR_SHADOW, 0, + &tmp); + if (error == 0) { + tpr_shadowing = 1; + TUNABLE_INT_FETCH("hw.vmm.vmx.use_tpr_shadowing", + &tpr_shadowing); + } + + if (tpr_shadowing) { + procbased_ctls |= PROCBASED_USE_TPR_SHADOW; + procbased_ctls &= ~PROCBASED_CR8_LOAD_EXITING; + procbased_ctls &= ~PROCBASED_CR8_STORE_EXITING; + } + + /* * Check support for virtual interrupt delivery. */ procbased2_vid_bits = (PROCBASED2_VIRTUALIZE_APIC_ACCESSES | @@ -758,13 +780,9 @@ vmx_init(int ipinum) PROCBASED2_APIC_REGISTER_VIRTUALIZATION | PROCBASED2_VIRTUAL_INTERRUPT_DELIVERY); - use_tpr_shadow = (vmx_set_ctlreg(MSR_VMX_PROCBASED_CTLS, - MSR_VMX_TRUE_PROCBASED_CTLS, PROCBASED_USE_TPR_SHADOW, 0, - &tmp) == 0); - error = vmx_set_ctlreg(MSR_VMX_PROCBASED_CTLS2, MSR_VMX_PROCBASED_CTLS2, procbased2_vid_bits, 0, &tmp); - if (error == 0 && use_tpr_shadow) { + if (error == 0 && tpr_shadowing) { virtual_interrupt_delivery = 1; TUNABLE_INT_FETCH("hw.vmm.vmx.use_apic_vid", &virtual_interrupt_delivery); @@ -776,13 +794,6 @@ vmx_init(int ipinum) procbased_ctls2 &= ~PROCBASED2_VIRTUALIZE_X2APIC_MODE; /* - * No need to emulate accesses to %CR8 if virtual - * interrupt delivery is enabled. - */ - procbased_ctls &= ~PROCBASED_CR8_LOAD_EXITING; - procbased_ctls &= ~PROCBASED_CR8_STORE_EXITING; - - /* * Check for Posted Interrupts only if Virtual Interrupt * Delivery is enabled. */ @@ -1051,10 +1062,13 @@ vmx_vminit(struct vm *vm, pmap_t pmap) vmx->ctx[i].guest_dr6 = DBREG_DR6_RESERVED1; error += vmwrite(VMCS_GUEST_DR7, DBREG_DR7_RESERVED1); - if (virtual_interrupt_delivery) { - error += vmwrite(VMCS_APIC_ACCESS, APIC_ACCESS_ADDRESS); + if (tpr_shadowing) { error += vmwrite(VMCS_VIRTUAL_APIC, vtophys(&vmx->apic_page[i])); + } + + if (virtual_interrupt_delivery) { + error += vmwrite(VMCS_APIC_ACCESS, APIC_ACCESS_ADDRESS); error += vmwrite(VMCS_EOI_EXIT0, 0); error += vmwrite(VMCS_EOI_EXIT1, 0); error += vmwrite(VMCS_EOI_EXIT2, 0); @@ -2645,6 +2659,12 @@ vmx_exit_process(struct vmx *vmx, int vcpu, struct vm_ SDT_PROBE3(vmm, vmx, exit, mwait, vmx, vcpu, vmexit); vmexit->exitcode = VM_EXITCODE_MWAIT; break; + case EXIT_REASON_TPR: + vlapic = vm_lapic(vmx->vm, vcpu); + vlapic_sync_tpr(vlapic); + vmexit->inst_length = 0; + handled = HANDLED; + break; case EXIT_REASON_VMCALL: case EXIT_REASON_VMCLEAR: case EXIT_REASON_VMLAUNCH: @@ -2931,6 +2951,16 @@ vmx_run(void *arg, int vcpu, register_t rip, pmap_t pm } /* + * If TPR Shadowing is enabled, the TPR Threshold + * must be updated right before entering the guest. + */ + if (tpr_shadowing && !virtual_interrupt_delivery) { + if ((vmx->cap[vcpu].proc_ctls & PROCBASED_USE_TPR_SHADOW) != 0) { + vmcs_write(VMCS_TPR_THRESHOLD, vlapic_get_cr8(vlapic)); + } + } + + /* * VM exits restore the base address but not the * limits of GDTR and IDTR. The VMCS only stores the * base address, so VM exits set the limits to 0xffff. @@ -3604,10 +3634,33 @@ vmx_set_tmr(struct vlapic *vlapic, int vector, bool le } static void -vmx_enable_x2apic_mode(struct vlapic *vlapic) +vmx_enable_x2apic_mode_ts(struct vlapic *vlapic) { struct vmx *vmx; struct vmcs *vmcs; + uint32_t proc_ctls; + int vcpuid; + + vcpuid = vlapic->vcpuid; + vmx = ((struct vlapic_vtx *)vlapic)->vmx; + vmcs = &vmx->vmcs[vcpuid]; + + proc_ctls = vmx->cap[vcpuid].proc_ctls; + proc_ctls &= ~PROCBASED_USE_TPR_SHADOW; + proc_ctls |= PROCBASED_CR8_LOAD_EXITING; + proc_ctls |= PROCBASED_CR8_STORE_EXITING; + vmx->cap[vcpuid].proc_ctls = proc_ctls; + + VMPTRLD(vmcs); + vmcs_write(VMCS_PRI_PROC_BASED_CTLS, proc_ctls); + VMCLEAR(vmcs); +} + +static void +vmx_enable_x2apic_mode_vid(struct vlapic *vlapic) +{ + struct vmx *vmx; + struct vmcs *vmcs; uint32_t proc_ctls2; int vcpuid, error; @@ -3765,12 +3818,16 @@ vmx_vlapic_init(void *arg, int vcpuid) vlapic_vtx->pir_desc = &vmx->pir_desc[vcpuid]; vlapic_vtx->vmx = vmx; + if (tpr_shadowing) { + vlapic->ops.enable_x2apic_mode = vmx_enable_x2apic_mode_ts; + } + if (virtual_interrupt_delivery) { vlapic->ops.set_intr_ready = vmx_set_intr_ready; vlapic->ops.pending_intr = vmx_pending_intr; vlapic->ops.intr_accepted = vmx_intr_accepted; vlapic->ops.set_tmr = vmx_set_tmr; - vlapic->ops.enable_x2apic_mode = vmx_enable_x2apic_mode; + vlapic->ops.enable_x2apic_mode = vmx_enable_x2apic_mode_vid; } if (posted_interrupts) Modified: stable/12/sys/amd64/vmm/io/vlapic.c ============================================================================== --- stable/12/sys/amd64/vmm/io/vlapic.c Sun Apr 19 17:01:21 2020 (r360102) +++ stable/12/sys/amd64/vmm/io/vlapic.c Sun Apr 19 17:19:29 2020 (r360103) @@ -549,6 +549,12 @@ vlapic_update_ppr(struct vlapic *vlapic) VLAPIC_CTR1(vlapic, "vlapic_update_ppr 0x%02x", ppr); } +void +vlapic_sync_tpr(struct vlapic *vlapic) +{ + vlapic_update_ppr(vlapic); +} + static VMM_STAT(VLAPIC_GRATUITOUS_EOI, "EOI without any in-service interrupt"); static void @@ -1094,6 +1100,8 @@ vlapic_pending_intr(struct vlapic *vlapic, int *vecptr int idx, i, bitpos, vector; uint32_t *irrptr, val; + vlapic_update_ppr(vlapic); + if (vlapic->ops.pending_intr) return ((*vlapic->ops.pending_intr)(vlapic, vecptr)); @@ -1151,7 +1159,6 @@ vlapic_intr_accepted(struct vlapic *vlapic, int vector panic("isrvec_stk_top overflow %d", stk_top); vlapic->isrvec_stk[stk_top] = vector; - vlapic_update_ppr(vlapic); } void Modified: stable/12/sys/amd64/vmm/io/vlapic.h ============================================================================== --- stable/12/sys/amd64/vmm/io/vlapic.h Sun Apr 19 17:01:21 2020 (r360102) +++ stable/12/sys/amd64/vmm/io/vlapic.h Sun Apr 19 17:19:29 2020 (r360103) @@ -75,6 +75,8 @@ void vlapic_set_error(struct vlapic *vlapic, uint32_t void vlapic_fire_cmci(struct vlapic *vlapic); int vlapic_trigger_lvt(struct vlapic *vlapic, int vector); +void vlapic_sync_tpr(struct vlapic *vlapic); + uint64_t vlapic_get_apicbase(struct vlapic *vlapic); int vlapic_set_apicbase(struct vlapic *vlapic, uint64_t val); void vlapic_set_x2apic_state(struct vm *vm, int vcpuid, enum x2apic_state s);