From nobody Fri Nov 18 18:26:26 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NDQGW2xq4z4d7mh; Fri, 18 Nov 2022 18:26:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NDQGW15mYz4HMD; Fri, 18 Nov 2022 18:26:27 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1668795987; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=oKKn+eRCDyzaH2bLHaDbu9xGt07WnHCeY7SVOG/LCYk=; b=kUviG6g/ZJiyLpNTr47Fpt+GXRb3k6HRCoy+g5NE9Jpnx+sxon1PXtqxbKcLMNDuAgmqPa msnXqGDF/krL7qE3gNluYadn6pfx6UyBYKHzkZTbhv1U5wdSIi1GxrM/wAaTReeFtlZnFL l2Z73CJByyZl0u/V8qtlxKH/l2QxdS1d30D1VQR3TmdGvzvTlzH1lwMyEtd/oXGfqeFGwL QGFP0XTXU9yT0u/UcO6/rbkcM6uzMJ/OAheF1UxA0KrqZOEEw2SZGTqIthQAikYl27r+pp qD+mbUUXVYoBoICWuDe07REOPuzUEQ8rMv7AyA1tDBsOipiO6kQqyLE3DYK1fg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1668795987; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=oKKn+eRCDyzaH2bLHaDbu9xGt07WnHCeY7SVOG/LCYk=; b=WCtbkFrkImxSfuxrM7zLv1k43lijPTnh/ZdtoP0tyDU00VvFQhHsIBVDzGysIh0K8/Yrpb ZcqSndJPatuCm3tlfWAwxO5qCNOy3U/+Cosh5iPpXdjSacQesrfSDRLkUra3yfqzsxyIwB gJmtjm1puhwiHdqDCfCWEP1RkZVsjivQvXSv9xXN4utjdfa1WcLSQyeOwuuWP+r3rdKtBY XvENZTlHxAnd1pK47i5rAKtiZ7HKkPSz1PRv67YB4XpWINy3UnYaqv/hOBPmhgbDopGqCb pDbXSZ4F2kCQz0OKVfVIwayT0L17bI1DlRtW+Iax7jGZJNmzcQhzD8ensTaM4g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1668795987; a=rsa-sha256; cv=none; b=w1XqQNRR4QwuPaCuaEEPKM6ZkTPvCs9gMws+K6IiIeOAy6Y/QRVXbl5EKsv+X61jtLJCtb 5hyZ5Ver5brpcOq/NwsCDRSAsegflm0CdMkPbvQM/3vR70i4txhhm0jPJn3ChoCG/82UZU d6sTShFHJzoS4dYP72J4/89e0CMo+mYegwBNu6DEa+MGlTA4iKafxhJEh6jPjsot/5cJVU x43cmZAFTRIZ/iv7ilXMjymOO8fPs+PotFZa4LPKKbk5njxo00cr7JagT1ZxlKsVLhI0Bd Udi9ajK5nMoYPAOnyUHUtjj5TPKS3YPcIy3nnW8LnvAUdKNMP/8pHyQxpDLLSw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4NDQGV70h0zGPd; Fri, 18 Nov 2022 18:26:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 2AIIQQMW029987; Fri, 18 Nov 2022 18:26:26 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 2AIIQQLZ029986; Fri, 18 Nov 2022 18:26:26 GMT (envelope-from git) Date: Fri, 18 Nov 2022 18:26:26 GMT Message-Id: <202211181826.2AIIQQLZ029986@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: 39ec056e6dbd - main - vmm: Rework snapshotting of CPU-specific per-vCPU data. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 39ec056e6dbd89e26ee21d2928dbd37335de0ebc Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=39ec056e6dbd89e26ee21d2928dbd37335de0ebc commit 39ec056e6dbd89e26ee21d2928dbd37335de0ebc Author: John Baldwin AuthorDate: 2022-11-18 17:58:41 +0000 Commit: John Baldwin CommitDate: 2022-11-18 18:25:35 +0000 vmm: Rework snapshotting of CPU-specific per-vCPU data. Previously some per-vCPU state was saved in vmmops_snapshot and other state was saved in vmmops_vcmx_snapshot. Consolidate all per-vCPU state into the latter routine and rename the hook to the more generic 'vcpu_snapshot'. Note that the CPU-independent per-vCPU data is still stored in a separate blob as well as the per-vCPU local APIC data. Reviewed by: corvink, markj Differential Revision: https://reviews.freebsd.org/D37150 --- sys/amd64/include/vmm.h | 4 +- sys/amd64/vmm/amd/svm.c | 239 ++++++++++++++++++++++------------------------ sys/amd64/vmm/intel/vmx.c | 87 ++++++++--------- sys/amd64/vmm/vmm.c | 4 +- 4 files changed, 157 insertions(+), 177 deletions(-) diff --git a/sys/amd64/include/vmm.h b/sys/amd64/include/vmm.h index 37a74f053fb3..f0707a10b804 100644 --- a/sys/amd64/include/vmm.h +++ b/sys/amd64/include/vmm.h @@ -185,7 +185,7 @@ typedef void (*vmi_vmspace_free)(struct vmspace *vmspace); typedef struct vlapic * (*vmi_vlapic_init)(void *vmi, int vcpu); typedef void (*vmi_vlapic_cleanup)(void *vmi, struct vlapic *vlapic); typedef int (*vmi_snapshot_t)(void *vmi, struct vm_snapshot_meta *meta); -typedef int (*vmi_snapshot_vmcx_t)(void *vmi, struct vm_snapshot_meta *meta, +typedef int (*vmi_snapshot_vcpu_t)(void *vmi, struct vm_snapshot_meta *meta, int vcpu); typedef int (*vmi_restore_tsc_t)(void *vmi, int vcpuid, uint64_t now); @@ -210,7 +210,7 @@ struct vmm_ops { /* checkpoint operations */ vmi_snapshot_t snapshot; - vmi_snapshot_vmcx_t vmcx_snapshot; + vmi_snapshot_vcpu_t vcpu_snapshot; vmi_restore_tsc_t restore_tsc; }; diff --git a/sys/amd64/vmm/amd/svm.c b/sys/amd64/vmm/amd/svm.c index d6502e331278..fca3722ed7f4 100644 --- a/sys/amd64/vmm/amd/svm.c +++ b/sys/amd64/vmm/amd/svm.c @@ -2428,224 +2428,213 @@ svm_vlapic_cleanup(void *arg, struct vlapic *vlapic) static int svm_snapshot(void *arg, struct vm_snapshot_meta *meta) { - /* struct svm_softc is AMD's representation for SVM softc */ - struct svm_softc *sc; - struct svm_vcpu *vcpu; - int ret; - uint16_t i, maxcpus; - - sc = arg; - - KASSERT(sc != NULL, ("%s: arg was NULL", __func__)); - - maxcpus = vm_get_maxcpus(sc->vm); - for (i = 0; i < maxcpus; i++) { - vcpu = &sc->vcpu[i]; - - /* Snapshot swctx for virtual cpu i */ - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_rbp, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_rbx, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_rcx, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_rdx, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_rdi, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_rsi, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r8, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r9, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r10, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r11, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r12, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r13, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r14, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r15, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_dr0, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_dr1, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_dr2, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_dr3, meta, ret, done); - - /* Restore other svm_vcpu struct fields */ - - /* Restore NEXTRIP field */ - SNAPSHOT_VAR_OR_LEAVE(vcpu->nextrip, meta, ret, done); - - /* Restore lastcpu field */ - SNAPSHOT_VAR_OR_LEAVE(vcpu->lastcpu, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->dirty, meta, ret, done); - - /* Restore EPTGEN field - EPT is Extended Page Table */ - SNAPSHOT_VAR_OR_LEAVE(vcpu->eptgen, meta, ret, done); - - SNAPSHOT_VAR_OR_LEAVE(vcpu->asid.gen, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vcpu->asid.num, meta, ret, done); - - /* Set all caches dirty */ - if (meta->op == VM_SNAPSHOT_RESTORE) - svm_set_dirty(sc, i, 0xffffffff); - } - if (meta->op == VM_SNAPSHOT_RESTORE) flush_by_asid(); -done: - return (ret); + return (0); } static int -svm_vmcx_snapshot(void *arg, struct vm_snapshot_meta *meta, int vcpu) +svm_vcpu_snapshot(void *arg, struct vm_snapshot_meta *meta, int vcpuid) { struct svm_softc *sc; + struct svm_vcpu *vcpu; int err, running, hostcpu; sc = (struct svm_softc *)arg; + vcpu = &sc->vcpu[vcpuid]; err = 0; KASSERT(arg != NULL, ("%s: arg was NULL", __func__)); - running = vcpu_is_running(sc->vm, vcpu, &hostcpu); - if (running && hostcpu !=curcpu) { - printf("%s: %s%d is running", __func__, vm_name(sc->vm), vcpu); + running = vcpu_is_running(sc->vm, vcpuid, &hostcpu); + if (running && hostcpu != curcpu) { + printf("%s: %s%d is running", __func__, vm_name(sc->vm), + vcpuid); return (EINVAL); } - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_CR0, meta); - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_CR2, meta); - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_CR3, meta); - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_CR4, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_CR0, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_CR2, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_CR3, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_CR4, meta); - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_DR6, meta); - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_DR7, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_DR6, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_DR7, meta); - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_RAX, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_RAX, meta); - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_RSP, meta); - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_RIP, meta); - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_RFLAGS, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_RSP, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_RIP, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_RFLAGS, meta); /* Guest segments */ /* ES */ - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_ES, meta); - err += vmcb_snapshot_desc(sc, vcpu, VM_REG_GUEST_ES, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_ES, meta); + err += vmcb_snapshot_desc(sc, vcpuid, VM_REG_GUEST_ES, meta); /* CS */ - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_CS, meta); - err += vmcb_snapshot_desc(sc, vcpu, VM_REG_GUEST_CS, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_CS, meta); + err += vmcb_snapshot_desc(sc, vcpuid, VM_REG_GUEST_CS, meta); /* SS */ - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_SS, meta); - err += vmcb_snapshot_desc(sc, vcpu, VM_REG_GUEST_SS, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_SS, meta); + err += vmcb_snapshot_desc(sc, vcpuid, VM_REG_GUEST_SS, meta); /* DS */ - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_DS, meta); - err += vmcb_snapshot_desc(sc, vcpu, VM_REG_GUEST_DS, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_DS, meta); + err += vmcb_snapshot_desc(sc, vcpuid, VM_REG_GUEST_DS, meta); /* FS */ - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_FS, meta); - err += vmcb_snapshot_desc(sc, vcpu, VM_REG_GUEST_FS, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_FS, meta); + err += vmcb_snapshot_desc(sc, vcpuid, VM_REG_GUEST_FS, meta); /* GS */ - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_GS, meta); - err += vmcb_snapshot_desc(sc, vcpu, VM_REG_GUEST_GS, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_GS, meta); + err += vmcb_snapshot_desc(sc, vcpuid, VM_REG_GUEST_GS, meta); /* TR */ - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_TR, meta); - err += vmcb_snapshot_desc(sc, vcpu, VM_REG_GUEST_TR, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_TR, meta); + err += vmcb_snapshot_desc(sc, vcpuid, VM_REG_GUEST_TR, meta); /* LDTR */ - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_LDTR, meta); - err += vmcb_snapshot_desc(sc, vcpu, VM_REG_GUEST_LDTR, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_LDTR, meta); + err += vmcb_snapshot_desc(sc, vcpuid, VM_REG_GUEST_LDTR, meta); /* EFER */ - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_EFER, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_EFER, meta); /* IDTR and GDTR */ - err += vmcb_snapshot_desc(sc, vcpu, VM_REG_GUEST_IDTR, meta); - err += vmcb_snapshot_desc(sc, vcpu, VM_REG_GUEST_GDTR, meta); + err += vmcb_snapshot_desc(sc, vcpuid, VM_REG_GUEST_IDTR, meta); + err += vmcb_snapshot_desc(sc, vcpuid, VM_REG_GUEST_GDTR, meta); /* Specific AMD registers */ - err += svm_snapshot_reg(sc, vcpu, VM_REG_GUEST_INTR_SHADOW, meta); + err += svm_snapshot_reg(sc, vcpuid, VM_REG_GUEST_INTR_SHADOW, meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_CR_INTERCEPT, 4), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_DR_INTERCEPT, 4), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_EXC_INTERCEPT, 4), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_INST1_INTERCEPT, 4), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_INST2_INTERCEPT, 4), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_PAUSE_FILTHRESH, 2), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_PAUSE_FILCNT, 2), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_ASID, 4), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_TLB_CTRL, 4), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_VIRQ, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_EXIT_REASON, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_EXITINFO1, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_EXITINFO2, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_EXITINTINFO, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_NP_ENABLE, 1), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_AVIC_BAR, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_AVIC_PAGE, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_AVIC_LT, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_AVIC_PT, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_CPL, 1), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_STAR, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_LSTAR, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_CSTAR, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_SFMASK, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_KERNELGBASE, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_SYSENTER_CS, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_SYSENTER_ESP, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_SYSENTER_EIP, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_GUEST_PAT, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_DBGCTL, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_BR_FROM, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_BR_TO, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_INT_FROM, 8), meta); - err += vmcb_snapshot_any(sc, vcpu, + err += vmcb_snapshot_any(sc, vcpuid, VMCB_ACCESS(VMCB_OFF_INT_TO, 8), meta); + if (err != 0) + goto done; + /* Snapshot swctx for virtual cpu */ + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_rbp, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_rbx, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_rcx, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_rdx, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_rdi, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_rsi, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r8, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r9, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r10, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r11, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r12, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r13, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r14, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_r15, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_dr0, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_dr1, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_dr2, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->swctx.sctx_dr3, meta, err, done); + + /* Restore other svm_vcpu struct fields */ + + /* Restore NEXTRIP field */ + SNAPSHOT_VAR_OR_LEAVE(vcpu->nextrip, meta, err, done); + + /* Restore lastcpu field */ + SNAPSHOT_VAR_OR_LEAVE(vcpu->lastcpu, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->dirty, meta, err, done); + + /* Restore EPTGEN field - EPT is Extended Page Table */ + SNAPSHOT_VAR_OR_LEAVE(vcpu->eptgen, meta, err, done); + + SNAPSHOT_VAR_OR_LEAVE(vcpu->asid.gen, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vcpu->asid.num, meta, err, done); + + /* Set all caches dirty */ + if (meta->op == VM_SNAPSHOT_RESTORE) + svm_set_dirty(sc, vcpuid, 0xffffffff); +done: return (err); } @@ -2679,7 +2668,7 @@ const struct vmm_ops vmm_ops_amd = { .vlapic_cleanup = svm_vlapic_cleanup, #ifdef BHYVE_SNAPSHOT .snapshot = svm_snapshot, - .vmcx_snapshot = svm_vmcx_snapshot, + .vcpu_snapshot = svm_vcpu_snapshot, .restore_tsc = svm_restore_tsc, #endif }; diff --git a/sys/amd64/vmm/intel/vmx.c b/sys/amd64/vmm/intel/vmx.c index 8b71ed2e39c1..7ece03a44952 100644 --- a/sys/amd64/vmm/intel/vmx.c +++ b/sys/amd64/vmm/intel/vmx.c @@ -4102,67 +4102,29 @@ vmx_vlapic_cleanup(void *arg, struct vlapic *vlapic) static int vmx_snapshot(void *arg, struct vm_snapshot_meta *meta) { - struct vmx *vmx; - struct vmx_vcpu *vcpu; - struct vmxctx *vmxctx; - int ret; - uint16_t i, maxcpus; - - vmx = arg; - - KASSERT(vmx != NULL, ("%s: arg was NULL", __func__)); - - maxcpus = vm_get_maxcpus(vmx->vm); - for (i = 0; i < maxcpus; i++) { - vcpu = &vmx->vcpus[i]; - - SNAPSHOT_BUF_OR_LEAVE(vcpu->guest_msrs, - sizeof(vcpu->guest_msrs), meta, ret, done); - - vmxctx = &vcpu->ctx; - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_rdi, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_rsi, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_rdx, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_rcx, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r8, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r9, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_rax, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_rbx, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_rbp, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r10, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r11, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r12, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r13, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r14, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r15, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_cr2, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_dr0, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_dr1, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_dr2, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_dr3, meta, ret, done); - SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_dr6, meta, ret, done); - } - -done: - return (ret); + return (0); } static int -vmx_vmcx_snapshot(void *arg, struct vm_snapshot_meta *meta, int vcpu) +vmx_vcpu_snapshot(void *arg, struct vm_snapshot_meta *meta, int vcpuid) { struct vmcs *vmcs; struct vmx *vmx; + struct vmx_vcpu *vcpu; + struct vmxctx *vmxctx; int err, run, hostcpu; vmx = (struct vmx *)arg; err = 0; KASSERT(arg != NULL, ("%s: arg was NULL", __func__)); - vmcs = vmx->vcpus[vcpu].vmcs; + vcpu = &vmx->vcpus[vcpuid]; + vmcs = vcpu->vmcs; - run = vcpu_is_running(vmx->vm, vcpu, &hostcpu); + run = vcpu_is_running(vmx->vm, vcpuid, &hostcpu); if (run && hostcpu != curcpu) { - printf("%s: %s%d is running", __func__, vm_name(vmx->vm), vcpu); + printf("%s: %s%d is running", __func__, vm_name(vmx->vm), + vcpuid); return (EINVAL); } @@ -4218,7 +4180,36 @@ vmx_vmcx_snapshot(void *arg, struct vm_snapshot_meta *meta, int vcpu) err += vmcs_snapshot_any(vmcs, run, VMCS_GUEST_ACTIVITY, meta); err += vmcs_snapshot_any(vmcs, run, VMCS_ENTRY_CTLS, meta); err += vmcs_snapshot_any(vmcs, run, VMCS_EXIT_CTLS, meta); + if (err != 0) + goto done; + + SNAPSHOT_BUF_OR_LEAVE(vcpu->guest_msrs, + sizeof(vcpu->guest_msrs), meta, err, done); + + vmxctx = &vcpu->ctx; + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_rdi, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_rsi, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_rdx, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_rcx, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r8, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r9, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_rax, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_rbx, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_rbp, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r10, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r11, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r12, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r13, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r14, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_r15, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_cr2, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_dr0, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_dr1, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_dr2, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_dr3, meta, err, done); + SNAPSHOT_VAR_OR_LEAVE(vmxctx->guest_dr6, meta, err, done); +done: return (err); } @@ -4268,7 +4259,7 @@ const struct vmm_ops vmm_ops_intel = { .vlapic_cleanup = vmx_vlapic_cleanup, #ifdef BHYVE_SNAPSHOT .snapshot = vmx_snapshot, - .vmcx_snapshot = vmx_vmcx_snapshot, + .vcpu_snapshot = vmx_vcpu_snapshot, .restore_tsc = vmx_restore_tsc, #endif }; diff --git a/sys/amd64/vmm/vmm.c b/sys/amd64/vmm/vmm.c index da1db09bdf58..89dedc676610 100644 --- a/sys/amd64/vmm/vmm.c +++ b/sys/amd64/vmm/vmm.c @@ -229,7 +229,7 @@ DEFINE_VMMOPS_IFUNC(void, vlapic_cleanup, (void *vmi, struct vlapic *vlapic)) #ifdef BHYVE_SNAPSHOT DEFINE_VMMOPS_IFUNC(int, snapshot, (void *vmi, struct vm_snapshot_meta *meta)) -DEFINE_VMMOPS_IFUNC(int, vmcx_snapshot, (void *vmi, struct vm_snapshot_meta +DEFINE_VMMOPS_IFUNC(int, vcpu_snapshot, (void *vmi, struct vm_snapshot_meta *meta, int vcpu)) DEFINE_VMMOPS_IFUNC(int, restore_tsc, (void *vmi, int vcpuid, uint64_t now)) #endif @@ -2860,7 +2860,7 @@ vm_snapshot_vmcx(struct vm *vm, struct vm_snapshot_meta *meta) maxcpus = vm_get_maxcpus(vm); for (i = 0; i < maxcpus; i++) { - error = vmmops_vmcx_snapshot(vm->cookie, meta, i); + error = vmmops_vcpu_snapshot(vm->cookie, meta, i); if (error != 0) { printf("%s: failed to snapshot vmcs/vmcb data for " "vCPU: %d; error: %d\n", __func__, i, error);