From nobody Tue Sep 23 17:09:38 2025 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cWRKt4Y66z68tT2; Tue, 23 Sep 2025 17:09:38 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4cWRKt2lX4z471v; Tue, 23 Sep 2025 17:09:38 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1758647378; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=orUVgLfyXwjrJyNpmA3gNC+goamIjQkYIHEGgRRbWco=; b=LlSyuZ0M1ilc3FQ+AmqMZBh5lTHr9Rf9RTU6RJmArSmXfZ/3KVNtc3s9tyT6GE49iGkNAf HV3xTTQbCxZ913k/KvyD65xRSN+8e7mztAR0GVZS6nhrQvbOulAgWzdFR5tMGY4J7ExdcF BVvDP9dQ+JNfeHrhPvkA9iLRZosfhKtue99jREmvdnRAPzG4hOAGDJrInO72FqsKaLBJ3s ZmsubdHbme9X0GF/bGr0oYlz5yGh0tWxNFI752DkCsYfPb5gg8Whv3qjShI3KsKo0vBg8P 8d76GoOd4wBPctvQkdTV8uBSz5e5Os06eYa9dGRfjZOhDRAX/OIgByg7Jjq3jA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1758647378; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=orUVgLfyXwjrJyNpmA3gNC+goamIjQkYIHEGgRRbWco=; b=YYVpcWo5MKkWbb6cfDThYb+lxNyZZGz7U4M6/YfLJO151qYfkF+dDO3J2AO3g+vvLu6r2N rWWeoVzQu6FmEzqQLJJJXYLbAVkU6Djdsg61Q8LghYg0VVkL7XMdoxN/JQffR59gtO6NVr x4uyS76gqMVy5pYoc1IY3FVBPxrFFSq0Ufa4iuhuSydXtZn3G6IgmdY4P+lkwnAw9S6zkS vFQ90m80rvSivEyGTqzbxGEIdoff43jBCrmShmStgjrHwlRlB+isoMfbrgA9n11v6WOLY0 9kTW0uhmPnO23dkSX4qYGOggmHquZ/S3xSkmCJVP9K8gw76KIlvRXb/VSdcvGQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1758647378; a=rsa-sha256; cv=none; b=YTFihJ0pYJdCpLjOxRS5ajUFJYrB81c/uk+NxK1umLZ4yLioUfY0TpxKmV3uVxmovwy8yP XzaIt2jlp64FXSAq+DPwxDJp31TYtd511PeXzw7amdAbw2YmVvQOK+vWOpG9BR9Y2kwmnJ NgCAB/DyTCfWOPLFNUh2+SmNqx247CEsxx9Rgf04k8oe5V5SFuhb/wCYgXgRwzjBgvyJSq WrlHhAwB3cmJnSM33yCbm+6Gx85aDbdo7EkzEYKPRnAshmFRcSHq0YWQox9NxgLo0znMWX nTcDY18REWr1tFeAqKn5gegnBZDZDn6OxorsR9bf+QkNK/MBdLtL6dmt4xpn3w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4cWRKt2LB0zvSD; Tue, 23 Sep 2025 17:09:38 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 58NH9c8l077404; Tue, 23 Sep 2025 17:09:38 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 58NH9c9N077401; Tue, 23 Sep 2025 17:09:38 GMT (envelope-from git) Date: Tue, 23 Sep 2025 17:09:38 GMT Message-Id: <202509231709.58NH9c9N077401@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Andrew Turner Subject: git: fbe076b2c837 - main - arm64/vmm: Use FEAT_ECV_POFF to support a timer List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: fbe076b2c837f396f96d4725a43745e741557df1 Auto-Submitted: auto-generated The branch main has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=fbe076b2c837f396f96d4725a43745e741557df1 commit fbe076b2c837f396f96d4725a43745e741557df1 Author: Andrew Turner AuthorDate: 2025-09-22 17:09:54 +0000 Commit: Andrew Turner CommitDate: 2025-09-23 17:08:37 +0000 arm64/vmm: Use FEAT_ECV_POFF to support a timer Support guest access to the physical timer when FEAT_ECV_POFF is supported. In this case we can set an offset for the physical timer. We can reuse the virtual timer support to also support the physical timer, with a few more registers needing to be handled when switching to a guest. As it is not clear how this will affect performance when the guest doesn't use it hide enabling it behind a sysctl. It is expected this will be useful when Nested Virtualisation is supported as guests are expected to use the physical timer registers. Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D51821 --- sys/arm64/vmm/arm64.h | 1 + sys/arm64/vmm/io/vtimer.c | 74 +++++++++++++++++++++++++++++++++++++---------- sys/arm64/vmm/vmm_arm64.c | 5 ++++ sys/arm64/vmm/vmm_hyp.c | 54 +++++++++++++++++++++++++++++----- 4 files changed, 110 insertions(+), 24 deletions(-) diff --git a/sys/arm64/vmm/arm64.h b/sys/arm64/vmm/arm64.h index 0bd5a933bee1..82c4481b8692 100644 --- a/sys/arm64/vmm/arm64.h +++ b/sys/arm64/vmm/arm64.h @@ -127,6 +127,7 @@ struct hyp { uint64_t el2_addr; /* The address of this in el2 space */ uint64_t feats; /* Which features are enabled */ #define HYP_FEAT_HCX (0x1ul << 0) +#define HYP_FEAT_ECV_POFF (0x1ul << 1) bool vgic_attached; struct vgic_v3 *vgic; struct hypctx *ctx[]; diff --git a/sys/arm64/vmm/io/vtimer.c b/sys/arm64/vmm/io/vtimer.c index ddc9e6e840a5..da0f0d96c431 100644 --- a/sys/arm64/vmm/io/vtimer.c +++ b/sys/arm64/vmm/io/vtimer.c @@ -36,6 +36,7 @@ #include #include #include +#include #include #include #include @@ -59,6 +60,14 @@ static uint32_t tmr_frq; #define timer_condition_met(ctl) ((ctl) & CNTP_CTL_ISTATUS) +SYSCTL_DECL(_hw_vmm); +SYSCTL_NODE(_hw_vmm, OID_AUTO, vtimer, CTLFLAG_RW, NULL, NULL); + +static bool allow_ecv_phys = false; +SYSCTL_BOOL(_hw_vmm_vtimer, OID_AUTO, allow_ecv_phys, CTLFLAG_RW, + &allow_ecv_phys, 0, + "Enable hardware access to the physical timer if FEAT_ECV_POFF is supported"); + static void vtimer_schedule_irq(struct hypctx *hypctx, bool phys); static int @@ -126,7 +135,12 @@ void vtimer_vminit(struct hyp *hyp) { uint64_t now; + bool ecv_poff; + + ecv_poff = false; + if (allow_ecv_phys && (hyp->feats & HYP_FEAT_ECV_POFF) != 0) + ecv_poff = true; /* * Configure the Counter-timer Hypervisor Control Register for the VM. @@ -151,23 +165,41 @@ vtimer_vminit(struct hyp *hyp) * TODO: Don't trap when FEAT_ECV is present */ hyp->vtimer.cnthctl_el2 = - CNTHCTL_E2H_EL0PCTEN_TRAP | CNTHCTL_E2H_EL0VCTEN_NOTRAP | - CNTHCTL_E2H_EL0VTEN_NOTRAP | - CNTHCTL_E2H_EL0PTEN_TRAP | - CNTHCTL_E2H_EL1PCTEN_TRAP | - CNTHCTL_E2H_EL1PTEN_TRAP; + CNTHCTL_E2H_EL0VTEN_NOTRAP; + if (ecv_poff) { + hyp->vtimer.cnthctl_el2 |= + CNTHCTL_E2H_EL0PCTEN_NOTRAP | + CNTHCTL_E2H_EL0PTEN_NOTRAP | + CNTHCTL_E2H_EL1PCTEN_NOTRAP | + CNTHCTL_E2H_EL1PTEN_NOTRAP; + } else { + hyp->vtimer.cnthctl_el2 |= + CNTHCTL_E2H_EL0PCTEN_TRAP | + CNTHCTL_E2H_EL0PTEN_TRAP | + CNTHCTL_E2H_EL1PCTEN_TRAP | + CNTHCTL_E2H_EL1PTEN_TRAP; + } } else { /* * CNTHCTL_EL1PCEN: trap access to CNTP_{CTL, CVAL, TVAL}_EL0 * from EL1 * CNTHCTL_EL1PCTEN: trap access to CNTPCT_EL0 */ - hyp->vtimer.cnthctl_el2 = - CNTHCTL_EL1PCTEN_TRAP | - CNTHCTL_EL1PCEN_TRAP; + if (ecv_poff) { + hyp->vtimer.cnthctl_el2 = + CNTHCTL_EL1PCTEN_NOTRAP | + CNTHCTL_EL1PCEN_NOTRAP; + } else { + hyp->vtimer.cnthctl_el2 = + CNTHCTL_EL1PCTEN_TRAP | + CNTHCTL_EL1PCEN_TRAP; + } } + if (ecv_poff) + hyp->vtimer.cnthctl_el2 |= CNTHCTL_ECV_EN; + now = READ_SPECIALREG(cntpct_el0); hyp->vtimer.cntvoff_el2 = now; @@ -233,15 +265,10 @@ vtimer_cleanup(void) { } -void -vtimer_sync_hwstate(struct hypctx *hypctx) +static void +vtime_sync_timer(struct hypctx *hypctx, struct vtimer_timer *timer, + uint64_t cntpct_el0) { - struct vtimer_timer *timer; - uint64_t cntpct_el0; - - timer = &hypctx->vtimer_cpu.virt_timer; - cntpct_el0 = READ_SPECIALREG(cntpct_el0) - - hypctx->hyp->vtimer.cntvoff_el2; if (!timer_enabled(timer->cntx_ctl_el0)) { vgic_inject_irq(hypctx->hyp, vcpu_vcpuid(hypctx->vcpu), timer->irqid, false); @@ -255,6 +282,21 @@ vtimer_sync_hwstate(struct hypctx *hypctx) } } +void +vtimer_sync_hwstate(struct hypctx *hypctx) +{ + uint64_t cntpct_el0; + + cntpct_el0 = READ_SPECIALREG(cntpct_el0) - + hypctx->hyp->vtimer.cntvoff_el2; + vtime_sync_timer(hypctx, &hypctx->vtimer_cpu.virt_timer, cntpct_el0); + /* If FEAT_ECV_POFF is in use then we need to sync the physical timer */ + if ((hypctx->hyp->vtimer.cnthctl_el2 & CNTHCTL_ECV_EN) != 0) { + vtime_sync_timer(hypctx, &hypctx->vtimer_cpu.phys_timer, + cntpct_el0); + } +} + static void vtimer_inject_irq_callout_phys(void *context) { diff --git a/sys/arm64/vmm/vmm_arm64.c b/sys/arm64/vmm/vmm_arm64.c index fa13fc76677a..618f4afaf8ee 100644 --- a/sys/arm64/vmm/vmm_arm64.c +++ b/sys/arm64/vmm/vmm_arm64.c @@ -523,6 +523,11 @@ vmmops_init(struct vm *vm, pmap_t pmap) hyp->vm = vm; hyp->vgic_attached = false; + if (get_kernel_reg(ID_AA64MMFR0_EL1, &idreg)) { + if (ID_AA64MMFR0_ECV_VAL(idreg) >= ID_AA64MMFR0_ECV_POFF) + hyp->feats |= HYP_FEAT_ECV_POFF; + } + if (get_kernel_reg(ID_AA64MMFR1_EL1, &idreg)) { if (ID_AA64MMFR1_HCX_VAL(idreg) >= ID_AA64MMFR1_HCX_IMPL) hyp->feats |= HYP_FEAT_HCX; diff --git a/sys/arm64/vmm/vmm_hyp.c b/sys/arm64/vmm/vmm_hyp.c index 6bbf0d7eb730..345535318f6e 100644 --- a/sys/arm64/vmm/vmm_hyp.c +++ b/sys/arm64/vmm/vmm_hyp.c @@ -42,11 +42,11 @@ struct hypctx; uint64_t VMM_HYP_FUNC(do_call_guest)(struct hypctx *); static void -vmm_hyp_reg_store(struct hypctx *hypctx, struct hyp *hyp, bool guest) +vmm_hyp_reg_store(struct hypctx *hypctx, struct hyp *hyp, bool guest, + bool ecv_poff) { uint64_t dfr0; - /* Store the guest VFP registers */ if (guest) { /* Store the timer registers */ hypctx->vtimer_cpu.cntkctl_el1 = @@ -55,7 +55,20 @@ vmm_hyp_reg_store(struct hypctx *hypctx, struct hyp *hyp, bool guest) READ_SPECIALREG(EL0_REG(CNTV_CVAL)); hypctx->vtimer_cpu.virt_timer.cntx_ctl_el0 = READ_SPECIALREG(EL0_REG(CNTV_CTL)); + } + if (guest_or_nonvhe(guest) && ecv_poff) { + /* + * If we have ECV then the guest could modify these registers. + * If VHE is enabled then the kernel will see a different view + * of the registers, so doesn't need to handle them. + */ + hypctx->vtimer_cpu.phys_timer.cntx_cval_el0 = + READ_SPECIALREG(EL0_REG(CNTP_CVAL)); + hypctx->vtimer_cpu.phys_timer.cntx_ctl_el0 = + READ_SPECIALREG(EL0_REG(CNTP_CTL)); + } + if (guest) { /* Store the GICv3 registers */ hypctx->vgic_v3_regs.ich_eisr_el2 = READ_SPECIALREG(ich_eisr_el2); @@ -262,7 +275,8 @@ vmm_hyp_reg_store(struct hypctx *hypctx, struct hyp *hyp, bool guest) } static void -vmm_hyp_reg_restore(struct hypctx *hypctx, struct hyp *hyp, bool guest) +vmm_hyp_reg_restore(struct hypctx *hypctx, struct hyp *hyp, bool guest, + bool ecv_poff) { uint64_t dfr0; @@ -440,6 +454,29 @@ vmm_hyp_reg_restore(struct hypctx *hypctx, struct hyp *hyp, bool guest) WRITE_SPECIALREG(cnthctl_el2, hyp->vtimer.cnthctl_el2); WRITE_SPECIALREG(cntvoff_el2, hyp->vtimer.cntvoff_el2); + if (ecv_poff) { + /* + * Load the same offset as the virtual timer + * to keep in sync. + */ + WRITE_SPECIALREG(CNTPOFF_EL2_REG, + hyp->vtimer.cntvoff_el2); + isb(); + } + } + if (guest_or_nonvhe(guest) && ecv_poff) { + /* + * If we have ECV then the guest could modify these registers. + * If VHE is enabled then the kernel will see a different view + * of the registers, so doesn't need to handle them. + */ + WRITE_SPECIALREG(EL0_REG(CNTP_CVAL), + hypctx->vtimer_cpu.phys_timer.cntx_cval_el0); + WRITE_SPECIALREG(EL0_REG(CNTP_CTL), + hypctx->vtimer_cpu.phys_timer.cntx_ctl_el0); + } + + if (guest) { /* Load the GICv3 registers */ WRITE_SPECIALREG(ich_hcr_el2, hypctx->vgic_v3_regs.ich_hcr_el2); WRITE_SPECIALREG(ich_vmcr_el2, @@ -497,9 +534,10 @@ vmm_hyp_call_guest(struct hyp *hyp, struct hypctx *hypctx) #endif uint64_t ret; uint64_t s1e1r, hpfar_el2; - bool hpfar_valid; + bool ecv_poff, hpfar_valid; - vmm_hyp_reg_store(&host_hypctx, NULL, false); + ecv_poff = (hyp->vtimer.cnthctl_el2 & CNTHCTL_ECV_EN) != 0; + vmm_hyp_reg_store(&host_hypctx, NULL, false, ecv_poff); #ifndef VMM_VHE if ((hyp->feats & HYP_FEAT_HCX) != 0) hcrx_el2 = READ_SPECIALREG(MRS_REG_ALT_NAME(HCRX_EL2)); @@ -513,7 +551,7 @@ vmm_hyp_call_guest(struct hyp *hyp, struct hypctx *hypctx) ich_hcr_el2 = READ_SPECIALREG(ich_hcr_el2); ich_vmcr_el2 = READ_SPECIALREG(ich_vmcr_el2); - vmm_hyp_reg_restore(hypctx, hyp, true); + vmm_hyp_reg_restore(hypctx, hyp, true, ecv_poff); /* Load the common hypervisor registers */ WRITE_SPECIALREG(vttbr_el2, hyp->vttbr_el2); @@ -529,7 +567,7 @@ vmm_hyp_call_guest(struct hyp *hyp, struct hypctx *hypctx) /* Store the exit info */ hypctx->exit_info.far_el2 = READ_SPECIALREG(far_el2); - vmm_hyp_reg_store(hypctx, hyp, true); + vmm_hyp_reg_store(hypctx, hyp, true, ecv_poff); hpfar_valid = true; if (ret == EXCP_TYPE_EL1_SYNC) { @@ -579,7 +617,7 @@ vmm_hyp_call_guest(struct hyp *hyp, struct hypctx *hypctx) } } - vmm_hyp_reg_restore(&host_hypctx, NULL, false); + vmm_hyp_reg_restore(&host_hypctx, NULL, false, ecv_poff); #ifndef VMM_VHE if ((hyp->feats & HYP_FEAT_HCX) != 0)