From nobody Thu Feb 23 15:36:26 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PMxvb0yyWz3s7q5; Thu, 23 Feb 2023 15:36:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PMxvb0Pdjz3Hk9; Thu, 23 Feb 2023 15:36:27 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1677166587; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=tiaN5MgJ9ic80Yz2YACe7w1lBcI4yIW0gei95aWw+c4=; b=OaCTIsvoCtbAEP3bn328q1J7VLnsNCjW3X5JUmWh42JyVjdknIU3Yg1AYTLNtScaKnVhJq HBse/PeHj+RODml4PGOJD0NxOTxFKG9uUR/9jZbvggzyywtJS5Jl8KGsiGr6XEGQDDXUAu Vmtl5aPCwD1UAjxXzwDD3xDjqS2U8LeOryv0NesrdOGNNG+yZyE9q1XS/7w4W/UysqoLdE 9x/4XfFaBFPuCs4o2rx8rSwbYQn1mb5qr15c61kvAN70Bwn0KVD00mDV4j0NfOwBefrEek RHFEXCYCKev3r120iVJKv8iP/S6JtEVQy2mKKTueIGRJLlxDnw101wXMozcfsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1677166587; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=tiaN5MgJ9ic80Yz2YACe7w1lBcI4yIW0gei95aWw+c4=; b=ScR+gaJRAB3lO51aucDUwJfzMWvjKCkesTnJxMs19XBk9HRjAw4Gp/S1J3qXLWWPiE9iwb XJ8LhnxZS5TGvO49skgeEdDRfYJfIRaBOqvwcln/CqqHbFvPVo4nTHFNTyPg3rhEJMx3xj VNmMz8mmXq9Kl+CO+AobsJcBl8IrfQ7rDNY7O5bAhDqND59rMnPKYpOaEeq5fsfPP4nfIO E1Nh5OUxm+2r6AllHVMJ0o1bxWmG7AOziMlfb/th9ul9/UOJOpD+5dvl4OpCRpDSnINxvW fBzQjLjQFQ8vQg1FiRn7xQenLdjOnMJe6aiOUXiX6T8UKv2A7b5g7q5LsXWeMQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1677166587; a=rsa-sha256; cv=none; b=sbF6bPdmZPzrpvj5E6LcXjHHoN/FJ5Vkg077i3GTewvyg/ZDjFcfMXWg+JW5ng6J8+kGo6 F6I9p14Zf6Ni9492SUBSdqC9gq6Pz/kqMoIgkOCuzilkhD+nKrCDYdPPcrT9/4ZSq2EpVi fVfYFtFD4nKwb8qWwwo0Ied4pZUPJd3InblZGrjfn5hHRjvsyGjM5E3m12maT0lzWVMsSO BZIvPFVNVCLm6lpW/zORSKGUZc4OjH4jXFinXs1is94v7frpJgHy7Mglr3pSAtZ3J5uD74 Xd9N/TjRuXvte4adJyMKzVOkhxwwgPUgUeE7j2EOH5jHVuWsG8NCY7J0nTy1DQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4PMxvZ6CSczn3n; Thu, 23 Feb 2023 15:36:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 31NFaQ4h075083; Thu, 23 Feb 2023 15:36:26 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 31NFaQPx075082; Thu, 23 Feb 2023 15:36:26 GMT (envelope-from git) Date: Thu, 23 Feb 2023 15:36:26 GMT Message-Id: <202302231536.31NFaQPx075082@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: 577a666c3c33 - stable/13 - vmm: Fix AP startup compatibility for old bhyve executables List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 577a666c3c33def1d4b996575dc43f8567fdac5c Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=577a666c3c33def1d4b996575dc43f8567fdac5c commit 577a666c3c33def1d4b996575dc43f8567fdac5c Author: Mark Johnston AuthorDate: 2023-02-09 20:52:35 +0000 Commit: Mark Johnston CommitDate: 2023-02-23 15:15:47 +0000 vmm: Fix AP startup compatibility for old bhyve executables These changes unbreak AP startup when using a 13.1-RELEASE bhyve executable with a newer kernel: - Correct the destination mask for the VM_EXITCODE_IPI message generated by an INIT or STARTUP IPI in vlapic_icrlo_write_handler(). - Only initialize vlapics on active vCPUs. 13.1-RELEASE bhyve activates AP vCPUs only after the BSP starts them with an IPI, and vmm now allocates vcpu structures lazily, so the STARTUP handling in vm_handle_ipi() could trigger a page fault. - Fix an off-by-one setting the vcpuid in a VM_EXITCODE_SPINUP_AP message. Fixes: 7c326ab5bb9a ("vmm: don't lock a mtx in the icr_low write handler") Reviewed by: jhb, corvink MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D38446 (cherry picked from commit b265a2e0d76422f4007e96dd7295ed0aeb846e2d) --- sys/amd64/vmm/io/vlapic.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/sys/amd64/vmm/io/vlapic.c b/sys/amd64/vmm/io/vlapic.c index a68c68073893..884e232b1422 100644 --- a/sys/amd64/vmm/io/vlapic.c +++ b/sys/amd64/vmm/io/vlapic.c @@ -1144,7 +1144,7 @@ vlapic_icrlo_write_handler(struct vlapic *vlapic, bool *retu) vmexit->exitcode = VM_EXITCODE_IPI; vmexit->u.ipi.mode = mode; vmexit->u.ipi.vector = vec; - vmexit->u.ipi.dmask = dmask; + vmexit->u.ipi.dmask = ipimask; *retu = true; } @@ -1169,16 +1169,22 @@ vm_handle_ipi(struct vcpu *vcpu, struct vm_exit *vme, bool *retu) *retu = true; switch (vme->u.ipi.mode) { - case APIC_DELMODE_INIT: - vm_smp_rendezvous(vcpu, *dmask, vlapic_handle_init, - NULL); + case APIC_DELMODE_INIT: { + cpuset_t active, reinit; + + active = vm_active_cpus(vcpu_vm(vcpu)); + CPU_AND(&reinit, &active, dmask); + if (!CPU_EMPTY(&reinit)) { + vm_smp_rendezvous(vcpu, reinit, vlapic_handle_init, + NULL); + } vm_await_start(vcpu_vm(vcpu), dmask); - if (!vlapic->ipi_exit) { + if (!vlapic->ipi_exit) *retu = false; - } break; + } case APIC_DELMODE_STARTUP: /* * Ignore SIPIs in any state other than wait-for-SIPI @@ -1196,13 +1202,13 @@ vm_handle_ipi(struct vcpu *vcpu, struct vm_exit *vme, bool *retu) */ if (!vlapic->ipi_exit) { vme->exitcode = VM_EXITCODE_SPINUP_AP; - vme->u.spinup_ap.vcpu = CPU_FFS(dmask); + vme->u.spinup_ap.vcpu = CPU_FFS(dmask) - 1; vme->u.spinup_ap.rip = vec << PAGE_SHIFT; } break; default: - return (1); + __assert_unreachable(); } return (0);