From nobody Tue Dec 6 18:27:08 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NRTR06vNcz4jbJh; Tue, 6 Dec 2022 18:27:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NRTR06PXVz4Nf9; Tue, 6 Dec 2022 18:27:08 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1670351228; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=XFhRm9t2KAFiA1KOIyyRJVGPlL+us/KbD9gfdEM2GP8=; b=oqCuzjvUBSXb5nVxI50G1ToQwv6GCc9KjfU7ONMNc2mjz4X7c89Li9137+eG4x/2q4i7d3 Ff0J1uRjbaw+lHzm5GHCqKSZYy12TjameVQKIfXA+4JfBD9rVpfpGBOdZ71FHB+KVAr07e RjMquoe2lFibrEXXJm7mRCSNdUX1fdjihzgXrZqO5j3ToFk1dA5rmGnVCgbpyKOUIEgVjt LYa+S28uNVqqUS/fof92TGzKPwMCKWPibgzS7A9J/xif8i7OVsEXa4qpj8mV7fPfM7LHVJ k9o0glXjhcepX9WTRhy8jTs3VDYNRLGVcBFO61BbCwuXZo8ygLm+0+ZgJP5Qaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1670351228; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=XFhRm9t2KAFiA1KOIyyRJVGPlL+us/KbD9gfdEM2GP8=; b=t3fAIRPlIxpDhoccGtAiCeucN2mIWLi5d1T4QybDwfmG67o+ogkMe0mY70SXZJURUWD3au hzaLXZMH23H0NWRQd0WJOmYtoCf/lJ4tuQ7O+waDBBB2HDeGe8Qr9pMcxzMXv1xcXFnlbW YuCKqj2gijWCI/1cRPtbkZxE895BocFXRRZQvNcIzlgSteVLcjkG7j++/YbykhKz6Q9Ein 9551EcOehxHVU1LT97ocjRrs9RuI+qxHodbcfeexwBwDT7QF5zPyobuNgy1XHT4YsIzEN0 ZQNy+GOU3TxvtvwduIRic5l23/OSS2LLud9SxUGn7AAJOMHw+YAen9rKFKuLWA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1670351228; a=rsa-sha256; cv=none; b=YpzOni9Nfh9DLSN5tXtcKgVm7+mo65STz2BNz8uomhb31CbyBASe/U745k3FX0GkpsDZmz A8TGtho2V6b9dPtho7U6Q49B6TNz7dSFpgtffk7FVeinDrnuPeKXGKhIjLnyuEQOrKQb4t crUXi146tkK6a3jDQe2EYdQpXOkGmdmuwQgE8pmwYT/9S5U65uqySCVqBNM14DBm9jc8Ve /o3xSnqPpGBF32iXtYChj0oPORUs9jv8V7WYP4G9fLfUmJYvaQM7lzSAdsP7QhyGUzjGw8 p8JXIGFxhDxIHym9t90G8lygXOi5KU+rj1uVwAtl/TCSK7BaK4lEGNW3m9MH8A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4NRTR058F5zn64; Tue, 6 Dec 2022 18:27:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 2B6IR8QW048030; Tue, 6 Dec 2022 18:27:08 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 2B6IR80D048029; Tue, 6 Dec 2022 18:27:08 GMT (envelope-from git) Date: Tue, 6 Dec 2022 18:27:08 GMT Message-Id: <202212061827.2B6IR80D048029@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kyle Evans Subject: git: 66a890f67f67 - stable/13 - arm64: add a spin-table implementation for Apple Silicon List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kevans X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 66a890f67f67d86fafb81625ade944547022de90 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by kevans: URL: https://cgit.FreeBSD.org/src/commit/?id=66a890f67f67d86fafb81625ade944547022de90 commit 66a890f67f67d86fafb81625ade944547022de90 Author: Kyle Evans AuthorDate: 2022-11-11 19:50:29 +0000 Commit: Kyle Evans CommitDate: 2022-12-06 18:26:56 +0000 arm64: add a spin-table implementation for Apple Silicon The M1 has no EL3, so we're limited to a spin-table implementation if we want to eventually use bhyve on it. Implement spin-table now, but note that we still prefer PSCI where possible. Reviewed by: mmel (cherry picked from commit 753a23ac152c5024de34f9eb02a2fdccf2881085) --- sys/arm64/arm64/mp_machdep.c | 113 +++++++++++++++++++++++++++++++++++++------ 1 file changed, 99 insertions(+), 14 deletions(-) diff --git a/sys/arm64/arm64/mp_machdep.c b/sys/arm64/arm64/mp_machdep.c index 4d3f9f4963f5..187e00576e03 100644 --- a/sys/arm64/arm64/mp_machdep.c +++ b/sys/arm64/arm64/mp_machdep.c @@ -487,12 +487,54 @@ cpu_mp_probe(void) return (1); } +static int +enable_cpu_psci(uint64_t target_cpu, vm_paddr_t entry, u_int cpuid) +{ + int err; + + err = psci_cpu_on(target_cpu, entry, cpuid); + if (err != PSCI_RETVAL_SUCCESS) { + /* + * Panic here if INVARIANTS are enabled and PSCI failed to + * start the requested CPU. psci_cpu_on() returns PSCI_MISSING + * to indicate we are unable to use it to start the given CPU. + */ + KASSERT(err == PSCI_MISSING || + (mp_quirks & MP_QUIRK_CPULIST) == MP_QUIRK_CPULIST, + ("Failed to start CPU %u (%lx), error %d\n", + cpuid, target_cpu, err)); + return (EINVAL); + } + + return (0); +} + +static int +enable_cpu_spin(uint64_t cpu, vm_paddr_t entry, vm_paddr_t release_paddr) +{ + vm_paddr_t *release_addr; + + release_addr = pmap_mapdev(release_paddr, sizeof(*release_addr)); + if (release_addr == NULL) + return (ENOMEM); + + *release_addr = entry; + pmap_unmapdev((vm_offset_t)release_addr, sizeof(*release_addr)); + + __asm __volatile( + "dsb sy \n" + "sev \n" + ::: "memory"); + + return (0); +} + /* * Starts a given CPU. If the CPU is already running, i.e. it is the boot CPU, * do nothing. Returns true if the CPU is present and running. */ static bool -start_cpu(u_int cpuid, uint64_t target_cpu, int domain) +start_cpu(u_int cpuid, uint64_t target_cpu, int domain, vm_paddr_t release_addr) { struct pcpu *pcpup; vm_offset_t pcpu_mem; @@ -531,18 +573,19 @@ start_cpu(u_int cpuid, uint64_t target_cpu, int domain) printf("Starting CPU %u (%lx)\n", cpuid, target_cpu); pa = pmap_extract(kernel_pmap, (vm_offset_t)mpentry); - err = psci_cpu_on(target_cpu, pa, cpuid); - if (err != PSCI_RETVAL_SUCCESS) { - /* - * Panic here if INVARIANTS are enabled and PSCI failed to - * start the requested CPU. psci_cpu_on() returns PSCI_MISSING - * to indicate we are unable to use it to start the given CPU. - */ - KASSERT(err == PSCI_MISSING || - (mp_quirks & MP_QUIRK_CPULIST) == MP_QUIRK_CPULIST, - ("Failed to start CPU %u (%lx), error %d\n", - cpuid, target_cpu, err)); + /* + * A limited set of hardware we support can only do spintables and + * remain useful, due to lack of EL3. Thus, we'll usually fall into the + * PSCI branch here. + */ + MPASS(release_addr == 0 || !psci_present); + if (release_addr != 0) + err = enable_cpu_spin(target_cpu, pa, release_addr); + else + err = enable_cpu_psci(target_cpu, pa, cpuid); + + if (err != 0) { pcpu_destroy(pcpup); dpcpu[cpuid - 1] = NULL; kmem_free((vm_offset_t)bootstacks[cpuid], MP_BOOTSTACK_SIZE); @@ -584,7 +627,7 @@ madt_handler(ACPI_SUBTABLE_HEADER *entry, void *arg) if (vm_ndomains > 1) domain = acpi_pxm_get_cpu_locality(intr->Uid); #endif - if (start_cpu(id, intr->ArmMpidr, domain)) { + if (start_cpu(id, intr->ArmMpidr, domain, 0)) { MPASS(cpuid_to_pcpu[id] != NULL); cpuid_to_pcpu[id]->pc_acpi_id = intr->Uid; /* @@ -631,10 +674,27 @@ cpu_init_acpi(void) #endif #ifdef FDT +/* + * Failure is indicated by failing to populate *release_addr. + */ +static void +populate_release_addr(phandle_t node, vm_paddr_t *release_addr) +{ + pcell_t buf[2]; + + if (OF_getencprop(node, "cpu-release-addr", buf, sizeof(buf)) != + sizeof(buf)) + return; + + *release_addr = (((uintptr_t)buf[0] << 32) | buf[1]); +} + static boolean_t start_cpu_fdt(u_int id, phandle_t node, u_int addr_size, pcell_t *reg) { uint64_t target_cpu; + vm_paddr_t release_addr; + char *enable_method; int domain; int cpuid; @@ -649,7 +709,32 @@ start_cpu_fdt(u_int id, phandle_t node, u_int addr_size, pcell_t *reg) else cpuid = fdt_cpuid; - if (!start_cpu(cpuid, target_cpu, 0)) + /* + * If PSCI is present, we'll always use that -- the cpu_on method is + * mandated in both v0.1 and v0.2. We'll check the enable-method if + * we don't have PSCI and use spin table if it's provided. + */ + release_addr = 0; + if (!psci_present && cpuid != 0) { + if (OF_getprop_alloc(node, "enable-method", + (void **)&enable_method) <= 0) + return (FALSE); + + if (strcmp(enable_method, "spin-table") != 0) { + OF_prop_free(enable_method); + return (FALSE); + } + + OF_prop_free(enable_method); + populate_release_addr(node, &release_addr); + if (release_addr == 0) { + printf("Failed to fetch release address for CPU %u", + cpuid); + return (FALSE); + } + } + + if (!start_cpu(cpuid, target_cpu, 0, release_addr)) return (FALSE); /*