From owner-svn-src-all@freebsd.org Fri May 1 21:58:20 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1B7202B0E36; Fri, 1 May 2020 21:58:20 +0000 (UTC) (envelope-from mhorne@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49DR2h0033z49nR; Fri, 1 May 2020 21:58:19 +0000 (UTC) (envelope-from mhorne@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EABD8F0D6; Fri, 1 May 2020 21:58:19 +0000 (UTC) (envelope-from mhorne@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 041LwJAs014783; Fri, 1 May 2020 21:58:19 GMT (envelope-from mhorne@FreeBSD.org) Received: (from mhorne@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 041LwJmo014782; Fri, 1 May 2020 21:58:19 GMT (envelope-from mhorne@FreeBSD.org) Message-Id: <202005012158.041LwJmo014782@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mhorne set sender to mhorne@FreeBSD.org using -f From: Mitchell Horne Date: Fri, 1 May 2020 21:58:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r360553 - head/sys/riscv/riscv X-SVN-Group: head X-SVN-Commit-Author: mhorne X-SVN-Commit-Paths: head/sys/riscv/riscv X-SVN-Commit-Revision: 360553 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 May 2020 21:58:20 -0000 Author: mhorne Date: Fri May 1 21:58:19 2020 New Revision: 360553 URL: https://svnweb.freebsd.org/changeset/base/360553 Log: Use the HSM SBI extension to start APs The addition of the HSM SBI extension to OpenSBI introduces a new breaking change: secondary harts will remain parked in the firmware, until they are brought up explicitly via sbi_hsm_hart_start(). Add the call to do this, sending the secondary harts to mpentry. If the HSM extension is not present, secondary harts are assumed to be released by the firmware, as is the case for OpenSBI =< v0.6 and BBL. In the case that the HSM call fails we exclude the CPU, notify the user, and allow the system to proceed with booting. Reviewed by: markj (older version) Differential Revision: https://reviews.freebsd.org/D24497 Modified: head/sys/riscv/riscv/mp_machdep.c Modified: head/sys/riscv/riscv/mp_machdep.c ============================================================================== --- head/sys/riscv/riscv/mp_machdep.c Fri May 1 21:55:51 2020 (r360552) +++ head/sys/riscv/riscv/mp_machdep.c Fri May 1 21:58:19 2020 (r360553) @@ -97,6 +97,7 @@ static uint32_t cpu_reg[MAXCPU][2]; #endif static device_t cpu_list[MAXCPU]; +void mpentry(u_long hartid); void init_secondary(uint64_t); static struct mtx ap_boot_mtx; @@ -297,7 +298,7 @@ smp_after_idle_runnable(void *arg __unused) struct pcpu *pc; int cpu; - for (cpu = 1; cpu < mp_ncpus; cpu++) { + for (cpu = 1; cpu <= mp_maxid; cpu++) { if (bootstacks[cpu] != NULL) { pc = pcpu_find(cpu); while (atomic_load_ptr(&pc->pc_curpcb) == NULL) @@ -399,9 +400,11 @@ static boolean_t cpu_init_fdt(u_int id, phandle_t node, u_int addr_size, pcell_t *reg) { struct pcpu *pcpup; + vm_paddr_t start_addr; uint64_t hart; u_int cpuid; int naps; + int error; /* Check if this hart supports MMU. */ if (OF_getproplen(node, "mmu-type") < 0) @@ -440,6 +443,23 @@ cpu_init_fdt(u_int id, phandle_t node, u_int addr_size /* Check if we are able to start this cpu */ if (cpuid > mp_maxid) return (0); + + /* + * Depending on the SBI implementation, APs are waiting either in + * locore.S or to be activated explicitly, via SBI call. + */ + if (sbi_probe_extension(SBI_EXT_ID_HSM) != 0) { + start_addr = pmap_kextract((vm_offset_t)mpentry); + error = sbi_hsm_hart_start(hart, start_addr, 0); + if (error != 0) { + mp_ncpus--; + + /* Send a warning to the user and continue. */ + printf("AP %u (hart %lu) failed to start, error %d\n", + cpuid, hart, error); + return (0); + } + } pcpup = &__pcpu[cpuid]; pcpu_init(pcpup, cpuid, sizeof(struct pcpu));