From owner-svn-src-head@freebsd.org Wed Jul 1 15:17:46 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EB97D354F09; Wed, 1 Jul 2020 15:17:46 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49xlGL5tzPz4cpW; Wed, 1 Jul 2020 15:17:46 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A8DC3B70E; Wed, 1 Jul 2020 15:17:46 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 061FHk0X084139; Wed, 1 Jul 2020 15:17:46 GMT (envelope-from andrew@FreeBSD.org) Received: (from andrew@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 061FHk9B084136; Wed, 1 Jul 2020 15:17:46 GMT (envelope-from andrew@FreeBSD.org) Message-Id: <202007011517.061FHk9B084136@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: andrew set sender to andrew@FreeBSD.org using -f From: Andrew Turner Date: Wed, 1 Jul 2020 15:17:46 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r362837 - head/sys/arm64/arm64 X-SVN-Group: head X-SVN-Commit-Author: andrew X-SVN-Commit-Paths: head/sys/arm64/arm64 X-SVN-Commit-Revision: 362837 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jul 2020 15:17:47 -0000 Author: andrew Date: Wed Jul 1 15:17:45 2020 New Revision: 362837 URL: https://svnweb.freebsd.org/changeset/base/362837 Log: Read the arm64 ID registers earlier in the boot process. Also move parsing the registers to just after the secondary CPUs have started. This means the kernel register view from all CPUs is available after the CPU SYSINITs have finished, e.g. for use by ifunc resolvers. Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D25505 Modified: head/sys/arm64/arm64/identcpu.c head/sys/arm64/arm64/mp_machdep.c Modified: head/sys/arm64/arm64/identcpu.c ============================================================================== --- head/sys/arm64/arm64/identcpu.c Wed Jul 1 15:02:56 2020 (r362836) +++ head/sys/arm64/arm64/identcpu.c Wed Jul 1 15:17:45 2020 (r362837) @@ -46,7 +46,6 @@ __FBSDID("$FreeBSD$"); #include #include -static int ident_lock; static void print_cpu_features(u_int cpu); static u_long parse_cpu_features_hwcap(u_int cpu); @@ -67,6 +66,8 @@ static int allow_idc = 1; SYSCTL_INT(_machdep_cache, OID_AUTO, allow_idc, CTLFLAG_RDTUN, &allow_idc, 0, "Allow optimizations based on the IDC cache bit"); +static void check_cpu_regs(u_int cpu); + /* * The default implementation of I-cache sync assumes we have an * aliasing cache until we know otherwise. @@ -1063,8 +1064,9 @@ identify_cpu_sysinit(void *dummy __unused) dic = (allow_dic != 0); idc = (allow_idc != 0); + CPU_FOREACH(cpu) { - print_cpu_features(cpu); + check_cpu_regs(cpu); hwcap = parse_cpu_features_hwcap(cpu); if (elf_hwcap == 0) elf_hwcap = hwcap; @@ -1096,8 +1098,18 @@ identify_cpu_sysinit(void *dummy __unused) install_undef_handler(true, user_mrs_handler); } -SYSINIT(identify_cpu, SI_SUB_SMP, SI_ORDER_ANY, identify_cpu_sysinit, NULL); +SYSINIT(identify_cpu, SI_SUB_CPU, SI_ORDER_ANY, identify_cpu_sysinit, NULL); +static void +cpu_features_sysinit(void *dummy __unused) +{ + u_int cpu; + + CPU_FOREACH(cpu) + print_cpu_features(cpu); +} +SYSINIT(cpu_features, SI_SUB_SMP, SI_ORDER_ANY, cpu_features_sysinit, NULL); + static u_long parse_cpu_features_hwcap(u_int cpu) { @@ -1468,7 +1480,8 @@ identify_cpu(void) if (impl_id == cpu_implementers[i].impl_id || cpu_implementers[i].impl_id == 0) { cpu_desc[cpu].cpu_impl = impl_id; - cpu_desc[cpu].cpu_impl_name = cpu_implementers[i].impl_name; + cpu_desc[cpu].cpu_impl_name = + cpu_implementers[i].impl_name; cpu_partsp = cpu_implementers[i].cpu_parts; break; } @@ -1505,77 +1518,68 @@ identify_cpu(void) cpu_desc[cpu].id_aa64mmfr2 = READ_SPECIALREG(id_aa64mmfr2_el1); cpu_desc[cpu].id_aa64pfr0 = READ_SPECIALREG(id_aa64pfr0_el1); cpu_desc[cpu].id_aa64pfr1 = READ_SPECIALREG(id_aa64pfr1_el1); +} - if (cpu != 0) { - /* - * This code must run on one cpu at a time, but we are - * not scheduling on the current core so implement a - * simple spinlock. - */ - while (atomic_cmpset_acq_int(&ident_lock, 0, 1) == 0) - __asm __volatile("wfe" ::: "memory"); +static void +check_cpu_regs(u_int cpu) +{ - switch (cpu_aff_levels) { - case 0: - if (CPU_AFF0(cpu_desc[cpu].mpidr) != - CPU_AFF0(cpu_desc[0].mpidr)) - cpu_aff_levels = 1; - /* FALLTHROUGH */ - case 1: - if (CPU_AFF1(cpu_desc[cpu].mpidr) != - CPU_AFF1(cpu_desc[0].mpidr)) - cpu_aff_levels = 2; - /* FALLTHROUGH */ - case 2: - if (CPU_AFF2(cpu_desc[cpu].mpidr) != - CPU_AFF2(cpu_desc[0].mpidr)) - cpu_aff_levels = 3; - /* FALLTHROUGH */ - case 3: - if (CPU_AFF3(cpu_desc[cpu].mpidr) != - CPU_AFF3(cpu_desc[0].mpidr)) - cpu_aff_levels = 4; - break; - } + switch (cpu_aff_levels) { + case 0: + if (CPU_AFF0(cpu_desc[cpu].mpidr) != + CPU_AFF0(cpu_desc[0].mpidr)) + cpu_aff_levels = 1; + /* FALLTHROUGH */ + case 1: + if (CPU_AFF1(cpu_desc[cpu].mpidr) != + CPU_AFF1(cpu_desc[0].mpidr)) + cpu_aff_levels = 2; + /* FALLTHROUGH */ + case 2: + if (CPU_AFF2(cpu_desc[cpu].mpidr) != + CPU_AFF2(cpu_desc[0].mpidr)) + cpu_aff_levels = 3; + /* FALLTHROUGH */ + case 3: + if (CPU_AFF3(cpu_desc[cpu].mpidr) != + CPU_AFF3(cpu_desc[0].mpidr)) + cpu_aff_levels = 4; + break; + } - if (cpu_desc[cpu].id_aa64afr0 != cpu_desc[0].id_aa64afr0) - cpu_print_regs |= PRINT_ID_AA64_AFR0; - if (cpu_desc[cpu].id_aa64afr1 != cpu_desc[0].id_aa64afr1) - cpu_print_regs |= PRINT_ID_AA64_AFR1; + if (cpu_desc[cpu].id_aa64afr0 != cpu_desc[0].id_aa64afr0) + cpu_print_regs |= PRINT_ID_AA64_AFR0; + if (cpu_desc[cpu].id_aa64afr1 != cpu_desc[0].id_aa64afr1) + cpu_print_regs |= PRINT_ID_AA64_AFR1; - if (cpu_desc[cpu].id_aa64dfr0 != cpu_desc[0].id_aa64dfr0) - cpu_print_regs |= PRINT_ID_AA64_DFR0; - if (cpu_desc[cpu].id_aa64dfr1 != cpu_desc[0].id_aa64dfr1) - cpu_print_regs |= PRINT_ID_AA64_DFR1; + if (cpu_desc[cpu].id_aa64dfr0 != cpu_desc[0].id_aa64dfr0) + cpu_print_regs |= PRINT_ID_AA64_DFR0; + if (cpu_desc[cpu].id_aa64dfr1 != cpu_desc[0].id_aa64dfr1) + cpu_print_regs |= PRINT_ID_AA64_DFR1; - if (cpu_desc[cpu].id_aa64isar0 != cpu_desc[0].id_aa64isar0) - cpu_print_regs |= PRINT_ID_AA64_ISAR0; - if (cpu_desc[cpu].id_aa64isar1 != cpu_desc[0].id_aa64isar1) - cpu_print_regs |= PRINT_ID_AA64_ISAR1; + if (cpu_desc[cpu].id_aa64isar0 != cpu_desc[0].id_aa64isar0) + cpu_print_regs |= PRINT_ID_AA64_ISAR0; + if (cpu_desc[cpu].id_aa64isar1 != cpu_desc[0].id_aa64isar1) + cpu_print_regs |= PRINT_ID_AA64_ISAR1; - if (cpu_desc[cpu].id_aa64mmfr0 != cpu_desc[0].id_aa64mmfr0) - cpu_print_regs |= PRINT_ID_AA64_MMFR0; - if (cpu_desc[cpu].id_aa64mmfr1 != cpu_desc[0].id_aa64mmfr1) - cpu_print_regs |= PRINT_ID_AA64_MMFR1; - if (cpu_desc[cpu].id_aa64mmfr2 != cpu_desc[0].id_aa64mmfr2) - cpu_print_regs |= PRINT_ID_AA64_MMFR2; + if (cpu_desc[cpu].id_aa64mmfr0 != cpu_desc[0].id_aa64mmfr0) + cpu_print_regs |= PRINT_ID_AA64_MMFR0; + if (cpu_desc[cpu].id_aa64mmfr1 != cpu_desc[0].id_aa64mmfr1) + cpu_print_regs |= PRINT_ID_AA64_MMFR1; + if (cpu_desc[cpu].id_aa64mmfr2 != cpu_desc[0].id_aa64mmfr2) + cpu_print_regs |= PRINT_ID_AA64_MMFR2; - if (cpu_desc[cpu].id_aa64pfr0 != cpu_desc[0].id_aa64pfr0) - cpu_print_regs |= PRINT_ID_AA64_PFR0; - if (cpu_desc[cpu].id_aa64pfr1 != cpu_desc[0].id_aa64pfr1) - cpu_print_regs |= PRINT_ID_AA64_PFR1; + if (cpu_desc[cpu].id_aa64pfr0 != cpu_desc[0].id_aa64pfr0) + cpu_print_regs |= PRINT_ID_AA64_PFR0; + if (cpu_desc[cpu].id_aa64pfr1 != cpu_desc[0].id_aa64pfr1) + cpu_print_regs |= PRINT_ID_AA64_PFR1; - if (cpu_desc[cpu].ctr != cpu_desc[0].ctr) { - /* - * If the cache type register is different we may - * have a different l1 cache type. - */ - identify_cache(cpu_desc[cpu].ctr); - cpu_print_regs |= PRINT_CTR_EL0; - } - - /* Wake up the other CPUs */ - atomic_store_rel_int(&ident_lock, 0); - __asm __volatile("sev" ::: "memory"); + if (cpu_desc[cpu].ctr != cpu_desc[0].ctr) { + /* + * If the cache type register is different we may + * have a different l1 cache type. + */ + identify_cache(cpu_desc[cpu].ctr); + cpu_print_regs |= PRINT_CTR_EL0; } } Modified: head/sys/arm64/arm64/mp_machdep.c ============================================================================== --- head/sys/arm64/arm64/mp_machdep.c Wed Jul 1 15:02:56 2020 (r362836) +++ head/sys/arm64/arm64/mp_machdep.c Wed Jul 1 15:17:45 2020 (r362837) @@ -212,6 +212,19 @@ init_secondary(uint64_t cpu) "mov x18, %0 \n" "msr tpidr_el1, %0" :: "r"(pcpup)); + /* + * Identify current CPU. This is necessary to setup + * affinity registers and to provide support for + * runtime chip identification. + * + * We need this before signalling the CPU is ready to + * let the boot CPU use the results. + */ + identify_cpu(); + + /* Ensure the stores in identify_cpu have completed */ + atomic_thread_fence_acq_rel(); + /* Signal the BSP and spin until it has released all APs. */ atomic_add_int(&aps_started, 1); while (!atomic_load_int(&aps_ready)) @@ -227,12 +240,6 @@ init_secondary(uint64_t cpu) ("pmap0 doesn't match cpu %ld's ttbr0", cpu)); pcpup->pc_curpmap = pmap0; - /* - * Identify current CPU. This is necessary to setup - * affinity registers and to provide support for - * runtime chip identification. - */ - identify_cpu(); install_cpu_errata(); intr_pic_init_secondary();