From owner-svn-src-all@freebsd.org Sun Jul 5 14:38:24 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 973F136D009; Sun, 5 Jul 2020 14:38:24 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B0BC43YMlz41hq; Sun, 5 Jul 2020 14:38:24 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 591BCEF27; Sun, 5 Jul 2020 14:38:24 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 065EcO1V020762; Sun, 5 Jul 2020 14:38:24 GMT (envelope-from andrew@FreeBSD.org) Received: (from andrew@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 065EcNxl020756; Sun, 5 Jul 2020 14:38:23 GMT (envelope-from andrew@FreeBSD.org) Message-Id: <202007051438.065EcNxl020756@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: andrew set sender to andrew@FreeBSD.org using -f From: Andrew Turner Date: Sun, 5 Jul 2020 14:38:23 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r362944 - in head/sys: amd64/amd64 arm64/arm64 kern sys X-SVN-Group: head X-SVN-Commit-Author: andrew X-SVN-Commit-Paths: in head/sys: amd64/amd64 arm64/arm64 kern sys X-SVN-Commit-Revision: 362944 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Jul 2020 14:38:24 -0000 Author: andrew Date: Sun Jul 5 14:38:22 2020 New Revision: 362944 URL: https://svnweb.freebsd.org/changeset/base/362944 Log: Rerun kernel ifunc resolvers after all CPUs have started On architectures that use RELA relocations it is safe to rerun the ifunc resolvers on after all CPUs have started, but while they are sill parked. On arm64 with big.LITTLE this is needed as some SoCs have shipped with different ID register values the big and little clusters meaning we were unable to rely on the register values from the boot CPU. Add support for rerunning the resolvers on arm64 and amd64 as these are both RELA using architectures. Reviewed by: kib Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D25455 Modified: head/sys/amd64/amd64/elf_machdep.c head/sys/amd64/amd64/machdep.c head/sys/arm64/arm64/elf_machdep.c head/sys/arm64/arm64/machdep.c head/sys/kern/link_elf.c head/sys/sys/linker.h Modified: head/sys/amd64/amd64/elf_machdep.c ============================================================================== --- head/sys/amd64/amd64/elf_machdep.c Sun Jul 5 13:15:13 2020 (r362943) +++ head/sys/amd64/amd64/elf_machdep.c Sun Jul 5 14:38:22 2020 (r362944) @@ -186,7 +186,7 @@ elf_is_ifunc_reloc(Elf_Size r_info) /* Process one elf relocation with addend. */ static int elf_reloc_internal(linker_file_t lf, Elf_Addr relocbase, const void *data, - int type, elf_lookup_fn lookup) + int type, bool late_ifunc, elf_lookup_fn lookup) { Elf64_Addr *where, val; Elf32_Addr *where32, val32; @@ -226,6 +226,13 @@ elf_reloc_internal(linker_file_t lf, Elf_Addr relocbas panic("unknown reloc type %d\n", type); } + if (late_ifunc) { + KASSERT(type == ELF_RELOC_RELA, + ("Only RELA ifunc relocations are supported")); + if (rtype != R_X86_64_IRELATIVE) + return (0); + } + switch (rtype) { case R_X86_64_NONE: /* none */ break; @@ -305,7 +312,7 @@ elf_reloc(linker_file_t lf, Elf_Addr relocbase, const elf_lookup_fn lookup) { - return (elf_reloc_internal(lf, relocbase, data, type, lookup)); + return (elf_reloc_internal(lf, relocbase, data, type, false, lookup)); } int @@ -313,7 +320,15 @@ elf_reloc_local(linker_file_t lf, Elf_Addr relocbase, int type, elf_lookup_fn lookup) { - return (elf_reloc_internal(lf, relocbase, data, type, lookup)); + return (elf_reloc_internal(lf, relocbase, data, type, false, lookup)); +} + +int +elf_reloc_late(linker_file_t lf, Elf_Addr relocbase, const void *data, + int type, elf_lookup_fn lookup) +{ + + return (elf_reloc_internal(lf, relocbase, data, type, true, lookup)); } int Modified: head/sys/amd64/amd64/machdep.c ============================================================================== --- head/sys/amd64/amd64/machdep.c Sun Jul 5 13:15:13 2020 (r362943) +++ head/sys/amd64/amd64/machdep.c Sun Jul 5 14:38:22 2020 (r362944) @@ -320,6 +320,13 @@ cpu_startup(dummy) cpu_setregs(); } +static void +late_ifunc_resolve(void *dummy __unused) +{ + link_elf_late_ireloc(); +} +SYSINIT(late_ifunc_resolve, SI_SUB_CPU, SI_ORDER_ANY, late_ifunc_resolve, NULL); + /* * Send an interrupt to process. * Modified: head/sys/arm64/arm64/elf_machdep.c ============================================================================== --- head/sys/arm64/arm64/elf_machdep.c Sun Jul 5 13:15:13 2020 (r362943) +++ head/sys/arm64/arm64/elf_machdep.c Sun Jul 5 14:38:22 2020 (r362944) @@ -143,8 +143,10 @@ reloc_instr_imm(Elf32_Addr *where, Elf_Addr val, u_int */ static int elf_reloc_internal(linker_file_t lf, Elf_Addr relocbase, const void *data, - int type, int local, elf_lookup_fn lookup) + int type, int flags, elf_lookup_fn lookup) { +#define ARM64_ELF_RELOC_LOCAL (1 << 0) +#define ARM64_ELF_RELOC_LATE_IFUNC (1 << 1) Elf_Addr *where, addr, addend, val; Elf_Word rtype, symidx; const Elf_Rel *rel; @@ -170,7 +172,14 @@ elf_reloc_internal(linker_file_t lf, Elf_Addr relocbas panic("unknown reloc type %d\n", type); } - if (local) { + if ((flags & ARM64_ELF_RELOC_LATE_IFUNC) != 0) { + KASSERT(type == ELF_RELOC_RELA, + ("Only RELA ifunc relocations are supported")); + if (rtype != R_AARCH64_IRELATIVE) + return (0); + } + + if ((flags & ARM64_ELF_RELOC_LOCAL) != 0) { if (rtype == R_AARCH64_RELATIVE) *where = elf_relocaddr(lf, relocbase + addend); return (0); @@ -229,7 +238,8 @@ elf_reloc_local(linker_file_t lf, Elf_Addr relocbase, int type, elf_lookup_fn lookup) { - return (elf_reloc_internal(lf, relocbase, data, type, 1, lookup)); + return (elf_reloc_internal(lf, relocbase, data, type, + ARM64_ELF_RELOC_LOCAL, lookup)); } /* Process one elf relocation with addend. */ @@ -239,6 +249,15 @@ elf_reloc(linker_file_t lf, Elf_Addr relocbase, const { return (elf_reloc_internal(lf, relocbase, data, type, 0, lookup)); +} + +int +elf_reloc_late(linker_file_t lf, Elf_Addr relocbase, const void *data, + int type, elf_lookup_fn lookup) +{ + + return (elf_reloc_internal(lf, relocbase, data, type, + ARM64_ELF_RELOC_LATE_IFUNC, lookup)); } int Modified: head/sys/arm64/arm64/machdep.c ============================================================================== --- head/sys/arm64/arm64/machdep.c Sun Jul 5 13:15:13 2020 (r362943) +++ head/sys/arm64/arm64/machdep.c Sun Jul 5 14:38:22 2020 (r362944) @@ -181,6 +181,13 @@ cpu_startup(void *dummy) SYSINIT(cpu, SI_SUB_CPU, SI_ORDER_FIRST, cpu_startup, NULL); +static void +late_ifunc_resolve(void *dummy __unused) +{ + link_elf_late_ireloc(); +} +SYSINIT(late_ifunc_resolve, SI_SUB_CPU, SI_ORDER_ANY, late_ifunc_resolve, NULL); + int cpu_idle_wakeup(int cpu) { Modified: head/sys/kern/link_elf.c ============================================================================== --- head/sys/kern/link_elf.c Sun Jul 5 13:15:13 2020 (r362943) +++ head/sys/kern/link_elf.c Sun Jul 5 14:38:22 2020 (r362944) @@ -1925,4 +1925,18 @@ link_elf_ireloc(caddr_t kmdp) link_elf_preload_parse_symbols(ef); relocate_file1(ef, elf_lookup_ifunc, elf_reloc, true); } + +#if defined(__aarch64__) || defined(__amd64__) +void +link_elf_late_ireloc(void) +{ + elf_file_t ef; + + KASSERT(linker_kernel_file != NULL, + ("link_elf_late_ireloc: No kernel linker file found")); + ef = (elf_file_t)linker_kernel_file; + + relocate_file1(ef, elf_lookup_ifunc, elf_reloc_late, true); +} +#endif #endif Modified: head/sys/sys/linker.h ============================================================================== --- head/sys/sys/linker.h Sun Jul 5 13:15:13 2020 (r362943) +++ head/sys/sys/linker.h Sun Jul 5 14:38:22 2020 (r362944) @@ -289,6 +289,12 @@ const Elf_Sym *elf_get_sym(linker_file_t _lf, Elf_Size const char *elf_get_symname(linker_file_t _lf, Elf_Size _symidx); void link_elf_ireloc(caddr_t kmdp); +#if defined(__aarch64__) || defined(__amd64__) +int elf_reloc_late(linker_file_t _lf, Elf_Addr base, const void *_rel, + int _type, elf_lookup_fn _lu); +void link_elf_late_ireloc(void); +#endif + typedef struct linker_ctf { const uint8_t *ctftab; /* Decompressed CTF data. */ int ctfcnt; /* Number of CTF data bytes. */