sG+r99mj89gzsbMmdpcVZDqKIxY2G9PtSuMV 4gG1dQjUo9SwvrefsBlhYFO7xSq7ANDL+5HYlWUF6ZsPGsT2X2H9uYukfmfleUbryhOEhJ 5B64bqeMuhAe53QsZcGvhr1J0dwJizlwNPxFd7dg+gVKSlCjkekSnGqkZf+dGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1765818005; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=K4Rq/rmmEvk+OlLRAmrRRdsf27t3rUwu4DFeJULg+zU=; b=I77U+TOQf2EMlvDP54GNNkNDt+zsOcbk6FNnV5963WzGKcitre0WXqPOAiJ6hzpsh9ztga zsY+YDylzIm5ce7VyFd0dL3L+qmRI0t16uBXEpTAWFxhlrilPaYFLIDwo4LmGZ2nab3On1 H5hllHD/dl79Pp1JY5aqCAzsE2EyBC8j3m7pZexVRtYeFDEoVaCebh5kS8WIs/ltISIE7P eVi2uwpdSKoS73FMTJRjUO8elZpL0WQ8XZEngMB6DBx7L0Fbbj6K7ZvkaCzkEFG/JUkIAe Q78SEshBQKsBduvTRBPTv0mhmJdbOEBoOaNwFjzP8LaQjxY8RnJb9pCztCWitQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1765818005; a=rsa-sha256; cv=none; b=E7vw8/d7JwFPhDtvU9fW2/wofsb8ILsDPyGn5N0ernQ53TCWjoKOgrJy+jIwZv1SmsQa0K +QFtgmizbTAo+NKtc2q7ka9eesVo6B2myF4uDJNcWhuenJ+B4nZdrkviZj5xMB+LS9dXJn TCFbTy4jQjE0nyaQKE2q6/4aELukh2XXBUrzoC3YQPBtAcA5q65A7M91vOCu1n7d7DQXXP DRR1vMhSrEvn31AsgggnSOgeeS4rvTi/eQV5MtQ5xYfwoxEqNUVNSw46k5WtlFJeh0Crbk 204Wqrs9JSZmHXrMawV6n+EK53gnK0pO6zHFu8YHSau08CR8Tfd//5Sgs5FnIw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4dVRBY0xbCzCnp for ; Mon, 15 Dec 2025 17:00:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id c6f0 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Mon, 15 Dec 2025 17:00:05 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Jessica Clarke Subject: git: f61eabc1f4cf - stable/14 - rtld-elf: Support IFUNCs on riscv List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-branches@freebsd.org Sender: owner-dev-commits-src-branches@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jrtc27 X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: f61eabc1f4cffaedb733262a8fd07f3011f6d827 Auto-Submitted: auto-generated Date: Mon, 15 Dec 2025 17:00:05 +0000 Message-Id: <69403e95.c6f0.5943e936@gitrepo.freebsd.org> The branch stable/14 has been updated by jrtc27: URL: https://cgit.FreeBSD.org/src/commit/?id=f61eabc1f4cffaedb733262a8fd07f3011f6d827 commit f61eabc1f4cffaedb733262a8fd07f3011f6d827 Author: Jessica Clarke AuthorDate: 2024-08-22 19:36:44 +0000 Commit: Jessica Clarke CommitDate: 2025-12-15 16:58:43 +0000 rtld-elf: Support IFUNCs on riscv GNU/Linux has historically had the following two resolver prototypes: 1. Elf_Addr(uint64_t, void *) 2. Elf_Addr(uint64_t, void *, void *) For the former, AT_HWCAP is passed in the first argument, and NULL in the second. For the latter, AT_HWCAP is still passed, and the second argument is a pointer to their home-grown __riscv_hwprobe function. Should they want to use the third argument in future, they'll have to introduce yet another prototype to allow for later expansion, and then all users will have to check whether the second argument is NULL to know if the third argument really exists. This is all rather silly and will surely prove fun in the face of type-checking CFI. Instead, be like arm64 and just define all 8 possible general purpose register arguments up front. To naive source code that forgets non-Linux OSes exist this will be compatible with prototype 1 above, since the second argument will be 0 and it won't look further (though should we start using the second argument for something that wouldn't be true any more and it might think it's __riscv_hwprobe, but that incompatibility is one we can defer committing to, and can choose to never adopt). Until the standard interface for querying extension information[1] is settled and implemented in FreeBSD there's not much you can do in a resolver other than use HWCAP_ISA_B, but this gets the infrastructure in place for when that day comes. [1] https://github.com/riscv-non-isa/riscv-c-api-doc/pull/74 Reviewed by: kib, mhorne MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D46278 (cherry picked from commit 729d2b16b74fa5207a12aa1de190bd930432810e) --- libexec/rtld-elf/riscv/reloc.c | 140 ++++++++++++++++++++++++++++------ libexec/rtld-elf/riscv/rtld_machdep.h | 5 +- 2 files changed, 122 insertions(+), 23 deletions(-) diff --git a/libexec/rtld-elf/riscv/reloc.c b/libexec/rtld-elf/riscv/reloc.c index 2b254803fabf..8dd225e5a920 100644 --- a/libexec/rtld-elf/riscv/reloc.c +++ b/libexec/rtld-elf/riscv/reloc.c @@ -153,10 +153,20 @@ reloc_plt(Obj_Entry *obj, int flags __unused, RtldLockState *lockstate __unused) for (rela = obj->pltrela; rela < relalim; rela++) { Elf_Addr *where; - assert(ELF_R_TYPE(rela->r_info) == R_RISCV_JUMP_SLOT); - where = (Elf_Addr *)(obj->relocbase + rela->r_offset); - *where += (Elf_Addr)obj->relocbase; + + switch (ELF_R_TYPE(rela->r_info)) { + case R_RISCV_JUMP_SLOT: + *where += (Elf_Addr)obj->relocbase; + break; + case R_RISCV_IRELATIVE: + obj->irelative = true; + break; + default: + _rtld_error("Unknown relocation type %u in PLT", + (unsigned int)ELF_R_TYPE(rela->r_info)); + return (-1); + } } return (0); @@ -188,6 +198,11 @@ reloc_jmpslots(Obj_Entry *obj, int flags, RtldLockState *lockstate) return (-1); } + if (ELF_ST_TYPE(def->st_info) == STT_GNU_IFUNC) { + obj->gnu_ifunc = true; + continue; + } + *where = (Elf_Addr)(defobj->relocbase + def->st_value); break; default: @@ -200,30 +215,89 @@ reloc_jmpslots(Obj_Entry *obj, int flags, RtldLockState *lockstate) return (0); } +static void +reloc_iresolve_one(Obj_Entry *obj, const Elf_Rela *rela, + RtldLockState *lockstate) +{ + Elf_Addr *where, target, *ptr; + + ptr = (Elf_Addr *)(obj->relocbase + rela->r_addend); + where = (Elf_Addr *)(obj->relocbase + rela->r_offset); + lock_release(rtld_bind_lock, lockstate); + target = call_ifunc_resolver(ptr); + wlock_acquire(rtld_bind_lock, lockstate); + *where = target; +} + int -reloc_iresolve(Obj_Entry *obj __unused, - struct Struct_RtldLockState *lockstate __unused) +reloc_iresolve(Obj_Entry *obj, struct Struct_RtldLockState *lockstate) { + const Elf_Rela *relalim; + const Elf_Rela *rela; + + if (!obj->irelative) + return (0); - /* XXX not implemented */ + obj->irelative = false; + relalim = (const Elf_Rela *)((const char *)obj->pltrela + + obj->pltrelasize); + for (rela = obj->pltrela; rela < relalim; rela++) { + if (ELF_R_TYPE(rela->r_info) == R_RISCV_IRELATIVE) + reloc_iresolve_one(obj, rela, lockstate); + } return (0); } int -reloc_iresolve_nonplt(Obj_Entry *obj __unused, - struct Struct_RtldLockState *lockstate __unused) +reloc_iresolve_nonplt(Obj_Entry *obj, struct Struct_RtldLockState *lockstate) { + const Elf_Rela *relalim; + const Elf_Rela *rela; - /* XXX not implemented */ + if (!obj->irelative_nonplt) + return (0); + + obj->irelative_nonplt = false; + relalim = (const Elf_Rela *)((const char *)obj->rela + obj->relasize); + for (rela = obj->rela; rela < relalim; rela++) { + if (ELF_R_TYPE(rela->r_info) == R_RISCV_IRELATIVE) + reloc_iresolve_one(obj, rela, lockstate); + } return (0); } int -reloc_gnu_ifunc(Obj_Entry *obj __unused, int flags __unused, - struct Struct_RtldLockState *lockstate __unused) +reloc_gnu_ifunc(Obj_Entry *obj, int flags, + struct Struct_RtldLockState *lockstate) { + const Elf_Rela *relalim; + const Elf_Rela *rela; + Elf_Addr *where, target; + const Elf_Sym *def; + const Obj_Entry *defobj; + + if (!obj->gnu_ifunc) + return (0); - /* XXX not implemented */ + relalim = (const Elf_Rela *)((const char *)obj->pltrela + obj->pltrelasize); + for (rela = obj->pltrela; rela < relalim; rela++) { + if (ELF_R_TYPE(rela->r_info) == R_RISCV_JUMP_SLOT) { + where = (Elf_Addr *)(obj->relocbase + rela->r_offset); + def = find_symdef(ELF_R_SYM(rela->r_info), obj, &defobj, + SYMLOOK_IN_PLT | flags, NULL, lockstate); + if (def == NULL) + return (-1); + if (ELF_ST_TYPE(def->st_info) != STT_GNU_IFUNC) + continue; + + lock_release(rtld_bind_lock, lockstate); + target = (Elf_Addr)rtld_resolve_ifunc(defobj, def); + wlock_acquire(rtld_bind_lock, lockstate); + reloc_jmpslot(where, target, defobj, obj, + (const Elf_Rel *)rela); + } + } + obj->gnu_ifunc = false; return (0); } @@ -233,7 +307,8 @@ reloc_jmpslot(Elf_Addr *where, Elf_Addr target, const Elf_Rel *rel) { - assert(ELF_R_TYPE(rel->r_info) == R_RISCV_JUMP_SLOT); + assert(ELF_R_TYPE(rel->r_info) == R_RISCV_JUMP_SLOT || + ELF_R_TYPE(rel->r_info) == R_RISCV_IRELATIVE); if (*where != target && !ld_bind_not) *where = target; @@ -252,13 +327,9 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int flags, const Elf_Rela *rela; const Elf_Sym *def; SymCache *cache; - Elf_Addr *where; + Elf_Addr *where, symval; unsigned long symnum; - if ((flags & SYMLOOK_IFUNC) != 0) - /* XXX not implemented */ - return (0); - /* * The dynamic loader may be called from a thread, we have * limited amounts of stack available so we cannot use alloca(). @@ -286,8 +357,27 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int flags, if (def == NULL) return (-1); - *where = (Elf_Addr)(defobj->relocbase + def->st_value + - rela->r_addend); + /* + * If symbol is IFUNC, only perform relocation + * when caller allowed it by passing + * SYMLOOK_IFUNC flag. Skip the relocations + * otherwise. + */ + if (ELF_ST_TYPE(def->st_info) == STT_GNU_IFUNC) { + if ((flags & SYMLOOK_IFUNC) == 0) { + obj->non_plt_gnu_ifunc = true; + continue; + } + symval = (Elf_Addr)rtld_resolve_ifunc(defobj, + def); + } else { + if ((flags & SYMLOOK_IFUNC) != 0) + continue; + symval = (Elf_Addr)(defobj->relocbase + + def->st_value); + } + + *where = symval + rela->r_addend; break; case R_RISCV_TLS_DTPMOD64: def = find_symdef(symnum, obj, &defobj, flags, cache, @@ -366,6 +456,9 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int flags, case R_RISCV_RELATIVE: *where = (Elf_Addr)(obj->relocbase + rela->r_addend); break; + case R_RISCV_IRELATIVE: + obj->irelative_nonplt = true; + break; default: rtld_printf("%s: Unhandled relocation %lu\n", obj->path, ELF_R_TYPE(rela->r_info)); @@ -376,10 +469,13 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int flags, return (0); } +unsigned long elf_hwcap; + void -ifunc_init(Elf_Auxinfo *aux_info[__min_size(AT_COUNT)] __unused) +ifunc_init(Elf_Auxinfo *aux_info[__min_size(AT_COUNT)]) { - + if (aux_info[AT_HWCAP] != NULL) + elf_hwcap = aux_info[AT_HWCAP]->a_un.a_val; } void diff --git a/libexec/rtld-elf/riscv/rtld_machdep.h b/libexec/rtld-elf/riscv/rtld_machdep.h index 381eba530c2d..d80843d2820a 100644 --- a/libexec/rtld-elf/riscv/rtld_machdep.h +++ b/libexec/rtld-elf/riscv/rtld_machdep.h @@ -77,8 +77,11 @@ Elf_Addr reloc_jmpslot(Elf_Addr *where, Elf_Addr target, __asm __volatile("mv gp, %0" :: "r"(old1)); \ }) +extern unsigned long elf_hwcap; #define call_ifunc_resolver(ptr) \ - (((Elf_Addr (*)(void))ptr)()) + (((Elf_Addr (*)(unsigned long, unsigned long, unsigned long, \ + unsigned long, unsigned long, unsigned long, unsigned long, \ + unsigned long))ptr)(elf_hwcap, 0, 0, 0, 0, 0, 0, 0)) /* * TLS