From nobody Fri May 29 22:08:22 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4gRyD71t9Lz6g9t2 for ; Fri, 29 May 2026 22:08:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R13" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4gRyD66f7Pz4GXY for ; Fri, 29 May 2026 22:08:22 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1780092503; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=8txWUtuH2lPM5LARM9Us4aN/zFRBZyBzC8LG0o9i/Hc=; b=ZvCI0Ekeleq9pdpVNR6MykKmlrU5zl/nR/nctTyAnfDzQLdPV5vQu/HS8xZbw5oD3fu6nc mcl863LhRf5XxeIlLTBHTGd9fFHZB0/dAP2l1h1w6jTxmsZ16EW5+2xMVad0bEvG0pEneH 4FcTJdeGFdLvbWLiJsr3lQpApKS+9GlOyEd/BSq2B/j//cEEc+CO773hu0Npm1bN+KWRvc 8bpENG+gQ0jghf6dVJA8hge7rrwQVUhJ4i6noyuqWwpR6DlallE11wYPJ/dsXxwoP44no1 MNPurDQHvtFbE9XqFcQsffT6zpNcjPdUdEfMcZAXmA5OVtX33BkjBYNTmvgcJg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1780092502; a=rsa-sha256; cv=none; b=H/yhmSS3af3TlwwyuA7UO8WuuEBV6FGQa3NJ+da0xfN5TtttxotHuE8csGPGk0Te5JQu88 Pii7JnVnkqk3tz9e8jAJcw12YF1mglJWuU6t76rxCcN4AAkR2eEyG58nVBYrFQxRnFEflO qLt7XMRWWU95jJ+RQg+IB0gcsqRpiKTm5uhxfd7z8xaiXN123zsqn9D2/F+cVG5qWKmLpR dD6UAM/d9SMyZ9bpJ8K3/pN1vuHJ2i9QxzfOoja7JH3YpmWj4hFELzT2HM0BdiChM7A32o VpBp32iV4noLEcvE24hsCwokrLYTdA84tmXU5bIQI/VJskXoXegd0scX0zChCA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1780092502; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=8txWUtuH2lPM5LARM9Us4aN/zFRBZyBzC8LG0o9i/Hc=; b=xcQoXhKRCGoPhQNwIr16HAwMTZrQOldZ/iZ+FDduc2BiZSvKnNCLjce5KQwXGrCfxzeV13 1GASI9TUa00W9YsbV9c2gCMN7G6TYXGu0Fcidu3E9I2Aw7piv93bPEelSayQHwAj/kqYnV RLxqptcSn6zzaS0Wqq/NAJg0hLlTmJkDX2dGRiHt83IuBxwBqEgRde4p0vuvDo22CT9LhI x5nhPgrwQIwNsycrSeg9Ad0GIWD1/MQr4DLH7GLJSmKRGGoFTttX3Ts4Mc0+VI5/VKb+Ge tYIcEE4UsZOJFSLB5aEWrEFK7HyQKoRGlrU3T51xiD/1/RBYk/7/Rs4lYVPWMg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4gRyD66FdLzs37 for ; Fri, 29 May 2026 22:08:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 3a970 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Fri, 29 May 2026 22:08:22 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: cb62bc13b2a5 - stable/15 - kinst/arm64: Handle an additional PC-relative instruction List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: cb62bc13b2a5b85ddfc9df539689744a16a082e1 Auto-Submitted: auto-generated Date: Fri, 29 May 2026 22:08:22 +0000 Message-Id: <6a1a0e56.3a970.3894ce90@gitrepo.freebsd.org> The branch stable/15 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=cb62bc13b2a5b85ddfc9df539689744a16a082e1 commit cb62bc13b2a5b85ddfc9df539689744a16a082e1 Author: Mark Johnston AuthorDate: 2026-05-20 14:49:41 +0000 Commit: Mark Johnston CommitDate: 2026-05-29 19:46:47 +0000 kinst/arm64: Handle an additional PC-relative instruction "ldr , " loads a value from a literal memory address into a register. It's PC-relative and so cannot be directly implemented using the trampoline mechanism. Unfortunately, on arm64 it can't easily be emulated either since the return-to-EL1 handler does not restore callee-saved registers, so like adr/adrp, we simply don't handle it. These instructions are fairly rare in an arm64 kernel. While here, refactor the code so that all instruction decoding is done in one place: introduce an enum type which characterizes the instruction type, add a helper to map instructions to enum values, and store the corresponding enum value in the probe description. Reviewed by: christos MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D56988 (cherry picked from commit e6be6dedeea1e6d2e5206e1e7422e2d556a6da0c) --- sys/cddl/dev/kinst/aarch64/kinst_isa.c | 121 +++++++++++++++------------------ sys/cddl/dev/kinst/aarch64/kinst_isa.h | 15 +++- 2 files changed, 69 insertions(+), 67 deletions(-) diff --git a/sys/cddl/dev/kinst/aarch64/kinst_isa.c b/sys/cddl/dev/kinst/aarch64/kinst_isa.c index 1ccfe20b8dcb..d9a8fd0276f2 100644 --- a/sys/cddl/dev/kinst/aarch64/kinst_isa.c +++ b/sys/cddl/dev/kinst/aarch64/kinst_isa.c @@ -18,6 +18,30 @@ DPCPU_DEFINE_STATIC(struct kinst_cpu_state, kinst_state); +static enum kinst_instr +kinst_instr_type(kinst_patchval_t instr) +{ + if (((instr >> 22) & 0xff) == 0b00100001) + return (KINST_INSTR_LDX); + else if (((instr >> 22) & 0xff) == 0b00100000) + return (KINST_INSTR_STX); + if (((instr >> 24) & 0x1f) == 0b10000) + return (KINST_INSTR_ADR); + else if (((instr >> 26) & 0x3f) == 0b000101) + return (KINST_INSTR_B); + else if (((instr >> 24) & 0xff) == 0b01010100) + return (KINST_INSTR_BCOND); + else if (((instr >> 26) & 0x3f) == 0b100101) + return (KINST_INSTR_BL); + else if (((instr >> 25) & 0x3f) == 0b011010) + return (KINST_INSTR_CBZ); + else if (((instr >> 25) & 0x3f) == 0b011011) + return (KINST_INSTR_TBZ); + else if (((instr >> 24) & 0xbf) == 0b11000) + return (KINST_INSTR_LDR_LITERAL); + return (KINST_INSTR_COMMON); +} + static void kinst_emulate(struct trapframe *frame, const struct kinst_probe *kp) { @@ -26,8 +50,8 @@ kinst_emulate(struct trapframe *frame, const struct kinst_probe *kp) uint8_t cond, reg, bitpos; bool res; - if (((instr >> 24) & 0x1f) == 0b10000) { - /* adr/adrp */ + switch (kp->kp_md.kp_type) { + case KINST_INSTR_ADR: reg = instr & 0x1f; imm = (instr >> 29) & 0x3; imm |= ((instr >> 5) & 0x0007ffff) << 2; @@ -44,14 +68,14 @@ kinst_emulate(struct trapframe *frame, const struct kinst_probe *kp) frame->tf_x[reg] = (frame->tf_elr & ~0xfff) + imm; } frame->tf_elr += INSN_SIZE; - } else if (((instr >> 26) & 0x3f) == 0b000101) { - /* b */ + break; + case KINST_INSTR_B: imm = instr & 0x03ffffff; if (imm & 0x0000000002000000) imm |= 0xfffffffffe000000; frame->tf_elr += imm << 2; - } else if (((instr >> 24) & 0xff) == 0b01010100) { - /* b.cond */ + break; + case KINST_INSTR_BCOND: imm = (instr >> 5) & 0x0007ffff; if (imm & 0x0000000000040000) imm |= 0xfffffffffffc0000; @@ -92,15 +116,15 @@ kinst_emulate(struct trapframe *frame, const struct kinst_probe *kp) frame->tf_elr += imm << 2; else frame->tf_elr += INSN_SIZE; - } else if (((instr >> 26) & 0x3f) == 0b100101) { - /* bl */ + break; + case KINST_INSTR_BL: imm = instr & 0x03ffffff; if (imm & 0x0000000002000000) imm |= 0xfffffffffe000000; frame->tf_lr = frame->tf_elr + INSN_SIZE; frame->tf_elr += imm << 2; - } else if (((instr >> 25) & 0x3f) == 0b011010) { - /* cbnz/cbz */ + break; + case KINST_INSTR_CBZ: cond = (instr >> 24) & 0x1; reg = instr & 0x1f; imm = (instr >> 5) & 0x0007ffff; @@ -114,8 +138,8 @@ kinst_emulate(struct trapframe *frame, const struct kinst_probe *kp) frame->tf_elr += imm << 2; else frame->tf_elr += INSN_SIZE; - } else if (((instr >> 25) & 0x3f) == 0b011011) { - /* tbnz/tbz */ + break; + case KINST_INSTR_TBZ: cond = (instr >> 24) & 0x1; reg = instr & 0x1f; bitpos = (instr >> 19) & 0x1f; @@ -131,6 +155,9 @@ kinst_emulate(struct trapframe *frame, const struct kinst_probe *kp) frame->tf_elr += imm << 2; else frame->tf_elr += INSN_SIZE; + break; + default: + __assert_unreachable(); } } @@ -211,7 +238,7 @@ kinst_invop(uintptr_t addr, struct trapframe *frame, uintptr_t scratch) dtrace_probe(kp->kp_id, 0, 0, 0, 0, 0); cpu->cpu_dtrace_caller = 0; - if (kp->kp_md.emulate) { + if (kp->kp_md.kp_type != KINST_INSTR_COMMON) { kinst_emulate(frame, kp); } else { ks->state = KINST_PROBE_FIRED; @@ -245,50 +272,6 @@ kinst_patch_tracepoint(struct kinst_probe *kp, kinst_patchval_t val) cpu_icache_sync_range(kp->kp_patchpoint, INSN_SIZE); } -static void -kinst_instr_dissect(struct kinst_probe *kp) -{ - struct kinst_probe_md *kpmd; - kinst_patchval_t instr = kp->kp_savedval; - - kpmd = &kp->kp_md; - kpmd->emulate = false; - - if (((instr >> 24) & 0x1f) == 0b10000) - kpmd->emulate = true; /* adr/adrp */ - else if (((instr >> 26) & 0x3f) == 0b000101) - kpmd->emulate = true; /* b */ - else if (((instr >> 24) & 0xff) == 0b01010100) - kpmd->emulate = true; /* b.cond */ - else if (((instr >> 26) & 0x3f) == 0b100101) - kpmd->emulate = true; /* bl */ - else if (((instr >> 25) & 0x3f) == 0b011010) - kpmd->emulate = true; /* cbnz/cbz */ - else if (((instr >> 25) & 0x3f) == 0b011011) - kpmd->emulate = true; /* tbnz/tbz */ - - if (!kpmd->emulate) - kinst_trampoline_populate(kp); -} - -static bool -kinst_instr_ldx(kinst_patchval_t instr) -{ - if (((instr >> 22) & 0xff) == 0b00100001) - return (true); - - return (false); -} - -static bool -kinst_instr_stx(kinst_patchval_t instr) -{ - if (((instr >> 22) & 0xff) == 0b00100000) - return (true); - - return (false); -} - int kinst_make_probe(linker_file_t lf, int symindx, linker_symval_t *symval, void *opaque) @@ -359,6 +342,8 @@ kinst_make_probe(linker_file_t lf, int symindx, linker_symval_t *symval, ldxstx_block = false; for (n = 0; instr < limit; instr++) { + enum kinst_instr type; + off = (int)((uint8_t *)instr - (uint8_t *)symval->value); /* @@ -366,9 +351,10 @@ kinst_make_probe(linker_file_t lf, int symindx, linker_symval_t *symval, * breakpoint is placed in a LDX/STX block, we violate the * operation and the loop might fail. */ - if (kinst_instr_ldx(*instr)) + type = kinst_instr_type(*instr); + if (type == KINST_INSTR_LDX) ldxstx_block = true; - else if (kinst_instr_stx(*instr)) { + else if (type == KINST_INSTR_STX) { ldxstx_block = false; continue; } @@ -376,13 +362,14 @@ kinst_make_probe(linker_file_t lf, int symindx, linker_symval_t *symval, continue; /* - * XXX: Skip ADR and ADRP instructions. The arm64 exception - * handler has a micro-optimization where it doesn't restore - * callee-saved registers when returning from exceptions in - * EL1. This results in a panic when the kinst emulation code - * modifies one of those registers. + * XXX: The arm64 exception handler has a micro-optimization + * where it doesn't restore callee-saved registers when + * returning from exceptions in EL1. As a result, instruction + * emulation doesn't work if a (callee-saved) register is + * modified. Hence, exclude the position-dependent ADR/ADRP and + * LDR instructions. */ - if (((*instr >> 24) & 0x1f) == 0b10000) + if (type == KINST_INSTR_ADR || type == KINST_INSTR_LDR_LITERAL) continue; if (pd->kpd_off != -1 && off != pd->kpd_off) @@ -408,12 +395,14 @@ kinst_make_probe(linker_file_t lf, int symindx, linker_symval_t *symval, kp->kp_patchpoint = instr; kp->kp_savedval = *instr; kp->kp_patchval = KINST_PATCHVAL; + kp->kp_md.kp_type = type; if ((kp->kp_tramp = kinst_trampoline_alloc(M_WAITOK)) == NULL) { KINST_LOG("cannot allocate trampoline for %p", instr); return (ENOMEM); } + if (kp->kp_md.kp_type == KINST_INSTR_COMMON) + kinst_trampoline_populate(kp); - kinst_instr_dissect(kp); kinst_probe_create(kp, lf); } if (ldxstx_block) diff --git a/sys/cddl/dev/kinst/aarch64/kinst_isa.h b/sys/cddl/dev/kinst/aarch64/kinst_isa.h index 7e1fd8d123e9..39cf6d49290a 100644 --- a/sys/cddl/dev/kinst/aarch64/kinst_isa.h +++ b/sys/cddl/dev/kinst/aarch64/kinst_isa.h @@ -19,8 +19,21 @@ typedef uint32_t kinst_patchval_t; +enum kinst_instr { + KINST_INSTR_ADR, /* adr/adrp */ + KINST_INSTR_B, + KINST_INSTR_BCOND, + KINST_INSTR_BL, + KINST_INSTR_CBZ, /* cbz/cbnz */ + KINST_INSTR_TBZ, /* tbnz/tbz */ + KINST_INSTR_LDR_LITERAL, + KINST_INSTR_LDX, + KINST_INSTR_STX, + KINST_INSTR_COMMON, +}; + struct kinst_probe_md { - bool emulate; /* emulate in sw */ + enum kinst_instr kp_type; }; #endif /* _KINST_ISA_H_ */