From nobody Tue Feb 10 15:43:34 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4f9Qny5nYlz6RWwD for ; Tue, 10 Feb 2026 15:43:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R13" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4f9Qny329cz3M2v for ; Tue, 10 Feb 2026 15:43:34 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1770738214; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=khwu9DUTlAQ1T4DsluWfe7Gru7DIVEEZYV7BhGeh2Qs=; b=i4IC5TtpX2gdawBYfPlO2IssfvjqKY+Ycb7+a+0cy448brt2fbWraxxNHOw1ewxFdfsJgf +onLSAdC9E40X1ZgkHwDkQQuR5pDCyuxb81ujjafcZp3D2HjAKoETALPshhqC9DK4NI/Zm /feE9NRoQ/zGit/jYaLs/2B0KhPBNUinCyFjYKt3U3JgnWi8JT8CdfUglJa8gA1jRzyVC7 nZHCxmIxB02dByx8mVjCVGKtBtfrUPibSEWtVNGlvxEjG/0M+uQx4R3yoCedSj9dWbZZXE AiLRdyXDwcR+mcTscDmZlOtITZoyQpXP0l0Ypjkf8gLH3MVgOIv/2UL70fnt0g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1770738214; a=rsa-sha256; cv=none; b=ayq820wRI2CAhwZKQslyzO6zcH7xCs0kq7jp+3p/MCys4ef6BcmOBpEq8NBLGits7U5TuY YN6fBeNNndcdvZ3MeaWBwxIcWEXHdp4QDU9AMtARS1Rxf1/DuNO6zam06w3tfmI3QcgpDs UrG0ORuXbW6wcPCNwtkvT4R5d41H90CxgPlJUQ3hY4v/ioRp2cJpQ/ev54nMhZR7n88m13 BtDTe7JznZR05rsw2LG1icCQ6tFr5trYH8/TowembSF9Hfl5/MwHjmSDIYlCgVhQoDzb68 897N7CMA9iqSIMnxs8m6rDNfav0CTQdph47RDJ2jp68sFXHhb7PygmLrkGon9g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1770738214; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=khwu9DUTlAQ1T4DsluWfe7Gru7DIVEEZYV7BhGeh2Qs=; b=QebiUFg/va8T2Grkrry89x6FDBFl5IF1tJG807PpyQnFJxcwUXHQxVZE23bYOACRCIw0dl jkWDKn/qasrOV8OfG/WkYMr5gizMCJ7wfqxn9D4Tc8mi/9C+6lSV5v0aOovVmJj0wXovAt hx7xXphTJiqB1662WpZNyme49GYrrXHGxL16PSs2uINmHhCu/BUy4kZagfJ/aTxh3yoSwi /5fyimsuVruEYt9fJzEoLy6KQuXI3e1jCnXPptw6EV98/lWccQlzFih+pTFvw1g0L6R41c 4pu2jAjA6tAO2cpYD7IKhoPnNlLbzQstFrnCKI9hv3HjhgfXCQ6u2x3QbzfGDA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4f9Qny2TXSz1NrY for ; Tue, 10 Feb 2026 15:43:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 46fea by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Tue, 10 Feb 2026 15:43:34 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Cc: Sarah Walker From: Andrew Turner Subject: git: 1224347817c4 - main - arm64: Add MOPS implementations of copyin/copyout List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 1224347817c450596797ae6bcbfcc81927cb1f88 Auto-Submitted: auto-generated Date: Tue, 10 Feb 2026 15:43:34 +0000 Message-Id: <698b5226.46fea.1d53d6e5@gitrepo.freebsd.org> The branch main has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=1224347817c450596797ae6bcbfcc81927cb1f88 commit 1224347817c450596797ae6bcbfcc81927cb1f88 Author: Sarah Walker AuthorDate: 2026-01-29 18:08:49 +0000 Commit: Andrew Turner CommitDate: 2026-02-10 15:39:56 +0000 arm64: Add MOPS implementations of copyin/copyout Reimplement copyin() & copyout() as ifuncs. Reviewed by: andrew Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D54947 --- sys/arm64/arm64/copyinout.S | 58 +++++++++++++++++++++++++++++++++++---- sys/arm64/arm64/copyinout_ifunc.c | 50 +++++++++++++++++++++++++++++++++ sys/conf/files.arm64 | 1 + 3 files changed, 103 insertions(+), 6 deletions(-) diff --git a/sys/arm64/arm64/copyinout.S b/sys/arm64/arm64/copyinout.S index e41c4b5f6734..8c6fee63f410 100644 --- a/sys/arm64/arm64/copyinout.S +++ b/sys/arm64/arm64/copyinout.S @@ -65,9 +65,9 @@ END(copyio_fault) /* * Copies from a kernel to user address * - * int copyout(const void *kaddr, void *udaddr, size_t len) + * int copyout_std(const void *kaddr, void *udaddr, size_t len) */ -ENTRY(copyout) +ENTRY(copyout_std) cbz x2, 1f check_user_access 1, 2, copyio_fault_nopcb @@ -76,14 +76,37 @@ ENTRY(copyout) 1: mov x0, xzr /* return 0 */ ret -END(copyout) +END(copyout_std) + +/* + * Copies from a kernel to user address + * + * int copyout_mops(const void *kaddr, void *udaddr, size_t len) + */ +ENTRY(copyout_mops) + cbz x2, 1f + check_user_access 1, 2, copyio_fault_nopcb + + adr x6, copyio_fault /* Get the handler address */ + SET_FAULT_HANDLER(x6, x7) /* Set the handler */ + + .inst 0x19001441 /* cpyfpwt [x1]!, [x0]!, x2! */ + .inst 0x19401441 /* cpyfmwt [x1]!, [x0]!, x2! */ + .inst 0x19801441 /* cpyfewt [x1]!, [x0]!, x2! */ + + SET_FAULT_HANDLER(xzr, x7) /* Clear the handler */ + +1: mov x0, xzr /* return 0 */ + ret + +END(copyout_mops) /* * Copies from a user to kernel address * - * int copyin(const void *uaddr, void *kdaddr, size_t len) + * int copyin_std(const void *uaddr, void *kdaddr, size_t len) */ -ENTRY(copyin) +ENTRY(copyin_std) cbz x2, 1f check_user_access 0, 2, copyio_fault_nopcb @@ -92,7 +115,30 @@ ENTRY(copyin) 1: mov x0, xzr /* return 0 */ ret -END(copyin) +END(copyin_std) + +/* + * Copies from a user to kernel address + * + * int copyin_mops(const void *uaddr, void *kdaddr, size_t len) + */ +ENTRY(copyin_mops) + cbz x2, 1f + check_user_access 0, 2, copyio_fault_nopcb + + adr x6, copyio_fault /* Get the handler address */ + SET_FAULT_HANDLER(x6, x7) /* Set the handler */ + + .inst 0x19002441 /* cpyfprt [x1]!, [x0]!, x2! */ + .inst 0x19402441 /* cpyfmrt [x1]!, [x0]!, x2! */ + .inst 0x19802441 /* cpyfert [x1]!, [x0]!, x2! */ + + SET_FAULT_HANDLER(xzr, x7) /* Clear the handler */ + +1: mov x0, xzr /* return 0 */ + ret + +END(copyin_mops) /* * Copies a string from a user to kernel address diff --git a/sys/arm64/arm64/copyinout_ifunc.c b/sys/arm64/arm64/copyinout_ifunc.c new file mode 100644 index 000000000000..33d9f11fc7e6 --- /dev/null +++ b/sys/arm64/arm64/copyinout_ifunc.c @@ -0,0 +1,50 @@ +/*- + * SPDX-License-Identifier: BSD-2-Clause + * + * Copyright (c) 2026 ARM Ltd + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include + +#include + +#include +#include +#include +#include + +int copyout_std(const void *kaddr, void *udaddr, size_t len); +int copyout_mops(const void *kaddr, void *udaddr, size_t len); +int copyin_std(const void *uaddr, void *kdaddr, size_t len); +int copyin_mops(const void *uaddr, void *kdaddr, size_t len); + +DEFINE_IFUNC(, int, copyout, (const void *, void *, size_t)) +{ + return ((elf_hwcap2 & HWCAP2_MOPS) != 0 ? copyout_mops : copyout_std); +} + +DEFINE_IFUNC(, int, copyin, (const void *, void *, size_t)) +{ + return ((elf_hwcap2 & HWCAP2_MOPS) != 0 ? copyin_mops : copyin_std); +} diff --git a/sys/conf/files.arm64 b/sys/conf/files.arm64 index 5439cc5edde0..3b12999a6c87 100644 --- a/sys/conf/files.arm64 +++ b/sys/conf/files.arm64 @@ -36,6 +36,7 @@ arm64/arm64/busdma_bounce.c standard arm64/arm64/busdma_machdep.c standard arm64/arm64/clock.c standard arm64/arm64/copyinout.S standard +arm64/arm64/copyinout_ifunc.c standard arm64/arm64/cpu_errata.c standard arm64/arm64/cpu_feat.c standard arm64/arm64/cpufunc_asm.S standard