From nobody Wed May 27 15:24:14 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4gQYLl4vJrz6fBTv for ; Wed, 27 May 2026 15:24:15 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R13" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4gQYLl0bPnz3PgR for ; Wed, 27 May 2026 15:24:15 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1779895455; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=OhyeRMfGtInq0oXB73yioXSxorLwzCrrkRN/6HaqZFs=; b=Y9wNOYNuR5gl3Zv+5FGlRmMNPm1oECdQLyQ+oS8ScEjZSCRJLhwL0ki8UAP7x4Mw7/iqL8 lJIbzVRWZnXZvJxogGZnr8DaYJxSp1g+hy+B/T6a9CtsFvMYOt+JUX+Phiu0D6YPG8t/E1 hRAAWlAu3P6LKeMkDf5hehy/0RmEmYpw5I9JqZuf4u5ou6Wp1ubhAXUp+Ic5SEUd2UO5h6 ybOFd3p9My58gY9ZLtIYYF9rJtx8d5xqCa+BamS3dHqjPgPwYphin0P1JD8zSHLWjNEMUn IjyhXmn/J4LaV0N0G5qCYK3+7EpDQ6dtAUPjGgeGbXRqiTzVJHFj/bkO9jjGmQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1779895455; a=rsa-sha256; cv=none; b=Ut/BrjwN2olR8T/+ICdTHGagk2CEx4YKQbOBaU7MwMcdDibQdqpE3T01JxHIRqlE/c3+x/ BqGVmr6RKAjZfaY9FflxchTfNOM+vA5fJ5TeD8X/vrPi0iHcS/rkDRu8aJJRmm4rjzjeE2 hU63Vg3ohePLyA10OC68D1yFW2NKRo7LhAVBf9bzwvcqUHEm1a00cQLtnRJOWv7c5rsxa8 NR2Q1h3zPYtIYBkMvj4CTrxAnCM9OICWJyam6omWrGl17DDRSfBZ+fZO7juIvw9pAkhRT9 4b2NRe6zhKMAvA+xRMCRdsQNZ3h1s5X5QsWM6z37vWNLPL91Jzgb8UivNROd8w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1779895455; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=OhyeRMfGtInq0oXB73yioXSxorLwzCrrkRN/6HaqZFs=; b=P7DlrJ+KZhrCtDQsu5N73QTj0BP6JIkZpTpmDbrrxUTuho+b8oEgMbq0iYTlyUM5hum4Ne F5+y34XT7kcrvrOYBhdifStfmE06vmEVdaIz19apLmgodbU+yIOV5gpqntjG46qfNeqUg7 I/qw+Rj14feLlUJU6hmGSpvYRh0RHUr57AAQ5GrxJQgVdfF4D86kpT3l9os6BD4jvzUMqG JNFsXrHsbXZHe6Bu3dWaenr+mzGPh2byd1jhERTdq9mqIEDm20WLqt2xKoR70E29GwgjVI TKucz+BzP5uyDUHsvjLITF7lT26NwhoTKnyR0Ht2ofZaR5YnglIk72GD7GJV1A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4gQYLk713Nz14ns for ; Wed, 27 May 2026 15:24:14 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 277fc by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Wed, 27 May 2026 15:24:14 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Cc: Harry Moulton From: Andrew Turner Subject: git: 7bb6b62394d3 - main - arm64: mte: copy/save tags on copy-on-write List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 7bb6b62394d3036567cf1453395d9e8b63d3dda1 Auto-Submitted: auto-generated Date: Wed, 27 May 2026 15:24:14 +0000 Message-Id: <6a170c9e.277fc.6fdde7de@gitrepo.freebsd.org> The branch main has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=7bb6b62394d3036567cf1453395d9e8b63d3dda1 commit 7bb6b62394d3036567cf1453395d9e8b63d3dda1 Author: Harry Moulton AuthorDate: 2026-05-18 09:29:03 +0000 Commit: Andrew Turner CommitDate: 2026-05-27 15:22:26 +0000 arm64: mte: copy/save tags on copy-on-write On a copy-on-write, copy the memory tags from the source pages to the destination pages so the forked process can continue to use MTE. Reviewed by: andrew Sponsored by: Arm Ltd Signed-off-by: Harry Moulton Differential Revision: https://reviews.freebsd.org/D55955 --- sys/arm64/arm64/mte.c | 41 +++++++++++++++++++++++++++++++++++++++++ sys/arm64/arm64/pmap.c | 13 +++++++++++++ sys/arm64/include/cpu.h | 1 + 3 files changed, 55 insertions(+) diff --git a/sys/arm64/arm64/mte.c b/sys/arm64/arm64/mte.c index ba5d9df3b01c..6e902858a8b9 100644 --- a/sys/arm64/arm64/mte.c +++ b/sys/arm64/arm64/mte.c @@ -49,6 +49,22 @@ static u_int __read_mostly mte_version = 0; struct thread *mte_switch(struct thread *); +#define load_tags(addr) ({ \ + uint64_t __val; \ + asm volatile( \ + ".arch_extension memtag \n" \ + "ldgm %0, [%1] \n" \ + ".arch_extension nomemtag" : "=r" (__val) : "r" (addr)); \ + __val; \ +}) + +#define set_tags(tags, addr) do { \ + asm volatile( \ + ".arch_extension memtag \n" \ + "stgm %0, [%1] \n" \ + ".arch_extension nomemtag" : "=r" (tags) : "r" (addr)); \ +} while (0) + /* Fetch the block size used by tag load and store instructions */ static inline size_t mte_block_size(void) @@ -95,6 +111,31 @@ mte_sync_tags(vm_page_t page) page->md.pv_flags |= PV_MTE_TAGGED; } +/** + * Copy the allocation tags from given target to destination page. This is called + * on a copy-on-write and anything that causes a pmap_copy_page call. + */ +void +mte_copy_tags(vm_page_t srcpage, vm_page_t dstpage, char *src, char *dst) +{ + size_t block_size; + uint64_t tags; + + MPASS((srcpage->md.pv_flags & PV_MTE_TAGGED) != 0); + + /* + * Copy the tags from the source page to the destination page, + * incrementing by the block count read from GMID_EL1 + */ + block_size = mte_block_size(); + for (size_t count = 0; count < PAGE_SIZE; + count += block_size, src += block_size, dst += block_size) { + tags = load_tags(src); + set_tags(tags, dst); + } + dstpage->md.pv_flags |= PV_MTE_TAGGED; +} + void mte_fork(struct thread *new_td, struct thread *orig_td) { diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index 6914fc13b065..1fb9ac2011aa 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -146,6 +146,7 @@ #include #include +#include #include #include #include @@ -6954,6 +6955,15 @@ pmap_copy_page(vm_page_t msrc, vm_page_t mdst) void *src = VM_PAGE_TO_DMAP(msrc); void *dst = VM_PAGE_TO_DMAP(mdst); + /* + * On a page copy, check whether the src page is tagged. If it is, + * we must copy the tags before copying the contents of the page. + */ + if ((msrc->md.pv_flags & PV_MTE_TAGGED) != 0) + mte_copy_tags(msrc, mdst, src, dst); + else + mdst->md.pv_flags &= ~PV_MTE_TAGGED; + pagecopy(src, dst); } @@ -6970,6 +6980,9 @@ pmap_copy_pages(vm_page_t ma[], vm_offset_t a_offset, vm_page_t mb[], int cnt; while (xfersize > 0) { + KASSERT(ADDR_IS_CANONICAL(a_offset), + ("%s: Address not in canonical form: %lx", __func__, a_offset)); + a_pg_offset = a_offset & PAGE_MASK; m_a = ma[a_offset >> PAGE_SHIFT]; p_a = m_a->phys_addr; diff --git a/sys/arm64/include/cpu.h b/sys/arm64/include/cpu.h index d36e1e56c91c..bdbc601edd26 100644 --- a/sys/arm64/include/cpu.h +++ b/sys/arm64/include/cpu.h @@ -285,6 +285,7 @@ void mte_thread_alloc(struct thread *); void mte_thread0(struct thread *); void mte_sync_tags(vm_page_t page); +void mte_copy_tags(vm_page_t, vm_page_t, char *, char *); /* Functions to read the sanitised view of the special registers */ void update_special_regs(u_int);