From nobody Fri Apr 14 23:54:09 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PytZn4qYGz454pZ; Fri, 14 Apr 2023 23:54:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PytZn454vz4Mcq; Fri, 14 Apr 2023 23:54:09 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681516449; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Jnz2Lw5jDilY9pt1FvTjk8HxQP0UgjRJr4HtM0SNx9I=; b=duNQ4LGdj/aoF2wBUVEyuDyEZ66KlVxzXdoEZ8NC7+plfso1JnalbV3FSSYjh49ruVnc9U nOps4wBAF2LhU5kO4LS7vpaXzOJVBjzjKO9s6ph93N7qKCC57fcIM0Yo9/5D3MqOcNjJl3 eRULC0/kiRHOCyoiLm5s2BEAEOZk4HUNTHHGNbhYE8/PAX5dJa3W2WnIM5V2vM+Lp9S+lR QrpmG0zTIv0/lXhGhClyCrx3Mg6oSb9KVWWqkqwbzE56GpSrtzndqd41fDcHQYBjwym2Rd vTkoy7tMvpeoyJAS+qvuIuF3xDWTU9glzsIH50luzAXTdln/cnEAJqm+yPhDhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681516449; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Jnz2Lw5jDilY9pt1FvTjk8HxQP0UgjRJr4HtM0SNx9I=; b=rygl71JzYg5ePzVKOnFR+9KBL6cmvkEmrwkiP9+5LuccV2670GdCUjxBj/XntsbmcX/qoU v+3S2mctJQrJyysPfq3qwHR5nDdrItwdyQBm1QBEYl854fKfF9qRADNwUx6xyzNzp+y86y mXxOdpNdLys0FZq2KsXllrL9dzcBwNxNhN9oNTnf8OhLIIJjvrQYh1EMSA0sfZAscQhbrc IQWu1CPJGyEJYqrAgtOnfZM8u1i/NkHoiJWfzmjcVvqySxEmr7TOMVpuCtPE6tJTVvLYaK kW4ZOMdZGDZJFpKAPv1Xc+vvcIPfubC8bKwaWA26et9IQsSm+SaWXWQx9k/pLw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681516449; a=rsa-sha256; cv=none; b=xBeuZQe/9IeglaPPXgAF8/kw3reCoRaThpr6usTUUsNtEj5dTShSAUCClRTvAGIZ0l3+L7 a0e0qVCRw0ZrRdA1AJxk4EeBnHXWvt5GaZOMWxOGoXzCY8ZSk8MXFUxRK1Yr8QIvVM3cGX k074ziObvvk1s10YN1xS766XXCx0a4Ls4XLTWs6qcq+sWdIdLKq8M6/gukBJOz9NX6MnkD Srz16QoAukWcqOhbzhIzip5fGsEEszM7OxT9vk0aam1p9p9UoPTbq5QMA3v+pGMks6+FS8 D+oJEF+LRjKmZ23kvEGxMN5HioOMcZSwePVSWT7AeksCHLy8POAFUHwCO5QY4A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4PytZn397Vzl74; Fri, 14 Apr 2023 23:54:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 33ENs9R4062221; Fri, 14 Apr 2023 23:54:09 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 33ENs9MH062220; Fri, 14 Apr 2023 23:54:09 GMT (envelope-from git) Date: Fri, 14 Apr 2023 23:54:09 GMT Message-Id: <202304142354.33ENs9MH062220@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Konstantin Belousov Subject: git: 1e0e335b0f0d - main - amd64: fix PKRU and swapout interaction List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 1e0e335b0f0dbae8ce49307377b23ef3673bd402 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=1e0e335b0f0dbae8ce49307377b23ef3673bd402 commit 1e0e335b0f0dbae8ce49307377b23ef3673bd402 Author: Konstantin Belousov AuthorDate: 2023-04-13 11:37:35 +0000 Commit: Konstantin Belousov CommitDate: 2023-04-14 23:53:59 +0000 amd64: fix PKRU and swapout interaction When vm_map_remove() is called from vm_swapout_map_deactivate_pages() due to swapout, PKRU attributes for the removed range must be kept intact. Provide a variant of pmap_remove(), pmap_map_delete(), to allow pmap to distinguish between real removes of the UVA mappings and any other internal removes, e.g. swapout. For non-amd64, pmap_map_delete() is stubbed by define to pmap_remove(). Reported by: andrew Reviewed by: markj Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D39556 --- sys/amd64/amd64/pmap.c | 37 ++++++++++++++++++++++++++++--------- sys/amd64/include/pmap.h | 1 + sys/arm/include/pmap.h | 1 + sys/arm64/include/pmap.h | 1 + sys/i386/include/pmap.h | 1 + sys/powerpc/include/pmap.h | 1 + sys/riscv/include/pmap.h | 1 + sys/vm/vm_map.c | 2 +- 8 files changed, 35 insertions(+), 10 deletions(-) diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c index a31cdb2a8215..e8a88aab3225 100644 --- a/sys/amd64/amd64/pmap.c +++ b/sys/amd64/amd64/pmap.c @@ -6280,14 +6280,8 @@ pmap_remove_ptes(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, return (anyvalid); } -/* - * Remove the given range of addresses from the specified map. - * - * It is assumed that the start and end are properly - * rounded to the page size. - */ -void -pmap_remove(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) +static void +pmap_remove1(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, bool map_delete) { struct rwlock *lock; vm_page_t mt; @@ -6319,7 +6313,8 @@ pmap_remove(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) pmap_delayed_invl_start(); PMAP_LOCK(pmap); - pmap_pkru_on_remove(pmap, sva, eva); + if (map_delete) + pmap_pkru_on_remove(pmap, sva, eva); /* * special handling of removing one page. a very @@ -6441,6 +6436,30 @@ out: vm_page_free_pages_toq(&free, true); } +/* + * Remove the given range of addresses from the specified map. + * + * It is assumed that the start and end are properly + * rounded to the page size. + */ +void +pmap_remove(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) +{ + pmap_remove1(pmap, sva, eva, false); +} + +/* + * Remove the given range of addresses as part of a logical unmap + * operation. This has the effect of calling pmap_remove(), but + * also clears any metadata that should persist for the lifetime + * of a logical mapping. + */ +void +pmap_map_delete(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) +{ + pmap_remove1(pmap, sva, eva, true); +} + /* * Routine: pmap_remove_all * Function: diff --git a/sys/amd64/include/pmap.h b/sys/amd64/include/pmap.h index a7e456d80fcf..d74f25cc8024 100644 --- a/sys/amd64/include/pmap.h +++ b/sys/amd64/include/pmap.h @@ -486,6 +486,7 @@ void pmap_force_invalidate_cache_range(vm_offset_t sva, vm_offset_t eva); void pmap_get_mapping(pmap_t pmap, vm_offset_t va, uint64_t *ptr, int *num); boolean_t pmap_map_io_transient(vm_page_t *, vm_offset_t *, int, boolean_t); void pmap_unmap_io_transient(vm_page_t *, vm_offset_t *, int, boolean_t); +void pmap_map_delete(pmap_t, vm_offset_t, vm_offset_t); void pmap_pti_add_kva(vm_offset_t sva, vm_offset_t eva, bool exec); void pmap_pti_remove_kva(vm_offset_t sva, vm_offset_t eva); void pmap_pti_pcid_invalidate(uint64_t ucr3, uint64_t kcr3); diff --git a/sys/arm/include/pmap.h b/sys/arm/include/pmap.h index b16651f789cc..977a1e989b42 100644 --- a/sys/arm/include/pmap.h +++ b/sys/arm/include/pmap.h @@ -45,6 +45,7 @@ extern vm_offset_t virtual_end; void *pmap_kenter_temporary(vm_paddr_t, int); #define pmap_page_is_write_mapped(m) (((m)->a.flags & PGA_WRITEABLE) != 0) void pmap_page_set_memattr(vm_page_t, vm_memattr_t); +#define pmap_map_delete(pmap, sva, eva) pmap_remove(pmap, sva, eva) void *pmap_mapdev(vm_paddr_t, vm_size_t); void pmap_unmapdev(void *, vm_size_t); diff --git a/sys/arm64/include/pmap.h b/sys/arm64/include/pmap.h index 2847fa8dd439..8e3e81f2a5d3 100644 --- a/sys/arm64/include/pmap.h +++ b/sys/arm64/include/pmap.h @@ -154,6 +154,7 @@ int pmap_pinit_stage(pmap_t, enum pmap_stage, int); bool pmap_ps_enabled(pmap_t pmap); uint64_t pmap_to_ttbr0(pmap_t pmap); void pmap_disable_promotion(vm_offset_t sva, vm_size_t size); +#define pmap_map_delete(pmap, sva, eva) pmap_remove(pmap, sva, eva) void *pmap_mapdev(vm_paddr_t, vm_size_t); void *pmap_mapbios(vm_paddr_t, vm_size_t); diff --git a/sys/i386/include/pmap.h b/sys/i386/include/pmap.h index 1ee13871bc32..87570929e0dd 100644 --- a/sys/i386/include/pmap.h +++ b/sys/i386/include/pmap.h @@ -258,6 +258,7 @@ void pmap_invalidate_cache_range(vm_offset_t sva, vm_offset_t eva); void pmap_force_invalidate_cache_range(vm_offset_t sva, vm_offset_t eva); void *pmap_trm_alloc(size_t size, int flags); void pmap_trm_free(void *addr, size_t size); +#define pmap_map_delete(pmap, sva, eva) pmap_remove(pmap, sva, eva) void invltlb_glob(void); diff --git a/sys/powerpc/include/pmap.h b/sys/powerpc/include/pmap.h index 5746378c64ee..4ee9b6a67ef6 100644 --- a/sys/powerpc/include/pmap.h +++ b/sys/powerpc/include/pmap.h @@ -326,6 +326,7 @@ const char *pmap_mmu_name(void); bool pmap_ps_enabled(pmap_t pmap); int pmap_nofault(pmap_t pmap, vm_offset_t va, vm_prot_t flags); boolean_t pmap_page_is_mapped(vm_page_t m); +#define pmap_map_delete(pmap, sva, eva) pmap_remove(pmap, sva, eva) void pmap_page_array_startup(long count); diff --git a/sys/riscv/include/pmap.h b/sys/riscv/include/pmap.h index 04808422c2c2..88579e6448b5 100644 --- a/sys/riscv/include/pmap.h +++ b/sys/riscv/include/pmap.h @@ -57,6 +57,7 @@ #define pmap_page_get_memattr(m) ((m)->md.pv_memattr) #define pmap_page_is_write_mapped(m) (((m)->a.flags & PGA_WRITEABLE) != 0) void pmap_page_set_memattr(vm_page_t m, vm_memattr_t ma); +#define pmap_map_delete(pmap, sva, eva) pmap_remove(pmap, sva, eva) /* * Pmap stuff diff --git a/sys/vm/vm_map.c b/sys/vm/vm_map.c index a7efa00da1e2..f5863a9b9939 100644 --- a/sys/vm/vm_map.c +++ b/sys/vm/vm_map.c @@ -3964,7 +3964,7 @@ vm_map_delete(vm_map_t map, vm_offset_t start, vm_offset_t end) */ if ((entry->eflags & MAP_ENTRY_IS_SUB_MAP) != 0 || entry->object.vm_object != NULL) - pmap_remove(map->pmap, entry->start, entry->end); + pmap_map_delete(map->pmap, entry->start, entry->end); if (entry->end == map->anon_loc) map->anon_loc = entry->start;