From owner-svn-src-all@freebsd.org Thu Sep 20 18:29:56 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 33D2810A4C01; Thu, 20 Sep 2018 18:29:56 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DB3267855A; Thu, 20 Sep 2018 18:29:55 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D632B21A12; Thu, 20 Sep 2018 18:29:55 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w8KITtHc063617; Thu, 20 Sep 2018 18:29:55 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w8KITt25063613; Thu, 20 Sep 2018 18:29:55 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201809201829.w8KITt25063613@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Thu, 20 Sep 2018 18:29:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r338836 - in head/sys: kern vm X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: in head/sys: kern vm X-SVN-Commit-Revision: 338836 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2018 18:29:56 -0000 Author: markj Date: Thu Sep 20 18:29:55 2018 New Revision: 338836 URL: https://svnweb.freebsd.org/changeset/base/338836 Log: Ensure that imports into per-domain kmem arenas are KVA_QUANTUM-aligned. The old code appears to assume that vmem_alloc() would import size-aligned KVA chunks from the parent kernel_arena, but vmem doesn't provide this guarantee. Also remove the unused global RWX arena and add comments explaining why we have per-domain arenas. Reported by: alc Reviewed by: alc, kib (previous version) Approved by: re (gjb) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D17249 Modified: head/sys/kern/subr_vmem.c head/sys/vm/vm_kern.c head/sys/vm/vm_kern.h Modified: head/sys/kern/subr_vmem.c ============================================================================== --- head/sys/kern/subr_vmem.c Thu Sep 20 18:25:26 2018 (r338835) +++ head/sys/kern/subr_vmem.c Thu Sep 20 18:29:55 2018 (r338836) @@ -241,9 +241,6 @@ static struct vmem buffer_arena_storage; static struct vmem transient_arena_storage; /* kernel and kmem arenas are aliased for backwards KPI compat. */ vmem_t *kernel_arena = &kernel_arena_storage; -#if VM_NRESERVLEVEL > 0 -vmem_t *kernel_rwx_arena = NULL; -#endif vmem_t *kmem_arena = &kernel_arena_storage; vmem_t *buffer_arena = &buffer_arena_storage; vmem_t *transient_arena = &transient_arena_storage; Modified: head/sys/vm/vm_kern.c ============================================================================== --- head/sys/vm/vm_kern.c Thu Sep 20 18:25:26 2018 (r338835) +++ head/sys/vm/vm_kern.c Thu Sep 20 18:29:55 2018 (r338836) @@ -669,7 +669,7 @@ kmem_init_zero_region(void) } /* - * Import kva into the kernel arena. + * Import KVA from the kernel map into the kernel arena. */ static int kva_import(void *unused, vmem_size_t size, int flags, vmem_addr_t *addrp) @@ -691,22 +691,20 @@ kva_import(void *unused, vmem_size_t size, int flags, return (0); } -#if VM_NRESERVLEVEL > 0 /* - * Import a superpage from the normal kernel arena into the special - * arena for allocations with different permissions. + * Import KVA from a parent arena into a per-domain arena. Imports must be + * KVA_QUANTUM-aligned and a multiple of KVA_QUANTUM in size. */ static int -kernel_rwx_alloc(void *arena, vmem_size_t size, int flags, vmem_addr_t *addrp) +kva_import_domain(void *arena, vmem_size_t size, int flags, vmem_addr_t *addrp) { KASSERT((size % KVA_QUANTUM) == 0, - ("kernel_rwx_alloc: Size %jd is not a multiple of %d", + ("kva_import_domain: Size %jd is not a multiple of %d", (intmax_t)size, (int)KVA_QUANTUM)); return (vmem_xalloc(arena, size, KVA_QUANTUM, 0, 0, VMEM_ADDR_MIN, VMEM_ADDR_MAX, flags, addrp)); } -#endif /* * kmem_init: @@ -744,30 +742,30 @@ kmem_init(vm_offset_t start, vm_offset_t end) vmem_init(kernel_arena, "kernel arena", 0, 0, PAGE_SIZE, 0, 0); vmem_set_import(kernel_arena, kva_import, NULL, NULL, KVA_QUANTUM); -#if VM_NRESERVLEVEL > 0 - /* - * In an architecture with superpages, maintain a separate arena - * for allocations with permissions that differ from the "standard" - * read/write permissions used for memory in the kernel_arena. - */ - kernel_rwx_arena = vmem_create("kernel rwx arena", 0, 0, PAGE_SIZE, - 0, M_WAITOK); - vmem_set_import(kernel_rwx_arena, kernel_rwx_alloc, - (vmem_release_t *)vmem_xfree, kernel_arena, KVA_QUANTUM); -#endif - for (domain = 0; domain < vm_ndomains; domain++) { + /* + * Initialize the per-domain arenas. These are used to color + * the KVA space in a way that ensures that virtual large pages + * are backed by memory from the same physical domain, + * maximizing the potential for superpage promotion. + */ vm_dom[domain].vmd_kernel_arena = vmem_create( "kernel arena domain", 0, 0, PAGE_SIZE, 0, M_WAITOK); vmem_set_import(vm_dom[domain].vmd_kernel_arena, - (vmem_import_t *)vmem_alloc, NULL, kernel_arena, - KVA_QUANTUM); + kva_import_domain, NULL, kernel_arena, KVA_QUANTUM); + + /* + * In architectures with superpages, maintain separate arenas + * for allocations with permissions that differ from the + * "standard" read/write permissions used for kernel memory, + * so as not to inhibit superpage promotion. + */ #if VM_NRESERVLEVEL > 0 vm_dom[domain].vmd_kernel_rwx_arena = vmem_create( "kernel rwx arena domain", 0, 0, PAGE_SIZE, 0, M_WAITOK); vmem_set_import(vm_dom[domain].vmd_kernel_rwx_arena, - kernel_rwx_alloc, (vmem_release_t *)vmem_xfree, - vm_dom[domain].vmd_kernel_arena, KVA_QUANTUM); + kva_import_domain, (vmem_release_t *)vmem_xfree, + kernel_arena, KVA_QUANTUM); #endif } } Modified: head/sys/vm/vm_kern.h ============================================================================== --- head/sys/vm/vm_kern.h Thu Sep 20 18:25:26 2018 (r338835) +++ head/sys/vm/vm_kern.h Thu Sep 20 18:29:55 2018 (r338836) @@ -70,7 +70,6 @@ extern vm_map_t kernel_map; extern vm_map_t exec_map; extern vm_map_t pipe_map; extern struct vmem *kernel_arena; -extern struct vmem *kernel_rwx_arena; extern struct vmem *kmem_arena; extern struct vmem *buffer_arena; extern struct vmem *transient_arena;