From owner-svn-src-head@freebsd.org Thu Sep 20 15:45:13 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7FF7310A09A8; Thu, 20 Sep 2018 15:45:13 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 354CD72604; Thu, 20 Sep 2018 15:45:13 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2C3651FFBF; Thu, 20 Sep 2018 15:45:13 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w8KFjCD2081309; Thu, 20 Sep 2018 15:45:12 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w8KFjCr5081308; Thu, 20 Sep 2018 15:45:12 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201809201545.w8KFjCr5081308@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Thu, 20 Sep 2018 15:45:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r338830 - head/sys/vm X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/sys/vm X-SVN-Commit-Revision: 338830 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2018 15:45:13 -0000 Author: markj Date: Thu Sep 20 15:45:12 2018 New Revision: 338830 URL: https://svnweb.freebsd.org/changeset/base/338830 Log: Change the domain selection policy in kmem_back(). Ensure that pages backing the same virtual large page come from the same physical domain, as kmem_malloc_domain() does. PR: 231038 Reviewed by: alc, kib Approved by: re (gjb) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D17248 Modified: head/sys/vm/vm_kern.c Modified: head/sys/vm/vm_kern.c ============================================================================== --- head/sys/vm/vm_kern.c Thu Sep 20 13:32:40 2018 (r338829) +++ head/sys/vm/vm_kern.c Thu Sep 20 15:45:12 2018 (r338830) @@ -122,11 +122,12 @@ SYSCTL_ULONG(_vm, OID_AUTO, max_kernel_address, CTLFLA "Max kernel address"); #if VM_NRESERVLEVEL > 0 -#define KVA_QUANTUM (1 << (VM_LEVEL_0_ORDER + PAGE_SHIFT)) +#define KVA_QUANTUM_SHIFT (VM_LEVEL_0_ORDER + PAGE_SHIFT) #else /* On non-superpage architectures want large import sizes. */ -#define KVA_QUANTUM (PAGE_SIZE * 1024) +#define KVA_QUANTUM_SHIFT (10 + PAGE_SHIFT) #endif +#define KVA_QUANTUM (1 << KVA_QUANTUM_SHIFT) /* * kva_alloc: @@ -416,9 +417,10 @@ kmem_malloc(vm_size_t size, int flags) } /* - * kmem_back: + * kmem_back_domain: * - * Allocate physical pages for the specified virtual address range. + * Allocate physical pages from the specified domain for the specified + * virtual address range. */ int kmem_back_domain(int domain, vm_object_t object, vm_offset_t addr, @@ -479,24 +481,39 @@ retry: return (KERN_SUCCESS); } +/* + * kmem_back: + * + * Allocate physical pages for the specified virtual address range. + */ int kmem_back(vm_object_t object, vm_offset_t addr, vm_size_t size, int flags) { - struct vm_domainset_iter di; - int domain; - int ret; + vm_offset_t end, next, start; + int domain, rv; KASSERT(object == kernel_object, ("kmem_back: only supports kernel object.")); - vm_domainset_iter_malloc_init(&di, kernel_object, &domain, &flags); - do { - ret = kmem_back_domain(domain, object, addr, size, flags); - if (ret == KERN_SUCCESS) + for (start = addr, end = addr + size; addr < end; addr = next) { + /* + * We must ensure that pages backing a given large virtual page + * all come from the same physical domain. + */ + if (vm_ndomains > 1) { + domain = (addr >> KVA_QUANTUM_SHIFT) % vm_ndomains; + next = roundup2(addr + 1, KVA_QUANTUM); + if (next > end || next < start) + next = end; + } else + next = end; + rv = kmem_back_domain(domain, object, addr, next - addr, flags); + if (rv != KERN_SUCCESS) { + kmem_unback(object, start, addr - start); break; - } while (vm_domainset_iter_malloc(&di, &domain, &flags) == 0); - - return (ret); + } + } + return (rv); } /*