From owner-dev-commits-src-all@freebsd.org Wed Feb 3 21:18:10 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EC4EC52C469; Wed, 3 Feb 2021 21:18:10 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DWF026Q0Vz3qPQ; Wed, 3 Feb 2021 21:18:10 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CF29427E8D; Wed, 3 Feb 2021 21:18:10 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 113LIA6R023751; Wed, 3 Feb 2021 21:18:10 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 113LIAqO023750; Wed, 3 Feb 2021 21:18:10 GMT (envelope-from git) Date: Wed, 3 Feb 2021 21:18:10 GMT Message-Id: <202102032118.113LIAqO023750@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Ryan Stone Subject: git: 660344ca44c6 - main - Add a VM flag to prevent reclaim on a failed contig allocation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rstone X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 660344ca44c63bfe4a16c3e57d0f6dbcbb5e083e Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Feb 2021 21:18:11 -0000 The branch main has been updated by rstone: URL: https://cgit.FreeBSD.org/src/commit/?id=660344ca44c63bfe4a16c3e57d0f6dbcbb5e083e commit 660344ca44c63bfe4a16c3e57d0f6dbcbb5e083e Author: Ryan Stone AuthorDate: 2021-01-29 21:13:57 +0000 Commit: Ryan Stone CommitDate: 2021-02-03 21:16:51 +0000 Add a VM flag to prevent reclaim on a failed contig allocation If a M_WAITOK contig alloc fails, the VM subsystem will try to reclaim contiguous memory twice before actually failing the request. On a system with 64GB of RAM I've observed this take 400-500ms before it finally gives up, and I believe that this will only be worse on systems with even more memory. In certain contexts this delay is extremely harmful, so add a flag that will skip reclaim for allocation requests to allow those paths to opt-out of doing an expensive reclaim. Sponsored by: Dell Inc Differential Revision: https://reviews.freebsd.org/D28422 Reviewed by: markj, kib --- sys/sys/malloc.h | 1 + sys/vm/vm_kern.c | 9 +++++++-- sys/vm/vm_page.h | 3 +++ 3 files changed, 11 insertions(+), 2 deletions(-) diff --git a/sys/sys/malloc.h b/sys/sys/malloc.h index a11dd767efc5..54a05e94a3a2 100644 --- a/sys/sys/malloc.h +++ b/sys/sys/malloc.h @@ -54,6 +54,7 @@ */ #define M_NOWAIT 0x0001 /* do not block */ #define M_WAITOK 0x0002 /* ok to block */ +#define M_NORECLAIM 0x0080 /* do not reclaim after failure */ #define M_ZERO 0x0100 /* bzero the allocation */ #define M_NOVM 0x0200 /* don't ask VM for pages */ #define M_USE_RESERVE 0x0400 /* can alloc out of reserve memory */ diff --git a/sys/vm/vm_kern.c b/sys/vm/vm_kern.c index f1704f834157..e8e24c3ca8a3 100644 --- a/sys/vm/vm_kern.c +++ b/sys/vm/vm_kern.c @@ -177,17 +177,22 @@ kmem_alloc_contig_pages(vm_object_t object, vm_pindex_t pindex, int domain, { vm_page_t m; int tries; - bool wait; + bool wait, reclaim; VM_OBJECT_ASSERT_WLOCKED(object); + /* Disallow an invalid combination of flags. */ + MPASS((pflags & (VM_ALLOC_WAITOK | VM_ALLOC_NORECLAIM)) != + (VM_ALLOC_WAITOK | VM_ALLOC_NORECLAIM)); + wait = (pflags & VM_ALLOC_WAITOK) != 0; + reclaim = (pflags & VM_ALLOC_NORECLAIM) == 0; pflags &= ~(VM_ALLOC_NOWAIT | VM_ALLOC_WAITOK | VM_ALLOC_WAITFAIL); pflags |= VM_ALLOC_NOWAIT; for (tries = wait ? 3 : 1;; tries--) { m = vm_page_alloc_contig_domain(object, pindex, domain, pflags, npages, low, high, alignment, boundary, memattr); - if (m != NULL || tries == 0) + if (m != NULL || tries == 0 || !reclaim) break; VM_OBJECT_WUNLOCK(object); diff --git a/sys/vm/vm_page.h b/sys/vm/vm_page.h index b05a3d1be861..ed372cae2a3b 100644 --- a/sys/vm/vm_page.h +++ b/sys/vm/vm_page.h @@ -541,6 +541,7 @@ vm_page_t PHYS_TO_VM_PAGE(vm_paddr_t pa); #define VM_ALLOC_WAITFAIL 0x0010 /* (acf) Sleep and return error */ #define VM_ALLOC_WIRED 0x0020 /* (acfgp) Allocate a wired page */ #define VM_ALLOC_ZERO 0x0040 /* (acfgp) Allocate a prezeroed page */ +#define VM_ALLOC_NORECLAIM 0x0080 /* (c) Do not reclaim after failure */ #define VM_ALLOC_NOOBJ 0x0100 /* (acg) No associated object */ #define VM_ALLOC_NOBUSY 0x0200 /* (acgp) Do not excl busy the page */ #define VM_ALLOC_NOCREAT 0x0400 /* (gp) Don't create a page */ @@ -570,6 +571,8 @@ malloc2vm_flags(int malloc_flags) pflags |= VM_ALLOC_NOWAIT; if ((malloc_flags & M_WAITOK)) pflags |= VM_ALLOC_WAITOK; + if ((malloc_flags & M_NORECLAIM)) + pflags |= VM_ALLOC_NORECLAIM; return (pflags); } #endif