From owner-svn-src-stable-12@freebsd.org Wed Sep 18 07:25:05 2019 Return-Path: Delivered-To: svn-src-stable-12@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EA316F4B51; Wed, 18 Sep 2019 07:25:05 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46YBMP5tdBz3y1g; Wed, 18 Sep 2019 07:25:05 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AD4872197D; Wed, 18 Sep 2019 07:25:05 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x8I7P5BY006078; Wed, 18 Sep 2019 07:25:05 GMT (envelope-from alc@FreeBSD.org) Received: (from alc@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x8I7P5NE006075; Wed, 18 Sep 2019 07:25:05 GMT (envelope-from alc@FreeBSD.org) Message-Id: <201909180725.x8I7P5NE006075@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: alc set sender to alc@FreeBSD.org using -f From: Alan Cox Date: Wed, 18 Sep 2019 07:25:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r352475 - in stable/12/sys: amd64/amd64 arm64/arm64 i386/i386 X-SVN-Group: stable-12 X-SVN-Commit-Author: alc X-SVN-Commit-Paths: in stable/12/sys: amd64/amd64 arm64/arm64 i386/i386 X-SVN-Commit-Revision: 352475 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable-12@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for only the 12-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Sep 2019 07:25:06 -0000 Author: alc Date: Wed Sep 18 07:25:04 2019 New Revision: 352475 URL: https://svnweb.freebsd.org/changeset/base/352475 Log: MFC r350463 In pmap_advise(), when we encounter a superpage mapping, we first demote the mapping and then destroy one of the 4 KB page mappings so that there is a potential trigger for repromotion. Currently, we destroy the first 4 KB page mapping that falls within the (current) superpage mapping or the virtual address range [sva, eva). However, I have found empirically that destroying the last 4 KB mapping produces slightly better results, specifically, more promotions and fewer failed promotion attempts. Accordingly, this revision changes pmap_advise() to destroy the last 4 KB page mapping. It also replaces some nearby uses of boolean_t with bool. Modified: stable/12/sys/amd64/amd64/pmap.c stable/12/sys/arm64/arm64/pmap.c stable/12/sys/i386/i386/pmap.c Directory Properties: stable/12/ (props changed) Modified: stable/12/sys/amd64/amd64/pmap.c ============================================================================== --- stable/12/sys/amd64/amd64/pmap.c Wed Sep 18 07:24:33 2019 (r352474) +++ stable/12/sys/amd64/amd64/pmap.c Wed Sep 18 07:25:04 2019 (r352475) @@ -7462,7 +7462,7 @@ pmap_advise(pmap_t pmap, vm_offset_t sva, vm_offset_t pt_entry_t *pte, PG_A, PG_G, PG_M, PG_RW, PG_V; vm_offset_t va, va_next; vm_page_t m; - boolean_t anychanged; + bool anychanged; if (advice != MADV_DONTNEED && advice != MADV_FREE) return; @@ -7481,7 +7481,7 @@ pmap_advise(pmap_t pmap, vm_offset_t sva, vm_offset_t PG_M = pmap_modified_bit(pmap); PG_V = pmap_valid_bit(pmap); PG_RW = pmap_rw_bit(pmap); - anychanged = FALSE; + anychanged = false; pmap_delayed_invl_start(); PMAP_LOCK(pmap); for (; sva < eva; sva = va_next) { @@ -7523,17 +7523,25 @@ pmap_advise(pmap_t pmap, vm_offset_t sva, vm_offset_t /* * Unless the page mappings are wired, remove the * mapping to a single page so that a subsequent - * access may repromote. Since the underlying page - * table page is fully populated, this removal never - * frees a page table page. + * access may repromote. Choosing the last page + * within the address range [sva, min(va_next, eva)) + * generally results in more repromotions. Since the + * underlying page table page is fully populated, this + * removal never frees a page table page. */ if ((oldpde & PG_W) == 0) { - pte = pmap_pde_to_pte(pde, sva); + va = eva; + if (va > va_next) + va = va_next; + va -= PAGE_SIZE; + KASSERT(va >= sva, + ("pmap_advise: no address gap")); + pte = pmap_pde_to_pte(pde, va); KASSERT((*pte & PG_V) != 0, ("pmap_advise: invalid PTE")); - pmap_remove_pte(pmap, pte, sva, *pde, NULL, + pmap_remove_pte(pmap, pte, va, *pde, NULL, &lock); - anychanged = TRUE; + anychanged = true; } if (lock != NULL) rw_wunlock(lock); @@ -7565,7 +7573,7 @@ pmap_advise(pmap_t pmap, vm_offset_t sva, vm_offset_t if (va == va_next) va = sva; } else - anychanged = TRUE; + anychanged = true; continue; maybe_invlrng: if (va != va_next) { Modified: stable/12/sys/arm64/arm64/pmap.c ============================================================================== --- stable/12/sys/arm64/arm64/pmap.c Wed Sep 18 07:24:33 2019 (r352474) +++ stable/12/sys/arm64/arm64/pmap.c Wed Sep 18 07:25:04 2019 (r352475) @@ -4882,15 +4882,23 @@ pmap_advise(pmap_t pmap, vm_offset_t sva, vm_offset_t /* * Unless the page mappings are wired, remove the * mapping to a single page so that a subsequent - * access may repromote. Since the underlying page - * table page is fully populated, this removal never - * frees a page table page. + * access may repromote. Choosing the last page + * within the address range [sva, min(va_next, eva)) + * generally results in more repromotions. Since the + * underlying page table page is fully populated, this + * removal never frees a page table page. */ if ((oldl2 & ATTR_SW_WIRED) == 0) { - l3 = pmap_l2_to_l3(l2, sva); + va = eva; + if (va > va_next) + va = va_next; + va -= PAGE_SIZE; + KASSERT(va >= sva, + ("pmap_advise: no address gap")); + l3 = pmap_l2_to_l3(l2, va); KASSERT(pmap_load(l3) != 0, ("pmap_advise: invalid PTE")); - pmap_remove_l3(pmap, l3, sva, pmap_load(l2), + pmap_remove_l3(pmap, l3, va, pmap_load(l2), NULL, &lock); } if (lock != NULL) Modified: stable/12/sys/i386/i386/pmap.c ============================================================================== --- stable/12/sys/i386/i386/pmap.c Wed Sep 18 07:24:33 2019 (r352474) +++ stable/12/sys/i386/i386/pmap.c Wed Sep 18 07:25:04 2019 (r352475) @@ -5246,19 +5246,19 @@ pmap_advise(pmap_t pmap, vm_offset_t sva, vm_offset_t pt_entry_t *pte; vm_offset_t va, pdnxt; vm_page_t m; - boolean_t anychanged, pv_lists_locked; + bool anychanged, pv_lists_locked; if (advice != MADV_DONTNEED && advice != MADV_FREE) return; if (pmap_is_current(pmap)) - pv_lists_locked = FALSE; + pv_lists_locked = false; else { - pv_lists_locked = TRUE; + pv_lists_locked = true; resume: rw_wlock(&pvh_global_lock); sched_pin(); } - anychanged = FALSE; + anychanged = false; PMAP_LOCK(pmap); for (; sva < eva; sva = pdnxt) { pdnxt = (sva + NBPDR) & ~PDRMASK; @@ -5272,7 +5272,7 @@ resume: if ((oldpde & PG_MANAGED) == 0) continue; if (!pv_lists_locked) { - pv_lists_locked = TRUE; + pv_lists_locked = true; if (!rw_try_wlock(&pvh_global_lock)) { if (anychanged) pmap_invalidate_all(pmap); @@ -5291,16 +5291,24 @@ resume: /* * Unless the page mappings are wired, remove the * mapping to a single page so that a subsequent - * access may repromote. Since the underlying page - * table page is fully populated, this removal never - * frees a page table page. + * access may repromote. Choosing the last page + * within the address range [sva, min(pdnxt, eva)) + * generally results in more repromotions. Since the + * underlying page table page is fully populated, this + * removal never frees a page table page. */ if ((oldpde & PG_W) == 0) { - pte = pmap_pte_quick(pmap, sva); + va = eva; + if (va > pdnxt) + va = pdnxt; + va -= PAGE_SIZE; + KASSERT(va >= sva, + ("pmap_advise: no address gap")); + pte = pmap_pte_quick(pmap, va); KASSERT((*pte & PG_V) != 0, ("pmap_advise: invalid PTE")); - pmap_remove_pte(pmap, pte, sva, NULL); - anychanged = TRUE; + pmap_remove_pte(pmap, pte, va, NULL); + anychanged = true; } } if (pdnxt > eva) @@ -5329,7 +5337,7 @@ resume: if (va == pdnxt) va = sva; } else - anychanged = TRUE; + anychanged = true; continue; maybe_invlrng: if (va != pdnxt) {