From owner-svn-src-head@freebsd.org Thu Feb 8 19:28:53 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 83F30F1E1BF; Thu, 8 Feb 2018 19:28:53 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 363DC780E2; Thu, 8 Feb 2018 19:28:53 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 311271A4A6; Thu, 8 Feb 2018 19:28:53 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w18JSrvl098592; Thu, 8 Feb 2018 19:28:53 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w18JSq0a098582; Thu, 8 Feb 2018 19:28:52 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201802081928.w18JSq0a098582@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Thu, 8 Feb 2018 19:28:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r329023 - in head/sys: amd64/amd64 arm64/arm64 i386/i386 mips/mips powerpc/powerpc riscv/riscv sparc64/sparc64 X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: in head/sys: amd64/amd64 arm64/arm64 i386/i386 mips/mips powerpc/powerpc riscv/riscv sparc64/sparc64 X-SVN-Commit-Revision: 329023 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Feb 2018 19:28:53 -0000 Author: markj Date: Thu Feb 8 19:28:51 2018 New Revision: 329023 URL: https://svnweb.freebsd.org/changeset/base/329023 Log: Use vm_page_unwire_noq() instead of directly modifying page wire counts. No functional change intended. Reviewed by: alc, kib (previous revision) MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D14266 Modified: head/sys/amd64/amd64/pmap.c head/sys/amd64/amd64/uma_machdep.c head/sys/arm64/arm64/pmap.c head/sys/arm64/arm64/uma_machdep.c head/sys/i386/i386/pmap.c head/sys/mips/mips/pmap.c head/sys/mips/mips/uma_machdep.c head/sys/powerpc/powerpc/uma_machdep.c head/sys/riscv/riscv/pmap.c head/sys/sparc64/sparc64/vm_machdep.c Modified: head/sys/amd64/amd64/pmap.c ============================================================================== --- head/sys/amd64/amd64/pmap.c Thu Feb 8 19:26:52 2018 (r329022) +++ head/sys/amd64/amd64/pmap.c Thu Feb 8 19:28:51 2018 (r329023) @@ -2723,8 +2723,7 @@ _pmap_allocpte(pmap_t pmap, vm_pindex_t ptepindex, str /* Have to allocate a new pdp, recurse */ if (_pmap_allocpte(pmap, NUPDE + NUPDPE + pml4index, lockp) == NULL) { - --m->wire_count; - atomic_subtract_int(&vm_cnt.v_wire_count, 1); + vm_page_unwire_noq(m); vm_page_free_zero(m); return (NULL); } @@ -2756,8 +2755,7 @@ _pmap_allocpte(pmap_t pmap, vm_pindex_t ptepindex, str /* Have to allocate a new pd, recurse */ if (_pmap_allocpte(pmap, NUPDE + pdpindex, lockp) == NULL) { - --m->wire_count; - atomic_subtract_int(&vm_cnt.v_wire_count, 1); + vm_page_unwire_noq(m); vm_page_free_zero(m); return (NULL); } @@ -2770,9 +2768,7 @@ _pmap_allocpte(pmap_t pmap, vm_pindex_t ptepindex, str /* Have to allocate a new pd, recurse */ if (_pmap_allocpte(pmap, NUPDE + pdpindex, lockp) == NULL) { - --m->wire_count; - atomic_subtract_int(&vm_cnt.v_wire_count, - 1); + vm_page_unwire_noq(m); vm_page_free_zero(m); return (NULL); } @@ -2904,18 +2900,16 @@ pmap_release(pmap_t pmap) pmap->pm_pml4[DMPML4I + i] = 0; pmap->pm_pml4[PML4PML4I] = 0; /* Recursive Mapping */ - m->wire_count--; - atomic_subtract_int(&vm_cnt.v_wire_count, 1); + vm_page_unwire_noq(m); vm_page_free_zero(m); if (pmap->pm_pml4u != NULL) { m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((vm_offset_t)pmap->pm_pml4u)); - m->wire_count--; - atomic_subtract_int(&vm_cnt.v_wire_count, 1); + vm_page_unwire_noq(m); vm_page_free(m); } } - + static int kvm_size(SYSCTL_HANDLER_ARGS) { @@ -7711,10 +7705,8 @@ pmap_pti_free_page(vm_page_t m) { KASSERT(m->wire_count > 0, ("page %p not wired", m)); - m->wire_count--; - if (m->wire_count != 0) + if (!vm_page_unwire_noq(m)) return (false); - atomic_subtract_int(&vm_cnt.v_wire_count, 1); vm_page_free_zero(m); return (true); } Modified: head/sys/amd64/amd64/uma_machdep.c ============================================================================== --- head/sys/amd64/amd64/uma_machdep.c Thu Feb 8 19:26:52 2018 (r329022) +++ head/sys/amd64/amd64/uma_machdep.c Thu Feb 8 19:28:51 2018 (r329023) @@ -74,7 +74,6 @@ uma_small_free(void *mem, vm_size_t size, u_int8_t fla pa = DMAP_TO_PHYS((vm_offset_t)mem); dump_drop_page(pa); m = PHYS_TO_VM_PAGE(pa); - m->wire_count--; + vm_page_unwire_noq(m); vm_page_free(m); - atomic_subtract_int(&vm_cnt.v_wire_count, 1); } Modified: head/sys/arm64/arm64/pmap.c ============================================================================== --- head/sys/arm64/arm64/pmap.c Thu Feb 8 19:26:52 2018 (r329022) +++ head/sys/arm64/arm64/pmap.c Thu Feb 8 19:28:51 2018 (r329023) @@ -1362,12 +1362,7 @@ _pmap_unwire_l3(pmap_t pmap, vm_offset_t va, vm_page_t } pmap_invalidate_page(pmap, va); - /* - * This is a release store so that the ordinary store unmapping - * the page table page is globally performed before TLB shoot- - * down is begun. - */ - atomic_subtract_rel_int(&vm_cnt.v_wire_count, 1); + atomic_subtract_int(&vm_cnt.v_wire_count, 1); /* * Put page on a list so that it is released after @@ -1493,9 +1488,7 @@ _pmap_alloc_l3(pmap_t pmap, vm_pindex_t ptepindex, str /* recurse for allocating page dir */ if (_pmap_alloc_l3(pmap, NUL2E + NUL1E + l0index, lockp) == NULL) { - --m->wire_count; - /* XXX: release mem barrier? */ - atomic_subtract_int(&vm_cnt.v_wire_count, 1); + vm_page_unwire_noq(m); vm_page_free_zero(m); return (NULL); } @@ -1521,8 +1514,7 @@ _pmap_alloc_l3(pmap_t pmap, vm_pindex_t ptepindex, str /* recurse for allocating page dir */ if (_pmap_alloc_l3(pmap, NUL2E + l1index, lockp) == NULL) { - --m->wire_count; - atomic_subtract_int(&vm_cnt.v_wire_count, 1); + vm_page_unwire_noq(m); vm_page_free_zero(m); return (NULL); } @@ -1537,10 +1529,7 @@ _pmap_alloc_l3(pmap_t pmap, vm_pindex_t ptepindex, str /* recurse for allocating page dir */ if (_pmap_alloc_l3(pmap, NUL2E + l1index, lockp) == NULL) { - --m->wire_count; - /* XXX: release mem barrier? */ - atomic_subtract_int( - &vm_cnt.v_wire_count, 1); + vm_page_unwire_noq(m); vm_page_free_zero(m); return (NULL); } @@ -1648,8 +1637,7 @@ pmap_release(pmap_t pmap) m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((vm_offset_t)pmap->pm_l0)); - m->wire_count--; - atomic_subtract_int(&vm_cnt.v_wire_count, 1); + vm_page_unwire_noq(m); vm_page_free_zero(m); } Modified: head/sys/arm64/arm64/uma_machdep.c ============================================================================== --- head/sys/arm64/arm64/uma_machdep.c Thu Feb 8 19:26:52 2018 (r329022) +++ head/sys/arm64/arm64/uma_machdep.c Thu Feb 8 19:28:51 2018 (r329023) @@ -72,7 +72,6 @@ uma_small_free(void *mem, vm_size_t size, u_int8_t fla pa = DMAP_TO_PHYS((vm_offset_t)mem); dump_drop_page(pa); m = PHYS_TO_VM_PAGE(pa); - m->wire_count--; + vm_page_unwire_noq(m); vm_page_free(m); - atomic_subtract_int(&vm_cnt.v_wire_count, 1); } Modified: head/sys/i386/i386/pmap.c ============================================================================== --- head/sys/i386/i386/pmap.c Thu Feb 8 19:26:52 2018 (r329022) +++ head/sys/i386/i386/pmap.c Thu Feb 8 19:28:51 2018 (r329023) @@ -2057,10 +2057,9 @@ pmap_release(pmap_t pmap) KASSERT(VM_PAGE_TO_PHYS(m) == (pmap->pm_pdpt[i] & PG_FRAME), ("pmap_release: got wrong ptd page")); #endif - m->wire_count--; + vm_page_unwire_noq(m); vm_page_free_zero(m); } - atomic_subtract_int(&vm_cnt.v_wire_count, NPGPTD); } static int Modified: head/sys/mips/mips/pmap.c ============================================================================== --- head/sys/mips/mips/pmap.c Thu Feb 8 19:26:52 2018 (r329022) +++ head/sys/mips/mips/pmap.c Thu Feb 8 19:28:51 2018 (r329023) @@ -1159,8 +1159,7 @@ _pmap_allocpte(pmap_t pmap, unsigned ptepindex, u_int if (_pmap_allocpte(pmap, NUPDE + segindex, flags) == NULL) { /* alloc failed, release current */ - --m->wire_count; - atomic_subtract_int(&vm_cnt.v_wire_count, 1); + vm_page_unwire_noq(m); vm_page_free_zero(m); return (NULL); } @@ -1238,8 +1237,7 @@ pmap_release(pmap_t pmap) ptdva = (vm_offset_t)pmap->pm_segtab; ptdpg = PHYS_TO_VM_PAGE(MIPS_DIRECT_TO_PHYS(ptdva)); - ptdpg->wire_count--; - atomic_subtract_int(&vm_cnt.v_wire_count, 1); + vm_page_unwire_noq(ptdpg); vm_page_free_zero(ptdpg); } Modified: head/sys/mips/mips/uma_machdep.c ============================================================================== --- head/sys/mips/mips/uma_machdep.c Thu Feb 8 19:26:52 2018 (r329022) +++ head/sys/mips/mips/uma_machdep.c Thu Feb 8 19:28:51 2018 (r329023) @@ -94,7 +94,6 @@ uma_small_free(void *mem, vm_size_t size, u_int8_t fla pa = MIPS_DIRECT_TO_PHYS((vm_offset_t)mem); dump_drop_page(pa); m = PHYS_TO_VM_PAGE(pa); - m->wire_count--; + vm_page_unwire_noq(m); vm_page_free(m); - atomic_subtract_int(&vm_cnt.v_wire_count, 1); } Modified: head/sys/powerpc/powerpc/uma_machdep.c ============================================================================== --- head/sys/powerpc/powerpc/uma_machdep.c Thu Feb 8 19:26:52 2018 (r329022) +++ head/sys/powerpc/powerpc/uma_machdep.c Thu Feb 8 19:28:51 2018 (r329023) @@ -95,8 +95,7 @@ uma_small_free(void *mem, vm_size_t size, u_int8_t fla (vm_offset_t)mem + PAGE_SIZE); m = PHYS_TO_VM_PAGE((vm_offset_t)mem); - m->wire_count--; + vm_page_unwire_noq(m); vm_page_free(m); - atomic_subtract_int(&vm_cnt.v_wire_count, 1); atomic_subtract_int(&hw_uma_mdpages, 1); } Modified: head/sys/riscv/riscv/pmap.c ============================================================================== --- head/sys/riscv/riscv/pmap.c Thu Feb 8 19:26:52 2018 (r329022) +++ head/sys/riscv/riscv/pmap.c Thu Feb 8 19:28:51 2018 (r329023) @@ -1153,12 +1153,7 @@ _pmap_unwire_l3(pmap_t pmap, vm_offset_t va, vm_page_t } pmap_invalidate_page(pmap, va); - /* - * This is a release store so that the ordinary store unmapping - * the page table page is globally performed before TLB shoot- - * down is begun. - */ - atomic_subtract_rel_int(&vm_cnt.v_wire_count, 1); + atomic_subtract_int(&vm_cnt.v_wire_count, 1); /* * Put page on a list so that it is released after @@ -1302,8 +1297,7 @@ _pmap_alloc_l3(pmap_t pmap, vm_pindex_t ptepindex, str /* recurse for allocating page dir */ if (_pmap_alloc_l3(pmap, NUPDE + l1index, lockp) == NULL) { - --m->wire_count; - atomic_subtract_int(&vm_cnt.v_wire_count, 1); + vm_page_unwire_noq(m); vm_page_free_zero(m); return (NULL); } @@ -1388,8 +1382,7 @@ pmap_release(pmap_t pmap) pmap->pm_stats.resident_count)); m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((vm_offset_t)pmap->pm_l1)); - m->wire_count--; - atomic_subtract_int(&vm_cnt.v_wire_count, 1); + vm_page_unwire_noq(m); vm_page_free_zero(m); /* Remove pmap from the allpmaps list */ Modified: head/sys/sparc64/sparc64/vm_machdep.c ============================================================================== --- head/sys/sparc64/sparc64/vm_machdep.c Thu Feb 8 19:26:52 2018 (r329022) +++ head/sys/sparc64/sparc64/vm_machdep.c Thu Feb 8 19:28:51 2018 (r329023) @@ -429,9 +429,8 @@ uma_small_free(void *mem, vm_size_t size, u_int8_t fla PMAP_STATS_INC(uma_nsmall_free); m = PHYS_TO_VM_PAGE(TLB_DIRECT_TO_PHYS((vm_offset_t)mem)); - m->wire_count--; + vm_page_unwire_noq(m); vm_page_free(m); - atomic_subtract_int(&vm_cnt.v_wire_count, 1); } void