From owner-svn-src-stable@freebsd.org Thu Sep 19 15:12:33 2019 Return-Path: Delivered-To: svn-src-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 917BD126046; Thu, 19 Sep 2019 15:12:33 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46Z0hK3PgXz3Mk0; Thu, 19 Sep 2019 15:12:33 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 57F4DF582; Thu, 19 Sep 2019 15:12:33 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x8JFCX7B035581; Thu, 19 Sep 2019 15:12:33 GMT (envelope-from alc@FreeBSD.org) Received: (from alc@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x8JFCW3H035578; Thu, 19 Sep 2019 15:12:32 GMT (envelope-from alc@FreeBSD.org) Message-Id: <201909191512.x8JFCW3H035578@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: alc set sender to alc@FreeBSD.org using -f From: Alan Cox Date: Thu, 19 Sep 2019 15:12:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r352519 - in stable/12/sys: amd64/amd64 arm64/arm64 i386/i386 riscv/riscv X-SVN-Group: stable-12 X-SVN-Commit-Author: alc X-SVN-Commit-Paths: in stable/12/sys: amd64/amd64 arm64/arm64 i386/i386 riscv/riscv X-SVN-Commit-Revision: 352519 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for all the -stable branches of the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Sep 2019 15:12:33 -0000 Author: alc Date: Thu Sep 19 15:12:32 2019 New Revision: 352519 URL: https://svnweb.freebsd.org/changeset/base/352519 Log: MFC r350335: Simplify the handling of superpages in pmap_clear_modify(). Specifically, if a demotion succeeds, then all of the 4KB page mappings within the superpage-sized region must be valid, so there is no point in testing the validity of the 4KB page mapping that is going to be write protected. Deindent the nearby code. Modified: stable/12/sys/amd64/amd64/pmap.c stable/12/sys/arm64/arm64/pmap.c stable/12/sys/i386/i386/pmap.c stable/12/sys/riscv/riscv/pmap.c Directory Properties: stable/12/ (props changed) Modified: stable/12/sys/amd64/amd64/pmap.c ============================================================================== --- stable/12/sys/amd64/amd64/pmap.c Thu Sep 19 14:45:04 2019 (r352518) +++ stable/12/sys/amd64/amd64/pmap.c Thu Sep 19 15:12:32 2019 (r352519) @@ -7599,7 +7599,7 @@ pmap_clear_modify(vm_page_t m) pmap_t pmap; pv_entry_t next_pv, pv; pd_entry_t oldpde, *pde; - pt_entry_t oldpte, *pte, PG_M, PG_RW, PG_V; + pt_entry_t *pte, PG_M, PG_RW; struct rwlock *lock; vm_offset_t va; int md_gen, pvh_gen; @@ -7635,33 +7635,23 @@ restart: } } PG_M = pmap_modified_bit(pmap); - PG_V = pmap_valid_bit(pmap); PG_RW = pmap_rw_bit(pmap); va = pv->pv_va; pde = pmap_pde(pmap, va); oldpde = *pde; - if ((oldpde & PG_RW) != 0) { - if (pmap_demote_pde_locked(pmap, pde, va, &lock)) { - if ((oldpde & PG_W) == 0) { - /* - * Write protect the mapping to a - * single page so that a subsequent - * write access may repromote. - */ - va += VM_PAGE_TO_PHYS(m) - (oldpde & - PG_PS_FRAME); - pte = pmap_pde_to_pte(pde, va); - oldpte = *pte; - if ((oldpte & PG_V) != 0) { - while (!atomic_cmpset_long(pte, - oldpte, - oldpte & ~(PG_M | PG_RW))) - oldpte = *pte; - vm_page_dirty(m); - pmap_invalidate_page(pmap, va); - } - } - } + /* If oldpde has PG_RW set, then it also has PG_M set. */ + if ((oldpde & PG_RW) != 0 && + pmap_demote_pde_locked(pmap, pde, va, &lock) && + (oldpde & PG_W) == 0) { + /* + * Write protect the mapping to a single page so that + * a subsequent write access may repromote. + */ + va += VM_PAGE_TO_PHYS(m) - (oldpde & PG_PS_FRAME); + pte = pmap_pde_to_pte(pde, va); + atomic_clear_long(pte, PG_M | PG_RW); + vm_page_dirty(m); + pmap_invalidate_page(pmap, va); } PMAP_UNLOCK(pmap); } Modified: stable/12/sys/arm64/arm64/pmap.c ============================================================================== --- stable/12/sys/arm64/arm64/pmap.c Thu Sep 19 14:45:04 2019 (r352518) +++ stable/12/sys/arm64/arm64/pmap.c Thu Sep 19 15:12:32 2019 (r352519) @@ -5008,28 +5008,22 @@ restart: va = pv->pv_va; l2 = pmap_l2(pmap, va); oldl2 = pmap_load(l2); - if ((oldl2 & ATTR_SW_DBM) != 0) { - if (pmap_demote_l2_locked(pmap, l2, va, &lock)) { - if ((oldl2 & ATTR_SW_WIRED) == 0) { - /* - * Write protect the mapping to a - * single page so that a subsequent - * write access may repromote. - */ - va += VM_PAGE_TO_PHYS(m) - - (oldl2 & ~ATTR_MASK); - l3 = pmap_l2_to_l3(l2, va); - oldl3 = pmap_load(l3); - if (pmap_l3_valid(oldl3)) { - while (!atomic_fcmpset_long(l3, - &oldl3, (oldl3 & ~ATTR_SW_DBM) | - ATTR_AP(ATTR_AP_RO))) - cpu_spinwait(); - vm_page_dirty(m); - pmap_invalidate_page(pmap, va); - } - } - } + /* If oldl2 has ATTR_SW_DBM set, then it is also dirty. */ + if ((oldl2 & ATTR_SW_DBM) != 0 && + pmap_demote_l2_locked(pmap, l2, va, &lock) && + (oldl2 & ATTR_SW_WIRED) == 0) { + /* + * Write protect the mapping to a single page so that + * a subsequent write access may repromote. + */ + va += VM_PAGE_TO_PHYS(m) - (oldl2 & ~ATTR_MASK); + l3 = pmap_l2_to_l3(l2, va); + oldl3 = pmap_load(l3); + while (!atomic_fcmpset_long(l3, &oldl3, + (oldl3 & ~ATTR_SW_DBM) | ATTR_AP(ATTR_AP_RO))) + cpu_spinwait(); + vm_page_dirty(m); + pmap_invalidate_page(pmap, va); } PMAP_UNLOCK(pmap); } Modified: stable/12/sys/i386/i386/pmap.c ============================================================================== --- stable/12/sys/i386/i386/pmap.c Thu Sep 19 14:45:04 2019 (r352518) +++ stable/12/sys/i386/i386/pmap.c Thu Sep 19 15:12:32 2019 (r352519) @@ -5366,7 +5366,7 @@ pmap_clear_modify(vm_page_t m) pv_entry_t next_pv, pv; pmap_t pmap; pd_entry_t oldpde, *pde; - pt_entry_t oldpte, *pte; + pt_entry_t *pte; vm_offset_t va; KASSERT((m->oflags & VPO_UNMANAGED) == 0, @@ -5393,33 +5393,24 @@ pmap_clear_modify(vm_page_t m) PMAP_LOCK(pmap); pde = pmap_pde(pmap, va); oldpde = *pde; - if ((oldpde & PG_RW) != 0) { - if (pmap_demote_pde(pmap, pde, va)) { - if ((oldpde & PG_W) == 0) { - /* - * Write protect the mapping to a - * single page so that a subsequent - * write access may repromote. - */ - va += VM_PAGE_TO_PHYS(m) - (oldpde & - PG_PS_FRAME); - pte = pmap_pte_quick(pmap, va); - oldpte = *pte; - if ((oldpte & PG_V) != 0) { - /* - * Regardless of whether a pte is 32 or 64 bits - * in size, PG_RW and PG_M are among the least - * significant 32 bits. - */ - while (!atomic_cmpset_int((u_int *)pte, - oldpte, - oldpte & ~(PG_M | PG_RW))) - oldpte = *pte; - vm_page_dirty(m); - pmap_invalidate_page(pmap, va); - } - } - } + /* If oldpde has PG_RW set, then it also has PG_M set. */ + if ((oldpde & PG_RW) != 0 && + pmap_demote_pde(pmap, pde, va) && + (oldpde & PG_W) == 0) { + /* + * Write protect the mapping to a single page so that + * a subsequent write access may repromote. + */ + va += VM_PAGE_TO_PHYS(m) - (oldpde & PG_PS_FRAME); + pte = pmap_pte_quick(pmap, va); + /* + * Regardless of whether a pte is 32 or 64 bits + * in size, PG_RW and PG_M are among the least + * significant 32 bits. + */ + atomic_clear_int((u_int *)pte, PG_M | PG_RW); + vm_page_dirty(m); + pmap_invalidate_page(pmap, va); } PMAP_UNLOCK(pmap); } Modified: stable/12/sys/riscv/riscv/pmap.c ============================================================================== --- stable/12/sys/riscv/riscv/pmap.c Thu Sep 19 14:45:04 2019 (r352518) +++ stable/12/sys/riscv/riscv/pmap.c Thu Sep 19 15:12:32 2019 (r352519) @@ -4104,7 +4104,7 @@ pmap_clear_modify(vm_page_t m) pmap_t pmap; pv_entry_t next_pv, pv; pd_entry_t *l2, oldl2; - pt_entry_t *l3, oldl3; + pt_entry_t *l3; vm_offset_t va; int md_gen, pvh_gen; @@ -4142,28 +4142,19 @@ restart: va = pv->pv_va; l2 = pmap_l2(pmap, va); oldl2 = pmap_load(l2); - if ((oldl2 & PTE_W) != 0) { - if (pmap_demote_l2_locked(pmap, l2, va, &lock)) { - if ((oldl2 & PTE_SW_WIRED) == 0) { - /* - * Write protect the mapping to a - * single page so that a subsequent - * write access may repromote. - */ - va += VM_PAGE_TO_PHYS(m) - - PTE_TO_PHYS(oldl2); - l3 = pmap_l2_to_l3(l2, va); - oldl3 = pmap_load(l3); - if ((oldl3 & PTE_V) != 0) { - while (!atomic_fcmpset_long(l3, - &oldl3, oldl3 & ~(PTE_D | - PTE_W))) - cpu_spinwait(); - vm_page_dirty(m); - pmap_invalidate_page(pmap, va); - } - } - } + /* If oldl2 has PTE_W set, then it also has PTE_D set. */ + if ((oldl2 & PTE_W) != 0 && + pmap_demote_l2_locked(pmap, l2, va, &lock) && + (oldl2 & PTE_SW_WIRED) == 0) { + /* + * Write protect the mapping to a single page so that + * a subsequent write access may repromote. + */ + va += VM_PAGE_TO_PHYS(m) - PTE_TO_PHYS(oldl2); + l3 = pmap_l2_to_l3(l2, va); + pmap_clear_bits(l3, PTE_D | PTE_W); + vm_page_dirty(m); + pmap_invalidate_page(pmap, va); } PMAP_UNLOCK(pmap); }