From owner-svn-src-all@freebsd.org Sun Dec 6 17:39:14 2015 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DD6CB9A03AC; Sun, 6 Dec 2015 17:39:14 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 917B41D19; Sun, 6 Dec 2015 17:39:14 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id tB6HdDhi069903; Sun, 6 Dec 2015 17:39:13 GMT (envelope-from cem@FreeBSD.org) Received: (from cem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id tB6HdDWh069901; Sun, 6 Dec 2015 17:39:13 GMT (envelope-from cem@FreeBSD.org) Message-Id: <201512061739.tB6HdDWh069901@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: cem set sender to cem@FreeBSD.org using -f From: "Conrad E. Meyer" Date: Sun, 6 Dec 2015 17:39:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r291906 - in head/sys: amd64/amd64 i386/i386 X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Dec 2015 17:39:15 -0000 Author: cem Date: Sun Dec 6 17:39:13 2015 New Revision: 291906 URL: https://svnweb.freebsd.org/changeset/base/291906 Log: pmap_invalidate_range: For very large ranges, flush the whole TLB Typical TLBs have 40-512 entries available. At some point, iterating every single page in a requested invalidation range and issuing invlpg on it is more expensive than flushing the TLB and allowing it to reload on demand. Broadwell CPUs have 1536 L2 TLB entries, so I've picked the arbitrary number 4096 entries as a hueristic at which point we flush TLB rather than invalidating every single potential page. Reviewed by: alc Feedback from: jhb, kib MFC notes: Depends on r291688 Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D4280 Modified: head/sys/amd64/amd64/pmap.c head/sys/i386/i386/pmap.c Modified: head/sys/amd64/amd64/pmap.c ============================================================================== --- head/sys/amd64/amd64/pmap.c Sun Dec 6 17:11:23 2015 (r291905) +++ head/sys/amd64/amd64/pmap.c Sun Dec 6 17:39:13 2015 (r291906) @@ -1421,6 +1421,9 @@ pmap_invalidate_page(pmap_t pmap, vm_off sched_unpin(); } +/* 4k PTEs -- Chosen to exceed the total size of Broadwell L2 TLB */ +#define PMAP_INVLPG_THRESHOLD (4 * 1024 * PAGE_SIZE) + void pmap_invalidate_range(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { @@ -1428,6 +1431,11 @@ pmap_invalidate_range(pmap_t pmap, vm_of vm_offset_t addr; u_int cpuid, i; + if (eva - sva >= PMAP_INVLPG_THRESHOLD) { + pmap_invalidate_all(pmap); + return; + } + if (pmap_type_guest(pmap)) { pmap_invalidate_ept(pmap); return; Modified: head/sys/i386/i386/pmap.c ============================================================================== --- head/sys/i386/i386/pmap.c Sun Dec 6 17:11:23 2015 (r291905) +++ head/sys/i386/i386/pmap.c Sun Dec 6 17:39:13 2015 (r291906) @@ -1032,6 +1032,9 @@ pmap_invalidate_page(pmap_t pmap, vm_off sched_unpin(); } +/* 4k PTEs -- Chosen to exceed the total size of Broadwell L2 TLB */ +#define PMAP_INVLPG_THRESHOLD (4 * 1024 * PAGE_SIZE) + void pmap_invalidate_range(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { @@ -1039,6 +1042,11 @@ pmap_invalidate_range(pmap_t pmap, vm_of vm_offset_t addr; u_int cpuid; + if (eva - sva >= PMAP_INVLPG_THRESHOLD) { + pmap_invalidate_all(pmap); + return; + } + sched_pin(); if (pmap == kernel_pmap || !CPU_CMP(&pmap->pm_active, &all_cpus)) { for (addr = sva; addr < eva; addr += PAGE_SIZE)