From owner-svn-src-stable@FreeBSD.ORG Sun Dec 4 18:55:19 2011 Return-Path: Delivered-To: svn-src-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB6EB1065672; Sun, 4 Dec 2011 18:55:19 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id B0FC58FC1E; Sun, 4 Dec 2011 18:55:19 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id pB4ItJ49098811; Sun, 4 Dec 2011 18:55:19 GMT (envelope-from alc@svn.freebsd.org) Received: (from alc@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id pB4ItJY3098809; Sun, 4 Dec 2011 18:55:19 GMT (envelope-from alc@svn.freebsd.org) Message-Id: <201112041855.pB4ItJY3098809@svn.freebsd.org> From: Alan Cox Date: Sun, 4 Dec 2011 18:55:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-8@freebsd.org X-SVN-Group: stable-8 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r228262 - stable/8/sys/amd64/amd64 X-BeenThere: svn-src-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for all the -stable branches of the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Dec 2011 18:55:20 -0000 Author: alc Date: Sun Dec 4 18:55:19 2011 New Revision: 228262 URL: http://svn.freebsd.org/changeset/base/228262 Log: MFC r219157 Make a change to the implementation of the direct map to improve performance on processors that support 1 GB pages. Specifically, if the end of physical memory is not aligned to a 1 GB page boundary, then map the residual physical memory with multiple 2 MB page mappings rather than a single 1 GB page mapping. When a 1 GB page mapping is used for this residual memory, access to the memory is slower than when multiple 2 MB page mappings are used. (I suspect that the reason for this slowdown is that the TLB is actually being loaded with 4 KB page mappings for the residual memory.) Modified: stable/8/sys/amd64/amd64/pmap.c Directory Properties: stable/8/sys/ (props changed) stable/8/sys/amd64/include/xen/ (props changed) stable/8/sys/cddl/contrib/opensolaris/ (props changed) stable/8/sys/contrib/dev/acpica/ (props changed) stable/8/sys/contrib/pf/ (props changed) Modified: stable/8/sys/amd64/amd64/pmap.c ============================================================================== --- stable/8/sys/amd64/amd64/pmap.c Sun Dec 4 18:43:09 2011 (r228261) +++ stable/8/sys/amd64/amd64/pmap.c Sun Dec 4 18:55:19 2011 (r228262) @@ -435,7 +435,7 @@ allocpages(vm_paddr_t *firstaddr, int n) static void create_pagetables(vm_paddr_t *firstaddr) { - int i; + int i, j, ndm1g; /* Allocate pages */ KPTphys = allocpages(firstaddr, NKPT); @@ -447,8 +447,11 @@ create_pagetables(vm_paddr_t *firstaddr) if (ndmpdp < 4) /* Minimum 4GB of dirmap */ ndmpdp = 4; DMPDPphys = allocpages(firstaddr, NDMPML4E); - if ((amd_feature & AMDID_PAGE1GB) == 0) - DMPDphys = allocpages(firstaddr, ndmpdp); + ndm1g = 0; + if ((amd_feature & AMDID_PAGE1GB) != 0) + ndm1g = ptoa(Maxmem) >> PDPSHIFT; + if (ndm1g < ndmpdp) + DMPDphys = allocpages(firstaddr, ndmpdp - ndm1g); dmaplimit = (vm_paddr_t)ndmpdp << PDPSHIFT; /* Fill in the underlying page table pages */ @@ -480,32 +483,28 @@ create_pagetables(vm_paddr_t *firstaddr) } /* - * Now, set up the direct map region using either 2MB or 1GB pages. - * Later, if pmap_mapdev{_attr}() uses the direct map for non-write- - * back memory, pmap_change_attr() will demote any 2MB or 1GB page - * mappings that are partially used. - */ - if ((amd_feature & AMDID_PAGE1GB) == 0) { - for (i = 0; i < NPDEPG * ndmpdp; i++) { - ((pd_entry_t *)DMPDphys)[i] = (vm_paddr_t)i << PDRSHIFT; - /* Preset PG_M and PG_A because demotion expects it. */ - ((pd_entry_t *)DMPDphys)[i] |= PG_RW | PG_V | PG_PS | - PG_G | PG_M | PG_A; - } - /* And the direct map space's PDP */ - for (i = 0; i < ndmpdp; i++) { - ((pdp_entry_t *)DMPDPphys)[i] = DMPDphys + - (i << PAGE_SHIFT); - ((pdp_entry_t *)DMPDPphys)[i] |= PG_RW | PG_V | PG_U; - } - } else { - for (i = 0; i < ndmpdp; i++) { - ((pdp_entry_t *)DMPDPphys)[i] = - (vm_paddr_t)i << PDPSHIFT; - /* Preset PG_M and PG_A because demotion expects it. */ - ((pdp_entry_t *)DMPDPphys)[i] |= PG_RW | PG_V | PG_PS | - PG_G | PG_M | PG_A; - } + * Now, set up the direct map region using 2MB and/or 1GB pages. If + * the end of physical memory is not aligned to a 1GB page boundary, + * then the residual physical memory is mapped with 2MB pages. Later, + * if pmap_mapdev{_attr}() uses the direct map for non-write-back + * memory, pmap_change_attr() will demote any 2MB or 1GB page mappings + * that are partially used. + */ + for (i = NPDEPG * ndm1g, j = 0; i < NPDEPG * ndmpdp; i++, j++) { + ((pd_entry_t *)DMPDphys)[j] = (vm_paddr_t)i << PDRSHIFT; + /* Preset PG_M and PG_A because demotion expects it. */ + ((pd_entry_t *)DMPDphys)[j] |= PG_RW | PG_V | PG_PS | PG_G | + PG_M | PG_A; + } + for (i = 0; i < ndm1g; i++) { + ((pdp_entry_t *)DMPDPphys)[i] = (vm_paddr_t)i << PDPSHIFT; + /* Preset PG_M and PG_A because demotion expects it. */ + ((pdp_entry_t *)DMPDPphys)[i] |= PG_RW | PG_V | PG_PS | PG_G | + PG_M | PG_A; + } + for (j = 0; i < ndmpdp; i++, j++) { + ((pdp_entry_t *)DMPDPphys)[i] = DMPDphys + (j << PAGE_SHIFT); + ((pdp_entry_t *)DMPDPphys)[i] |= PG_RW | PG_V | PG_U; } /* And recursively map PML4 to itself in order to get PTmap */