From owner-svn-src-head@FreeBSD.ORG Tue Jul 15 13:52:55 2014 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43B8D2D5; Tue, 15 Jul 2014 13:52:55 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E96B92896; Tue, 15 Jul 2014 13:52:54 +0000 (UTC) Received: from c-50-155-136-3.hsd1.co.comcast.net ([50.155.136.3] helo=ilsoft.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1X739a-0007ko-LP; Tue, 15 Jul 2014 13:52:22 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id s6FDqLTW011024; Tue, 15 Jul 2014 07:52:21 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 50.155.136.3 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1+x5prJxNMOj7i/9cySOU1k X-Authentication-Warning: paranoia.hippie.lan: Host revolution.hippie.lan [172.22.42.240] claimed to be [172.22.42.240] Subject: Re: svn commit: r268660 - head/sys/amd64/amd64 From: Ian Lepore To: Konstantin Belousov In-Reply-To: <201407150930.s6F9Uhgi052394@svn.freebsd.org> References: <201407150930.s6F9Uhgi052394@svn.freebsd.org> Content-Type: text/plain; charset="us-ascii" Date: Tue, 15 Jul 2014 07:52:20 -0600 Message-ID: <1405432340.1312.26.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 13:52:55 -0000 On Tue, 2014-07-15 at 09:30 +0000, Konstantin Belousov wrote: > Author: kib > Date: Tue Jul 15 09:30:43 2014 > New Revision: 268660 > URL: http://svnweb.freebsd.org/changeset/base/268660 > > Log: > Make amd64 pmap_copy_pages() functional for pages not mapped by DMAP. > > Requested and reviewed by: royger > Tested by: pho, royger > Sponsored by: The FreeBSD Foundation > MFC after: 1 week > > Modified: > head/sys/amd64/amd64/pmap.c > > Modified: head/sys/amd64/amd64/pmap.c > ============================================================================== > --- head/sys/amd64/amd64/pmap.c Tue Jul 15 05:45:50 2014 (r268659) > +++ head/sys/amd64/amd64/pmap.c Tue Jul 15 09:30:43 2014 (r268660) > @@ -390,6 +390,11 @@ SYSCTL_PROC(_vm_pmap, OID_AUTO, pcid_sav > CTLFLAG_MPSAFE, NULL, 0, pmap_pcid_save_cnt_proc, "QU", > "Count of saved TLB context on switch"); > > +/* pmap_copy_pages() over non-DMAP */ > +static struct mtx cpage_lock; > +static vm_offset_t cpage_a; > +static vm_offset_t cpage_b; > + > /* > * Crashdump maps. > */ > @@ -1055,6 +1060,10 @@ pmap_init(void) > M_WAITOK | M_ZERO); > for (i = 0; i < pv_npg; i++) > TAILQ_INIT(&pv_table[i].pv_list); > + > + mtx_init(&cpage_lock, "cpage", NULL, MTX_DEF); > + cpage_a = kva_alloc(PAGE_SIZE); > + cpage_b = kva_alloc(PAGE_SIZE); > } > > static SYSCTL_NODE(_vm_pmap, OID_AUTO, pde, CTLFLAG_RD, 0, > @@ -5064,19 +5073,58 @@ pmap_copy_pages(vm_page_t ma[], vm_offse > vm_offset_t b_offset, int xfersize) > { > void *a_cp, *b_cp; > + vm_page_t m_a, m_b; > + vm_paddr_t p_a, p_b; > + pt_entry_t *pte; > vm_offset_t a_pg_offset, b_pg_offset; > int cnt; > + boolean_t pinned; > > + pinned = FALSE; > while (xfersize > 0) { > a_pg_offset = a_offset & PAGE_MASK; > - cnt = min(xfersize, PAGE_SIZE - a_pg_offset); > - a_cp = (char *)PHYS_TO_DMAP(ma[a_offset >> PAGE_SHIFT]-> > - phys_addr) + a_pg_offset; > + m_a = ma[a_offset >> PAGE_SHIFT]; > + p_a = m_a->phys_addr; > b_pg_offset = b_offset & PAGE_MASK; > + m_b = mb[b_offset >> PAGE_SHIFT]; > + p_b = m_b->phys_addr; > + cnt = min(xfersize, PAGE_SIZE - a_pg_offset); > cnt = min(cnt, PAGE_SIZE - b_pg_offset); > - b_cp = (char *)PHYS_TO_DMAP(mb[b_offset >> PAGE_SHIFT]-> > - phys_addr) + b_pg_offset; > + if (__predict_false(p_a < DMAP_MIN_ADDRESS || > + p_a > DMAP_MIN_ADDRESS + dmaplimit)) { > + mtx_lock(&cpage_lock); > + sched_pin(); > + pinned = TRUE; > + pte = vtopte(cpage_a); > + *pte = p_a | X86_PG_A | X86_PG_V | > + pmap_cache_bits(kernel_pmap, m_a->md.pat_mode, 0); > + invlpg(cpage_a); > + a_cp = (char *)cpage_a + a_pg_offset; > + } else { > + a_cp = (char *)PHYS_TO_DMAP(p_a) + a_pg_offset; > + } > + if (__predict_false(p_b < DMAP_MIN_ADDRESS || > + p_b > DMAP_MIN_ADDRESS + dmaplimit)) { > + if (!pinned) { > + mtx_lock(&cpage_lock); > + sched_pin(); > + pinned = TRUE; > + } > + pte = vtopte(cpage_b); > + *pte = p_b | X86_PG_A | X86_PG_M | X86_PG_RW | > + X86_PG_V | pmap_cache_bits(kernel_pmap, > + m_b->md.pat_mode, 0); > + invlpg(cpage_b); > + b_cp = (char *)cpage_b + b_pg_offset; > + } else { > + b_cp = (char *)PHYS_TO_DMAP(p_b) + b_pg_offset; > + } > bcopy(a_cp, b_cp, cnt); > + if (__predict_false(pinned)) { > + sched_unpin(); > + mtx_unlock(&cpage_lock); > + pinned = FALSE; Should this pinned = FALSE be done under the cpage_lock to avoid a race? -- Ian > + } > a_offset += cnt; > b_offset += cnt; > xfersize -= cnt; >