From owner-svn-src-head@freebsd.org Fri Aug 14 20:08:17 2015 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7333E9BA713; Fri, 14 Aug 2015 20:08:17 +0000 (UTC) (envelope-from jah@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 610D613C5; Fri, 14 Aug 2015 20:08:17 +0000 (UTC) (envelope-from jah@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.70]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t7EK8Hmj037330; Fri, 14 Aug 2015 20:08:17 GMT (envelope-from jah@FreeBSD.org) Received: (from jah@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t7EK8Hkt037329; Fri, 14 Aug 2015 20:08:17 GMT (envelope-from jah@FreeBSD.org) Message-Id: <201508142008.t7EK8Hkt037329@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jah set sender to jah@FreeBSD.org using -f From: "Jason A. Harmening" Date: Fri, 14 Aug 2015 20:08:17 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r286787 - head/sys/x86/x86 X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Aug 2015 20:08:17 -0000 Author: jah Date: Fri Aug 14 20:08:16 2015 New Revision: 286787 URL: https://svnweb.freebsd.org/changeset/base/286787 Log: Use pmap_quick_enter_page() to handle bouncing of unmapped buffers in the x86 busdma_bounce implementation. Also treat user buffers as unmapped. This allows two things: 1. Sync'ing bounced maps in non-sleepable contexts. The physcopy* calls previously used could sleep on sf_buf operations in some cases. 2. Sync'ing user buffers outside the context of the owning process Approved by: kib (mentor) Modified: head/sys/x86/x86/busdma_bounce.c Modified: head/sys/x86/x86/busdma_bounce.c ============================================================================== --- head/sys/x86/x86/busdma_bounce.c Fri Aug 14 18:38:39 2015 (r286786) +++ head/sys/x86/x86/busdma_bounce.c Fri Aug 14 20:08:16 2015 (r286787) @@ -79,7 +79,8 @@ struct bounce_page { vm_offset_t vaddr; /* kva of bounce buffer */ bus_addr_t busaddr; /* Physical address */ vm_offset_t datavaddr; /* kva of client data */ - bus_addr_t dataaddr; /* client physical address */ + vm_page_t datapage; /* physical page of client data */ + vm_offset_t dataoffs; /* page offset of client data */ bus_size_t datacount; /* client data count */ STAILQ_ENTRY(bounce_page) links; }; @@ -658,7 +659,7 @@ bounce_bus_dmamap_load_buffer(bus_dma_ta { bus_size_t sgsize, max_sgsize; bus_addr_t curaddr; - vm_offset_t vaddr; + vm_offset_t kvaddr, vaddr; int error; if (map == NULL) @@ -681,10 +682,13 @@ bounce_bus_dmamap_load_buffer(bus_dma_ta /* * Get the physical address for this segment. */ - if (pmap == kernel_pmap) + if (pmap == kernel_pmap) { curaddr = pmap_kextract(vaddr); - else + kvaddr = vaddr; + } else { curaddr = pmap_extract(pmap, vaddr); + kvaddr = 0; + } /* * Compute the segment size, and adjust counts. @@ -696,7 +700,7 @@ bounce_bus_dmamap_load_buffer(bus_dma_ta bus_dma_run_filter(&dmat->common, curaddr)) { sgsize = roundup2(sgsize, dmat->common.alignment); sgsize = MIN(sgsize, max_sgsize); - curaddr = add_bounce_page(dmat, map, vaddr, curaddr, + curaddr = add_bounce_page(dmat, map, kvaddr, curaddr, sgsize); } else { sgsize = MIN(sgsize, max_sgsize); @@ -757,6 +761,7 @@ bounce_bus_dmamap_sync(bus_dma_tag_t dma bus_dmasync_op_t op) { struct bounce_page *bpage; + vm_offset_t datavaddr, tempvaddr; if ((bpage = STAILQ_FIRST(&map->bpages)) == NULL) return; @@ -770,13 +775,19 @@ bounce_bus_dmamap_sync(bus_dma_tag_t dma if ((op & BUS_DMASYNC_PREWRITE) != 0) { while (bpage != NULL) { - if (bpage->datavaddr != 0) { - bcopy((void *)bpage->datavaddr, - (void *)bpage->vaddr, bpage->datacount); - } else { - physcopyout(bpage->dataaddr, - (void *)bpage->vaddr, bpage->datacount); + tempvaddr = 0; + datavaddr = bpage->datavaddr; + if (datavaddr == 0) { + tempvaddr = + pmap_quick_enter_page(bpage->datapage); + datavaddr = tempvaddr | bpage->dataoffs; } + + bcopy((void *)datavaddr, + (void *)bpage->vaddr, bpage->datacount); + + if (tempvaddr != 0) + pmap_quick_remove_page(tempvaddr); bpage = STAILQ_NEXT(bpage, links); } dmat->bounce_zone->total_bounced++; @@ -784,14 +795,19 @@ bounce_bus_dmamap_sync(bus_dma_tag_t dma if ((op & BUS_DMASYNC_POSTREAD) != 0) { while (bpage != NULL) { - if (bpage->datavaddr != 0) { - bcopy((void *)bpage->vaddr, - (void *)bpage->datavaddr, - bpage->datacount); - } else { - physcopyin((void *)bpage->vaddr, - bpage->dataaddr, bpage->datacount); + tempvaddr = 0; + datavaddr = bpage->datavaddr; + if (datavaddr == 0) { + tempvaddr = + pmap_quick_enter_page(bpage->datapage); + datavaddr = tempvaddr | bpage->dataoffs; } + + bcopy((void *)bpage->vaddr, + (void *)datavaddr, bpage->datacount); + + if (tempvaddr != 0) + pmap_quick_remove_page(tempvaddr); bpage = STAILQ_NEXT(bpage, links); } dmat->bounce_zone->total_bounced++; @@ -989,7 +1005,8 @@ add_bounce_page(bus_dma_tag_t dmat, bus_ bpage->busaddr |= addr & PAGE_MASK; } bpage->datavaddr = vaddr; - bpage->dataaddr = addr; + bpage->datapage = PHYS_TO_VM_PAGE(addr & ~PAGE_MASK); + bpage->dataoffs = addr & PAGE_MASK; bpage->datacount = size; STAILQ_INSERT_TAIL(&(map->bpages), bpage, links); return (bpage->busaddr);