From owner-svn-src-head@freebsd.org Wed Jun 21 18:23:29 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F644D95B82; Wed, 21 Jun 2017 18:23:29 +0000 (UTC) (envelope-from zbb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 456C6778E9; Wed, 21 Jun 2017 18:23:29 +0000 (UTC) (envelope-from zbb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v5LINSH0065648; Wed, 21 Jun 2017 18:23:28 GMT (envelope-from zbb@FreeBSD.org) Received: (from zbb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v5LINScf065647; Wed, 21 Jun 2017 18:23:28 GMT (envelope-from zbb@FreeBSD.org) Message-Id: <201706211823.v5LINScf065647@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: zbb set sender to zbb@FreeBSD.org using -f From: Zbigniew Bodek Date: Wed, 21 Jun 2017 18:23:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r320197 - head/sys/arm/arm X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2017 18:23:29 -0000 Author: zbb Date: Wed Jun 21 18:23:28 2017 New Revision: 320197 URL: https://svnweb.freebsd.org/changeset/base/320197 Log: Introduce support for DMA coherent ARM platforms - Inherit BUS_DMA_COHERENT flag from parent buses - Use cacheable memory attributes on dma coherent platform - Disable cache synchronization on coherent platform Changes are based on ARMv8 busdma code and commit r299683. Submitted by: Michal Mazur Obtained from: Semihalf Sponsored by: Stormshield Reviewed by: ian Differential revision: https://reviews.freebsd.org/D11201 Modified: head/sys/arm/arm/busdma_machdep-v6.c Modified: head/sys/arm/arm/busdma_machdep-v6.c ============================================================================== --- head/sys/arm/arm/busdma_machdep-v6.c Wed Jun 21 18:20:17 2017 (r320196) +++ head/sys/arm/arm/busdma_machdep-v6.c Wed Jun 21 18:23:28 2017 (r320197) @@ -491,6 +491,7 @@ bus_dma_tag_create(bus_dma_tag_t parent, bus_size_t al newtag->highaddr = MAX(parent->highaddr, newtag->highaddr); newtag->alignment = MAX(parent->alignment, newtag->alignment); newtag->flags |= parent->flags & BUS_DMA_COULD_BOUNCE; + newtag->flags |= parent->flags & BUS_DMA_COHERENT; if (newtag->boundary == 0) newtag->boundary = parent->boundary; else if (parent->boundary != 0) @@ -755,11 +756,19 @@ bus_dmamem_alloc(bus_dma_tag_t dmat, void **vaddr, int } map->flags = DMAMAP_DMAMEM_ALLOC; - /* Choose a busdma buffer allocator based on memory type flags. */ - if (flags & BUS_DMA_COHERENT) { + /* For coherent memory, set the map flag that disables sync ops. */ + if (flags & BUS_DMA_COHERENT) + map->flags |= DMAMAP_COHERENT; + + /* + * Choose a busdma buffer allocator based on memory type flags. + * If the tag's COHERENT flag is set, that means normal memory + * is already coherent, use the normal allocator. + */ + if ((flags & BUS_DMA_COHERENT) && + ((dmat->flags & BUS_DMA_COHERENT) == 0)) { memattr = VM_MEMATTR_UNCACHEABLE; ba = coherent_allocator; - map->flags |= DMAMAP_COHERENT; } else { memattr = VM_MEMATTR_DEFAULT; ba = standard_allocator; @@ -829,7 +838,8 @@ bus_dmamem_free(bus_dma_tag_t dmat, void *vaddr, bus_d struct busdma_bufzone *bufzone; busdma_bufalloc_t ba; - if (map->flags & DMAMAP_COHERENT) + if ((map->flags & DMAMAP_COHERENT) && + ((dmat->flags & BUS_DMA_COHERENT) == 0)) ba = coherent_allocator; else ba = standard_allocator; @@ -1030,7 +1040,7 @@ _bus_dmamap_load_phys(bus_dma_tag_t dmat, bus_dmamap_t sgsize = MIN(sgsize, PAGE_SIZE - (curaddr & PAGE_MASK)); curaddr = add_bounce_page(dmat, map, 0, curaddr, sgsize); - } else { + } else if ((dmat->flags & BUS_DMA_COHERENT) == 0) { if (map->sync_count > 0) sl_end = sl->paddr + sl->datacount; @@ -1144,7 +1154,7 @@ _bus_dmamap_load_buffer(bus_dma_tag_t dmat, bus_dmamap sgsize)) { curaddr = add_bounce_page(dmat, map, kvaddr, curaddr, sgsize); - } else { + } else if ((dmat->flags & BUS_DMA_COHERENT) == 0) { if (map->sync_count > 0) { sl_pend = sl->paddr + sl->datacount; sl_vend = sl->vaddr + sl->datacount; @@ -1353,8 +1363,9 @@ _bus_dmamap_sync(bus_dma_tag_t dmat, bus_dmamap_t map, bpage->datacount); if (tempvaddr != 0) pmap_quick_remove_page(tempvaddr); - dcache_wb_poc(bpage->vaddr, bpage->busaddr, - bpage->datacount); + if ((dmat->flags & BUS_DMA_COHERENT) == 0) + dcache_wb_poc(bpage->vaddr, + bpage->busaddr, bpage->datacount); bpage = STAILQ_NEXT(bpage, links); } dmat->bounce_zone->total_bounced++; @@ -1374,8 +1385,9 @@ _bus_dmamap_sync(bus_dma_tag_t dmat, bus_dmamap_t map, if ((op & BUS_DMASYNC_PREREAD) && !(op & BUS_DMASYNC_PREWRITE)) { bpage = STAILQ_FIRST(&map->bpages); while (bpage != NULL) { - dcache_inv_poc_dma(bpage->vaddr, bpage->busaddr, - bpage->datacount); + if ((dmat->flags & BUS_DMA_COHERENT) == 0) + dcache_inv_poc_dma(bpage->vaddr, + bpage->busaddr, bpage->datacount); bpage = STAILQ_NEXT(bpage, links); } } @@ -1391,8 +1403,9 @@ _bus_dmamap_sync(bus_dma_tag_t dmat, bus_dmamap_t map, */ if (op & BUS_DMASYNC_POSTREAD) { while (bpage != NULL) { - dcache_inv_poc(bpage->vaddr, bpage->busaddr, - bpage->datacount); + if ((dmat->flags & BUS_DMA_COHERENT) == 0) + dcache_inv_poc(bpage->vaddr, + bpage->busaddr, bpage->datacount); tempvaddr = 0; datavaddr = bpage->datavaddr; if (datavaddr == 0) { @@ -1421,7 +1434,8 @@ _bus_dmamap_sync(bus_dma_tag_t dmat, bus_dmamap_t map, if (map->flags & DMAMAP_COHERENT) { if (op & BUS_DMASYNC_PREWRITE) { dsb(); - cpu_l2cache_drain_writebuf(); + if ((dmat->flags & BUS_DMA_COHERENT) == 0) + cpu_l2cache_drain_writebuf(); } return; }