From owner-svn-src-head@freebsd.org Thu Dec 29 20:11:51 2016 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4DB11C9633D; Thu, 29 Dec 2016 20:11:51 +0000 (UTC) (envelope-from kan@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 07A991F1F; Thu, 29 Dec 2016 20:11:50 +0000 (UTC) (envelope-from kan@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id uBTKBobQ080979; Thu, 29 Dec 2016 20:11:50 GMT (envelope-from kan@FreeBSD.org) Received: (from kan@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id uBTKBofm080964; Thu, 29 Dec 2016 20:11:50 GMT (envelope-from kan@FreeBSD.org) Message-Id: <201612292011.uBTKBofm080964@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kan set sender to kan@FreeBSD.org using -f From: Alexander Kabaev Date: Thu, 29 Dec 2016 20:11:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r310784 - head/sys/mips/mips X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2016 20:11:51 -0000 Author: kan Date: Thu Dec 29 20:11:50 2016 New Revision: 310784 URL: https://svnweb.freebsd.org/changeset/base/310784 Log: Use mips_dcache_wbinv_range instead of mips_dcache_wb_range on CPU_XBURST Ingenic CPUs treat plain cache writeback as local-only operation and do nothing if that is a remote CPU that holds the dirty cache line. They do broadcast invalidate and write-and-invalidate to other cores though, so take advantage of that and use wbinv in place of wb as this still gives us required busdma semantics. Otherwise we'd have to do IPI to remote CPU ourselves. Modified: head/sys/mips/mips/busdma_machdep.c Modified: head/sys/mips/mips/busdma_machdep.c ============================================================================== --- head/sys/mips/mips/busdma_machdep.c Thu Dec 29 19:57:46 2016 (r310783) +++ head/sys/mips/mips/busdma_machdep.c Thu Dec 29 20:11:50 2016 (r310784) @@ -66,6 +66,16 @@ __FBSDID("$FreeBSD$"); #define BUS_DMA_COULD_BOUNCE BUS_DMA_BUS3 #define BUS_DMA_MIN_ALLOC_COMP BUS_DMA_BUS4 +/* + * On XBurst cores from Ingenic, cache-line writeback is local + * only, unless accompanied by invalidation. Invalidations force + * dirty line writeout and invalidation requests forwarded to + * other cores if other cores have the cache line dirty. + */ +#if defined(SMP) && defined(CPU_XBURST) +#define BUS_DMA_FORCE_WBINV +#endif + struct bounce_zone; struct bus_dma_tag { @@ -1069,7 +1079,7 @@ bus_dmamap_sync_buf(vm_offset_t buf, int /* * dcache invalidation operates on cache line aligned addresses * and could modify areas of memory that share the same cache line - * at the beginning and the ending of the buffer. In order to + * at the beginning and the ending of the buffer. In order to * prevent a data loss we save these chunks in temporary buffer * before invalidation and restore them afer it. * @@ -1099,7 +1109,7 @@ bus_dmamap_sync_buf(vm_offset_t buf, int case BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE: case BUS_DMASYNC_POSTREAD: - /* + /* * Save buffers that might be modified by invalidation */ if (size_cl) @@ -1107,14 +1117,14 @@ bus_dmamap_sync_buf(vm_offset_t buf, int if (size_clend) memcpy (tmp_clend, (void*)buf_clend, size_clend); mips_dcache_inv_range(buf, len); - /* + /* * Restore them */ if (size_cl) memcpy ((void*)buf_cl, tmp_cl, size_cl); if (size_clend) memcpy ((void*)buf_clend, tmp_clend, size_clend); - /* + /* * Copies above have brought corresponding memory * cache lines back into dirty state. Write them back * out and invalidate affected cache lines again if @@ -1132,7 +1142,7 @@ bus_dmamap_sync_buf(vm_offset_t buf, int break; case BUS_DMASYNC_PREREAD: - /* + /* * Save buffers that might be modified by invalidation */ if (size_cl) @@ -1147,7 +1157,7 @@ bus_dmamap_sync_buf(vm_offset_t buf, int memcpy ((void *)buf_cl, tmp_cl, size_cl); if (size_clend) memcpy ((void *)buf_clend, tmp_clend, size_clend); - /* + /* * Copies above have brought corresponding memory * cache lines back into dirty state. Write them back * out and invalidate affected cache lines again if @@ -1161,7 +1171,11 @@ bus_dmamap_sync_buf(vm_offset_t buf, int break; case BUS_DMASYNC_PREWRITE: +#ifdef BUS_DMA_FORCE_WBINV + mips_dcache_wbinv_range(buf, len); +#else mips_dcache_wb_range(buf, len); +#endif break; } } @@ -1175,19 +1189,24 @@ _bus_dmamap_sync_bp(bus_dma_tag_t dmat, if (op & BUS_DMASYNC_PREWRITE) { if (bpage->datavaddr != 0) bcopy((void *)bpage->datavaddr, - (void *)(bpage->vaddr_nocache != 0 ? + (void *)(bpage->vaddr_nocache != 0 ? bpage->vaddr_nocache : bpage->vaddr), bpage->datacount); else physcopyout(bpage->dataaddr, - (void *)(bpage->vaddr_nocache != 0 ? + (void *)(bpage->vaddr_nocache != 0 ? bpage->vaddr_nocache : bpage->vaddr), bpage->datacount); if (bpage->vaddr_nocache == 0) { +#ifdef BUS_DMA_FORCE_WBINV + mips_dcache_wbinv_range(bpage->vaddr, + bpage->datacount); +#else mips_dcache_wb_range(bpage->vaddr, bpage->datacount); +#endif } dmat->bounce_zone->total_bounced++; } @@ -1197,11 +1216,11 @@ _bus_dmamap_sync_bp(bus_dma_tag_t dmat, bpage->datacount); } if (bpage->datavaddr != 0) - bcopy((void *)(bpage->vaddr_nocache != 0 ? + bcopy((void *)(bpage->vaddr_nocache != 0 ? bpage->vaddr_nocache : bpage->vaddr), (void *)bpage->datavaddr, bpage->datacount); else - physcopyin((void *)(bpage->vaddr_nocache != 0 ? + physcopyin((void *)(bpage->vaddr_nocache != 0 ? bpage->vaddr_nocache : bpage->vaddr), bpage->dataaddr, bpage->datacount); dmat->bounce_zone->total_bounced++; @@ -1214,7 +1233,7 @@ _bus_dmamap_sync(bus_dma_tag_t dmat, bus { struct sync_list *sl, *end; int aligned; - + if (op == BUS_DMASYNC_POSTWRITE) return; if (STAILQ_FIRST(&map->bpages)) @@ -1233,7 +1252,7 @@ _bus_dmamap_sync(bus_dma_tag_t dmat, bus if (map->sync_count) { end = &map->slist[map->sync_count]; for (sl = &map->slist[0]; sl != end; sl++) - bus_dmamap_sync_buf(sl->vaddr, sl->datacount, op, + bus_dmamap_sync_buf(sl->vaddr, sl->datacount, op, aligned); } }