From owner-freebsd-arch@FreeBSD.ORG Thu Apr 30 08:38:38 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A4F49446; Thu, 30 Apr 2015 08:38:38 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47685136A; Thu, 30 Apr 2015 08:38:38 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t3U8cWuR049983 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 30 Apr 2015 11:38:32 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t3U8cWuR049983 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t3U8cW59049982; Thu, 30 Apr 2015 11:38:32 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 30 Apr 2015 11:38:32 +0300 From: Konstantin Belousov To: Ian Lepore Cc: Jason Harmening , Adrian Chadd , Svatopluk Kraus , freebsd-arch Subject: Re: bus_dmamap_sync() for bounced client buffers from user address space Message-ID: <20150430083832.GR2390@kib.kiev.ua> References: <20150429132017.GM2390@kib.kiev.ua> <20150429165432.GN2390@kib.kiev.ua> <20150429185019.GO2390@kib.kiev.ua> <20150429193337.GQ2390@kib.kiev.ua> <1430346204.1157.107.camel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1430346204.1157.107.camel@freebsd.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Apr 2015 08:38:38 -0000 On Wed, Apr 29, 2015 at 04:23:24PM -0600, Ian Lepore wrote: > For what we call armv6 (which is mostly armv7)... > > The cache maintenance operations require virtual addresses, which means > it looks a lot like a VIPT cache. Under the hood the implementation > behaves as if it were a PIPT cache so even in the presence of multiple > mappings of the same physical page into different virtual addresses, the > SMP coherency hardware works correctly. > > The ARM ARM says... > > [Stuff about ARMv6 and page coloring when a cache way exceeds > 4K.] > > ARMv7 does not support page coloring, and requires that all data > and unified caches behave as Physically Indexed Physically > Tagged (PIPT) caches. > > The only true armv6 chip we support isn't SMP and has a 16K/4-way cache > that neatly sidesteps the aliasing problem that requires page coloring > solutions. So modern arm chips we get to act like we've got PIPT data > caches, but with the quirk that cache ops are initiated by virtual > address. > > Basically, when you perform a cache maintainence operation, a > translation table walk is done on the core that issued the cache op, > then from that point on the physical address is used within the cache > hardware and that's what gets broadcast to the other cores by the snoop > control unit or cache coherency fabric (depending on the chip). This is the same as it is done on the x86. There is a CLFLUSH instruction, which takes virtual address and invalidates the cache line, maintaining cache coherency in the coherency domain and possibly doing write-back. It takes a virtual address, and even set the accessed bit in the page table entry. My understanding is that such decision to operate on the virtual addresses for x86 was done to allow the instruction to work from the user mode. Still, an instruction to flush cache line addressed by the physical address would be nice. The required circuits are already there, since CPUs must react on the coherency requests from other CPUs. On amd64, the pmap_invalidate_cache_pages() uses direct map, but on i386 kernel has to use specially allocated KVA page frame for temporal mapping (per-cpu CMAP2), see i386/i386/pmap.c:pmap_flush_page().