From owner-freebsd-arch@FreeBSD.ORG Thu Apr 30 09:53:07 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 39CDE48E; Thu, 30 Apr 2015 09:53:07 +0000 (UTC) Received: from mail-ie0-x231.google.com (mail-ie0-x231.google.com [IPv6:2607:f8b0:4001:c03::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F234A1CB9; Thu, 30 Apr 2015 09:53:06 +0000 (UTC) Received: by iedfl3 with SMTP id fl3so70652438ied.1; Thu, 30 Apr 2015 02:53:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=vE8UxfyFvnSUa7y0Ao9fY7DbeYyy+An+QqpnG4OUzu0=; b=E6bwNNpBoA69OaqVDEDr8MO/+MuR8+ZZC/6uXz8KU0VUwxe/SuXuU+O7FsTZu4d+l9 QRnhxdW1LwP/yTZJUOjxugXB8cEZyoT5kzKUEH+wbqekoxGVdMyM6VASwi54OBxkerhn LjYH/qcXEWBWdlyiOlDW/yx0jEEVkI+5SNgOXgrZmX9vzGm91f0jTnfMT5ccs85L5ZhR HJxKvtn00Sz9p+WiZSkxlL3q4Lq6r6XoQqlSP8cPJw8k1Vo/j/LJgqVeyArq8YMIp004 6PQK78DBSx00SLjKoumNfojAcu9rLdbSZOKc/UUhuuvp2tAhaRXTi0Y/yOwD652TJ3Pf cg/g== MIME-Version: 1.0 X-Received: by 10.107.28.146 with SMTP id c140mr4399830ioc.67.1430387586495; Thu, 30 Apr 2015 02:53:06 -0700 (PDT) Received: by 10.64.13.81 with HTTP; Thu, 30 Apr 2015 02:53:06 -0700 (PDT) In-Reply-To: References: <38574E63-2D74-4ECB-8D68-09AC76DFB30C@bsdimp.com> <1761247.Bq816CMB8v@ralph.baldwin.cx> <20150429132017.GM2390@kib.kiev.ua> <20150429165432.GN2390@kib.kiev.ua> <20150429185019.GO2390@kib.kiev.ua> <20150429193337.GQ2390@kib.kiev.ua> <1430346204.1157.107.camel@freebsd.org> Date: Thu, 30 Apr 2015 11:53:06 +0200 Message-ID: Subject: Re: bus_dmamap_sync() for bounced client buffers from user address space From: Svatopluk Kraus To: Jason Harmening Cc: Ian Lepore , Konstantin Belousov , Adrian Chadd , freebsd-arch Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Apr 2015 09:53:07 -0000 On Thu, Apr 30, 2015 at 1:10 AM, Jason Harmening wrote: >> >> For what we call armv6 (which is mostly armv7)... >> >> The cache maintenance operations require virtual addresses, which means >> it looks a lot like a VIPT cache. Under the hood the implementation >> behaves as if it were a PIPT cache so even in the presence of multiple >> mappings of the same physical page into different virtual addresses, the >> SMP coherency hardware works correctly. >> >> The ARM ARM says... >> >> [Stuff about ARMv6 and page coloring when a cache way exceeds >> 4K.] >> >> ARMv7 does not support page coloring, and requires that all data >> and unified caches behave as Physically Indexed Physically >> Tagged (PIPT) caches. >> >> The only true armv6 chip we support isn't SMP and has a 16K/4-way cache >> that neatly sidesteps the aliasing problem that requires page coloring >> solutions. So modern arm chips we get to act like we've got PIPT data >> caches, but with the quirk that cache ops are initiated by virtual >> address. > > > Cool, thanks for the explanation! > To satisfy my own curiosity, since it "looks like VIPT", does that mean we > still have to flush the cache on context switch? No, in general, there is no need to flush PIPT caches (even if they "look like VIPT") on context switch. When it comes (cache maintainance), physical page is either mapped in correct context or you have to map it somewhere in current context (KVA is used for that). >> >> >> Basically, when you perform a cache maintainence operation, a >> translation table walk is done on the core that issued the cache op, >> then from that point on the physical address is used within the cache >> hardware and that's what gets broadcast to the other cores by the snoop >> control unit or cache coherency fabric (depending on the chip). > > > So, if we go back to the original problem of wanting to do bus_dmamap_sync() > on userspace buffers from some asynchronous context: > > Say the process that owns the buffer is running on one core and prefetches > some data into a cacheline for the buffer, and bus_dmamap_sync(POSTREAD) is > done by a kernel thread running on another core. Since the core running the > kernel thread is responsible for the TLB lookup to get the physical address, > then since that core has no UVA the cache ops will be treated as misses and > the cacheline on the core that owns the UVA won't be invalidated, correct? > > That means the panic on !pmap_dmap_iscurrent() in busdma_machdep-v6.c should > stay? Not for unmapped buffers. For user space buffers, it's still a question how this will be resolved. It looks now that it's aiming to not using UVA for DMA buffers in kernel at all. In any case, even if UVA will be used, the panic won't be needed if correct implementation will be done. > > Sort of the same problem would apply to drivers using > vm_fault_quick_hold_pages + bus_dmamap_load_ma...no cache maintenance, since > there are no VAs to operate on. Indeed, both arm and mips implementation of > _bus_dmamap_load_phys don't do anything with the sync list. I'm just working on _bus_dmamap_load_phys() implementation for armv6. It means that I'm adding sync list for unmapped buffers (with virtual address set to zero) and implement cache maintainance operations with physical address passed as argument. It means that given range will be temporarily mapped to kernel (page by page) and cache operation using virtual address willl be called. It's the same scenarion taken in i386 pmap.