From owner-freebsd-hackers@FreeBSD.ORG Mon May 21 19:27:43 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D1A821065674; Mon, 21 May 2012 19:27:43 +0000 (UTC) (envelope-from marktinguely@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9B5918FC19; Mon, 21 May 2012 19:27:43 +0000 (UTC) Received: by dadv36 with SMTP id v36so7789614dad.13 for ; Mon, 21 May 2012 12:27:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=ioRQofufxc0jHzZA6LgY7/JNjOZBWaADmPyCh7gTsPA=; b=yjd+JQyJPUoH/y/MzgpxYgRYNWWcSaYHwxVmsWooKlnkD+G2rXKdxDOPyhOVucuRsj P+cX1c1l0r2ZmZOK9aRv7XkvkGKgeBFBW57qCwSH9SrH282uK7Wl/3s1PB1NEqmd8fQg NEBWCNjIUe4x6z//qrf0P3Em5JyyzuvQjkPdROG2e1aso0eRU+JbJ5nGRPBJj7+8XR/5 Z/HHD8KVH8eY+MqEJD2BNKodq+BhGxdHT7ucpZ6phyV9iAsL2EuFthGxEfkEIc3bohkB lpc/2gHjn/AYmqCW8r5P4TrikRg+KL21Tn1qi9lBba1VXIad+zjA4NjeNdYjRLVwNIKc Blkw== MIME-Version: 1.0 Received: by 10.68.232.135 with SMTP id to7mr520502pbc.143.1337628463040; Mon, 21 May 2012 12:27:43 -0700 (PDT) Received: by 10.68.72.195 with HTTP; Mon, 21 May 2012 12:27:42 -0700 (PDT) In-Reply-To: <1337617221.2516.38.camel@revolution.hippie.lan> References: <1337285248.1503.308.camel@revolution.hippie.lan> <1337617221.2516.38.camel@revolution.hippie.lan> Date: Mon, 21 May 2012 14:27:42 -0500 Message-ID: From: Mark Tinguely To: Ian Lepore Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Richard Hodges , freebsd-arm@freebsd.org, hackers@freebsd.org, Svatopluk Kraus Subject: Re: ARM + CACHE_LINE_SIZE + DMA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 May 2012 19:27:43 -0000 On Mon, May 21, 2012 at 11:20 AM, Ian Lepore wrote: > On Fri, 2012-05-18 at 16:13 +0200, Svatopluk Kraus wrote: >> On Thu, May 17, 2012 at 10:07 PM, Ian Lepore >> wrote: >> > On Thu, 2012-05-17 at 15:20 +0200, Svatopluk Kraus wrote: >> >> Hi, >> >> >> >> I'm working on DMA bus implementation for ARM11mpcore platform. I've >> >> looked at implementation in ARM tree, but IMHO it only works with som= e >> >> assumptions. There is a problem with DMA on memory block which is not >> >> aligned on CACHE_LINE_SIZE (start and end) if memory is not coherent. >> >> >> >> Let's have a buffer for DMA which is no aligned on CACHE_LINE_SIZE. >> >> Then first cache line associated with the buffer can be divided into >> >> two parts, A and B, where A is a memory we know nothing about it and = B >> >> is buffer memory. The same stands for last cache line associatted wit= h >> >> the buffer. We have no problem if a memory is coherent. Otherwise it >> >> depends on memory attributes. ... > My suggestion of making a temporary writable mapping was the answer to > how to correctly change memory attributes on a page which is in use, at > least in the existing code, which is for a single processor. > > You don't need, and won't even use, the temporary mapping. =A0You would > make it just because doing so invokes logic in arm/arm/pmap.c which will > find all existing virtual mappings of the given physical pages, and > disable caching in each of those existing mappings. =A0In effect, it make= s > all existing mappings of the physical pages DMA_COHERENT. =A0When you > later free the temporary mapping, all other existing mappings are > changed back to being cacheable (as long as no more than one of the > mappings that remain is writable). > > I don't know that making a temporary mapping just for its side effect of > changing other existing mappings is a good idea, it's just a quick and > easy thing to do if you want to try changing all existing mappings to > non-cacheable. =A0It could be that a better way would be to have the > busdma_machdep code call directly to lower-level routines in pmap.c to > change existing mappings without making a new temporary mapping in the > kernel pmap. =A0The actual changes to the existing mappings are made by > pmap_fix_cache() but that routine isn't directly callable right now. > > Also, as far as I know all of this automatic disabling of cache for > multiple writable mappings applies only to VIVT cache architectures. > I'm not sure how the pmap code is going to change to support VIPT and > PIPT caches, but it may no longer be true that making a second writable > mapping of a page will lead to changing all existing mappings to > non-cacheable. > > -- Ian We don't want to carry the VIVT cache fixing code to VIPT/PIPT. I like the x86 approach of marking the page with a cache type (default/device/uncached/etc). The page mapping routines (for example pmap_qenter() on a clustered write) will honor that cache type in all future mappings. It is easy to implement. Other allocations, such as page tables, can benefit from an attributed allocation too. I also like having a separate allocator for the sub-page bus_dmamem_alloc() requests that want uncached buffers. These entries can stick around for a long time. If we just malloced the entries, then the other threads that happen to allocate data from the same page are penalized with uncached buffers too. --Mark.