From owner-freebsd-hackers@FreeBSD.ORG Wed May 23 13:13:08 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C902A1065670; Wed, 23 May 2012 13:13:08 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 19EC88FC17; Wed, 23 May 2012 13:13:07 +0000 (UTC) Received: by lbon10 with SMTP id n10so7316692lbo.13 for ; Wed, 23 May 2012 06:13:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Ht/a4Yyu+Pi+OKYcTM9+ILnoF4hBDFZ0IOsqqrqpdL8=; b=EIx3g1ttNA+GumVZXz6bWZcqZ2cLPodavKMklHQaSYteLszLjAf8eEyfaVFyVb9IIx 4nuqyzmWok3648ffFhtbh8AFy7RUonIqjBr3IZKscAsgrx5zJ1qi0bm2Pjv45Y+gNYsJ PdAVsf2S4mMoA4aNQvC8pl6zn5cu4Q6EMHHiA6e2d19hKg60+yT6mkw94/SN04Gvdcsq Il7CwPNK07tVJocagoddc+YD/yV0XT6L0b3M8KkGP1mTrsPZknaxBVjgrRn+W62Cz+FT 3BVFytDZ3ACQ4buptvOEafI/tKh9xeUP2geRsQAKTqokpxE6VuAa+tV7gJ87fX9QNlbZ iYJQ== MIME-Version: 1.0 Received: by 10.112.44.132 with SMTP id e4mr2622509lbm.51.1337778780810; Wed, 23 May 2012 06:13:00 -0700 (PDT) Received: by 10.112.60.228 with HTTP; Wed, 23 May 2012 06:13:00 -0700 (PDT) In-Reply-To: <1337617221.2516.38.camel@revolution.hippie.lan> References: <1337285248.1503.308.camel@revolution.hippie.lan> <1337617221.2516.38.camel@revolution.hippie.lan> Date: Wed, 23 May 2012 15:13:00 +0200 Message-ID: From: Svatopluk Kraus To: Ian Lepore Content-Type: text/plain; charset=ISO-8859-1 Cc: Richard Hodges , freebsd-arm@freebsd.org, hackers@freebsd.org Subject: Re: ARM + CACHE_LINE_SIZE + DMA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 May 2012 13:13:08 -0000 Hi, with respect to your replies and among other things, the following summary could be made: There are three kinds of DMA buffers according to their origin: 1. driver buffers As Alexander wrote, the buffers should be allocated by bus_dmamap_alloc(). The function should be implemented to allocate the buffers correctly aligned with help of bus_dma_tag_t. For these buffers, we can avoid bouncing totally just by correct driver implementation. For badly implemented drivers, bouncing penalty is paid in case of unaligned buffers. For BUS_DMA_COHERENT allocations, as Mark wrote, an allocation pool of coherent pages is good optimalization. 2. well-known system buffers Mbufs and vfs buffers. The buffers should be aligned on CACHE_LINE_SIZE (start and size). It should be enough for vfs buffers as they are carring data only and only whole buffers should be accessed by DMA. The mbuf is a structure and data can be carried on three possible locations. The first one, the external buffer, should be aligned on CACHE_LINE_SIZE. The next two locations, which are parts of the mbuf structure, could be unaligned in general. If we assume that no one else is writing any part of the mbuf during DMA access, we can set BUS_DMA_UNALIGNED_SAFE flag in mbuf load functions. I.e., we don't bounce unaligned buffers if the flag is set in dmamap. A tunable can be implemented to suppres the flag for debugging purposes. 3. other buffers As we know nothing about these buffers, we must always bounce unaligned ones. Just two more notes. The DMA buffer should not be access by anyone (except DMA itself) after PRESYNC and before POSTSYNC. For DMA descriptors (for example), using bus_dmamap_alloc() with BUS_DMA_COHERENT flag could be inevitable. As I'm implementing bus dma for ARM11mpcore, I'm doing it with next assumptions: 1. ARMv6k and higher 2. PIPT data cache 3. SMP ready Svata