From owner-freebsd-arm@freebsd.org Mon Apr 3 14:59:12 2017 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB780D2CD5B for ; Mon, 3 Apr 2017 14:59:12 +0000 (UTC) (envelope-from zbb@semihalf.com) Received: from mail-lf0-x22f.google.com (mail-lf0-x22f.google.com [IPv6:2a00:1450:4010:c07::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 456D9336 for ; Mon, 3 Apr 2017 14:59:12 +0000 (UTC) (envelope-from zbb@semihalf.com) Received: by mail-lf0-x22f.google.com with SMTP id j90so76335691lfk.2 for ; Mon, 03 Apr 2017 07:59:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=2aW8SrANOHuPdEFNgUgctZJTlXxdnSkd1NGUvndsgfM=; b=Vc67gSa4plB00nOGRWH5sb4mybSwBG605zcMu5312EdDfVmvOb1HKaVniJNv144t3c +rPEDLBUvzL9FB7MeoxviL+OiMu3ngUC080aPujR16sjmdyeotU7eOGo+gVjpxKTEvda 066n8r6/1WAnp+mt8QlmhLirMM8l3zUfJwjDgpB7V9OEWwW/a5faRlI0uSvZOAjbHxjo W7luk2pkyD1w4PiPis8zbdbATzhRk3axbPGbbC2ZX96XC2phxaHoHjdInZvaikHGpnOz Akv/nPi49lXQU9LsDFo6vnIkN5v37RlC7Krr3HIiazmjIoxGjD/LPbNBuoPKb5FLV3nW oFPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=2aW8SrANOHuPdEFNgUgctZJTlXxdnSkd1NGUvndsgfM=; b=pEc5rl8OmTcNSvV1UGfhaHl7fYsOBRkHsN3fafQGJckWBuPER5xr1W0mH9vZuGWTR4 H5xhKNINwKLcPg4Gyj04Rfy3O2dP27KrRo0NJWY/OZ9Q093a14MTH6xiWVpbQWfvcOPb VQ5zK7udi5A3H+xEUHOqOXiVl1W2SF9G6Q4v5tRJIfb1fw+H7R3xcm2n+My+azUvPNwM i4Zn93H8WE6RaOG6BMFVAOezihcs9wd4Ia9qIy3rJIAoZE1gKpaqikVb/mJsHvQPvt15 wpmGlcU+eL0UCct+tzXAe3BMJGRX0mjE+VXIxWiLE538IjsYgLNP9a51UMao89ZnEGlH lrBw== X-Gm-Message-State: AFeK/H0QpSTqpSBBu1+bAb39Vm6YSR5FsMGrhCQWMcI6EY+dGLDTttMWMMWGhRJNwz667+8zfCo75WFkd4OdfQ== X-Received: by 10.25.32.80 with SMTP id g77mr4853909lfg.76.1491231550186; Mon, 03 Apr 2017 07:59:10 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.181.220 with HTTP; Mon, 3 Apr 2017 07:58:49 -0700 (PDT) In-Reply-To: <5586A20F-125D-41EC-9741-BBFFDB0A7A38@fubar.geek.nz> References: <0EA39E6B-3460-45B9-8247-CB6CC8631C5F@fubar.geek.nz> <5586A20F-125D-41EC-9741-BBFFDB0A7A38@fubar.geek.nz> From: Zbigniew Bodek Date: Mon, 3 Apr 2017 16:58:49 +0200 Message-ID: Subject: Re: Coherent bus_dma for ARMv7 To: Andrew Turner Cc: Marcin Wojtas , Adrian Chadd , "freebsd-arm@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Apr 2017 14:59:12 -0000 2017-04-03 16:37 GMT+02:00 Andrew Turner : > > > On 3 Apr 2017, at 15:14, Zbigniew Bodek wrote: > > > > 2017-04-03 15:37 GMT+02:00 Andrew Turner : > > > > > On 3 Apr 2017, at 14:16, Marcin Wojtas wrote: > > > > > > Hi Adrian, > > > > > > Frankly we are not such experts in armv6 bus_dma, which looks more > > > complicated than one in arm64, so we thought it's much safer no to mi= x > > > the two solutions and leave for the user a single switch to decide, > > > which one to pick. Afaik Andrew Turner did the oposite for arm64 > > > (implement not coherent solution on top of coherent bus_dma), however > > > I'm not sure if it's possible here in an easy way - there's also > > > pretty significant risk of regression for all platforms. Please let m= e > > > know your opinion. Do you think some sort of update of armv6 is > > > doable? > > > > I don=E2=80=99t see any reason to think it would be difficult to add su= pport for > coherent hardware to the existing armv6 busdma code. It=E2=80=99s mostly = skipping > cache operations based on a flag in the dam tag. > > > > Andrew > > > > Hello Andrew, > > > > I don't think anyone uses flags related to DMA coherency in > bus_dma_tag_create. > > The generic PCI and ThunderX PCIe PEM drivers do. The former based on the > FDT dma-coherent flag. > In this particular example this will work as almost all (not all) devices on ThunderX are PCIe devices. For most ARMv7-based SoCs this is not true. We would need to create a coherent DMA tag for the top level buses and ensure that this is propagated correctly down to the subordinate buses and devices. > > > > > Nevertheless, for coherent platforms we want bus_dma to always map DMA > memory as normal WBWA regardless of the flags passed to create a bus_dma > MAP. > > For example, we don't want to perform any synchronization and we want t= o > have the cacheable memory regardless of BUS_DMA_COHERENT flag used. > > That=E2=80=99s already the case on arm64, the only synchronisation used w= hen the > tag is created with BUS_DMA_COHERENT is a memory barrier. > For PCI. > > > Otherwise the performance improvement will apply only to those drivers > that dare to use BUS_DMA_COHERENT flag and very few of them does that. In > other words, what is the point of having coherent DMA if you do cache > maintenance anyway? > > The drivers should be getting the parent DMA tag and passing this to > bus_dma_tag_create. If this was created with BUS_DMA_COHERENT it will pas= s > this to the child tag. This is how the above PCI drivers work. > > This basically makes sense to me if we do the same for all buses or once for every platform. The question is how much additional stuff is added to busdma_machdep-v6.c to make it work on all relevant platforms because it is quite different from the ARM64 implementation. Still we can go with the ARM64 approach and add new DMA handling, parallel to the existing one. Improve it over time to handle non-coherent DMA and replace the old one with the new one when it is proven to be correct for all. Kind regards zbb