From owner-freebsd-arm@FreeBSD.ORG Mon Aug 27 19:18:55 2012 Return-Path: Delivered-To: freebsd-arm@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ABA0A106566B for ; Mon, 27 Aug 2012 19:18:55 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 091A98FC1F for ; Mon, 27 Aug 2012 19:18:53 +0000 (UTC) Received: by obbun3 with SMTP id un3so11731249obb.13 for ; Mon, 27 Aug 2012 12:18:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=NWAxBZraqJ39VsS1Q2psNADkTl2YA8RO4lXjmSt0qtk=; b=Oo6SneAT0Eh4ptRwaawUkSTodEx3ahTcGtKUS1+TojGfMB7/pqxo7jEAEqyFd/3TBI bteCo991V6yA/QgVh1Na0zqrkAXElRx917pZa9jFnpBwOWUbIbZYm4uz2L+k2QCc40Kv TR4j2fqfFRLB4pNOriqNlGBQGJqVM1XJByDBVizQsBbrhrWSdD3HYChFB4glwEPmh/CP oHwCcu2lb6ebKUHGDSZ19jfUogDSrx2U75Y+GW99wt+hzMTPNUynsjJSjaH5eUggbta4 rJyENSUYvR+0fSNmoYWyOpw+EgKFAiFXY7DEHPClCqdH4yM6EZvqARqci7O+1bWuJuF1 3MUA== Received: by 10.182.182.71 with SMTP id ec7mr8049310obc.22.1346095133419; Mon, 27 Aug 2012 12:18:53 -0700 (PDT) Received: from [10.30.101.53] ([209.117.142.2]) by mx.google.com with ESMTPS id r1sm13110813oea.4.2012.08.27.12.18.51 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 27 Aug 2012 12:18:52 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1346094047.1140.264.camel@revolution.hippie.lan> Date: Mon, 27 Aug 2012 13:18:49 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> <1346002922.1140.56.camel@revolution.hippie.lan> <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> <1346083716.1140.212.camel@revolution.hippie.lan> <1346094047.1140.264.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQlttZzqkhCfJ0lgiheQOo9AGr9a600oEtPzF1I1U9kRw2ljShs0ihBl1vA9WC5xwrwxiImW Cc: freebsd-arch@freebsd.org, freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, Hans Petter Selasky Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the StrongARM Processor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 19:18:55 -0000 On Aug 27, 2012, at 1:00 PM, Ian Lepore wrote: > On Mon, 2012-08-27 at 09:53 -0700, Adrian Chadd wrote: >> On 27 August 2012 09:08, Ian Lepore = wrote: >>=20 >>> If two DMAs are going on concurrently in the same buffer, one is = going >>> to finish before the other, leading to a POSTxxxx sync op happening = for >>> one DMA operation while the other is still in progress. The unit of >>> granularity for sync operations is the mapped region, so now you're >>> syncing access to a region which still has active DMA happening = within >>> it. >>=20 >> Right. But the enforced idea is "DMA up to this point should be >> flushed to memory." >>=20 >>> While I think it's really an API definition issue, think about it in >>> terms of a potential implementation... What if the CPU had to access = the >>> memory as part of the sync for the first DMA that completes, while = the >>> second is still running? Now you've got pretty much exactly the = same >>> situation as when a driver subdivides a buffer without knowing about = the >>> cache alignment; you end up with the CPU and DMA touching data in = the >>> same cachline and no sequence of flush/invalidate can be g'teed to >>> preserve all data correctly. >>=20 >> Right. So you realise at that point you can't win and you stick each >> of those pieces in a different cache line. >>=20 >=20 > Actually, I think that even discussing cache lines in this context is = a > mistake (yeah, I'm the one who did so above, in trying to relate an > abstract API design concept to a real-world hardware example). >=20 > Drivers are not supposed to know about interactions between DMA > transfers and cache lines or other machine-specific constraints; that > info is supposed to be encapsulated and hidden within busdma. I think = a > driver making the assumption that it can do DMA safely on a buffer as > long as that buffer is cacheline-granular is just as flawed as = assuming > that it can do DMA safely on any arbitrarily sized and aligned buffer. >=20 > So the right way to "stick each of those pieces in a different cache > line" is to allocate two different buffers, one per concurrent DMA > transfer. Or, really, to use two separate busdma mappings would be = the > more rigorous way to say it, since the mapping is the operation at = which > constraints come into play. Thinking of it that way then drives the > need to document that if multiple mappings describe the same area of > physical memory, then concurrent operations on those maps yield > unpredictable results. Despite what I said earlier, I think this is sane. busdma only should = support one DMA active at a time into a buffer. If the driver wants to = do two different ones, they are on their own. Warner=