Date: Mon, 27 Aug 2012 13:00:47 -0600 From: Ian Lepore <freebsd@damnhippie.dyndns.org> To: Adrian Chadd <adrian@freebsd.org> Cc: freebsd-arm@freebsd.org, freebsd-arch@freebsd.org, freebsd-mips@freebsd.org, Hans Petter Selasky <hans.petter.selasky@bitfrost.no> Subject: Re: Partial cacheline flush problems on ARM and MIPS Message-ID: <1346094047.1140.264.camel@revolution.hippie.lan> In-Reply-To: <CAJ-Vmo=Mh1dav4DYW8=yjyBh-qmuAfSKpbFFU9JZSXqYqwLLVg@mail.gmail.com> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <FD8DC82C-AD3B-4EBC-A625-62A37B9ECBF1@bsdimp.com> <1345765503.27688.602.camel@revolution.hippie.lan> <CAJ-VmonOwgR7TNuYGtTOhAbgz-opti_MRJgc8G%2BB9xB3NvPFJQ@mail.gmail.com> <1345766109.27688.606.camel@revolution.hippie.lan> <CAJ-VmomFhqV5rTDf-kKQfbSuW7SSiSnqPEjGPtxWjaHFA046kQ@mail.gmail.com> <F8C9E811-8597-4ED0-9F9D-786EB2301D6F@bsdimp.com> <1346002922.1140.56.camel@revolution.hippie.lan> <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> <1346083716.1140.212.camel@revolution.hippie.lan> <CAJ-Vmo=Mh1dav4DYW8=yjyBh-qmuAfSKpbFFU9JZSXqYqwLLVg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 2012-08-27 at 09:53 -0700, Adrian Chadd wrote: > On 27 August 2012 09:08, Ian Lepore <freebsd@damnhippie.dyndns.org> wrote: > > > If two DMAs are going on concurrently in the same buffer, one is going > > to finish before the other, leading to a POSTxxxx sync op happening for > > one DMA operation while the other is still in progress. The unit of > > granularity for sync operations is the mapped region, so now you're > > syncing access to a region which still has active DMA happening within > > it. > > Right. But the enforced idea is "DMA up to this point should be > flushed to memory." > > > While I think it's really an API definition issue, think about it in > > terms of a potential implementation... What if the CPU had to access the > > memory as part of the sync for the first DMA that completes, while the > > second is still running? Now you've got pretty much exactly the same > > situation as when a driver subdivides a buffer without knowing about the > > cache alignment; you end up with the CPU and DMA touching data in the > > same cachline and no sequence of flush/invalidate can be g'teed to > > preserve all data correctly. > > Right. So you realise at that point you can't win and you stick each > of those pieces in a different cache line. > Actually, I think that even discussing cache lines in this context is a mistake (yeah, I'm the one who did so above, in trying to relate an abstract API design concept to a real-world hardware example). Drivers are not supposed to know about interactions between DMA transfers and cache lines or other machine-specific constraints; that info is supposed to be encapsulated and hidden within busdma. I think a driver making the assumption that it can do DMA safely on a buffer as long as that buffer is cacheline-granular is just as flawed as assuming that it can do DMA safely on any arbitrarily sized and aligned buffer. So the right way to "stick each of those pieces in a different cache line" is to allocate two different buffers, one per concurrent DMA transfer. Or, really, to use two separate busdma mappings would be the more rigorous way to say it, since the mapping is the operation at which constraints come into play. Thinking of it that way then drives the need to document that if multiple mappings describe the same area of physical memory, then concurrent operations on those maps yield unpredictable results. -- Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1346094047.1140.264.camel>