Date: Sun, 26 Aug 2012 12:25:07 -0600 From: Ian Lepore <freebsd@damnhippie.dyndns.org> To: Mark Tinguely <marktinguely@gmail.com> Cc: freebsd-arm@freebsd.org, Hans Petter Selasky <hans.petter.selasky@bitfrost.no>, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS Message-ID: <1346005507.1140.69.camel@revolution.hippie.lan> In-Reply-To: <CAP%2BM-_HZ4yARwZA2koPJDeJWHT-1LORupjymuVnMtLBzeXe=DA@mail.gmail.com> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <FD8DC82C-AD3B-4EBC-A625-62A37B9ECBF1@bsdimp.com> <1345765503.27688.602.camel@revolution.hippie.lan> <CAJ-VmonOwgR7TNuYGtTOhAbgz-opti_MRJgc8G%2BB9xB3NvPFJQ@mail.gmail.com> <1345766109.27688.606.camel@revolution.hippie.lan> <CAJ-VmomFhqV5rTDf-kKQfbSuW7SSiSnqPEjGPtxWjaHFA046kQ@mail.gmail.com> <F8C9E811-8597-4ED0-9F9D-786EB2301D6F@bsdimp.com> <1346002922.1140.56.camel@revolution.hippie.lan> <CAP%2BM-_HZ4yARwZA2koPJDeJWHT-1LORupjymuVnMtLBzeXe=DA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, 2012-08-26 at 13:05 -0500, Mark Tinguely wrote: > I did a quick look at the drivers last summer. > > Most drivers do the right thing and use memory allocated from > bus_dmamem_alloc(). It is easy for us to give them a cache aligned > buffer. > > Some drivers use mbufs - 256 bytes which cache safe. > > Some drivers directly or indirectly malloc() a buffer and then use it > to dma - rather than try to fix them all, I was okay with making the > smallest malloc() amount equal to the cache line size. It amounts to > getting rid of the 16 byte allocation on some ARM architectures. The > power of 2 allocator will then give us cache line safe allocation. > > A few drivers take a small memory amount from the kernel stack and dma > to it <- broken driver. > > The few drivers that use data from a structure and that memory is not > cached aligned <- broken driver. > I disagree about those last two points -- drivers that choose to use stack memory or malloc'd memory as IO buffers are not broken. Drivers can do IO directly to/from userland buffers, do we say that an application that calls read(2) and passes the address of a stack variable is broken? In this regard, it's the busdma implementation that's broken, because it should bounce those IOs through a DMA-safe buffer. There's absolutely no rule that I've ever heard of in FreeBSD that says IO can only take place using memory allocated from busdma. The rule is only that the proper sequence of busdma operation must be called, and beyond that it's up to the busdma implementation to make it work. Our biggest problem, I think, is that we don't have a sufficient definition of "the proper sequence of busdma operations." I don't think it will be very hard to make the arm and mips busdma implementations work correctly. It won't even be too hard to make them fairly efficient at bouncing small IOs (my thinking is that we can make small bounces no more expensive than the current partial cacheline flush implementation which copies the data multiple times). Bouncing large IO will never be efficient, but the inefficiency will be a powerful motivator to update drivers that do large IO to work better, such as using buffers allocated from busdma. -- Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1346005507.1140.69.camel>