Date: Thu, 23 Aug 2012 17:27:08 -0600 From: Ian Lepore <freebsd@damnhippie.dyndns.org> To: Adrian Chadd <adrian@freebsd.org> Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS Message-ID: <1345764428.27688.591.camel@revolution.hippie.lan> In-Reply-To: <CAJ-VmokuY0WGYfKyBR_zemZnEuE=X04Z0y_rUpmWB-qY_N68Jw@mail.gmail.com> References: <1345757300.27688.535.camel@revolution.hippie.lan> <CAJ-VmokuY0WGYfKyBR_zemZnEuE=X04Z0y_rUpmWB-qY_N68Jw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 2012-08-23 at 16:09 -0700, Adrian Chadd wrote: > [snip] > > Whoa, there's USB code that has these small buffers straddle cache lines? > > Why aren't they just allocated to always be in their own separate > buffers, so they don't ever have to worry about overlapping cache > lines? Yes, but note that with our current code, a small buffer doesn't have to straddle a cacheline boundary to invoke the partial cacheline flush logic. Its very smallness does so -- if a buffer doesn't start on a cacheline boundary, and/or if its size doesn't exactly equal a multiple of the cacheline size, then it's a partial cacheline flush situation. So even a driver striving to work around this trouble can get into trouble -- it uses bus_dmamem_alloc() to grab a buffer for a 13 byte status message. It gets a 16-byte chunk from the kernel allocator, and the other 16 bytes of that cacheline are given to something else in the kernel, and now you have DMA and CPU access going on in the same cacheline despite your efforts to Do The Right Thing with buffer allocation. (At work we avoid this by setting the kernel minimum allocation size to 32 bytes instead of 16. We did so after being burned by exactly this situation -- the "other 16 bytes" in our case belonged to a struct ithread, and trying to track down a lockup that was caused by a partial cacheline flush corrupting the struct ithread kept me entertained for a couple weeks.) -- Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1345764428.27688.591.camel>