Date: Mon, 27 Aug 2012 09:40:19 -0600 From: Warner Losh <imp@bsdimp.com> To: Adrian Chadd <adrian@freebsd.org> Cc: Tim Kientzle <tim@kientzle.com>, Hans Petter Selasky <hans.petter.selasky@bitfrost.no>, freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS Message-ID: <9642068B-3C66-42BD-8515-14F734B3FF89@bsdimp.com> In-Reply-To: <CAJ-VmomMCvR7nr73sLLmwsotU=TT399asBcJvinHCqspo6BN1w@mail.gmail.com> References: <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> <zarafa.503b0e81.5c36.1a2f71091ebf9bd2@eric2.bitfrost> <A749E691-BF25-4B72-B929-56ABEB10F3E9@bsdimp.com> <DA9750F9-7B8A-49AF-8ECA-AC7D565CF3F5@kientzle.com> <CAJ-VmomMCvR7nr73sLLmwsotU=TT399asBcJvinHCqspo6BN1w@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Aug 27, 2012, at 9:20 AM, Adrian Chadd wrote: > On 27 August 2012 08:12, Tim Kientzle <tim@kientzle.com> wrote: >> But don't mbuf structures do pretty much what Hans is suggesting? >>=20 >> Why is mbuf okay? >=20 > Which part of mbufs? >=20 > I think the difference here is that there's no concurrent access. Ie, > although you may have an mbuf header + data inside the same cache > line, you wouldn't go fondling the mbuf once it's handed to the > hardware. >=20 > But I was under the impression that mbuf + mbuf buffers are already > correctly aligned. Not quite. mbufs are large enough and aligned enough so that no two = mbufs are in the same cache line. The data inside the mbuf on 32-bit = platforms is 24 bytes from the start of the mbuf, so the mbuf data and = mbuf header do share the same cache line, but as you say, nobody touches = any part of the mbuf will the hardware has it. Recall that it isn't = alignment of where the buffer starts that matters here, but rather = alignment to ensure that there's no sharing of cache lines between = structures. > This all honestly looks like a very i386-centric interpretation of the > busdma "intention", which I think illustrates a need to better > document what assumptions busdma actually makes. Yes. x86 lets you play fast and loose with many of these things. We've = tried to accommodate this for a long time, but that is lame. > That does remind me, I think the ath(4) driver does the same (since it > allocates its own descriptor block and then treats it as an array of > descriptors for the hardware to access) - I should ensure that > sizeof(ath_desc) is aligned on the relevant architecture. It gets > slightly scary - AR93xx TX descriptors are "L1 cache =3D=3D 128 byte > aligned" which is an enormous waste of memory compared to a 16 or 32 > byte aligned platform. Alas.. The problem is with cache line sharing, not necessarily with alignment. = If you are only ever using one of them at a time, or if you have perfect = hygiene, you can cope with this situation without undue waste. The = perfect hygiene might be hard sometimes. Warner
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9642068B-3C66-42BD-8515-14F734B3FF89>