Date: Wed, 29 Apr 2015 16:23:24 -0600 From: Ian Lepore <ian@freebsd.org> To: Jason Harmening <jason.harmening@gmail.com> Cc: Konstantin Belousov <kostikbel@gmail.com>, Adrian Chadd <adrian@freebsd.org>, Svatopluk Kraus <onwahe@gmail.com>, freebsd-arch <freebsd-arch@freebsd.org> Subject: Re: bus_dmamap_sync() for bounced client buffers from user address space Message-ID: <1430346204.1157.107.camel@freebsd.org> In-Reply-To: <CAM=8qak0qRw5MsSG4e1Zqxo_x9VFGQ2rQpjUBFX_UA6P9_-2cA@mail.gmail.com> References: <38574E63-2D74-4ECB-8D68-09AC76DFB30C@bsdimp.com> <CAJ-VmomqGkEFVauya%2BrmPGcD_-=Z-mmg1RSDf1D2bT_DfwPBGA@mail.gmail.com> <1761247.Bq816CMB8v@ralph.baldwin.cx> <CAFHCsPX9rgmCAPABct84a000NuBPQm5sprOAQr9BTT6Ev6KZcQ@mail.gmail.com> <20150429132017.GM2390@kib.kiev.ua> <CAFHCsPWjEFBF%2B-7SR7EJ3UHP6oAAa9xjbu0CbRaQvd_-6gKuAQ@mail.gmail.com> <20150429165432.GN2390@kib.kiev.ua> <CAM=8qakzkKX8TZNYE33H=JqL_r5z%2BAU9fyp5%2B7Z0mixmF5t63w@mail.gmail.com> <20150429185019.GO2390@kib.kiev.ua> <CAM=8qanPHbCwUeu0-zi-ccY4WprHaOGzCm44PwNSgb==nwgGGw@mail.gmail.com> <20150429193337.GQ2390@kib.kiev.ua> <CAM=8qak0qRw5MsSG4e1Zqxo_x9VFGQ2rQpjUBFX_UA6P9_-2cA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 2015-04-29 at 14:59 -0500, Jason Harmening wrote:
> >
> > Even without SMP, VIPT cache cannot hold two mappings of the same page.
> > As I understand, sometimes it is more involved, eg if mappings have
> > correct color (eg. on ultrasparcs), then cache can deal with aliasing.
> > Otherwise pmap has to map the page uncached for all mappings.
> >
>
> Yes, you are right. Regardless of whatever logic the cache uses (or
> doesn't use), FreeBSD's page-coloring scheme should prevent that.
>
>
> >
> > I do not see what would make this case special for SMP after that.
> > Cache invalidation would be either not needed, or coherency domain
> > propagation of the virtual address does the right thing.
> >
>
> Since VIPT cache operations require a virtual address, I'm wondering about
> the case where different processes are running on different cores, and the
> same UVA corresponds to a completely different physical page for each of
> those processes. If the d-cache for each core contains that UVA, then what
> does it mean when one core issues a flush/invalidate instruction for that
> UVA?
>
> Admittedly, there's a lot I don't know about how that's supposed to work in
> the arm/mips SMP world. For all I know, the SMP targets could be
> fully-snooped and we don't need to worry about cache maintenance at all.
For what we call armv6 (which is mostly armv7)...
The cache maintenance operations require virtual addresses, which means
it looks a lot like a VIPT cache. Under the hood the implementation
behaves as if it were a PIPT cache so even in the presence of multiple
mappings of the same physical page into different virtual addresses, the
SMP coherency hardware works correctly.
The ARM ARM says...
[Stuff about ARMv6 and page coloring when a cache way exceeds
4K.]
ARMv7 does not support page coloring, and requires that all data
and unified caches behave as Physically Indexed Physically
Tagged (PIPT) caches.
The only true armv6 chip we support isn't SMP and has a 16K/4-way cache
that neatly sidesteps the aliasing problem that requires page coloring
solutions. So modern arm chips we get to act like we've got PIPT data
caches, but with the quirk that cache ops are initiated by virtual
address.
Basically, when you perform a cache maintainence operation, a
translation table walk is done on the core that issued the cache op,
then from that point on the physical address is used within the cache
hardware and that's what gets broadcast to the other cores by the snoop
control unit or cache coherency fabric (depending on the chip).
Not that it's germane to this discussion, but an ARM instruction cache
can really be VIPT with no "behave as if" restrictions in the spec.
That means when doing i-cache maintenance on a virtual address that
could be multiply-mapped our only option a rather expensive all-cores
"invalidate entire i-cache and branch predictor cache".
For the older armv4/v5 world which is VIVT, we have a restriction that a
page that is multiply-mapped cannot have cache enabled (it's handled in
pmap). That's also probably not very germane to this discussion,
because it doesn't seem likely that anyone is going to try to add
physical IO or userspace DMA support to that old code.
-- Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1430346204.1157.107.camel>