Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 29 Jun 2009 13:37:41 +0200
From:      Hans Petter Selasky <hselasky@c2i.net>
To:        Piotr =?utf-8?q?Zi=C4=99cik?= <kosmo@semihalf.com>
Cc:        Rafal Jaworowski <raj@semihalf.com>, freebsd-arm@freebsd.org, freebsd-usb@freebsd.org, thompsa@freebsd.org
Subject:   Re: CPU Cache and busdma usage in USB
Message-ID:  <200906291337.43635.hselasky@c2i.net>
In-Reply-To: <200906291249.43983.kosmo@semihalf.com>
References:  <200906231035.43096.kosmo@semihalf.com> <200906291155.13941.hselasky@c2i.net> <200906291249.43983.kosmo@semihalf.com>

next in thread | previous in thread | raw e-mail | index | archive | help
Hi,

On Monday 29 June 2009 12:49:43 Piotr Zi=C4=99cik wrote:
> Monday 29 June 2009 11:55:11 Hans Petter Selasky napisa=C5=82(a):
> > Hi Piotr and Rafal,

>
> Look into ehci_check_transfer() function
> (http://fxr.watson.org/fxr/source/dev/usb/controller/ehci.c#L1294)
>
> usb_pc_cpu_invalidate() [bus_dmamap_sync()] is not used in this function
> correcly. It is not paired with usb_pc_cpu_flush() [opposite
> bus_dmamap_sync()] as busdma requires (see part of manpage cited above).
> The same problem is in part of code shown in previous mail.
>
> If usb_pc_cpu_invalidate()/usb_pc_cpu_flush() functions had been
> implemented without using busdma, for example as cpu_*() functions,
> ehci_check_transfer() would have been 100% correct. In current code busdma
> requirements are simply not met.

Good point. Unfortunately I cannot pair like you suggest, because then I ca=
n=20
overwrite values updated by the hardware, if I first read the status for=20
example, and the same 32-bits are updated by hardware, before the old value=
 is=20
flushed over the new one.. Also I cannot find the word "pair" in the busdma=
=20
manpage? Where is this requirement stated?

I see some things:

You wrote earlier that the COHERENT flag was not set. That means your setup=
 is=20
using bounce pages for DMA? Right?

Then I see a problem, if I do several POST operations in a row then I can s=
ee=20
that the real DMA memory can get cached:

In: "src/sys/arm/arm/busdma_machdep.c"

Is "vaddr_nocache" set or cleared?

                if (op & BUS_DMASYNC_POSTREAD) {
                        if (bpage->vaddr_nocache =3D=3D 0) {
                                cpu_dcache_inv_range(bpage->vaddr,
                                    bpage->datacount);
                                cpu_l2cache_inv_range(bpage->vaddr,
                                    bpage->datacount);
                        }
                        bcopy((void *)(bpage->vaddr_nocache !=3D 0 ?=20
                            bpage->vaddr_nocache : bpage->vaddr),
                            (void *)bpage->datavaddr, bpage->datacount);
                        dmat->bounce_zone->total_bounced++;
                }

USB is currently _updating_ (!!) the PAGE offset part of "vaddr". If=20
cpu_dcache_inv_range() is called with an address not starting at the cache=
=20
line what will the cpu_dcache_inv_range() do? Will it skip to the next cach=
e=20
line? Or will it completely skip the whole cache sync operation?! In the=20
function just above in the file I refer to, "bus_dmamap_sync_buf()", there =
is=20
quite some more code to handle invalidation when the destination address is=
=20
not properly aligned. I'm not an ARM expert. Maybe you can do an experiment=
=20
for me:

Change: bpage->vaddr into (bpage->vaddr & ~arm_dcache_align_mask)
Change: bpage->datacount into (bpage->datacount + (bpage->vaddr &=20
arm_dcache_align_mask) + arm_dcache_align_mask - 1) & ~arm_dcache_align_mas=
k;

You don't need any memcpy there, because we own the complete memory page wh=
ich=20
vaddr is a part of!

>
> Ok. So if you use bus_dma(), please use it in correct way.

What has to be changed in busdma_machdep.c for ARM/MIPS so that the problem=
 is=20
resolved. I think there are something missing there and not in USB!

=2D-HPS




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200906291337.43635.hselasky>