Date: Sun, 16 Apr 2023 09:06:58 +0200 From: Dmitry Salychev <dsl@mcusim.org> To: Mitchell Horne <mhorne@freebsd.org> Cc: freebsd-arm@freebsd.org Subject: Re: About PHYS_TO_DMAP Message-ID: <86zg78s2bj.fsf@peasant.tower.home> In-Reply-To: <eec51f08-8f20-df60-d736-a9cf37f8b11f@freebsd.org> References: <86ildyucuv.fsf@peasant.tower.home> <eec51f08-8f20-df60-d736-a9cf37f8b11f@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Mitchell Horne <mhorne@freebsd.org> writes:
> On 4/14/23 04:31, Dmitry Salychev wrote:
>> Hi,
>> I'm struggling to understand which KVA will be returned by
>> PHYS_TO_DMAP
>> on arm64. For example, if I'll create a DMA tag this way:
>> bus_dma_tag_create(
>> bus_get_dma_tag(dev),
>> sc->buf_align, 0, /* alignment, boundary */
>> DMAP_MAX_PHYSADDR, /* low restricted addr */
>> DMAP_MIN_PHYSADDR, /* high restricted addr */
>
> I think you are confused about the purpose of lowaddr and
> highaddr. They specify the window of bus-space addresses that are
> *inaccessible* *to the device* for DMA. It does not prevent you from
> providing a buffer backed by physical memory within this range, but in
> this case bounce pages will be used as an intermediate to perform the
> DMA transaction.
>
> Most commonly, if the device can only address a 32-bit range, then the
> values lowaddr=BUS_SPACE_MAXADDR_32BIT and highaddr=BUS_SPACE_MAXADDR
> will be given. If the entire bus range is accessible to the device for
> DMA, a zero-sized window will be given: lowaddr=BUS_SPACE_MAXADDR,
> highaddr=BUS_SPACE_MAXADDR.
>
> This is all described in adequate detail in busdma(9), but it is still
> not easily understood without a lot of code study, in my experience.
>
According to busdma(9), the window contains all addresses *greater than*
lowaddr and *less than or equal to* highaddr, i.e. your example with the
32-bit range looks like:
allowed prohibited
------------------------------(xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
lowaddr highaddr
BUS_SPACE_MAXADDR_32BIT BUS_SPACE_MAXADDR
But my example looks a bit different:
prohibited allowed prohibited
xxxxxxxxxxxxxxxx]----------------------------(xxxxxxxxxxxxxxxxxxx>
highaddr lowaddr
DMAP_MIN_PHYSADDR DMAP_MAX_PHYSADDR
I've found the only example of such DMA tag configuration in
src/sys/dev/vnic/nicvf_queues.c at line 379 (rbdr_buff_dmat).
>> NULL, NULL, /* filter, filterarg */
>> BUF_SIZE, 1, /* maxsize, nsegments */
>> BUF_SIZE, 0, /* maxsegsize, flags */
>> NULL, NULL, /* lockfunc, lockarg */
>> &dmat);
>> in order to restrict any physical addresses but a window defined by
>> DMAP_MIN_PHYSADDR and DMAP_MAX_PHYSADDR. Later on when I'll be
>> mapping my mbuf (BUF_SIZE) with
>> bus_dmamap_load_mbuf_sg(dmat, dmap,
>> m, &segs, &nsegs, BUS_DMA_NOWAIT);
>> I expect that
>> m->m_data == PHYS_TO_DMAP(segs[0].ds_addr)
>
> Why do you expect or need this to be the case?
>
> busdma is not responsible for setting or modifying m_data, it comes
> from wherever you allocated the mbuf. Calling
> bus_dmamap_load_mbuf_sg() will prepare a DMA mapping where the m_data
> buffer is used as the source/destination for the DMA transaction, but
> busdma does not allocate the buffer itself.
>
I don't need this to be the case exactly, but I'd like to be able to
access a "frame annotation" (64 bytes long) which is located exactly at
the start of m_data buffer having that physical address is provided, i.e.
fa = (struct dpaa2_fa *) mbuf->m_data;
/* ... fa populated ... */
/* ... DMA transaction of the Tx frame (together with fa) ... */
/* ... Tx confirmation from HW (bus address of the frame only) ...*/
fa = (struct dpaa2_fa *) PHYS_TO_DMAP(paddr);
>> but it isn't true. Could somebody explain what exactly is returned
>> by
>> PHYS_TO_DMAP in this case and whether it's possible to translate
>> physical address to KVA as fast as possible (O(1) ideally).
>>
>
> PHYS_TO_DMAP is always a linear calculation of: physaddr + DMAP_MIN_ADDRESS.
>
> I do not think PHYS_TO_DMAP is in use at all in this example, or
> anywhere within busdma really.
>
Is there any other way to obtain KVA of a buffer mapped for DMA
transaction by the physical address? I've been crawling source code for
sometime already, but DMAP is the only thing I managed to find.
>> Regards,
>> Dmitry
>> --
>> Open source software/hardware enthusiast
>> hackaday.io/dsl | github.com/mcusim | patreon.com/salychev
>>
Regards,
Dmitry
--
Open source software/hardware enthusiast
hackaday.io/dsl | github.com/mcusim | patreon.com/salychev
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?86zg78s2bj.fsf>