Date: Mon, 17 Dec 2018 16:54:30 +0530 From: Rajesh Kumar <rajfbsd@gmail.com> To: cem@freebsd.org Cc: asomers@freebsd.org, freebsd-hackers@freebsd.org, freebsd-drivers@freebsd.org Subject: Re: How to use the DMA Engine in FreeBSD? Message-ID: <CAAO%2BANPa0uTvZPbq%2BM%2BX1ozA5ytUoAG6RwXhyM5dGzq1ZasFiw@mail.gmail.com> In-Reply-To: <CAG6CVpV5hL%2Bk3OcJZmMfvJnSPo5wU1w8o0A85TbBAU49a661zA@mail.gmail.com> References: <CAAO%2BANOty8FeOR7VQdXmxWy5y288pm=Q4cswHJ1BbgT2h1cUWg@mail.gmail.com> <CANCZdfrEv9o6_ydp9Qe73O1v6NqHFtivHpw4m2PKhTyAVaPR9g@mail.gmail.com> <26df913b-a2f8-2709-1cec-d11ad7d113a8@pix.net> <CAOtMX2jXMz3D8_JvOk4M2DrEZDP8sztgy2bdBuv-yNwz9zR2JQ@mail.gmail.com> <CAG6CVpV5hL%2Bk3OcJZmMfvJnSPo5wU1w8o0A85TbBAU49a661zA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, Thanks everyone for your inputs. I assume, "PCI busmaster"ing refers to a peripheral device being made as the owner of the bus by the DMA engine and do data transfer. Please correct me if I am wrong. But, will this bus_dma API uses the DMA engine in hardware (does it support PCI busmastering)? As I understand, DMA engine will expose DMA channels, which can be used by the client drivers to queue their DMA requests (without CPU intervention) and completions are given through the callback functions. If this is considered the real DMA operation, I don't see options to request DMA channels using bus_dma API's, and the callbacks in bus_dma API is meant to receive just the DMA map details (doesn't indicate a DMA completion). ioat(4) driver seems to be doing what I described above, but that seems to be Intel specific. Can it be used with any DMA controllers? So, I am wondering what needs to be done in FreeBSD to do the actual DMA involving DMA engine in a more generic way? Thanks, Rajesh. On Sat, Dec 15, 2018 at 6:30 AM Conrad Meyer <cem@freebsd.org> wrote: > On Fri, Dec 14, 2018 at 8:17 AM Alan Somers <asomers@freebsd.org> wrote: > > > For some Intel based server hardware, there is the "ioat" driver, which > > > allows for user code to schedule DMA operations. See ioat(4) for > > > details, including a pointer to the test program. > > This isn't true (or at least, only for testing). It's only usable by > the kernel at the moment. > > > * In what context are callbacks called? Are they called from a signal > > handler, or in a separate thread, or something else? > > Directly from the ithread, mostly. Callers can't take sleep locks or > do very much besides set a flag and wakeup, or kick off a callout or > taskqueue. > > > * Why isn't ioat.h installed? > > Why would it be? It's a kernel interface. > > > * Are "interrupts" synonymous with callbacks? > > I don't understand the question. Interrupts are synonymous with > interrupts. Like, MSI-X. > > > * Do you have a rough idea for about the minimum buffer size that > > makes sense to use with ioat? > > What makes sense will depend on your use case. It's not necessarily > faster than CPU memcpy, but it may be worth some slowdown to offload > latency-insensitive bulk copies. (It may make a lot of sense for > asynchronous page zero, if someone wants to bring that back.) It's > worth benchmarking your specific model. > > We use it for >= 8kB copies on Broadwell, although that's largely an > artifact of our use case (write sizes are exactly 8 bytes, 512 bytes, > or multiples of 8kB). IIRC, it shows up as a gen3 PCIe device with > some small number of lanes on Broadwell, although I may be mistaken; > and it may not communicate with RAM over the PCIe bus anyway, given > the tight integration with the CPU. > > Best, > Conrad > _______________________________________________ > freebsd-drivers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-drivers > To unsubscribe, send any mail to "freebsd-drivers-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAAO%2BANPa0uTvZPbq%2BM%2BX1ozA5ytUoAG6RwXhyM5dGzq1ZasFiw>