Date: Tue, 18 Dec 2018 13:59:06 +0530 From: Rajesh Kumar <rajfbsd@gmail.com> To: cem@freebsd.org Cc: asomers@freebsd.org, freebsd-hackers@freebsd.org, freebsd-drivers@freebsd.org Subject: Re: How to use the DMA Engine in FreeBSD? Message-ID: <CAAO%2BANNHHJpMMgXuDSC7ftpTwPr1XPj9_ioYoA83WJooiELwog@mail.gmail.com> In-Reply-To: <CAAO%2BANPa0uTvZPbq%2BM%2BX1ozA5ytUoAG6RwXhyM5dGzq1ZasFiw@mail.gmail.com> References: <CAAO%2BANOty8FeOR7VQdXmxWy5y288pm=Q4cswHJ1BbgT2h1cUWg@mail.gmail.com> <CANCZdfrEv9o6_ydp9Qe73O1v6NqHFtivHpw4m2PKhTyAVaPR9g@mail.gmail.com> <26df913b-a2f8-2709-1cec-d11ad7d113a8@pix.net> <CAOtMX2jXMz3D8_JvOk4M2DrEZDP8sztgy2bdBuv-yNwz9zR2JQ@mail.gmail.com> <CAG6CVpV5hL%2Bk3OcJZmMfvJnSPo5wU1w8o0A85TbBAU49a661zA@mail.gmail.com> <CAAO%2BANPa0uTvZPbq%2BM%2BX1ozA5ytUoAG6RwXhyM5dGzq1ZasFiw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Any thoughts (or) suggestions here? On Mon, Dec 17, 2018 at 4:54 PM Rajesh Kumar <rajfbsd@gmail.com> wrote: > Hi, > > Thanks everyone for your inputs. I assume, "PCI busmaster"ing refers to a > peripheral device being made as the owner of the bus by the DMA engine and > do data transfer. Please correct me if I am wrong. But, will this bus_dma > API uses the DMA engine in hardware (does it support PCI busmastering)? > > As I understand, DMA engine will expose DMA channels, which can be used by > the client drivers to queue their DMA requests (without CPU intervention) > and completions are given through the callback functions. If this is > considered the real DMA operation, I don't see options to request DMA > channels using bus_dma API's, and the callbacks in bus_dma API is meant to > receive just the DMA map details (doesn't indicate a DMA completion). > ioat(4) driver seems to be doing what I described above, but that seems to > be Intel specific. Can it be used with any DMA controllers? > > So, I am wondering what needs to be done in FreeBSD to do the actual DMA > involving DMA engine in a more generic way? > > Thanks, > Rajesh. > > > On Sat, Dec 15, 2018 at 6:30 AM Conrad Meyer <cem@freebsd.org> wrote: > >> On Fri, Dec 14, 2018 at 8:17 AM Alan Somers <asomers@freebsd.org> wrote: >> > > For some Intel based server hardware, there is the "ioat" driver, >> which >> > > allows for user code to schedule DMA operations. See ioat(4) for >> > > details, including a pointer to the test program. >> >> This isn't true (or at least, only for testing). It's only usable by >> the kernel at the moment. >> >> > * In what context are callbacks called? Are they called from a signal >> > handler, or in a separate thread, or something else? >> >> Directly from the ithread, mostly. Callers can't take sleep locks or >> do very much besides set a flag and wakeup, or kick off a callout or >> taskqueue. >> >> > * Why isn't ioat.h installed? >> >> Why would it be? It's a kernel interface. >> >> > * Are "interrupts" synonymous with callbacks? >> >> I don't understand the question. Interrupts are synonymous with >> interrupts. Like, MSI-X. >> >> > * Do you have a rough idea for about the minimum buffer size that >> > makes sense to use with ioat? >> >> What makes sense will depend on your use case. It's not necessarily >> faster than CPU memcpy, but it may be worth some slowdown to offload >> latency-insensitive bulk copies. (It may make a lot of sense for >> asynchronous page zero, if someone wants to bring that back.) It's >> worth benchmarking your specific model. >> >> We use it for >= 8kB copies on Broadwell, although that's largely an >> artifact of our use case (write sizes are exactly 8 bytes, 512 bytes, >> or multiples of 8kB). IIRC, it shows up as a gen3 PCIe device with >> some small number of lanes on Broadwell, although I may be mistaken; >> and it may not communicate with RAM over the PCIe bus anyway, given >> the tight integration with the CPU. >> >> Best, >> Conrad >> _______________________________________________ >> freebsd-drivers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-drivers >> To unsubscribe, send any mail to "freebsd-drivers-unsubscribe@freebsd.org >> " >> >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAAO%2BANNHHJpMMgXuDSC7ftpTwPr1XPj9_ioYoA83WJooiELwog>