Date: Fri, 14 Dec 2018 16:59:37 -0800 From: Conrad Meyer <cem@freebsd.org> To: Alan Somers <asomers@freebsd.org> Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>, freebsd-drivers@freebsd.org Subject: Re: How to use the DMA Engine in FreeBSD? Message-ID: <CAG6CVpV5hL%2Bk3OcJZmMfvJnSPo5wU1w8o0A85TbBAU49a661zA@mail.gmail.com> In-Reply-To: <CAOtMX2jXMz3D8_JvOk4M2DrEZDP8sztgy2bdBuv-yNwz9zR2JQ@mail.gmail.com> References: <CAAO%2BANOty8FeOR7VQdXmxWy5y288pm=Q4cswHJ1BbgT2h1cUWg@mail.gmail.com> <CANCZdfrEv9o6_ydp9Qe73O1v6NqHFtivHpw4m2PKhTyAVaPR9g@mail.gmail.com> <26df913b-a2f8-2709-1cec-d11ad7d113a8@pix.net> <CAOtMX2jXMz3D8_JvOk4M2DrEZDP8sztgy2bdBuv-yNwz9zR2JQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Dec 14, 2018 at 8:17 AM Alan Somers <asomers@freebsd.org> wrote: > > For some Intel based server hardware, there is the "ioat" driver, which > > allows for user code to schedule DMA operations. See ioat(4) for > > details, including a pointer to the test program. This isn't true (or at least, only for testing). It's only usable by the kernel at the moment. > * In what context are callbacks called? Are they called from a signal > handler, or in a separate thread, or something else? Directly from the ithread, mostly. Callers can't take sleep locks or do very much besides set a flag and wakeup, or kick off a callout or taskqueue. > * Why isn't ioat.h installed? Why would it be? It's a kernel interface. > * Are "interrupts" synonymous with callbacks? I don't understand the question. Interrupts are synonymous with interrupts. Like, MSI-X. > * Do you have a rough idea for about the minimum buffer size that > makes sense to use with ioat? What makes sense will depend on your use case. It's not necessarily faster than CPU memcpy, but it may be worth some slowdown to offload latency-insensitive bulk copies. (It may make a lot of sense for asynchronous page zero, if someone wants to bring that back.) It's worth benchmarking your specific model. We use it for >= 8kB copies on Broadwell, although that's largely an artifact of our use case (write sizes are exactly 8 bytes, 512 bytes, or multiples of 8kB). IIRC, it shows up as a gen3 PCIe device with some small number of lanes on Broadwell, although I may be mistaken; and it may not communicate with RAM over the PCIe bus anyway, given the tight integration with the CPU. Best, Conrad
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAG6CVpV5hL%2Bk3OcJZmMfvJnSPo5wU1w8o0A85TbBAU49a661zA>