Date: Mon, 27 Apr 2015 09:46:52 -0500 From: Jason Harmening <jason.harmening@gmail.com> To: Svatopluk Kraus <onwahe@gmail.com> Cc: Konstantin Belousov <kostikbel@gmail.com>, FreeBSD Arch <freebsd-arch@freebsd.org> Subject: Re: bus_dmamap_sync() for bounced client buffers from user address space Message-ID: <CAM=8qam%2B-VnvroYF9J30JWYDCMLt=r1RfgodkqfG3ZVWwDzROw@mail.gmail.com> In-Reply-To: <CAFHCsPXzU_Cu51bj8vc_5e7to4GqdFDUvHvwWM1mAxJ5LvemXw@mail.gmail.com> References: <CAFHCsPXMjge84AR2cR8KXMXWP4kH2YvuV_uqtPKUvn5C3ygknw@mail.gmail.com> <CAM=8qan-4SbKJaddrfkv=HG3n%2BHaOPDL5MEPS9DoaTvnhrJPZQ@mail.gmail.com> <20150425094152.GE2390@kib.kiev.ua> <CAFHCsPXzU_Cu51bj8vc_5e7to4GqdFDUvHvwWM1mAxJ5LvemXw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> > Using of vslock() is proposed method in bus_dma man page. IMO, the >> function looks complex and can be a big time eater. However, are you >> saying that vslock() does not work for that? Then for what reason >> does that function exist? >> > There's been some misunderstanding here, I think. If you use vslock (or vm_map_wire, which vslock wraps), then the UVAs should be safe from teardown and you should be able to use bcopy if you are in the correct context. See the post elsewhere in this thread where I dig through the sys_munmap path and find vm_map_delete waiting on system-wired PTEs. > > >> >> >> > The only safe method to work with the userspace regions is to >> > vm_fault_quick_hold() them to get hold on the pages, and then either >> > pass pages array down, or remap them in the KVA with pmap_qenter(). >> > >> >> >> So, even vm_fault_quick_hold() does not keep valid user mapping? >> > vm_fault_quick_hold_pages() doesn't do any bookkeeping on UVAs, only the underlying physical pages. That means it is possible for the UVA region to be munmap'ed if vm_fault_quick_hold_pages() has been used. So if you use vm_fault_quick_hold_pages() instead of vslock(), you can't use bus_dmamap_load_uio(UIO_USERSPACE) because that assumes valid UVA mappings. You must instead deal only with the underlying vm_page_t's, which means using _bus_dmamap_load_ma(). Here's my take on it: vslock(), as you mention, is very complex. It not only keeps the physical pages from being swapped out, but it also removes them from page queues (see https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054890.html) and does a lot of bookkeeping on the UVA mappings for those pages. Part of that involves simulating a pagefault, which as kib mentions can lead to a lot of UVA fragmentation. vm_fault_quick_hold_pages() is much cheaper and seems mostly intended for short-term DMA operations. So, you might use vslock() + bus_dmamap_load_uio() for long-duration DMA transfers, like continuous streaming to a circular buffer that could last minutes or longer. Then, the extra cost of the vslock will be amortized over the long time of the transfer, and UVA fragmentation will be less of a concern since you presumably will have a limited number of vslock() calls over the lifetime of the process. Also, you will probably be keeping the DMA map for a long duration anyway, so it should be OK to wait and call bus_dmamap_sync() in the process context. Since vslock() removed the pages from the page queues, there will also be less work for pagedaemon to do during the long transfer. OTOH, vm_fault_quick_hold_pages() + _bus_dmamap_load_ma() seems much better to do for frequent short transfers to widely-varying buffers, such as block I/O. The extra pagedaemon work is inconsequential here, and since the DMA operations are frequent and you may have many in-flight at once, the reduced setup cost and fragmentation are much more important.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAM=8qam%2B-VnvroYF9J30JWYDCMLt=r1RfgodkqfG3ZVWwDzROw>