Date: Thu, 22 Mar 2012 08:11:12 -0400 From: John Baldwin <jhb@freebsd.org> To: freebsd-hackers@freebsd.org Cc: Eric Saint-Etienne <eric.saintetienne@gmail.com> Subject: Re: malloc pages map to user space Message-ID: <201203220811.12954.jhb@freebsd.org> In-Reply-To: <CACJX%2B52DPPYea5k6QxQ6WQ62GEA9YHGr4y=ETUpPArNL83kh5w@mail.gmail.com> References: <CACJX%2B52DPPYea5k6QxQ6WQ62GEA9YHGr4y=ETUpPArNL83kh5w@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wednesday, March 21, 2012 7:27:58 pm Eric Saint-Etienne wrote: > Hi, > > >From within the freeBSD kernel, not all malloc are made equal: > * malloc() smaller than KMEM_ZMAX (set to one page size) end up in > UMA SLABs, themselves laid out in powers of 2 (from 16 bytes, 32... to > 4096 bytes) > * bigger malloc() are done through uma_large_malloc() which uses the > kmem wired space > > In my driver, I need to map some malloc-ed memory, obtained from > another module, into userspace. > > The problem: on the smaller mallocs, as well as on some bigeer ones > (8k seems fine, 64k fails): vm_map_lookup() fails finding the > underlying vm object. > > Do somebody know how (or better, have a piece of code!) to retrieve > the vm_object associated with malloc-ed memory? (small and big ones) > > As far as I can see in the vm code, there isn't any object associated > with the slabs (the smaller mallocs), it seems that a huge chunk of > virtual space is used "as is", so I presume the virtual addresses > where the SLABs are have some remarkable property, with respect to > physical addresses, that could allow creating an object from scratch? > > The usual answer is: use mmap(). It seems mmap() is the solution to > everything. But what I dislike with mmap() is the following cost *for > each page*: > 1/ a page fault > 2/ a call to a pager function that will do the "on demand" mapping. You can prefault each page in userland after you call mmap() if you want to pay the cost up front vs. later during runtime. However, there is another option you can use (though it might require you to rework your interfaces a bit, esp. depending on how you get the malloc'd memory in the first place). Specifically, I recently added an extension to the 'shm_open()' API in the kernel that lets you map a shared memory object into the kernel (shm_map() and shm_unmap(), the change should be easy to MFC to 8 and 9 (I use it in 8 and will MFC it soonish to those stable branches)). The way the workflow works in this case is that userland creates a shared memory object (usually an anonymous one, so using SHM_ANON), and then passes the fd in as part of an ioctl request to something in the kernel. The kernel code uses fget() to convert the fd to a reference to a 'struct file *'. You can then pass that file to shm_map(). Once the kernel code is done with the mapping it should call shm_unmap() to release it as well as using fdrop() to release the reference to the 'fp' obtained from fget(). shm_map() will wire down pages for the mapped region of the shm (and alloc them if needed). As a bonus then, if you mmap() the shm in userland after the ioctl, you can pass MAP_PREFAULT_READ to mmap() and it should remove all the page faults in userland. -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201203220811.12954.jhb>