Date: Mon, 13 Jun 2005 10:10:57 -0700 From: Julian Elischer <julian@elischer.org> To: Apache Xie <apachexm@hotmail.com> Cc: freebsd-hackers@freebsd.org Subject: Re: contigmalloc() and mmap() Message-ID: <42ADBE21.6030007@elischer.org> In-Reply-To: <BAY19-F33CED9E68F54579EE49ACCC4F00@phx.gbl> References: <BAY19-F33CED9E68F54579EE49ACCC4F00@phx.gbl>
next in thread | previous in thread | raw e-mail | index | archive | help
Apache Xie wrote: > I have some experiences in writing Linux device driver, > but I am new to FreeBSD kernel, although from the first > glimpse, there seems no big differences between the kernel > operations a char device driver can use, but I met some > problems when the driver is running in FreeBSD. > > Our device is an experimental communication card, which can > do remote DMA between two cards, which means the card in one > node can DMA memory data to network, and when data are transfered > to the card on another node, it will be DMAed to memory too. sounds like a firewire card :-) > > Because the card can only use contiguous physical memory for > DMA operations, so data in user process will be copied to an > contiguous memory buffer, then DMA will tranfer data in this > buffer, and this buffer is allocated by driver using __get_free_pages() > in Linux. The buffer is mmaped to user process space, so user > process can do the copy directly in user space, it can directly > orgnize data in this mmaped memory too. > > When I am porting my driver to FreeBSD, I use /dev/bktr driver as the > example, seems easily, just using contigmalloc() to allocate the > buffer, and in driver's _mmap() function, return the physical > address for each page to be mmaped. > > The problem is, in Linux, I allocate buffer in driver's ioctl() > function, and free it in a timer function, many processes may use the > driver > at the same time, each process use a different kernel buffer, when > the process first use the driver, it calls __get_free_pages() to allocate > kernel buffer, and when it exit, it trigger timer function, the timer > function will can free_pages() to free the buffer, so these two kernel > interfaces will be called frequently, but this usage pattern works > correctly in Linux. > > In FreeBSD, the driver works in the same pattern, but when a user process > mmap driver's buffer (allocated by contigmalloc()) and is killed, then > when > another process mmap the same buffer again, sometimes it cannot get > correct > data from the mmaped pages, which means the user space virtual aderess > may > not point to the correct physical page of driver's buffer, sometimes > the OS > even panic with some information such as "Trap 12, Page not present" etc. > > I browsed kernel tree, I found those drivers which use contigmalloc() and > contigfree() always call these two kernel interfaces in _attach() and > _detach(), but in my driver, I call contigmalloc() in ioctl(), and call > contigfree() in a callout function which is set by callout_reset(). > What I want to know is if contigmalloc() and contigfree() can only be > used > under some conditions? Maybe I don't understand the problem but.. I think the problem is that you want to keep a separate buffer for each user, ] while the drivers you are looking at expect to have only one buffer per device. One answer to this would be to make each user open a different 'instance' of the device. (i.e. a differnt minor number). otherwise there is no really good place to store the information. The device does not track users as such and even if it did, how would it track when a user process forks and becomes 2? it is not notified of this event. > > And recently, I modified my driver, I allocated a big chunk of contiguous > physical memory using contigmalloc() in the driver's _attach() function, > and I use a simple first-fit algorithm to manage this memory myself, > which > mean in ioctl() I use my allocate/deallocate functions instead of > contigmalloc(), > in _detach() function contigfree() is called to free the big chunk of > memory, > no panic again, but sometimes, process cannot get the correct data from > the mmaped memory. I don't know why. > > Any help is welcomed. > > Thanks. > > _________________________________________________________________ > Express yourself instantly with MSN Messenger! Download today it's > FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/ > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?42ADBE21.6030007>