Date: Mon, 7 Jan 2013 17:37:40 -0800 From: Mehmet Erol Sanliturk <m.e.sanliturk@gmail.com> To: "Dr. Rolf Jansen" <rj@cyclaero.com> Cc: freebsd-drivers@freebsd.org Subject: Re: How to map device addresses into user space Message-ID: <CAOgwaMsSZfr0SRi4t6sXvGP_zh2dmetQK9R39b9%2BY4cUUnDc=w@mail.gmail.com> In-Reply-To: <8BDABFD4-D371-49D4-B21C-EBCD7C86B47C@cyclaero.com> References: <DFC983B8-91B2-4ED8-89B0-FC4BD4AB2576@cyclaero.com> <201301071011.49234.jhb@freebsd.org> <F62E8A3F-7D16-49F9-9375-5FFE2CBAAF09@cyclaero.com> <201301071421.10559.jhb@freebsd.org> <8BDABFD4-D371-49D4-B21C-EBCD7C86B47C@cyclaero.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Jan 7, 2013 at 3:40 PM, Dr. Rolf Jansen <rj@cyclaero.com> wrote:
> Am 07.01.2013 um 17:21 schrieb John Baldwin:
>
> > On Monday, January 07, 2013 12:25:33 PM Dr. Rolf Jansen wrote:
> >> Am 07.01.2013 um 13:11 schrieb John Baldwin:
> >>> On Monday, January 07, 2013 08:52:01 AM Dr. Rolf Jansen wrote:
> >>>> Am 03.01.2013 um 14:45 schrieb Dr. Rolf Jansen:
> >>>>> ...
> >>>>>
> >>>>> So, how can I map device addresses into user space on FreeBSD?
> >>>>
> >>>> Many Thanks to everybody, who responded.
> >>>>
> >>>> I got it working. I wrote a PCI device driver, which in addition to
> >>>> open/close/read/write responds also to ioctl and mmap requests from
> user
> >>>> space. In its pci_attach routine, it assembles a custom address_space
> >>>> struct containing the addresses of the two BAR's of my NI PCI-DAQ
> board
> >>>> and two huge allocated memory regions for DMA...
> >>>
> >>> Note that if your BARs are not contiguous you can allow mapping of
> things
> >>> in the middle. This is a potentital security hole if your driver can
> be
> >>> opened by non-root.
> >>
> >> I can understand this. Actually, on the present model of the DAQ board,
> the
> >> BARs are contiguous, and in the present incarnation, my driver can only
> be
> >> addressed by root. Anyway, I want to do it correctly, that means, I
> would
> >> like to avoid right now potential holes bubbling up in the future, when
> >> using my driver with other DAQ boards, and in different usage scenarios.
> >
> > Note that the BARs being contiguous is a property of your BIOS or OS,
> not your
> > board. :) The firmware/OS are free to assign whatever ranges to your
> BARs
> > that they wish.
> >
> >>> A more robust approach is to define a virtual address space for
> >>> your device. This is also a lot easier to program against in userland.
> >>> For example, you could define offset 0 as mapping to BAR 0 and the next
> >>> chunk
> >>
> >>> of virtual address space (the offset) map to BAR 1, etc.:
> >> I did this, but id didn't work out. The 2 BARs are contiguous, and the 2
> >> DMA regions are also, but there is huge gap between BAR and DMA. And
> >> depending on which mapping is done first, either the DMAs or the BARs
> >> receive invalid paddr. So, I guess, defining a virtual address space is
> a
> >> little bit more involved than simply getting the offsets straight as you
> >> lined out.
> >>
> >> I am still learning things, so please bear with me, if it is a dumb
> >> question. How do I define a virtual address space for the device?
> >
> > Ah, I'll explain a bit. When you mmap a section of a file, the OS is
> treating
> > the file as if it were its own section of virtual memory, where files
> contents
> > were addressed by the offset. That is, virtual address 0 of the file
> > corresponds to the data at file offset 0, and virtual address N
> corresponds to
> > the data at file offset N.
> >
> > Now, think of your /dev/foo device as a file. When you mmap() your
> /dev/foo,
> > the OS has assumes it has a similar virtual memory that is addressed by
> the
> > offset into the file. However, for a character device, the OS needs the
> > driver to describe this virtual address space. That is what you are
> doing
> > with d_mmap() and the 'offset' parameter. You are telling it what
> physical
> > page backs each "virtual page". It is important to note that when using
> > d_mmap(), these mappings are immutable. Once you provide a mapping for a
> > given virtual page, the OS will cache it forever (or until reboot).
> >
> > In other words, you can write your d_mmap() to give the appearance that
> your
> > /dev/foo "file" contains BAR 0 followed by BAR 1, etc.
> >
> >>> static int
> >>> mydaq_mmap(...)
> >>> {
> >>>
> >>> struct mydaq_softc *sc; // you should be storing things like the
> >>> pointers
> >>>
> >>> // to your BARs in a softc
> >>>
> >>> sc = dev->si_drv1;
> >>> if (offset <= rman_get_size(sc->bar0res))
> >>>
> >>> return (rman_get_start(sc->bar0res) + offset);
> >>>
> >>> offset -= rman_get_size(sc->bar0res);
> >>> if (offset <= rman_get_size(sc->bar1res))
> >>>
> >>> return (rman_get_start(sc->bar1res) + offset);
> >>>
> >>> offset -= rman_get_size(sc->bar1res);
> >>> if (offset <= dmaAIBufSize)
> >>>
> >>> return (vtophys(sc->dma[0]) + offset);
> >>>
> >>> offset -= dmaAIBufSize;
> >>> if (offset <= dmaAOBufSize)
> >>>
> >>> return (vtophys(sc->dma[1]) + offset);
> >>>
> >>> /* Invalid offset */
> >>> return (-1);
> >>>
> >>> }
> >>
> >> I guess, you hacked this quickly together, and I picked the concept, so
> >> this is fine. In order to make this working, the addresses should be
> >> assigned to the *paddr parameter, and a status code should be returned.
> In
> >> order to check things, of course I applied these minor corrections, but
> it
> >> doesn't work for either the BAR block or the DMA block, depending on
> which
> >> is mapped first.
> >
> > Yes, it needs the changes you made. Can you post your updated d_mmap()
> > routine along with how you are mapping it in userland? With what I
> posted
> > above you would do something like this in userland:
> >
> > const char *bar0, *bar1;
> >
> > int fd = open("/dev/foo");
> >
> > bar0 = mmap(NULL, <size of BAR 0>, PROT_READ, MAP_SHARED, fd, 0);
> > bar1 = mmap(NULL, <size of BAR 1>, PROT_READ, MAP_SHARED, fd,
> > <size of BAR 0>);
> >
> > etc.
>
>
> Yes, now I understood it. I realized, that mydaq_mmap() is not called only
> once
> for each mapping, but once for each page in each mapping. And finally I
> got it
> working. Besides the corrections already mentioned, the comparison
> operators
> needed to be changed from "<=" to "<". The working d_mmap() is now:
>
>
> static int mydaq_mmap(struct cdev *dev, vm_ooffset_t offset, vm_paddr_t
> *paddr,
> int nprot, vm_memattr_t *memattr)
> {
> mydaq_pci_softc *sc = dev->si_drv1;
>
> if (offset < rman_get_size(sc->bar0res))
> {
> *paddr = rman_get_start(sc->bar0res) + offset;
> return 0;
> }
>
> offset -= rman_get_size(sc->bar0res);
> if (offset < rman_get_size(sc->bar1res))
> {
> *paddr = rman_get_start(sc->bar1res) + offset;
> return 0;
> }
>
> offset -= rman_get_size(sc->bar1res);
> if (offset < dmaAIBufferSize)
> {
> *paddr = vtophys(sc->dma[0]) + offset;
> return 0;
> }
>
> offset -= dmaAIBufferSize;
> if (offset < dmaAOBufferSize)
> {
> *paddr = vtophys(sc->dma[1]) + offset;
> return 0;
> }
>
> return -1;
> }
>
>
> Thank you very much again, for the big help.
>
> Best regards
>
> Rolf
> _______________________________________________
>
>
If it is suitable for you , is it possible to supply your sources with a
permissive license for
using them / studying them and learning how to develop such a software ?
Thank you very much .
Mehmet Erol Sanliturk
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOgwaMsSZfr0SRi4t6sXvGP_zh2dmetQK9R39b9%2BY4cUUnDc=w>