Date: Mon, 7 Jan 2013 21:40:11 -0200 From: "Dr. Rolf Jansen" <rj@cyclaero.com> To: freebsd-drivers@freebsd.org Subject: Re: How to map device addresses into user space Message-ID: <8BDABFD4-D371-49D4-B21C-EBCD7C86B47C@cyclaero.com> In-Reply-To: <201301071421.10559.jhb@freebsd.org> References: <DFC983B8-91B2-4ED8-89B0-FC4BD4AB2576@cyclaero.com> <201301071011.49234.jhb@freebsd.org> <F62E8A3F-7D16-49F9-9375-5FFE2CBAAF09@cyclaero.com> <201301071421.10559.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Am 07.01.2013 um 17:21 schrieb John Baldwin:
> On Monday, January 07, 2013 12:25:33 PM Dr. Rolf Jansen wrote:
>> Am 07.01.2013 um 13:11 schrieb John Baldwin:
>>> On Monday, January 07, 2013 08:52:01 AM Dr. Rolf Jansen wrote:
>>>> Am 03.01.2013 um 14:45 schrieb Dr. Rolf Jansen:
>>>>> ...
>>>>>=20
>>>>> So, how can I map device addresses into user space on FreeBSD?
>>>>=20
>>>> Many Thanks to everybody, who responded.
>>>>=20
>>>> I got it working. I wrote a PCI device driver, which in addition to
>>>> open/close/read/write responds also to ioctl and mmap requests from =
user
>>>> space. In its pci_attach routine, it assembles a custom =
address_space
>>>> struct containing the addresses of the two BAR's of my NI PCI-DAQ =
board
>>>> and two huge allocated memory regions for DMA...
>>>=20
>>> Note that if your BARs are not contiguous you can allow mapping of =
things
>>> in the middle. This is a potentital security hole if your driver =
can be
>>> opened by non-root.
>>=20
>> I can understand this. Actually, on the present model of the DAQ =
board, the
>> BARs are contiguous, and in the present incarnation, my driver can =
only be
>> addressed by root. Anyway, I want to do it correctly, that means, I =
would
>> like to avoid right now potential holes bubbling up in the future, =
when
>> using my driver with other DAQ boards, and in different usage =
scenarios.
>=20
> Note that the BARs being contiguous is a property of your BIOS or OS, =
not your=20
> board. :) The firmware/OS are free to assign whatever ranges to your =
BARs=20
> that they wish.
>=20
>>> A more robust approach is to define a virtual address space for
>>> your device. This is also a lot easier to program against in =
userland.
>>> For example, you could define offset 0 as mapping to BAR 0 and the =
next
>>> chunk
>>=20
>>> of virtual address space (the offset) map to BAR 1, etc.:
>> I did this, but id didn't work out. The 2 BARs are contiguous, and =
the 2
>> DMA regions are also, but there is huge gap between BAR and DMA. And
>> depending on which mapping is done first, either the DMAs or the BARs
>> receive invalid paddr. So, I guess, defining a virtual address space =
is a
>> little bit more involved than simply getting the offsets straight as =
you
>> lined out.
>>=20
>> I am still learning things, so please bear with me, if it is a dumb
>> question. How do I define a virtual address space for the device?
>=20
> Ah, I'll explain a bit. When you mmap a section of a file, the OS is =
treating=20
> the file as if it were its own section of virtual memory, where files =
contents=20
> were addressed by the offset. That is, virtual address 0 of the file=20=
> corresponds to the data at file offset 0, and virtual address N =
corresponds to=20
> the data at file offset N.
>=20
> Now, think of your /dev/foo device as a file. When you mmap() your =
/dev/foo,=20
> the OS has assumes it has a similar virtual memory that is addressed =
by the=20
> offset into the file. However, for a character device, the OS needs =
the=20
> driver to describe this virtual address space. That is what you are =
doing=20
> with d_mmap() and the 'offset' parameter. You are telling it what =
physical=20
> page backs each "virtual page". It is important to note that when =
using=20
> d_mmap(), these mappings are immutable. Once you provide a mapping =
for a=20
> given virtual page, the OS will cache it forever (or until reboot).
>=20
> In other words, you can write your d_mmap() to give the appearance =
that your=20
> /dev/foo "file" contains BAR 0 followed by BAR 1, etc.
>=20
>>> static int
>>> mydaq_mmap(...)
>>> {
>>>=20
>>> struct mydaq_softc *sc; // you should be storing things like =
the
>>> pointers
>>> =09
>>> // to your BARs in a softc
>>> =09
>>> sc =3D dev->si_drv1;
>>> if (offset <=3D rman_get_size(sc->bar0res))
>>> =09
>>> return (rman_get_start(sc->bar0res) + offset);
>>> =09
>>> offset -=3D rman_get_size(sc->bar0res);
>>> if (offset <=3D rman_get_size(sc->bar1res))
>>> =09
>>> return (rman_get_start(sc->bar1res) + offset);
>>> =09
>>> offset -=3D rman_get_size(sc->bar1res);
>>> if (offset <=3D dmaAIBufSize)
>>> =09
>>> return (vtophys(sc->dma[0]) + offset);
>>> =09
>>> offset -=3D dmaAIBufSize;
>>> if (offset <=3D dmaAOBufSize)
>>> =09
>>> return (vtophys(sc->dma[1]) + offset);
>>> =09
>>> /* Invalid offset */
>>> return (-1);
>>>=20
>>> }
>>=20
>> I guess, you hacked this quickly together, and I picked the concept, =
so
>> this is fine. In order to make this working, the addresses should be
>> assigned to the *paddr parameter, and a status code should be =
returned. In
>> order to check things, of course I applied these minor corrections, =
but it
>> doesn't work for either the BAR block or the DMA block, depending on =
which
>> is mapped first.
>=20
> Yes, it needs the changes you made. Can you post your updated =
d_mmap()
> routine along with how you are mapping it in userland? With what I =
posted
> above you would do something like this in userland:
>=20
> const char *bar0, *bar1;
>=20
> int fd =3D open("/dev/foo");
>=20
> bar0 =3D mmap(NULL, <size of BAR 0>, PROT_READ, MAP_SHARED, fd, 0);
> bar1 =3D mmap(NULL, <size of BAR 1>, PROT_READ, MAP_SHARED, fd,
> <size of BAR 0>);
>=20
> etc.
Yes, now I understood it. I realized, that mydaq_mmap() is not called =
only once
for each mapping, but once for each page in each mapping. And finally I =
got it
working. Besides the corrections already mentioned, the comparison =
operators
needed to be changed from "<=3D" to "<". The working d_mmap() is now:
static int mydaq_mmap(struct cdev *dev, vm_ooffset_t offset, vm_paddr_t =
*paddr,
int nprot, vm_memattr_t *memattr)
{
mydaq_pci_softc *sc =3D dev->si_drv1;
if (offset < rman_get_size(sc->bar0res))
{
*paddr =3D rman_get_start(sc->bar0res) + offset;
return 0;
}
offset -=3D rman_get_size(sc->bar0res);
if (offset < rman_get_size(sc->bar1res))
{
*paddr =3D rman_get_start(sc->bar1res) + offset;
return 0;
}
offset -=3D rman_get_size(sc->bar1res);
if (offset < dmaAIBufferSize)
{
*paddr =3D vtophys(sc->dma[0]) + offset;
return 0;
}
offset -=3D dmaAIBufferSize;
if (offset < dmaAOBufferSize)
{
*paddr =3D vtophys(sc->dma[1]) + offset;
return 0;
}
return -1;
}
Thank you very much again, for the big help.
Best regards
Rolf=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8BDABFD4-D371-49D4-B21C-EBCD7C86B47C>