From owner-freebsd-drivers@FreeBSD.ORG Tue Jan 8 01:37:42 2013 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 52B93A9F for ; Tue, 8 Jan 2013 01:37:42 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-vb0-f46.google.com (mail-vb0-f46.google.com [209.85.212.46]) by mx1.freebsd.org (Postfix) with ESMTP id D52891AA for ; Tue, 8 Jan 2013 01:37:41 +0000 (UTC) Received: by mail-vb0-f46.google.com with SMTP id b13so20320960vby.33 for ; Mon, 07 Jan 2013 17:37:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=EhXwZ2k74PXm2/evf8OdqYos6hTaGOm2SQPiROpV4nw=; b=tkLzZyifXAFoGLT4JAdAHM7Vf4+aaS8oYwXvHbH/mknn7ZXSFZbmcaDkfIqg/5F7ap 1H0cWhtcr4jHn0jH6s8qzZFcGMf1zP2RGC+Kk0EzNV1hv3hyh7+fJTNKnpWskJRguSDR sW/kj7+oPHlae0wPR/LXZJPrtcaeiNUrnxYk8+kzwfurUKQzNEdlQAld+r5ZfnzBaIVU 5wc6JVwKTYUQGwpkTf9gV6U6RV5AoADywG3niTSBPnaKgaApboKR6BdvpYRBwngmcLj4 18VmwPR8J7+T73h6zCxhq709jtsZB0Dbvnccgcex1UjUKnQL5qqUrlRseRhSeIHGstNQ 1kug== MIME-Version: 1.0 Received: by 10.52.93.77 with SMTP id cs13mr52386971vdb.118.1357609060709; Mon, 07 Jan 2013 17:37:40 -0800 (PST) Received: by 10.58.214.226 with HTTP; Mon, 7 Jan 2013 17:37:40 -0800 (PST) In-Reply-To: <8BDABFD4-D371-49D4-B21C-EBCD7C86B47C@cyclaero.com> References: <201301071011.49234.jhb@freebsd.org> <201301071421.10559.jhb@freebsd.org> <8BDABFD4-D371-49D4-B21C-EBCD7C86B47C@cyclaero.com> Date: Mon, 7 Jan 2013 17:37:40 -0800 Message-ID: Subject: Re: How to map device addresses into user space From: Mehmet Erol Sanliturk To: "Dr. Rolf Jansen" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-drivers@freebsd.org X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2013 01:37:42 -0000 On Mon, Jan 7, 2013 at 3:40 PM, Dr. Rolf Jansen wrote: > Am 07.01.2013 um 17:21 schrieb John Baldwin: > > > On Monday, January 07, 2013 12:25:33 PM Dr. Rolf Jansen wrote: > >> Am 07.01.2013 um 13:11 schrieb John Baldwin: > >>> On Monday, January 07, 2013 08:52:01 AM Dr. Rolf Jansen wrote: > >>>> Am 03.01.2013 um 14:45 schrieb Dr. Rolf Jansen: > >>>>> ... > >>>>> > >>>>> So, how can I map device addresses into user space on FreeBSD? > >>>> > >>>> Many Thanks to everybody, who responded. > >>>> > >>>> I got it working. I wrote a PCI device driver, which in addition to > >>>> open/close/read/write responds also to ioctl and mmap requests from > user > >>>> space. In its pci_attach routine, it assembles a custom address_space > >>>> struct containing the addresses of the two BAR's of my NI PCI-DAQ > board > >>>> and two huge allocated memory regions for DMA... > >>> > >>> Note that if your BARs are not contiguous you can allow mapping of > things > >>> in the middle. This is a potentital security hole if your driver can > be > >>> opened by non-root. > >> > >> I can understand this. Actually, on the present model of the DAQ board, > the > >> BARs are contiguous, and in the present incarnation, my driver can only > be > >> addressed by root. Anyway, I want to do it correctly, that means, I > would > >> like to avoid right now potential holes bubbling up in the future, when > >> using my driver with other DAQ boards, and in different usage scenarios. > > > > Note that the BARs being contiguous is a property of your BIOS or OS, > not your > > board. :) The firmware/OS are free to assign whatever ranges to your > BARs > > that they wish. > > > >>> A more robust approach is to define a virtual address space for > >>> your device. This is also a lot easier to program against in userland. > >>> For example, you could define offset 0 as mapping to BAR 0 and the next > >>> chunk > >> > >>> of virtual address space (the offset) map to BAR 1, etc.: > >> I did this, but id didn't work out. The 2 BARs are contiguous, and the 2 > >> DMA regions are also, but there is huge gap between BAR and DMA. And > >> depending on which mapping is done first, either the DMAs or the BARs > >> receive invalid paddr. So, I guess, defining a virtual address space is > a > >> little bit more involved than simply getting the offsets straight as you > >> lined out. > >> > >> I am still learning things, so please bear with me, if it is a dumb > >> question. How do I define a virtual address space for the device? > > > > Ah, I'll explain a bit. When you mmap a section of a file, the OS is > treating > > the file as if it were its own section of virtual memory, where files > contents > > were addressed by the offset. That is, virtual address 0 of the file > > corresponds to the data at file offset 0, and virtual address N > corresponds to > > the data at file offset N. > > > > Now, think of your /dev/foo device as a file. When you mmap() your > /dev/foo, > > the OS has assumes it has a similar virtual memory that is addressed by > the > > offset into the file. However, for a character device, the OS needs the > > driver to describe this virtual address space. That is what you are > doing > > with d_mmap() and the 'offset' parameter. You are telling it what > physical > > page backs each "virtual page". It is important to note that when using > > d_mmap(), these mappings are immutable. Once you provide a mapping for a > > given virtual page, the OS will cache it forever (or until reboot). > > > > In other words, you can write your d_mmap() to give the appearance that > your > > /dev/foo "file" contains BAR 0 followed by BAR 1, etc. > > > >>> static int > >>> mydaq_mmap(...) > >>> { > >>> > >>> struct mydaq_softc *sc; // you should be storing things like the > >>> pointers > >>> > >>> // to your BARs in a softc > >>> > >>> sc = dev->si_drv1; > >>> if (offset <= rman_get_size(sc->bar0res)) > >>> > >>> return (rman_get_start(sc->bar0res) + offset); > >>> > >>> offset -= rman_get_size(sc->bar0res); > >>> if (offset <= rman_get_size(sc->bar1res)) > >>> > >>> return (rman_get_start(sc->bar1res) + offset); > >>> > >>> offset -= rman_get_size(sc->bar1res); > >>> if (offset <= dmaAIBufSize) > >>> > >>> return (vtophys(sc->dma[0]) + offset); > >>> > >>> offset -= dmaAIBufSize; > >>> if (offset <= dmaAOBufSize) > >>> > >>> return (vtophys(sc->dma[1]) + offset); > >>> > >>> /* Invalid offset */ > >>> return (-1); > >>> > >>> } > >> > >> I guess, you hacked this quickly together, and I picked the concept, so > >> this is fine. In order to make this working, the addresses should be > >> assigned to the *paddr parameter, and a status code should be returned. > In > >> order to check things, of course I applied these minor corrections, but > it > >> doesn't work for either the BAR block or the DMA block, depending on > which > >> is mapped first. > > > > Yes, it needs the changes you made. Can you post your updated d_mmap() > > routine along with how you are mapping it in userland? With what I > posted > > above you would do something like this in userland: > > > > const char *bar0, *bar1; > > > > int fd = open("/dev/foo"); > > > > bar0 = mmap(NULL, , PROT_READ, MAP_SHARED, fd, 0); > > bar1 = mmap(NULL, , PROT_READ, MAP_SHARED, fd, > > ); > > > > etc. > > > Yes, now I understood it. I realized, that mydaq_mmap() is not called only > once > for each mapping, but once for each page in each mapping. And finally I > got it > working. Besides the corrections already mentioned, the comparison > operators > needed to be changed from "<=" to "<". The working d_mmap() is now: > > > static int mydaq_mmap(struct cdev *dev, vm_ooffset_t offset, vm_paddr_t > *paddr, > int nprot, vm_memattr_t *memattr) > { > mydaq_pci_softc *sc = dev->si_drv1; > > if (offset < rman_get_size(sc->bar0res)) > { > *paddr = rman_get_start(sc->bar0res) + offset; > return 0; > } > > offset -= rman_get_size(sc->bar0res); > if (offset < rman_get_size(sc->bar1res)) > { > *paddr = rman_get_start(sc->bar1res) + offset; > return 0; > } > > offset -= rman_get_size(sc->bar1res); > if (offset < dmaAIBufferSize) > { > *paddr = vtophys(sc->dma[0]) + offset; > return 0; > } > > offset -= dmaAIBufferSize; > if (offset < dmaAOBufferSize) > { > *paddr = vtophys(sc->dma[1]) + offset; > return 0; > } > > return -1; > } > > > Thank you very much again, for the big help. > > Best regards > > Rolf > _______________________________________________ > > If it is suitable for you , is it possible to supply your sources with a permissive license for using them / studying them and learning how to develop such a software ? Thank you very much . Mehmet Erol Sanliturk