From owner-freebsd-drivers@FreeBSD.ORG Mon Jan 7 20:13:20 2013 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id B8478416 for ; Mon, 7 Jan 2013 20:13:20 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 6139B307 for ; Mon, 7 Jan 2013 20:13:20 +0000 (UTC) Received: from ralph.baldwin.cx (c-68-39-198-164.hsd1.de.comcast.net [68.39.198.164]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 87B07B97C; Mon, 7 Jan 2013 15:13:19 -0500 (EST) From: John Baldwin To: "Dr. Rolf Jansen" Subject: Re: How to map device addresses into user space Date: Mon, 7 Jan 2013 14:21:10 -0500 User-Agent: KMail/1.13.7 (FreeBSD/9.1-PRERELEASE; KDE/4.8.4; amd64; ; ) References: <201301071011.49234.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201301071421.10559.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 07 Jan 2013 15:13:19 -0500 (EST) Cc: freebsd-drivers@freebsd.org X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2013 20:13:20 -0000 On Monday, January 07, 2013 12:25:33 PM Dr. Rolf Jansen wrote: > Am 07.01.2013 um 13:11 schrieb John Baldwin: > > On Monday, January 07, 2013 08:52:01 AM Dr. Rolf Jansen wrote: > >> Am 03.01.2013 um 14:45 schrieb Dr. Rolf Jansen: > >>> ... > >>> > >>> So, how can I map device addresses into user space on FreeBSD? > >> > >> Many Thanks to everybody, who responded. > >> > >> I got it working. I wrote a PCI device driver, which in addition to > >> open/close/read/write responds also to ioctl and mmap requests from user > >> space. In its pci_attach routine, it assembles a custom address_space > >> struct containing the addresses of the two BAR's of my NI PCI-DAQ board > >> and two huge allocated memory regions for DMA... > > > > Note that if your BARs are not contiguous you can allow mapping of things > > in the middle. This is a potentital security hole if your driver can be > > opened by non-root. > > I can understand this. Actually, on the present model of the DAQ board, the > BARs are contiguous, and in the present incarnation, my driver can only be > addressed by root. Anyway, I want to do it correctly, that means, I would > like to avoid right now potential holes bubbling up in the future, when > using my driver with other DAQ boards, and in different usage scenarios. Note that the BARs being contiguous is a property of your BIOS or OS, not your board. :) The firmware/OS are free to assign whatever ranges to your BARs that they wish. > > A more robust approach is to define a virtual address space for > > your device. This is also a lot easier to program against in userland. > > For example, you could define offset 0 as mapping to BAR 0 and the next > > chunk > > > of virtual address space (the offset) map to BAR 1, etc.: > I did this, but id didn't work out. The 2 BARs are contiguous, and the 2 > DMA regions are also, but there is huge gap between BAR and DMA. And > depending on which mapping is done first, either the DMAs or the BARs > receive invalid paddr. So, I guess, defining a virtual address space is a > little bit more involved than simply getting the offsets straight as you > lined out. > > I am still learning things, so please bear with me, if it is a dumb > question. How do I define a virtual address space for the device? Ah, I'll explain a bit. When you mmap a section of a file, the OS is treating the file as if it were its own section of virtual memory, where files contents were addressed by the offset. That is, virtual address 0 of the file corresponds to the data at file offset 0, and virtual address N corresponds to the data at file offset N. Now, think of your /dev/foo device as a file. When you mmap() your /dev/foo, the OS has assumes it has a similar virtual memory that is addressed by the offset into the file. However, for a character device, the OS needs the driver to describe this virtual address space. That is what you are doing with d_mmap() and the 'offset' parameter. You are telling it what physical page backs each "virtual page". It is important to note that when using d_mmap(), these mappings are immutable. Once you provide a mapping for a given virtual page, the OS will cache it forever (or until reboot). In other words, you can write your d_mmap() to give the appearance that your /dev/foo "file" contains BAR 0 followed by BAR 1, etc. > > static int > > mydaq_mmap(...) > > { > > > > struct mydaq_softc *sc; // you should be storing things like the > > pointers > > > > // to your BARs in a softc > > > > sc = dev->si_drv1; > > if (offset <= rman_get_size(sc->bar0res)) > > > > return (rman_get_start(sc->bar0res) + offset); > > > > offset -= rman_get_size(sc->bar0res); > > if (offset <= rman_get_size(sc->bar1res)) > > > > return (rman_get_start(sc->bar1res) + offset); > > > > offset -= rman_get_size(sc->bar1res); > > if (offset <= dmaAIBufSize) > > > > return (vtophys(sc->dma[0]) + offset); > > > > offset -= dmaAIBufSize; > > if (offset <= dmaAOBufSize) > > > > return (vtophys(sc->dma[1]) + offset); > > > > /* Invalid offset */ > > return (-1); > > > > } > > I guess, you hacked this quickly together, and I picked the concept, so > this is fine. In order to make this working, the addresses should be > assigned to the *paddr parameter, and a status code should be returned. In > order to check things, of course I applied these minor corrections, but it > doesn't work for either the BAR block or the DMA block, depending on which > is mapped first. Yes, it needs the changes you made. Can you post your updated d_mmap() routine along with how you are mapping it in userland? With what I posted above you would do something like this in userland: const char *bar0, *bar1; int fd = open("/dev/foo"); bar0 = mmap(NULL, , PROT_READ, MAP_SHARED, fd, 0); bar1 = mmap(NULL, , PROT_READ, MAP_SHARED, fd, ); etc. -- John Baldwin