From owner-freebsd-virtualization@FreeBSD.ORG Wed Sep 28 04:21:37 2011 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C46A0106564A for ; Wed, 28 Sep 2011 04:21:37 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 967D68FC14 for ; Wed, 28 Sep 2011 04:21:37 +0000 (UTC) Received: from julian-mac.elischer.org (70-90-212-91-saltlake.hfc.comcastbusiness.net [70.90.212.91]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id p8S4LY4b058996 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 27 Sep 2011 21:21:36 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <4E82A103.9010900@freebsd.org> Date: Tue, 27 Sep 2011 22:22:27 -0600 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.22) Gecko/20110902 Thunderbird/3.1.14 MIME-Version: 1.0 To: Neel Natu References: <4E8168BB.9050802@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org Subject: Re: question on Bhyve X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2011 04:21:37 -0000 On 9/27/11 1:35 PM, Neel Natu wrote: > Hi Julian, > > On Mon, Sep 26, 2011 at 11:10 PM, Julian Elischer wrote: >> does anyone know what is needed for a hypervisor to support PCI pass >> through? >> > I can speak about the bhyve implementation of pci passthru but I > suspect that other hypervisors do it similarly. BHyVe requires that > the platform support nested page tables and has an iommu to be able to > support pci passthru. > > There are two parts to it. > > The simpler part is "hiding" the pci device that is to be used to > passthru. In bhyve we do this by setting the tunable "pptdevs" to a > list of pci devices that we may want to assign as passthru devices. > > This tunable is used by the "blackhole" pci device driver to claim > each device in "pptdevs" during probe but then reject it in attach. > The net effect of this is leave the pci device un-attached from the > host's perspective. > > For e.g. in /boot/loader.conf: > > pptdevs="1/0/0 2/0/0 3/0/0" > blackhole_load="YES" > > The second part is a bit more work for the hypervisor. > > We assign the pci device to a guest with the following command line > option to /usr/sbin/bhyve: -s 1,passthru,3/0/0. This says to make the > pci device identified by "3/0/0" appear as a passthru pci device on > slot 1 of the virtual pci bus. > > When we assign the device to the guest we need to provide it with > passthru access to the device's config space, mmio space, io space. In > the other direction we need to provide the pci device with access to > the guest's address space. > > The pci config space is a bit tricky because we need to intercept > accesses to the PCI BARs and also to the MSI capability. But beyond > those registers we simply intercept config space accesses and fulfill > them by issuing PCIIOCREAD and PCIIOCWRITE ioctls on /dev/pci > > Accesses to the MMIO space are straightforward and require no > hypervisor intervention after setting up the nested page tables. > > Accesses to the I/O space allocated to the device are trapped by the > hypervisor and handled by the opening /dev/io and issuing IODEV_PIO > ioctl on it. > > And finally dma transfers generated by the pci device into the guest's > address space are transparent to the hypervisor once it has set up the > IOMMU. > > Hope that helps. thanks for your explanation.. I will see if I can use it to pass-through a fusion-io flash card as soon as I get a change to set it all up.. Julian > best > Neel > >> >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to >> "freebsd-virtualization-unsubscribe@freebsd.org" >>