From owner-freebsd-virtualization@freebsd.org Tue Dec 22 03:45:01 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 80FD3A4C042 for ; Tue, 22 Dec 2015 03:45:01 +0000 (UTC) (envelope-from sm@ara-ler.com) Received: from mail-ig0-x234.google.com (mail-ig0-x234.google.com [IPv6:2607:f8b0:4001:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5216215E5 for ; Tue, 22 Dec 2015 03:45:01 +0000 (UTC) (envelope-from sm@ara-ler.com) Received: by mail-ig0-x234.google.com with SMTP id to18so49526603igc.0 for ; Mon, 21 Dec 2015 19:45:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ara-ler-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=cNWXKyh35/mqWrJoTd5Oic7uxzQPCfQ5iP4xDd3qNuk=; b=D72hMKWHBnEEb2JYFpwyFh/4wH9Ubo0JjHX7spuPapQI5zLuTVTsRFDWBlaRMiuTud H8WVSLKbgFvqd0r1LyEdry4IlzX5muytvkSMcPwCziIcOZ0BpKWXb8cydDeNoCyD8/lp 05IS4b8PKgf4va0Sr5qCS5et1aG0DB9HfEnSQFGByDgHPvazcz9AzLdwG9/LuEM7Pjmm VY+AeAMRHLARe3Bldv8lLFZiInT4k3XeZ3iYyiAXQFaqv4PVtgmmTL+LWCTrVIsDvLi9 zN4bQiWhKyKRPvmTbKI6Ot4SulfHU3c/LQfncdu+LBZTNwp5Rn4E96hEJueNdKVm6bEe 7bDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:subject:message-id:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=cNWXKyh35/mqWrJoTd5Oic7uxzQPCfQ5iP4xDd3qNuk=; b=ZSSBuJ6VwuIYthOVK/vHQFrxU2TCUWJ7MRhXIivh3M0ummWlF+UpqJnPNYlWYlnwfk DjEtAKiXyxrCF0YtqchlJgLJYQVhEGZLiv1IeOl4TDnS7q7CqB7gLx6A28elkzNTQiHD h38nsJtyb8yatmwyk22zNaTB/hBdcaXfRe4dTCXZfltWdGUaXcpsVNdQnmNqI2AcyECp RNM8PYbNqGkCcVZyJa3pz5TAPeSO3SS7B4leLbIT7HSAKh1rraU7heQ1qcgJzUGSBkN6 nURlsjgJ28mMDiC/IofQ/zaHOvqU0KjN48QeunLSqifZXFC0wbPZG6aNrezH3En5yOrG asQg== X-Gm-Message-State: ALoCoQmO5YyjolQk4G6xxgf1fp1YKfYiFPk+jurqO9NQw3tOMBm8UFv02/C4DWEBgzhMhQNmU99Fq3cPVTvrdTFmvKmVhXHMXw== X-Received: by 10.50.142.99 with SMTP id rv3mr20834833igb.72.1450755900433; Mon, 21 Dec 2015 19:45:00 -0800 (PST) Received: from dendrobates.araler.com (70-59-19-232.hlrn.qwest.net. [70.59.19.232]) by smtp.gmail.com with ESMTPSA id c67sm4653112iod.5.2015.12.21.19.44.59 for (version=TLSv1/SSLv3 cipher=OTHER); Mon, 21 Dec 2015 19:44:59 -0800 (PST) Date: Mon, 21 Dec 2015 20:44:57 -0700 From: Sergey Manucharian To: FreeBSD virtualization Subject: Re: bhyve PCI pass-through to Linux guest Message-ID: <20151222034457.GA40078@dendrobates.araler.com> References: <20151220045821.GG22018@dendrobates.araler.com> <56763672.3090207@freebsd.org> <20151220051015.GH22018@dendrobates.araler.com> <56763A0B.8010802@freebsd.org> <20151220053644.GI22018@dendrobates.araler.com> <20151220215240.GB4064@dendrobates.araler.com> <5678BECC.7090200@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5678BECC.7090200@freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Dec 2015 03:45:01 -0000 Excerpts from Peter Grehan's message from Tue 22-Dec-15 13:09: > Hi Sergey, > > > This is pretty reproducible: > > > > I pass trough a PCI device (USB controller) to a Linux guest. It works > > properly. Then I halt the VM, make sure that bhyve destroyed it and run > > Windows guest with the same PCI device passed-through. > > > > Windows device manager does show the device, however, e.g. a flash drive > > plugged in is not presented to Windows, instead it's being processed by > > FreeBSD. > > > > After that it does not work in Linux guest as well. Kernel module (vmm) > > unloading and reloading does not help. > > The flash drive being processed by FreeBSD would indicate that it has > ownership of the device. Would you be able to try a 'pciconf -vl' after > the Linux guest exists, and after the Windows guest exits ? OK, I've experimented more and found that even with the same Linux VM after halting it once, the pass-through device doesn't work when that Linux guest is restarted again. It looks that both host and guest report exactly the same data before and after restarting the guest: Linux - first run: ================= $ lspci -v 00:07.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04) (prog-if 30 [XHCI]) Subsystem: Lenovo Device 21f3 Flags: bus master, medium devsel, latency 0, IRQ 24 Memory at c0010000 (64-bit, prefetchable) [size=64K] Capabilities: [70] Power Management version 2 Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+ Kernel driver in use: xhci_hcd $ lsusb Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 002: ID 0781:5530 SanDisk Corp. Cruzer Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub FreeBSD when Linux is running first time: ========================================= $ pciconf -v ppt0@pci0:0:20:0: class=0x0c0330 card=0x21f317aa chip=0x1e318086 rev=0x04 hdr=0x00 vendor = 'Intel Corporation' device = '7 Series/C210 Series Chipset Family USB xHCI Host Controller' class = serial bus subclass = USB Linux - second run: =================== $ lspci -v 00:07.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04) (prog-if 30 [XHCI]) Subsystem: Lenovo Device 21f3 Flags: bus master, medium devsel, latency 0, IRQ 24 Memory at c0010000 (64-bit, prefetchable) [size=64K] Capabilities: [70] Power Management version 2 Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+ Kernel driver in use: xhci_hcd FreeBSD when Linux is running second time: ========================================== $ pciconf -lv ppt0@pci0:0:20:0: class=0x0c0330 card=0x21f317aa chip=0x1e318086 rev=0x04 hdr=0x00 vendor = 'Intel Corporation' device = '7 Series/C210 Series Chipset Family USB xHCI Host Controller' class = serial bus subclass = USB -- Thanks, Sergey