From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 30 01:37:14 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D2CF203; Sun, 30 Nov 2014 01:37:14 +0000 (UTC) Received: from mail-wi0-x236.google.com (mail-wi0-x236.google.com [IPv6:2a00:1450:400c:c05::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 174E5FBE; Sun, 30 Nov 2014 01:37:14 +0000 (UTC) Received: by mail-wi0-f182.google.com with SMTP id h11so14159160wiw.15 for ; Sat, 29 Nov 2014 17:37:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Lv0RSQjmHUXoCrOx6D6fZ40c1sA01waDWrmGt2ZweeU=; b=LMdI7lHEWOKotUBzrArwFyjIcz7bEp2ivXxYShA4RWSOTgte+CtDUCMuJDAB3rtoLO 9kVBpib2L3WlT32ce4gHHULTlFnbVyKbEsqxoT2JOhb+g0/zbmGtwMA3u4y6PLdgzU/f BFzHzL+lBqrAkenp96RUIxBDmNH5O6of4PTbI58LzJdWYILkIzoEc3Q+0y5R9Gnwkoj1 BpPQGFvShcuxaS+Ui6OsTgXc10OedMWu78E09ef/umvFp+ly4J0Ky0pKyVSTz84DolSF S0gHd7XZnbGWDmvX29iFL3br8HrUSm+UYY36gfoXtEVrCSKWxhSgSOT35jGnFQ2Ukzi/ irkw== MIME-Version: 1.0 X-Received: by 10.194.238.3 with SMTP id vg3mr43638223wjc.69.1417311431969; Sat, 29 Nov 2014 17:37:11 -0800 (PST) Received: by 10.27.12.168 with HTTP; Sat, 29 Nov 2014 17:37:11 -0800 (PST) In-Reply-To: <5479D1BE.2010106@gmx.com> References: <5478E4C4.3080203@gmx.com> <5479D1BE.2010106@gmx.com> Date: Sat, 29 Nov 2014 17:37:11 -0800 Message-ID: Subject: Re: bhyve: Unsupported MSI-X configuration: 2/0/0 From: Neel Natu To: Nikos Vassiliadis Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Nov 2014 01:37:14 -0000 Hi Nikos, On Sat, Nov 29, 2014 at 6:01 AM, Nikos Vassiliadis wrote: > > > On 11/29/14 02:05, Craig Rodrigues wrote: >> >> On Fri, Nov 28, 2014 at 1:10 PM, Nikos Vassiliadis > > wrote: >> >> Hi, >> >> I am trying to use PCI passthrough to pass the bge device to a guest >> but I am doing something wrong. I have tried several combinations of >> pptdevs like "2/0/0 2/0/1 2/0/2 2/0/3" but it always fails on 2/0/0. >> Any ideas what I could try? >> >> >> Did you read this: https://wiki.freebsd.org/bhyve/pci_passthru ? >> >> After following those instructions, what is the output of pciconf -l -v >> before and after enabling PCI passthru? >> On the host and in the guest? > > > > Before loading vmm.ko: >> >> hostb0@pci0:0:0:0: class=0x060000 card=0x064b1025 chip=0x01548086 >> rev=0x09 hdr=0x00 >> pcib1@pci0:0:1:0: class=0x060400 card=0x064b1025 chip=0x01518086 >> rev=0x09 hdr=0x01 >> vgapci1@pci0:0:2:0: class=0x030000 card=0x064b1025 chip=0x01668086 >> rev=0x09 hdr=0x00 >> none0@pci0:0:22:0: class=0x078000 card=0x064b1025 chip=0x1e3a8086 >> rev=0x04 hdr=0x00 >> ehci0@pci0:0:26:0: class=0x0c0320 card=0x064b1025 chip=0x1e2d8086 >> rev=0x04 hdr=0x00 >> hdac0@pci0:0:27:0: class=0x040300 card=0x064b1025 chip=0x1e208086 >> rev=0x04 hdr=0x00 >> pcib2@pci0:0:28:0: class=0x060400 card=0x064b1025 chip=0x1e108086 >> rev=0xc4 hdr=0x01 >> pcib3@pci0:0:28:1: class=0x060400 card=0x064b1025 chip=0x1e128086 >> rev=0xc4 hdr=0x01 >> ehci1@pci0:0:29:0: class=0x0c0320 card=0x064b1025 chip=0x1e268086 >> rev=0x04 hdr=0x00 >> isab0@pci0:0:31:0: class=0x060100 card=0x064b1025 chip=0x1e578086 >> rev=0x04 hdr=0x00 >> ahci0@pci0:0:31:2: class=0x010601 card=0x064b1025 chip=0x1e038086 >> rev=0x04 hdr=0x00 >> none1@pci0:0:31:3: class=0x0c0500 card=0x064b1025 chip=0x1e228086 >> rev=0x04 hdr=0x00 >> vgapci0@pci0:1:0:0: class=0x030000 card=0x06911025 chip=0x114010de >> rev=0xa1 hdr=0x00 >> none2@pci0:2:0:0: class=0x020000 card=0x06471025 chip=0x16b514e4 >> rev=0x10 hdr=0x00 >> none3@pci0:2:0:1: class=0x080501 card=0x06471025 chip=0x16bc14e4 >> rev=0x10 hdr=0x00 >> none4@pci0:2:0:2: class=0x088000 card=0x06471025 chip=0x16be14e4 >> rev=0x10 hdr=0x00 >> none5@pci0:2:0:3: class=0x088000 card=0x06471025 chip=0x16bf14e4 >> rev=0x10 hdr=0x00 >> none6@pci0:3:0:0: class=0x028000 card=0xe042105b chip=0x472714e4 >> rev=0x01 hdr=0x00 > > > After loading vmm.ko: >> >> hostb0@pci0:0:0:0: class=0x060000 card=0x064b1025 chip=0x01548086 >> rev=0x09 hdr=0x00 >> pcib1@pci0:0:1:0: class=0x060400 card=0x064b1025 chip=0x01518086 >> rev=0x09 hdr=0x01 >> vgapci1@pci0:0:2:0: class=0x030000 card=0x064b1025 chip=0x01668086 >> rev=0x09 hdr=0x00 >> none0@pci0:0:22:0: class=0x078000 card=0x064b1025 chip=0x1e3a8086 >> rev=0x04 hdr=0x00 >> ehci0@pci0:0:26:0: class=0x0c0320 card=0x064b1025 chip=0x1e2d8086 >> rev=0x04 hdr=0x00 >> hdac0@pci0:0:27:0: class=0x040300 card=0x064b1025 chip=0x1e208086 >> rev=0x04 hdr=0x00 >> pcib2@pci0:0:28:0: class=0x060400 card=0x064b1025 chip=0x1e108086 >> rev=0xc4 hdr=0x01 >> pcib3@pci0:0:28:1: class=0x060400 card=0x064b1025 chip=0x1e128086 >> rev=0xc4 hdr=0x01 >> ehci1@pci0:0:29:0: class=0x0c0320 card=0x064b1025 chip=0x1e268086 >> rev=0x04 hdr=0x00 >> isab0@pci0:0:31:0: class=0x060100 card=0x064b1025 chip=0x1e578086 >> rev=0x04 hdr=0x00 >> ahci0@pci0:0:31:2: class=0x010601 card=0x064b1025 chip=0x1e038086 >> rev=0x04 hdr=0x00 >> none1@pci0:0:31:3: class=0x0c0500 card=0x064b1025 chip=0x1e228086 >> rev=0x04 hdr=0x00 >> vgapci0@pci0:1:0:0: class=0x030000 card=0x06911025 chip=0x114010de >> rev=0xa1 hdr=0x00 >> ppt0@pci0:2:0:0: class=0x020000 card=0x06471025 chip=0x16b514e4 >> rev=0x10 hdr=0x00 >> ppt1@pci0:2:0:1: class=0x080501 card=0x06471025 chip=0x16bc14e4 >> rev=0x10 hdr=0x00 >> ppt2@pci0:2:0:2: class=0x088000 card=0x06471025 chip=0x16be14e4 >> rev=0x10 hdr=0x00 >> ppt3@pci0:2:0:3: class=0x088000 card=0x06471025 chip=0x16bf14e4 >> rev=0x10 hdr=0x00 >> ppt4@pci0:3:0:0: class=0x028000 card=0xe042105b chip=0x472714e4 >> rev=0x01 hdr=0x00 > > > Then I start "bhyve ... -s 10,passthru,2/0/0 ..." and it spits: > "Unsupported MSI-X configuration: 2/0/0" > and quits. > The "Unsupported MSI-X configuration" referred to here is that bhyve doesn't emulate the 'Pending Bit Array'. In most cases this is not relevant because the PBA and the MSI-X tables are in different page frames. In this case the MSI-X tables are emulated and the pending bit array page is passed through to the guest. If the PBA and MSI-X tables are on the same page frame then you see this error. You can confirm this by doing 'pciconf -lvbc pci0:2:0:0'. Having said that the drivers I have seen don't rely at all on the PBA. So it may be possible that this works just fine with this patch: https://people.freebsd.org/~neel/patches/bhyve_ignore_unsupported_pba.patch If it does then I can add a tunable to bhyve to ignore this check. best Neel > This happens *only* with 2/0/0, that is the bge device. When I try to > passthrough this: >> >> none6@pci0:3:0:0: class=0x028000 card=0xe042105b chip=0x472714e4 >> rev=0x01 hdr=0x00 >> vendor = 'Broadcom Corporation' >> device = 'BCM4313 802.11b/g/n Wireless LAN Controller' >> class = network > > > it goes further, bhyve starts and the device is passed to the host but I > cannot use it from the VM. Actually my goal is to use the wifi adapter from > a VM. The bge device is just testing and I thought I should report it/ask > about it. > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org"