From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 30 15:50:36 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 45F72E6; Sun, 30 Nov 2014 15:50:36 +0000 (UTC) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B80DFC2; Sun, 30 Nov 2014 15:50:35 +0000 (UTC) Received: from moby.local ([109.193.238.174]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0MJByE-1XsLsc0Fv0-002s6q; Sun, 30 Nov 2014 16:50:28 +0100 Message-ID: <547B3CC2.4040301@gmx.com> Date: Sun, 30 Nov 2014 16:50:26 +0100 From: Nikos Vassiliadis User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: Neel Natu Subject: Re: bhyve: Unsupported MSI-X configuration: 2/0/0 References: <5478E4C4.3080203@gmx.com> <5479D1BE.2010106@gmx.com> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:6EQ43SN6GMvcGIRmr5d6BQQShsqV8GUQKIbgNwd1N0PwtK1ZJ13 E+3p5Rr8vQ3OAnb0Q86U1kpCKqNjYnpJxj0aaF9QgL/lrAZEEAgY9DGsUKAGuf5VdcmNknX +UXvEVDqt049wMfLGmSu3Zhs2e78QK263MoaDdK3YcS0OIDyqyRJaj2hbRvhpE5sPQHnqDu 7k4ARYQsMat44ynEVszug== X-UI-Out-Filterresults: notjunk:1; Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Nov 2014 15:50:36 -0000 Hi, On 11/30/14 02:37, Neel Natu wrote: > The "Unsupported MSI-X configuration" referred to here is that bhyve > doesn't emulate the 'Pending Bit Array'. > > In most cases this is not relevant because the PBA and the MSI-X > tables are in different page frames. In this case the MSI-X tables are > emulated and the pending bit array page is passed through to the > guest. > > If the PBA and MSI-X tables are on the same page frame then you see > this error. You can confirm this by doing 'pciconf -lvbc pci0:2:0:0'. Here is the ouput: > bge0@pci0:2:0:0: class=0x020000 card=0x06471025 chip=0x16b514e4 rev=0x10 hdr=0x00 > vendor = 'Broadcom Corporation' > device = 'NetLink BCM57785 Gigabit Ethernet PCIe' > class = network > subclass = ethernet > bar [10] = type Prefetchable Memory, range 64, base 0xb3430000, size 65536, enabled > bar [18] = type Prefetchable Memory, range 64, base 0xb3440000, size 65536, enabled > cap 01[48] = powerspec 3 supports D0 D3 current D0 > cap 05[58] = MSI supports 8 messages, 64 bit enabled with 1 message > cap 11[a0] = MSI-X supports 5 messages > Table in map 0x18[0x0], PBA in map 0x18[0x120] > cap 10[ac] = PCI-Express 2 endpoint max data 128(128) link x1(x1) > speed 2.5(2.5) ASPM L1(L0s/L1) > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected > ecap 0003[13c] = Serial 1 0000208984c6b5ed > ecap 0004[150] = Power Budgeting 1 > ecap 0002[160] = VC 1 max VC0 > Having said that the drivers I have seen don't rely at all on the PBA. > So it may be possible that this works just fine with this patch: > https://people.freebsd.org/~neel/patches/bhyve_ignore_unsupported_pba.patch > > If it does then I can add a tunable to bhyve to ignore this check. It seems a tunable can help. I tried the patch and now bhyve continues and indeed I see the bge device in my VM. What is interesting/strange is that the device is partly functioning. For example, it does detect a link change when I remove the cable but that's all. The kernel also complains with a "bge watchdog timeout" message. In case there is no hardware support for VT-d, as Craig noticed, how could this behaviour be explained? Why does link detection work? Again, thank you both!