From owner-freebsd-virtualization@freebsd.org Sun Aug 28 21:00:32 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9BA4BC1904 for ; Sun, 28 Aug 2016 21:00:32 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A6F08152 for ; Sun, 28 Aug 2016 21:00:32 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u7SL01Lu005972 for ; Sun, 28 Aug 2016 21:00:32 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201608282100.u7SL01Lu005972@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-virtualization@FreeBSD.org Subject: Problem reports for freebsd-virtualization@FreeBSD.org that need special attention Date: Sun, 28 Aug 2016 21:00:32 +0000 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Aug 2016 21:00:32 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 202322 | [bhyve,patch] add option to have bhyve write its 1 problems total for which you should take action. From owner-freebsd-virtualization@freebsd.org Fri Sep 2 12:55:18 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9DE07BCCFF1 for ; Fri, 2 Sep 2016 12:55:18 +0000 (UTC) (envelope-from misc-freebsd@talk2dom.com) Received: from mail.shmtech.biz (mail.shmtech.biz [IPv6:2001:41c9:1:37::e]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.shmtech.biz", Issuer "Go Daddy Secure Certification Authority" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A120A1C for ; Fri, 2 Sep 2016 12:55:17 +0000 (UTC) (envelope-from misc-freebsd@talk2dom.com) Received: from [10.0.1.100] (originc9729.pndsl.co.uk [84.92.225.78]) (authenticated bits=0) by mail.shmtech.biz (8.15.2/8.14.9) with ESMTPSA id u82CtFOi074425 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Fri, 2 Sep 2016 13:55:15 +0100 (BST) (envelope-from misc-freebsd@talk2dom.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=talk2dom.com; s=shmtech4; t=1472820915; x=1473425715; bh=KozCggj0Lnj7TdVw8XfdwV2lHXSRWGD0aQyFtfrm8nI=; h=From:Subject:To:Date; b=nAlszLTBchrnAJcYBysM0XW0wV06yXj+yc1UkPzXPJG8119awKBbJzp7tGdTgkl6x Ylhd5yZj2ghRu0NOySMQm5e8sLmpDaXTtM77uZ+n1kzCzglS7WklzF6vOWoWavKZ9X 1/cc6qO3RzTk2kQRVATVkqwg03a62hvyIXZYZV4k= X-Authentication-Warning: sendmail: Host originc9729.pndsl.co.uk [84.92.225.78] claimed to be [10.0.1.100] From: Dom Subject: Issues with GTX960 on CentOS7 using bhyve PCI passthru (FreeBSD 11-RC2) To: freebsd-virtualization@freebsd.org Message-ID: <727839e2-4be1-5661-8c19-6d13f73ad36f@talk2dom.com> Date: Fri, 2 Sep 2016 13:54:11 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Sep 2016 12:55:18 -0000 Hello, Setup: nvidia GTX960 in PCIe slot intel i7-4790K CPU FreeBSD 11-RC2 host CentOS 7 guest with kernel 3.10.0-327.28.3.el7.x86_64 Using vm-bhyve port I've hit two issues: 1. BAR allocation Workaround (for me) is adding "pci=nocrs" to linux guest's kernel command line. Without "pci=nocrs" (or with "pci=use_crs"), the GTX960 doesn't get its 256MB block allocated. Bhyve's ACPI table produces this error linux-side regardless of "pci=" setting: [ 0.163085] acpi PNP0A03:00: host bridge window [0xd000000000-0xd0100fffff] (ignored, not CPU addressable) which then leads to this: [ 0.215369] pci 0000:00:06.0: can't claim BAR 1 [mem 0xd000000000-0xd00fffffff 64bit pref]: no compatible bridge window and then, with "pci=use_crs" (i.e. use ACPI host bridge windows): [ 0.164030] pci_bus 0000:00: root bus resource [bus 00] [ 0.164379] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7] [ 0.164799] pci_bus 0000:00: root bus resource [io 0x0d00-0x1fff] [ 0.165206] pci_bus 0000:00: root bus resource [io 0x2000-0x211f] [ 0.165623] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xc41fffff] ... [ 0.231762] pci 0000:00:06.0: BAR 1: no space for [mem size 0x10000000 64bit pref] [ 0.232263] pci 0000:00:06.0: BAR 1: trying firmware assignment [mem size 0x10000000 64bit pref] [ 0.232855] pci 0000:00:06.0: BAR 1: [mem size 0x10000000 64bit pref] conflicts with PCI mem [mem 0x00000000-0x7fffffffff] [ 0.233579] pci 0000:00:06.0: BAR 1: failed to assign [mem size 0x10000000 64bit pref] but with "pci=nocrs" (i.e. ignore ACPI host bridge windows): [ 0.163967] pci_bus 0000:00: root bus resource [bus 00] [ 0.164323] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] [ 0.164745] pci_bus 0000:00: root bus resource [mem 0x00000000-0x7fffffffff] ... [ 0.230203] pci 0000:00:06.0: BAR 1: assigned [mem 0x140000000-0x14fffffff 64bit pref] 2. "can't derive routing for PCI INT" Linux-side dmesg related output: [ 1.677168] nvidia 0000:00:06.0: can't derive routing for PCI INT A [ 1.677600] nvidia 0000:00:06.0: PCI INT A: no GSI Host-side info (when GTX960 is NOT configured as a pass-thru dev): vgapci0@pci0:1:0:0: class=0x030000 card=0x19623842 chip=0x140110de rev=0xa1 hdr=0x00 vendor = 'NVIDIA Corporation' device = 'GM206 [GeForce GTX 960]' class = display subclass = VGA bar [10] = type Memory, range 32, base 0xf6000000, size 16777216, enabled bar [14] = type Prefetchable Memory, range 64, base 0xe0000000, size 268435456, enabled bar [1c] = type Prefetchable Memory, range 64, base 0xf0000000, size 33554432, enabled bar [24] = type I/O Port, range 32, base 0xe000, size 128, enabled cap 01[60] = powerspec 3 supports D0 D3 current D0 cap 05[68] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[78] = PCI-Express 2 legacy endpoint max data 128(256) RO link x4(x16) speed 2.5(8.0) ecap 0002[100] = VC 1 max VC0 ecap 001e[258] = unknown 1 ecap 0004[128] = Power Budgeting 1 ecap 0001[420] = AER 2 0 fatal 0 non-fatal 4 corrected ecap 000b[600] = Vendor 1 ID 1 ecap 0019[900] = PCIe Sec 1 lane errors 0xf Linux-side info (when GTX960 IS configured as a pass-thru dev, also with "pci=nocrs"): 00:06.0 VGA compatible controller: NVIDIA Corporation GM206 [GeForce GTX 960] (rev a1) (prog-if 00 [VGA controller]) Subsystem: eVga.com. Corp. Device 1962 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR-