From owner-freebsd-virtualization@freebsd.org Wed Jan 11 13:52:49 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E963CA982E for ; Wed, 11 Jan 2017 13:52:49 +0000 (UTC) (envelope-from misc-freebsd@talk2dom.com) Received: from mail.shmtech.biz (mail.shmtech.biz [IPv6:2001:41c9:1:37::e]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 43ED71737 for ; Wed, 11 Jan 2017 13:52:48 +0000 (UTC) (envelope-from misc-freebsd@talk2dom.com) Received: from [10.0.1.100] (originc9729.pndsl.co.uk [84.92.225.78]) (authenticated bits=0) by mail.shmtech.biz (8.15.2/8.15.2) with ESMTPSA id v0BDqhBj052238 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Wed, 11 Jan 2017 13:52:44 GMT (envelope-from misc-freebsd@talk2dom.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=talk2dom.com; s=shmtech4; t=1484142764; x=1484747564; bh=V+ZJAicwvSt6f0NoK72u94LquylUUzp3J3smQbXXWiU=; h=Subject:To:References:Cc:From:Date:In-Reply-To; b=kvkZu5xw0wCXigMjUeW+wXdrfrY5wom8lq3MMqyecWdoXagVHkR4/rHmUmvaWoJ5p x8I8csenpg9Ff5D9gi1tQu5AOJgF8utvQKHiaPMZDZj/YwXtYSXLKTHxoTYM2smrT0 TjVhFO8edyYSLAi+09aJ2BVnBgHkSOgN/jsRCCLo= X-Authentication-Warning: sendmail: Host originc9729.pndsl.co.uk [84.92.225.78] claimed to be [10.0.1.100] Subject: Re: Issues with GTX960 on CentOS7 using bhyve PCI passthru (FreeBSD 11-RC2) To: soralx@cydem.org References: <20170110003332.7cf8ba15@mscad14> <0de7e0fe-5680-b1be-bd57-6bf446c2fd38@talk2dom.com> <0c927784-3e3f-7946-fba9-c25001f4156c@talk2dom.com> <20170110180117.7f246b5a@mscad14> Cc: freebsd-virtualization@freebsd.org From: Dom Message-ID: <75abdb83-8902-1c6e-e881-5af24e5bba05@talk2dom.com> Date: Wed, 11 Jan 2017 13:52:43 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20170110180117.7f246b5a@mscad14> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jan 2017 13:52:49 -0000 On 11/01/2017 02:01, soralx@cydem.org wrote: > Dom wote: >> There doesn't seem to be support for CPUID 0x40000001 in bhyve either. > What is it supposed to do? As far as I can tell it's the Hypervisor extension flags list. The lack of these extensions/optimisations might explain why your FreeBSD VM runs slow but their presence also causes the nVidia driver to refuse to run. (Can't remember where I read this, sorry) With your change to PCI_EMUL_MEMBASE64 I can boot a CentOS VM without the "pci=nocrs" kernel option and nVidia card is assigned BARs without issue. However, even with reapplying the changes to vmm.ko to hide/remove the 0x40000000 CPUID support and CPUID2_HV, I still have the same "RmInitAdapter failed" issue. Allegedly[0] nVidia VM checking came in with driver version 337.88, with more checking after version 344.11. I couldn't install version 319 as it failed to build the Linux kernel module. I currently have 370.28 installed which supports both my GT610 and my GTX960. Maybe the next thing for me to try is to replicate your tests with a FreeBSD VM. [0] https://ubuntuforums.org/showthread.php?t=2266916 search for "337.88"