Date: Wed, 11 Jan 2017 18:33:35 -0800 From: <soralx@cydem.org> To: <misc-freebsd@talk2dom.com> Cc: <freebsd-virtualization@freebsd.org> Subject: Re: Issues with GTX960 on CentOS7 using bhyve PCI passthru (FreeBSD 11-RC2) Message-ID: <20170111183335.50a93727@mscad14> In-Reply-To: <75abdb83-8902-1c6e-e881-5af24e5bba05@talk2dom.com> References: <20170110003332.7cf8ba15@mscad14> <0de7e0fe-5680-b1be-bd57-6bf446c2fd38@talk2dom.com> <0c927784-3e3f-7946-fba9-c25001f4156c@talk2dom.com> <20170110180117.7f246b5a@mscad14> <75abdb83-8902-1c6e-e881-5af24e5bba05@talk2dom.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> As far as I can tell it's the Hypervisor extension flags list. The lack
> of these extensions/optimisations might explain why your FreeBSD VM
> runs slow
The guest isn't slow, actually -- just the `nvidia-smi` tool was
much slower than normal to produce output. CPU speed in the guest
is less than 2% slower than bare hardware. Memory bandwidth is
over 2.5 times slower. [0] Still, compiling world and ports feels
quite snappy. Disk I/O is not bad, too! (but latencies are bit
high) [1]
> However, even with reapplying the changes to vmm.ko to hide/remove the
> 0x40000000 CPUID support and CPUID2_HV, I still have the same
> "RmInitAdapter failed" issue.
Well, I, too, couldn't get nVidia driver to work at all in Linux.
The driver loaded, but when trying to use it, gave the "RmInitAdapter
failed! (0x53:0x3:1856)" errors. Is is the same in your case? Or
the driver does not load at all (like I experienced with Quadro 600)?
Maybe I should try the same driver version in Linux as in FreeBSD...
> Allegedly[0] nVidia VM checking came in with driver version 337.88,
> with more checking after version 344.11. I couldn't install version 319
> as it failed to build the Linux kernel module. I currently have 370.28
> installed which supports both my GT610 and my GTX960.
`nvidia-smi -q` thinks it's running on bare hardware [when in VM]:
GPU Virtualization Mode
Virtualization mode : None
Yet we know the driver refuses to load for Quadro 600 and your GTX.
(that two-faced bastard nVidia!) So there must be multiple checks.
> Maybe the next thing for me to try is to replicate your tests with a
> FreeBSD VM.
Yes, give it a try. I'm about to give up, for nVidia card at least
(I was only using it for testing, as I didn't have AMD GPU handy
until recently).
[0]
# Host with idle FreeBSD VM guest running
# ubench
Unix Benchmark Utility v.0.3
FreeBSD 10.3-STABLE FreeBSD 10.3-STABLE #0 r311343M: Thu Jan 5 02:31:50 PST 2017 xxx@yyy:/usr/obj/usr/src/sys/SORALX amd64
Ubench CPU: 2076542
Ubench MEM: 1296221
--------------------
Ubench AVG: 1686381
# Guest
# ubench
Unix Benchmark Utility v.0.3
FreeBSD 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r311659: Tue Jan 10 14:18:44 PST 2017 xxx@zzz:/usr/obj/usr/src/sys/GENERIC amd64
Ubench CPU: 2053983
Ubench MEM: 489953
--------------------
Ubench AVG: 1271968
[1]
# Host
# diskinfo -tv /dev/ada0
/dev/ada0
512 # sectorsize
250059350016 # mediasize in bytes (233G)
488397168 # mediasize in sectors
4096 # stripesize
Seek times:
Full stroke: 250 iter in 0.029146 sec = 0.117 msec
Half stroke: 250 iter in 0.029950 sec = 0.120 msec
Quarter stroke: 500 iter in 0.057272 sec = 0.115 msec
Short forward: 400 iter in 0.045711 sec = 0.114 msec
Short backward: 400 iter in 0.045748 sec = 0.114 msec
Seq outer: 2048 iter in 0.072799 sec = 0.036 msec
Seq inner: 2048 iter in 0.072256 sec = 0.035 msec
Transfer rates:
outside: 102400 kbytes in 0.222412 sec = 460407 kbytes/sec
middle: 102400 kbytes in 0.222026 sec = 461207 kbytes/sec
inside: 102400 kbytes in 0.222842 sec = 459518 kbytes/sec
# Guest
# diskinfo -tv /dev/vtbd0
/dev/vtbd0
512 # sectorsize
22548644864 # mediasize in bytes (21G)
44040322 # mediasize in sectors
32768 # stripesize
Seek times:
Full stroke: 250 iter in 0.112714 sec = 0.451 msec
Half stroke: 250 iter in 0.100388 sec = 0.402 msec
Quarter stroke: 500 iter in 0.132808 sec = 0.266 msec
Short forward: 400 iter in 0.074504 sec = 0.186 msec
Short backward: 400 iter in 0.096154 sec = 0.240 msec
Seq outer: 2048 iter in 0.208239 sec = 0.102 msec
Seq inner: 2048 iter in 0.061853 sec = 0.030 msec
Transfer rates:
outside: 102400 kbytes in 0.201375 sec = 508504 kbytes/sec
middle: 102400 kbytes in 0.196044 sec = 522332 kbytes/sec
inside: 102400 kbytes in 0.197697 sec = 517964 kbytes/sec
--
[SorAlx] ridin' VN2000 Classic LT
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20170111183335.50a93727>