Date: Fri, 9 Sep 2016 10:20:41 +0100 From: Dom <misc-freebsd@talk2dom.com> To: freebsd-virtualization@freebsd.org Subject: Re: Issues with GTX960 on CentOS7 using bhyve PCI passthru (FreeBSD 11-RC2) Message-ID: <22df7de9-1dfb-561c-125d-99af2b3da70b@talk2dom.com> In-Reply-To: <62b2a72a-3662-5b3e-5029-f284a625d05b@freebsd.org> References: <727839e2-4be1-5661-8c19-6d13f73ad36f@talk2dom.com> <62b2a72a-3662-5b3e-5029-f284a625d05b@freebsd.org>
index | next in thread | previous in thread | raw e-mail
Hi Peter,
Thanks for getting back to me. Here's the info you requested:
>> [ 0.163085] acpi PNP0A03:00: host bridge window
>> [0xd000000000-0xd0100fffff] (ignored, not CPU addressable)
>
> That one is most likely a bug in bhyve, where the space used for 64-bit
> BAR placement isn't tested against the max physaddr width of the host
> CPU.
>
> To confirm, would you be able to report on this value on your system ?
>
> # sudo pkg install cpuid
> # cpuid | grep ^80000008
On my Intel i7-4790K CPU:
# cpuid | grep ^80000008
80000008 00003027 00000000 00000000 00000000
> The device has an MSI capability, but the nvidia driver may not use it.
> bhyve PCI passthrough requires the use of MSI/MSI-x interrupts, and
> doesn't support using legacy interrupts.
>
> This could be confirmed from the output of /proc/interrupts when
> booting Linux on the system.
Output of /proc/interrupts:
CPU0
0: 137 IO-APIC-edge timer
1: 9 IO-APIC-edge i8042
4: 965 IO-APIC-edge serial
8: 0 IO-APIC-edge rtc0
9: 0 IO-APIC-fasteoi acpi
12: 138 IO-APIC-edge i8042
17: 0 IO-APIC-fasteoi snd_hda_intel
24: 0 PCI-MSI-edge virtio0-config
25: 8535 PCI-MSI-edge virtio0-req.0
26: 0 PCI-MSI-edge virtio1-config
27: 123 PCI-MSI-edge virtio1-input.0
28: 1 PCI-MSI-edge virtio1-output.0
NMI: 0 Non-maskable interrupts
LOC: 6050 Local timer interrupts
SPU: 0 Spurious interrupts
PMI: 0 Performance monitoring interrupts
IWI: 2484 IRQ work interrupts
RTR: 0 APIC ICR read retries
RES: 0 Rescheduling interrupts
CAL: 0 Function call interrupts
TLB: 0 TLB shootdowns
TRM: 0 Thermal event interrupts
THR: 0 Threshold APIC interrupts
MCE: 0 Machine check exceptions
MCP: 1 Machine check polls
ERR: 0
MIS: 0
I guess the lack of a line containing PCI-MSI-* here indicates the
nvidia driver isn't using an MSI/MSI-x interrupt?
However, searching the web suggests the Linux nvidia driver does use MSI
interrupts. This taken from a working non-VM Linux dmesg:
[ 4.330536] nvidia 0000:05:00.0: PCI INT A -> GSI 16 (level, low) ->
IRQ 16
[ 4.330542] nvidia 0000:05:00.0: setting latency timer to 64
Source: https://bugzilla.kernel.org/show_bug.cgi?id=20432#c2
(Thread also mentions disabling MSI)
I'll try some Linux boot options and reordering the devices when calling
bhyve to see if that changes anything.
Thanks,
Dom
home |
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?22df7de9-1dfb-561c-125d-99af2b3da70b>