Date: Fri, 23 Mar 2018 19:53:37 +0100 From: Wouter Snels <nospam@ofloo.net> To: Julian Elischer <julian@freebsd.org>, freebsd-virtualization@freebsd.org Subject: Re: bhyve pci passthrough usb controller Message-ID: <8945b900-ca21-26e1-fa15-5dbeac8716b4@ofloo.net> In-Reply-To: <2ca981b4-9651-253a-7549-eaece4a58ff0@freebsd.org> References: <5bd3c8b6-0579-0269-1ada-ab6f0a55ce8f@ofloo.net> <2ca981b4-9651-253a-7549-eaece4a58ff0@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --MQOKdVQJ8Vjc1Uz5qruwntr4Siq4VacOy Content-Type: multipart/mixed; boundary="eUUAWQ4DS1XcIw7pzMGtyK9ck2aFdgVIR"; protected-headers="v1" From: Wouter Snels <nospam@ofloo.net> To: Julian Elischer <julian@freebsd.org>, freebsd-virtualization@freebsd.org Message-ID: <8945b900-ca21-26e1-fa15-5dbeac8716b4@ofloo.net> Subject: Re: bhyve pci passthrough usb controller References: <5bd3c8b6-0579-0269-1ada-ab6f0a55ce8f@ofloo.net> <2ca981b4-9651-253a-7549-eaece4a58ff0@freebsd.org> In-Reply-To: <2ca981b4-9651-253a-7549-eaece4a58ff0@freebsd.org> --eUUAWQ4DS1XcIw7pzMGtyK9ck2aFdgVIR Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Content-Language: nl-BE No the xhci driver is not attached only the the ppt driver is, but that's normal because you set pptdevs=3D"0/20/0 4/0/0" in /boot/loader.co= nf Op 23-03-18 om 18:25 schreef Julian Elischer: > On 23/3/18 11:40 pm, Wouter Snels wrote: > > > new wrapping is no better > It looks like the device you are passing though is not removed from > freebsd first. > but it's hard to tell. > >> |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve >> windows 7 virtualization. When I start the virtual machine it freezes >> the entire system and this shows up on the console. To be clear the >> system runs without any problems without the pci passthrough. | >> ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 = 00 >> 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid >> (ada0:ahcich0:0:0:0): Error 22, Unretryable error >> (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00= >> 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid >> (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 b= us >> 0 scbus0 target 0 lun 0 ada0: <ST500DM002-1SB10A CC43> s/n ZA4292G6 >> detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40= >> 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was= >> invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: >> Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss >> 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 >> (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 0= 0 >> 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 4= 0 >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Pol= l >> timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000= >> rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0)= : >> SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 >> (aprobe0:ahcich0:0:0:0): CAM status: Command timeout >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 4= 0 >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error >> (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-= - >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: = hw >> tdh =3D 428, hw tdt =3D 471 em0: Tx Queue Status =3D -2147483648 em0: = TX >> descriptors avail =3D 981 em0: Tx Descriptors avail failure =3D 0 em0:= RX >> Queue 0 ------ em0: hw rdh =3D 912, hw rdt =3D 908 em0: RX discarded p= ackets >> =3D 0 em0: RX Next to Check =3D 909 em0: RX Next to Refresh =3D 908 xl= 0: >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface = is >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D= 58 >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966= em0: >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh = =3D 0, >> hw rdt =3D 1023 em0: RX discarded packets =3D 0 em0: RX Next to Check = =3D 0 >> em0: RX Next to Refresh =3D 1023 em0: Watchdog timeout Queue[0]-- >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: = hw >> tdh =3D 0, hw tdt =3D 58 em0: Tx Queue Status =3D -2147483648 em0: TX >> descriptors avail =3D 966 em0: Tx Descriptors avail failure =3D 0 em0:= RX >> Queue 0 ------ em0: hw rdh =3D 0, hw rdt =3D 1023 em0: RX discarded pa= ckets >> =3D 0 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D 1023 xl0= : >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface = is >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D= 54 >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 970= em0: >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh = =3D 0, >> hw rdt =3D 1023 em0: RX discarded packets =3D 0include GENERIC ident O= FL >> options IPFIREWALL|||||| >> >> |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve >> windows 7 virtualization. When I start the virtual machine it freezes >> the entire system and this shows up on the console. To be clear the >> system runs without any problems without the pci passthrough. | >> ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 = 00 >> 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid >> (ada0:ahcich0:0:0:0): Error 22, Unretryable error >> (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00= >> 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid >> (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 b= us >> 0 scbus0 target 0 lun 0 ada0: <ST500DM002-1SB10A CC43> s/n ZA4292G6 >> detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40= >> 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was= >> invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: >> Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss >> 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 >> (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 0= 0 >> 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 4= 0 >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Pol= l >> timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000= >> rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0)= : >> SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 >> (aprobe0:ahcich0:0:0:0): CAM status: Command timeout >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 4= 0 >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error >> (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-= - >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: = hw >> tdh =3D 428, hw tdt =3D 471 em0: Tx Queue Status =3D -2147483648 em0: = TX >> descriptors avail =3D 981 em0: Tx Descriptors avail failure =3D 0 em0:= RX >> Queue 0 ------ em0: hw rdh =3D 912, hw rdt =3D 908 em0: RX discarded p= ackets >> =3D 0 em0: RX Next to Check =3D 909 em0: RX Next to Refresh =3D 908 xl= 0: >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface = is >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D= 58 >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966= em0: >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh = =3D 0, >> hw rdt =3D 1023 em0: RX discarded packets =3D 0 em0: RX Next to Check = =3D 0 >> em0: RX Next to Refresh =3D 1023 em0: Watchdog timeout Queue[0]-- >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: = hw >> tdh =3D 0, hw tdt =3D 58 em0: Tx Queue Status =3D -2147483648 em0: TX >> descriptors avail =3D 966 em0: Tx Descriptors avail failure =3D 0 em0:= RX >> Queue 0 ------ em0: hw rdh =3D 0, hw rdt =3D 1023 em0: RX discarded pa= ckets >> =3D 0 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D 1023 xl0= : >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface = is >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D= 54 >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 970= em0: >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh = =3D 0, >> hw rdt =3D 1023 em0: RX discarded packets =3D 0include GENERIC ident O= FL >> options IPFIREWALL options IPFIREWALL_VERBOSE options >> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options= >> CONSPEED=3D115200 em0: RX Next to Check =3D 0 em0: RX Next to Refresh = =3D >> 1023 |||| >> |||I've recompiled the kernel without the xhci driver. I've tested bot= h >> usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI >> Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both >> showed the same error on the console screen.| # uname FreeBSD >> 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL >> options IPFIREWALL options IPFIREWALL_VERBOSE options >> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options= >> CONSPEED=3D115200 * note commented "device xhci" in generic # svn info= >> |||Path: . Working Copy Root Path: /usr/src URL: >> https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 >> Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: >> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: >> directory Schedule: normal Last Changed Author: eadler Last Changed Re= v: >> 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018)= >> |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon >> E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Register= s >> pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe >> Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes >> Sunrise Point-H USB 3.0 xHCI Controller none0 0/20/2 >> No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H >> CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] >> pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No >> Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H >> LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No >> Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 >> 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI >> Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0= /0 >> Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: >> class=3D0x0c0330 card=3D0x00151912 chip=3D0x00151912 rev=3D0x02 >> hdr=3D0x00 vendor =3D 'Renesas Technology Corp.' device =3D 'uPD720202= USB 3.0 >> Host Controller' class =3D serial bus subclass =3D USB >> |||ppt|0@pci0:0:20:0: class=3D0x0c0330 card=3D0x06ba1028 chip=3D0xa12f= 8086 >> rev=3D0x31 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Sunri= se Point-H >> USB 3.0 xHCI Controller' class =3D serial bus subclass =3D USB # grep = -i >> pptdevfs loader.conf ||pptdevs=3D"0/20/0 4/0/0" # vm configure >> ||passthru0=3D"0/20/0"| >> >> |# vm configure ||passthru0=3D"4/0/0"| >> ||Any suggestions on what I'm doing wrong?|| >> =C2=A0 Regards, Ofloo >> |||||| >> >> ||options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT >> options >> DUMMYNET options IPDIVERT options CONSPEED=3D115200 em0: RX Next to Ch= eck >> =3D 0 em0: RX Next to Refresh =3D 1023 |||| >> |||I've recompiled the kernel without the xhci driver. I've tested bot= h >> usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI >> Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both >> showed the same error on the console screen.| # uname FreeBSD >> 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL >> options IPFIREWALL options IPFIREWALL_VERBOSE options >> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options= >> CONSPEED=3D115200 * note commented "device xhci" in generic # svn info= >> |||Path: . Working Copy Root Path: /usr/src URL: >> https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 >> Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: >> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: >> directory Schedule: normal Last Changed Author: eadler Last Changed Re= v: >> 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018)= >> |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon >> E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Register= s >> pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe >> Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes >> Sunrise Point-H USB 3.0 xHCI Controller none0 0/20/2 >> No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H >> CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] >> pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No >> Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H >> LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No >> Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 >> 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI >> Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0= /0 >> Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: >> class=3D0x0c0330 card=3D0x00151912 chip=3D0x00151912 rev=3D0x02 >> hdr=3D0x00 vendor =3D 'Renesas Technology Corp.' device =3D 'uPD720202= USB 3.0 >> Host Controller' class =3D serial bus subclass =3D USB >> |||ppt|0@pci0:0:20:0: class=3D0x0c0330 card=3D0x06ba1028 chip=3D0xa12f= 8086 >> rev=3D0x31 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Sunri= se Point-H >> USB 3.0 xHCI Controller' class =3D serial bus subclass =3D USB # grep = -i >> pptdevfs loader.conf ||pptdevs=3D"0/20/0 4/0/0" # vm configure >> ||passthru0=3D"0/20/0"| >> >> |# vm configure ||passthru0=3D"4/0/0"| >> ||Any suggestions on what I'm doing wrong?|| >> >> edit: previous wrapping was all wrong. >> =C2=A0=C2=A0 Regards, Ofloo >> |||||| >> >> --eUUAWQ4DS1XcIw7pzMGtyK9ck2aFdgVIR-- --MQOKdVQJ8Vjc1Uz5qruwntr4Siq4VacOy Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJatU0xAAoJENWumses7+9mrIYQAIPKiaK4LkdeAZjrgtj48hv3 r18Af2jzNOTueIZde6v64T6AbWlh/P+U/N7ii4skV8hM0OIBX0zoHb0tmXHM1Amg hb8gGAFwQauHjfoV78VMjtqjVi7ps9Cu+ch4TlJHRwOvkRrYu+JyW1fQYgTsMRPg 2NkzwD44W2pizQpUHnL3SqFkxWZwRCQ7zYihzuTz0BtotXmxcb89iH9d0/iwGILv 4ub+qLuA6TA316Sa3sSir6tauBvQIRTqnPhlJPhxuZijisoGCt3gFy7UiuC4zj/v md4BgH35PqYQn4zPMaS8O5Q3eeVHGKHz99HDl+7GLUOqTXMHllAm27pJ4QMmmSdJ FaumUVthzA6+CE0kkHsxFZ5QU42rfz7mpZIytw8psobg80+xphEA2EPQcQB0UgcF TXBPJbVr/Qbh/3WSTK3IX4PesMW4rumieRRmapwuw6VOvz9VIPE3xmHjmkR7r2GH F9apMBkMeD9hjN9ppURxuPJd+UEBpOrAuRR/4t3g3exoTjZUhOnANaCHcSNIaszx H1zcWWtKl6jbQ/NYsmo5ZXRI08iqiUNF/JV2lLeAoQaRnL4Bnwn4l2p5i4ijVhm/ NhS3uI6U+28TUB5UgrHXKtNS0EtsQD2LTpXwtjI5SB9eVSvyLkmF0clKwdwXOLIf voVd7vAkQ+tAQTqBM92B =+2X9 -----END PGP SIGNATURE----- --MQOKdVQJ8Vjc1Uz5qruwntr4Siq4VacOy--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8945b900-ca21-26e1-fa15-5dbeac8716b4>