From owner-freebsd-virtualization@freebsd.org Fri Mar 23 15:30:35 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A0B1F517CD for ; Fri, 23 Mar 2018 15:30:35 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from loki.ofloo.net (unknown [IPv6:2001:470:7acd:0:1:1:0:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "loki.ofloo.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 16C8E799B0 for ; Fri, 23 Mar 2018 15:30:34 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from [IPv6:fc03:1337:dead:cafe:247f:77bc:e37d:82ad] ([IPv6:fc03:1337:dead:cafe:247f:77bc:e37d:82ad]) by loki.ofloo.net (8.15.2/8.15.2) with ESMTPS id w2NFTb74038269 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Fri, 23 Mar 2018 15:29:43 GMT (envelope-from nospam@ofloo.net) To: freebsd-virtualization@freebsd.org From: Wouter Snels Subject: bhyve pci passthrough usb controller Message-ID: <686906fa-b531-455b-713a-796361d0741d@ofloo.net> Date: Fri, 23 Mar 2018 16:29:31 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="t920u2XkOZwgHAMsAVQlk9l9MBzLDBw8w" X-Spam-Status: No, score=-1.8 required=5.0 tests=ALL_TRUSTED,BAYES_00, HTML_MESSAGE,RDNS_NONE,SPF_PASS,TW_EV,TW_SV,TW_TD,TW_XH autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on loki.ofloo.net X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 15:30:35 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --t920u2XkOZwgHAMsAVQlk9l9MBzLDBw8w Content-Type: multipart/mixed; boundary="TlkQYThvklOLoToqeA1wi7vQnJSQadnSK"; protected-headers="v1" From: Wouter Snels To: freebsd-virtualization@freebsd.org Message-ID: <686906fa-b531-455b-713a-796361d0741d@ofloo.net> Subject: bhyve pci passthrough usb controller --TlkQYThvklOLoToqeA1wi7vQnJSQadnSK Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve windows 7 virtualization. When I start the virtual machine it freezes the entire system and this shows up on the console. To be clear the system runs without any problems without the pci passthrough. | ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: s/n ZA4292G6 detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000 rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 40 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 428, hw tdt =3D 471 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 981 em0: Tx Descriptors avail failure =3D 0 em0: RX= Queue 0 ------ em0: hw rdh =3D 912, hw rdt =3D 908 em0: RX discarded pack= ets =3D 0 em0: RX Next to Check =3D 909 em0: RX Next to Refresh =3D 908 xl0: watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D 5= 8 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966 em= 0: Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh =3D= 0, hw rdt =3D 1023 em0: RX discarded packets =3D 0 em0: RX Next to Check =3D= 0 em0: RX Next to Refresh =3D 1023 em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D 58 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966 em0: Tx Descriptors avail failure =3D 0 em0: RX= Queue 0 ------ em0: hw rdh =3D 0, hw rdt =3D 1023 em0: RX discarded packe= ts =3D 0 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D 1023 xl0: watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D 5= 4 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 970 em= 0: Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh =3D= 0, hw rdt =3D 1023 em0: RX discarded packets =3D 0include GENERIC ident OFL options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options CONSPEED=3D115200 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D= 1023 |||| |||I've recompiled the kernel without the xhci driver. I've tested both usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both showed the same error on the console screen.| # uname FreeBSD 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options CONSPEED=3D115200 * note commented "device xhci" in generic # svn info ||= |Path: . Working Copy Root Path: /usr/src URL: https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: directory Schedule: normal Last Changed Author: eadler Last Changed Rev: 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018) |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes Sun= rise Point-H USB 3.0 xHCI Controller none0 0/20/2 No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0/0 Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: class=3D0x0c0330= card=3D0x00151912 chip=3D0x00151912 rev=3D0x02 hdr=3D0x00 vendor =3D 'Renesas Technology Corp.' device =3D 'uPD720202 US= B 3.0 Host Controller' class =3D serial bus subclass =3D USB |||ppt|0@pci0:0:20= :0: class=3D0x0c0330 card=3D0x06ba1028 chip=3D0xa12f8086 rev=3D0x31 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Sunrise = Point-H USB 3.0 xHCI Controller' class =3D serial bus subclass =3D USB # grep -i pptdevfs loader.conf ||pptdevs=3D"0/20/0 4/0/0" # vm configure ||passthru= 0=3D"0/20/0"| |# vm configure ||passthru0=3D"4/0/0"| ||Any suggestions on what I'm doing wrong?|| =C2=A0 Regards, Ofloo |||||| --TlkQYThvklOLoToqeA1wi7vQnJSQadnSK-- --t920u2XkOZwgHAMsAVQlk9l9MBzLDBw8w Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJatR1bAAoJENWumses7+9mzDEQALA5HHUmC0/3IrJ+SzhUjlQk uLb0KGZdeK592J/2ERToMY8IN5WjndPDhuH4IOkGDSaNQhbUDYx0B3y7mKKHGygV vrT34hvcxaLYcuXu41vQBRxrRJB55CQJiGg0BKFkCoVknv2Opop6b0QawZrzHaH6 vGfOjXuMRIALrmvRK955kxWR9uIGoTanHJL8Hs4PGrIPL3LgJoefgvrkzXV/1DZF vMqZdpiaJqI5Aswj9ifrzpQonXxXtiSVi0XhyOAfnq0tO2mNWHjhDyhP1tTD1kQK ZAcBYk/TNqdjC0QY1Vvn3kpuL9qL0ElvhZimzRCEchg+UME7jUwTX/65y6gk5jbL 8c1ggHdOyDiy7lOn8BgMfJbBSVF5OrLh2rfsIS/so+tB1dDEmg+rdGVme9nUtKcE qtEG95S05jgrGG7UomhsfKgFAIDplJMdl8znXIJ5DutaZqGEvZvPpGidWfAci1kl esbw+SHSI9xmr39WJWqTj2jEGeqcjHpMNQPQOdUnGBx3h7Vp1pGiLvLotBFIin0J yM5kz62E9nNIewgbgy4L6DV0YJMc5mfvpmOUe/KC3oO53jsmUvXXcaOcgjt3PHwm Hd1Ry2LHvRoS7OLhsEGgNYWdm23x3BZ1djGxIsO9JwDC0gWqSj685bvfgF27NOuD aALqiWrkoNPChoNkOHgp =I8od -----END PGP SIGNATURE----- --t920u2XkOZwgHAMsAVQlk9l9MBzLDBw8w--