Date: Sat, 24 Mar 2018 01:25:01 +0800 From: Julian Elischer <julian@freebsd.org> To: Wouter Snels <nospam@ofloo.net>, freebsd-virtualization@freebsd.org Subject: Re: bhyve pci passthrough usb controller Message-ID: <2ca981b4-9651-253a-7549-eaece4a58ff0@freebsd.org> In-Reply-To: <5bd3c8b6-0579-0269-1ada-ab6f0a55ce8f@ofloo.net> References: <5bd3c8b6-0579-0269-1ada-ab6f0a55ce8f@ofloo.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 23/3/18 11:40 pm, Wouter Snels wrote: new wrapping is no better It looks like the device you are passing though is not removed from freebsd first. but it's hard to tell. > |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve > windows 7 virtualization. When I start the virtual machine it freezes > the entire system and this shows up on the console. To be clear the > system runs without any problems without the pci passthrough. | > ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 00 > 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid > (ada0:ahcich0:0:0:0): Error 22, Unretryable error > (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid > (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 bus > 0 scbus0 target 0 lun 0 ada0: <ST500DM002-1SB10A CC43> s/n ZA4292G6 > detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 > 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was > invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: > Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss > 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 > (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 > 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 > 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll > timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000 > rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0): > SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 > (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 40 > 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error > (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-- > resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw > tdh = 428, hw tdt = 471 em0: Tx Queue Status = -2147483648 em0: TX > descriptors avail = 981 em0: Tx Descriptors avail failure = 0 em0: RX > Queue 0 ------ em0: hw rdh = 912, hw rdt = 908 em0: RX discarded packets > = 0 em0: RX Next to Check = 909 em0: RX Next to Refresh = 908 xl0: > watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is > RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 58 > em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 966 em0: > Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0, > hw rdt = 1023 em0: RX discarded packets = 0 em0: RX Next to Check = 0 > em0: RX Next to Refresh = 1023 em0: Watchdog timeout Queue[0]-- > resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw > tdh = 0, hw tdt = 58 em0: Tx Queue Status = -2147483648 em0: TX > descriptors avail = 966 em0: Tx Descriptors avail failure = 0 em0: RX > Queue 0 ------ em0: hw rdh = 0, hw rdt = 1023 em0: RX discarded packets > = 0 em0: RX Next to Check = 0 em0: RX Next to Refresh = 1023 xl0: > watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is > RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 54 > em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 970 em0: > Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0, > hw rdt = 1023 em0: RX discarded packets = 0include GENERIC ident OFL > options IPFIREWALL|||||| > > |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve > windows 7 virtualization. When I start the virtual machine it freezes > the entire system and this shows up on the console. To be clear the > system runs without any problems without the pci passthrough. | > ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 00 > 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid > (ada0:ahcich0:0:0:0): Error 22, Unretryable error > (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid > (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 bus > 0 scbus0 target 0 lun 0 ada0: <ST500DM002-1SB10A CC43> s/n ZA4292G6 > detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 > 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was > invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: > Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss > 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 > (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 > 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 > 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll > timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000 > rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0): > SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 > (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 40 > 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error > (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-- > resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw > tdh = 428, hw tdt = 471 em0: Tx Queue Status = -2147483648 em0: TX > descriptors avail = 981 em0: Tx Descriptors avail failure = 0 em0: RX > Queue 0 ------ em0: hw rdh = 912, hw rdt = 908 em0: RX discarded packets > = 0 em0: RX Next to Check = 909 em0: RX Next to Refresh = 908 xl0: > watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is > RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 58 > em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 966 em0: > Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0, > hw rdt = 1023 em0: RX discarded packets = 0 em0: RX Next to Check = 0 > em0: RX Next to Refresh = 1023 em0: Watchdog timeout Queue[0]-- > resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw > tdh = 0, hw tdt = 58 em0: Tx Queue Status = -2147483648 em0: TX > descriptors avail = 966 em0: Tx Descriptors avail failure = 0 em0: RX > Queue 0 ------ em0: hw rdh = 0, hw rdt = 1023 em0: RX discarded packets > = 0 em0: RX Next to Check = 0 em0: RX Next to Refresh = 1023 xl0: > watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is > RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 54 > em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 970 em0: > Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0, > hw rdt = 1023 em0: RX discarded packets = 0include GENERIC ident OFL > options IPFIREWALL options IPFIREWALL_VERBOSE options > IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options > CONSPEED=115200 em0: RX Next to Check = 0 em0: RX Next to Refresh = 1023 |||| > |||I've recompiled the kernel without the xhci driver. I've tested both > usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI > Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both > showed the same error on the console screen.| # uname FreeBSD > 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL > options IPFIREWALL options IPFIREWALL_VERBOSE options > IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options > CONSPEED=115200 * note commented "device xhci" in generic # svn info |||Path: . Working Copy Root Path: /usr/src URL: > https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 > Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: > ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: > directory Schedule: normal Last Changed Author: eadler Last Changed Rev: > 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018) > |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon > E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers > pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe > Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes Sunrise Point-H USB 3.0 xHCI Controller none0 0/20/2 > No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H > CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] > pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No > Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H > LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No > Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 > 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI > Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0/0 > Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: class=0x0c0330 card=0x00151912 chip=0x00151912 rev=0x02 > hdr=0x00 vendor = 'Renesas Technology Corp.' device = 'uPD720202 USB 3.0 > Host Controller' class = serial bus subclass = USB |||ppt|0@pci0:0:20:0: class=0x0c0330 card=0x06ba1028 chip=0xa12f8086 > rev=0x31 hdr=0x00 vendor = 'Intel Corporation' device = 'Sunrise Point-H > USB 3.0 xHCI Controller' class = serial bus subclass = USB # grep -i > pptdevfs loader.conf ||pptdevs="0/20/0 4/0/0" # vm configure ||passthru0="0/20/0"| > > |# vm configure ||passthru0="4/0/0"| > ||Any suggestions on what I'm doing wrong?|| > > Regards, Ofloo > |||||| > > ||options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options > DUMMYNET options IPDIVERT options CONSPEED=115200 em0: RX Next to Check > = 0 em0: RX Next to Refresh = 1023 |||| > |||I've recompiled the kernel without the xhci driver. I've tested both > usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI > Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both > showed the same error on the console screen.| # uname FreeBSD > 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL > options IPFIREWALL options IPFIREWALL_VERBOSE options > IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options > CONSPEED=115200 * note commented "device xhci" in generic # svn info |||Path: . Working Copy Root Path: /usr/src URL: > https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 > Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: > ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: > directory Schedule: normal Last Changed Author: eadler Last Changed Rev: > 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018) > |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon > E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers > pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe > Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes Sunrise Point-H USB 3.0 xHCI Controller none0 0/20/2 > No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H > CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] > pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No > Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H > LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No > Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 > 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI > Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0/0 > Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: class=0x0c0330 card=0x00151912 chip=0x00151912 rev=0x02 > hdr=0x00 vendor = 'Renesas Technology Corp.' device = 'uPD720202 USB 3.0 > Host Controller' class = serial bus subclass = USB |||ppt|0@pci0:0:20:0: class=0x0c0330 card=0x06ba1028 chip=0xa12f8086 > rev=0x31 hdr=0x00 vendor = 'Intel Corporation' device = 'Sunrise Point-H > USB 3.0 xHCI Controller' class = serial bus subclass = USB # grep -i > pptdevfs loader.conf ||pptdevs="0/20/0 4/0/0" # vm configure ||passthru0="0/20/0"| > > |# vm configure ||passthru0="4/0/0"| > ||Any suggestions on what I'm doing wrong?|| > > edit: previous wrapping was all wrong. > > Regards, Ofloo > |||||| > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2ca981b4-9651-253a-7549-eaece4a58ff0>