Date: Tue, 23 Feb 2016 01:52:53 +0000 From: Eric Joyner <erj@freebsd.org> To: Ultima <ultima1252@gmail.com> Cc: freebsd-current@freebsd.org, freebsd-virtualization@freebsd.org Subject: Re: sr-iov issues, reset_hw() failed with error -100 Message-ID: <CA%2Bb0zg-mFqZJLu8AmVVrxSUOzgBpC8xWrbCQxzexy4vG7khKMA@mail.gmail.com> In-Reply-To: <CANJ8om5vjNEWr_SpEqtJjdPQj5PPZRHF%2B1S=NdOULGWeQUtQ=A@mail.gmail.com> References: <CANJ8om4_QnXU_4qywqAjrUW%2B25xNrt2nTE_ED7QQgfygLSqxzA@mail.gmail.com> <CA%2Bb0zg_F99P5ufChNtqdB_VWsNdHCGP_C7iapvCKYmyqL6KYiA@mail.gmail.com> <CANJ8om5vjNEWr_SpEqtJjdPQj5PPZRHF%2B1S=NdOULGWeQUtQ=A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
I don't really have any ideas on the error -100. Error -100 means there was
a mailbox error, so something failed in the initial communications setup
between the PF and VF, but I don't know what exactly went wrong.
I'm grasping at straws, but try using a smaller number of VFs initially,
like 2? And check to see if VT-d is enabled in your BIOS? (Though I
would've expected iovctl to fail).
- Eric
On Mon, Feb 22, 2016 at 12:01 PM Ultima <ultima1252@gmail.com> wrote:
> After reboot...
>
> ifconfig ix1 up
>
> dhclient ix1
> DHCPDISCOVER on ix1 to 255.255.255.255 port 67 interval 4
> DHCPOFFER from 192.168.1.1
> DHCPREQUEST on ix1 to 255.255.255.255 port 67
> DHCPACK from 192.168.1.1
> bound to 192.168.1.145 -- renewal in 21600 seconds.
>
> ix0 down
> ping 192.168.1.1
> PING 192.168.1.1 (192.168.1.1): 56 data bytes
> 64 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=0.149 ms
> 64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.171 ms
> 64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.167 ms
>
> iovctl -Cf /etc/iovctl.conf
>
> ping 192.168.1.1
> PING 192.168.1.1 (192.168.1.1): 56 data bytes
> ^C
> --- 192.168.1.1 ping statistics ---
> 29 packets transmitted, 0 packets received, 100.0% packet loss
> ifconfig ix1 up
> ping 192.168.1.1
> PING 192.168.1.1 (192.168.1.1): 56 data bytes
> ^C
> --- 192.168.1.1 ping statistics ---
> 12 packets transmitted, 0 packets received, 100.0% packet loss
>
> ix1 is no longer usable until a restart...
>
> iovctl -Dd ix1
> ifconfig ix1 up
> ping 192.168.1.1
> PING 192.168.1.1 (192.168.1.1): 56 data bytes
> ^C
> --- 192.168.1.1 ping statistics ---
> 9 packets transmitted, 0 packets received, 100.0% packet loss
>
>
>
> Is there anything else that maybe useful?
>
> here is my ifconfig at the end (after ifconfig ix0 up)
>
>
> ix0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0
> mtu 1500
>
> options=e400b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
> ether -Hidden-
> inet 192.168.1.8 netmask 0xffffff00 broadcast 192.168.1.255
> inet 192.168.1.9 netmask 0xffffff00 broadcast 192.168.1.255
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> media: Ethernet autoselect (10Gbase-T <full-duplex,rxpause,txpause>)
> status: active
> ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>
> options=e407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
> ether -Hidden-
> inet 192.168.1.145 netmask 0xffffff00 broadcast 192.168.1.255
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> media: Ethernet autoselect (10Gbase-T <full-duplex,rxpause,txpause>)
> status: active
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
> options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
> inet6 ::1 prefixlen 128
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
> inet 127.0.0.1 netmask 0xff000000
> nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
> groups: lo
> bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
> 1500
> ether -Hidden-
> nd6 options=9<PERFORMNUD,IFDISABLED>
> groups: bridge
> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
> maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
> member: ix0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
> ifmaxaddr 0 port 1 priority 128 path cost 2000
> member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
> ifmaxaddr 0 port 5 priority 128 path cost 2000
> epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric
> 0 mtu 1500
> options=8<VLAN_MTU>
> ether -Hidden-
> inet6 fe80::ff:70ff:fe00:50a%epair0a prefixlen 64 scopeid 0x5
> nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
> status: active
> groups: epair
>
> On Mon, Feb 22, 2016 at 1:51 PM, Eric Joyner <erj@freebsd.org> wrote:
>
>> Did you do an ifconfig up on ix1 before loading the VF driver?
>>
>> On Sat, Feb 20, 2016 at 11:57 AM Ultima <ultima1252@gmail.com> wrote:
>>
>>> Decided to do some testing with iovctl to see how sr-iov is coming
>>> along.
>>> Currently when adding the vf's there are a couple errors, and the network
>>> no longer function after iovctl is started. My guess is the reset_hw()
>>> call
>>> that is failing. Any ideas why this call would fail? I tested this on
>>> both
>>> ports, ix1 is detached and unused for this test, however inserting a
>>> cable
>>> results in an unusable port. iovctl -Dd ix1 removes the vf's, however
>>> functionality is still not restored without a system restart.
>>>
>>> FreeBSD S1 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r295736: Wed Feb 17
>>> 21:17:28 EST 2016 root@S1:/usr/obj/usr/src/sys/MYKERNEL amd64
>>>
>>> /boot/loader.conf
>>> hw.ix.num_queues="4"
>>>
>>> /etc/iovctl.conf
>>> PF {
>>> device : ix1;
>>> num_vfs : 31;
>>> }
>>>
>>> DEFAULT {
>>> passthrough : true;
>>> }
>>> VF-0 {
>>> passthrough : false;
>>> }
>>> VF-1 {
>>> passthrough : false;
>>> }
>>>
>>> # iovctl -C -f /etc/iovctl.conf
>>>
>>> dmesg
>>> ixv0: <Intel(R) PRO/10GbE Virtual Function Network Driver, Version -
>>> 1.4.6-k> at device 0.129 on pci12
>>> ixv0: Using MSIX interrupts with 2 vectors
>>> ixv0: ixgbe_reset_hw() failed with error -100
>>> device_attach: ixv0 attach returned 5
>>> ixv0: <Intel(R) PRO/10GbE Virtual Function Network Driver, Version -
>>> 1.4.6-k> at device 0.131 on pci12
>>> ixv0: Using MSIX interrupts with 2 vectors
>>> ixv0: ixgbe_reset_hw() failed with error -100
>>> device_attach: ixv0 attach returned 5
>>> pci12: <network, ethernet> at device 0.133 (no driver attached)
>>> pci12: <network, ethernet> at device 0.135 (no driver attached)
>>> pci12: <network, ethernet> at device 0.137 (no driver attached)
>>> pci12: <network, ethernet> at device 0.139 (no driver attached)
>>> pci12: <network, ethernet> at device 0.141 (no driver attached)
>>> pci12: <network, ethernet> at device 0.143 (no driver attached)
>>> pci12: <network, ethernet> at device 0.145 (no driver attached)
>>> pci12: <network, ethernet> at device 0.147 (no driver attached)
>>> pci12: <network, ethernet> at device 0.149 (no driver attached)
>>> pci12: <network, ethernet> at device 0.151 (no driver attached)
>>> pci12: <network, ethernet> at device 0.153 (no driver attached)
>>> pci12: <network, ethernet> at device 0.155 (no driver attached)
>>> pci12: <network, ethernet> at device 0.157 (no driver attached)
>>> pci12: <network, ethernet> at device 0.159 (no driver attached)
>>> pci12: <network, ethernet> at device 0.161 (no driver attached)
>>> pci12: <network, ethernet> at device 0.163 (no driver attached)
>>> pci12: <network, ethernet> at device 0.165 (no driver attached)
>>> pci12: <network, ethernet> at device 0.167 (no driver attached)
>>> pci12: <network, ethernet> at device 0.169 (no driver attached)
>>> pci12: <network, ethernet> at device 0.171 (no driver attached)
>>> pci12: <network, ethernet> at device 0.173 (no driver attached)
>>> pci12: <network, ethernet> at device 0.175 (no driver attached)
>>> pci12: <network, ethernet> at device 0.177 (no driver attached)
>>> pci12: <network, ethernet> at device 0.179 (no driver attached)
>>> pci12: <network, ethernet> at device 0.181 (no driver attached)
>>> pci12: <network, ethernet> at device 0.183 (no driver attached)
>>> pci12: <network, ethernet> at device 0.185 (no driver attached)
>>> pci12: <network, ethernet> at device 0.187 (no driver attached)
>>> pci12: <network, ethernet> at device 0.189 (no driver attached)
>>>
>>> pciconf -lv
>>> ix1@pci0:129:0:1: class=0x020000 card=0x00001458 chip=0x15288086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'Ethernet Controller 10-Gigabit X540-AT2'
>>> class = network
>>> subclass = ethernet
>>> none155@pci0:129:0:129: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> none156@pci0:129:0:131: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt0@pci0:129:0:133: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt1@pci0:129:0:135: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt2@pci0:129:0:137: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt3@pci0:129:0:139: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt4@pci0:129:0:141: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt5@pci0:129:0:143: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt6@pci0:129:0:145: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt7@pci0:129:0:147: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt8@pci0:129:0:149: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt9@pci0:129:0:151: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt10@pci0:129:0:153: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt11@pci0:129:0:155: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt12@pci0:129:0:157: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt13@pci0:129:0:159: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt14@pci0:129:0:161: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt15@pci0:129:0:163: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt16@pci0:129:0:165: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt17@pci0:129:0:167: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt18@pci0:129:0:169: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt19@pci0:129:0:171: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt20@pci0:129:0:173: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt21@pci0:129:0:175: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt22@pci0:129:0:177: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt23@pci0:129:0:179: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt24@pci0:129:0:181: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt25@pci0:129:0:183: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt26@pci0:129:0:185: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt27@pci0:129:0:187: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>> ppt28@pci0:129:0:189: class=0x020000 card=0x00001458 chip=0x15158086
>>> rev=0x01 hdr=0x00
>>> vendor = 'Intel Corporation'
>>> device = 'X540 Ethernet Controller Virtual Function'
>>> class = network
>>> subclass = ethernet
>>>
>>> Ultima
>>> _______________________________________________
>>> freebsd-current@freebsd.org mailing list
>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>>> To unsubscribe, send any mail to "
>>> freebsd-current-unsubscribe@freebsd.org"
>>>
>>
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2Bb0zg-mFqZJLu8AmVVrxSUOzgBpC8xWrbCQxzexy4vG7khKMA>