From owner-freebsd-virtualization@freebsd.org Sat Feb 20 19:57:37 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7EAFFAAEDBC; Sat, 20 Feb 2016 19:57:37 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: from mail-vk0-x22e.google.com (mail-vk0-x22e.google.com [IPv6:2607:f8b0:400c:c05::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 39410160B; Sat, 20 Feb 2016 19:57:37 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: by mail-vk0-x22e.google.com with SMTP id k196so102077395vka.0; Sat, 20 Feb 2016 11:57:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=2Z9JxVpdJh+332GZx+OuFz0ZFwFyWmg2E5rlnVa6YNA=; b=mXg0/8hDQ6D3i/MUMRKqUcZEXpk2rhcXM+X+7CdiFubmKT0uvLMJa/Tb6MtIVwglk/ fMI57trkYKUxeSzavNOy5Axsb1igXhHzjMmG7+HMyvOO19sGWufjVY4NT9L0GB2a7wnw d45XE9+omoYYsfEtRK4Tb1dlRL8G6ORVKV3s0xu+JPsplGtSxnO0Ux561TqMgS8UEe3p 9+cXVJMzv0twnahhBzqNvcUa8VRAdW/WfdEIJsOOR+czvLWYAP3faXhvXY80WkKeI4lt ywphkeXJedDOVssvSFrnHwr3v+/OJXyp2MgPyTnCh4qtC+5Da88TuFJBe0GA5Yel+Xu8 8X8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=2Z9JxVpdJh+332GZx+OuFz0ZFwFyWmg2E5rlnVa6YNA=; b=S87WVEHkrOkAZBWdvkSqh+TJrZ9W3oNBeL5pt+kjhlAlG93KynrJektMIu4FSSyWN5 p66VUYtY+mFtTfZZq5URH/LyO71Uup+zuCz8Bugqzk98GzLCY2dY/qLdx3RMwNerQK5O NgeqQ2rXGwCc7atoNK5OMiBRoGzTFwbEEnO1aCE/uEb/+9lKCs1rysKrH15ptg2DOr6k MU6sIBwDOHE/Pb6L5LoR3TyoArTw9UohqqgbdiKJJ0c2tyJH2ngUMwMUxgOBw/AaJra5 M35tNLoiGn6UzQgc75q5S7EmNK86vEZVtGDNqXx+458ulskrk51YD1QZWlx+0egbHINI kaVQ== X-Gm-Message-State: AG10YOQeJadlO4DctM1CshgWsz4E4pBKaRyLx9iO54xtqMJ0NGOzZbu26djYPqguyh2Aw5HnH0XcpwxTn8LVQg== MIME-Version: 1.0 X-Received: by 10.31.34.67 with SMTP id i64mr7873202vki.140.1455998256155; Sat, 20 Feb 2016 11:57:36 -0800 (PST) Received: by 10.31.194.194 with HTTP; Sat, 20 Feb 2016 11:57:36 -0800 (PST) Date: Sat, 20 Feb 2016 14:57:36 -0500 Message-ID: Subject: sr-iov issues, reset_hw() failed with error -100 From: Ultima To: freebsd-current@freebsd.org, freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2016 19:57:37 -0000 Decided to do some testing with iovctl to see how sr-iov is coming along. Currently when adding the vf's there are a couple errors, and the network no longer function after iovctl is started. My guess is the reset_hw() call that is failing. Any ideas why this call would fail? I tested this on both ports, ix1 is detached and unused for this test, however inserting a cable results in an unusable port. iovctl -Dd ix1 removes the vf's, however functionality is still not restored without a system restart. FreeBSD S1 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r295736: Wed Feb 17 21:17:28 EST 2016 root@S1:/usr/obj/usr/src/sys/MYKERNEL amd64 /boot/loader.conf hw.ix.num_queues="4" /etc/iovctl.conf PF { device : ix1; num_vfs : 31; } DEFAULT { passthrough : true; } VF-0 { passthrough : false; } VF-1 { passthrough : false; } # iovctl -C -f /etc/iovctl.conf dmesg ixv0: at device 0.129 on pci12 ixv0: Using MSIX interrupts with 2 vectors ixv0: ixgbe_reset_hw() failed with error -100 device_attach: ixv0 attach returned 5 ixv0: at device 0.131 on pci12 ixv0: Using MSIX interrupts with 2 vectors ixv0: ixgbe_reset_hw() failed with error -100 device_attach: ixv0 attach returned 5 pci12: at device 0.133 (no driver attached) pci12: at device 0.135 (no driver attached) pci12: at device 0.137 (no driver attached) pci12: at device 0.139 (no driver attached) pci12: at device 0.141 (no driver attached) pci12: at device 0.143 (no driver attached) pci12: at device 0.145 (no driver attached) pci12: at device 0.147 (no driver attached) pci12: at device 0.149 (no driver attached) pci12: at device 0.151 (no driver attached) pci12: at device 0.153 (no driver attached) pci12: at device 0.155 (no driver attached) pci12: at device 0.157 (no driver attached) pci12: at device 0.159 (no driver attached) pci12: at device 0.161 (no driver attached) pci12: at device 0.163 (no driver attached) pci12: at device 0.165 (no driver attached) pci12: at device 0.167 (no driver attached) pci12: at device 0.169 (no driver attached) pci12: at device 0.171 (no driver attached) pci12: at device 0.173 (no driver attached) pci12: at device 0.175 (no driver attached) pci12: at device 0.177 (no driver attached) pci12: at device 0.179 (no driver attached) pci12: at device 0.181 (no driver attached) pci12: at device 0.183 (no driver attached) pci12: at device 0.185 (no driver attached) pci12: at device 0.187 (no driver attached) pci12: at device 0.189 (no driver attached) pciconf -lv ix1@pci0:129:0:1: class=0x020000 card=0x00001458 chip=0x15288086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'Ethernet Controller 10-Gigabit X540-AT2' class = network subclass = ethernet none155@pci0:129:0:129: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet none156@pci0:129:0:131: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt0@pci0:129:0:133: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt1@pci0:129:0:135: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt2@pci0:129:0:137: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt3@pci0:129:0:139: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt4@pci0:129:0:141: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt5@pci0:129:0:143: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt6@pci0:129:0:145: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt7@pci0:129:0:147: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt8@pci0:129:0:149: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt9@pci0:129:0:151: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt10@pci0:129:0:153: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt11@pci0:129:0:155: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt12@pci0:129:0:157: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt13@pci0:129:0:159: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt14@pci0:129:0:161: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt15@pci0:129:0:163: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt16@pci0:129:0:165: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt17@pci0:129:0:167: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt18@pci0:129:0:169: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt19@pci0:129:0:171: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt20@pci0:129:0:173: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt21@pci0:129:0:175: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt22@pci0:129:0:177: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt23@pci0:129:0:179: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt24@pci0:129:0:181: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt25@pci0:129:0:183: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt26@pci0:129:0:185: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt27@pci0:129:0:187: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet ppt28@pci0:129:0:189: class=0x020000 card=0x00001458 chip=0x15158086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'X540 Ethernet Controller Virtual Function' class = network subclass = ethernet Ultima