From owner-freebsd-current@freebsd.org Mon Mar 28 17:29:57 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C33DAE1104 for ; Mon, 28 Mar 2016 17:29:57 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: from mail-vk0-x234.google.com (mail-vk0-x234.google.com [IPv6:2607:f8b0:400c:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B5094117E for ; Mon, 28 Mar 2016 17:29:56 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: by mail-vk0-x234.google.com with SMTP id e6so164413025vkh.2 for ; Mon, 28 Mar 2016 10:29:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:cc; bh=NXeF6OXz1tjJV6CKbQgvOmRzIb4RmoovfVa8vCDivkE=; b=XDxEvoE08M7ecyfoAokZnk92ewx0fjtPmoZs5EdZ6VmRmHRMKCUbgr3WU2HrrmccYZ 26mdGTftJYHnPYnFmaBmLlmgbyorNYO20jVZSGkhayuuGMAWnRyQed+R8XmRn+iwQMMf dPT515u5ZtvMA+xPNokYsdOHPES402BNobdbYKlPawxp6UGzZejCyQJ7YckQcc4uZft4 TUhZ2pVBYXBLGDrUWbwcUWflo1KOi/+EvUoyZr3ReG7qk6iTWri443fXiAYRDCjmd417 0qGUfvtXcth877iGxFyTjgVEg9pZxvy0XKOC1V5hiSAB2fcHnnR46PFWp0e/pfI0DPM0 rRqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:cc; bh=NXeF6OXz1tjJV6CKbQgvOmRzIb4RmoovfVa8vCDivkE=; b=Cdmlt/y6EPop/h/sgtzwTH93zSJLCHCxhxTECUwE+CvjO+Fg4i++FgTfGXA695r53J gMzJLBdcC8Ye92XT4jmyPkN9FUx9jU5B+NL+hWgHq0bdFO/846Efvqa8/J8b0VLhqQKh YcWLAlzugT7X019qUf7HfjB6baUB6yhOTV9qcLp3DX1M4hapC1Zi5inaB/msl/c4IG4q 21cS5i6kqf0mhXX+6x5VWu0AC0ouAMI010Y7mZBjDEO3/CwldbSOsRVAeFh53HbbThdE HnVUPVuKhwDKwapymn/9d4WelOFZFKqRSxwI7Lq3Yk2LolUhvgNYAY9Hc8XslCc6VGhS YSUQ== X-Gm-Message-State: AD7BkJIbNR9uBeduwXKNC4SJ4L3BeCWnAE8/P2Jota11KaX3hmBVVVhKI9//zo/QKzO4aj5huF8TcMNeEo2X+A== MIME-Version: 1.0 X-Received: by 10.31.1.138 with SMTP id 132mt16095848vkb.140.1459186195645; Mon, 28 Mar 2016 10:29:55 -0700 (PDT) Received: by 10.31.194.194 with HTTP; Mon, 28 Mar 2016 10:29:55 -0700 (PDT) In-Reply-To: References: Date: Mon, 28 Mar 2016 13:29:55 -0400 Message-ID: Subject: Re: sr-iov issues, reset_hw() failed with error -100 From: Ultima Cc: freebsd-current@freebsd.org, freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Mar 2016 17:29:57 -0000 Recently upgraded to r297351, once running iovctl -C -f /etc/iovctl.conf everything appears to work as it should. The main network interface ramains connected and pinging works fine. iovctl.conf PF { device : ix1; num_vfs : 31; } DEFAULT { passthrough : true; } VF-0 { passthrough : false; } VF-1 { passthrough : false; } Once vf's have been initialized, I have found none-passthrough vf's are unusable. # devctl attach pci0:129:0:129 devctl: Failed to attach pci0:129:0:129: Input/output error The dmesg spits out the same error as before, so it appears that the errors I was mentioning before is actually the none-passthrough vf's attempting to attach, but fails. /var/log/messages: ixv0: at device 0.131 on pci12 ixv0: Using MSIX interrupts with 2 vectors ixv0: ixgbe_reset_hw() failed with error -100 device_attach: ixv0 attach returned 5 ixv0: at device 0.129 on pci12 ixv0: Using MSIX interrupts with 2 vectors ixv0: ixgbe_reset_hw() failed with error -100 device_attach: ixv0 attach returned 5 On Wed, Mar 9, 2016 at 5:36 PM, Ultima wrote: > I have been interested in this when I first read about it in 2012. =] > > Can this be done in loader.conf? I have vmm_load="YES" > > I'm not sure if the vf's are usable, I have not actually tested the vf's. > The parent ix1 still shows no response. > > kldunload vmm > > kenv hw.vmm.force_iommu=1 > > kldload vmm > > iovctl -Cf /etc/iovctl.conf > > The same error messages appear, I currently on hmm i'm not sure, I > upgraded with git and it doesn't show rev(last time I use git for > source?) 8b372d1(master) > > On Wed, Mar 9, 2016 at 5:00 PM, Eric Joyner wrote: > >> I don't know if you're still interested in this, but did you do "kenv >> hw.vmm.force_iommu=1" before loading the vmm module? I think that might be >> necessary. >> >> On Wed, Feb 24, 2016 at 5:12 PM Ultima wrote: >> >>> Yeah, still getting the -100 error, I do have sendmail disabled. I just >>> tested with sendmail up and running then add the VF's and it still shows >>> the error message. >>> >>> On Wed, Feb 24, 2016 at 8:04 PM, Eric Joyner wrote: >>> >>>> Are you still getting the -100 errors when trying to load the VF driver? >>>> >>>> I've tried SR-IOV on a system here, and I can confirm that traffic >>>> stops passing on the PF interface when you create a VF interface. That >>>> didn't used to happen, so I'm investigating why that is right now. >>>> >>>> On Wed, Feb 24, 2016 at 8:09 AM Ultima wrote: >>>> >>>>> Decided to do some more tests, I actually have a second board with >>>>> sr-iov >>>>> capabilities that I used for awhile with vmware esxi. I decided to test >>>>> this out and unfortunately it won't activate, it is giving the no space >>>>> left on device error message. I double checked bios and all VT-d >>>>> related >>>>> options are enabled and have hw.ix.num_queues="4" in >>>>> /boot/loader.conf. Is >>>>> there anything else that may need to be set? .(It did work on vmware) >>>>> >>>>> For my second test, I moved the X540-AT1 to the board with the >>>>> X540-AT2. >>>>> It functioned with the same issues as the AT2 tho. >>>>> >>>>> >>>>> I don't think I listed the motherboards in question yet so ill list >>>>> them >>>>> now. >>>>> >>>>> S1200BTLRM - >>>>> >>>>> http://ark.intel.com/products/69633/Intel-Server-Board-S1200BTLRM?q=S1200BTLRM >>>>> MD80-TM0 >>>>> >>>>> - http://b2b.gigabyte.com/products/product-page.aspx?pid=5146#ov >>>>> >>>>> I'm not sure if it will be of any help tho. >>>>> >>>>> Ultima >>>>> _______________________________________________ >>>>> freebsd-current@freebsd.org mailing list >>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current >>>>> To unsubscribe, send any mail to " >>>>> freebsd-current-unsubscribe@freebsd.org" >>>>> >>>> >>> >