From owner-freebsd-xen@FreeBSD.ORG Mon Dec 2 04:12:52 2013 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9E9239E4 for ; Mon, 2 Dec 2013 04:12:52 +0000 (UTC) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id 4D05F6EC8 for ; Mon, 2 Dec 2013 04:12:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from:mime-version:to:subject:content-type:content-transfer-encoding; s=smtpapi; bh=ZY3c4V5UZdlRZzVcJBeBT0Jy0Z8=; b=sQ4pPZYjUxLbjNGKUU dHmwfKFLOAji4Le2MaukjPtIWvYTWNzcKJIYrKHdJZD19hM3YVcZDLMe3b8z/AOM jIQWDQQHvEUyx9j1LIcE/sV3rTzqph8Fq6HiOCFKmU01Vo3hWuWyMe4znS8XNunb Dl2fQXxM+8sqEagboBL0sVx5Y= Received: by mf85 with SMTP id mf85.19570.529C08C26 Mon, 02 Dec 2013 04:12:50 +0000 (GMT) Received: from mail.tarsnap.com (unknown [10.60.208.15]) by mi47 (SG) with ESMTP id 142b18238d7.46b9.10d2425 for ; Sun, 01 Dec 2013 22:12:50 -0600 (CST) Received: (qmail 3017 invoked from network); 2 Dec 2013 04:12:49 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 2 Dec 2013 04:12:49 -0000 Received: (qmail 20224 invoked from network); 2 Dec 2013 04:10:15 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 2 Dec 2013 04:10:15 -0000 Message-ID: <529C0827.4020003@freebsd.org> Date: Sun, 01 Dec 2013 20:10:15 -0800 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: "freebsd-xen@freebsd.org" , jfv@freebsd.org Subject: Is ixgbe usable with SR-IOV? X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-SG-EID: RUbAm5H8PjswBj/QH+sYVehaJogg3iBnZcyVi1bw/IxW+rO/P7BcbCJh1NmKDR2hFptSIOoQYfD7CEE7zDQ5O3rru4FxEbbpy7M8LgiUOhDi4eMighFmfJQeLq56pMfNXwrEZE1et+d33sP1Ypa2bvEEALEg6k386JYqRgnyh/U= X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Dec 2013 04:12:52 -0000 Hi Xen people & jfv, Amazon's new "C3" instance type has support for SR-IOV using ixgbe hardware; our driver looks like it should support this, but when I turn it on it doesn't seem to work. Boot dmesg: http://pastebin.com/1Kkpfzbi The virtual hardware is recognized: > ix0: mem 0xf3000000-0xf3003fff,0xf3004000-0xf3007fff at device 3.0 on pci0 > ix0: Using MSIX interrupts with 2 vectors > ix0: Ethernet address: 02:91:23:76:38:77 And we seem to be able to send packets, but never see any responses: > ix0: link state changed to UP > Starting Network: lo0 ix0. > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > ix0: flags=8843 metric 0 mtu 1500 > options=401bb > ether 02:91:23:76:38:77 > nd6 options=29 > media: Ethernet autoselect > status: active > Starting devd. > Starting dhclient. > DHCPDISCOVER on ix0 to 255.255.255.255 port 67 interval 8 > DHCPDISCOVER on ix0 to 255.255.255.255 port 67 interval 12 > DHCPDISCOVER on ix0 to 255.255.255.255 port 67 interval 12 > DHCPDISCOVER on ix0 to 255.255.255.255 port 67 interval 9 > DHCPDISCOVER on ix0 to 255.255.255.255 port 67 interval 10 > DHCPDISCOVER on ix0 to 255.255.255.255 port 67 interval 10 > No DHCPOFFERS received. Under exactly the same Xen configuration except with SR-IOV turned off and EC2 presenting a Xen netback to us instead, the DHCP works just fine. Has anyone managed to use {FreeBSD, SR-IOV, ixgbe, Xen}? Was any magic necessary in order to make it work? -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid