From owner-freebsd-virtualization@FreeBSD.ORG Fri Jan 14 02:51:49 2011 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26F6A1065670; Fri, 14 Jan 2011 02:51:49 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 889628FC1E; Fri, 14 Jan 2011 02:51:48 +0000 (UTC) Received: by eyf6 with SMTP id 6so1221784eyf.13 for ; Thu, 13 Jan 2011 18:51:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=HW7X4/m+7kqoR4P4Am6QJylt9l0t3Wzbb5ftsnBfr1s=; b=r4Ji5T/8IQRiYMdnyTlhGngFqkYRKr2SGJJ25BTA/Mfb4UAwVkqFA2DydlHvbmWKtu HYPiJ6O99cWEs1rHS0cRwZn+5CElla6ADWf42nj6SNhERfUQ978R0ifIGKCYJVwH6SRo JYlWVju53zW1dFS71evuiGBLJxHGI2hboYPTE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=jWeh6w1BYpJQc89AuSgbeokoVf4go5ZAGsE10BulxZGgyMUD7v0V2M+Vkyxd6886GA RdR6v2r0HLNwFphUtHBbL2vW2pjaTqJ7JSNKChbiFUO//n1QREQt4TDGYC0YRDIqNmqN Q8fPaFDG4jEBYn4PWzO3Xb9Nq/dhPC1Ib/M9I= MIME-Version: 1.0 Received: by 10.213.23.10 with SMTP id p10mr1163573ebb.0.1294973507358; Thu, 13 Jan 2011 18:51:47 -0800 (PST) Received: by 10.213.22.14 with HTTP; Thu, 13 Jan 2011 18:51:47 -0800 (PST) In-Reply-To: References: Date: Thu, 13 Jan 2011 21:51:47 -0500 Message-ID: From: Ryan Stone To: Brandon Gooch Content-Type: text/plain; charset=ISO-8859-1 X-Mailman-Approved-At: Fri, 14 Jan 2011 07:37:13 +0000 Cc: Jack F Vogel , freebsd-virtualization@freebsd.org Subject: Re: Would there be interest in virtualization of the ixgbe driver? X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Jan 2011 02:51:49 -0000 On Thu, Jan 13, 2011 at 3:04 PM, Brandon Gooch wrote: > It would be nice to split up the hardware for use with vnet jails. The > virtualization technique you are describing -- it sounds similar to > how network device virtualization is done in the Solaris "Project > Crossbow" implementation. Can you comment on this? It looks like what I've done is implement what they call "L2 virtualization" in Project Crossbow. > In other words, would we have the ability to have a vnet jail tied to > specific hardware resources (Rx/Tx rings with their own DMA channels > and interrupts, etc...). Exactly right. And the rx ring has a unique MAC, so that's how incoming packets are multiplexed across multiple rings(and ultimately vnets). Also, you can use RSS on top of VMDq. To use the terminology used in the 82599's datasheet, each MAC(and vnet) would be associated with a pool of 1 or more rx and tx rings. Packets are multiplexed across the pools by MAC, and then packets are multiplexed across the rx rings in that pool by a hash over the IP addresses and the TCP/UDP ports. All of this, of course, is subject to the limits of the hardware. The 82598 is quite restrictive: something like 16 pools and up to 4 rings per pool. The 82599 has a lot more pools and queues to work with. On Thu, Jan 13, 2011 at 10:46 AM, Nikos Vassiliadis wrote: > Yet, I don't know if the number of changes in the infrastructure worth the > labor, for just one specific hardware. Is ixgbe the only hardware that > support such things? > Or maybe it is some trend of the future? Basically all of the changes are within the ixgbe driver. No infrastructure should have to change to support the feature. Also, Project Crossbow was implemented for a number of different drivers, including ixgbe and igb, so it should be possible to implement similar features for other drivers. However, this will always end up being quite hardware-specific so while it'd probably be possible to use the same concepts across the different drivers, it would have to re-implemented for each driver. The if_cloner used to create the virtual ifnets could be shared but that's probably <1% of the work.