From owner-freebsd-net@FreeBSD.ORG Thu May 31 04:01:16 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6558106564A for ; Thu, 31 May 2012 04:01:16 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.freebsd.org (Postfix) with ESMTP id 7F33B8FC0A for ; Thu, 31 May 2012 04:01:16 +0000 (UTC) Received: from [192.168.200.2] (c-24-125-204-77.hsd1.va.comcast.net [24.125.204.77]) (authenticated bits=0) by duke.cs.duke.edu (8.14.5/8.14.5) with ESMTP id q4V41FZp021735 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 31 May 2012 00:01:15 -0400 (EDT) X-DKIM: Sendmail DKIM Filter v2.8.3 duke.cs.duke.edu q4V41FZp021735 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=cs.duke.edu; s=mail; t=1338436875; bh=B1OOx69Vr1TMS6rUWg+IgqpgSwJ07/xFuHorvou2ZMw=; h=Message-ID:Date:From:MIME-Version:To:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=a0fk4GVeONb7/yFx0Y8+hZVZotsrVoTPPJhz4pAcvuYq1ZR5/rSpDnGeeNbJ/eJo0 Qaz5hFTN5Rb34S7BMrCqXU6pbKPFq+E9PlIasaVGk7qYZTYTZvxWPLKVrJ/Z9Epgf1 +evzIj8krT6Jix5v5SXO5U9ZC97S2NShmiJGn1PH/Rz81DMpGvNHvpLwCdo/ZI+hed 0Csk8PLlVJAhdEhArlfC8cPpnaroc+F0AFsg/iC/3o8PloM3zOoq0jX7ASc40lt9/t xKHpMdNzLflfrBYGRgu+LAAka82UBzoZvVKYvxyJyYdas1R7W5lEE+dPfcZIK0qzkA gD3pQqtU20Ekg== Message-ID: <4FC6ED0B.5080000@cs.duke.edu> Date: Thu, 31 May 2012 00:01:15 -0400 From: Andrew Gallatin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:11.0) Gecko/20120329 Thunderbird/11.0.1 MIME-Version: 1.0 To: freebsd-net@freebsd.org References: <20120528161240.GA38291@onelab2.iet.unipi.it> In-Reply-To: <20120528161240.GA38291@onelab2.iet.unipi.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: some questions on virtual machine bridging. X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 04:01:16 -0000 On 05/28/12 12:12, Luigi Rizzo wrote: > I am doing some experiments with implementing a software bridge > between virtual machines, using netmap as the communication API. > > I have a first prototype up and running and it is quite fast (10 Mpps > with 60-byte frames, 4 Mpps with 1500 byte frames, compared to the > ~500-800Kpps @60 bytes that you get with the tap interface used by > openvswitch or the native linux bridging). That is awesome! > - and of course, using PCI passthrough you get more or less hw speed > (constrained by the OS), but need support from an external switch > or the NIC itself to do forwarding between different ports. > anything else ? In terms of PCI passthrough / SR-IOV there are the emerging/competing EVB and VEPA standards to allow VM<->VM communication to go on the wire to a "real" switch, then back to the correct VM. > * any high-performance virtual switching solution around ? > As mentioned, i have measured native linux bridging and in-kernel ovs > and the numbers are above (not surprising; the tap involves a syscall > on each packet if i am not mistaken, and internally you need a > data copy) You should probably compare to ESXi. I've seen ~1Mpps going to or from from 1..N VMs and in or out a port on a 10GbE interface with ESX4 and newer. Drew