From owner-freebsd-xen@FreeBSD.ORG Fri May 13 05:55:20 2011 Return-Path: Delivered-To: freebsd-xen@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE81C1065674; Fri, 13 May 2011 05:55:20 +0000 (UTC) (envelope-from snabb@epipe.com) Received: from tiktik.epipe.com (tiktik.epipe.com [IPv6:2001:1828:0:3::2]) by mx1.freebsd.org (Postfix) with ESMTP id B0B8B8FC13; Fri, 13 May 2011 05:55:20 +0000 (UTC) Received: from tiktik.epipe.com (tiktik.epipe.com [IPv6:2001:1828:0:3::2]) by tiktik.epipe.com (8.14.4/8.14.4) with ESMTP id p4D5tJPx083515 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 13 May 2011 05:55:19 GMT (envelope-from snabb@epipe.com) X-DKIM: Sendmail DKIM Filter v2.8.3 tiktik.epipe.com p4D5tJPx083515 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=epipe.com; s=default; t=1305266120; x=1305870920; bh=P5w76SbKwrWhOpXY3KFza5tSs6qvhBb6uB/K1bhvSTg=; h=Date:From:To:cc:Subject:In-Reply-To:Message-ID:References: MIME-Version:Content-Type; b=PGrspSAybnwzR89IulgPNlntVDZWnbCyW52mVj7+U47mcnwESq2WapMLCvGUWItdJ vzQma5YgF9CoWG5A6Ul9vUXZTNgNlMjrAJPje9gzXqeTFoRm6YeXPuuHA++EQtHm74 +KPsLqL5j2jpZogui6oM92Y76Uc5zQjSIpwcyu98= Date: Fri, 13 May 2011 05:55:19 +0000 (UTC) From: Janne Snabb To: "Justin T. Gibbs" In-Reply-To: <4DCC3FAA.9030608@FreeBSD.org> Message-ID: References: <4DCBEEE0.9060705@steadinet.fr> <4DCC3FAA.9030608@FreeBSD.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (tiktik.epipe.com [IPv6:2001:1828:0:3::2]); Fri, 13 May 2011 05:55:20 +0000 (UTC) Cc: freebsd-xen@FreeBSD.org, Laurent Cligny Subject: Re: [FreeBSD 8.2 amd64 XENHVM] DomU terrible network performance trought NAT X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 May 2011 05:55:21 -0000 On Thu, 12 May 2011, Justin T. Gibbs wrote: > Do you recall which path was slow (rx or tx from the perspective of > the FreeBSD driver) and what the relative difference in performance > was between the two approaches? No, which is really stupid (that is why I haven't published these numbers before). The numbers are as follows, for simple iperf TCP test, with all networking, kernel, etc. settings on default values: input (Mbit/s) output (Mbit/s) Xen FreeBSD (rtl) 724 200 Xen FreeBSD (xn) 44 1700 Xen Linux (xn) 8570 2340 KVM FreeBSD (e1000) 475 495 KVM FreeBSD (rtl) 1100 85 KVM Linux (e1000) 785 890 KVM Linux (virtio) 585 715 dom0 <-> dom0 18500 18500 The table above will probably not render correctly, but hopefully it is somewhat readable. The first column indicates the hypervisor used, the guest OS and the network driver in the guest. dom0 was running Debian's 2.6.32-5-amd64 Linux kernel. Xen was Debian's 4.0.1. Linux guests were running Debian's 2.6.32-5-amd64. FreeBSD was 8.2 amd64 with the "too many frags" and "panic: do something smart" patches to make it usable with Xen at all. I did not test multiple concurrent connections, multiple virtual machines transferring data simultaneously, mixtures of different kinds of data, CPU load nor anything like that which is also relevant. Only the raw TCP speed was measured to figure out if there is any significant difference: and yes there was as can be seen from the numbers above. Unfortunately I have no recollection and forgot to write down if input was input from dom0's perspective or domU's perspective. They should be the same way around for all the tests though. My lesson was that it does make sense to select your network drivers carefully in a virtualized environment. I also tried two different virtio patches for FreeBSD with KVM, but one of them did not work at all and the another one gave worse results than any sensible emulated hardware. -- Janne Snabb / EPIPE Communications snabb@epipe.com - http://epipe.com/