From owner-freebsd-xen@FreeBSD.ORG  Sat Apr 26 11:57:18 2014
Return-Path: <owner-freebsd-xen@FreeBSD.ORG>
Delivered-To: freebsd-xen@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 060C1136
 for <freebsd-xen@freebsd.org>; Sat, 26 Apr 2014 11:57:18 +0000 (UTC)
Received: from mail-we0-x22c.google.com (mail-we0-x22c.google.com
 [IPv6:2a00:1450:400c:c03::22c])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (Client CN "smtp.gmail.com",
 Issuer "Google Internet Authority G2" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 85F4313A3
 for <freebsd-xen@freebsd.org>; Sat, 26 Apr 2014 11:57:17 +0000 (UTC)
Received: by mail-we0-f172.google.com with SMTP id k48so746516wev.17
 for <freebsd-xen@freebsd.org>; Sat, 26 Apr 2014 04:57:15 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:in-reply-to:references:date:message-id:subject:from:to
 :cc:content-type;
 bh=AQN45VvAUdSASvh1uH+KXB4+kj2GJkk6abAGYPWP/V8=;
 b=wT7o3NdSRDY2nzW405n/BgzShGbtSrF8/+1cVUpbTfpwxSft03qkQ3i8A/lHOxL0Ax
 4lYYquvgS5QMJ4V8A4oH+JUuB5ievYO5kVw0g2CxJyYmFK2UL+ZUJx+8n8BbhEw2XJy2
 vXvtwOZIojlO5c+4g7PQWk/+tG/R/M0UBM3sb4sEW75vlfbJHWpXUBWVXBQf4asDeA/H
 dGIYL5L8LlMxAVQO2/ewPK830Lo9a7MfNmfVj7Exe0GhgmE+j7r3n00bfHChEVSwrtxd
 Kn9ebAX84uOp9FEztwTtayBURTFMXbQUnuFexulU2ufjsmqrn4L9x+SGup4DvKsjRUrv
 5mbg==
MIME-Version: 1.0
X-Received: by 10.180.39.175 with SMTP id q15mr7679156wik.4.1398513435764;
 Sat, 26 Apr 2014 04:57:15 -0700 (PDT)
Received: by 10.216.61.203 with HTTP; Sat, 26 Apr 2014 04:57:15 -0700 (PDT)
In-Reply-To: <53567847.10203@citrix.com>
References: <CAJGy1F0+G1zq9hVbifTM2Vq6HHEmCM9hnFvQ=4t-4d5x=npaCA@mail.gmail.com>
 <53567847.10203@citrix.com>
Date: Sat, 26 Apr 2014 12:57:15 +0100
Message-ID: <CAJGy1F3hcAXi4xh3Yd-QEoWrWuJb2+GfmZG1x9tVFS4Qo7ar9w@mail.gmail.com>
Subject: Re: VM in Xen 4.1; poor packet forwarding performance on xn0
From: "seanrees@gmail.com" <seanrees@gmail.com>
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Content-Filtered-By: Mailman/MimeDel 2.1.17
Cc: "freebsd-xen@freebsd.org" <freebsd-xen@freebsd.org>
X-BeenThere: freebsd-xen@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Discussion of the freebsd port to xen - implementation and usage
 <freebsd-xen.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-xen>,
 <mailto:freebsd-xen-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-xen/>
List-Post: <mailto:freebsd-xen@freebsd.org>
List-Help: <mailto:freebsd-xen-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-xen>,
 <mailto:freebsd-xen-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 26 Apr 2014 11:57:18 -0000

Hi Roger,

Thanks for the patch -- sadly, it didn't work. No change.

I did have to modify a bit for releng/10.0; for some reason patch refused
to apply it cleanly. It looked fairly straightforward but I attached inline
the patch I ultimately applied below just in case I got it wrong.

Are there any other potential differences between Xen 3.4 and 4.1? (my
provider migrated my problem VPS to a 3.4 host and the problem evaporated;
I am trying this on a new 4.1 VPS that I was able to reproduce the problem
on).

Sean


Index: hvm.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- hvm.c (revision 264963)
+++ hvm.c (working copy)
@@ -626,6 +626,7 @@

  xhp.domid =3D DOMID_SELF;
  xhp.index =3D HVM_PARAM_CALLBACK_IRQ;
+#if 0
  if (xen_feature(XENFEAT_hvm_callback_vector) !=3D 0) {
  int error;

@@ -638,6 +639,7 @@
  printf("Xen HVM callback vector registration failed (%d). "
     "Falling back to emulated device interrupt\n", error);
  }
+#endif
  xen_vector_callback_enabled =3D 0;
  if (dev =3D=3D NULL) {
  /*
@@ -783,7 +785,7 @@
  info.mfn =3D vtophys(vcpu_info) >> PAGE_SHIFT;
  info.offset =3D vtophys(vcpu_info) - trunc_page(vtophys(vcpu_info));

- rc =3D HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info, cpu, &info);
+ rc =3D 1;
  if (rc !=3D 0)
  DPCPU_SET(vcpu_info, &HYPERVISOR_shared_info->vcpu_info[cpu]);
  else


On Tue, Apr 22, 2014 at 3:10 PM, Roger Pau Monn=C3=A9 <roger.pau@citrix.com=
>wrote:

> On 18/04/14 16:57, seanrees@gmail.com wrote:
> > Hi there freebsd-xen,
> >
> > I tried first on freebsd-questions@ without success, so I thought to
> retry
> > here.
> >
> > I run OpenVPN on a FreeBSD 10.0-Rp1 VM in Xen 4.1 (HVM). I am
> experiencing
> > slow network performance on xn0 that seems to have developed after I
> > upgraded to FreeBSD 10 (no other changes). I can only achieve about
> 0.5mbps
> > through this interface when forwarding packets; packets in a single
> > direction are fine (e.g; downloading to the VPS or pushing from the VPS=
)
> > and clock in at many (>10 usually) mbps.
> >
> > Interestingly, my identical VM (configuration managed centrally) runnin=
g
> on
> > Xen 3.4 (HVM) does *not* have this issue.
>
> Hello,
>
> The difference between Xen 3.4 and Xen 4.1 is that FreeBSD will make use
> of the vector callback, the PV timer and PV IPIs when running on Xen >
> 4.0 (which should provide better performance). I'm attaching a patch
> that will make FreeBSD behave the same way when running on either Xen
> 3.4 or Xen 4.1 (by disabling all this new additions), could you please
> give it a try?
>
> >
> > I did a little debugging and here's what I've noticed:
> >    - Not related to OpenVPN, repro'd using ssh -d.
> >    - Slow VM has a very low rate of context switches (~250) while
> > forwarding, fast VM has a lot more (~2000) sampled over 5 seconds using
> > systat -v.
> >   - I can't repro a context switch limit (tried a limited fork() bomb).
> >   - Tried with *and* without LRO and TSO on xn0 (and all combinations o=
f
> > LRO and TSO on/off)
>
> I've got the feeling that the issue you are seeing is not related to the
> Xen version itself, but the Linux Dom0 kernel version (which I suppose
> is different in the Xen 3.4 and Xen 4.1 hosts). Could you ask your
> provider which Linux Dom0 kernel are they using on the different hosts?
>
> Roger.
>