From owner-freebsd-virtualization@FreeBSD.ORG  Thu Sep 26 09:50:42 2013
Return-Path: <owner-freebsd-virtualization@FreeBSD.ORG>
Delivered-To: virtualization@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id 2760E78A;
 Thu, 26 Sep 2013 09:50:42 +0000 (UTC)
 (envelope-from roger.pau@citrix.com)
Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89])
 (using TLSv1 with cipher RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46C562EA3;
 Thu, 26 Sep 2013 09:50:41 +0000 (UTC)
X-IronPort-AV: E=Sophos;i="4.90,984,1371081600"; d="scan'208";a="57588493"
Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net)
 ([10.9.154.239])
 by FTLPIPO01.CITRIX.COM with ESMTP; 26 Sep 2013 09:50:38 +0000
Received: from [127.0.0.1] (10.80.16.47) by smtprelay.citrix.com
 (10.13.107.78) with Microsoft SMTP Server id 14.2.342.4; Thu, 26 Sep 2013
 05:50:38 -0400
Message-ID: <5244036D.7060004@citrix.com>
Date: Thu, 26 Sep 2013 11:50:37 +0200
From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= <roger.pau@citrix.com>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7;
 rv:17.0) Gecko/20130801 Thunderbird/17.0.8
MIME-Version: 1.0
To: Julian Elischer <julian@freebsd.org>
Subject: Re: Xen (and others Hypervisors) how do they handle IPIs?
References: <5242F9B8.9090300@freebsd.org> <5242FB81.4090002@citrix.com>
 <5243927C.9080609@freebsd.org>
In-Reply-To: <5243927C.9080609@freebsd.org>
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-DLP: MIA2
Cc: virtualization@FreeBSD.org
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
 <freebsd-virtualization.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-virtualization>, 
 <mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>, 
 <mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 26 Sep 2013 09:50:42 -0000

On 26/09/13 03:48, Julian Elischer wrote:
> On 9/25/13 11:04 PM, Roger Pau Monné wrote:
>> On 25/09/13 16:56, Julian Elischer wrote:
>>> If CPUs are mapped around, how are IPIs handled? I assume they must be
>>> emulated?
>>>
>>> I've noticed that under Xen (on both Amazon EC2 and a Redhat server)
>>> whenever you schedule a thread it always sits on the run queue for 20
>>> uSecs before it starts running. It looks to me like it's the IPI taking
>>> a long time to be emulated.
>> This has been improved on the FreeBSD Xen PVHVM port by using PV IPIs
>> instead of the emulated ones, see r255331. It should be faster than the
>> previous emulated implementation.
>>
> I missed that.. thanks!
> Do you (or anyone else) know if this can be used on Amazon EC2?
> And do you need a specific version/configuration of Xen to be able to
> use it?

The PV IPIs require Xen version 4.0 or greater, and the PV timer
requires 4.0.1 or greater if I'm not mistaken.

If you are lucky to get an Amazon instance that uses this Xen version
(or any superior one) they will be activated by default, if not FreeBSD
will switch to the old event delivery method and PV IPIs and PV timer
will be disabled in favour of the emulated ones.

I guess it's just a matter of time before Amazon switches all their
servers to Xen 4.x.

Roger.