From owner-freebsd-virtualization@FreeBSD.ORG Fri Sep 27 07:44:35 2013 Return-Path: Delivered-To: virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 81AA5422 for ; Fri, 27 Sep 2013 07:44:35 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 38E8528A0 for ; Fri, 27 Sep 2013 07:44:34 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-245-177.lns20.per2.internode.on.net [121.45.245.177]) (authenticated bits=0) by vps1.elischer.org (8.14.6/8.14.6) with ESMTP id r8R7iBdB038733 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 27 Sep 2013 00:44:24 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <52453747.6090706@freebsd.org> Date: Fri, 27 Sep 2013 15:44:07 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= Subject: Re: Xen (and others Hypervisors) how do they handle IPIs? References: <5242F9B8.9090300@freebsd.org> <5242FB81.4090002@citrix.com> <5243927C.9080609@freebsd.org> <5244036D.7060004@citrix.com> In-Reply-To: <5244036D.7060004@citrix.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 07:44:35 -0000 On 9/26/13 5:50 PM, Roger Pau Monné wrote: > On 26/09/13 03:48, Julian Elischer wrote: >> On 9/25/13 11:04 PM, Roger Pau Monné wrote: >>> On 25/09/13 16:56, Julian Elischer wrote: >>>> If CPUs are mapped around, how are IPIs handled? I assume they must be >>>> emulated? >>>> >>>> I've noticed that under Xen (on both Amazon EC2 and a Redhat server) >>>> whenever you schedule a thread it always sits on the run queue for 20 >>>> uSecs before it starts running. It looks to me like it's the IPI taking >>>> a long time to be emulated. >>> This has been improved on the FreeBSD Xen PVHVM port by using PV IPIs >>> instead of the emulated ones, see r255331. It should be faster than the >>> previous emulated implementation. >>> >> I missed that.. thanks! >> Do you (or anyone else) know if this can be used on Amazon EC2? >> And do you need a specific version/configuration of Xen to be able to >> use it? I ran a new GENERIC kernel on an Amazon Xen 4.2 system and it performed a bit better. delays in the running of threads reduced from a constant 20uSecs to a variable amount between 1.5uSecs and 9uSecs. It's still very slow when compared with real hardware. (timing viewed using Schedgraph) Is there a way to see whether PV IPIs are active? (it looks it from the results but that could be so many other things too). Julian > The PV IPIs require Xen version 4.0 or greater, and the PV timer > requires 4.0.1 or greater if I'm not mistaken. > > If you are lucky to get an Amazon instance that uses this Xen version > (or any superior one) they will be activated by default, if not FreeBSD > will switch to the old event delivery method and PV IPIs and PV timer > will be disabled in favour of the emulated ones. > > I guess it's just a matter of time before Amazon switches all their > servers to Xen 4.x. > > Roger. > > >