From owner-freebsd-virtualization@FreeBSD.ORG Tue May 28 19:19:10 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1AA3DF0D; Tue, 28 May 2013 19:19:10 +0000 (UTC) (envelope-from prvs=8537416cf=msw@amazon.com) Received: from smtp-fw-2101.amazon.com (smtp-fw-2101.amazon.com [72.21.196.25]) by mx1.freebsd.org (Postfix) with ESMTP id A74C89E; Tue, 28 May 2013 19:19:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=msw@amazon.com; q=dns/txt; s=amazon201209; t=1369768749; x=1401304749; h=date:from:to:cc:subject:message-id:references: mime-version:content-transfer-encoding:in-reply-to; bh=yG55VFh0WNP2/pY5xjZtRFXtjo7WsphkGwXc+KaVu14=; b=OvCxN2tJpmlGyiYjR4GGFsiei9/fHj/u/klfTLSK4pfZquLZJsMBYqsF Mr5ZcSWV27vycTm4TJtnwuOe9nwiryEy0t9xNRLrUKnXqL7u5Bh6ocEF+ k/QCb/ij5Nlm/r6aaTphrM7/V3DsOXijEQg8Y2Qx1p590nd7QkjO9ne2y Q=; X-IronPort-AV: E=Sophos;i="4.87,759,1363132800"; d="scan'208";a="568049173" Received: from smtp-in-1002.vdc.amazon.com ([10.118.132.23]) by smtp-border-fw-out-2101.iad2.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 28 May 2013 19:18:58 +0000 Received: from ex10-hub-9002.ant.amazon.com (ex10-hub-9002.ant.amazon.com [10.185.137.130]) by smtp-in-1002.vdc.amazon.com (8.13.8/8.13.8) with ESMTP id r4SJIubK015557 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=OK); Tue, 28 May 2013 19:18:58 GMT Received: from u109add4315675089e695.ant.amazon.com (10.184.8.86) by ex10-hub-9002.ant.amazon.com (10.185.137.130) with Microsoft SMTP Server id 14.2.247.3; Tue, 28 May 2013 12:18:56 -0700 Received: by u109add4315675089e695.ant.amazon.com (sSMTP sendmail emulation); Tue, 28 May 2013 12:18:55 -0700 Date: Tue, 28 May 2013 12:18:55 -0700 From: Matt Wilson To: Roger Pau =?iso-8859-1?Q?Monn=E9?= Subject: Re: [Xen-devel] FreeBSD PVHVM call for testing Message-ID: <20130528191855.GA13736@u109add4315675089e695.ant.amazon.com> References: <51957D42.9060801@citrix.com> <51959ED9.6040405@freebsd.org> <51974EC9.9030204@citrix.com> <5197A1EA.2040404@freebsd.org> <519CAFC7.1070908@citrix.com> <519D24A9.3050407@freebsd.org> <519DDC0A.9000201@citrix.com> <519E6958.6020606@freebsd.org> <519F3CD0.5090405@citrix.com> <51A4D804.9050208@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <51A4D804.9050208@citrix.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-xen@freebsd.org" , xen-devel , "freebsd-virtualization@freebsd.org" , Colin Percival , "xen-users@lists.xen.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 May 2013 19:19:10 -0000 On Tue, May 28, 2013 at 06:15:00PM +0200, Roger Pau Monné wrote: > On 24/05/13 12:11, Roger Pau Monné wrote: > > > > Thanks for the test, this is what I expected. I'm a little bit out of > > ideas since I'm not able to reproduce this on upstream Xen 4.2. Without > > knowing what's happening inside the hypervisor it's hard to tell what's > > wrong. It would be interesting to try if the same happens with a Linux > > PVHVM (not PV) running on the same instance type. > > Hello Matt, > > Colin has found an issue on the FreeBSD PVHVM port that I haven't been > able to reproduce using open source Xen, even when using the same > version as the one reported by EC2. Is there anyway you could provide > some help debugging this? Without seeing the Xen code that causes > VCPUOP_set_singleshot_timer to return EINVAL it is quite hard to figure > out what's happening inside the hypervisor. Hi Roger, VCPUOP_set_singleshot_timer returns -EINVAL when: 1) the specified vCPU ID is out of range (<0 or >MAX_VIRT_CPUS) 2) the specified vCPU ID doesn't match the running vCPU. It seems that there is a confusion between the logical vCPU ID and the local APIC physical ID. I added some debugging to case 2): diff --git a/xen/common/domain.c b/xen/common/domain.c index e728819..e3efb8c 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -901,7 +901,12 @@ long do_vcpu_op(int cmd, int vcpuid, XEN_GUEST_HANDLE(void) arg) struct vcpu_set_singleshot_timer set; if ( v != current ) + { + printk("Domain %d (vcpu#%d) VCPUOP_set_singleshot_timer specified vcpuid %d\n", + d->domain_id, current->vcpu_id, vcpuid); + return -EINVAL; + } if ( copy_from_guest(&set, arg, 1) ) return -EFAULT; The output from booting ami-e75c358e on a cr1.8xlarge: (XEN) Domain 1 (vcpu#16) VCPUOP_set_singleshot_timer specified vcpuid 1 (XEN) Domain 1 (vcpu#7) VCPUOP_set_singleshot_timer specified vcpuid 14 (XEN) Domain 1 (vcpu#23) VCPUOP_set_singleshot_timer specified vcpuid 15 (XEN) Domain 1 (vcpu#11) VCPUOP_set_singleshot_timer specified vcpuid 22 (XEN) Domain 1 (vcpu#27) VCPUOP_set_singleshot_timer specified vcpuid 23 (XEN) Domain 1 (vcpu#18) VCPUOP_set_singleshot_timer specified vcpuid 5 (XEN) Domain 1 (vcpu#2) VCPUOP_set_singleshot_timer specified vcpuid 4 (XEN) Domain 1 (vcpu#9) VCPUOP_set_singleshot_timer specified vcpuid 18 (XEN) Domain 1 (vcpu#25) VCPUOP_set_singleshot_timer specified vcpuid 19 (XEN) Domain 1 (vcpu#1) VCPUOP_set_singleshot_timer specified vcpuid 2 (XEN) Domain 1 (vcpu#6) VCPUOP_set_singleshot_timer specified vcpuid 12 (XEN) Domain 1 (vcpu#22) VCPUOP_set_singleshot_timer specified vcpuid 13 (XEN) Domain 1 (vcpu#26) VCPUOP_set_singleshot_timer specified vcpuid 21 (XEN) Domain 1 (vcpu#10) VCPUOP_set_singleshot_timer specified vcpuid 20 (XEN) Domain 1 (vcpu#14) VCPUOP_set_singleshot_timer specified vcpuid 28 (XEN) Domain 1 (vcpu#30) VCPUOP_set_singleshot_timer specified vcpuid 29 (XEN) Domain 1 (vcpu#3) VCPUOP_set_singleshot_timer specified vcpuid 6 (XEN) Domain 1 (vcpu#19) VCPUOP_set_singleshot_timer specified vcpuid 7 (XEN) Domain 1 (vcpu#12) VCPUOP_set_singleshot_timer specified vcpuid 24 (XEN) Domain 1 (vcpu#28) VCPUOP_set_singleshot_timer specified vcpuid 25 (XEN) Domain 1 (vcpu#5) VCPUOP_set_singleshot_timer specified vcpuid 10 (XEN) Domain 1 (vcpu#21) VCPUOP_set_singleshot_timer specified vcpuid 11 (XEN) Domain 1 (vcpu#24) VCPUOP_set_singleshot_timer specified vcpuid 17 (XEN) Domain 1 (vcpu#8) VCPUOP_set_singleshot_timer specified vcpuid 16 (XEN) Domain 1 (vcpu#17) VCPUOP_set_singleshot_timer specified vcpuid 3 (XEN) Domain 1 (vcpu#20) VCPUOP_set_singleshot_timer specified vcpuid 9 (XEN) Domain 1 (vcpu#4) VCPUOP_set_singleshot_timer specified vcpuid 8 (XEN) Domain 1 (vcpu#13) VCPUOP_set_singleshot_timer specified vcpuid 26 (XEN) Domain 1 (vcpu#29) VCPUOP_set_singleshot_timer specified vcpuid 27 (XEN) Domain 1 (vcpu#15) VCPUOP_set_singleshot_timer specified vcpuid 30 Note from the FreeBSD boot output: APIC: CPU 0 has ACPI ID 0 APIC: CPU 1 has ACPI ID 16 APIC: CPU 2 has ACPI ID 1 APIC: CPU 3 has ACPI ID 17 APIC: CPU 4 has ACPI ID 2 APIC: CPU 5 has ACPI ID 18 APIC: CPU 6 has ACPI ID 3 APIC: CPU 7 has ACPI ID 19 APIC: CPU 8 has ACPI ID 4 APIC: CPU 9 has ACPI ID 20 APIC: CPU 10 has ACPI ID 5 APIC: CPU 11 has ACPI ID 21 APIC: CPU 12 has ACPI ID 6 APIC: CPU 13 has ACPI ID 22 APIC: CPU 14 has ACPI ID 7 APIC: CPU 15 has ACPI ID 23 APIC: CPU 16 has ACPI ID 8 APIC: CPU 17 has ACPI ID 24 APIC: CPU 18 has ACPI ID 9 APIC: CPU 19 has ACPI ID 25 APIC: CPU 20 has ACPI ID 10 APIC: CPU 21 has ACPI ID 26 APIC: CPU 22 has ACPI ID 11 APIC: CPU 23 has ACPI ID 27 APIC: CPU 24 has ACPI ID 12 APIC: CPU 25 has ACPI ID 28 APIC: CPU 26 has ACPI ID 13 APIC: CPU 27 has ACPI ID 29 APIC: CPU 28 has ACPI ID 14 APIC: CPU 29 has ACPI ID 30 APIC: CPU 30 has ACPI ID 15 APIC: CPU 31 has ACPI ID 31 --msw