From owner-freebsd-xen@FreeBSD.ORG Thu Jun 2 07:22:30 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07A05106566B for ; Thu, 2 Jun 2011 07:22:30 +0000 (UTC) (envelope-from sergi@estrafolari.com) Received: from xen-smtp02.srv.cat (xen-smtp02.srv.cat [212.36.75.133]) by mx1.freebsd.org (Postfix) with ESMTP id 90ACF8FC12 for ; Thu, 2 Jun 2011 07:22:28 +0000 (UTC) Received: from [192.168.0.30] (155.Red-88-2-251.staticIP.rima-tde.net [88.2.251.155]) (Authenticated sender: estrafolar02@noverificar) by xen-smtp02.srv.cat (Postfix) with ESMTPSA id B92257C073; Thu, 2 Jun 2011 09:24:39 +0200 (CEST) Message-ID: <4DE73A39.2060609@estrafolari.com> Date: Thu, 02 Jun 2011 09:22:33 +0200 From: Sergi User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.9) Gecko/20100515 Lightning/1.0b1 Icedove/3.0.4 MIME-Version: 1.0 To: Kostik Belousov References: <4DE5EDD7.20105@estrafolari.com> <20110601082156.GB48734@deviant.kiev.zoral.com.ua> <4DE60820.1080409@estrafolari.com> <4DE63803.7090504@estrafolari.com> <20110601213616.GE48734@deviant.kiev.zoral.com.ua> In-Reply-To: <20110601213616.GE48734@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-xen@freebsd.org Subject: Re: fpudna: fpcurthread == curthread XXXX times X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Jun 2011 07:22:30 -0000 On 01/06/11 23:36, Kostik Belousov wrote: > On Wed, Jun 01, 2011 at 03:00:51PM +0200, Sergi wrote: > >> On 01/06/11 11:36, Sergi wrote: >> >>> On 01/06/11 10:21, Kostik Belousov wrote: >>> >>>> On Wed, Jun 01, 2011 at 09:44:23AM +0200, Sergi wrote: >>>> >>>>> Hello, >>>>> >>>>> I'm working with full virtual FreeBSD 8.2-RELEASE-p1 domU under debian >>>>> squeeze and xen-hypervisor-4.0-amd64. >>>>> >>>>> If I cfg this hvm with cpu> 4 : >>>>> >>>>> vcpus = 5 >>>>> >>>>> these messages block the server : >>>>> >>>>> fpudna: fpcurthread == curthread XXXX times >>>>> >>>>> The machine is pingable but I'm unable to ssh to it. >>>>> >>>>> On single user it works fine, fsck an so on ok, but when switching to >>>>> multiuser these fpudna messages start flooding. >>>>> >>>>> I've googled but haven't found anything; something from 2005 about >>>>> fpudna : >>>>> >>>>> >>>>> http://lists.freebsd.org/pipermail/freebsd-amd64/2005-April/004413.html >>>>> >>>>> and this link, but I don't have the options he mentions enabled on the >>>>> kernel : >>>>> >>>>> http://forums.freebsd.org/showthread.php?t=17979 >>>>> >>>>> Has anyone stepped on this behaviour before?, is there any workaround? >>>>> The machine really seems to detect cpu's available and responds to >>>>> keyboard >>>>> on VNC, but it's impossible to see whats written down because of the >>>>> messages flooding the screen. >>>>> >>>> You did not specified the architecture of the domu. From the message, >>>> I can >>>> guess that your guest is running amd64 kernel. There are slight >>>> differences >>>> in the handling of the FPU in i386 and amd64 that may matter there. >>>> >>>> The message you reported means that the FreeBSD kernel assumes that FPU >>>> is currently loaded with the context of the current thread, but the >>>> CR0.TS bit is set, meaning that FPU context is set for switch. >>>> >>>> AFAIR, HVM means that you run bare-metal kernel, right ? Most likely, >>>> it is some issue with Xen itself. I am curious whether the following >>>> will cause any usermode-visible regression for you: >>>> >>>> diff --git a/sys/amd64/amd64/fpu.c b/sys/amd64/amd64/fpu.c >>>> index 08e5e57..a5ee853 100644 >>>> --- a/sys/amd64/amd64/fpu.c >>>> +++ b/sys/amd64/amd64/fpu.c >>>> @@ -394,14 +394,8 @@ fpudna(void) >>>> struct pcb *pcb; >>>> >>>> critical_enter(); >>>> - if (PCPU_GET(fpcurthread) == curthread) { >>>> - printf("fpudna: fpcurthread == curthread %d times\n", >>>> - ++err_count); >>>> - stop_emulating(); >>>> - critical_exit(); >>>> - return; >>>> - } >>>> - if (PCPU_GET(fpcurthread) != NULL) { >>>> + if (PCPU_GET(fpcurthread) != NULL&& >>>> + PCPU_GET(fpcurthread) != curthread) { >>>> printf("fpudna: fpcurthread = %p (%d), curthread = %p (%d)\n", >>>> PCPU_GET(fpcurthread), >>>> PCPU_GET(fpcurthread)->td_proc->p_pid, >>>> >>> Hello, >>> >>> yes, sorry, amd64, and yes, hvm hardware virtual machine, not >>> paravirtual. >>> >>> So, you mean patching fpu.c and recompiling the kernel, right?, I'm >>> new to modifiying src files. >>> >>> Thanks for your help, >>> Sergi >>> >>> >>> _______________________________________________ >>> freebsd-xen@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-xen >>> To unsubscribe, send any mail to "freebsd-xen-unsubscribe@freebsd.org" >>> >>> >> Hello, >> >> well, I patched fpu.c, recompiled the kernel, and booted ok with 4 vcpu. >> Then I tried to boot with 5 vcpus and got : >> >> kernel trap 22 with interrupts disabled >> ... >> kernel trap 22 with interrupts disabled >> Fatal double fault >> rip = 0xffffffff8067865a >> rsp = 0xffffff8000000000 >> rbp = 0xffffff8000000040 >> cpuid = 4; apic id = 08 >> panic: double fault >> cpuid = 4 >> >> 4 vcpus is the maximum number of vcpus I can use. >> >> How do you think I can debug this in order to provide more information? >> > At least you can add KDB/DDB to the kernel config and get a backtrace > at panic. > > My feeling right now is that the issue is in the hypervisor, and not in > the kernel. > Hello, well, I'll try to add debugging to the kernel and see if I get somewhere. I'll post on the xen-user mailing-list to see if there is some issue known in the hypervisor. It's strange that nobody in this list has had this same issue. Thanks for your help, regards, Sergi