From owner-freebsd-xen@FreeBSD.ORG Wed Jun 1 21:36:33 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 21EED106564A for ; Wed, 1 Jun 2011 21:36:33 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id AF0A38FC08 for ; Wed, 1 Jun 2011 21:36:32 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p51LaGk8015943 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 2 Jun 2011 00:36:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p51LaGoC012810; Thu, 2 Jun 2011 00:36:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p51LaG2o012809; Thu, 2 Jun 2011 00:36:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 2 Jun 2011 00:36:16 +0300 From: Kostik Belousov To: Sergi Message-ID: <20110601213616.GE48734@deviant.kiev.zoral.com.ua> References: <4DE5EDD7.20105@estrafolari.com> <20110601082156.GB48734@deviant.kiev.zoral.com.ua> <4DE60820.1080409@estrafolari.com> <4DE63803.7090504@estrafolari.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="W9261hGV6KsBFa7P" Content-Disposition: inline In-Reply-To: <4DE63803.7090504@estrafolari.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-xen@freebsd.org Subject: Re: fpudna: fpcurthread == curthread XXXX times X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jun 2011 21:36:33 -0000 --W9261hGV6KsBFa7P Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jun 01, 2011 at 03:00:51PM +0200, Sergi wrote: > On 01/06/11 11:36, Sergi wrote: > >On 01/06/11 10:21, Kostik Belousov wrote: > >>On Wed, Jun 01, 2011 at 09:44:23AM +0200, Sergi wrote: > >>>Hello, > >>> > >>>I'm working with full virtual FreeBSD 8.2-RELEASE-p1 domU under debian > >>>squeeze and xen-hypervisor-4.0-amd64. > >>> > >>>If I cfg this hvm with cpu> 4 : > >>> > >>> vcpus =3D 5 > >>> > >>>these messages block the server : > >>> > >>> fpudna: fpcurthread =3D=3D curthread XXXX times > >>> > >>>The machine is pingable but I'm unable to ssh to it. > >>> > >>>On single user it works fine, fsck an so on ok, but when switching to > >>>multiuser these fpudna messages start flooding. > >>> > >>>I've googled but haven't found anything; something from 2005 about=20 > >>>fpudna : > >>> > >>> =20 > >>>http://lists.freebsd.org/pipermail/freebsd-amd64/2005-April/004413.html > >>> > >>>and this link, but I don't have the options he mentions enabled on the > >>>kernel : > >>> > >>> http://forums.freebsd.org/showthread.php?t=3D17979 > >>> > >>>Has anyone stepped on this behaviour before?, is there any workaround? > >>>The machine really seems to detect cpu's available and responds to=20 > >>>keyboard > >>>on VNC, but it's impossible to see whats written down because of the > >>>messages flooding the screen. > >>You did not specified the architecture of the domu. From the message,= =20 > >>I can > >>guess that your guest is running amd64 kernel. There are slight=20 > >>differences > >>in the handling of the FPU in i386 and amd64 that may matter there. > >> > >>The message you reported means that the FreeBSD kernel assumes that FPU > >>is currently loaded with the context of the current thread, but the > >>CR0.TS bit is set, meaning that FPU context is set for switch. > >> > >>AFAIR, HVM means that you run bare-metal kernel, right ? Most likely, > >>it is some issue with Xen itself. I am curious whether the following > >>will cause any usermode-visible regression for you: > >> > >>diff --git a/sys/amd64/amd64/fpu.c b/sys/amd64/amd64/fpu.c > >>index 08e5e57..a5ee853 100644 > >>--- a/sys/amd64/amd64/fpu.c > >>+++ b/sys/amd64/amd64/fpu.c > >>@@ -394,14 +394,8 @@ fpudna(void) > >> struct pcb *pcb; > >> > >> critical_enter(); > >>- if (PCPU_GET(fpcurthread) =3D=3D curthread) { > >>- printf("fpudna: fpcurthread =3D=3D curthread %d times\n", > >>- ++err_count); > >>- stop_emulating(); > >>- critical_exit(); > >>- return; > >>- } > >>- if (PCPU_GET(fpcurthread) !=3D NULL) { > >>+ if (PCPU_GET(fpcurthread) !=3D NULL&& > >>+ PCPU_GET(fpcurthread) !=3D curthread) { > >> printf("fpudna: fpcurthread =3D %p (%d), curthread =3D %p (%d= )\n", > >> PCPU_GET(fpcurthread), > >> PCPU_GET(fpcurthread)->td_proc->p_pid, > >Hello, > > > >yes, sorry, amd64, and yes, hvm hardware virtual machine, not=20 > >paravirtual. > > > >So, you mean patching fpu.c and recompiling the kernel, right?, I'm=20 > >new to modifiying src files. > > > >Thanks for your help, > >Sergi > > > > > >_______________________________________________ > >freebsd-xen@freebsd.org mailing list > >http://lists.freebsd.org/mailman/listinfo/freebsd-xen > >To unsubscribe, send any mail to "freebsd-xen-unsubscribe@freebsd.org" > > > Hello, >=20 > well, I patched fpu.c, recompiled the kernel, and booted ok with 4 vcpu. > Then I tried to boot with 5 vcpus and got : >=20 > kernel trap 22 with interrupts disabled > ... > kernel trap 22 with interrupts disabled > Fatal double fault > rip =3D 0xffffffff8067865a > rsp =3D 0xffffff8000000000 > rbp =3D 0xffffff8000000040 > cpuid =3D 4; apic id =3D 08 > panic: double fault > cpuid =3D 4 >=20 > 4 vcpus is the maximum number of vcpus I can use. >=20 > How do you think I can debug this in order to provide more information? At least you can add KDB/DDB to the kernel config and get a backtrace at panic. My feeling right now is that the issue is in the hypervisor, and not in the kernel. --W9261hGV6KsBFa7P Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk3msM8ACgkQC3+MBN1Mb4gq7wCaA0CaRHrWO3F2Y9NwI8VKIpb0 wzsAn1VpPFdgrZY++aSdOh6DRDF0s1oV =MA7t -----END PGP SIGNATURE----- --W9261hGV6KsBFa7P--