From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 25 23:46:36 2012 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 53F069C4; Sun, 25 Nov 2012 23:46:36 +0000 (UTC) (envelope-from sree.openwrk@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id ECB998FC08; Sun, 25 Nov 2012 23:46:35 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id 16so13299903obc.13 for ; Sun, 25 Nov 2012 15:46:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ilSZSikZVjAXjIJ/FHZw0Pk/4rphIMt+DlKhe3pxPPY=; b=FOvEi6HE8BBBqlIq289ohE7slgEqFxugoAvDHQBnnotFjfpd+Ap+kVh2Wfv6+IKK0P jzqwUvzlCRARBHTr/GhLMbhctuEfSLwtLkbzvG3CDtSx4zMpL99fQY/7U0J80JEjdz3l SgqwMBDDRu0NccBPM1T2JrB+qE1DZOxRm9BIJ3O6MsHffSFnkVX0owu+XIo6TgaT7+Cg tjrCZz3e84584gWMf/oMggZrpXH2EZG/1Yqy+QM0xXW9ltQ8DYKVwcLKeMts3kSYpBnr pp5UNbh09tIRNjW6WZ+QzoB5Fltvw7NztYE3zdxmjy7CuWpnV+lIDav+Tcra/Wx3OoW5 4tMw== MIME-Version: 1.0 Received: by 10.182.21.175 with SMTP id w15mr8015425obe.28.1353887188914; Sun, 25 Nov 2012 15:46:28 -0800 (PST) Received: by 10.60.140.136 with HTTP; Sun, 25 Nov 2012 15:46:28 -0800 (PST) In-Reply-To: References: Date: Sun, 25 Nov 2012 15:46:28 -0800 Message-ID: Subject: Re: Bhyve: panic: "Unregistered use of FPU in kernel" when starting guest From: "sree.openwrk" To: Neel Natu Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Nov 2012 23:46:36 -0000 Hey Neel I tried to see if the problem is reproducible after the fix went in. Looks like I have landed in to another issue. Now when I start the guest vm, the host hangs and could only be rebooted. Since its not a crash I am not able to get a crash dump. Is this a known issue? Also is there a way I can config and get to kick off a trace using something lie a hw-watchdog or something? Reproduction steps- 1. I have 8gb memory and I have limited the host to see only 4gb. svas-fbsd# sysctl -a | grep hw.physmem hw.physmem: 3116384256 output from dmesg- real memory = 8589934592 (8192 MB) avail memory = 2999402496 (2860 MB) 2. I checkedout the latest code from svn:// svn.freebsd.org/base/projects/bhyve. The revision number is 243326. The host has this image built out of this. 3. These are the steps I followed, before starting the guest. Then it hangs. a) kldload vmm b) kldload if_tap c) ifconfig tap0 create 4. I use the prebuilt tarball given at http://people.freebsd.org/~neel/bhyve/vm1.tar.gz. 5. I start the vm as ./vmrun.sh vm1 and it hangs after displaying the message "Launching virtual machine "vm1" with 768MB memory below 4GB and 2048MB memory above 4GB ..." Thanks On Thu, Oct 25, 2012 at 8:16 PM, Neel Natu wrote: > Hi svas, > > I just submitted a fix for this: > http://svnweb.freebsd.org/base?view=revision&revision=242122 > > Please let me know if this fixes the panic you reported. > > best > Neel > > On Sun, Oct 7, 2012 at 11:12 AM, sree.openwrk > wrote: > > Hey Neel > > Sure. I will reproduce the issue and try to narrow down the problem. > > Will update you, if I am able to narrow down the problem. > > > > Thanks > > > > > > On Thu, Oct 4, 2012 at 9:07 PM, Neel Natu wrote: > >> > >> Hi svas, > >> > >> On Thu, Oct 4, 2012 at 5:58 PM, sree.openwrk > >> wrote: > >> > Hi > >> > I am seeing frequent kernel panic when I try to run the guest OS. I > have > >> > the > >> > latest Freebsd (I compiled the latest code from > >> > svn://svn.freebsd.org/base/projects/bhyve and installed it in my > >> > machine). > >> > While trying to run the pre-built guest provided at > >> > http://people.freebsd.org/~neel/bhyve/vm1.tar.gz I get the kernel > panic > >> > (panic: Unregistered use of FPU in kernel) > >> > I am attaching the core.txt. > >> > I also want to mention that I was successfully able to boot in to the > >> > guest > >> > OS couple of times. But around 7/10 times I get thsi host kernel > panic. > >> > > >> > stack trace: > >> > vm_run() at vm_run+0xec > >> > vmmdev_ioctl() at vmmdev_ioctl+0x4ea > >> > devfs_ioctl_f() at devfs_ioctl_f+0x7a > >> > kern_ioctl() at kern_ioctl+0xcd > >> > sys_ioctl() at sys_ioctl+0xfd > >> > amd64_syscall() at amd64_syscall+0x304 > >> > Xfast_syscall() at Xfast_syscall+0xf7 > >> > > >> > >> I am seeing this too but nowhere as frequently as you are. I hit this > >> last night for the first time and not since. > >> > >> There hasn't been any change in this part of the code for quite some > time > >> now. > >> > >> Since you can trigger this much more frequently that me, would it be > >> possible for you to narrow down which change caused this? It would be > >> hugely helpful. > >> > >> best > >> Neel > >> > >> > > >> > regards > >> > svas > >> > > >> > > > > > > From owner-freebsd-virtualization@FreeBSD.ORG Mon Nov 26 01:48:55 2012 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 59CDC8E8; Mon, 26 Nov 2012 01:48:55 +0000 (UTC) (envelope-from sree.openwrk@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id E55228FC08; Mon, 26 Nov 2012 01:48:54 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id 16so13363861obc.13 for ; Sun, 25 Nov 2012 17:48:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ej8eUkyeuUCpKrtk3MpwYkGpla+GkNQvaXAu+Bt7akQ=; b=aCN8//+fs958H3LNMsuZ9EteO5aGp9ugvS59KPsWxH4hc4DncX8yOeVOcI6nqCK4fW X1JwUyr9lgsceaoDa33gaFc1qkx8MjJQzlekmHMeGjdAOEiLHKum+G+VxDC8OGhCEl/v LTc4BdMoZ6lQ0h1/sqQgTiOdf++FZJC6tAYTUv/8z8oiinYi3E1aQViUV9ZlGeznwaFq eMvV+HllDqqPg1opoP46gWNs1Zih5SA5ExuLRt0DNEpSQctOfXCrXkbWl2gYyyIZ39Nh OXZ2PU4MtTj2Hms1MEXYFSsGviOUJGHFpDk0oqUGX52ajQ6chYetNuQJbADN9BfLJWMA VQwA== MIME-Version: 1.0 Received: by 10.182.21.135 with SMTP id v7mr8141694obe.101.1353894534114; Sun, 25 Nov 2012 17:48:54 -0800 (PST) Received: by 10.60.140.136 with HTTP; Sun, 25 Nov 2012 17:48:54 -0800 (PST) In-Reply-To: References: Date: Sun, 25 Nov 2012 17:48:54 -0800 Message-ID: Subject: Re: Bhyve: panic: "Unregistered use of FPU in kernel" when starting guest From: "sree.openwrk" To: Neel Natu Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Nov 2012 01:48:55 -0000 Update- After enabling hw watchdog- by loading the ichwd kernel module and starting the watchdog daemon I am not getting any clue. After enabling hw-watchdog and starting the guest, the system becomes unresponsive. Neither does the watchdogd trigger a system reboot. I followed this- http://koitsu.wordpress.com/2010/07/13/freebsd-and-hardwaresoftware-watchdogs/- to enable hw-watchdog. On Sun, Nov 25, 2012 at 3:46 PM, sree.openwrk wrote: > Hey Neel > I tried to see if the problem is reproducible after the fix went in. > Looks like I have landed in to another issue. Now when I start the guest > vm, the host hangs and could only be rebooted. Since its not a crash I am > not able to get a crash dump. Is this a known issue? Also is there a way I > can config and get to kick off a trace using something lie a hw-watchdog or > something? > > Reproduction steps- > 1. I have 8gb memory and I have limited the host to see only 4gb. > svas-fbsd# sysctl -a | grep hw.physmem > hw.physmem: 3116384256 > > output from dmesg- > real memory = 8589934592 (8192 MB) > avail memory = 2999402496 (2860 MB) > > 2. I checkedout the latest code from svn:// > svn.freebsd.org/base/projects/bhyve. The revision number is 243326. The > host has this image built out of this. > > 3. These are the steps I followed, before starting the guest. Then it > hangs. > a) kldload vmm > b) kldload if_tap > c) ifconfig tap0 create > > 4. I use the prebuilt tarball given at > http://people.freebsd.org/~neel/bhyve/vm1.tar.gz. > > 5. I start the vm as ./vmrun.sh vm1 and it hangs after displaying the > message "Launching virtual machine "vm1" with 768MB memory below 4GB and > 2048MB memory above 4GB ..." > > Thanks > > > > > On Thu, Oct 25, 2012 at 8:16 PM, Neel Natu wrote: > >> Hi svas, >> >> I just submitted a fix for this: >> http://svnweb.freebsd.org/base?view=revision&revision=242122 >> >> Please let me know if this fixes the panic you reported. >> >> best >> Neel >> >> On Sun, Oct 7, 2012 at 11:12 AM, sree.openwrk >> wrote: >> > Hey Neel >> > Sure. I will reproduce the issue and try to narrow down the problem. >> > Will update you, if I am able to narrow down the problem. >> > >> > Thanks >> > >> > >> > On Thu, Oct 4, 2012 at 9:07 PM, Neel Natu wrote: >> >> >> >> Hi svas, >> >> >> >> On Thu, Oct 4, 2012 at 5:58 PM, sree.openwrk >> >> wrote: >> >> > Hi >> >> > I am seeing frequent kernel panic when I try to run the guest OS. I >> have >> >> > the >> >> > latest Freebsd (I compiled the latest code from >> >> > svn://svn.freebsd.org/base/projects/bhyve and installed it in my >> >> > machine). >> >> > While trying to run the pre-built guest provided at >> >> > http://people.freebsd.org/~neel/bhyve/vm1.tar.gz I get the kernel >> panic >> >> > (panic: Unregistered use of FPU in kernel) >> >> > I am attaching the core.txt. >> >> > I also want to mention that I was successfully able to boot in to the >> >> > guest >> >> > OS couple of times. But around 7/10 times I get thsi host kernel >> panic. >> >> > >> >> > stack trace: >> >> > vm_run() at vm_run+0xec >> >> > vmmdev_ioctl() at vmmdev_ioctl+0x4ea >> >> > devfs_ioctl_f() at devfs_ioctl_f+0x7a >> >> > kern_ioctl() at kern_ioctl+0xcd >> >> > sys_ioctl() at sys_ioctl+0xfd >> >> > amd64_syscall() at amd64_syscall+0x304 >> >> > Xfast_syscall() at Xfast_syscall+0xf7 >> >> > >> >> >> >> I am seeing this too but nowhere as frequently as you are. I hit this >> >> last night for the first time and not since. >> >> >> >> There hasn't been any change in this part of the code for quite some >> time >> >> now. >> >> >> >> Since you can trigger this much more frequently that me, would it be >> >> possible for you to narrow down which change caused this? It would be >> >> hugely helpful. >> >> >> >> best >> >> Neel >> >> >> >> > >> >> > regards >> >> > svas >> >> > >> >> > >> > >> > >> > > From owner-freebsd-virtualization@FreeBSD.ORG Mon Nov 26 11:06:53 2012 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BD0545AE for ; Mon, 26 Nov 2012 11:06:53 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id A16C48FC1E for ; Mon, 26 Nov 2012 11:06:53 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id qAQB6roE019580 for ; Mon, 26 Nov 2012 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id qAQB6r5i019578 for freebsd-virtualization@FreeBSD.org; Mon, 26 Nov 2012 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 26 Nov 2012 11:06:53 GMT Message-Id: <201211261106.qAQB6r5i019578@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Nov 2012 11:06:53 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/170096 virtualization[vimage] Dynamically-attached network interface will c o kern/169991 virtualization[run] [vimage] panic after device plugged in o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF/IPFilter + VIMAGE k a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail a kern/141696 virtualization[rum] [vimage] [panic] rum(4)+ vimage = kernel panic 10 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Tue Nov 27 03:37:40 2012 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1561DBC3; Tue, 27 Nov 2012 03:37:40 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: from mail-ia0-f182.google.com (mail-ia0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9AD428FC08; Tue, 27 Nov 2012 03:37:39 +0000 (UTC) Received: by mail-ia0-f182.google.com with SMTP id x2so10903377iad.13 for ; Mon, 26 Nov 2012 19:37:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Ll7BYO/BkHWFBtaKJ93PgrbEhL6M4FCRyiHnsibu0tM=; b=0CGsd/UhmgNjmYWdXRyNB0VEcoWgwBuZuKjtirkaNf9UoBqmESBu5FCeYrHwgENI57 QBm06PShot27pWf9dJTUFMo2VRzYtetjoOxIqRsNHAb/7vsbBWUHaHnANT67aiQZlNTu 4MW9lpY1V66VkJTw5JRPbmX0s4DJyd1vVIerPK20vb/Su388ufS+DNNe0kEHRE7pQlwJ 9g+vCNrPwgnatfGwjXPTxQBg7ZrmGC7ZEbTrs6ZZYP8SqreAqNSqNsSN4Ba49M7FDs9D 80StjVXdPJOBn96QnPKDa/Rhvny8VyTxk3uZ9KZ4hSbYH+7PhtLolDoKhuTt20Oteb0g 6TnA== MIME-Version: 1.0 Received: by 10.50.203.100 with SMTP id kp4mr14028438igc.35.1353987458910; Mon, 26 Nov 2012 19:37:38 -0800 (PST) Received: by 10.64.49.44 with HTTP; Mon, 26 Nov 2012 19:37:38 -0800 (PST) In-Reply-To: References: Date: Mon, 26 Nov 2012 19:37:38 -0800 Message-ID: Subject: Re: Bhyve: panic: "Unregistered use of FPU in kernel" when starting guest From: Neel Natu To: "sree.openwrk" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Nov 2012 03:37:40 -0000 Hi Sree, On Sun, Nov 25, 2012 at 3:46 PM, sree.openwrk wrote: > Hey Neel > I tried to see if the problem is reproducible after the fix went in. > Looks like I have landed in to another issue. Now when I start the guest vm, > the host hangs and could only be rebooted. Since its not a crash I am not > able to get a crash dump. Is this a known issue? Also is there a way I can > config and get to kick off a trace using something lie a hw-watchdog or > something? > > Reproduction steps- > 1. I have 8gb memory and I have limited the host to see only 4gb. > svas-fbsd# sysctl -a | grep hw.physmem > hw.physmem: 3116384256 > > output from dmesg- > real memory = 8589934592 (8192 MB) > avail memory = 2999402496 (2860 MB) > > 2. I checkedout the latest code from > svn://svn.freebsd.org/base/projects/bhyve. The revision number is 243326. > The host has this image built out of this. > > 3. These are the steps I followed, before starting the guest. Then it > hangs. > a) kldload vmm > b) kldload if_tap > c) ifconfig tap0 create > > 4. I use the prebuilt tarball given at > http://people.freebsd.org/~neel/bhyve/vm1.tar.gz. > > 5. I start the vm as ./vmrun.sh vm1 and it hangs after displaying the > message "Launching virtual machine "vm1" with 768MB memory below 4GB and > 2048MB memory above 4GB ..." > I am not 100% certain about this but it is simple enough to quickly verify. We recently did away with the need for static partitioning of the memory between the host and the guest. So, we end up asking for free pages from the host and using them for guest memory. This means that your guest is able to only allocate memory out of the 4GB you have set aside for the host. Can you reboot without the "hw.physmem" setting and see if you can still reproduce the problem? best Neel > Thanks > > > > > On Thu, Oct 25, 2012 at 8:16 PM, Neel Natu wrote: >> >> Hi svas, >> >> I just submitted a fix for this: >> http://svnweb.freebsd.org/base?view=revision&revision=242122 >> >> Please let me know if this fixes the panic you reported. >> >> best >> Neel >> >> On Sun, Oct 7, 2012 at 11:12 AM, sree.openwrk >> wrote: >> > Hey Neel >> > Sure. I will reproduce the issue and try to narrow down the problem. >> > Will update you, if I am able to narrow down the problem. >> > >> > Thanks >> > >> > >> > On Thu, Oct 4, 2012 at 9:07 PM, Neel Natu wrote: >> >> >> >> Hi svas, >> >> >> >> On Thu, Oct 4, 2012 at 5:58 PM, sree.openwrk >> >> wrote: >> >> > Hi >> >> > I am seeing frequent kernel panic when I try to run the guest OS. I >> >> > have >> >> > the >> >> > latest Freebsd (I compiled the latest code from >> >> > svn://svn.freebsd.org/base/projects/bhyve and installed it in my >> >> > machine). >> >> > While trying to run the pre-built guest provided at >> >> > http://people.freebsd.org/~neel/bhyve/vm1.tar.gz I get the kernel >> >> > panic >> >> > (panic: Unregistered use of FPU in kernel) >> >> > I am attaching the core.txt. >> >> > I also want to mention that I was successfully able to boot in to the >> >> > guest >> >> > OS couple of times. But around 7/10 times I get thsi host kernel >> >> > panic. >> >> > >> >> > stack trace: >> >> > vm_run() at vm_run+0xec >> >> > vmmdev_ioctl() at vmmdev_ioctl+0x4ea >> >> > devfs_ioctl_f() at devfs_ioctl_f+0x7a >> >> > kern_ioctl() at kern_ioctl+0xcd >> >> > sys_ioctl() at sys_ioctl+0xfd >> >> > amd64_syscall() at amd64_syscall+0x304 >> >> > Xfast_syscall() at Xfast_syscall+0xf7 >> >> > >> >> >> >> I am seeing this too but nowhere as frequently as you are. I hit this >> >> last night for the first time and not since. >> >> >> >> There hasn't been any change in this part of the code for quite some >> >> time >> >> now. >> >> >> >> Since you can trigger this much more frequently that me, would it be >> >> possible for you to narrow down which change caused this? It would be >> >> hugely helpful. >> >> >> >> best >> >> Neel >> >> >> >> > >> >> > regards >> >> > svas >> >> > >> >> > >> > >> > > >