From owner-freebsd-virtualization@FreeBSD.ORG Mon Dec 9 04:48:33 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D4A38DD9 for ; Mon, 9 Dec 2013 04:48:33 +0000 (UTC) Received: from mail-qc0-x22e.google.com (mail-qc0-x22e.google.com [IPv6:2607:f8b0:400d:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 926A211E9 for ; Mon, 9 Dec 2013 04:48:33 +0000 (UTC) Received: by mail-qc0-f174.google.com with SMTP id n7so2338430qcx.33 for ; Sun, 08 Dec 2013 20:48:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=kMVnlYF/R4lZSyWeSjwUuM+ar534vS95h6IzaukkNfY=; b=nvstnlDIhfSgSajfXZvDlnSKoM1u+u9dCADvxeU2GIfnyAg1FW/gbwyrL3UL6SGfPT ghK6iMVsKdrehKM/a+LemfgN4mO6FnAXrw6Ok5NqeyveVE7dLgYnd4n7swbfQQsCzEuv Lu2PBGOJ/83j00CtZ2CgqpD3e5GGXhfPgDJoNg5y/td7l9kKpp7ejAJpD8aegcLW/lbW j9JVomJ790molPDn2m0SWUNuPYyf764OYC8CpdmYHW/2GyD0QvSQ6JzFamf3aGxumqDn FGBl0MyVYtNKAlNicGpausLcSbGRKmSK/ItBZbdKjS5uIrwtvbL1+MvuQq5Hp0q9TGgt RsIA== MIME-Version: 1.0 X-Received: by 10.224.4.70 with SMTP id 6mr30330905qaq.86.1386564512033; Sun, 08 Dec 2013 20:48:32 -0800 (PST) Received: by 10.96.126.38 with HTTP; Sun, 8 Dec 2013 20:48:31 -0800 (PST) Received: by 10.96.126.38 with HTTP; Sun, 8 Dec 2013 20:48:31 -0800 (PST) In-Reply-To: References: Date: Sun, 8 Dec 2013 22:48:31 -0600 Message-ID: Subject: Bhyve not able to get CentOS to boot From: Matt Donovan To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Dec 2013 04:48:33 -0000 After getting CentOS installed trying to get it to boot after wards gives me a segfault or back to the shell. Not near computer to try but anyone know of an idea as using the following (hd0,msdos1)/vmlinux-2.6.32.el6.x86_64 root=/dev/sda2 as it's seeing the drive as physical instead of virtual like vda2. From owner-freebsd-virtualization@FreeBSD.ORG Mon Dec 9 07:15:46 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 37C078CD for ; Mon, 9 Dec 2013 07:15:46 +0000 (UTC) Received: from mail-ee0-x22b.google.com (mail-ee0-x22b.google.com [IPv6:2a00:1450:4013:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C314E1ACE for ; Mon, 9 Dec 2013 07:15:45 +0000 (UTC) Received: by mail-ee0-f43.google.com with SMTP id c13so1323309eek.16 for ; Sun, 08 Dec 2013 23:15:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:content-transfer-encoding:subject:message-id:date :to:mime-version; bh=ntV2pxFGTVJ7E26E0+rDDKuaOIO0QzJ52ob+If5KaC4=; b=jW9E5cJjOCaoj6+Gs3NDzdkOP8v7xLyV5OhA0/Ag0OAXyGMSCFBFDqGfGDn2Q7J43N rYANuq/Y+34heMFUOxR5uqYMhLBnZWT8b/WNJ9o2dm+e3VD4bxxDyOXwIWoj218AEwoK uo8G+3C4fANRAAw0uUIehe8K87QuaVCRIFjMFHb53bHYdROwvp5vwUD2OgYe9jtvFrb9 t1AyiK7GqkVV0Nj9bskTfVTcfU4ca6FvwWri5jMQ0rr1+fgyXhC9+Q9FmQWD0Z1n55NX RBRhuaHPGWB466Uw9dT/0d+rbmMfDfTTj4B2sdTeKsW4PeED4lE35+ZgEVY0Rg+J1O+J S9pA== X-Received: by 10.14.109.5 with SMTP id r5mr1613358eeg.110.1386573344265; Sun, 08 Dec 2013 23:15:44 -0800 (PST) Received: from [10.0.3.25] (188.165-67-87.adsl-dyn.isp.belgacom.be. [87.67.165.188]) by mx.google.com with ESMTPSA id m1sm25317561eeg.0.2013.12.08.23.15.43 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 08 Dec 2013 23:15:43 -0800 (PST) From: Sydney Meyer Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: =?windows-1252?Q?bhyve-scripts-vm=92s_with_zvol=92s_exiting?= Message-Id: Date: Mon, 9 Dec 2013 08:15:41 +0100 To: freebsd-virtualization@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1822\)) X-Mailer: Apple Mail (2.1822) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Dec 2013 07:15:46 -0000 Hello Everyone, i`m trying to start a bhyve vm with a zvol as a disk backend. I downloaded bhyve-scripts, prepared the host (.ko=92s,network,zvol), = changed =93DEVTYPE=3D=93 and =93VMDEV=3Dzvol/zroot/vm0=94, provisioned = the vm, =93sh vm0 start=94 returns the Bootloader and exits with: Booting... vmload: vm0 appears to have loaded. vmboot: zvol/zroot/vm0 does not exist! Exiting. I=92m on 10-BETA4. What am i doing wrong? Thanks, Sydney.= From owner-freebsd-virtualization@FreeBSD.ORG Mon Dec 9 11:06:56 2013 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3B88E272 for ; Mon, 9 Dec 2013 11:06:56 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 266411E8F for ; Mon, 9 Dec 2013 11:06:56 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id rB9B6uPp071197 for ; Mon, 9 Dec 2013 11:06:56 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id rB9B6tbb071195 for freebsd-virtualization@FreeBSD.org; Mon, 9 Dec 2013 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 9 Dec 2013 11:06:55 GMT Message-Id: <201312091106.rB9B6tbb071195@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Dec 2013 11:06:56 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 7 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Mon Dec 9 18:13:35 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A7485E97 for ; Mon, 9 Dec 2013 18:13:35 +0000 (UTC) Received: from mail-qe0-x229.google.com (mail-qe0-x229.google.com [IPv6:2607:f8b0:400d:c02::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6579111E3 for ; Mon, 9 Dec 2013 18:13:35 +0000 (UTC) Received: by mail-qe0-f41.google.com with SMTP id gh4so3090103qeb.14 for ; Mon, 09 Dec 2013 10:13:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=ami190pufmTKoBqzR5zeAvaCOogXjUtRM1nwZyUgw6c=; b=gL8I7wFim7+V4T4z8FvK4epKK6hL+3whDmjPs8zLD98GuBdwNYnG+4448cpbj9Ee+B nLGAhQpnwJ56HBhlydHP1nMfZIBQK8xTgEVuIIixoexoeXM7I3kI8PAwSwolNDvHjS08 MFERdPRiQUfg0mknMT4Q8lRlNiZnlpaKHQ71Zp/DV0+uUolkImheK/IZc4xQxsqGQp+D VOAMDMEnFmYsvDQxOPgL9RebRXPyMkKluph5DVTzAUYjTW51jywb++epsh9LXsj7k6gn dCm2FK6oWZTUDze1tuVDAFQlfEdxavYSizv4MSQqSraxZeFm3ZzoeRdbW7XbJZl+kWkR hq6Q== X-Received: by 10.49.127.205 with SMTP id ni13mr35828769qeb.40.1386612814475; Mon, 09 Dec 2013 10:13:34 -0800 (PST) MIME-Version: 1.0 Received: by 10.96.126.38 with HTTP; Mon, 9 Dec 2013 10:13:14 -0800 (PST) In-Reply-To: References: From: Matt Donovan Date: Mon, 9 Dec 2013 12:13:14 -0600 Message-ID: Subject: Re: Bhyve not able to get CentOS to boot To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Dec 2013 18:13:35 -0000 Ok the following is what I do get/ grub> linux (hd0,msdos1)/vmlinuz-2.6.32-431.el6.x86_64 ro root=/dev/mapper/vg_server-lv_root rd_LVM_LV=vg_server/lv_swap rd_NO_L UKS LANG=en_US.UTF-8 rd_NO_MD KEYTABLE=us SYSFONT=latarcyrheb-sun16 rd_LVM_LV=vg_server/lv_root console=ttyS0 rd_NO_DM grub> initrd (hd0,msdos1)/initramfs-2.6.32-431.el6.x86_64.img grub> boot vps# I am not sure what I am doing wrong exactly /usr/local/sbin/grub-bhyve -r hd0 -m vm/vm0/device.map -M 1024 vm0 grub> ls (cd0) (hd0) (hd0,msdos2) (hd0,msdos1) (host) (lvm/vg_server-lv_swap) (lvm/vg_server-lv_home) (lvm/vg_server-lv_root) grub> On Sun, Dec 8, 2013 at 10:48 PM, Matt Donovan wrote: > After getting CentOS installed trying to get it to boot after wards gives > me a segfault or back to the shell. Not near computer to try but anyone > know of an idea as using the following > (hd0,msdos1)/vmlinux-2.6.32.el6.x86_64 root=/dev/sda2 as it's seeing the > drive as physical instead of virtual like vda2. > -- Technological progress is like an ax in the hands of a pathological criminal. - *Albert EinsteinBreadth of Unix experience and depth of knowledge in the world of servers.* From owner-freebsd-virtualization@FreeBSD.ORG Mon Dec 9 19:32:28 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1EE563D7 for ; Mon, 9 Dec 2013 19:32:28 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id D29E4195B for ; Mon, 9 Dec 2013 19:32:27 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id BA6021220E; Tue, 10 Dec 2013 05:32:19 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BQQ02549 (AUTH peterg@ptree32.com.au); Tue, 10 Dec 2013 05:32:18 +1000 Message-ID: <52A61AC2.8050308@freebsd.org> Date: Mon, 09 Dec 2013 11:32:18 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.1.1 MIME-Version: 1.0 To: Matt Donovan , freebsd-virtualization@freebsd.org Subject: Re: Bhyve not able to get CentOS to boot References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Dec 2013 19:32:28 -0000 Hi Matt, > Ok the following is what I do get/ > > grub> linux (hd0,msdos1)/vmlinuz-2.6.32-431.el6.x86_64 ro > root=/dev/mapper/vg_server-lv_root rd_LVM_LV=vg_server/lv_swap rd_NO_L > UKS LANG=en_US.UTF-8 rd_NO_MD KEYTABLE=us SYSFONT=latarcyrheb-sun16 > rd_LVM_LV=vg_server/lv_root console=ttyS0 rd_NO_DM > grub> initrd (hd0,msdos1)/initramfs-2.6.32-431.el6.x86_64.img > grub> boot > vps# That looks good. The "boot" command doesn't actually start the VM - it finalizes setting register state so that usr/sbin/bhyve can then run it. The bhyve.org script hides that from you. > I am not sure what I am doing wrong exactly > > /usr/local/sbin/grub-bhyve -r hd0 -m vm/vm0/device.map -M 1024 vm0 > > grub> ls > (cd0) (hd0) (hd0,msdos2) (hd0,msdos1) (host) (lvm/vg_server-lv_swap) > (lvm/vg_server-lv_home) (lvm/vg_server-lv_root) > grub> Using "-r hd0,msdos1" will allow "ls /" to work, though the (hd0,msdos1) qualifier in the first example does the right thing. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Tue Dec 10 20:28:00 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 683BBF13; Tue, 10 Dec 2013 20:28:00 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 235551DB9; Tue, 10 Dec 2013 20:27:57 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 188E3B986; Tue, 10 Dec 2013 15:27:56 -0500 (EST) From: John Baldwin To: grehan@freebsd.org, neel@freebsd.org Subject: [PATCH] Additional lapic support for bhyve Date: Tue, 10 Dec 2013 15:27:47 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201312101527.47234.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 10 Dec 2013 15:27:56 -0500 (EST) Cc: virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Dec 2013 20:28:00 -0000 This patch extends the local APIC emulation a bit by adding the following things: - Add an ioctl and a bhyvectl command to trigger local interrupts on a local APIC. The "fixed" and "NMI" delivery modes are enabled. - Add support for the CMCI LVT entry. - Add the ability to report local APIC errors and trigger errors for invalid vectors when sending IPIs or firing an external interrupt that references an invalid IDT vector. - Silently force all reserved fields in LVT entries to zero when they are written (removes the need for clearing those bits when triggering an LVT interrupt). - Add entries to the MP Table and MADT to advertise the typical x86 LINT configuration (ExtINT on LINT0 and NMI on LINT1). - Add a bhyvectl command to inject an NMI on an arbitrary CPU (this latter should probably be a separate patch) In particular, while bhyectl --inject-nmi can inject an NMI on a single vcpu, the more traditional way of signalling a system-wide error (such as SERR# or PERR#) is to assert the LINT1 pins on all CPUs. This can now be done via 'bhyectl --vm=foo --cpu=-1 --assert-lapic-lvt=1' (cpu of -1 is a broadcast to all CPUs for the LVT ioctl). The patch is at people.freebsd.org/~jhb/patches/bhyve_lapic.patch I'm certainly open to suggestions on ways to make this be more consistent with the style/design/flow of the existing code. (I started on this because I want to add support for machine check injection so I can use bhyve to test the machine check code, but for that I wanted CMCI support and it kind of snowballed from there) -- John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Wed Dec 11 07:19:15 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C8E8C813; Wed, 11 Dec 2013 07:19:15 +0000 (UTC) Received: from mail-qe0-x230.google.com (mail-qe0-x230.google.com [IPv6:2607:f8b0:400d:c02::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 62FD911BD; Wed, 11 Dec 2013 07:19:15 +0000 (UTC) Received: by mail-qe0-f48.google.com with SMTP id gc15so5035782qeb.7 for ; Tue, 10 Dec 2013 23:19:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=qFJv3rFAZWCqpSVWKmwLIwL4QoJbznH+WluVS+3QIMw=; b=PrJEYH6AV4nnq2qCCO9sdcUtQDTvac9YcSA+QeiGO4FFh8MgRjUou3Ew/n7bx2zAcR 9s0jTtK5urPuwnmdlLcdO1izDovkw0V8s/H+tDAWVM+IeJvSD/hSsaQzWdD+/sTjDEen WikUt4YYTGgm5qyhZXkHn2ox9zoBnm2SS0y4b1xzGWkCOhXIu/q1M+jDhBlX5nxL24pm Re+asVK0SHwH9hUSonASgVtHn3ajBDJ329GKDJD0WykzbRMoqMTQQPNONEb/1rEN2SKe GAwTW3c7M9E8yP2x5IKxajxAz1qDTkc4zg3enyGC2J/eJjYswR5L+vvKbeiK2hgDP25T 8f4w== MIME-Version: 1.0 X-Received: by 10.224.112.68 with SMTP id v4mr51533538qap.90.1386746354584; Tue, 10 Dec 2013 23:19:14 -0800 (PST) Received: by 10.140.34.17 with HTTP; Tue, 10 Dec 2013 23:19:14 -0800 (PST) In-Reply-To: <201312101527.47234.jhb@freebsd.org> References: <201312101527.47234.jhb@freebsd.org> Date: Tue, 10 Dec 2013 23:19:14 -0800 Message-ID: Subject: Re: [PATCH] Additional lapic support for bhyve From: Neel Natu To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Dec 2013 07:19:16 -0000 Hi John, On Tue, Dec 10, 2013 at 12:27 PM, John Baldwin wrote: > This patch extends the local APIC emulation a bit by adding the following > things: > - Add an ioctl and a bhyvectl command to trigger local interrupts on a > local APIC. The "fixed" and "NMI" delivery modes are enabled. > - Add support for the CMCI LVT entry. > - Add the ability to report local APIC errors and trigger errors for > invalid vectors when sending IPIs or firing an external interrupt that > references an invalid IDT vector. > - Silently force all reserved fields in LVT entries to zero when they > are written (removes the need for clearing those bits when triggering > an LVT interrupt). > - Add entries to the MP Table and MADT to advertise the typical x86 LINT > configuration (ExtINT on LINT0 and NMI on LINT1). > - Add a bhyvectl command to inject an NMI on an arbitrary CPU (this latter > should probably be a separate patch) > > In particular, while bhyectl --inject-nmi can inject an NMI on a single vcpu, > the more traditional way of signalling a system-wide error (such as SERR# > or PERR#) is to assert the LINT1 pins on all CPUs. This can now be done via > 'bhyectl --vm=foo --cpu=-1 --assert-lapic-lvt=1' (cpu of -1 is a broadcast > to all CPUs for the LVT ioctl). The patch is at > people.freebsd.org/~jhb/patches/bhyve_lapic.patch I'm certainly open to > suggestions on ways to make this be more consistent with the style/design/flow > of the existing code. > This looks good. vlapic_fire_lvt() and vlapic_set_error() can call each other recursively if guest has programmed the ERROR_LVT vector incorrectly. This can probably be fixed by passing the LVT index to vlapic_fire_lvt() and treating LVT_ERROR specially. A couple of nits: - Could you add a /* fallthrough */ comment after setting the 'mask' for the LINT_LVT entries in vlapic_set_lvt()? - Any reason that vlapic_set_error() and vlapic_fire_cmci() are not static to vlapic.c? Do you anticipate calling them from outside this file in the future? best Neel > (I started on this because I want to add support for machine check injection > so I can use bhyve to test the machine check code, but for that I wanted CMCI > support and it kind of snowballed from there) > > -- > John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Wed Dec 11 17:38:37 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C44ED4C9; Wed, 11 Dec 2013 17:38:37 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9AB7D144C; Wed, 11 Dec 2013 17:38:37 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id C3C31B984; Wed, 11 Dec 2013 12:38:35 -0500 (EST) From: John Baldwin To: Neel Natu Subject: Re: [PATCH] Additional lapic support for bhyve Date: Wed, 11 Dec 2013 12:38:11 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <201312101527.47234.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201312111238.12232.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 11 Dec 2013 12:38:35 -0500 (EST) Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Dec 2013 17:38:37 -0000 On Wednesday, December 11, 2013 2:19:14 am Neel Natu wrote: > Hi John, > > On Tue, Dec 10, 2013 at 12:27 PM, John Baldwin wrote: > > This patch extends the local APIC emulation a bit by adding the following > > things: > > - Add an ioctl and a bhyvectl command to trigger local interrupts on a > > local APIC. The "fixed" and "NMI" delivery modes are enabled. > > - Add support for the CMCI LVT entry. > > - Add the ability to report local APIC errors and trigger errors for > > invalid vectors when sending IPIs or firing an external interrupt that > > references an invalid IDT vector. > > - Silently force all reserved fields in LVT entries to zero when they > > are written (removes the need for clearing those bits when triggering > > an LVT interrupt). > > - Add entries to the MP Table and MADT to advertise the typical x86 LINT > > configuration (ExtINT on LINT0 and NMI on LINT1). > > - Add a bhyvectl command to inject an NMI on an arbitrary CPU (this latter > > should probably be a separate patch) > > > > In particular, while bhyectl --inject-nmi can inject an NMI on a single vcpu, > > the more traditional way of signalling a system-wide error (such as SERR# > > or PERR#) is to assert the LINT1 pins on all CPUs. This can now be done via > > 'bhyectl --vm=foo --cpu=-1 --assert-lapic-lvt=1' (cpu of -1 is a broadcast > > to all CPUs for the LVT ioctl). The patch is at > > people.freebsd.org/~jhb/patches/bhyve_lapic.patch I'm certainly open to > > suggestions on ways to make this be more consistent with the style/design/flow > > of the existing code. > > > > This looks good. > > vlapic_fire_lvt() and vlapic_set_error() can call each other > recursively if guest has programmed the ERROR_LVT vector incorrectly. > This can probably be fixed by passing the LVT index to > vlapic_fire_lvt() and treating LVT_ERROR specially. Oh, cute, and very true. Another option might be to have set_error set a per vlapic flag and avoid posting the interrupt if it detects recursion. > A couple of nits: > - Could you add a /* fallthrough */ comment after setting the 'mask' > for the LINT_LVT entries in vlapic_set_lvt()? Sure. > - Any reason that vlapic_set_error() and vlapic_fire_cmci() are not > static to vlapic.c? Do you anticipate calling them from outside this > file in the future? Yes. I imagine the machine check injection code I plan to work on would be in a different file, and it would need to be able to trigger CMCI. For vlapic_set_error(), there is one error case we currently do not handle which is access to an invalid register (right now we ignore those). For x2APIC we should be generating general protection fault. For mmio access an error is supposed to be asserted instead. I figured for the latter that it would be done in lapic_mmio_read() and lapic_mmio_write(). -- John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Wed Dec 11 20:00:21 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 278AC69B; Wed, 11 Dec 2013 20:00:21 +0000 (UTC) Received: from ims7bld4.solarus.net (smtp4.solarus.net [209.94.171.124]) by mx1.freebsd.org (Postfix) with ESMTP id E7EB8121F; Wed, 11 Dec 2013 20:00:20 +0000 (UTC) Received: from mail.solarus.net (mail.tznet.com [209.94.172.150]) by ims7bld4.solarus.net with ESMTP id rBBJf4XO042062; Wed, 11 Dec 2013 13:41:34 -0600 (CST) Date: Wed, 11 Dec 2013 13:41:04 -0600 (CST) From: Scott Pilz To: Peter Grehan Subject: Re: bhyve - non *bsd guests In-Reply-To: Message-ID: References: <529579CD.9010509@freebsd.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Solarus-VirusScanner: Clean X-Spam-Status: No Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Dec 2013 20:00:21 -0000 No issues with the Intel E5-2650v2 running multiple processors per guest and Linux/OpenBSD? Just want to verify before I place a rather large order of hardware. Thanks. Scott On Tue, 26 Nov 2013, Scott Pilz wrote: > > Thank you for your quick answer, and I'm not surprised by this. Makes > perfect sense. I ran into the 1-CPU issue when I first started my testing. > > I'll ensure that the new IX blade system we're going with as > supported processors. > > Keep up the great work. I'll be happy to do some extensive testing > under various environments on this project if needed. > > Scott > > On Tue, 26 Nov 2013, Peter Grehan wrote: > >> Hi Scott, >> >>> The error I'm getting is: >>> >>> VM unrestricted guest capability required >>> Error in initializing VM >> >> It's this one: >> >>> b) the CPU does not support what I'm trying to do >> >> Your 5570 CPU is a first-gen Nehalem. These models don't have >> 'unrestricted-guest' VT-x support which is required to run code in >> 16/32-bit mode with paging disabled. This feature arrived with Westmere and >> successors. >> >> grub-bhyve boots Linux and OpenBSD in 32-bit "flat" mode - protected mode >> with paging disabled. This can only work if unrestricted-guest is supported >> by the CPU. Other hypervisors work around this by using s/w instruction >> emulation until paging is enabled. >> >> You'll find the same error if you start up a FreeBSD VM with > 1 vCPU since >> the AP spinup code is started in 16-bit real mode. >> >> This doesn't show up with a single-vCPU FreeBSD guest since bhyveload sets >> up a 64-bit execution environment for the kernel. >> >> later, >> >> Peter. >> >> > From owner-freebsd-virtualization@FreeBSD.ORG Wed Dec 11 20:04:50 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A0C9C7F8 for ; Wed, 11 Dec 2013 20:04:50 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 6041E129B for ; Wed, 11 Dec 2013 20:04:50 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 89B3012240; Thu, 12 Dec 2013 06:04:42 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BQR22505 (AUTH peterg@ptree32.com.au); Thu, 12 Dec 2013 06:04:41 +1000 Message-ID: <52A8C557.20909@freebsd.org> Date: Wed, 11 Dec 2013 12:04:39 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.1.1 MIME-Version: 1.0 To: Scott Pilz Subject: Re: bhyve - non *bsd guests References: <529579CD.9010509@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Dec 2013 20:04:50 -0000 Hi Scott, > No issues with the Intel E5-2650v2 running multiple processors per > guest and Linux/OpenBSD? Just want to verify before I place a rather > large order of hardware. I've not personally run bhyve on that exact model, but it runs fine on Ivy Bridge desktop and other server models so you should be good. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Thu Dec 12 00:44:37 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0C641D89; Thu, 12 Dec 2013 00:44:37 +0000 (UTC) Received: from mail-qa0-x22b.google.com (mail-qa0-x22b.google.com [IPv6:2607:f8b0:400d:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9C5EF1924; Thu, 12 Dec 2013 00:44:36 +0000 (UTC) Received: by mail-qa0-f43.google.com with SMTP id ii20so5393374qab.9 for ; Wed, 11 Dec 2013 16:44:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=gS7I4s/VtRZtkOAWFR/SL0RwAvwG2RlEa/1Ouwwpq00=; b=tvk/+mr7GsNRc0+vg0sI0Ec3uwksMM2rf+0xNW3PtU0Dsr4pUIzTTyPN9eEv2hAc5q Dn3iDlW7+x6174DVIBj6pWskHT9hZDGVXICYi5pZpVIJpqx0jlNA1u1rmC3eN5Wzc+HA wjhepA1MqCmKk63mt3SbXkwqOd85k7pdfZHnetnoVU3QmBGrNUUVV9Ik2NmcknrQ4Aft +P5n3OHX/li3PhlQczt037WUZPvVKi2JEPi+dW+ltPqlcn5MMpmH64IDl5hsny6LpOmL 0Nrz5L6Zievbgc8K5PRFpbJykocW/yuvwZuMaR6RtHHDi0+xnbWYYfSH8NkolBsh6nB5 JGyQ== MIME-Version: 1.0 X-Received: by 10.224.120.71 with SMTP id c7mr820072qar.37.1386809075134; Wed, 11 Dec 2013 16:44:35 -0800 (PST) Received: by 10.140.34.17 with HTTP; Wed, 11 Dec 2013 16:44:35 -0800 (PST) In-Reply-To: <201312111238.12232.jhb@freebsd.org> References: <201312101527.47234.jhb@freebsd.org> <201312111238.12232.jhb@freebsd.org> Date: Wed, 11 Dec 2013 16:44:35 -0800 Message-ID: Subject: Re: [PATCH] Additional lapic support for bhyve From: Neel Natu To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Dec 2013 00:44:37 -0000 Hi John, On Wed, Dec 11, 2013 at 9:38 AM, John Baldwin wrote: > On Wednesday, December 11, 2013 2:19:14 am Neel Natu wrote: >> Hi John, >> >> On Tue, Dec 10, 2013 at 12:27 PM, John Baldwin wrote: >> > This patch extends the local APIC emulation a bit by adding the following >> > things: >> > - Add an ioctl and a bhyvectl command to trigger local interrupts on a >> > local APIC. The "fixed" and "NMI" delivery modes are enabled. >> > - Add support for the CMCI LVT entry. >> > - Add the ability to report local APIC errors and trigger errors for >> > invalid vectors when sending IPIs or firing an external interrupt that >> > references an invalid IDT vector. >> > - Silently force all reserved fields in LVT entries to zero when they >> > are written (removes the need for clearing those bits when triggering >> > an LVT interrupt). >> > - Add entries to the MP Table and MADT to advertise the typical x86 LINT >> > configuration (ExtINT on LINT0 and NMI on LINT1). >> > - Add a bhyvectl command to inject an NMI on an arbitrary CPU (this latter >> > should probably be a separate patch) >> > >> > In particular, while bhyectl --inject-nmi can inject an NMI on a single vcpu, >> > the more traditional way of signalling a system-wide error (such as SERR# >> > or PERR#) is to assert the LINT1 pins on all CPUs. This can now be done via >> > 'bhyectl --vm=foo --cpu=-1 --assert-lapic-lvt=1' (cpu of -1 is a broadcast >> > to all CPUs for the LVT ioctl). The patch is at >> > people.freebsd.org/~jhb/patches/bhyve_lapic.patch I'm certainly open to >> > suggestions on ways to make this be more consistent with the style/design/flow >> > of the existing code. >> > >> >> This looks good. >> >> vlapic_fire_lvt() and vlapic_set_error() can call each other >> recursively if guest has programmed the ERROR_LVT vector incorrectly. >> This can probably be fixed by passing the LVT index to >> vlapic_fire_lvt() and treating LVT_ERROR specially. > > Oh, cute, and very true. Another option might be to have set_error set > a per vlapic flag and avoid posting the interrupt if it detects recursion. > Yup, that would work as well. >> A couple of nits: >> - Could you add a /* fallthrough */ comment after setting the 'mask' >> for the LINT_LVT entries in vlapic_set_lvt()? > > Sure. > >> - Any reason that vlapic_set_error() and vlapic_fire_cmci() are not >> static to vlapic.c? Do you anticipate calling them from outside this >> file in the future? > > Yes. I imagine the machine check injection code I plan to work on would > be in a different file, and it would need to be able to trigger CMCI. > > For vlapic_set_error(), there is one error case we currently do not handle > which is access to an invalid register (right now we ignore those). For > x2APIC we should be generating general protection fault. For mmio access > an error is supposed to be asserted instead. I figured for the latter > that it would be done in lapic_mmio_read() and lapic_mmio_write(). > Sounds good. best Neel > -- > John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Thu Dec 12 20:12:56 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DDE337E4 for ; Thu, 12 Dec 2013 20:12:56 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B876D1395 for ; Thu, 12 Dec 2013 20:12:56 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id C43A6B922 for ; Thu, 12 Dec 2013 15:12:55 -0500 (EST) From: John Baldwin To: virtualization@freebsd.org Subject: Panic starting a bhyve guest after resume Date: Thu, 12 Dec 2013 15:11:38 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201312121511.38608.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 12 Dec 2013 15:12:55 -0500 (EST) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Dec 2013 20:12:56 -0000 If I suspend and resume my laptop and then try to start a guest after the resume, I get an odd panic. It generates a privileged instruction fault (in kernel mode) for 'vmclear'. I've checked CR4 and it claims that VMXE is set. I dont have any other ideas off the top of my head on what I should be poking at? It looks like we read a bunch of MSRs in vmx_init(), but we don't write to them, and all vmx_enable() does on each CPU is set VMXE in CR4 from what I can tell. -- John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Thu Dec 12 21:00:09 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 52C6CCA1; Thu, 12 Dec 2013 21:00:09 +0000 (UTC) Received: from mail-qe0-x234.google.com (mail-qe0-x234.google.com [IPv6:2607:f8b0:400d:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0383F1826; Thu, 12 Dec 2013 21:00:08 +0000 (UTC) Received: by mail-qe0-f52.google.com with SMTP id ne12so841416qeb.25 for ; Thu, 12 Dec 2013 13:00:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=2yYdbF9xkk7v/AEjmDcWOK4ei+7eeguhfRBHug4CTEo=; b=gC1xKX+EMWuDoK/UGR5tBsVfb4A2zyz6t3QrEh0Gydg5vhNBzpJRYnVKc8VTUpEPXQ tizAn2peiFUljBgHxFd+1XFzDXARNlPsfaiDzf9+3PITTxvYspaPjFVFhTWKsHe8JwwY YZY5j/375/jOGdfhM13kU65UXfaj0NZ8xk18JuWy120szjc8pGYIY0E9GMKZIBk1/jqq WIrwjmwkV94pM5kNtn4LB4XfCmin5wmecof9CIpsUKhZF2Y0URaojK/At4R2YSP2PZCw Z+ao6dvlPiijEf5tXYJT1s2PYl916UGSdlmLhNfg9OIgNPGmNFG3EjnygI0pFfBiN8Iz Bf/A== MIME-Version: 1.0 X-Received: by 10.224.50.195 with SMTP id a3mr15980753qag.25.1386882008158; Thu, 12 Dec 2013 13:00:08 -0800 (PST) Received: by 10.140.34.17 with HTTP; Thu, 12 Dec 2013 13:00:08 -0800 (PST) In-Reply-To: <201312121511.38608.jhb@freebsd.org> References: <201312121511.38608.jhb@freebsd.org> Date: Thu, 12 Dec 2013 13:00:08 -0800 Message-ID: Subject: Re: Panic starting a bhyve guest after resume From: Neel Natu To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Dec 2013 21:00:09 -0000 Hi John, On Thu, Dec 12, 2013 at 12:11 PM, John Baldwin wrote: > If I suspend and resume my laptop and then try to start a guest after the > resume, I get an odd panic. It generates a privileged instruction fault (in > kernel mode) for 'vmclear'. I've checked CR4 and it claims that VMXE is set. > I dont have any other ideas off the top of my head on what I should be poking > at? It looks like we read a bunch of MSRs in vmx_init(), but we don't write > to them, and all vmx_enable() does on each CPU is set VMXE in CR4 from what I > can tell. > It also does a "vmxon" on each logical cpu which may also need to be done after a resume. best Neel > -- > John Baldwin > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Fri Dec 13 01:42:47 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 974CDE4C for ; Fri, 13 Dec 2013 01:42:47 +0000 (UTC) Received: from mail-ob0-x22b.google.com (mail-ob0-x22b.google.com [IPv6:2607:f8b0:4003:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5F9081E53 for ; Fri, 13 Dec 2013 01:42:47 +0000 (UTC) Received: by mail-ob0-f171.google.com with SMTP id wp18so1337944obc.30 for ; Thu, 12 Dec 2013 17:42:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=yNYfukvhxjrQH+D3dXZk/VX069HmrdEoTtOGkkiLA3E=; b=SIUA/nvLVIr78Sojs9N7uqE3d3hUUtXD08wBmtNdnA2g9jWzqWUQyrSX+ZDJ2P/982 iwmfqhUKS+2QN4VsqagCSSRI0V5Gj53rnYh93I0W/jEz7tfQfmlTVqtxIaWgwqOLt4t1 UkWRmM9rGnK4LtHMuq1w3f1FoK3qIKAFOTxd+hkt/kxIp/ANy8sCXKJNR+QGGytp5drV HnB8yjRmXF/gBeYFNeiQwk7ki1gJbDf39EKRiUJBWSx54MocZh9zDiqsysTbYtVmeDWb 1cgmEhkllmplseR7Fb/1oDdsGmI41Gv17on1Sp82w9+sZ+sPASB2e7XmCkGpeIJepvkF k3oQ== MIME-Version: 1.0 X-Received: by 10.60.65.101 with SMTP id w5mr18897oes.0.1386898966567; Thu, 12 Dec 2013 17:42:46 -0800 (PST) Received: by 10.76.70.202 with HTTP; Thu, 12 Dec 2013 17:42:46 -0800 (PST) Date: Thu, 12 Dec 2013 20:42:46 -0500 Message-ID: Subject: Whats missing... From: Outback Dingo To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Dec 2013 01:42:47 -0000 taking the stock vm0 on 10.0 PRERELEASE svn r259280 Booting... 2 vCPUs requested but only 1 available FreeBSD detects 16 with threads.... uname -a FreeBSD bsd.com 10.0-PRERELEASE FreeBSD 10.0-PRERELEASE #0 r259280: Fri Dec 13 01:18:42 EST 2013 dingo@bsd.com:/usr/obj/usr/src/sys/GENERIC amd64 dmesg Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-PRERELEASE #0 r259280: Fri Dec 13 01:18:42 EST 2013 dingo@bsd.optimcloud.com:/usr/obj/usr/src/sys/GENERIC amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 CPU: Intel(R) Xeon(R) CPU X5560 @ 2.80GHz (2800.16-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x106a5 Family = 0x6 Model = 0x1a Stepping = 5 Features=0xbfebfbff Features2=0x9ce3bd AMD Features=0x28100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 38654705664 (36864 MB) avail memory = 37461794816 (35726 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: <111610 APIC1528> FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 SMT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 cpu8 (AP): APIC ID: 16 cpu9 (AP): APIC ID: 17 cpu10 (AP): APIC ID: 18 cpu11 (AP): APIC ID: 19 cpu12 (AP): APIC ID: 20 cpu13 (AP): APIC ID: 21 cpu14 (AP): APIC ID: 22 cpu15 (AP): APIC ID: 23 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard random: initialized kbd1 at kbdmux0 acpi0: on motherboard acpi0: Overriding SCI from IRQ 9 to IRQ 20 acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, bff00000 (3) failed cpu0: on acpi0 ACPI BIOS Warning (bug): Incorrect checksum in table [OEMB] - 0x8D, should be 0x8A (20130823/tbprint-233) cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 cpu8: on acpi0 cpu9: on acpi0 cpu10: on acpi0 cpu11: on acpi0 cpu12: on acpi0 cpu13: on acpi0 cpu14: on acpi0 cpu15: on acpi0 attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 igb0: port 0xe880-0xe89f mem 0xfbc20000-0xfbc3ffff,0xfbc00000-0xfbc1ffff,0xfbbdc000-0xfbbdffff irq 28 at device 0.0 on pci1 igb0: Using MSIX interrupts with 9 vectors igb0: Ethernet address: 00:30:48:f2:4f:50 igb0: Bound queue 0 to cpu 0 igb0: Bound queue 1 to cpu 1 igb0: Bound queue 2 to cpu 2 igb0: Bound queue 3 to cpu 3 igb0: Bound queue 4 to cpu 4 igb0: Bound queue 5 to cpu 5 igb0: Bound queue 6 to cpu 6 igb0: Bound queue 7 to cpu 7 igb1: port 0xec00-0xec1f mem 0xfbce0000-0xfbcfffff,0xfbcc0000-0xfbcdffff,0xfbc9c000-0xfbc9ffff irq 40 at device 0.1 on pci1 igb1: Using MSIX interrupts with 9 vectors igb1: Ethernet address: 00:30:48:f2:4f:51 igb1: Bound queue 0 to cpu 8 igb1: Bound queue 1 to cpu 9 igb1: Bound queue 2 to cpu 10 igb1: Bound queue 3 to cpu 11 igb1: Bound queue 4 to cpu 12 igb1: Bound queue 5 to cpu 13 igb1: Bound queue 6 to cpu 14 igb1: Bound queue 7 to cpu 15 pcib2: at device 3.0 on pci0 pci3: on pcib2 pci3: at device 0.0 (no driver attached) pcib3: at device 5.0 on pci0 pci4: on pcib3 pcib4: at device 7.0 on pci0 pci5: on pcib4 pci0: at device 20.0 (no driver attached) pci0: at device 20.1 (no driver attached) pci0: at device 20.2 (no driver attached) pci0: at device 20.3 (no driver attached) pci0: at device 22.0 (no driver attached) pci0: at device 22.1 (no driver attached) pci0: at device 22.2 (no driver attached) pci0: at device 22.3 (no driver attached) pci0: at device 22.4 (no driver attached) pci0: at device 22.5 (no driver attached) pci0: at device 22.6 (no driver attached) pci0: at device 22.7 (no driver attached) uhci0: port 0xc880-0xc89f irq 16 at device 26.0 on pci0 uhci0: LegSup = 0x2f00 usbus0 on uhci0 uhci1: port 0xcc00-0xcc1f irq 21 at device 26.1 on pci0 uhci1: LegSup = 0x2f00 usbus1 on uhci1 uhci2: port 0xd000-0xd01f irq 19 at device 26.2 on pci0 uhci2: LegSup = 0x2f00 usbus2 on uhci2 ehci0: mem 0xfbed6000-0xfbed63ff irq 18 at device 26.7 on pci0 usbus3: EHCI version 1.0 usbus3 on ehci0 uhci3: port 0xc400-0xc41f irq 23 at device 29.0 on pci0 uhci3: LegSup = 0x2f00 usbus4 on uhci3 uhci4: port 0xc480-0xc49f irq 19 at device 29.1 on pci0 uhci4: LegSup = 0x2f00 usbus5 on uhci4 uhci5: port 0xc800-0xc81f irq 18 at device 29.2 on pci0 uhci5: LegSup = 0x2f00 usbus6 on uhci5 ehci1: mem 0xfbed4000-0xfbed43ff irq 23 at device 29.7 on pci0 usbus7: EHCI version 1.0 usbus7 on ehci1 pcib5: at device 30.0 on pci0 pci6: on pcib5 vgapci0: mem 0xf9000000-0xf9ffffff,0xfaffc000-0xfaffffff,0xfb000000-0xfb7fffff irq 18 at device 1.0 on pci6 isab0: at device 31.0 on pci0 isa0: on isab0 ahci0: port 0xd400-0xd407,0xdc00-0xdc03,0xd880-0xd887,0xd800-0xd803,0xd480-0xd49f mem 0xfbefa000-0xfbefa7ff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.20 with 6 3Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahcich4: at channel 4 on ahci0 ahcich5: at channel 5 on ahci0 ahciem0: on ahci0 pci0: at device 31.3 (no driver attached) acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 qpi0: on motherboard pcib6: pcibus 255 on qpi0 pci255: on pcib6 pcib7: pcibus 254 on qpi0 pci254: on pcib7 orm0: at iomem 0xc0000-0xc7fff,0xcb000-0xcbfff,0xcc000-0xccfff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] ppc0: cannot reserve I/O port range est0: on cpu0 p4tcc0: on cpu0 est1: on cpu1 p4tcc1: on cpu1 est2: on cpu2 p4tcc2: on cpu2 est3: on cpu3 p4tcc3: on cpu3 est4: on cpu4 p4tcc4: on cpu4 est5: on cpu5 p4tcc5: on cpu5 est6: on cpu6 p4tcc6: on cpu6 est7: on cpu7 p4tcc7: on cpu7 est8: on cpu8 p4tcc8: on cpu8 est9: on cpu9 p4tcc9: on cpu9 est10: on cpu10 p4tcc10: on cpu10 est11: on cpu11 p4tcc11: on cpu11 est12: on cpu12 p4tcc12: on cpu12 est13: on cpu13 p4tcc13: on cpu13 est14: on cpu14 p4tcc14: on cpu14 est15: on cpu15 p4tcc15: on cpu15 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec random: unblocking device. usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 usbus4: 12Mbps Full Speed USB v1.0 usbus5: 12Mbps Full Speed USB v1.0 usbus6: 12Mbps Full Speed USB v1.0 usbus7: 480Mbps High Speed USB v2.0 ugen0.1: at usbus0 uhub0: on usbus0 ugen2.1: at usbus2 uhub1: on usbus2 ugen1.1: at usbus1 uhub2: on usbus1 ugen4.1: at usbus4 uhub3: on usbus4 ugen3.1: at usbus3 uhub4: on usbus3 ugen7.1: at usbus7 uhub5: on usbus7 ugen6.1: at usbus6 uhub6: on usbus6 ugen5.1: at usbus5 uhub7: on usbus5 ses0 at ahciem0 bus 0 scbus6 target 0 lun 0 ses0: SEMB S-E-S 2.00 device ses0: SEMB SES Device ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: ATA-8 SATA 2.x device ada0: Serial Number WD-WXD0CB9C3566 ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 286168MB (586072368 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad4 Netvsc initializing... SMP: AP CPU #1 Launched! SMP: AP CPU #14 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #10 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #9 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #15 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #12 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #11 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #8 Launched! SMP: AP CPU #13 Launched! Timecounter "TSC-low" frequency 1400078562 Hz quality 1000 Root mount waiting for: usbus7 usbus6 usbus5 usbus4 usbus3 usbus2 usbus1 usbus0 uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered uhub7: 2 ports with 2 removable, self powered uhub6: 2 ports with 2 removable, self powered Root mount waiting for: usbus7 usbus3 Root mount waiting for: usbus7 usbus3 uhub5: 6 ports with 6 removable, self powered uhub4: 6 ports with 6 removable, self powered Trying to mount root from zfs:zroot/ROOT/default []... ugen1.2: at usbus1 ukbd0: on usbus1 kbd2 at ukbd0 ums0: on usbus1 From owner-freebsd-virtualization@FreeBSD.ORG Fri Dec 13 01:55:36 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E723D27B for ; Fri, 13 Dec 2013 01:55:36 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id A8D301F35 for ; Fri, 13 Dec 2013 01:55:36 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 38D2411DF0; Fri, 13 Dec 2013 11:55:28 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-67-161-27-37.hsd1.ca.comcast.net [67.161.27.37]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BQS02822 (AUTH peterg@ptree32.com.au); Fri, 13 Dec 2013 11:55:27 +1000 Message-ID: <52AA6905.8040108@freebsd.org> Date: Thu, 12 Dec 2013 17:55:17 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: Outback Dingo Subject: Re: Whats missing... References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Dec 2013 01:55:37 -0000 Hi, > CPU: Intel(R) Xeon(R) CPU X5560 @ 2.80GHz (2800.16-MHz K8-class Same issue as http://docs.freebsd.org/cgi/mid.cgi?529579CD.9010509 later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Fri Dec 13 02:03:20 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CED0C606; Fri, 13 Dec 2013 02:03:20 +0000 (UTC) Received: from mail-ob0-x232.google.com (mail-ob0-x232.google.com [IPv6:2607:f8b0:4003:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 88A681FE3; Fri, 13 Dec 2013 02:03:20 +0000 (UTC) Received: by mail-ob0-f178.google.com with SMTP id uz6so1350878obc.9 for ; Thu, 12 Dec 2013 18:03:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=zkXGxkK/T77UROVASPhNm31r4KB//yxONidVKUTm7ko=; b=zsmSBujjYxiw21pAoa715usCp9Npu4MCI3o06ukVZH2/sNWhOSmUMpHamrbR5n1XxI 2s/2d4OGrf13N/GZfAH+PJ79cv4Qkf8Ud4TFA1c2NyMayDMiBh1CO8uf2SRvxuH8Cjkl gti+3n6pPlSiHcAO7QAX0oeNfLk0Mwnd5p2UamAvGSy345gV0LA/Gs0wjs05x3OsBKHD JLiGORg9WQ0MnrbWBU6ks2fgCcL6JsI20ClFjXCg/Aww9DFpuIeEI9U2v+U6rnwzuntL LPixPhWAoPl37niWl+ZO48YLlDHMmYrZlxmIb+RuuFeoDvks1HoYXgbY3A0JpDsFiPq3 qPNg== MIME-Version: 1.0 X-Received: by 10.182.103.133 with SMTP id fw5mr7900348obb.43.1386900199846; Thu, 12 Dec 2013 18:03:19 -0800 (PST) Received: by 10.76.70.202 with HTTP; Thu, 12 Dec 2013 18:03:19 -0800 (PST) In-Reply-To: <52AA6905.8040108@freebsd.org> References: <52AA6905.8040108@freebsd.org> Date: Thu, 12 Dec 2013 21:03:19 -0500 Message-ID: Subject: Re: Whats missing... From: Outback Dingo To: Peter Grehan Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Dec 2013 02:03:20 -0000 On Thu, Dec 12, 2013 at 8:55 PM, Peter Grehan wrote: > Hi, > > > CPU: Intel(R) Xeon(R) CPU X5560 @ 2.80GHz (2800.16-MHz K8-class >> > > Same issue as http://docs.freebsd.org/cgi/mid.cgi?529579CD.9010509 > > later, > > Peter. > Well damn..... guess its back to running XEN, which loads and runs fine on the box From owner-freebsd-virtualization@FreeBSD.ORG Fri Dec 13 10:43:28 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C9F56DEB; Fri, 13 Dec 2013 10:43:28 +0000 (UTC) Received: from mail-wi0-x236.google.com (mail-wi0-x236.google.com [IPv6:2a00:1450:400c:c05::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0D8CF1AC8; Fri, 13 Dec 2013 10:43:27 +0000 (UTC) Received: by mail-wi0-f182.google.com with SMTP id en1so866233wid.3 for ; Fri, 13 Dec 2013 02:43:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=qLuGjFjIbTBmrmCLhz1P/xOv4g9bl283RuDDVYt5XmA=; b=C8+rGrdNFCNPGwtS1JWocpDvpUBOYVVeWu/2G4SAhc3Y/2IHyjnGjkxpvL4XZuTVQ2 CWcAJ8bhRs2x9qolYzP76Y3Y+4sHMJSMObQOaVQv/HWzcoJ47vKH2yaHlCYM/YZlAr2+ 1umfMsTN8B1JUlKhqhKgHckaXZzznhZG8q9tITdqdXjiZz2l3gyzmSsMYrcK8iwKv5Xr 9Kgga8ePC+TGOaIXJU/kzea7tR4kSUBg/BFSe+mN3aX+RpSxp0Ni9GUGy8tXPsRnq4QK qI34DV+c+Khiljs7WYb4kn3MZZYaW+/yQvlB+ulyLloxCcAIef5IA2Wfj8/E4/psKavA r7JA== MIME-Version: 1.0 X-Received: by 10.194.237.99 with SMTP id vb3mr1700916wjc.28.1386931406475; Fri, 13 Dec 2013 02:43:26 -0800 (PST) Received: by 10.227.226.12 with HTTP; Fri, 13 Dec 2013 02:43:26 -0800 (PST) Date: Fri, 13 Dec 2013 12:43:26 +0200 Message-ID: Subject: From: Markiyan Kushnir To: freebsd-current@freebsd.org, freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Dec 2013 10:43:29 -0000 I started some ports to compile inside a bhyve instance: root@vm:~ # uname -a FreeBSD vm.mkushnir.mooo.com 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r259250: Thu Dec 12 14:17:32 EET 2013 root@vm.mkushnir.zapto.org:/ usr/obj/usr/src.svnup/sys/MAREK amd64 and left it running unattended. Approx. 2 hours later the host went to panic. The bhyve instance survived after the panic and I could be able to complete my ports compilation. core.txt attached (gzipped) From owner-freebsd-virtualization@FreeBSD.ORG Fri Dec 13 10:46:23 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CF9C3DC; Fri, 13 Dec 2013 10:46:23 +0000 (UTC) Received: from mail-wg0-x229.google.com (mail-wg0-x229.google.com [IPv6:2a00:1450:400c:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 176DF1AF0; Fri, 13 Dec 2013 10:46:22 +0000 (UTC) Received: by mail-wg0-f41.google.com with SMTP id y10so1753697wgg.4 for ; Fri, 13 Dec 2013 02:46:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=77kT1YahZhEcw43ht1E+/iHYaI/Tl6CHeJ61lGyax0Q=; b=gj+UUT9jab/cyc/OpHRsO+wD3P7xpsI3DNUfMXZqJHEPI0zFqB73wFkeGHuUUWM+mI Gh/lyaAw29bEB9Znm4NgrNGlXnL+hrBTRQNwxEqv8/z5WiJkbTprtMm3sA5Ww1ykXlDN LH62KGsf/OW+DfkJTt9muZpwWQHu/Er7DK99WzJymujEOKAD0iz9sCw5lTA8IY6N9V2F wTpDsHkvgCGgZMyzP3hfWoDFTo2iEWcFpSKCyFz3Cw+ZMxStS/tUqdcfE4yDVz4egv6W fzUvnPqcgr0k8pDWX3V1LuczSmjwwHTLQiMmaWgHGa/R8KSJBnp9wsX8Knotku+o0XVW F34Q== MIME-Version: 1.0 X-Received: by 10.180.86.198 with SMTP id r6mr2246075wiz.27.1386931580718; Fri, 13 Dec 2013 02:46:20 -0800 (PST) Received: by 10.227.226.12 with HTTP; Fri, 13 Dec 2013 02:46:20 -0800 (PST) Date: Fri, 13 Dec 2013 12:46:20 +0200 Message-ID: Subject: 11.0-CURRENT panic while running a bhyve instance From: Markiyan Kushnir To: freebsd-current@freebsd.org, freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Dec 2013 10:46:23 -0000 Forgot to fill the Subject: header, re-posting it fixed. -- Markiyan ---------- Forwarded message ---------- From: Markiyan Kushnir Date: 2013/12/13 Subject: To: freebsd-current@freebsd.org, freebsd-virtualization@freebsd.org I started some ports to compile inside a bhyve instance: root@vm:~ # uname -a FreeBSD vm.mkushnir.mooo.com 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r259250: Thu Dec 12 14:17:32 EET 2013 root@vm.mkushnir.zapto.org:/ usr/obj/usr/src.svnup/sys/MAREK amd64 and left it running unattended. Approx. 2 hours later the host went to panic. The bhyve instance survived after the panic and I could be able to complete my ports compilation. core.txt attached (gzipped) From owner-freebsd-virtualization@FreeBSD.ORG Fri Dec 13 20:57:42 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 209A2B61; Fri, 13 Dec 2013 20:57:42 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id ECCEA1CC8; Fri, 13 Dec 2013 20:57:41 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id DD75AB9B9; Fri, 13 Dec 2013 15:57:40 -0500 (EST) From: John Baldwin To: freebsd-current@freebsd.org Subject: Re: 11.0-CURRENT panic while running a bhyve instance Date: Fri, 13 Dec 2013 13:04:57 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201312131304.57455.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 13 Dec 2013 15:57:41 -0500 (EST) Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Dec 2013 20:57:42 -0000 On Friday, December 13, 2013 5:46:20 am Markiyan Kushnir wrote: > Forgot to fill the Subject: header, re-posting it fixed. The mailing lists strips attachments, can you post it at a URL? > -- > Markiyan > > > ---------- Forwarded message ---------- > From: Markiyan Kushnir > Date: 2013/12/13 > Subject: > To: freebsd-current@freebsd.org, freebsd-virtualization@freebsd.org > > > I started some ports to compile inside a bhyve instance: > > root@vm:~ # uname -a > FreeBSD vm.mkushnir.mooo.com 11.0-CURRENT FreeBSD 11.0-CURRENT #0 > r259250: Thu Dec 12 14:17:32 EET 2013 > root@vm.mkushnir.zapto.org:/ > usr/obj/usr/src.svnup/sys/MAREK amd64 > > and left it running unattended. Approx. 2 hours later the host went to > panic. The bhyve instance survived after the panic and I could be able > to complete my ports compilation. > > core.txt attached (gzipped) > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > -- John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Fri Dec 13 22:17:35 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 81D0AE7 for ; Fri, 13 Dec 2013 22:17:35 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5B08F130A for ; Fri, 13 Dec 2013 22:17:35 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 4AC2EB9DD; Fri, 13 Dec 2013 17:17:34 -0500 (EST) From: John Baldwin To: Neel Natu Subject: Re: Panic starting a bhyve guest after resume Date: Fri, 13 Dec 2013 17:09:20 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <201312121511.38608.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201312131709.20264.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 13 Dec 2013 17:17:34 -0500 (EST) Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Dec 2013 22:17:35 -0000 On Thursday, December 12, 2013 4:00:08 pm Neel Natu wrote: > Hi John, > > On Thu, Dec 12, 2013 at 12:11 PM, John Baldwin wrote: > > If I suspend and resume my laptop and then try to start a guest after the > > resume, I get an odd panic. It generates a privileged instruction fault (in > > kernel mode) for 'vmclear'. I've checked CR4 and it claims that VMXE is set. > > I dont have any other ideas off the top of my head on what I should be poking > > at? It looks like we read a bunch of MSRs in vmx_init(), but we don't write > > to them, and all vmx_enable() does on each CPU is set VMXE in CR4 from what I > > can tell. > > > > It also does a "vmxon" on each logical cpu which may also need to be > done after a resume. Ah, yes it does. That was sufficient both for starting a new guest after resume and even doing a suspend/resume while a guest was active (and the guest continued to run fine). I have a hacky patch for this. One, it includes both a suspend and resume hook for VMM, though for my testing I only needed a resume hook to invoke vmxon. Second, the name of vmx_resume2() is a total hack (because vmx_resume() was already taken. I think for now if I were to commit this, I'd just add the resme hook and maybe call the Intel method vmx_reset() or vmx_restore()? http://people.freebsd.org/~jhb/patches/bhyve_resume.patch -- John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Fri Dec 13 22:35:56 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 801506A7; Fri, 13 Dec 2013 22:35:56 +0000 (UTC) Received: from mail-wi0-x234.google.com (mail-wi0-x234.google.com [IPv6:2a00:1450:400c:c05::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C3E15151D; Fri, 13 Dec 2013 22:35:55 +0000 (UTC) Received: by mail-wi0-f180.google.com with SMTP id hn9so1737828wib.7 for ; Fri, 13 Dec 2013 14:35:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=cl91Au8B2XzTIieWPImQlyvmK7i1TtXQFja6dr738lE=; b=iSUKWAr5aNsk0hPHWa4E9BgNjJyRsoCillWLnmnZ/c1ScPC33xX52UfX01x3qFolYi H0rioT24DpQ5r0P+jutZ3cjNCpIyE1ouaMrumEWYN9j7q4WphmKOcMVwTz3bRuvTDccF Sl6akBq+9KzSDgsdf25TRTkLk8sMRCm7NjhckLtLyFV8iMv0+OfDdC3r98yKUxClTGQg MJH88PdJ3ZxDir3WxQ9bD+6JTDX0YE+MBnYNKIWLF0593/2wBuriHxYOZoev/blt52g4 mrLbHBptKRaERvdSxS4CsbJv/aZDgNgJ3TByVieL5BbhyK5JsX7BZvXomxndE0mFIG3n I0HQ== MIME-Version: 1.0 X-Received: by 10.194.59.240 with SMTP id c16mr4192951wjr.13.1386974153983; Fri, 13 Dec 2013 14:35:53 -0800 (PST) Received: by 10.227.226.12 with HTTP; Fri, 13 Dec 2013 14:35:53 -0800 (PST) In-Reply-To: <201312131304.57455.jhb@freebsd.org> References: <201312131304.57455.jhb@freebsd.org> Date: Sat, 14 Dec 2013 00:35:53 +0200 Message-ID: Subject: Re: 11.0-CURRENT panic while running a bhyve instance From: Markiyan Kushnir To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-current@freebsd.org, freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Dec 2013 22:35:56 -0000 2013/12/13 John Baldwin : > On Friday, December 13, 2013 5:46:20 am Markiyan Kushnir wrote: >> Forgot to fill the Subject: header, re-posting it fixed. > > The mailing lists strips attachments, can you post it at a URL? > Shared here: https://drive.google.com/file/d/0B9Q-zpUXxqCnem5iYTVqLUxrcWo4cmlhdkM1c2lJa2dKak5R/edit?usp=sharing -- Markiyan. >> -- >> Markiyan >> >> >> ---------- Forwarded message ---------- >> From: Markiyan Kushnir >> Date: 2013/12/13 >> Subject: >> To: freebsd-current@freebsd.org, freebsd-virtualization@freebsd.org >> >> >> I started some ports to compile inside a bhyve instance: >> >> root@vm:~ # uname -a >> FreeBSD vm.mkushnir.mooo.com 11.0-CURRENT FreeBSD 11.0-CURRENT #0 >> r259250: Thu Dec 12 14:17:32 EET 2013 >> root@vm.mkushnir.zapto.org:/ >> usr/obj/usr/src.svnup/sys/MAREK amd64 >> >> and left it running unattended. Approx. 2 hours later the host went to >> panic. The bhyve instance survived after the panic and I could be able >> to complete my ports compilation. >> >> core.txt attached (gzipped) >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >> > > -- > John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Sat Dec 14 00:16:07 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7D685EFD; Sat, 14 Dec 2013 00:16:07 +0000 (UTC) Received: from mail-qe0-x22a.google.com (mail-qe0-x22a.google.com [IPv6:2607:f8b0:400d:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 189B31E34; Sat, 14 Dec 2013 00:16:07 +0000 (UTC) Received: by mail-qe0-f42.google.com with SMTP id b4so2257800qen.1 for ; Fri, 13 Dec 2013 16:16:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=YwOd22Saon+Pot6MFTTXE9IxGf1ih78H//T5I6EgR9c=; b=DvAOpbwq99SxYFE/VMgwxQNfdQHb9mm2kB91BdJb8arD+ZqLlMo1ggRZqkOZPNyXDF rNdf5nhquotBT0KnL4tujhDcwr597HYsIcp1HB4rfmU0N01XLt4jzKmZPN7u+i1CzGeL FNHpQcpf+G7jiLgRP2f3eM67ovR9Sn5t/PmSb59QRlq0EeKxghxHtdkMu325uc2s/EjI LQJ2FnSZ3xTKQow6rJJuWqpw+UtPLEB+fLP0M+EqdvLOo1UlX1TnIKW6bujI5WW0irMr jk0XAGahkERSQDC//sDz492T3KYDdgtmQr+phQMJ98jD2zxNZblfGOGUSBUotro3HsuR jsqw== MIME-Version: 1.0 X-Received: by 10.49.132.65 with SMTP id os1mr9533290qeb.39.1386980166278; Fri, 13 Dec 2013 16:16:06 -0800 (PST) Received: by 10.140.34.17 with HTTP; Fri, 13 Dec 2013 16:16:06 -0800 (PST) In-Reply-To: References: <201312131304.57455.jhb@freebsd.org> Date: Fri, 13 Dec 2013 16:16:06 -0800 Message-ID: Subject: Re: 11.0-CURRENT panic while running a bhyve instance From: Neel Natu To: Markiyan Kushnir Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD current , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Dec 2013 00:16:07 -0000 Hi Markiyan, On Fri, Dec 13, 2013 at 2:35 PM, Markiyan Kushnir wrote: > 2013/12/13 John Baldwin : >> On Friday, December 13, 2013 5:46:20 am Markiyan Kushnir wrote: >>> Forgot to fill the Subject: header, re-posting it fixed. >> >> The mailing lists strips attachments, can you post it at a URL? >> > > Shared here: > > https://drive.google.com/file/d/0B9Q-zpUXxqCnem5iYTVqLUxrcWo4cmlhdkM1c2lJa2dKak5R/edit?usp=sharing > Thanks. It looks like something funky going on with the vcpu state. Do you know if there was any access to the VM via 'bhyvectl' close to the time of the panic? best Neel > -- > Markiyan. > >>> -- >>> Markiyan >>> >>> >>> ---------- Forwarded message ---------- >>> From: Markiyan Kushnir >>> Date: 2013/12/13 >>> Subject: >>> To: freebsd-current@freebsd.org, freebsd-virtualization@freebsd.org >>> >>> >>> I started some ports to compile inside a bhyve instance: >>> >>> root@vm:~ # uname -a >>> FreeBSD vm.mkushnir.mooo.com 11.0-CURRENT FreeBSD 11.0-CURRENT #0 >>> r259250: Thu Dec 12 14:17:32 EET 2013 >>> root@vm.mkushnir.zapto.org:/ >>> usr/obj/usr/src.svnup/sys/MAREK amd64 >>> >>> and left it running unattended. Approx. 2 hours later the host went to >>> panic. The bhyve instance survived after the panic and I could be able >>> to complete my ports compilation. >>> >>> core.txt attached (gzipped) >>> _______________________________________________ >>> freebsd-current@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >>> >> >> -- >> John Baldwin > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Sat Dec 14 02:28:30 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7E739DA3; Sat, 14 Dec 2013 02:28:30 +0000 (UTC) Received: from mail-qa0-x232.google.com (mail-qa0-x232.google.com [IPv6:2607:f8b0:400d:c00::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2C8C816DE; Sat, 14 Dec 2013 02:28:30 +0000 (UTC) Received: by mail-qa0-f50.google.com with SMTP id i13so57824qae.9 for ; Fri, 13 Dec 2013 18:28:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ZTPGNLC0I4wgGSG7YksE+vqow+biGJcDo/vc6DE85h8=; b=TviDRcZz2w+oNB8q2guLg8vt0C4BVolELj0DnBwKjCkg9VKoGafbYUZdTUg4KKDSSS i+VSAn4IISqzQZsYzL5HhvMZXZm3gaufpTb+/6AbIGxpICPue6y5s59UhimbnFWtcRGN fnwl5Vf4m6sFwAavd886Sgs2RQaP+hyw80t8LTyv4gn7g3amgLuQk5IvPv1y04D14qHN riwwnsC2TZYC/RBdzn5B9m96qeglE5U6ilkBneiDsdZsxiXthOOQc9M/soUSFQm+7PDP SY/kBld9v7XDjDoCAZx//3GTg977Sbt8UlOS92X4okJETB5K265f9bBHO5OdXQPdMT5P ZFvA== MIME-Version: 1.0 X-Received: by 10.224.120.71 with SMTP id c7mr10213189qar.37.1386988109416; Fri, 13 Dec 2013 18:28:29 -0800 (PST) Received: by 10.140.34.17 with HTTP; Fri, 13 Dec 2013 18:28:29 -0800 (PST) In-Reply-To: <201312131709.20264.jhb@freebsd.org> References: <201312121511.38608.jhb@freebsd.org> <201312131709.20264.jhb@freebsd.org> Date: Fri, 13 Dec 2013 18:28:29 -0800 Message-ID: Subject: Re: Panic starting a bhyve guest after resume From: Neel Natu To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Dec 2013 02:28:30 -0000 Hi John, On Fri, Dec 13, 2013 at 2:09 PM, John Baldwin wrote: > On Thursday, December 12, 2013 4:00:08 pm Neel Natu wrote: >> Hi John, >> >> On Thu, Dec 12, 2013 at 12:11 PM, John Baldwin wrote: >> > If I suspend and resume my laptop and then try to start a guest after the >> > resume, I get an odd panic. It generates a privileged instruction fault (in >> > kernel mode) for 'vmclear'. I've checked CR4 and it claims that VMXE is set. >> > I dont have any other ideas off the top of my head on what I should be poking >> > at? It looks like we read a bunch of MSRs in vmx_init(), but we don't write >> > to them, and all vmx_enable() does on each CPU is set VMXE in CR4 from what I >> > can tell. >> > >> >> It also does a "vmxon" on each logical cpu which may also need to be >> done after a resume. > > Ah, yes it does. That was sufficient both for starting a new guest after > resume and even doing a suspend/resume while a guest was active (and the > guest continued to run fine). I have a hacky patch for this. One, it > includes both a suspend and resume hook for VMM, though for my testing I only > needed a resume hook to invoke vmxon. Second, the name of vmx_resume2() > is a total hack (because vmx_resume() was already taken. I think for now > if I were to commit this, I'd just add the resme hook and maybe call the > Intel method vmx_reset() or vmx_restore()? > > http://people.freebsd.org/~jhb/patches/bhyve_resume.patch > There seems to be a race after the APs are restarted and before 'vmm_resume_p()' where it would be problematic to execute a VMX instruction. Perhaps we should enable VMX on each cpu before they return to the interrupted code? best Neel > -- > John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Sat Dec 14 07:57:43 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A6880108; Sat, 14 Dec 2013 07:57:43 +0000 (UTC) Received: from mail-we0-x22c.google.com (mail-we0-x22c.google.com [IPv6:2a00:1450:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E8B9B1016; Sat, 14 Dec 2013 07:57:42 +0000 (UTC) Received: by mail-we0-f172.google.com with SMTP id w62so2805398wes.17 for ; Fri, 13 Dec 2013 23:57:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=fYQQpQ+Q8XWRgbhR12G/jW+bbJGwvYBJCD8JpWSjrF0=; b=j5BiRoPOrqJvUfR64KaZyRNS9ElZysIJX3YpnyjHE9pBrk8iEpTCH+DMmSro+YGK82 tBGDwFXB5q23F6w0oVon/aznchkS56amheGS66YKgFgfZyEcW5/N3Gr5KtjJ97ZWoOKM qo6Mt2jBMRX9rpQ8BlP+dPClldECy5+K+t3LrezidA0e7A07RSEfLtFXncCJ4roCfIIw xNNrJkbAqAQX/GFcL32eJ2k9z07j0NKY/vtpDrhn0Iv7mDpBqdoAv2q5tIyzlr2T3e4O EwvUVDdvovSPS5+6QJiCCivjPG1kA5j/JaUl/u/soQAKsXEKumX6Mp9i7Dg9Y7nHm8Db c6mQ== MIME-Version: 1.0 X-Received: by 10.180.107.168 with SMTP id hd8mr5463818wib.32.1387007861459; Fri, 13 Dec 2013 23:57:41 -0800 (PST) Received: by 10.227.226.12 with HTTP; Fri, 13 Dec 2013 23:57:41 -0800 (PST) In-Reply-To: References: <201312131304.57455.jhb@freebsd.org> Date: Sat, 14 Dec 2013 09:57:41 +0200 Message-ID: Subject: Re: 11.0-CURRENT panic while running a bhyve instance From: Markiyan Kushnir To: Neel Natu Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD current , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Dec 2013 07:57:43 -0000 2013/12/14 Neel Natu : > Hi Markiyan, > > On Fri, Dec 13, 2013 at 2:35 PM, Markiyan Kushnir > wrote: >> 2013/12/13 John Baldwin : >>> On Friday, December 13, 2013 5:46:20 am Markiyan Kushnir wrote: >>>> Forgot to fill the Subject: header, re-posting it fixed. >>> >>> The mailing lists strips attachments, can you post it at a URL? >>> >> >> Shared here: >> >> https://drive.google.com/file/d/0B9Q-zpUXxqCnem5iYTVqLUxrcWo4cmlhdkM1c2lJa2dKak5R/edit?usp=sharing >> > > Thanks. > > It looks like something funky going on with the vcpu state. Do you > know if there was any access to the VM via 'bhyvectl' close to the > time of the panic? > Well, I don't know if there was. I would set up the same scenario again + a script running on the host querying bhyvectl. May be I would catch it again. Please let me know if all it makes sense, and if so, how it can be made better. -- Markiyan. > best > Neel > >> -- >> Markiyan. >> >>>> -- >>>> Markiyan >>>> >>>> >>>> ---------- Forwarded message ---------- >>>> From: Markiyan Kushnir >>>> Date: 2013/12/13 >>>> Subject: >>>> To: freebsd-current@freebsd.org, freebsd-virtualization@freebsd.org >>>> >>>> >>>> I started some ports to compile inside a bhyve instance: >>>> >>>> root@vm:~ # uname -a >>>> FreeBSD vm.mkushnir.mooo.com 11.0-CURRENT FreeBSD 11.0-CURRENT #0 >>>> r259250: Thu Dec 12 14:17:32 EET 2013 >>>> root@vm.mkushnir.zapto.org:/ >>>> usr/obj/usr/src.svnup/sys/MAREK amd64 >>>> >>>> and left it running unattended. Approx. 2 hours later the host went to >>>> panic. The bhyve instance survived after the panic and I could be able >>>> to complete my ports compilation. >>>> >>>> core.txt attached (gzipped) >>>> _______________________________________________ >>>> freebsd-current@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >>>> >>> >>> -- >>> John Baldwin >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Sat Dec 14 08:17:12 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1392441F; Sat, 14 Dec 2013 08:17:12 +0000 (UTC) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 146F31124; Sat, 14 Dec 2013 08:17:10 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.95,484,1384300800"; d="scan'208";a="82241502" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 14 Dec 2013 08:15:45 +0000 Received: from [IPv6:::1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.78) with Microsoft SMTP Server id 14.2.342.4; Sat, 14 Dec 2013 03:15:44 -0500 Message-ID: <52AC13B1.8060402@citrix.com> Date: Sat, 14 Dec 2013 09:15:45 +0100 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Neel Natu , John Baldwin Subject: Re: Panic starting a bhyve guest after resume References: <201312121511.38608.jhb@freebsd.org> <201312131709.20264.jhb@freebsd.org> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit X-DLP: MIA1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Dec 2013 08:17:12 -0000 On 14/12/13 03:28, Neel Natu wrote: > Hi John, > > On Fri, Dec 13, 2013 at 2:09 PM, John Baldwin wrote: >> On Thursday, December 12, 2013 4:00:08 pm Neel Natu wrote: >>> Hi John, >>> >>> On Thu, Dec 12, 2013 at 12:11 PM, John Baldwin wrote: >>>> If I suspend and resume my laptop and then try to start a guest after the >>>> resume, I get an odd panic. It generates a privileged instruction fault (in >>>> kernel mode) for 'vmclear'. I've checked CR4 and it claims that VMXE is set. >>>> I dont have any other ideas off the top of my head on what I should be poking >>>> at? It looks like we read a bunch of MSRs in vmx_init(), but we don't write >>>> to them, and all vmx_enable() does on each CPU is set VMXE in CR4 from what I >>>> can tell. >>>> >>> >>> It also does a "vmxon" on each logical cpu which may also need to be >>> done after a resume. >> >> Ah, yes it does. That was sufficient both for starting a new guest after >> resume and even doing a suspend/resume while a guest was active (and the >> guest continued to run fine). I have a hacky patch for this. One, it >> includes both a suspend and resume hook for VMM, though for my testing I only >> needed a resume hook to invoke vmxon. Second, the name of vmx_resume2() >> is a total hack (because vmx_resume() was already taken. I think for now >> if I were to commit this, I'd just add the resme hook and maybe call the >> Intel method vmx_reset() or vmx_restore()? >> >> http://people.freebsd.org/~jhb/patches/bhyve_resume.patch >> > > There seems to be a race after the APs are restarted and before > 'vmm_resume_p()' where it would be problematic to execute a VMX > instruction. > > Perhaps we should enable VMX on each cpu before they return to the > interrupted code? Can you use the hook in cpususpend_handler? It's cpu_ops.cpu_resume, and gets called on each CPU before returning from the handler. From owner-freebsd-virtualization@FreeBSD.ORG Sat Dec 14 13:30:50 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EFC31571; Sat, 14 Dec 2013 13:30:49 +0000 (UTC) Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3D16512B4; Sat, 14 Dec 2013 13:30:49 +0000 (UTC) Received: by mail-wg0-f44.google.com with SMTP id a1so2974092wgh.35 for ; Sat, 14 Dec 2013 05:30:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=DM7rs4+MN2QAlgey5h4v24nwCfz9+FCCplBhHg3ke5k=; b=QCT9VcdB6SVz824ImmZX++DuEF2oLWGqIEPRvA/7rs1omi9ZqAs8T6eee3CE18FKm3 0+DUCR7h2wYc/M1MlTL3JRPrNw5wTXdJZh+5DmFiBpwa1hyub6a2owSMr9WTYLJcuJPC 3z8ENwBCZ2JviSrXJciud9exznqORa6d1Lesp1s75hX3aF9kJsvIMq6J8Raj1GUb9QHI z4YCQnptO+ZTaqGd08PPS62nKgiU7q7jYYaPLlq41OnPH2yeBwwjHyKlpRiBgcw8FrLW gQ+OdtaR2zvoVOv1s6QwUUgHm4VRFRyCdm4vDM8P49nO4/rVBTzo+tLiRkTiFvAkq3X7 u8Ug== MIME-Version: 1.0 X-Received: by 10.194.189.42 with SMTP id gf10mr6146083wjc.24.1387027847652; Sat, 14 Dec 2013 05:30:47 -0800 (PST) Received: by 10.227.226.12 with HTTP; Sat, 14 Dec 2013 05:30:47 -0800 (PST) In-Reply-To: References: <201312131304.57455.jhb@freebsd.org> Date: Sat, 14 Dec 2013 15:30:47 +0200 Message-ID: Subject: Re: 11.0-CURRENT panic while running a bhyve instance From: Markiyan Kushnir To: Neel Natu Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD current , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Dec 2013 13:30:50 -0000 2013/12/14 Markiyan Kushnir : > 2013/12/14 Neel Natu : >> Hi Markiyan, >> >> On Fri, Dec 13, 2013 at 2:35 PM, Markiyan Kushnir >> wrote: >>> 2013/12/13 John Baldwin : >>>> On Friday, December 13, 2013 5:46:20 am Markiyan Kushnir wrote: >>>>> Forgot to fill the Subject: header, re-posting it fixed. >>>> >>>> The mailing lists strips attachments, can you post it at a URL? >>>> >>> >>> Shared here: >>> >>> https://drive.google.com/file/d/0B9Q-zpUXxqCnem5iYTVqLUxrcWo4cmlhdkM1c2lJa2dKak5R/edit?usp=sharing >>> >> >> Thanks. >> >> It looks like something funky going on with the vcpu state. Do you >> know if there was any access to the VM via 'bhyvectl' close to the >> time of the panic? >> > > Well, I don't know if there was. I would set up the same scenario > again + a script running on the host querying bhyvectl. May be I would > catch it again. Please let me know if all it makes sense, and if so, > how it can be made better. > To make it clear -- I didn't run bhyvectl when the VM was running, so I can tell for sure that nobody was trying to interact with the VM during its run. My first thought was that it would make sense to (periodically) query state of the VM using bhyvectl to get info about what was going on when it comes close to the crash... -- Markiyan. > -- > Markiyan. > >> best >> Neel >> >>> -- >>> Markiyan. >>> >>>>> -- >>>>> Markiyan >>>>> >>>>> >>>>> ---------- Forwarded message ---------- >>>>> From: Markiyan Kushnir >>>>> Date: 2013/12/13 >>>>> Subject: >>>>> To: freebsd-current@freebsd.org, freebsd-virtualization@freebsd.org >>>>> >>>>> >>>>> I started some ports to compile inside a bhyve instance: >>>>> >>>>> root@vm:~ # uname -a >>>>> FreeBSD vm.mkushnir.mooo.com 11.0-CURRENT FreeBSD 11.0-CURRENT #0 >>>>> r259250: Thu Dec 12 14:17:32 EET 2013 >>>>> root@vm.mkushnir.zapto.org:/ >>>>> usr/obj/usr/src.svnup/sys/MAREK amd64 >>>>> >>>>> and left it running unattended. Approx. 2 hours later the host went to >>>>> panic. The bhyve instance survived after the panic and I could be able >>>>> to complete my ports compilation. >>>>> >>>>> core.txt attached (gzipped) >>>>> _______________________________________________ >>>>> freebsd-current@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>>>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >>>>> >>>> >>>> -- >>>> John Baldwin >>> _______________________________________________ >>> freebsd-current@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Sat Dec 14 14:00:34 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B5085ED0; Sat, 14 Dec 2013 14:00:34 +0000 (UTC) Received: from mail-wg0-x232.google.com (mail-wg0-x232.google.com [IPv6:2a00:1450:400c:c00::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 03A3D14AE; Sat, 14 Dec 2013 14:00:33 +0000 (UTC) Received: by mail-wg0-f50.google.com with SMTP id a1so2982080wgh.29 for ; Sat, 14 Dec 2013 06:00:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ytxNJAI5K2cbVTRrfDhYkTs5sBLR9RXv2p9Fr/5r3WY=; b=Gt9yV6uQYs6z/vsC6nU+NC6PDJ4Pbb1f4FBTDtiYmT7sMkqTb+F9cs06Uml9KvwkU3 GRx55yUddYSan56H2RXKpBe+Sz1GDsaU62lBa6L8oXG5oE17eVxlCNz2+43q+R3bRTZe 0jNp1JxVqsCp88azeHjW5ntg/6jWf/3mxtfTxKtio6k+wNss5oTJY0XpuW8YchWbfO8D QFAuk4OD8lr0qbJsPsA6dWQf7va3Tcm9TSE1xYVhviUkls2lP8IuljMY34DMoBtJgXPO VxF270lnETz+Kfnz1UaOdjP4zuuQxELIxWXtaPwE9pit/Qmx0WoF8nNp+aLQEmOcY68D vgKA== MIME-Version: 1.0 X-Received: by 10.180.108.97 with SMTP id hj1mr6688364wib.59.1387029631779; Sat, 14 Dec 2013 06:00:31 -0800 (PST) Received: by 10.227.226.12 with HTTP; Sat, 14 Dec 2013 06:00:31 -0800 (PST) In-Reply-To: References: <201312131304.57455.jhb@freebsd.org> Date: Sat, 14 Dec 2013 16:00:31 +0200 Message-ID: Subject: Re: 11.0-CURRENT panic while running a bhyve instance From: Markiyan Kushnir To: Neel Natu Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD current , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Dec 2013 14:00:34 -0000 2013/12/14 Markiyan Kushnir : > 2013/12/14 Markiyan Kushnir : >> 2013/12/14 Neel Natu : >>> Hi Markiyan, >>> >>> On Fri, Dec 13, 2013 at 2:35 PM, Markiyan Kushnir >>> wrote: >>>> 2013/12/13 John Baldwin : >>>>> On Friday, December 13, 2013 5:46:20 am Markiyan Kushnir wrote: >>>>>> Forgot to fill the Subject: header, re-posting it fixed. >>>>> >>>>> The mailing lists strips attachments, can you post it at a URL? >>>>> >>>> >>>> Shared here: >>>> >>>> https://drive.google.com/file/d/0B9Q-zpUXxqCnem5iYTVqLUxrcWo4cmlhdkM1c2lJa2dKak5R/edit?usp=sharing >>>> >>> >>> Thanks. >>> >>> It looks like something funky going on with the vcpu state. Do you >>> know if there was any access to the VM via 'bhyvectl' close to the >>> time of the panic? >>> >> >> Well, I don't know if there was. I would set up the same scenario >> again + a script running on the host querying bhyvectl. May be I would >> catch it again. Please let me know if all it makes sense, and if so, >> how it can be made better. >> > > To make it clear -- I didn't run bhyvectl when the VM was running, so > I can tell for sure that nobody was trying to interact with the VM > during its run. My first thought was that it would make sense to > (periodically) query state of the VM using bhyvectl to get info about > what was going on when it comes close to the crash... > > -- > Markiyan. Ok, I was able to hit a similarly looking panic exactly when trying to run "bhyvectl --vm=altroot-bhyve --get-all" while the VM was busy with its own "pkg delete -af". When the VM is mostly idle, bhyvectl --get-all goes smoothly. Now I'm thinking I might inadvertently run bhyvectl over my heavily running VM once when I was not at the desktop and wanted to see how things were going remotely from my office... Here are core.txt of the two panics in a row: https://drive.google.com/file/d/0B9Q-zpUXxqCnMUxVdGwxSEs1dE0/edit?usp=sharing https://drive.google.com/file/d/0B9Q-zpUXxqCnREtuTXpabnQ5QXM/edit?usp=sharing -- Markiyan. > >> -- >> Markiyan. >> >>> best >>> Neel >>> >>>> -- >>>> Markiyan. >>>> >>>>>> -- >>>>>> Markiyan >>>>>> >>>>>> >>>>>> ---------- Forwarded message ---------- >>>>>> From: Markiyan Kushnir >>>>>> Date: 2013/12/13 >>>>>> Subject: >>>>>> To: freebsd-current@freebsd.org, freebsd-virtualization@freebsd.org >>>>>> >>>>>> >>>>>> I started some ports to compile inside a bhyve instance: >>>>>> >>>>>> root@vm:~ # uname -a >>>>>> FreeBSD vm.mkushnir.mooo.com 11.0-CURRENT FreeBSD 11.0-CURRENT #0 >>>>>> r259250: Thu Dec 12 14:17:32 EET 2013 >>>>>> root@vm.mkushnir.zapto.org:/ >>>>>> usr/obj/usr/src.svnup/sys/MAREK amd64 >>>>>> >>>>>> and left it running unattended. Approx. 2 hours later the host went to >>>>>> panic. The bhyve instance survived after the panic and I could be able >>>>>> to complete my ports compilation. >>>>>> >>>>>> core.txt attached (gzipped) >>>>>> _______________________________________________ >>>>>> freebsd-current@freebsd.org mailing list >>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>>>>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >>>>>> >>>>> >>>>> -- >>>>> John Baldwin >>>> _______________________________________________ >>>> freebsd-current@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Sat Dec 14 23:23:03 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4B985A68; Sat, 14 Dec 2013 23:23:03 +0000 (UTC) Received: from mail-qe0-x230.google.com (mail-qe0-x230.google.com [IPv6:2607:f8b0:400d:c02::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D57441661; Sat, 14 Dec 2013 23:23:02 +0000 (UTC) Received: by mail-qe0-f48.google.com with SMTP id gc15so2807555qeb.7 for ; Sat, 14 Dec 2013 15:23:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=gK1j9JXKanmJxVl8cyJCzEcBEPGZdL9OeWUbEAkjXD8=; b=ZiXFjgxClJFUMM609ZfmdMZ3EehId+EX854PXCPABQQnSsvUGPP1t1/1lEF3kWlz8Z WsbGctto6Aep9n0w8NKgWvy/9W9rxiWeehqNMRyU6fZKTPq4ravytM88XE+C1Nw09orp XoIQb5SnHCwBDz1RSxo/2Dv4Ww7Tz8Q9G+V65Qxwzi1F8u43vhEAm5JdeY2wq1CKArLC H69vFmJQIwE/t5Ua/X0YKCAy+JhXRXvC7YMUXUAu4DBgCspJXRoYQuNrENSjk9UkttTU DygtQGPZ8fCWgWKzerO3O6jQq9T4LXoMZsgNLVMRG+Obi07xYBT9aDlax/pjKiVVmsTp qYnQ== MIME-Version: 1.0 X-Received: by 10.49.27.234 with SMTP id w10mr18939238qeg.22.1387063381926; Sat, 14 Dec 2013 15:23:01 -0800 (PST) Received: by 10.140.34.17 with HTTP; Sat, 14 Dec 2013 15:23:01 -0800 (PST) In-Reply-To: References: <201312131304.57455.jhb@freebsd.org> Date: Sat, 14 Dec 2013 15:23:01 -0800 Message-ID: Subject: Re: 11.0-CURRENT panic while running a bhyve instance From: Neel Natu To: Markiyan Kushnir Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD current , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Dec 2013 23:23:03 -0000 Hi Markiyan, On Sat, Dec 14, 2013 at 6:00 AM, Markiyan Kushnir wrote: > 2013/12/14 Markiyan Kushnir : >> 2013/12/14 Markiyan Kushnir : >>> 2013/12/14 Neel Natu : >>>> Hi Markiyan, >>>> >>>> On Fri, Dec 13, 2013 at 2:35 PM, Markiyan Kushnir >>>> wrote: >>>>> 2013/12/13 John Baldwin : >>>>>> On Friday, December 13, 2013 5:46:20 am Markiyan Kushnir wrote: >>>>>>> Forgot to fill the Subject: header, re-posting it fixed. >>>>>> >>>>>> The mailing lists strips attachments, can you post it at a URL? >>>>>> >>>>> >>>>> Shared here: >>>>> >>>>> https://drive.google.com/file/d/0B9Q-zpUXxqCnem5iYTVqLUxrcWo4cmlhdkM1= c2lJa2dKak5R/edit?usp=3Dsharing >>>>> >>>> >>>> Thanks. >>>> >>>> It looks like something funky going on with the vcpu state. Do you >>>> know if there was any access to the VM via 'bhyvectl' close to the >>>> time of the panic? >>>> >>> >>> Well, I don't know if there was. I would set up the same scenario >>> again + a script running on the host querying bhyvectl. May be I would >>> catch it again. Please let me know if all it makes sense, and if so, >>> how it can be made better. >>> >> >> To make it clear -- I didn't run bhyvectl when the VM was running, so >> I can tell for sure that nobody was trying to interact with the VM >> during its run. My first thought was that it would make sense to >> (periodically) query state of the VM using bhyvectl to get info about >> what was going on when it comes close to the crash... >> >> -- >> Markiyan. > > Ok, I was able to hit a similarly looking panic exactly when trying to > run "bhyvectl --vm=3Daltroot-bhyve --get-all" while the VM was busy with > its own "pkg delete -af". When the VM is mostly idle, bhyvectl > --get-all goes smoothly. > > Now I'm thinking I might inadvertently run bhyvectl over my heavily > running VM once when I was not at the desktop and wanted to see how > things were going remotely from my office=85 > Thanks, that helps a lot because I can see how bhyvectl could perturb the state of the vcpu and eventually lead to a panic. I'll submit a fix for this ASAP. best Neel > Here are core.txt of the two panics in a row: > > https://drive.google.com/file/d/0B9Q-zpUXxqCnMUxVdGwxSEs1dE0/edit?usp=3Ds= haring > > https://drive.google.com/file/d/0B9Q-zpUXxqCnREtuTXpabnQ5QXM/edit?usp=3Ds= haring > > -- > Markiyan. > >> >>> -- >>> Markiyan. >>> >>>> best >>>> Neel >>>> >>>>> -- >>>>> Markiyan. >>>>> >>>>>>> -- >>>>>>> Markiyan >>>>>>> >>>>>>> >>>>>>> ---------- Forwarded message ---------- >>>>>>> From: Markiyan Kushnir >>>>>>> Date: 2013/12/13 >>>>>>> Subject: >>>>>>> To: freebsd-current@freebsd.org, freebsd-virtualization@freebsd.org >>>>>>> >>>>>>> >>>>>>> I started some ports to compile inside a bhyve instance: >>>>>>> >>>>>>> root@vm:~ # uname -a >>>>>>> FreeBSD vm.mkushnir.mooo.com 11.0-CURRENT FreeBSD 11.0-CURRENT #0 >>>>>>> r259250: Thu Dec 12 14:17:32 EET 2013 >>>>>>> root@vm.mkushnir.zapto.org:/ >>>>>>> usr/obj/usr/src.svnup/sys/MAREK amd64 >>>>>>> >>>>>>> and left it running unattended. Approx. 2 hours later the host went= to >>>>>>> panic. The bhyve instance survived after the panic and I could be a= ble >>>>>>> to complete my ports compilation. >>>>>>> >>>>>>> core.txt attached (gzipped) >>>>>>> _______________________________________________ >>>>>>> freebsd-current@freebsd.org mailing list >>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>>>>>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freeb= sd.org" >>>>>>> >>>>>> >>>>>> -- >>>>>> John Baldwin >>>>> _______________________________________________ >>>>> freebsd-current@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>>>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd= .org"