From owner-freebsd-virtualization@freebsd.org Wed Jul 15 07:02:18 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ECA329A2BAE for ; Wed, 15 Jul 2015 07:02:18 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: from mail-wg0-x22e.google.com (mail-wg0-x22e.google.com [IPv6:2a00:1450:400c:c00::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 87E4F1C66; Wed, 15 Jul 2015 07:02:18 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: by wgjx7 with SMTP id x7so25422307wgj.2; Wed, 15 Jul 2015 00:02:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=aJdGqZ8pMDuvzQEosCufAZsWd2zOotPAAe+R9HTllyY=; b=RCihH0N5Ks4+PlJbL1qi/lkM3xUWUmIO/7mFdeK80+wbyb0YhG6LQSrtLqkyBz4gfa 080tUfFK7sVV1zTfbXmTVK3IBGs/AKNW1XUYJ8U87XnL3sT7J9fJgyBFOiEZ8oyYgQJ/ ZbFVIBB8ZPh+r+eFdU/ZKlPaBcK7cL8PFQLsS/jIYix014k9VYkBtH6NdQK7tjOUMjHB oL9fVOHvkMGuzRtPBvDm4WgPkGQDlcU36RrRgQu1XpxX3sw2pJVKpECbYaHLfb/QQCYH ZwVn7OE6464xjxnzJU1SVpERkgaC4p0DFI7eTlJr9ZFVhFnH+AHSpgE5rNb0x9vrqmkz nkKA== MIME-Version: 1.0 X-Received: by 10.194.60.131 with SMTP id h3mr5161286wjr.156.1436943737092; Wed, 15 Jul 2015 00:02:17 -0700 (PDT) Received: by 10.27.214.139 with HTTP; Wed, 15 Jul 2015 00:02:17 -0700 (PDT) In-Reply-To: <55A514D1.7040805@FreeBSD.org> References: <5587EE05.2020001@FreeBSD.org> <558900A7.40609@FreeBSD.org> <55891335.5090403@FreeBSD.org> <55A514D1.7040805@FreeBSD.org> Date: Wed, 15 Jul 2015 00:02:17 -0700 Message-ID: Subject: Re: bhyve: centos 7.1 with multiple virtual processors From: Neel Natu To: Andriy Gapon Cc: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Jul 2015 07:02:19 -0000 Hi Andriy, On Tue, Jul 14, 2015 at 6:55 AM, Andriy Gapon wrote: > On 23/06/2015 11:05, Andriy Gapon wrote: >> On 23/06/2015 10:26, Neel Natu wrote: > [snip] >>> Does this ever happen with a single vcpu guest? >> >> Never seen the problem with a single CPU so far. >> Also, never had that problem with FreeBSD guests. >> >>> The other mystery is the NMIs the host is receiving. I (re)verified to >>> make sure that bhyve/vmm.ko do not assert NMIs so it has to be >>> something else on the host that's doing it ... >> >> But the correlation with the multi-CPU non-FreeBSD guests seems to be significant. > > Today I've got another NMI with a Linux guest, but a few things were different: > - the host was fresh after a reboot > - the guest was single CPU > - the NMI happened during boot up of the very first VM > > kernel: NMI ISA 2c, EISA ff > kernel: NMI ... going to debugger > > Not what causes those NMIs but when they happen they always happen during a > bhyve guest boot... > Could you update the host with the following patch? https://people.freebsd.org/~neel/patches/ktr_stray_nmi.patch It enables KTR_GEN logging by default and turns it off when a VM-exit due to NMI is detected. If there is a pattern to the NMIs then it can be correlated with the guest activity around that time. best Neel > -- > Andriy Gapon