From owner-freebsd-virtualization@FreeBSD.ORG Sat Dec 27 19:23:03 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 637EF721 for ; Sat, 27 Dec 2014 19:23:03 +0000 (UTC) Received: from mail-qc0-x22e.google.com (mail-qc0-x22e.google.com [IPv6:2607:f8b0:400d:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1456B1339 for ; Sat, 27 Dec 2014 19:23:03 +0000 (UTC) Received: by mail-qc0-f174.google.com with SMTP id c9so8169729qcz.33 for ; Sat, 27 Dec 2014 11:23:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:user-agent:in-reply-to :references:mime-version:content-type; bh=ScixE67LjN7LqxVwEHrC8aD5m2OAHqr3WOVoIYPKnZg=; b=lUBH/O2//dP747EUb6/h8QUAE3B08abc/WzWiEsCTm8sk06tlXe+lQjdlEwipIOmwr DOavol0eklKcMSpGSsH+1qw1CLIkWpgnqc9jqlQvthSELZLHIISYgUlH1tFHZnbcVviV iI65T2rt7F65J7M5F0Pw0P2jLt/Aq20/b0GoSYn4JLmhWO4iM/rwgJ6/kI5ojXlSbOPD 6B54y6m8b6e51tO73a+OHRAMcpyeCgBBq7Mgxj/n3P+lvNF/HmU+hTu6gRl2oft+TTY0 ciwbKJ/agtj8qY9U1ygF2blOupryBr08tg5IcScFpzeb5QCshPsUS5iTuK3Aaitd6Cf3 AH/Q== X-Received: by 10.224.171.129 with SMTP id h1mr33064337qaz.74.1419708182082; Sat, 27 Dec 2014 11:23:02 -0800 (PST) Received: from hbsd-dev-local.localnet ([73.173.99.185]) by mx.google.com with ESMTPSA id k102sm8990096qgd.7.2014.12.27.11.23.01 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 27 Dec 2014 11:23:01 -0800 (PST) From: Shawn Webb To: Neel Natu Subject: Re: FreeBSD bhyve VM Exiting Date: Sat, 27 Dec 2014 14:22:57 -0500 Message-ID: <2487562.Nj6jnlQzjX@hbsd-dev-local> User-Agent: KMail/4.14.2 (FreeBSD/11.0-CURRENT; KDE/4.14.2; amd64; ; ) In-Reply-To: References: MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1728346.4RnBHVCunU"; micalg="pgp-sha256"; protocol="application/pgp-signature" Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Dec 2014 19:23:03 -0000 --nextPart1728346.4RnBHVCunU Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" On Friday, December 26, 2014 12:07:44 PM Neel Natu wrote: > Hi Shawn, > > On Fri, Dec 26, 2014 at 6:26 AM, Shawn Webb wrote: > > Hey All, > > > > I'm running a FreeBSD/amd64 11-current (r276204) guest VM in bhyve. It > > seems that running Poudriere in it causes the VM to exit. > > > > This is what bhyve outputs when the VM dies: > > reason VMX > > rip 0xffffffff80d68564 > > inst_length 5 > > status 0 > > exit_reason 2 > > qualification 0x0000000000000000 > > inst_type 0 > > inst_error 0 > > > > I have no idea what "reason VMX" means or what "exit_reason 2" means. Is > > there a way to decode that? > > Exit reason '2' is a triple fault. > > > What kinds of info/data do you guys need to debug this? > > You'll need a FreeBSD host with svn revision >= 276098 and a custom > kernel config to enable KTR. > https://svnweb.freebsd.org/base?view=revision&revision=276098 > > Then you can set the following tunable "hw.vmm.trace_guest_exceptions" > to "1" before loading vmm.ko. This will cause bhyve to trace all guest > exceptions via KTR. You can verify that the tunable took effect using > 'sysctl hw.vmm.trace_guest_exceptions'. > > When the bhyve process exits you can then dump the KTR entries using > something like this: 'sudo ktrdump -cto /tmp/ktrdump.out'. You'll > probably want to do this in a script so the ktrdump(8) happens > immediately after bhyve(8) exits. > > I use the following options in my kernel config to enable KTR: > options KTR > options KTR_MASK=(KTR_GEN) > options KTR_ENTRIES=(4*1024*1024) > > Hope this helps. > > best > Neel Hey Neel, Thanks for the helpful suggestions! The good thing is that my host is already on a revision >= 276098 and my kernel is already built with KTR. With 31c3 going on, the soonest I can do that will be Tuesday or Wednesday of next week. I'll update you then. Thanks, Shawn --nextPart1728346.4RnBHVCunU Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJUnwcRAAoJEGqEZY9SRW7udhAP/1T0NtXGwdTLFdlpSg9YNy1f UkmIC4KYgdGLYoy57y13Fp/6doUoPqPLpH6/N2iU32pUvluPXwEhzYg1Zu6e1gaC MzdSrCCMwV0nfGuoxpcHcY1C7xneKfXX6L+vqPi7axIL59SGmx+TI/fPRvjvTtee il8vT8QCNYTNyYEo7IGCQKptDRjvg/JTHn8PCBEdG3K4M20uxa2QU/XfxwCWC5Ui E+JRm6ZFoJtVFUd5ViqNrPik57s0mBJ09SjtjsXaLx/biC7z0VQNqHBRjnm7LNgL YoCFkTgjq93BnQAZucmDBEbKjS1B9Ty1pezmRSfS9ln5+ylrTQ8B05XOlJrT72ra jJ0mom995Zzb75epZ7KeHHBbCvxfy8nhXfBweZiYPBxQlzU7SHbINC+bE1fN1xQs nW0RmBIfCuByy3O/IYXTNsu8dhwknJAVWvru7gXKSV17SWAfKbtl2oJGr68swkgu fk5hspmLN+MK+zMR5LveARjNDN77VxFF1MZob51QBbCTgvHkpC1WnUcbRmQ8sQ/g faaWx1beIy3egjB+ZgAbeCQjRqNWZ8BhDh1ZiNLrOylaAdO2BcQM5FUXhHNqcQhu fxR1dlNCp0Lc5eIrrZQCmR1LrFUnwREcgT42fKslFG9EnNtLBIaeC6eSRSkf0Mqe UzgN/BgP+LnPMZtw9p9w =nEMY -----END PGP SIGNATURE----- --nextPart1728346.4RnBHVCunU--