From owner-freebsd-virtualization@FreeBSD.ORG Mon Dec 16 00:36:36 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 78EEAEE3 for ; Mon, 16 Dec 2013 00:36:36 +0000 (UTC) Received: from smtp141.ord.emailsrvr.com (smtp141.ord.emailsrvr.com [173.203.6.141]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4ACEF10C8 for ; Mon, 16 Dec 2013 00:36:35 +0000 (UTC) Received: from smtp10.relay.ord1a.emailsrvr.com (localhost.localdomain [127.0.0.1]) by smtp10.relay.ord1a.emailsrvr.com (SMTP Server) with ESMTP id 51E9F3701B6 for ; Sun, 15 Dec 2013 19:27:33 -0500 (EST) X-SMTPDoctor-Processed: csmtpprox 2.7.4 Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp10.relay.ord1a.emailsrvr.com (SMTP Server) with ESMTP id 4E2703701B8 for ; Sun, 15 Dec 2013 19:27:33 -0500 (EST) X-Virus-Scanned: OK Received: from smtp192.mex05.mlsrvr.com (unknown [184.106.31.85]) by smtp10.relay.ord1a.emailsrvr.com (SMTP Server) with ESMTPS id 3B9503701B6 for ; Sun, 15 Dec 2013 19:27:33 -0500 (EST) Received: from ORD2MBX05C.mex05.mlsrvr.com ([fe80::90e2:baff:fe30:7498]) by ORD2HUB05.mex05.mlsrvr.com ([fe80::9805:206:86ac:bda2%20]) with mapi id 14.03.0158.001; Sun, 15 Dec 2013 18:27:32 -0600 From: Michael Berman To: "freebsd-virtualization@freebsd.org" Subject: break to debugger in bhyve Thread-Topic: break to debugger in bhyve Thread-Index: AQHO+fWcAdZnUKNB1Eyr8jDJeSsP0A== Date: Mon, 16 Dec 2013 00:27:32 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [71.198.217.229] MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 00:36:36 -0000 I=92ve tried the various kernel and sysctl options, and it may be the way I= am starting bhyve but I can=92t seem to get a the FreeBSD guest to enter t= he kernel debugger with a hotkey. Michael From owner-freebsd-virtualization@FreeBSD.ORG Mon Dec 16 01:10:00 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C49705A7 for ; Mon, 16 Dec 2013 01:10:00 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 95F04124E for ; Mon, 16 Dec 2013 01:10:00 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-246-96.lns20.per2.internode.on.net [121.45.246.96]) (authenticated bits=0) by vps1.elischer.org (8.14.7/8.14.7) with ESMTP id rBG19s64055136 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 15 Dec 2013 17:09:58 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <52AE52DD.6020500@freebsd.org> Date: Mon, 16 Dec 2013 09:09:49 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Michael Berman , "freebsd-virtualization@freebsd.org" Subject: Re: break to debugger in bhyve References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 01:10:00 -0000 On 12/16/13, 8:27 AM, Michael Berman wrote: > I’ve tried the various kernel and sysctl options, and it may be the way I am starting bhyve but I can’t seem to get a the FreeBSD guest to enter the kernel debugger with a hotkey. it depends of course on which 'console' you are using,, for serial console with /dev/nmdm you can use the ALT_BREAK_TO_DEBUGGER option so that {enter| {~} {^B} will drop you into the debugger. When I was doing this last I had the console on the first serial port and the debugger on the second. (that's why I wrote the nmdm node.) > > Michael > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > > From owner-freebsd-virtualization@FreeBSD.ORG Mon Dec 16 05:02:00 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90546850 for ; Mon, 16 Dec 2013 05:02:00 +0000 (UTC) Received: from smtp157.ord.emailsrvr.com (smtp157.ord.emailsrvr.com [173.203.6.157]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 62F851351 for ; Mon, 16 Dec 2013 05:02:00 +0000 (UTC) Received: from smtp16.relay.ord1a.emailsrvr.com (localhost.localdomain [127.0.0.1]) by smtp16.relay.ord1a.emailsrvr.com (SMTP Server) with ESMTP id 8E7C73D0454; Mon, 16 Dec 2013 00:00:03 -0500 (EST) X-SMTPDoctor-Processed: csmtpprox 2.7.4 Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp16.relay.ord1a.emailsrvr.com (SMTP Server) with ESMTP id 889B53D0475; Mon, 16 Dec 2013 00:00:03 -0500 (EST) X-Virus-Scanned: OK Received: from smtp192.mex05.mlsrvr.com (unknown [184.106.31.85]) by smtp16.relay.ord1a.emailsrvr.com (SMTP Server) with ESMTPS id 6E86C3D0454; Mon, 16 Dec 2013 00:00:03 -0500 (EST) Received: from ORD2MBX05C.mex05.mlsrvr.com ([fe80::90e2:baff:fe30:7498]) by ORD2HUB34.mex05.mlsrvr.com ([::1]) with mapi id 14.03.0158.001; Sun, 15 Dec 2013 23:00:03 -0600 From: Michael Berman To: Julian Elischer Subject: Re: break to debugger in bhyve Thread-Topic: break to debugger in bhyve Thread-Index: AQHO+fWcAdZnUKNB1Eyr8jDJeSsP0JpWZ7KA///bvds= Date: Mon, 16 Dec 2013 05:00:02 +0000 Message-ID: <03322001-03C7-491F-B97D-64EA2E89BF6C@tidalscale.com> References: , <52AE52DD.6020500@freebsd.org> In-Reply-To: <52AE52DD.6020500@freebsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 05:02:00 -0000 Unfortunately, this does not seem to work with the bhyve /dev/console devic= e.=20 Sent from my iPhone > On Dec 15, 2013, at 5:10 PM, "Julian Elischer" wrote= : >=20 >> On 12/16/13, 8:27 AM, Michael Berman wrote: >> I=92ve tried the various kernel and sysctl options, and it may be the wa= y I am starting bhyve but I can=92t seem to get a the FreeBSD guest to ente= r the kernel debugger with a hotkey. > it depends of course on which 'console' you are using,, for serial conso= le with /dev/nmdm you can use the ALT_BREAK_TO_DEBUGGER > option so that {enter| {~} {^B} will drop you into the debugger. > When I was doing this last I had the console on the first serial port and= the debugger on the second. > (that's why I wrote the nmdm node.) >>=20 >> Michael >>=20 >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@fre= ebsd.org" >=20 From owner-freebsd-virtualization@FreeBSD.ORG Mon Dec 16 11:06:58 2013 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F72DF7A for ; Mon, 16 Dec 2013 11:06:58 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EEE321361 for ; Mon, 16 Dec 2013 11:06:57 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id rBGB6vuq019552 for ; Mon, 16 Dec 2013 11:06:57 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id rBGB6v13019550 for freebsd-virtualization@FreeBSD.org; Mon, 16 Dec 2013 11:06:57 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 16 Dec 2013 11:06:57 GMT Message-Id: <201312161106.rBGB6v13019550@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 11:06:58 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail 7 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Tue Dec 17 01:03:55 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 70E173DB; Tue, 17 Dec 2013 01:03:55 +0000 (UTC) Received: from mail-qc0-x22a.google.com (mail-qc0-x22a.google.com [IPv6:2607:f8b0:400d:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1D01E1361; Tue, 17 Dec 2013 01:03:55 +0000 (UTC) Received: by mail-qc0-f170.google.com with SMTP id x13so4422059qcv.29 for ; Mon, 16 Dec 2013 17:03:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=i5VIQCr4tWlcG0bQWycqCG0HrtT3inK8ADhU330zGPE=; b=pwFQtan3i6Zx/JK2GnbJrqk0gF/dqz6uRDfwsnh2Kk2WM8h2Ukf90Itl/ftlD2JxM1 eoyTrXj4gG48nms5g7zRoMvKhsj9gppSBUScJIvwFWJ/WyO1vTfDBgkOKrSVBbRAlLpo YaLvWuRABcxmT38+BRaTVHOvxmjvm0/un1et2ZtqT2bAfLL55QN2XwaMarLkgi55RBI8 0uNs1SXvO29yg0tE4rTYaKsYPLRBekq0zTOUCoCdVj6uQs/4tpSj6PYOEr46ODxogVTw 1ntkf9QAvdozs/774hR1B02i74RISsXSa9/KCf37G3Ceambuz3OUCV2TyJu+ufro6xvp 6xEw== MIME-Version: 1.0 X-Received: by 10.49.27.234 with SMTP id w10mr38015147qeg.22.1387242234350; Mon, 16 Dec 2013 17:03:54 -0800 (PST) Received: by 10.140.34.17 with HTTP; Mon, 16 Dec 2013 17:03:54 -0800 (PST) In-Reply-To: <03322001-03C7-491F-B97D-64EA2E89BF6C@tidalscale.com> References: <52AE52DD.6020500@freebsd.org> <03322001-03C7-491F-B97D-64EA2E89BF6C@tidalscale.com> Date: Mon, 16 Dec 2013 17:03:54 -0800 Message-ID: Subject: Re: break to debugger in bhyve From: Neel Natu To: Michael Berman Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2013 01:03:55 -0000 Hi Michael, On Sun, Dec 15, 2013 at 9:00 PM, Michael Berman wrote: > > Unfortunately, this does not seem to work with the bhyve /dev/console dev= ice. > Works fine for me once I compile with the ALT_BREAK_TO_DEBUGGER config opti= on. root@b2:~ # sysctl -a kern.conftxt | grep ALT_BREAK_TO_DEBUGGER options ALT_BREAK_TO_DEBUGGER root@b2:~ # root@b2:~ # KDB: enter: Break to debugger [ thread pid 11 tid 100003 ] Stopped at kdb_alt_break_internal+0x17f: movq $0,kdb_why db> best Neel > Sent from my iPhone > >> On Dec 15, 2013, at 5:10 PM, "Julian Elischer" wrot= e: >> >>> On 12/16/13, 8:27 AM, Michael Berman wrote: >>> I=92ve tried the various kernel and sysctl options, and it may be the w= ay I am starting bhyve but I can=92t seem to get a the FreeBSD guest to ent= er the kernel debugger with a hotkey. >> it depends of course on which 'console' you are using,, for serial cons= ole with /dev/nmdm you can use the ALT_BREAK_TO_DEBUGGER >> option so that {enter| {~} {^B} will drop you into the debugger. >> When I was doing this last I had the console on the first serial port an= d the debugger on the second. >> (that's why I wrote the nmdm node.) >>> >>> Michael >>> >>> _______________________________________________ >>> freebsd-virtualization@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >>> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@fr= eebsd.org" >> > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@free= bsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Tue Dec 17 02:06:48 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7649EF02 for ; Tue, 17 Dec 2013 02:06:48 +0000 (UTC) Received: from smtp149.ord.emailsrvr.com (smtp149.ord.emailsrvr.com [173.203.6.149]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 382F918EE for ; Tue, 17 Dec 2013 02:06:47 +0000 (UTC) Received: from smtp15.relay.ord1a.emailsrvr.com (localhost.localdomain [127.0.0.1]) by smtp15.relay.ord1a.emailsrvr.com (SMTP Server) with ESMTP id 02ACA270142; Mon, 16 Dec 2013 20:57:59 -0500 (EST) X-SMTPDoctor-Processed: csmtpprox 2.7.4 Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp15.relay.ord1a.emailsrvr.com (SMTP Server) with ESMTP id EF55E27018D; Mon, 16 Dec 2013 20:57:58 -0500 (EST) X-Virus-Scanned: OK Received: from smtp192.mex05.mlsrvr.com (unknown [184.106.31.85]) by smtp15.relay.ord1a.emailsrvr.com (SMTP Server) with ESMTPS id BEF0E270142; Mon, 16 Dec 2013 20:57:58 -0500 (EST) Received: from ORD2MBX05C.mex05.mlsrvr.com ([fe80::90e2:baff:fe30:7498]) by ORD2HUB07.mex05.mlsrvr.com ([fe80::d6ae:52ff:fe7f:66a7%15]) with mapi id 14.03.0158.001; Mon, 16 Dec 2013 19:57:56 -0600 From: Michael Berman To: Neel Natu Subject: Re: break to debugger in bhyve Thread-Topic: break to debugger in bhyve Thread-Index: AQHO+fWcAdZnUKNB1Eyr8jDJeSsP0JpWZ7KA///bvduAAbTwAP//iPsA Date: Tue, 17 Dec 2013 01:57:55 +0000 Message-ID: References: <52AE52DD.6020500@freebsd.org> <03322001-03C7-491F-B97D-64EA2E89BF6C@tidalscale.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [152.179.42.173] Content-Type: text/plain; charset="iso-8859-1" Content-ID: <74DFC4540633D94DA02974B57CFD19D0@mex05.mlsrvr.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2013 02:06:48 -0000 Yes, thank you. I had this in my sysctl.conf. It seems that my problem was in use of ssh in my environment. =B3~~^B=B2 finally did the trick. I=B9ve also moved it to boot/loader.conf as this sets it a little earlier i= n the boot sequence. It is still unreliable for me, but it may be my environent. On 12/16/13, 5:03 PM, "Neel Natu" wrote: >Hi Michael, > >On Sun, Dec 15, 2013 at 9:00 PM, Michael Berman > wrote: >> >> Unfortunately, this does not seem to work with the bhyve /dev/console >>device. >> > >Works fine for me once I compile with the ALT_BREAK_TO_DEBUGGER config >option. > >root@b2:~ # sysctl -a kern.conftxt | grep ALT_BREAK_TO_DEBUGGER >options ALT_BREAK_TO_DEBUGGER >root@b2:~ # >root@b2:~ # KDB: enter: Break to debugger >[ thread pid 11 tid 100003 ] >Stopped at kdb_alt_break_internal+0x17f: movq $0,kdb_why >db> > >best >Neel > >> Sent from my iPhone >> >>> On Dec 15, 2013, at 5:10 PM, "Julian Elischer" >>>wrote: >>> >>>> On 12/16/13, 8:27 AM, Michael Berman wrote: >>>> I=B9ve tried the various kernel and sysctl options, and it may be the >>>>way I am starting bhyve but I can=B9t seem to get a the FreeBSD guest t= o >>>>enter the kernel debugger with a hotkey. >>> it depends of course on which 'console' you are using,, for serial >>>console with /dev/nmdm you can use the ALT_BREAK_TO_DEBUGGER >>> option so that {enter| {~} {^B} will drop you into the debugger. >>> When I was doing this last I had the console on the first serial port >>>and the debugger on the second. >>> (that's why I wrote the nmdm node.) >>>> >>>> Michael >>>> >>>> _______________________________________________ >>>> freebsd-virtualization@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >>>> To unsubscribe, send any mail to >>>>"freebsd-virtualization-unsubscribe@freebsd.org" >>> >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to >>"freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Tue Dec 17 02:09:08 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5E0BFCE for ; Tue, 17 Dec 2013 02:09:08 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 928DE1901 for ; Tue, 17 Dec 2013 02:09:08 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id B39EC11E4C; Tue, 17 Dec 2013 12:09:06 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-67-161-27-37.hsd1.ca.comcast.net [67.161.27.37]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BQT54636 (AUTH peterg@ptree32.com.au); Tue, 17 Dec 2013 12:09:05 +1000 Message-ID: <52AFB23C.9000802@freebsd.org> Date: Mon, 16 Dec 2013 18:09:00 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: Neel Natu Subject: Re: break to debugger in bhyve References: <52AE52DD.6020500@freebsd.org> <03322001-03C7-491F-B97D-64EA2E89BF6C@tidalscale.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2013 02:09:08 -0000 > Works fine for me once I compile with the ALT_BREAK_TO_DEBUGGER config option. Also works without that, but the sysctl (or tunable) has to be set: root@bhyve-b1:~ # sysctl debug.kdb.alt_break_to_debugger=1 debug.kdb.alt_break_to_debugger: 0 -> 1 (issued the 3-char sequence CR ~ ctl-b) root@bhyve-b1:~ # KDB: enter: Break to debugger [ thread pid 11 tid 100003 ] Stopped at kdb_alt_break_internal+0x17f: movq $0,kdb_why db> later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Wed Dec 18 22:38:44 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D68E575 for ; Wed, 18 Dec 2013 22:38:44 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 635F71E24 for ; Wed, 18 Dec 2013 22:38:44 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id A45CBB964; Wed, 18 Dec 2013 17:38:42 -0500 (EST) From: John Baldwin To: Roger Pau =?iso-8859-1?q?Monn=E9?= Subject: Re: Panic starting a bhyve guest after resume Date: Wed, 18 Dec 2013 15:02:27 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <201312121511.38608.jhb@freebsd.org> <52AC13B1.8060402@citrix.com> In-Reply-To: <52AC13B1.8060402@citrix.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Message-Id: <201312181502.27806.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 18 Dec 2013 17:38:43 -0500 (EST) Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2013 22:38:44 -0000 On Saturday, December 14, 2013 3:15:45 am Roger Pau Monn=E9 wrote: > On 14/12/13 03:28, Neel Natu wrote: > > Hi John, > >=20 > > On Fri, Dec 13, 2013 at 2:09 PM, John Baldwin wrote: > >> On Thursday, December 12, 2013 4:00:08 pm Neel Natu wrote: > >>> Hi John, > >>> > >>> On Thu, Dec 12, 2013 at 12:11 PM, John Baldwin wrot= e: > >>>> If I suspend and resume my laptop and then try to start a guest afte= r=20 the > >>>> resume, I get an odd panic. It generates a privileged instruction=20 fault (in > >>>> kernel mode) for 'vmclear'. I've checked CR4 and it claims that VMX= E=20 is set. > >>>> I dont have any other ideas off the top of my head on what I should = be=20 poking > >>>> at? It looks like we read a bunch of MSRs in vmx_init(), but we don= 't=20 write > >>>> to them, and all vmx_enable() does on each CPU is set VMXE in CR4 fr= om=20 what I > >>>> can tell. > >>>> > >>> > >>> It also does a "vmxon" on each logical cpu which may also need to be > >>> done after a resume. > >> > >> Ah, yes it does. That was sufficient both for starting a new guest af= ter > >> resume and even doing a suspend/resume while a guest was active (and t= he > >> guest continued to run fine). I have a hacky patch for this. One, it > >> includes both a suspend and resume hook for VMM, though for my testing= I=20 only > >> needed a resume hook to invoke vmxon. Second, the name of vmx_resume2= () > >> is a total hack (because vmx_resume() was already taken. I think for = now > >> if I were to commit this, I'd just add the resme hook and maybe call t= he > >> Intel method vmx_reset() or vmx_restore()? > >> > >> http://people.freebsd.org/~jhb/patches/bhyve_resume.patch > >> > >=20 > > There seems to be a race after the APs are restarted and before > > 'vmm_resume_p()' where it would be problematic to execute a VMX > > instruction. > >=20 > > Perhaps we should enable VMX on each cpu before they return to the > > interrupted code? >=20 > Can you use the hook in cpususpend_handler? It's cpu_ops.cpu_resume, and > gets called on each CPU before returning from the handler. That is the right place, yes. However, I'm worried about collisions. Can = you=20 run nested VMM's under Xen? That is, can a xenhvm guest start a bhyve vmm?= =20 If so, then you would need to run both cpu_resume handlers. Also, cpu_resu= me=20 isn't run on the CPU that initiates the suspend. For now I will stick with= a dedicated vmm_resume_p hook, but we may want to revisit that at some point. =2D-=20 John Baldwin From owner-freebsd-virtualization@FreeBSD.ORG Thu Dec 19 08:37:25 2013 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 81B01315; Thu, 19 Dec 2013 08:37:25 +0000 (UTC) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9ADCB1908; Thu, 19 Dec 2013 08:37:24 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.95,512,1384300800"; d="scan'208";a="83823740" Received: from accessns.citrite.net (HELO FTLPEX01CL03.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 19 Dec 2013 08:37:15 +0000 Received: from [IPv6:::1] (10.80.16.47) by smtprelay.citrix.com (10.13.107.80) with Microsoft SMTP Server id 14.2.342.4; Thu, 19 Dec 2013 03:37:15 -0500 Message-ID: <52B2B03B.7060903@citrix.com> Date: Thu, 19 Dec 2013 09:37:15 +0100 From: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: John Baldwin Subject: Re: Panic starting a bhyve guest after resume References: <201312121511.38608.jhb@freebsd.org> <52AC13B1.8060402@citrix.com> <201312181502.27806.jhb@freebsd.org> In-Reply-To: <201312181502.27806.jhb@freebsd.org> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-DLP: MIA2 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2013 08:37:25 -0000 On 18/12/13 21:02, John Baldwin wrote: > On Saturday, December 14, 2013 3:15:45 am Roger Pau Monné wrote: >> On 14/12/13 03:28, Neel Natu wrote: >>> Hi John, >>> >>> On Fri, Dec 13, 2013 at 2:09 PM, John Baldwin wrote: >>>> On Thursday, December 12, 2013 4:00:08 pm Neel Natu wrote: >>>>> Hi John, >>>>> >>>>> On Thu, Dec 12, 2013 at 12:11 PM, John Baldwin wrote: >>>>>> If I suspend and resume my laptop and then try to start a guest after > the >>>>>> resume, I get an odd panic. It generates a privileged instruction > fault (in >>>>>> kernel mode) for 'vmclear'. I've checked CR4 and it claims that VMXE > is set. >>>>>> I dont have any other ideas off the top of my head on what I should be > poking >>>>>> at? It looks like we read a bunch of MSRs in vmx_init(), but we don't > write >>>>>> to them, and all vmx_enable() does on each CPU is set VMXE in CR4 from > what I >>>>>> can tell. >>>>>> >>>>> >>>>> It also does a "vmxon" on each logical cpu which may also need to be >>>>> done after a resume. >>>> >>>> Ah, yes it does. That was sufficient both for starting a new guest after >>>> resume and even doing a suspend/resume while a guest was active (and the >>>> guest continued to run fine). I have a hacky patch for this. One, it >>>> includes both a suspend and resume hook for VMM, though for my testing I > only >>>> needed a resume hook to invoke vmxon. Second, the name of vmx_resume2() >>>> is a total hack (because vmx_resume() was already taken. I think for now >>>> if I were to commit this, I'd just add the resme hook and maybe call the >>>> Intel method vmx_reset() or vmx_restore()? >>>> >>>> http://people.freebsd.org/~jhb/patches/bhyve_resume.patch >>>> >>> >>> There seems to be a race after the APs are restarted and before >>> 'vmm_resume_p()' where it would be problematic to execute a VMX >>> instruction. >>> >>> Perhaps we should enable VMX on each cpu before they return to the >>> interrupted code? >> >> Can you use the hook in cpususpend_handler? It's cpu_ops.cpu_resume, and >> gets called on each CPU before returning from the handler. > > That is the right place, yes. However, I'm worried about collisions. Can you > run nested VMM's under Xen? That is, can a xenhvm guest start a bhyve vmm? In theory yes, but AFAIK nested virtualization support in Xen is not really stable (it's still marked as experimental). > If so, then you would need to run both cpu_resume handlers. Also, cpu_resume > isn't run on the CPU that initiates the suspend. For now I will stick with a > dedicated vmm_resume_p hook, but we may want to revisit that at some point. Using something like an event handler would be the best solution, but that doesn't work on interrupt context due to the usage of mutexes. From owner-freebsd-virtualization@FreeBSD.ORG Fri Dec 20 10:00:50 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2AC9EDE5 for ; Fri, 20 Dec 2013 10:00:50 +0000 (UTC) Received: from mail-bk0-f50.google.com (mail-bk0-f50.google.com [209.85.214.50]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B38611365 for ; Fri, 20 Dec 2013 10:00:49 +0000 (UTC) Received: by mail-bk0-f50.google.com with SMTP id e11so1116025bkh.37 for ; Fri, 20 Dec 2013 02:00:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=PU9lt0bSVJ6z91WzzPru2UQrOXAI25TvobqgbVRmaGA=; b=WdMcDD5uuZE3RnGDEWCadWlU2fv2M3ZRmSOOuYwwvSsXTYt4uE/1OJ6MXAS/nahXAf Mllx1wulG+AcRHxZtZmN5nii+XTkHECrCZSkAlyWxMMKwWUZrRncJQTOeooutMNm7O9E b4xuq+qoorWkfXo+pdOVhPXULwMJ0L6bIAKmJRbqekbY4U3bAeWBTt4/9jYpI+djUqI4 6egMnMNw0zJv3+iNWKCqqZ9R7FqVcdZ95BCbpBW+ed227UTiujAuqwkNolmOA6b1gZux IMB+8cscnqkNOAEhsfLfmJ/aDLsUh8fPNItaltiam6O886OzyRrtg3nuIx3oUdBCgN6d qjJw== X-Gm-Message-State: ALoCoQkxc9T6QvX+D/8sb2pxe4AP2HXrs/1/nCBwnXu57oF3r3A/Aw9IPiyePgJ7Zlu0wBK7KSRu MIME-Version: 1.0 X-Received: by 10.204.99.205 with SMTP id v13mr4041576bkn.21.1387533245870; Fri, 20 Dec 2013 01:54:05 -0800 (PST) Received: by 10.204.62.136 with HTTP; Fri, 20 Dec 2013 01:54:05 -0800 (PST) Date: Fri, 20 Dec 2013 10:54:05 +0100 Message-ID: Subject: Means to control a bhyve VM instance From: =?UTF-8?Q?Micha=C5=82_Dubiel?= To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Dec 2013 10:00:50 -0000 Hi, May I ask you about the means to control a VM instance. I mean by that something providing similar features as QEMU's monitor interface along with QEMU Machine Protocol (QMP) do. Has it already been decided/designed how bhyve is going to handle features like a soft start/stop of a VM instance, hot-plugging of additional blk/net devices, etc? Regards, Michal. From owner-freebsd-virtualization@FreeBSD.ORG Sat Dec 21 22:56:40 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 72FDB20B for ; Sat, 21 Dec 2013 22:56:40 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 330681898 for ; Sat, 21 Dec 2013 22:56:40 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id D40701214B; Sun, 22 Dec 2013 08:56:13 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-71-196-188-222.hsd1.co.comcast.net [71.196.188.222]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BQV81603 (AUTH peterg@ptree32.com.au); Sun, 22 Dec 2013 08:56:12 +1000 Message-ID: <52B61C94.4000006@freebsd.org> Date: Sat, 21 Dec 2013 14:56:20 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: =?UTF-8?B?TWljaGHFgiBEdWJpZWw=?= Subject: Re: Means to control a bhyve VM instance References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Dec 2013 22:56:40 -0000 Hi MichaĹ‚, > May I ask you about the means to control a VM instance. I mean by that > something providing similar features as QEMU's monitor interface along > with QEMU > Machine Protocol (QMP) do. Has it already been decided/designed how bhyve > is going to handle features like a soft start/stop of a VM instance, > hot-plugging of additional blk/net devices, etc? Decided, no, though the broad direction we're moving in is for bhyve to be a single process (reboots in-process, no external loader), and control from bhyvectl via some form of RPC. later, Peter.