From owner-freebsd-virtualization@freebsd.org Sun Dec 15 02:07:28 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 220211DE485 for ; Sun, 15 Dec 2019 02:07:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47b78J06Hhz41RC for ; Sun, 15 Dec 2019 02:07:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 03B7E1DE484; Sun, 15 Dec 2019 02:07:28 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0378A1DE483 for ; Sun, 15 Dec 2019 02:07:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47b78H6NK2z41R9 for ; Sun, 15 Dec 2019 02:07:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BC7F5602F for ; Sun, 15 Dec 2019 02:07:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBF27RIJ031561 for ; Sun, 15 Dec 2019 02:07:27 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBF27R4x031560 for virtualization@FreeBSD.org; Sun, 15 Dec 2019 02:07:27 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 203874] [patch] MSI/MSI-X interrupts don't work in VMware virtual machines Date: Sun, 15 Dec 2019 02:07:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: mp@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Dec 2019 02:07:28 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D203874 Mark Peek changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mp@FreeBSD.org --- Comment #29 from Mark Peek --- I looked into this issue a bit to better understand the lineage. There was a report of issues for FreeBSD in ESXi 5.0 back in late 2011: https://forums.freebsd.org/threads/freebsd-9-rc2-breaks-mpt-driver-when-use= d-with-vmware-and-pcie-passthrough.27899/ And the blacklist quirk was added here in 2013-03: https://svnweb.freebsd.org/base/head/sys/dev/pci/pci.c?revision=3D247632&vi= ew=3Dmarkup I could not find a bug report for the above FreeBSD issue within the VMware= bug tracker. But it did show up as a Linux 4.3+ issue in 2016-01 reported again= st ESXi 6.0. The issue and resolution was documented in this knowledge base article which lists fixes in ESXi 5.5 patch 08 and 6.0 patch 03 which came out in 2016. https://kb.vmware.com/s/article/2150585 I will note that Comment #19 indicates it is broken in ESXi 5.5 build 32485= 47 which appears to be in the patch 07 branch prior to this fix being availabl= e. Now, let=E2=80=99s look at EOL dates: vSphere 5.0 and 5.1 EOL - August 24, 2016 vSphere 5.5 EOL =E2=80=93 September 19, 2018 vSphere 6.0 EOL =E2=80=93 March 2020 There have been patch releases made available in 2016 to address both 5.5 a= nd 6.0, plus having 5.5 already EOL and 6.0 in a few months. Given that it is = my recommendation we 1) remove the blacklist for these devices and 2) not investigate or add version checking (which likely would be error prone and would not take into account patch releases). I just proposed a patch to remove the quirk which I can commit with approva= ls: https://reviews.freebsd.org/D22819 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Mon Dec 16 16:35:13 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A59191C84E8 for ; Mon, 16 Dec 2019 16:35:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47c6M541Tvz40xn for ; Mon, 16 Dec 2019 16:35:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 87EBA1C84E7; Mon, 16 Dec 2019 16:35:13 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 86A841C84E6 for ; Mon, 16 Dec 2019 16:35:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47c6M52v5lz40xm for ; Mon, 16 Dec 2019 16:35:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5A43E3F for ; Mon, 16 Dec 2019 16:35:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBGGZDau056535 for ; Mon, 16 Dec 2019 16:35:13 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBGGZDvn056534 for virtualization@FreeBSD.org; Mon, 16 Dec 2019 16:35:13 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242666] Stopped VM cant restart again Date: Mon, 16 Dec 2019 16:35:13 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: bernhard@blberger.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2019 16:35:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242666 Bug ID: 242666 Summary: Stopped VM cant restart again Product: Base System Version: 12.1-RELEASE Hardware: amd64 OS: Any Status: New Severity: Affects Only Me Priority: --- Component: bhyve Assignee: virtualization@FreeBSD.org Reporter: bernhard@blberger.de Hi, I have an old problem with BHYVE VM's Sometimes a VM that has been stopped cannot be restarted. When the VM is restarted, an error message appears stating that it is still running. Currently this can only be solved by rebooting the server. This is also the case with the current FreeBSD 12.1 release :-( Here is my VM list root@superserver:~ # vm list NAME DATASTORE LOADER CPU MEMORY VNC AUTOSTART STATE nextcloud default bhyveload 2 2G - Yes [3] Running (86383) samba default bhyveload 2 2G - Yes [2] Running (15621) test-pc default bhyveload 2 2G - No Stopped unifi default grub 1 2G - Yes [1] Running (15601) Here I try to start the VM "test-pc root@superserver:~ # vm start test-pc Starting test-pc * found guest in /data/bhyve/test-pc ! guest appears to be running already However, there is no process=20 root@superserver:~ # ps xaf | grep test-pc 19050 14 S+ 0:00.00 grep test-pc compared to samba root@superserver:~ # ps xaf | grep samba 15361 2- I 0:00.01 /bin/sh /usr/local/sbin/vm _run samba 15621 2- IC 102:54.67 bhyve: samba (bhyve) 19971 14 S+ 0:00.00 grep samba but in /dev/vmm the test-pc exists as running VM root@superserver:~ # ll /dev/vmm total 0 crw------- 1 root wheel 0xb1 Dec 16 13:35 nextcloud crw------- 1 root wheel 0xb3 Dec 15 23:25 samba crw------- 1 root wheel 0xb2 Dec 16 15:26 test-pc crw------- 1 root wheel 0x7d Dec 15 23:24 unifi Translated with www.DeepL.com/Translator (free version) --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Mon Dec 16 16:45:12 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B6B231C89C3 for ; Mon, 16 Dec 2019 16:45:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47c6Zc4Pprz41kn for ; Mon, 16 Dec 2019 16:45:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 973581C89C2; Mon, 16 Dec 2019 16:45:12 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 96F8F1C89BF for ; Mon, 16 Dec 2019 16:45:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47c6Zc388jz41km for ; Mon, 16 Dec 2019 16:45:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6569023D for ; Mon, 16 Dec 2019 16:45:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBGGjCNN078518 for ; Mon, 16 Dec 2019 16:45:12 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBGGjC0i078517 for virtualization@FreeBSD.org; Mon, 16 Dec 2019 16:45:12 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242666] Stopped VM cant restart again Date: Mon, 16 Dec 2019 16:45:12 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: bernhard@blberger.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: keywords Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2019 16:45:12 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242666 Bernhard Berger changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |bhyve --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue Dec 17 04:44:11 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 21E4E1DB87D for ; Tue, 17 Dec 2019 04:44:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47cQXC05vtz3KKw for ; Tue, 17 Dec 2019 04:44:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 0371C1DB87C; Tue, 17 Dec 2019 04:44:11 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 033041DB87B for ; Tue, 17 Dec 2019 04:44:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47cQXB6NP3z3KKv for ; Tue, 17 Dec 2019 04:44:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D5FE589A7 for ; Tue, 17 Dec 2019 04:44:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBH4iAgu068150 for ; Tue, 17 Dec 2019 04:44:10 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBH4iAm7068149 for virtualization@FreeBSD.org; Tue, 17 Dec 2019 04:44:10 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242666] Stopped VM cant restart again Date: Tue, 17 Dec 2019 04:44:10 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: aleksandr.fedorov@itglobal.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2019 04:44:11 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242666 Aleksandr Fedorov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |aleksandr.fedorov@itglobal. | |com --- Comment #1 from Aleksandr Fedorov --- As I understand it, you are using sysutils/vm-bhyve port. Then, this may no= t be a bhyve problem. If the bhyve process unexpectedly terminates with an error or is killed via kill -9, the device file remains in /dev/vmm/. And the attempt to restart the virtual machine will fail until you run the bhyvectl --destroy = --vm command. But, I don't know how well sysutils/vm-bhyve handles such situations. Can you show us the output of the bhyvectl --get-exit-reason --vm test-pc command to determine the exit reason for the VM? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue Dec 17 05:21:47 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A2F521DC45F for ; Tue, 17 Dec 2019 05:21:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47cRMb3vllz3M2b for ; Tue, 17 Dec 2019 05:21:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 860B21DC45E; Tue, 17 Dec 2019 05:21:47 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 85C7E1DC45D for ; Tue, 17 Dec 2019 05:21:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47cRMb31vRz3M2Z for ; Tue, 17 Dec 2019 05:21:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6373A8F58 for ; Tue, 17 Dec 2019 05:21:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBH5LlDU052989 for ; Tue, 17 Dec 2019 05:21:47 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBH5LlCK052966 for virtualization@FreeBSD.org; Tue, 17 Dec 2019 05:21:47 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242666] Stopped VM cant restart again Date: Tue, 17 Dec 2019 05:21:46 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jason@tubnor.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2019 05:21:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242666 Jason Tubnor changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jason@tubnor.net --- Comment #2 from Jason Tubnor --- Can you also check for a run.lock file? This will be located in the guest = vm configuration directory: /data/bhyve/test-pc If this is present, vm-bhyve will argue. If you remove it, vm-bhyve will c= lean up and launch your vm again. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue Dec 17 10:20:54 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 282481E2C84 for ; Tue, 17 Dec 2019 10:20:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47cZ0k0HP3z486V for ; Tue, 17 Dec 2019 10:20:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 07D651E2C83; Tue, 17 Dec 2019 10:20:54 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0798A1E2C82 for ; Tue, 17 Dec 2019 10:20:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47cZ0j6Rd5z486T for ; Tue, 17 Dec 2019 10:20:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D8B86C66C for ; Tue, 17 Dec 2019 10:20:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBHAKrw2001024 for ; Tue, 17 Dec 2019 10:20:53 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBHAKrJg000916 for virtualization@FreeBSD.org; Tue, 17 Dec 2019 10:20:53 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242666] Stopped VM cant restart again Date: Tue, 17 Dec 2019 10:20:53 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: bernhard@blberger.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2019 10:20:54 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242666 --- Comment #3 from Bernhard Berger --- (In reply to Jason Tubnor from comment #2) no, the file dont exists --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue Dec 17 11:01:46 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7526B1E3DC6 for ; Tue, 17 Dec 2019 11:01:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47cZvt2YW1z4Bf3 for ; Tue, 17 Dec 2019 11:01:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 57B591E3DC5; Tue, 17 Dec 2019 11:01:46 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 577CB1E3DC4 for ; Tue, 17 Dec 2019 11:01:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47cZvt1h8mz4Bf2 for ; Tue, 17 Dec 2019 11:01:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 34D6CCEE0 for ; Tue, 17 Dec 2019 11:01:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBHB1k8q080227 for ; Tue, 17 Dec 2019 11:01:46 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBHB1kjW080226 for virtualization@FreeBSD.org; Tue, 17 Dec 2019 11:01:46 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242666] Stopped VM cant restart again Date: Tue, 17 Dec 2019 11:01:46 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: bernhard@blberger.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2019 11:01:46 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242666 --- Comment #4 from Bernhard Berger --- (In reply to Aleksandr Fedorov from comment #1) root@superserver:/data/bhyve # bhyvectl --get-exit-reason --vm test-pc VM:test-pc is not created. root@superserver:/data/bhyve # bhyvectl --destroy --vm test-pc VM:test-pc is not created. root@superserver:/data/bhyve # vm start test-pc Starting test-pc * found guest in /data/bhyve/test-pc ! guest appears to be running already root@superserver:/data/bhyve # ls /dev/vmm nextcloud samba test-pc unifi root@superserver:/data/bhyve # ls test-pc disk0.img test-pc.conf root@superserver:/data/bhyve # vm list NAME DATASTORE LOADER CPU MEMORY VNC AUTOSTART STATE nextcloud default bhyveload 2 2G - Yes [3] Running (8638= 3) samba default bhyveload 2 2G - Yes [2] Running (1562= 1) test-pc default bhyveload 2 2G - No Stopped unifi default grub 1 2G - Yes [1] Running (1560= 1) root@superserver:/data/bhyve # ps xaf | grep bhyve 86383 0- SC 450:08.40 bhyve: nextcloud (bhyve) 15601 2- SC 53:29.89 bhyve: unifi (bhyve) 15621 2- IC 207:39.88 bhyve: samba (bhyve) root@superserver:/data/bhyve # ls /dev | grep nmdm nmdm-nextcloud.1A nmdm-nextcloud.1B nmdm-samba.1A nmdm-samba.1B nmdm-unifi.1A nmdm-unifi.1B Definitely the VM "test-pc" is not started and cannot be started because: /dev/vmm/test-pc still exists.=20 There must be a way to remove the /dev/vmm/test-pc device. Just to understand, this is not the first time this problem has occurred, b= ut since I've been working with bhyve since FreeBSD 11, it's very off.=20 It often happens that if you type "reboot" in the guest system (FreeBSD), t= his is exactly what happens. If you enter "shutdown -p now" in the guest system (FreeBSD), it may also happen that the VM cannot be started afterwards.=20 I think it has to do with a non-closed console. In the past, I solved this by rebooting the host system. But that can't be = the solution and that's why I opened this ticket. VM list apparently only checks if the process bhyve exists bhyveload seems to only check if the device /dev/vmm/ exists.=20 bhyvectl A process "bhyve " must also exist for /dev/vm/.= If the process does not exist, the device must be removed. This should not be = so difficult to program. Yours sincerely There is another problem which has to do with VM-BHYVE but how to set a tic= ket --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue Dec 17 20:09:07 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C5D4D1C8760 for ; Tue, 17 Dec 2019 20:09:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47cq3R4xf9z3DkQ for ; Tue, 17 Dec 2019 20:09:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id A7A791C875F; Tue, 17 Dec 2019 20:09:07 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A64761C875E for ; Tue, 17 Dec 2019 20:09:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47cq3R3lNwz3DkP for ; Tue, 17 Dec 2019 20:09:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7C02B1B10D for ; Tue, 17 Dec 2019 20:09:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBHK97G2001150 for ; Tue, 17 Dec 2019 20:09:07 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBHK97Ld001149 for virtualization@FreeBSD.org; Tue, 17 Dec 2019 20:09:07 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242666] Stopped VM cant restart again Date: Tue, 17 Dec 2019 20:09:07 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jason@tubnor.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2019 20:09:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242666 --- Comment #5 from Jason Tubnor --- FWIW: We don't use anything except UEFI in bhyve now. Our platform is bas= ed on 11.3-RELEASE and follows the 11 tree. Do you have any issues with UEFI guests or is it purely bhyveload and/or grub-bhyve?? For issues with vm-bhyve, raise an issue on churchers github page for the project: https://github.com/churchers/vm-bhyve/issues --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue Dec 17 20:10:46 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EDDBF1C8810 for ; Tue, 17 Dec 2019 20:10:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47cq5L65k9z3Dw3 for ; Tue, 17 Dec 2019 20:10:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id D152F1C880F; Tue, 17 Dec 2019 20:10:46 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D117A1C880D for ; Tue, 17 Dec 2019 20:10:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47cq5L5DbLz3Dw1 for ; Tue, 17 Dec 2019 20:10:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AF0BE1B126 for ; Tue, 17 Dec 2019 20:10:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBHKAk9c004043 for ; Tue, 17 Dec 2019 20:10:46 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBHKAknG004042 for virtualization@FreeBSD.org; Tue, 17 Dec 2019 20:10:46 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242666] Stopped VM cant restart again Date: Tue, 17 Dec 2019 20:10:46 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2019 20:10:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242666 Rodney W. Grimes changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |rgrimes@FreeBSD.org --- Comment #6 from Rodney W. Grimes --- (In reply to Bernhard Berger from comment #4) > root@superserver:/data/bhyve # bhyvectl --destroy --vm test-pc > VM:test-pc is not created. I believe bhyvectl is arg order sensitive, try that command as bhyvectl --vm test-pc --destroy --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Wed Dec 18 15:27:37 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 984861E1389 for ; Wed, 18 Dec 2019 15:27:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47dJm93cphz4hcL for ; Wed, 18 Dec 2019 15:27:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 7C0B91E1388; Wed, 18 Dec 2019 15:27:37 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7BC951E1386 for ; Wed, 18 Dec 2019 15:27:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dJm92nlBz4hcJ for ; Wed, 18 Dec 2019 15:27:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5B4BB271 for ; Wed, 18 Dec 2019 15:27:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBIFRbpb042205 for ; Wed, 18 Dec 2019 15:27:37 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBIFRbdr042204 for virtualization@FreeBSD.org; Wed, 18 Dec 2019 15:27:37 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242666] Stopped VM cant restart again Date: Wed, 18 Dec 2019 15:27:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: bernhard@blberger.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2019 15:27:37 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242666 --- Comment #7 from Bernhard Berger --- (In reply to Rodney W. Grimes from comment #6) how many times do you want me to try?=20 Do you read what I write? It's a bug in the BHYVE system.=20 Under certain circumstances, a VM that has not ended cleanly can no longer = be started.=20 In such cases I solve the problem by restarting the host server but that ca= n't be a general solution.=20 It shouldn't be a problem to program a check or repair function in bhyvectl that detects such problems and fixes them.=20=20 With kind regards=20 Translated with www.DeepL.com/Translator (free version) --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Wed Dec 18 16:11:40 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 33FF31E2385 for ; Wed, 18 Dec 2019 16:11:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47dKl00hdSz4l0p for ; Wed, 18 Dec 2019 16:11:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 15CDB1E2384; Wed, 18 Dec 2019 16:11:40 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 158CA1E2383 for ; Wed, 18 Dec 2019 16:11:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dKkz6wy9z4l0m for ; Wed, 18 Dec 2019 16:11:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E9693AEB for ; Wed, 18 Dec 2019 16:11:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBIGBdLw047550 for ; Wed, 18 Dec 2019 16:11:39 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBIGBdtE047549 for virtualization@FreeBSD.org; Wed, 18 Dec 2019 16:11:39 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242666] Stopped VM cant restart again Date: Wed, 18 Dec 2019 16:11:39 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: bernhard@blberger.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2019 16:11:40 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242666 --- Comment #8 from Bernhard Berger --- I'm going to describe what this is all about. It is "not" about a solution on "my" server but about a global solution for= all who use BHYVE.=20 My VM "test-pc" which can no longer be started is only the trigger of a pro= blem that has been occurring to me for a long time to finally send you a bug rep= ort so that you get to know about it and look for a solution. I found out the following: 1. a stopped VM cannot be started anymore if the /dev/nmdm-.1A/B s= till exists 2 A terminated VM can no longer be started if the device in /dev/vmm/ still exists. So, take the problem seriously and forward it to the appropriate developers. I hope that this will be solved in the next version of BHYVE. Yours sincerely --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Wed Dec 18 17:15:49 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0197A1E35D3 for ; Wed, 18 Dec 2019 17:15:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47dM906JKqz4pCF for ; Wed, 18 Dec 2019 17:15:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id D64301E35D2; Wed, 18 Dec 2019 17:15:48 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D4DAE1E35D1 for ; Wed, 18 Dec 2019 17:15:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dM9051bkz4pCD for ; Wed, 18 Dec 2019 17:15:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A309115DF for ; Wed, 18 Dec 2019 17:15:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBIHFm4Y030200 for ; Wed, 18 Dec 2019 17:15:48 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBIHFmgs030198 for virtualization@FreeBSD.org; Wed, 18 Dec 2019 17:15:48 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242666] Stopped VM cant restart again Date: Wed, 18 Dec 2019 17:15:48 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: aleksandr.fedorov@itglobal.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2019 17:15:49 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242666 --- Comment #9 from Aleksandr Fedorov --- Yes, this is the bhyve problem. But we need to know what is happened. Can you run: truss bhyvectl --destroy --vm test-pc. This will help us to find out the cause of the openat() system call error. Also, please add to the attachments a complete list of processes on the hos= t. The output of fstat utility is also very interesting. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Wed Dec 18 19:13:29 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 266A81E4E4E for ; Wed, 18 Dec 2019 19:13:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47dPmn0Fhpz4vCc for ; Wed, 18 Dec 2019 19:13:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 06A961E4E4D; Wed, 18 Dec 2019 19:13:29 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0666F1E4E4C for ; Wed, 18 Dec 2019 19:13:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dPmm6Nhlz4vCb for ; Wed, 18 Dec 2019 19:13:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D69142AF1 for ; Wed, 18 Dec 2019 19:13:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBIJDS9l079439 for ; Wed, 18 Dec 2019 19:13:28 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBIJDStK079438 for virtualization@FreeBSD.org; Wed, 18 Dec 2019 19:13:28 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242666] Stopped VM cant restart again Date: Wed, 18 Dec 2019 19:13:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: bernhard@blberger.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2019 19:13:29 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242666 --- Comment #10 from Bernhard Berger --- Created attachment 210041 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D210041&action= =3Dedit fstat output --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Wed Dec 18 19:14:59 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5FF7E1E4EBB for ; Wed, 18 Dec 2019 19:14:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47dPpW1v3Cz4vGH for ; Wed, 18 Dec 2019 19:14:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 40E2A1E4EBA; Wed, 18 Dec 2019 19:14:59 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 40A8F1E4EB8 for ; Wed, 18 Dec 2019 19:14:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dPpW12sQz4vGG for ; Wed, 18 Dec 2019 19:14:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1F4E32AF6 for ; Wed, 18 Dec 2019 19:14:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBIJEx4m081269 for ; Wed, 18 Dec 2019 19:14:59 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBIJExY7081268 for virtualization@FreeBSD.org; Wed, 18 Dec 2019 19:14:59 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242666] Stopped VM cant restart again Date: Wed, 18 Dec 2019 19:14:59 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: bernhard@blberger.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2019 19:14:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242666 --- Comment #11 from Bernhard Berger --- Created attachment 210042 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D210042&action= =3Dedit Task list --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Wed Dec 18 19:15:22 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AFAB91E4EF8 for ; Wed, 18 Dec 2019 19:15:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47dPpy4H58z4vK1 for ; Wed, 18 Dec 2019 19:15:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 92D4D1E4EF7; Wed, 18 Dec 2019 19:15:22 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 929B91E4EF5 for ; Wed, 18 Dec 2019 19:15:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dPpy3Rtmz4vK0 for ; Wed, 18 Dec 2019 19:15:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 71B5A2AFC for ; Wed, 18 Dec 2019 19:15:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBIJFMLV082656 for ; Wed, 18 Dec 2019 19:15:22 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBIJFM74082655 for virtualization@FreeBSD.org; Wed, 18 Dec 2019 19:15:22 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242666] Stopped VM cant restart again Date: Wed, 18 Dec 2019 19:15:22 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: bernhard@blberger.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2019 19:15:22 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242666 --- Comment #12 from Bernhard Berger --- Created attachment 210043 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D210043&action= =3Dedit Truss file --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Wed Dec 18 19:16:59 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6DA3B1E4F94 for ; Wed, 18 Dec 2019 19:16:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47dPrq2LbHz4vNh for ; Wed, 18 Dec 2019 19:16:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 505611E4F93; Wed, 18 Dec 2019 19:16:59 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 501821E4F92 for ; Wed, 18 Dec 2019 19:16:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dPrq1V6xz4vNg for ; Wed, 18 Dec 2019 19:16:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2E0862B01 for ; Wed, 18 Dec 2019 19:16:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBIJGxMB084604 for ; Wed, 18 Dec 2019 19:16:59 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBIJGxLq084602 for virtualization@FreeBSD.org; Wed, 18 Dec 2019 19:16:59 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242666] Stopped VM cant restart again Date: Wed, 18 Dec 2019 19:16:59 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: bernhard@blberger.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2019 19:16:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242666 --- Comment #13 from Bernhard Berger --- the expenses as requested. i hope it helps Yours sincerely --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Wed Dec 18 23:01:06 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 57D011C87C1 for ; Wed, 18 Dec 2019 23:01:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47dVqQ1gPLz3NCw for ; Wed, 18 Dec 2019 23:01:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 3930F1C87C0; Wed, 18 Dec 2019 23:01:06 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 38F551C87BF for ; Wed, 18 Dec 2019 23:01:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dVqQ0r62z3NCv for ; Wed, 18 Dec 2019 23:01:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 16D7C55F7 for ; Wed, 18 Dec 2019 23:01:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBIN1595033689 for ; Wed, 18 Dec 2019 23:01:05 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBIN15ra033688 for virtualization@FreeBSD.org; Wed, 18 Dec 2019 23:01:05 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 203874] [patch] MSI/MSI-X interrupts don't work in VMware virtual machines Date: Wed, 18 Dec 2019 23:01:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2019 23:01:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D203874 --- Comment #30 from commit-hook@freebsd.org --- A commit references this bug: Author: mp Date: Wed Dec 18 23:00:56 UTC 2019 New revision: 355889 URL: https://svnweb.freebsd.org/changeset/base/355889 Log: Remove VMware MSI-X from the PCI blacklist. First reported against ESXi 5.0, PCI passthrough was not working due to MSI-X issues. However, this issue was fixed via patch releases against ESXi 5.5 and 6.0 in 2016. Given ESXi 5.5 and earlier have been EOL, this patch removes the VMware MSI-X blacklist entries in the quirk table. PR: 203874 Reviewed by: imp, jhb MFC after: 1 month Sponsored by: VMware Differential Revision: https://reviews.freebsd.org/D22819 Changes: head/sys/dev/pci/pci.c --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Wed Dec 18 23:15:08 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F19011C8E0B for ; Wed, 18 Dec 2019 23:15:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47dW7c6CKtz3Nwy for ; Wed, 18 Dec 2019 23:15:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id D509A1C8E0A; Wed, 18 Dec 2019 23:15:08 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D4D0C1C8E08 for ; Wed, 18 Dec 2019 23:15:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dW7c5M2nz3Nwx for ; Wed, 18 Dec 2019 23:15:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B35E25857 for ; Wed, 18 Dec 2019 23:15:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBINF8vN069557 for ; Wed, 18 Dec 2019 23:15:08 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBINF8dl069556 for virtualization@FreeBSD.org; Wed, 18 Dec 2019 23:15:08 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 203874] [patch] MSI/MSI-X interrupts don't work in VMware virtual machines Date: Wed, 18 Dec 2019 23:15:07 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: jhb@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2019 23:15:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D203874 --- Comment #31 from John Baldwin --- Thanks for the digging Mark. It is amusing that the description of the bug= in VMware matches almost exactly the bug Xen had. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Wed Dec 18 23:44:13 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5AD471C9566 for ; Wed, 18 Dec 2019 23:44:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47dWn91pCLz3Pxb for ; Wed, 18 Dec 2019 23:44:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 3DC291C9563; Wed, 18 Dec 2019 23:44:13 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3D8941C9562 for ; Wed, 18 Dec 2019 23:44:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dWn90yvqz3PxY for ; Wed, 18 Dec 2019 23:44:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1CA025DC2 for ; Wed, 18 Dec 2019 23:44:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBINiCAv062429 for ; Wed, 18 Dec 2019 23:44:12 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBINiCK3062428 for virtualization@FreeBSD.org; Wed, 18 Dec 2019 23:44:12 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 203874] [patch] MSI/MSI-X interrupts don't work in VMware virtual machines Date: Wed, 18 Dec 2019 23:44:11 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: mp@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2019 23:44:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D203874 Mark Peek changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|New |Closed --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Dec 19 13:04:00 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EAE161E0249 for ; Thu, 19 Dec 2019 13:04:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47dsX061n8z4fMn for ; Thu, 19 Dec 2019 13:04:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id CEB531E0248; Thu, 19 Dec 2019 13:04:00 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CE7D61E0247 for ; Thu, 19 Dec 2019 13:04:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dsX058wgz4fMm for ; Thu, 19 Dec 2019 13:04:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id ABA1EF146 for ; Thu, 19 Dec 2019 13:04:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBJD40nG038047 for ; Thu, 19 Dec 2019 13:04:00 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBJD40YC038046 for virtualization@FreeBSD.org; Thu, 19 Dec 2019 13:04:00 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242724] bhyve: Unkillable processes stuck in 'STOP' state (vmm::vm_handle_suspend()) Date: Thu, 19 Dec 2019 13:04:00 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: aleksandr.fedorov@itglobal.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2019 13:04:01 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242724 Bug ID: 242724 Summary: bhyve: Unkillable processes stuck in 'STOP' state (vmm::vm_handle_suspend()) Product: Base System Version: CURRENT Hardware: Any OS: Any Status: New Severity: Affects Many People Priority: --- Component: bhyve Assignee: virtualization@FreeBSD.org Reporter: aleksandr.fedorov@itglobal.com On one of our servers, there are three unkillable bhyve processes which stu= ck in 'STOP' state. All has the same symptoms: # procstat -kk 9277 | sort -k 4 PID TID COMM TDNAME KSTACK 9277 102504 bhyve blk-1:0:0-0 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102514 bhyve blk-1:0:0-1 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102752 bhyve blk-1:0:0-2 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102760 bhyve blk-1:0:0-3 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102761 bhyve blk-1:0:0-4 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102763 bhyve blk-1:0:0-5 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102764 bhyve blk-1:0:0-6 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102766 bhyve blk-1:0:0-7 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102767 bhyve blk-3:0-0 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102768 bhyve blk-3:0-1 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102769 bhyve blk-3:0-2 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102770 bhyve blk-3:0-3 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102771 bhyve blk-3:0-4 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102772 bhyve blk-3:0-5 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102773 bhyve blk-3:0-6 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102774 bhyve blk-3:0-7 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102775 bhyve blk-4:0:0-0 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102780 bhyve blk-4:0:0-1 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102887 bhyve blk-4:0:0-2 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102888 bhyve blk-4:0:0-3 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102889 bhyve blk-4:0:0-4 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102890 bhyve blk-4:0:0-5 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102891 bhyve blk-4:0:0-6 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102892 bhyve blk-4:0:0-7 mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102687 bhyve mevent mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102941 bhyve pci-devstat mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102983 bhyve pci-reconfig mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102937 bhyve rfb mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f 9277 102991 bhyve vcpu 0 mi_switch+0xe2 sleepq_timedwait+0x2f msleep_spin_sbt+0xd8 vm_run+0x6c7 vmmdev_ioctl+0x9b2 devfs_ioctl+0xc7 VOP_IOCTL_APV+0x56 vn_ioctl+0x16a devfs_ioctl_f+0x1f kern_ioctl+0x27d sys_ioctl+0x15d amd64_syscall+0x3b0 fast_syscall_common+0x= 101 9277 100782 bhyve vcpu 1 mi_switch+0xe2 thread_suspend_switch+0xd4 thread_single+0x47b sigexit+0x57 postsig+0x2f8 ast+0x327 fast_syscall_common+0x198 9277 100788 bhyve vcpu 2 mi_switch+0xe2 sleepq_timedwait+0x2f msleep_spin_sbt+0xd8 vm_run+0x6c7 vmmdev_ioctl+0x9b2 devfs_ioctl+0xc7 VOP_IOCTL_APV+0x56 vn_ioctl+0x16a devfs_ioctl_f+0x1f kern_ioctl+0x27d sys_ioctl+0x15d amd64_syscall+0x3b0 fast_syscall_common+0x= 101 9277 101385 bhyve vcpu 3 mi_switch+0xe2 sleepq_timedwait+0x2f msleep_spin_sbt+0xd8 vm_run+0x6c7 vmmdev_ioctl+0x9b2 devfs_ioctl+0xc7 VOP_IOCTL_APV+0x56 vn_ioctl+0x16a devfs_ioctl_f+0x1f kern_ioctl+0x27d sys_ioctl+0x15d amd64_syscall+0x3b0 fast_syscall_common+0x= 101 9277 103402 bhyve vcpu 4 mi_switch+0xe2 sleepq_timedwait+0x2f msleep_spin_sbt+0xd8 vm_run+0x6c7 vmmdev_ioctl+0x9b2 devfs_ioctl+0xc7 VOP_IOCTL_APV+0x56 vn_ioctl+0x16a devfs_ioctl_f+0x1f kern_ioctl+0x27d sys_ioctl+0x15d amd64_syscall+0x3b0 fast_syscall_common+0x= 101 9277 102206 bhyve vcpu 5 mi_switch+0xe2 sleepq_timedwait+0x2f msleep_spin_sbt+0xd8 vm_run+0x6c7 vmmdev_ioctl+0x9b2 devfs_ioctl+0xc7 VOP_IOCTL_APV+0x56 vn_ioctl+0x16a devfs_ioctl_f+0x1f kern_ioctl+0x27d sys_ioctl+0x15d amd64_syscall+0x3b0 fast_syscall_common+0x= 101 9277 102932 bhyve vtnet-5:0 tx mi_switch+0xe2 thread_suspend_check+0x297 ast+0x3b9 doreti_ast+0x1f # procstat threads 9277|grep vcpu 9277 100782 bhyve vcpu 1 -1 124 stop - 9277 100788 bhyve vcpu 2 -1 123 stop vmsusp 9277 101385 bhyve vcpu 3 -1 123 stop vmsusp 9277 102206 bhyve vcpu 5 -1 122 stop vmsusp 9277 102991 bhyve vcpu 0 -1 124 stop vmsusp 9277 103402 bhyve vcpu 4 -1 123 stop vmsusp As you can see, One of the VCPU threads exit, the rest are sleeping in ker= nel mode: https://svnweb.freebsd.org/base/head/sys/amd64/vmm/vmm.c?view=3Dmarkup#l1515 The main problem is that if the bhyve process is killed, the threads driving those vCPUs will be gone, resulting in the remaining thread waiting in kernelspace forever: https://svnweb.freebsd.org/base/head/sys/amd64/vmm/vmm.c?view=3Dmarkup#l1507 I found that the same bug was fixed in SmartOS: https://github.com/joyent/illumos-joyent/commit/dce228e4331f185347c3e0325ca= b8a3af72d6410#diff-721b4b2a86bbb8d11c6e01a2995868e6 Unfortunately the site https://smartos.org/bugview/OS-6888 is not available= , so it is impossible to see the detailed description. Can we apply similar fix to our bhyve code? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Dec 19 13:04:50 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3C15A1E0325 for ; Thu, 19 Dec 2019 13:04:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47dsXy0rnDz4fR0 for ; Thu, 19 Dec 2019 13:04:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 1D2111E0322; Thu, 19 Dec 2019 13:04:50 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1CE4D1E0321 for ; Thu, 19 Dec 2019 13:04:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dsXx758gz4fQy for ; Thu, 19 Dec 2019 13:04:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EED35F147 for ; Thu, 19 Dec 2019 13:04:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBJD4ni4038971 for ; Thu, 19 Dec 2019 13:04:49 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBJD4nP9038970 for virtualization@FreeBSD.org; Thu, 19 Dec 2019 13:04:49 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242724] bhyve: Unkillable processes stuck in 'STOP' state (vmm::vm_handle_suspend()) Date: Thu, 19 Dec 2019 13:04:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: aleksandr.fedorov@itglobal.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2019 13:04:50 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242724 --- Comment #1 from Aleksandr Fedorov --- Created attachment 210059 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D210059&action= =3Dedit procstat -k --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Dec 19 13:05:15 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 52C2C1E03FA for ; Thu, 19 Dec 2019 13:05:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47dsYR1Zd4z4fVK for ; Thu, 19 Dec 2019 13:05:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 361751E03F8; Thu, 19 Dec 2019 13:05:15 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 35E1F1E03F7 for ; Thu, 19 Dec 2019 13:05:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dsYR0lgvz4fVJ for ; Thu, 19 Dec 2019 13:05:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 155D0F148 for ; Thu, 19 Dec 2019 13:05:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBJD5EkW039533 for ; Thu, 19 Dec 2019 13:05:14 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBJD5E2E039532 for virtualization@FreeBSD.org; Thu, 19 Dec 2019 13:05:14 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242724] bhyve: Unkillable processes stuck in 'STOP' state (vmm::vm_handle_suspend()) Date: Thu, 19 Dec 2019 13:05:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: aleksandr.fedorov@itglobal.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2019 13:05:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242724 --- Comment #2 from Aleksandr Fedorov --- Created attachment 210060 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D210060&action= =3Dedit prockstat threads --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Dec 19 13:05:55 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4430D1E0480 for ; Thu, 19 Dec 2019 13:05:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47dsZC16CKz4fY5 for ; Thu, 19 Dec 2019 13:05:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 260701E047F; Thu, 19 Dec 2019 13:05:55 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 25CE91E047E for ; Thu, 19 Dec 2019 13:05:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dsZC0BNpz4fY4 for ; Thu, 19 Dec 2019 13:05:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 01EE8F149 for ; Thu, 19 Dec 2019 13:05:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBJD5sYh040299 for ; Thu, 19 Dec 2019 13:05:54 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBJD5swK040298 for virtualization@FreeBSD.org; Thu, 19 Dec 2019 13:05:54 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242724] bhyve: Unkillable processes stuck in 'STOP' state (vmm::vm_handle_suspend()) Date: Thu, 19 Dec 2019 13:05:55 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: aleksandr.fedorov@itglobal.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2019 13:05:55 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242724 --- Comment #3 from Aleksandr Fedorov --- Created attachment 210061 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D210061&action= =3Dedit prockstat signals --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Dec 19 13:06:12 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C25BD1E04BA for ; Thu, 19 Dec 2019 13:06:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47dsZX4pZYz4fbL for ; Thu, 19 Dec 2019 13:06:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id A4D7E1E04B8; Thu, 19 Dec 2019 13:06:12 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A49591E04B6 for ; Thu, 19 Dec 2019 13:06:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dsZX3wqGz4fbK for ; Thu, 19 Dec 2019 13:06:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 819F1F14A for ; Thu, 19 Dec 2019 13:06:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBJD6CdP040616 for ; Thu, 19 Dec 2019 13:06:12 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBJD6CfJ040615 for virtualization@FreeBSD.org; Thu, 19 Dec 2019 13:06:12 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242724] bhyve: Unkillable processes stuck in 'STOP' state (vmm::vm_handle_suspend()) Date: Thu, 19 Dec 2019 13:06:12 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: aleksandr.fedorov@itglobal.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2019 13:06:12 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242724 --- Comment #4 from Aleksandr Fedorov --- Created attachment 210062 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D210062&action= =3Dedit procstat tsignals --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Dec 19 13:07:00 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 510171E056B for ; Thu, 19 Dec 2019 13:07:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47dsbS1X3Xz4ffQ for ; Thu, 19 Dec 2019 13:07:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 3470E1E056A; Thu, 19 Dec 2019 13:07:00 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 343801E0569 for ; Thu, 19 Dec 2019 13:07:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dsbS0bJhz4ffP for ; Thu, 19 Dec 2019 13:07:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0FC3FF14B for ; Thu, 19 Dec 2019 13:07:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBJD6xg3041528 for ; Thu, 19 Dec 2019 13:06:59 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBJD6xkC041527 for virtualization@FreeBSD.org; Thu, 19 Dec 2019 13:06:59 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242724] bhyve: Unkillable processes stuck in 'STOP' state (vmm::vm_handle_suspend()) Date: Thu, 19 Dec 2019 13:07:00 +0000 X-Bugzilla-Reason: AssignedTo CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: aleksandr.fedorov@itglobal.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2019 13:07:00 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242724 Aleksandr Fedorov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jhb@FreeBSD.org, | |kib@FreeBSD.org, | |markj@FreeBSD.org, | |rgrimes@FreeBSD.org, | |virtualization@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Dec 19 15:30:42 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7768E1E32CB for ; Thu, 19 Dec 2019 15:30:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47dwnG2czRz3JXC for ; Thu, 19 Dec 2019 15:30:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 57D091E32CA; Thu, 19 Dec 2019 15:30:42 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 567371E32C9 for ; Thu, 19 Dec 2019 15:30:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dwnG1Qcxz3JX9 for ; Thu, 19 Dec 2019 15:30:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2C4C618A20 for ; Thu, 19 Dec 2019 15:30:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBJFUgkC073954 for ; Thu, 19 Dec 2019 15:30:42 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBJFUghm073953 for virtualization@FreeBSD.org; Thu, 19 Dec 2019 15:30:42 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242724] bhyve: Unkillable processes stuck in 'STOP' state (vmm::vm_handle_suspend()) Date: Thu, 19 Dec 2019 15:30:41 +0000 X-Bugzilla-Reason: CC AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2019 15:30:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242724 --- Comment #5 from Konstantin Belousov --- Show line number for vm_run+0x6c7. What you wrote about vm_handle_rendezvous() might be reasonable but I am not sure that this is the case. Anyway, the line number information should con= firm or deny. If indeed the issue is the infinite wait for rendezvous, it seems to be eas= y to handle it. --=20 You are receiving this mail because: You are on the CC list for the bug. You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Dec 19 16:17:23 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F401B1E45A7 for ; Thu, 19 Dec 2019 16:17:22 +0000 (UTC) (envelope-from vhpc.dist@gmail.com) Received: from mail-il1-x142.google.com (mail-il1-x142.google.com [IPv6:2607:f8b0:4864:20::142]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dxq627HBz3Llt for ; Thu, 19 Dec 2019 16:17:22 +0000 (UTC) (envelope-from vhpc.dist@gmail.com) Received: by mail-il1-x142.google.com with SMTP id b15so5336369iln.3 for ; Thu, 19 Dec 2019 08:17:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to :content-transfer-encoding; bh=yWJEsXtNWZDOzVcWxb9L6pCtXF7AS5GaRwBQfaUqIWU=; b=VnSAHKne+JBtJ3Ms0lXbp9fKrb/q49g45vtij+VHmqz+0whkRbTuarJKcAaMqGVpzd HKHwbt59kAz5znwlM+VckBB7MxQnTIr6WeyVkWyYz3GgqDDp6Ivt6RTEo7xmXDBocEm9 3HUNhEeWbWU/EwbCnCG4MC2lkOGtrGlY8N9bojhYurFI7ZFKmiQqFfd+XjcSgq1ma2Vj YgFWPM8+XSytFr/m7eD/W3JTVaAHHDhao5TyFuSw59eNDryjXDnr/KZRESb0snlBqiaz TwQhvqKc2Z1lvQ04iWPgKYLBiKb1j28+RiDtS3sSLAD6mdXPzR1iHUAN/bCIaH10y7FO QrLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to :content-transfer-encoding; bh=yWJEsXtNWZDOzVcWxb9L6pCtXF7AS5GaRwBQfaUqIWU=; b=Pu5juhxtHizgTwKcmZiLEQVlL+MCVq2MzwkMmp35VdEFTyQTQbBbKNqtfAeq4XoJca 5iG7k+ApqwrlN3RZ221MxOP1YbK6Lcx5BtNBK1Ekhg1frmYxVsvqrSgNkJuw7mR83H3I Ylqu1Mgm77ROQK2gERTVWiQ9UvZpWatPl7XWgzLb5SHlWyRuJURs8UA7WdRLrp/ix4Hq KBQaZqXlekZGvEfrwLbFo1f77Qsa7+yOboXPztoCg/D9oGC50+AZi3hJ3AwXw+/mb/w5 ssON3u0ripWWtdRbbZ/zvRRkuOX3VPT48UabkI+epLqZv722+AmWdnNDvgpIBjJ5XLbq 16AA== X-Gm-Message-State: APjAAAVFGUw6OzNOK3AK1UghGIZDeS35t0XmHR0ei057rv6hXxwM4PVx WV2Waplos766W9IXsm7QhPVL5EUz71JgoLDeC2zhdEmu5fw= X-Google-Smtp-Source: APXvYqyFQR5adXn81qegYpJsseo04Qzdx5ZGwgXfSv+jgD5dU3wqXLbdMqYOHwzNwJdUxEoPgTtvIM0KmkSvkNB6PNU= X-Received: by 2002:a92:cb10:: with SMTP id s16mr7257013ilo.176.1576772240790; Thu, 19 Dec 2019 08:17:20 -0800 (PST) MIME-Version: 1.0 From: VHPC 20 Date: Thu, 19 Dec 2019 17:17:09 +0100 Message-ID: Subject: CfP VHPC20: HPC Containers-Kubernetes To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 47dxq627HBz3Llt X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=VnSAHKne; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of vhpcdist@gmail.com designates 2607:f8b0:4864:20::142 as permitted sender) smtp.mailfrom=vhpcdist@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_DN_NONE(0.00)[]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(0.00)[ip: (1.92), ipnet: 2607:f8b0::/32(-2.18), asn: 15169(-1.90), country: US(-0.05)]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2.4.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2019 16:17:23 -0000 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D CALL FOR PAPERSa 15th Workshop on Virtualization in High-Performance Cloud Computing (VHPC 20) held in conjunction with the International Supercomputing Conference - High Performance, June 21-25, 2020, Frankfurt, Germany. (Springer LNCS Proceedings) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Date: June 25, 2020 Workshop URL: vhpc[dot]org Abstract registration Deadline: Jan 31st, 2020 Paper Submission Deadline: Apr 05th, 2020 Springer LNCS Call for Papers Containers and virtualization technologies constitute key enabling factors for flexible resource management in modern data centers, and particularly in cloud environments. Cloud providers need to manage complex infrastructures in a seamless fashion to support the highly dynamic and heterogeneous workloads and hosted applications customers deploy. Similarly, HPC environments have been increasingly adopting techniques that enable flexible management of vast computing and networking resources, close to marginal provisioning cost, which is unprecedented in the history of scientific and commercial computing. Most recently, Function as a Service (Faas) and Serverless computing, utilizing lightweight VMs-containers widens the spectrum of applications that can be deployed in a cloud environment, especially in an HPC context. Here, HPC-provided services can be become accessible to distributed workloads outside of large cluster environments. Various virtualization-containerization technologies contribute to the overall picture in different ways: machine virtualization, with its capability to enable consolidation of multiple under=C2=ADutilized servers with heterogeneous software and operating systems (OSes), and its capability to live=C2=AD-migrate a fully operating virtual machine (VM) with a very short downtime, enables novel and dynamic ways to manage physical servers; OS-=C2=ADlevel virtualization (i.e., containerization), with its capability to isolate multiple user=C2=AD-space environments and to allow for their co=C2=ADexistence within the same OS kernel, promises to provide many of the advantages of machine virtualization with high levels of responsiveness and performance; lastly, unikernels provide for many virtualization benefits with a minimized OS/library surface. I/O Virtualization in turn allows physical network interfaces to take traffic from multiple VMs or containers; network virtualization, with its capability to create logical network overlays that are independent of the underlying physical topology is furthermore enabling virtualization of HPC infrastructures. Publication Accepted papers will be published in a Springer LNCS proceedings volume. Topics of Interest The VHPC program committee solicits original, high-quality submissions related to virtualization across the entire software stack with a special focus on the intersection of HPC, containers-virtualization and the cloud. Major Topics: - HPC workload orchestration (Kubernetes) - Kubernetes HPC batch - HPC Container Environments Landscape - HW Heterogeneity - Container ecosystem (Docker alternatives) - Networking - Lightweight Virtualization - Unikernels / LibOS - State-of-the-art processor virtualization (RISC-V, EPI) - Containerizing HPC Stacks/Apps/Codes: Climate model containers each major topic encompassing design/architecture, management, performance management, modeling and configuration/tooling. Specifically, we invite papers that deal with the following topics: - HPC orchestration (Kubernetes) - Virtualizing Kubernetes for HPC - Deployment paradigms - Multitenancy - Serverless - Declerative data center integration - Network provisioning - Storage - OCI i.a. images - Isolation/security - HW Accelerators, including GPUs, FPGAs, AI, and others - State-of-practice/art, including transition to cloud - Frameworks, system software - Programming models, runtime systems, and APIs to facilitate cloud adoption - Edge use-cases - Application adaptation, success stories - Kubernetes Batch - Scheduling, job management - Execution paradigm - workflow - Data management - Deployment paradigm - Multi-cluster/scalability - Performance improvement - Workflow / execution paradigm - Podman: end-to-end Docker alternative container environment & use-cases - Creating, Running containers as non-root (rootless) - Running rootless containers with MPI - Container live migration - Running containers in restricted environments without setuid - Networking - Software defined networks and network virtualization - New virtualization NICs/Nitro alike ASICs for the data center? - Kubernetes SDN policy (Calico i.a.) - Kubernetes network provisioning (Flannel i.a.) - Lightweight Virtualization - Micro VMMs (Rust-VMM, Firecracker, solo5) - Xen - Nitro hypervisor (KVM) - RVirt - Cloud Hypervisor - Unikernels / LibOS - HPC Storage in Virtualization - HPC container storage - Cloud-native storage - Hypervisors in storage virtualization - Processor Virtualization - RISC-V hypervisor extensions - RISC-V Hypervisor ports - EPI - Composable HPC microservices - Containerizing Scientific Codes - Building - Deploying - Securing - Storage - Monitoring - Use case for containerizing HPC codes: Climate model containers for portability, reproducibility, traceability, immutability, provenance, data & software preservation The Workshop on Virtualization in High=C2=AD-Performance Cloud Computing (VHPC) aims to bring together researchers and industrial practitioners facing the challenges posed by virtualization in order to foster discussion, collaboration, mutual exchange of knowledge and experience, enabling research to ultimately provide novel solutions for virtualized computing systems of tomorrow. The workshop will be one day in length, composed of 20 min paper presentations, each followed by 10 min discussion sections, plus lightning talks that are limited to 5 minutes. Presentations may be accompanied by interactive demonstrations. Important Dates Jan 31st, 2020 - Abstract Apr 5th, 2020 - Paper submission deadline (Springer LNCS) Apr 26th, 2020 - Acceptance notification June 25th, 2020 - Workshop Day July 10th, 2020 - Camera-ready version due Chair Michael Alexander (chair), BOKU, Vienna, Austria Anastassios Nanos (co-=C2=ADchair), Sunlight.io, UK Program committee Stergios Anastasiadis, University of Ioannina, Greece Paolo Bonzini, Redhat, Italy Jakob Blomer, CERN, Europe Eduardo C=C3=A9sar, Universidad Autonoma de Barcelona, Spain Taylor Childers, Argonne National Laboratory, USA Stephen Crago, USC ISI, USA Tommaso Cucinotta, St. Anna School of Advanced Studies, Italy Fran=C3=A7ois Diakhat=C3=A9 CEA DAM Ile de France, France Kyle Hale, Northwestern University, USA Brian Kocoloski, Washington University, USA John Lange, University of Pittsburgh, USA Giuseppe Lettieri, University of Pisa, Italy Klaus Ma, Huawei, China Alberto Madonna, Swiss National Supercomputing Center, Switzerland Nikos Parlavantzas, IRISA, France Anup Patel, Western Digital, USA Kevin Pedretti, Sandia National Laboratories, USA Amer Qouneh, Western New England University, USA Carlos Rea=C3=B1o, Queen=E2=80=99s University Belfast, UK Adrian Reber, Redhat, Germany Riccardo Rocha, CERN, Europe Borja Sotomayor, University of Chicago, USA Jonathan Sparks, Cray, USA Kurt Tutschku, Blekinge Institute of Technology, Sweden John Walters, USC ISI, USA Yasuhiro Watashiba, Osaka University, Japan Chao-Tung Yang, Tunghai University, Taiwan Paper Submission-Publication Papers submitted to the workshop will be reviewed by at least two members of the program committee and external reviewers. Submissions should include abstract, keywords, the e-mail address of the corresponding author, and must not exceed 10 pages, including tables and figures at a main font size no smaller than 11 point. Submission of a paper should be regarded as a commitment that, should the paper be accepted, at least one of the authors will register and attend the conference to present the work. Accepted papers will be published in a Springer LNCS volume. The format must be according to the Springer LNCS Style. Initial submissions are in PDF; authors of accepted papers will be requested to provide source files. Abstract, Paper Submission Link: edas[dot]info/newPaper.php?c=3D26973 Lightning Talks Lightning Talks are non-paper track, synoptical in nature and are strictly limited to 5 minutes. They can be used to gain early feedback on ongoing research, for demonstrations, to present research results, early research ideas, perspectives and positions of interest to the community. Submit abstract via the main submission link. General Information The workshop is one day in length and will be held in conjunction with the International Supercomputing Conference - High Performance (ISC) 2019, June 21-25, Frankfurt, Germany. From owner-freebsd-virtualization@freebsd.org Thu Dec 19 19:36:01 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0FA971E9684 for ; Thu, 19 Dec 2019 19:36:01 +0000 (UTC) (envelope-from lauren@copierquotes.site) Received: from s177327.cloud.flynet.pro (alias26s177327.cloud.flynet.pro [193.47.34.40]) by mx1.freebsd.org (Postfix) with ESMTP id 47f2DG2z2Rz43cB for ; Thu, 19 Dec 2019 19:35:57 +0000 (UTC) (envelope-from lauren@copierquotes.site) Received: from alias5s177136.cloud.flynet.pro (unknown [85.117.233.34]) by s177327.cloud.flynet.pro (Postfix) with ESMTP id F2F976BABB for ; Thu, 19 Dec 2019 21:22:13 +0200 (IST) Received: from m1.bizquotes.click (unknown [10.20.15.1]) by alias5s177136.cloud.flynet.pro (Postfix) with ESMTP id AE8F5419D9 for ; Thu, 19 Dec 2019 21:22:10 +0200 (IST) Received: from list.bizquotes.click (unknown [192.168.100.137]) by m1.bizquotes.click (Postfix) with ESMTP id D8DD7101B4 for ; Thu, 19 Dec 2019 19:30:04 +0200 (IST) DKIM-Filter: OpenDKIM Filter v2.11.0 m1.bizquotes.click D8DD7101B4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bizquotes.org; s=zmail; t=1576776605; bh=XpK6zNwQ+up4FW4Nb87w8NMIjWxn50v63eRsPBtF5VI=; h=To:Subject:Date:From:Reply-To:List-Unsubscribe:From; b=MYMHY/c5iFKGQl9RxIye6t3Tk6PBJ2IV4hQNbEksMrXVbdqXEYA8x224citbeYGMT ds8ISN9T1b1DBm6VhvvEgdM++kysC4hMv6afsHNu4i0hs5s7TPuw+H2zrAFL0FqaCR OkNfWfOvPooREvyMc21xZLDSzpE+m994bAVKTa/I= To: freebsd-virtualization@freebsd.org Subject: Copier Quotes for Charlie's Choice Message-ID: Date: Thu, 19 Dec 2019 16:49:51 +0000 From: "Lauren Harris" Reply-To: lauren@copierquotes.site MIME-Version: 1.0 X-Mailer-LID: 41,41 X-Mailer-RecptId: 34790047 X-Mailer-SID: 26078 X-Mailer-Sent-By: 1 X-Rspamd-Queue-Id: 47f2DG2z2Rz43cB X-Spamd-Bar: +++++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bizquotes.org header.s=zmail header.b=MYMHY/c5; dmarc=none; spf=pass (mx1.freebsd.org: domain of lauren@copierquotes.site designates 193.47.34.40 as permitted sender) smtp.mailfrom=lauren@copierquotes.site X-Spamd-Result: default: False [5.26 / 15.00]; ARC_NA(0.00)[]; HAS_REPLYTO(0.00)[lauren@copierquotes.site]; R_DKIM_ALLOW(-0.20)[bizquotes.org:s=zmail]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+Ip4:193.47.34.0/24]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; HAS_PHPMAILER_SIG(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[bizquotes.org:+]; NEURAL_SPAM_LONG(1.00)[0.999,0]; DMARC_NA(0.00)[copierquotes.site]; NEURAL_SPAM_MEDIUM(0.98)[0.980,0]; IP_SCORE(0.69)[ipnet: 193.47.34.0/24(2.08), asn: 51724(1.33), country: RU(0.01)]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; INTRODUCTION(2.00)[]; HAS_INTERSPIRE_SIG(1.00)[]; ASN(0.00)[asn:51724, ipnet:193.47.34.0/24, country:RU]; MIME_TRACE(0.00)[0:+,1:+,2:~]; REPLYTO_ADDR_EQ_FROM(0.00)[] Content-Type: text/plain; format=flowed; charset="UTF-8" Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2019 19:36:01 -0000 Hi, My name is Lauren and I am with CopierQuotes[.]site which keeps businesses such as Charlie's Choice up to date on the best deals offered on photocopiers. Benefits of using our quote service: * Comfort: Fully equip your office in a cheap and easy way. * Budget effectively: Lease the latest models, with no down payment required. Anticipate your monthly expenses in order to help you balance your costs better. * Flexibility: The option to lease or buy a photocopier provides flexibility to business owners. The following information will be needed in order to compose a Quote: * Full Name * Company Name * Phone number * Postal Code * What type of copier are you interested in? I encourage you to check out our pricing and upgrade your office by replying with your needs and details. Best regards, Lauren Harris Lauren@CopierQuotes.site Unsubscribe by reply back "REMOVE" From owner-freebsd-virtualization@freebsd.org Thu Dec 19 20:57:47 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8CFFB1EB13C for ; Thu, 19 Dec 2019 20:57:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47f42g3BJKz46yB for ; Thu, 19 Dec 2019 20:57:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 6B6F71EB13B; Thu, 19 Dec 2019 20:57:47 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6A1FE1EB139 for ; Thu, 19 Dec 2019 20:57:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47f42g27P9z46y9 for ; Thu, 19 Dec 2019 20:57:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 449691C4ED for ; Thu, 19 Dec 2019 20:57:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBJKvlqw089934 for ; Thu, 19 Dec 2019 20:57:47 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBJKvlwe089933 for virtualization@FreeBSD.org; Thu, 19 Dec 2019 20:57:47 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242724] bhyve: Unkillable processes stuck in 'STOP' state (vmm::vm_handle_suspend()) Date: Thu, 19 Dec 2019 20:57:46 +0000 X-Bugzilla-Reason: CC AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2019 20:57:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242724 --- Comment #6 from Konstantin Belousov --- Try https://reviews.freebsd.org/D22881. I did not tested that. --=20 You are receiving this mail because: You are on the CC list for the bug. You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Fri Dec 20 07:26:42 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2799E1D0305 for ; Fri, 20 Dec 2019 07:26:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47fL0L0Jsgz4btZ for ; Fri, 20 Dec 2019 07:26:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 0A9081D0304; Fri, 20 Dec 2019 07:26:42 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0A4DE1D0303 for ; Fri, 20 Dec 2019 07:26:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47fL0K6Rkdz4btY for ; Fri, 20 Dec 2019 07:26:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D402B235F9 for ; Fri, 20 Dec 2019 07:26:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBK7Qflv033950 for ; Fri, 20 Dec 2019 07:26:41 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBK7QfW5033939 for virtualization@FreeBSD.org; Fri, 20 Dec 2019 07:26:41 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242724] bhyve: Unkillable processes stuck in 'STOP' state (vmm::vm_handle_suspend()) Date: Fri, 20 Dec 2019 07:26:40 +0000 X-Bugzilla-Reason: AssignedTo CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: andrey@bsdnir.info X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Dec 2019 07:26:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242724 Andrey Fesenko changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |andrey@bsdnir.info --- Comment #7 from Andrey Fesenko --- https://smartos.org/bugview/OS-6888 yandex save :) https://yandexwebcache.net/yandbtm?lang=3Den&fmode=3Dinject&tm=3D1576826734= &tld=3Dru&la=3D1560740864&text=3DOS-6888%20bhyve%20wedged%20on%20vioapic%20= write&url=3Dhttps%3A%2F%2Fsmartos.org%2Fbugview%2FOS-6888&l10n=3Dru&mime=3D= html&sign=3D0c8dee200fcb04690f50b361149607d8&keyno=3D0 --=20 You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug.= From owner-freebsd-virtualization@freebsd.org Fri Dec 20 14:11:15 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E6E341DAC3E for ; Fri, 20 Dec 2019 14:11:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47fVz75rkGz40rf for ; Fri, 20 Dec 2019 14:11:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id C898C1DAC3D; Fri, 20 Dec 2019 14:11:15 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C74EE1DAC3C for ; Fri, 20 Dec 2019 14:11:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47fVz74qBVz40rd for ; Fri, 20 Dec 2019 14:11:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A0DBB27E6D for ; Fri, 20 Dec 2019 14:11:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBKEBFZs043545 for ; Fri, 20 Dec 2019 14:11:15 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBKEBFb8043542 for virtualization@FreeBSD.org; Fri, 20 Dec 2019 14:11:15 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 240945] [hyper-v] [netvsc] hn network driver incorrectly detects baudrate Date: Fri, 20 Dec 2019 14:11:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: geostel@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? mfc-stable12? X-Bugzilla-Changed-Fields: version Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Dec 2019 14:11:16 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D240945 Nova_Logic changed: What |Removed |Added ---------------------------------------------------------------------------- Version|11.3-RELEASE |12.1-RELEASE --- Comment #1 from Nova_Logic --- also it's not working in freebsd 12.1,seems it affects every freebsd installation on hyper-v --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Fri Dec 20 14:12:41 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 417CF1DACC1 for ; Fri, 20 Dec 2019 14:12:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47fW0n144Kz415S for ; Fri, 20 Dec 2019 14:12:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 242861DACC0; Fri, 20 Dec 2019 14:12:41 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 23EC61DACBF for ; Fri, 20 Dec 2019 14:12:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47fW0n09B5z415R for ; Fri, 20 Dec 2019 14:12:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 014E827EE4 for ; Fri, 20 Dec 2019 14:12:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBKECeHt053741 for ; Fri, 20 Dec 2019 14:12:40 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBKECeSl053740 for virtualization@FreeBSD.org; Fri, 20 Dec 2019 14:12:40 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 240945] [hyper-v] [netvsc] hn network driver incorrectly detects baudrate Date: Fri, 20 Dec 2019 14:12:40 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: geostel@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: mfc-stable12? mfc-stable11? mfc-stable10? exp-run? X-Bugzilla-Changed-Fields: flagtypes.name Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Dec 2019 14:12:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D240945 Nova_Logic changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|maintainer-feedback? |mfc-stable11?, | |mfc-stable10?, exp-run? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Fri Dec 20 14:13:33 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 124B71DAD22 for ; Fri, 20 Dec 2019 14:13:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47fW1m6myhz4181 for ; Fri, 20 Dec 2019 14:13:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id E89BA1DAD21; Fri, 20 Dec 2019 14:13:32 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E86271DAD20 for ; Fri, 20 Dec 2019 14:13:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47fW1m5w4Qz417y for ; Fri, 20 Dec 2019 14:13:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C67E027EE8 for ; Fri, 20 Dec 2019 14:13:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBKEDWk5054744 for ; Fri, 20 Dec 2019 14:13:32 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBKEDWH4054743 for virtualization@FreeBSD.org; Fri, 20 Dec 2019 14:13:32 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 240945] [hyper-v] [netvsc] hn network driver incorrectly detects baudrate Date: Fri, 20 Dec 2019 14:13:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: antoine@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: mfc-stable12? mfc-stable11? mfc-stable10? X-Bugzilla-Changed-Fields: flagtypes.name Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Dec 2019 14:13:33 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D240945 Antoine Brodin changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|exp-run? | --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Fri Dec 20 14:37:46 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id ACA7B1DB2A2 for ; Fri, 20 Dec 2019 14:37:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47fWYk49Nyz4284 for ; Fri, 20 Dec 2019 14:37:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 8D5AD1DB2A1; Fri, 20 Dec 2019 14:37:46 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8D1A71DB2A0 for ; Fri, 20 Dec 2019 14:37:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47fWYk3Hymz4282 for ; Fri, 20 Dec 2019 14:37:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 683CA2B1 for ; Fri, 20 Dec 2019 14:37:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBKEbkNo031175 for ; Fri, 20 Dec 2019 14:37:46 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBKEbk3F031172 for virtualization@FreeBSD.org; Fri, 20 Dec 2019 14:37:46 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242724] bhyve: Unkillable processes stuck in 'STOP' state (vmm::vm_handle_suspend()) Date: Fri, 20 Dec 2019 14:37:45 +0000 X-Bugzilla-Reason: AssignedTo CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Dec 2019 14:37:46 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242724 --- Comment #8 from Konstantin Belousov --- (In reply to Andrey Fesenko from comment #7) I have no idea what did you tried to say. smartos link opens up as a blank page, and the yandex link returns 404. --=20 You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug.= From owner-freebsd-virtualization@freebsd.org Fri Dec 20 16:03:58 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 68C401DCEC9 for ; Fri, 20 Dec 2019 16:03:58 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from wout1-smtp.messagingengine.com (wout1-smtp.messagingengine.com [64.147.123.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47fYT94WyMz46PR for ; Fri, 20 Dec 2019 16:03:57 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 99DB9632 for ; Fri, 20 Dec 2019 11:03:55 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Fri, 20 Dec 2019 11:03:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h= date:from:to:subject:message-id:mime-version:content-type; s= fm1; bh=OkTvgFOnv/R9xrPVjKNbGe+E/+5uR1efvzzhRc9bEVU=; b=hS9EGjp2 n/jk6z4uYkJh3x2Ih8HDj8ePvL9k0+3etCy6AkY1x6whyXsyUVBwxzeDb1+inTto itbWnH0WHuKMq3R7UbvFc1tMehLmlGWgkWOXg3NkuJcRQGAtv/XYSMNQmHewVE5p df+WkqcsttlJpZxXlAVowQP+lOAMzQv0PGvirtTwvvK+9HG7suiegZ8qZMh7oqqA FCylfNQEuL2C8NqPKdBEA1xZFSYqKDcEJS3l7OmV3E/1AXKYC2hTevhHcjMcSeWM Dij1hbaoQzpLrfhtEu3ufaPn+9bfI7EsoJFZAYTRLTC8So6RJAXY3+N8WSrS4yH4 901faFwZc+uUAQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:message-id :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; bh=OkTvgFOnv/R9xrPVjKNbGe+E/+5uR 1efvzzhRc9bEVU=; b=KkauZ/NqtOIyQoFu1haKSWXESA5CDvSgHmkCm2uIJ/3w5 pMAViv23Szs90qcrrnwIg3ZT7Cl6axc/tPpKxe9SypFLEjuN9IIJaxOKNNiCr7rw OS5qvqB4Dhb5hPrIdv9ggY2KNUI+rUgIQzFoEQLBuDj01TqkLYqy0W0Nae50oFYQ 2pSDichTUJpMisPNVpBhbqmtfr47/2vRrGt7elRtt32i4xtFxzWpRQMQ51tmhxnH ItUWX/0vF0J92Py9hMAuK4id2salVRt355j24YU2sOvwEE329+GDU+keaA+VQQv8 SkRWER+8hd9IVCl5SewbfylQdhW8mq0xq81WEFKFQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedrvddufedgkedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkgggtuggfsehgtderre dtredvnecuhfhrohhmpehtvggthhdqlhhishhtshcuoehtvggthhdqlhhishhtshesiiih gihsthdrnhgvtheqnecukfhppeekvddrjedtrdeluddrleelnecurfgrrhgrmhepmhgrih hlfhhrohhmpehtvggthhdqlhhishhtshesiiihgihsthdrnhgvthenucevlhhushhtvghr ufhiiigvpedt X-ME-Proxy: Received: from bastion.zyxst.net (bastion.zyxst.net [82.70.91.99]) by mail.messagingengine.com (Postfix) with ESMTPA id 6870680059 for ; Fri, 20 Dec 2019 11:03:54 -0500 (EST) Date: Fri, 20 Dec 2019 16:03:25 +0000 From: tech-lists To: freebsd-virtualization@freebsd.org Subject: bhyve device emulation initialization error Message-ID: <20191220160325.GI63749@bastion.zyxst.net> Mail-Followup-To: freebsd-virtualization@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="BghK6+krpKHjj+jk" Content-Disposition: inline User-Agent: Mutt/1.12.2 (2019-09-21) X-Rspamd-Queue-Id: 47fYT94WyMz46PR X-Spamd-Bar: -------- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=zyxst.net header.s=fm1 header.b=hS9EGjp2; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=KkauZ/Nq; dmarc=none; spf=pass (mx1.freebsd.org: domain of tech-lists@zyxst.net designates 64.147.123.24 as permitted sender) smtp.mailfrom=tech-lists@zyxst.net X-Spamd-Result: default: False [-8.19 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[zyxst.net:s=fm1,messagingengine.com:s=fm1]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.24]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; DMARC_NA(0.00)[zyxst.net]; DKIM_TRACE(0.00)[zyxst.net:+,messagingengine.com:+]; SIGNED_PGP(-2.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[24.123.147.64.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-3.49)[ip: (-9.81), ipnet: 64.147.123.0/24(-4.92), asn: 11403(-2.68), country: US(-0.05)] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Dec 2019 16:03:58 -0000 --BghK6+krpKHjj+jk Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, I'm trying to install an old 11.1-release like this: vmrun.sh -c 4 -m 32768M -tap2 -d test-host.img -i -I FreeBSD-11.1-RELEASE-amd64-dvd1.iso test-host I get the boot screen and countdown but it fails to boot like this: /boot/kernel/kernel text=3D0x14972f8 data=3D0x1384c0+0x4c15e8 syms=3D[0x8+0x15e8b0+0x8+0x178422] Booting... device emulation initialization error: No such file or directory =2E..and returns me to the command prompt. Just in case it was an outated bhyve on this 12-stable r354992 system, I did this as root, on r355929 /usr/src: # cd /usr/src/usr.sbin/bhyve && make && make install installed without error tried the vmrun.sh command again, same result. Is there a way around this? thanks, --=20 J. --BghK6+krpKHjj+jk Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEE8n3tWhxW11Ccvv9/s8o7QhFzNAUFAl388N8ACgkQs8o7QhFz NAUEkw/+K7Wh1Su62kAgkSxknt02gSKSRSk00LADrFj0DX44AxTUdd6Ss8OztKu7 RQ2OhVxULs62LnR3fMXsVMgm0RajpYZV/6g3FJ6p+HyCn/kI4WU/ZHcpa6bm/82S JyhdsqRNwcU3r4ctUsMcaJB3Feygr/wbCxMRAPUNuMF+mzDOfWwjZRhpyRlXwbKC 8t0AT7m1gDiVoCkCTuLqojs0IrSlWH77inm5307M7YUREjfumB8rX3myt7fQZUZA 9w+q41BDcsg4tiFcSancMiQDr/y6dUHQidVCG06qwCOdoM9iqBiOScji6ojApB4J r/4wvz68GuRx/XnCF6De3bFBU/3nxP8QP1FuB+X8NU9rz//KRpCG4adR/o2o/F42 SyjVuv8EZG2gr4igkB8oKd7qw8zorf5X0VcbHFXVKNSpvQPB8yEI0iZUfdiE+pgv OQI7sjCxCPBeCM6vn1dI1fh+UfzaTbAZQHScUR9RHlfhaLc2G0j7Pt7GBGVz9sHb K05FiMH0rJddCl3+Z+mtBVYELUozsNNleSBJLlE1byg+bmuxhzR6qKEgFvb5nnoC 4oJal9TpVO41JWm0igpNjQ/TCfbA6qGFmZH3AY09b2qinY+w9W+NqlF6UMbEJ0JW 2uYZjQd52Hq0hVTUI2FcJEejaOOKcuINsOmfekkIvJLOhjX6+CY= =JeKs -----END PGP SIGNATURE----- --BghK6+krpKHjj+jk-- From owner-freebsd-virtualization@freebsd.org Fri Dec 20 20:02:29 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2A2651E1FBD for ; Fri, 20 Dec 2019 20:02:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47ffmP0PdGz4L7G for ; Fri, 20 Dec 2019 20:02:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 0DDD91E1FBC; Fri, 20 Dec 2019 20:02:29 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0DA031E1FBB for ; Fri, 20 Dec 2019 20:02:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47ffmN6hdpz4L7D for ; Fri, 20 Dec 2019 20:02:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E154A41C8 for ; Fri, 20 Dec 2019 20:02:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id xBKK2SVL065601 for ; Fri, 20 Dec 2019 20:02:28 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id xBKK2SHZ065600 for virtualization@FreeBSD.org; Fri, 20 Dec 2019 20:02:28 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242724] bhyve: Unkillable processes stuck in 'STOP' state (vmm::vm_handle_suspend()) Date: Fri, 20 Dec 2019 20:02:28 +0000 X-Bugzilla-Reason: CC AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: andrey@bsdnir.info X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Dec 2019 20:02:29 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242724 --- Comment #9 from Andrey Fesenko --- (In reply to Konstantin Belousov from comment #8) hm... yandex cach not permanent link :( original page bug OS-6888 long time not work, now working, save https://web.archive.org/web/20191220195714/https://smartos.org/bugview/OS-6= 888 if smartos bugview down again. --=20 You are receiving this mail because: You are on the CC list for the bug. You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Sat Dec 21 05:49:11 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3265D1C8146 for ; Sat, 21 Dec 2019 05:49:11 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47fvnL0D8Rz3QGP for ; Sat, 21 Dec 2019 05:49:09 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id xBL5n1Xr013454; Fri, 20 Dec 2019 21:49:01 -0800 (PST) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id xBL5n0Dp013453; Fri, 20 Dec 2019 21:49:01 -0800 (PST) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201912210549.xBL5n0Dp013453@gndrsh.dnsmgr.net> Subject: Re: bhyve device emulation initialization error In-Reply-To: <20191220160325.GI63749@bastion.zyxst.net> To: tech-lists Date: Fri, 20 Dec 2019 21:49:00 -0800 (PST) CC: freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 47fvnL0D8Rz3QGP X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd-rwg@gndrsh.dnsmgr.net has no SPF policy when checking 69.59.192.140) smtp.mailfrom=freebsd-rwg@gndrsh.dnsmgr.net X-Spamd-Result: default: False [-0.56 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.82)[-0.821,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-0.67)[-0.672,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dnsmgr.net]; AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.04)[ip: (0.14), ipnet: 69.59.192.0/19(0.07), asn: 13868(0.02), country: US(-0.05)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Dec 2019 05:49:11 -0000 -- Start of PGP signed section. > Hi, > > I'm trying to install an old 11.1-release like this: > > vmrun.sh -c 4 -m 32768M -tap2 -d test-host.img -i -I ^^^^^ is this a typo? Should be -t tap2 > FreeBSD-11.1-RELEASE-amd64-dvd1.iso test-host > > I get the boot screen and countdown but it fails to boot like this: > > /boot/kernel/kernel text=0x14972f8 data=0x1384c0+0x4c15e8 > syms=[0x8+0x15e8b0+0x8+0x178422] > Booting... > device emulation initialization error: No such file or directory > > ...and returns me to the command prompt. > > Just in case it was an outated bhyve on this 12-stable r354992 system, I did > this as root, on r355929 /usr/src: > > # cd /usr/src/usr.sbin/bhyve && make && make install > > installed without error > > tried the vmrun.sh command again, same result. > > Is there a way around this? > > thanks, > > > -- > J. -- End of PGP section, PGP failed! -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Sat Dec 21 12:54:43 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 66C601D37F8 for ; Sat, 21 Dec 2019 12:54:43 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47g5DL2hRfz4FZ7 for ; Sat, 21 Dec 2019 12:54:42 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 112AE22139 for ; Sat, 21 Dec 2019 07:54:41 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Sat, 21 Dec 2019 07:54:41 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h= date:from:to:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm1; bh=Dnh5RLN+pWNWXPBj7PeI8xZeI2L 7DtTbGGR4faiz53w=; b=ItOZIA6m2/JV5TYj1SD4JJxOGMJcbX7OPO6VwDwXbVS 2Qmfzya2S9DMDpUaNIHwn2FIGkfpQrj+BQGDwGswdMrAeh4/Oj/COo1xYwFlA6n4 AyZ53cD0bd3UAkky6srmPnRwW/TDHqTNke6vO+iyZDtcdv4TFxwB7qjk1AQEkfB/ 0mMe8vPMT/8j9i1Ljd16BDpBI4qolnpkXnv05dI2BuL/VIJP6HvJhhKfAP9kaXkP UFm2wjEyFUujg00sSyu/Tqxo2nUC8mh4rbE8JtcwQZ5WGYSuWDNpMA6x6Wln8Qxs c/mHurdZY6+vnaKVXMqq/S95N2NPXB7XGJGe0eYQ9sw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=Dnh5RL N+pWNWXPBj7PeI8xZeI2L7DtTbGGR4faiz53w=; b=CqSiWAyR+228E7cGCv/Thk Xeww9E/Y+6an/OBCGCcLWX4xOWFZcyONU1fTnxxErwPyS/MqHVzGpLFP/qsqspa2 nb2H8hULVSDRTeHnPlqvIxAXJ3hvia54LKh9i1/pw1E8cwLJVlkwAcs0TM9jrnEq h1HtCip9Vug4YE0IAfmyX5WaULbSX0lRrtqUHQfR2AO51xpxe/zO63OFruhnGitC 0toN5OzVMz2qNT3HFa/oRJWq7FFS2LWl3LUq9CkU/o6F30oKZtCxxGTJqMW8GRWU fBgykcunQtt+6qGYCqoa4qW+NbV1JJuGuwGZOmvagtl2z6Nr3Zd9bCQT9qd6lrfg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedrvdduhedggeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujggfsehgtd erredtreejnecuhfhrohhmpehtvggthhdqlhhishhtshcuoehtvggthhdqlhhishhtshes iiihgihsthdrnhgvtheqnecukfhppeekvddrjedtrdeluddrleelnecurfgrrhgrmhepmh grihhlfhhrohhmpehtvggthhdqlhhishhtshesiiihgihsthdrnhgvthenucevlhhushht vghrufhiiigvpedt X-ME-Proxy: Received: from bastion.zyxst.net (bastion.zyxst.net [82.70.91.99]) by mail.messagingengine.com (Postfix) with ESMTPA id 75A6E8005C for ; Sat, 21 Dec 2019 07:54:40 -0500 (EST) Date: Sat, 21 Dec 2019 12:54:11 +0000 From: tech-lists To: freebsd-virtualization@freebsd.org Subject: Re: bhyve device emulation initialization error Message-ID: <20191221125411.GM63749@bastion.zyxst.net> Mail-Followup-To: freebsd-virtualization@freebsd.org References: <20191220160325.GI63749@bastion.zyxst.net> <201912210549.xBL5n0Dp013453@gndrsh.dnsmgr.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="m+utxuhC6KVTvgNz" Content-Disposition: inline In-Reply-To: <201912210549.xBL5n0Dp013453@gndrsh.dnsmgr.net> User-Agent: Mutt/1.12.2 (2019-09-21) X-Rspamd-Queue-Id: 47g5DL2hRfz4FZ7 X-Spamd-Bar: -------- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=zyxst.net header.s=fm1 header.b=ItOZIA6m; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=CqSiWAyR; dmarc=none; spf=pass (mx1.freebsd.org: domain of tech-lists@zyxst.net designates 66.111.4.28 as permitted sender) smtp.mailfrom=tech-lists@zyxst.net X-Spamd-Result: default: False [-8.19 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[zyxst.net:s=fm1,messagingengine.com:s=fm1]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[28.4.111.66.rep.mailspike.net : 127.0.0.18]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.28]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; DMARC_NA(0.00)[zyxst.net]; DKIM_TRACE(0.00)[zyxst.net:+,messagingengine.com:+]; SIGNED_PGP(-2.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[28.4.111.66.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-3.49)[ip: (-9.84), ipnet: 66.111.4.0/24(-4.88), asn: 11403(-2.68), country: US(-0.05)] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Dec 2019 12:54:43 -0000 --m+utxuhC6KVTvgNz Content-Type: text/plain; charset=utf-8; format=flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 20, 2019 at 09:48:33PM -0800, Rodney W. Grimes wrote: >-- Start of PGP signed section. >> Hi, >> >> I'm trying to install an old 11.1-release like this: >> >> vmrun.sh -c 4 -m 32768M -tap2 -d test-host.img -i -I > ^^^^^ is this a typo? Should be -t tap2 argh, it's what I typed as the command. Well spotted. Must have more coffee before doing stu=EF=AC=82f like this. sorry for the noise --=20 J. --m+utxuhC6KVTvgNz Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEE8n3tWhxW11Ccvv9/s8o7QhFzNAUFAl3+FgQACgkQs8o7QhFz NAW+2w//dBqjqLD91BkCGo4K0diq5bOzMwBhAhdE6aANqf42jnM8KIPJxlm1jhMl GI7i6Pn2+Vo18pbdmiLena1yIB/H7LAR8fbdgQMHTiRF65C4ANjRmv7Io/gNPFxC gwznpuuvEMNBYGbXvOsoXq9YolJNZaYLdQHIcOTtLcXlfMNLATVoLB6EeuBhBgZA u5dpnHQkpUnwZA2d8K+oYVunU4GiComsp0qttI0ax1JKSzgVMMyU46Pwi8eyyjyJ 8zh+Jz35UAQiwAVfP+TpMmnVOO9q/vBOao6xRymjuN23Jy4lPirKFrd4lIYGcQCU vKJ+fzH9yyUQTIspxkKDTy4ozuncoWckk2IyEtzDbkhymth+kMOPMkogk9e6dlzt FDBiW1q7mNm/0fksR1KQAZe5PFPEH3y9id35nuZcSmXhtjjW8z8HxPRVBc/WevNK xNi36ntVIp39yeDrmFqzoOvRJR18BAGAwkXILlYFkFY/FJiJ55io8VtIBcd1rFdL 2+OQxw9RTiOghNdgBsoXOrsbb0fw4T9FCCte765gHFm031z44BTzv15UgX+itcx6 viF7ASm8nGDGH3r5c01ablE0tJ8lFyk6MM92HKMe1tjSWzrzjN9cL/cY+5p8nX6V pztUggYI7Qdsnlcs6lr+kmrJlnkKjrftv5yEMs6No+4VJV3nY70= =iujW -----END PGP SIGNATURE----- --m+utxuhC6KVTvgNz-- From owner-freebsd-virtualization@freebsd.org Sat Dec 21 19:26:05 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id ABAC11DAEA6 for ; Sat, 21 Dec 2019 19:26:05 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from mail1.yamagi.org (mail1.yamagi.org [IPv6:2001:19f0:b001:853::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47gFvw5CgPz4YVN for ; Sat, 21 Dec 2019 19:26:04 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from [2001:470:6845:1:bc89:e268:2ad5:517c] (helo=killua.home.yamagi.org) by mail1.yamagi.org with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92.3 (FreeBSD)) (envelope-from ) id 1iikNx-000LQB-3o for freebsd-virtualization@freebsd.org; Sat, 21 Dec 2019 20:25:58 +0100 Date: Sat, 21 Dec 2019 20:25:46 +0100 From: Yamagi To: freebsd-virtualization@freebsd.org Subject: [PATCH] Untangle TPR shadowing and APIC virtualization / Make Win guests on Bhyve _fast_ Message-Id: <20191221202546.caca1f242a907cf50b5562e3@yamagi.org> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; x86_64-unknown-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="PGP-SHA256"; boundary="Signature=_Sat__21_Dec_2019_20_25_46_+0100_88vibovcNf1BDUAv" X-Rspamd-Queue-Id: 47gFvw5CgPz4YVN X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of lists@yamagi.org designates 2001:19f0:b001:853::3 as permitted sender) smtp.mailfrom=lists@yamagi.org X-Spamd-Result: default: False [-4.27 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[yamagi.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(-0.38)[asn: 20473(-1.83), country: US(-0.05)]; MV_CASE(0.50)[]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:20473, ipnet:2001:19f0:b000::/38, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Dec 2019 19:26:05 -0000 --Signature=_Sat__21_Dec_2019_20_25_46_+0100_88vibovcNf1BDUAv Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, a long known problem with Bhyve is that Windows guests are rather slow. With Windows 10 1903 this became much worse, to the point that the guest is unusable. I have found the reason for this: Windows hammers on the %cr8 control register. For example, Windows 10 1909 on an i7-2620M has about 68,000 %cr8 accesses per second. Each of them triggers a vm exit. The most common solution is TPR shadowing. Many thanks to royger in #bhyve for getting me on the right track. Bhyve already implements TPR shadowing. On AMD SVM it just works, but the implementation for Intel VT-x is bound to APIC virtualization. And APIC virtualization is a Xeon feature that is missing on most (all?) desktop CPUs. The patch - further down inline or under [0] - separates TPR shadowing from APIC virtualization, so TPR shadowing can be used on desktop CPUs as well. The patch doesn't just give a small speed boost, it's a difference like day and night. As an example, without the patch, the installation of Windows 10 1909 takes about 2280 seconds from start to first reboot. With the patch, only 370 seconds. On an old Thinkpad X220, Windows 10 guests were previously unusable, now they are resonable fast. The patch does: * Add a new tuneable 'hw.vmm.vmx.use_tpr_shadowing' to disable TLP shadowing. Also add 'hw.vmm.vmx.cap.tpr_shadowing' to be able to query if TPR shadowing is used. * Detach the initialization of TPR shadowing from the initialization of APIC virtualization. APIC virtualization still needs TPR shadowing, but not vice versa. Any CPU that supports APIC virtualization should also support TPR shadowing. * When TPR shadowing is used, the APIC page of each vCPU is written to the VMCS_VIRTUAL_APIC field of the VMCS so that the CPU can write directly to the page without intercept. * On vm exit, vlapic_update_ppr() is called to update the PPR. The patch was tested on an i7-2620M, an i7-6700k and a Xeon Silver 4110. Both Windows and FreeBSD guests work correctly. Regards, Yamagi 0: https://gist.github.com/Yamagi/de70c08eadeeef14eec4cb42aeb5957f ---- diff --git a/sys/amd64/vmm/intel/vmx.c b/sys/amd64/vmm/intel/vmx.c index 605fd0bda766..324a1e9d0c3c 100644 --- a/sys/amd64/vmm/intel/vmx.c +++ b/sys/amd64/vmm/intel/vmx.c @@ -172,6 +172,10 @@ static int cap_invpcid; SYSCTL_INT(_hw_vmm_vmx_cap, OID_AUTO, invpcid, CTLFLAG_RD, &cap_invpcid, 0, "Guests are allowed to use INVPCID"); =20 +static int tpr_shadowing; +SYSCTL_INT(_hw_vmm_vmx_cap, OID_AUTO, tpr_shadowing, CTLFLAG_RD, + &tpr_shadowing, 0, "TPR shadowin support"); + static int virtual_interrupt_delivery; SYSCTL_INT(_hw_vmm_vmx_cap, OID_AUTO, virtual_interrupt_delivery, CTLFLAG_= RD, &virtual_interrupt_delivery, 0, "APICv virtual interrupt delivery supp= ort"); @@ -627,7 +631,7 @@ vmx_restore(void) static int vmx_init(int ipinum) { - int error, use_tpr_shadow; + int error; uint64_t basic, fixed0, fixed1, feature_control; uint32_t tmp, procbased2_vid_bits; =20 @@ -750,6 +754,24 @@ vmx_init(int ipinum) MSR_VMX_PROCBASED_CTLS2, PROCBASED2_ENABLE_INVPCID, 0, &tmp) =3D=3D 0); =20 + /* + * Check support for TPR shadow. + */ + error =3D vmx_set_ctlreg(MSR_VMX_PROCBASED_CTLS, + MSR_VMX_TRUE_PROCBASED_CTLS, PROCBASED_USE_TPR_SHADOW, 0, + &tmp); + if (error =3D=3D 0) { + tpr_shadowing =3D 1; + TUNABLE_INT_FETCH("hw.vmm.vmx.use_tpr_shadowing", + &tpr_shadowing); + } + + if (tpr_shadowing) { + procbased_ctls |=3D PROCBASED_USE_TPR_SHADOW; + procbased_ctls &=3D ~PROCBASED_CR8_LOAD_EXITING; + procbased_ctls &=3D ~PROCBASED_CR8_STORE_EXITING; + } + /* * Check support for virtual interrupt delivery. */ @@ -758,13 +780,9 @@ vmx_init(int ipinum) PROCBASED2_APIC_REGISTER_VIRTUALIZATION | PROCBASED2_VIRTUAL_INTERRUPT_DELIVERY); =20 - use_tpr_shadow =3D (vmx_set_ctlreg(MSR_VMX_PROCBASED_CTLS, - MSR_VMX_TRUE_PROCBASED_CTLS, PROCBASED_USE_TPR_SHADOW, 0, - &tmp) =3D=3D 0); - error =3D vmx_set_ctlreg(MSR_VMX_PROCBASED_CTLS2, MSR_VMX_PROCBASED_CTLS2, procbased2_vid_bits, 0, &tmp); - if (error =3D=3D 0 && use_tpr_shadow) { + if (error =3D=3D 0 && tpr_shadowing) { virtual_interrupt_delivery =3D 1; TUNABLE_INT_FETCH("hw.vmm.vmx.use_apic_vid", &virtual_interrupt_delivery); @@ -775,13 +793,6 @@ vmx_init(int ipinum) procbased_ctls2 |=3D procbased2_vid_bits; procbased_ctls2 &=3D ~PROCBASED2_VIRTUALIZE_X2APIC_MODE; =20 - /* - * No need to emulate accesses to %CR8 if virtual - * interrupt delivery is enabled. - */ - procbased_ctls &=3D ~PROCBASED_CR8_LOAD_EXITING; - procbased_ctls &=3D ~PROCBASED_CR8_STORE_EXITING; - /* * Check for Posted Interrupts only if Virtual Interrupt * Delivery is enabled. @@ -1051,10 +1062,13 @@ vmx_vminit(struct vm *vm, pmap_t pmap) vmx->ctx[i].guest_dr6 =3D DBREG_DR6_RESERVED1; error +=3D vmwrite(VMCS_GUEST_DR7, DBREG_DR7_RESERVED1); =20 - if (virtual_interrupt_delivery) { - error +=3D vmwrite(VMCS_APIC_ACCESS, APIC_ACCESS_ADDRESS); + if (tpr_shadowing) { error +=3D vmwrite(VMCS_VIRTUAL_APIC, vtophys(&vmx->apic_page[i])); + } + + if (virtual_interrupt_delivery) { + error +=3D vmwrite(VMCS_APIC_ACCESS, APIC_ACCESS_ADDRESS); error +=3D vmwrite(VMCS_EOI_EXIT0, 0); error +=3D vmwrite(VMCS_EOI_EXIT1, 0); error +=3D vmwrite(VMCS_EOI_EXIT2, 0); @@ -2313,6 +2327,14 @@ vmx_exit_process(struct vmx *vmx, int vcpu, struct v= m_exit *vmexit) } } =20 + /* + * If 'TPR shadowing' is used, update the local APICs PPR. + */ + if (tpr_shadowing) { + vlapic =3D vm_lapic(vmx->vm, vcpu); + vlapic_update_ppr(vlapic); + } + switch (reason) { case EXIT_REASON_TASK_SWITCH: ts =3D &vmexit->u.task_switch; diff --git a/sys/amd64/vmm/io/vlapic.c b/sys/amd64/vmm/io/vlapic.c index 74e6cd967396..289fdb7e077d 100644 --- a/sys/amd64/vmm/io/vlapic.c +++ b/sys/amd64/vmm/io/vlapic.c @@ -490,7 +490,7 @@ dump_isrvec_stk(struct vlapic *vlapic) * Algorithm adopted from section "Interrupt, Task and Processor Priority" * in Intel Architecture Manual Vol 3a. */ -static void +void vlapic_update_ppr(struct vlapic *vlapic) { int isrvec, tpr, ppr; diff --git a/sys/amd64/vmm/io/vlapic.h b/sys/amd64/vmm/io/vlapic.h index 2a5f54003253..71b97feab6bc 100644 --- a/sys/amd64/vmm/io/vlapic.h +++ b/sys/amd64/vmm/io/vlapic.h @@ -74,6 +74,8 @@ void vlapic_post_intr(struct vlapic *vlapic, int hostcpu,= int ipinum); void vlapic_fire_cmci(struct vlapic *vlapic); int vlapic_trigger_lvt(struct vlapic *vlapic, int vector); =20 +void vlapic_update_ppr(struct vlapic *vlapic); + uint64_t vlapic_get_apicbase(struct vlapic *vlapic); int vlapic_set_apicbase(struct vlapic *vlapic, uint64_t val); void vlapic_set_x2apic_state(struct vm *vm, int vcpuid, enum x2apic_state = s); --=20 Homepage: https://www.yamagi.org Github: https://github.com/yamagi GPG: 0x1D502515 --Signature=_Sat__21_Dec_2019_20_25_46_+0100_88vibovcNf1BDUAv Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEOXu/lxyufwz0gC5x6xRy5x1QJRUFAl3+cboACgkQ6xRy5x1Q JRUvAA/+PuqhedCjcAl45VRGx1qiW6erTxctKu8nDe0OQCRDa9736NggtuzvV7tx CIWMy5KLzkddbqkxnQc9h5thzhdbo1UJYHSivp4rQWvjj4iPE7NLyz+Ug3jOg9LG Nrc/dmz1/OvWmp6HZSiVVZYACs1iv3W9QSaDnh6Q1krLrkwEC5T2bCIpBK+nZ7Ga 0rHp+GMdkYr5ys6QGsDjhGBuGIpVtAsmhw96l1dYhaxVco2Ox09cxepz+exDDFIp 3YzO3HVt2U/NCHT1cz6kPcYrN0CT6eRhWp+aLIzdQgJolia2aZxnWAHxR8BvdVmg yPeiNYeAopntUuURMu9ClXVxzZA3+VLchy55Y2TzQuVHe06GozEGAJ9cyCbIBbSR 854z6pPdaTc4I+y1WZucGEielHSHVYN94hQK2Gqxtlb27dRW7p08aL79eBB8vQz5 etKGKRtT5BR44WdeGowJ6DXTYUYBRPMOICQb+5ih8wr2JgGzqWO7WkXlzNH8D/MX 2GT0eFW0nXs6jkpixHdcQECgycADz2Ex8GGTDkLIZP/nh5JuSuLbHu8WUfGErxxb 2uUGmEHewLTsLVtbl3VwF9erpQmwmIzLv3Nc9vRqbesR1235V4trBtZjJWZgR65Q lCCEvjU+7fecc2RMIi7X9uAatt0rQLUCc/DQVjOL1Jly+VFpYiw= =BGMY -----END PGP SIGNATURE----- --Signature=_Sat__21_Dec_2019_20_25_46_+0100_88vibovcNf1BDUAv--