From owner-freebsd-virtualization@freebsd.org Sun Sep 13 06:30:22 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 985DF3EE79E for ; Sun, 13 Sep 2020 06:30:22 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from vito-vars.onthenet.com.au (vito-vars.OntheNet.com.au [203.13.68.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bq03f0fgVz3WyY for ; Sun, 13 Sep 2020 06:30:21 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto2.onthenet.com.au [203.13.68.14]) by vito-vars.onthenet.com.au (Postfix) with ESMTPS id 9BB3C2094E69 for ; Sun, 13 Sep 2020 16:30:16 +1000 (AEST) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id 8417A20AE980 for ; Sun, 13 Sep 2020 16:30:16 +1000 (AEST) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 7A4E9209C01F for ; Sun, 13 Sep 2020 16:30:16 +1000 (AEST) Received: from iredmail.onthenet.com.au ([127.0.0.1]) by iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ZcSsfQ3yxNvZ for ; Sun, 13 Sep 2020 16:30:16 +1000 (AEST) Received: from MacBook-Air-4.local (CPE-120-29-42-110.dsl.OntheNet.net [120.29.42.110]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 98AA2209C01A; Sun, 13 Sep 2020 16:30:14 +1000 (AEST) Subject: Re: Nested Virtualization Under KVM Hypervisor To: Pete Wright Cc: FreeBSD virtualization References: <5eacd0a5-89bd-cc98-6c5a-d5490e33b100@freebsd.org> <5e62480f-14cb-5963-f753-8562aab381e3@nomadlogic.org> From: Peter Grehan Message-ID: <5ffc62fe-96ac-a834-1682-1be2a9f50d6d@freebsd.org> Date: Sun, 13 Sep 2020 16:30:14 +1000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.3 cv=bM+N3M2Z c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=jXL/WSj5nX36MkVNSbrI6Q==:17 a=IkcTkHD0fZMA:10 a=reM5J-MqmosA:10 a=6I5d2MoRAAAA:8 a=csACVZsZR9-lm7JpNIUA:9 a=QEXdDO2ut3YA:10 a=IjZwj45LgO3ly-622nXo:22 X-CMAE-Score: 0 X-CMAE-Analysis: v=2.3 cv=LtAJNkVc c=1 sm=1 tr=0 a=eF5wA05EhuPcU0j7PNT2Vg==:117 a=jXL/WSj5nX36MkVNSbrI6Q==:17 a=IkcTkHD0fZMA:10 a=reM5J-MqmosA:10 a=6I5d2MoRAAAA:8 a=csACVZsZR9-lm7JpNIUA:9 a=QEXdDO2ut3YA:10 a=IjZwj45LgO3ly-622nXo:22 X-Rspamd-Queue-Id: 4Bq03f0fgVz3WyY X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; ASN(0.00)[asn:9313, ipnet:203.13.68.0/23, country:AU]; local_wl_from(0.00)[freebsd.org] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Sep 2020 06:30:22 -0000 Hi Pete, > Ah I see - once I realized we had a newer kernel I didn't focus too much > on the first bug - here's the MSR's > > > MSR 0x480: 0x00981000 0x11e57ed0 > MSR 0x482: 0xfff9fffe 0x0401e172 > MSR 0x48b: 0x000003fb 0x00000000 > MSR 0x48c: 0x00000f01 0x04114040 This looks very similar to comment 14 in the bug report https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994#c14 INS/OUTS isn't marked as supported, and single-context INVEPT is missing. As mentioned in comment 17, these are simple to work around but other issues started to show up, indicating that other problems are at play. Given kernel 4.15 looks like it could be from Ubuntu 16-LTS, I'd recommend moving to 18-LTS if that is possible, since that kernel version is known to work. later, Peter.