From owner-freebsd-virtualization@freebsd.org Wed Mar 17 16:32:25 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 240E756FCCE for ; Wed, 17 Mar 2021 16:32:25 +0000 (UTC) (envelope-from seb@strollo.org) Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com [64.147.123.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F0wfv6tNzz4gWV for ; Wed, 17 Mar 2021 16:32:23 +0000 (UTC) (envelope-from seb@strollo.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.west.internal (Postfix) with ESMTP id AF00A225C; Wed, 17 Mar 2021 12:32:22 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Wed, 17 Mar 2021 12:32:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=strollo.org; h= from:content-type:mime-version:date:subject:message-id:to; s= fm3; bh=CO9samz7JCV8AXMlweMcY+xGKv6oH+arg2gYcWKaxNo=; b=RMq/VHVg CCmUkZLIy0lryr+s5EXjjN+LEGyr4xdpBCb+9A6k2UZfA4nY0mSd1BJp8xHUc6IA MQkJDp5FUnmGEVvVtAZar3k+Yt8k2FtogZ6VpIldRDR2eHN7phtBuLKNKvBY4kTx FfdBM4YfaHRIXPO10hPrvTe94OSfCPp5inWGezzREHGyqdiYWGl76vhJ2zpUJ0bv uoIdqzK2b3jyEL+6fT64NhoNflO/FBBmwv/LotvnoiuD+7GypHGIg9UGOKAMu05A YqTvKuCHzp9NKt25683VXhflvZu8xQrEjYNrV9TVqCunFya5Dpe2GPV/Ew3S2zht ZuWSbktldKGQMA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:message-id :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; bh=CO9samz7JCV8AXMlweMcY+xGKv6oH +arg2gYcWKaxNo=; b=pbnKW0UhOEbrtuF9dKyp32tkICH8ttP/KsV4UKHKt54PQ c5SSDf97U7QtrX5SLcBdIkDc5F8f4I3J5jO6YGStQf2frsJqMIb0Npy2ZIMgtia5 BPrsi7FtpB4XleOYRaQ4Vdpg/e91Rz7nZah7zE7Hl1gKXnqNu1SKRu06Z0XqiQLS aId2p9/21C3k9ObfCwD29ayP1WbS9oYd0uALZ7KPmhLAePgrsa0FouEPLANxv1BP BMW1YZr9dQBQlIikpvlgpq1h+piW+9CG22u73I4lVV9e09xb5m/Jd572U2rkJEcG PDaebT6sdMPDamIOSbNWp0ErVclPdYshVGuYEH3DQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudefgedgledvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhtgggfffukffvofesmhdtmherhh dtjeenucfhrhhomhepufgvsggrshhtihgrnhcuufhtrhholhhlohcuoehsvggssehsthhr ohhllhhordhorhhgqeenucggtffrrghtthgvrhhnpeegkeegffdugfehffduteefleduue dvgfeuudffgfekledvgfevfeettedvgefhhfenucfkphepleegrddvheehrddvudekrddv ieenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehsvg gssehsthhrohhllhhordhorhhg X-ME-Proxy: Received: from spinther.strollo.org (94-255-218-26.cust.bredband2.com [94.255.218.26]) by mail.messagingengine.com (Postfix) with ESMTPA id 8BD52240054; Wed, 17 Mar 2021 12:32:21 -0400 (EDT) From: Sebastian Strollo Content-Type: multipart/mixed; boundary="Apple-Mail=_2B081441-6F8F-4266-9AE8-EB4BBDD14A25" Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Date: Wed, 17 Mar 2021 17:32:19 +0100 Subject: Nested virtualization with VirtualBox OSE 6.1.8, any success? Message-Id: <2963C20D-25C2-4613-A878-908D16D5C4C0@strollo.org> To: freebsd-virtualization@freebsd.org X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4F0wfv6tNzz4gWV X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=strollo.org header.s=fm3 header.b=RMq/VHVg; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=pbnKW0Uh; dmarc=none; spf=pass (mx1.freebsd.org: domain of seb@strollo.org designates 64.147.123.21 as permitted sender) smtp.mailfrom=seb@strollo.org X-Spamd-Result: default: False [-2.10 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.21]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[strollo.org:+,messagingengine.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[64.147.123.21:from]; ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:~]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.21:from]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[strollo.org:s=fm3,messagingengine.com:s=fm2]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; DMARC_NA(0.00)[strollo.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[64.147.123.21:from:127.0.2.255]; RWL_MAILSPIKE_POSSIBLE(0.00)[64.147.123.21:from]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2021 16:32:25 -0000 --Apple-Mail=_2B081441-6F8F-4266-9AE8-EB4BBDD14A25 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi I=E2=80=99m running 12.2-RELEASE-p4 (on an Intel i7-10710U cpu w 64G = RAM) as the host OS and using virtualbox-ose-nox11-6.1.18 (from pkg). I have a guest VM that runs Ubuntu 20.04, and for some testing and = development I need it to be able to run KVM, hence I need =E2=80=9Cnested = virtualization=E2=80=9D. Has anyone had any luck with the = "--nested-hw-virt on=E2=80=9D option? I=E2=80=99m not having any luck at all, if I try to enable it then = startup of my VM just hangs, I get the following error in VBoxSVC.log = (which I think is the relevant one, the whole log is attached - I = suspect the disk that it can=E2=80=99t close is just a result of it = shutting down even though it comes before in the log) 00:54:00.390457 Watcher ERROR [COM]: aRC=3DE_ACCESSDENIED = (0x80070005) aIID=3D{d0a0163f-e254-4e5b-a1f2-011cf991c38d} = aComponent=3D{VirtualBoxWrap} aText=3D{The object is not ready}, = preserve=3Dfalse aResultDetail=3D0 And looking for that id in xpti.dat I find =E2=80=9CIVirtualBox=E2=80=9D: % grep d0a0163f-e254-4e5b-a1f2-011cf991c38d = ~/.config/VirtualBox/xpti.dat 329,IVirtualBox,{d0a0163f-e254-4e5b-a1f2-011cf991c38d},0,-1,1 If I =E2=80=9Cvboxmanage modifyvm --nested-hw-virt off=E2=80=9D then my = VM starts up fine, so I=E2=80=99ve eliminated other problems. I assume that the kernel has to allow VirtualBox to do this kind of = stuff? Is there perhaps a sysctl setting to permit this, or is it simply = not doable in FreeBSD-12? Thanks, /Sebastian PS. I=E2=80=99d actually much rather run my VM under bhyve, anyone know = the status of nested virtualization there? --Apple-Mail=_2B081441-6F8F-4266-9AE8-EB4BBDD14A25 Content-Disposition: attachment; filename=VBoxSVC.log Content-Type: application/octet-stream; x-unix-mode=0600; name="VBoxSVC.log" Content-Transfer-Encoding: 7bit 00:00:00.000349 main VirtualBox XPCOM Server 6.1.18 r142142 freebsd.amd64 (Mar 13 2021 17:24:09) release log 00:00:00.000354 main Log opened 2021-03-17T15:18:23.329141000Z 00:00:00.000355 main Build Type: release 00:00:00.000360 main OS Product: FreeBSD 00:00:00.000363 main OS Release: 12.2-RELEASE-p4 00:00:00.000365 main OS Version: FreeBSD 12.2-RELEASE-p4 GENERIC 00:00:00.000366 main Firmware type: failed - VERR_NOT_SUPPORTED 00:00:00.000378 main Host RAM: 65202MB (63.6GB) total, 62900MB (61.4GB) available 00:00:00.000382 main Executable: /usr/local/lib/virtualbox/VBoxSVC 00:00:00.000382 main Process ID: 1323 00:00:00.000383 main Package type: BSD_64BITS_GENERIC (OSE) 00:00:00.001353 main IPC socket path: /tmp/.vbox-seb-ipc/ipcd 00:00:00.108950 nspr-2 VirtualBox: object creation starts 00:00:00.109005 nspr-2 Home directory: '/home/seb/.config/VirtualBox' 00:00:00.109184 nspr-2 Loading settings file "/home/seb/.config/VirtualBox/VirtualBox.xml" with version "1.12-freebsd" 00:00:01.172367 nspr-2 HostDnsMonitor: initializing 00:00:01.172407 nspr-2 NAT: resolv.conf: nameserver 10.11.11.1 00:00:01.172430 nspr-2 HostDnsMonitor: updating information 00:00:01.172440 nspr-2 HostDnsMonitor: old information 00:00:01.172444 nspr-2 no server entries 00:00:01.172448 nspr-2 no domain set 00:00:01.172451 nspr-2 no search string entries 00:00:01.172454 nspr-2 HostDnsMonitor: new information 00:00:01.172457 nspr-2 server 1: 10.11.11.1 00:00:01.172461 nspr-2 domain: kanten.strollo.org 00:00:01.172467 nspr-2 search string 1: kanten.strollo.org 00:00:01.172470 nspr-2 search string 2: strollo.org 00:00:01.172791 nspr-2 VD: VDInit finished with VINF_SUCCESS 00:00:01.172854 nspr-2 ERROR [COM]: aRC=NS_ERROR_FAILURE (0x80004005) aIID={027bc463-929c-40e8-bf16-fea557cd8e7e} aComponent={SystemPropertiesWrap} aText={Cannot determine default Guest Additions ISO location. Most likely they are not available}, preserve=false aResultDetail=0 00:00:01.173131 nspr-2 Loading settings file "/store/vbox-vms/rhea/rhea.vbox" with version "1.17-freebsd" 00:00:01.173400 nspr-2 Loading settings file "/store/vbox-vms/utest/utest.vbox" with version "1.16-freebsd" 00:00:01.173597 nspr-2 VirtualBox: object created 00:00:01.178295 nspr-2 Saving settings file "/store/vbox-vms/rhea/rhea.vbox" with version "1.17-freebsd" 00:53:55.317121 nspr-2 Saving settings file "/store/vbox-vms/rhea/rhea.vbox" with version "1.17-freebsd" 00:54:00.389943 main VirtualBox: object deletion starts 00:54:00.389975 main ERROR [COM]: aRC=VBOX_E_OBJECT_IN_USE (0x80bb000c) aIID={ad47ad09-787b-44ab-b343-a082a3f2dfb1} aComponent={MediumWrap} aText={Medium '/store/vbox-vms/rhea/disk0.vmdk' cannot be closed because it is still attached to 1 virtual machines}, preserve=false aResultDetail=0 00:54:00.390275 main HostDnsMonitor: shutting down ... 00:54:00.390282 main HostDnsMonitor: shut down 00:54:00.390457 Watcher ERROR [COM]: aRC=E_ACCESSDENIED (0x80070005) aIID={d0a0163f-e254-4e5b-a1f2-011cf991c38d} aComponent={VirtualBoxWrap} aText={The object is not ready}, preserve=false aResultDetail=0 00:54:00.390522 main VirtualBox: object deleted --Apple-Mail=_2B081441-6F8F-4266-9AE8-EB4BBDD14A25--