From owner-freebsd-virtualization@freebsd.org Thu Oct 11 20:40:18 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1B2610C6C99 for ; Thu, 11 Oct 2018 20:40:18 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smarthost2.sentex.ca", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6523780E32; Thu, 11 Oct 2018 20:40:18 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5:0:0:0:11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id w9BKeGmR002377 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 11 Oct 2018 16:40:16 -0400 (EDT) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (saphire3.sentex.net [192.168.43.26]) by lava.sentex.ca (8.15.2/8.15.2) with ESMTP id w9BKeE6Q019936; Thu, 11 Oct 2018 16:40:14 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: CFT: Reload LDTR after #VMEXIT on AMD-v in bhyve To: John Baldwin , freebsd-virtualization@freebsd.org References: From: Mike Tancsa Openpgp: preference=signencrypt Autocrypt: addr=mike@sentex.net; prefer-encrypt=mutual; keydata= xsBNBEzcA24BCACpwI/iqOrs0GfQSfhA1v6Z8AcXVeGsRyKEKUpxoOYxXWc2z3vndbYlIP6E YJeifzKhS/9E+VjhhICaepLHfw865TDTUPr5D0Ed+edSsKjlnDtb6hfNJC00P7eoiuvi85TW F/gAxRY269A5d856bYrzLbkWp2lKUR3Bg6NnORtflGzx9ZWAltZbjYjjRqegPv0EQNYcHqWo eRpXilEo1ahT6nmOU8V7yEvT2j4wlLcQ6qg7w+N/vcBvyd/weiwHU+vTQ9mT61x5/wUrQhdw 2gJHeQXeDGMJV49RT2EEz+QVxaf477eyWsdQzPVjAKRMT3BVdK8WvpYAEfBAbXmkboOxABEB AAHNHG1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5jYT7CwHgEEwECACIFAkzcA24CGwMGCwkI BwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEJXHwM2kc8rX+sMH/2V6pTBKsQ5mpWWLgs6wVP2k BC+6r/YKNXv9Rw/PrC6+9hTbgA+sSjJ+8gxsCbJsOQXZrxF0x3l9oYdYfuKcwdwXFX1/FS8p HfBeDkmlH+dI709xT9wgrR4dS5aMmKp0scPrXPIAKiYVOHjOlNItcLYTEEWEFBepheEVsgmk GrNbcrHwOx/u4igUQ8vcpyXPyUki+BsftPw8ZQvBU887igh0OxaCR8AurJppQ5UQd63r81cX E1ZjoFoWCaGK/SjPb/OhpYpu5swoZIhOxQbn7OtakYPsDd5t2A5KhvjI8BMTnd5Go+2xsCmr jlIEq8Bi29gCcfQUvNiClevi13ifmnnOwE0ETNwDbgEIALWGNJHRAhpd0A4vtd3G0oRqMBcM FGThQr3qORmEBTPPEomTdBaHcn+Xl+3YUvTBD/67/mutWBwgp2R5gQOSqcM7axvgMSHbKqBL 9sd1LsLw0UT2O5AYxv3EwzhG84pwRg3XcUqvWA4lA8tIj/1q4Jzi5qOkg1zxq4W9qr9oiYK5 bBR638JUvr3eHMaz/Nz+sDVFgwHmXZj3M6aE5Ce9reCGbvrae7H5D5PPvtT3r22X8SqfVAiO TFKedCf/6jbSOedPN931FJQYopj9P6b3m0nI3ZiCDVSqeyOAIBLzm+RBUIU3brzoxDhYR8pz CJc2sK8l6YjqivPakrD86bFDff8AEQEAAcLAXwQYAQIACQUCTNwDbgIbDAAKCRCVx8DNpHPK 1+iQB/99aqNtez9ZTBWELj269La8ntuRx6gCpzfPXfn6SDIfTItDxTh1hrdRVP5QNGGF5wus N4EMwXouskva1hbFX3Pv72csYSxxEJXjW16oV8WK4KjKXoskLg2RyRP4uXqL7Mp2ezNtVY5F 9nu3fj4ydpHCSaqKy5xd70A8D50PfZsFgkrsa5gdQhPiGGEdxhq/XSeAAnZ4uVLJKarH+mj5 MEhgZPEBWkGrbDZpezl9qbFcUem/uT9x8FYT/JIztMVh9qDcdP5tzANW5J7nvgXjska+VFGY ryZK4SPDczh74mn6GI/+RBi7OUzXXPgpPBrhS5FByjwCqjjsSpTjTds+NGIY Organization: Sentex Communications Message-ID: Date: Thu, 11 Oct 2018 16:40:15 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Scanned-By: MIMEDefang 2.83 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Oct 2018 20:40:19 -0000 On 10/11/2018 3:02 PM, John Baldwin wrote: > Can someone using bhyve on an AMD host test this patch? Just booting a > guest to multiuser is probably sufficient testing: > > https://github.com/bsdjhb/freebsd/commit/97323364e196900548f5293ac97bfb22b8a2ba72.patch > > Thanks. > well, if I let the box boot fully and then load the vmm.ko, all is good. But if I load it from /boot/loader.conf, I get a panic at boot up (img attached) But other than that, it works fine. 0{ryzenbsd12}# fetch -o p https://github.com/bsdjhb/freebsd/commit/97323364e196900548f5293ac97bfb22b8a2ba72.patch fetch: https://github.com/bsdjhb/freebsd/commit/97323364e196900548f5293ac97bfb22b8a2ba72.patch: size of remote file is not known p                                                     1618  B   15 MBps    00s 0{ryzenbsd12}# patch < p Hmm...  Looks like a unified diff to me... The text leading up to this was: -------------------------- |From 97323364e196900548f5293ac97bfb22b8a2ba72 Mon Sep 17 00:00:00 2001 |From: John Baldwin |Date: Tue, 9 Oct 2018 14:49:37 -0700 |Subject: [PATCH] Reload the LDT selector after an AMD-v #VMEXIT. | |cpu_switch() always reloads the LDT, so this can only affect the |hypervisor process itself.  Fix this by explicitly reloading the host |LDT selector after each #VMEXIT.  The stock bhyve process on FreeBSD |never uses a custom LDT, so this change is cosmetic. |--- | sys/amd64/vmm/amd/svm.c | 13 +++++++++++++ | 1 file changed, 13 insertions(+) | |diff --git a/sys/amd64/vmm/amd/svm.c b/sys/amd64/vmm/amd/svm.c |index 2597bf9775706..c420db550bc7e 100644 |--- a/sys/amd64/vmm/amd/svm.c |+++ b/sys/amd64/vmm/amd/svm.c -------------------------- Patching file sys/amd64/vmm/amd/svm.c using Plan A... Hunk #1 succeeded at 1940. Hunk #2 succeeded at 2025. Hunk #3 succeeded at 2064. done 0{ryzenbsd12}# I confirmed prior to the patch I could run stock 11.2R  i386 and amd64 guest images on 12.0-ALPHA9 FreeBSD 12.0-ALPHA9 r339287 as the hypervisor CPU: AMD Ryzen 5 1600X Six-Core Processor            (3593.34-MHz K8-class CPU)   Origin="AuthenticAMD"  Id=0x800f11  Family=0x17  Model=0x1  Stepping=1   Features=0x178bfbff   Features2=0x7ed8320b   AMD Features=0x2e500800   AMD Features2=0x35c233ff   Structured Extended Features=0x209c01a9   XSAVE Features=0xf   AMD Extended Feature Extensions ID EBX=0x1007   SVM: NP,NRIP,VClean,AFlush,DAssist,NAsids=32768   TSC: P-state invariant, performance statistics with the patched version ivhd0: on acpi0 ivhd0: Flag:b0 ivhd0: Features(type:0x11) MsiNumPPR = 0 PNBanks= 2 PNCounters= 0 ivhd0: Extended features[31:0]:22294ada HATS = 0x2 GATS = 0x0 GLXSup = 0x1 SmiFSup = 0x1 SmiFRC = 0x2 GAMSup = 0x1 DualPortLogSup = 0x2 DualEventLogSup = 0x2 ivhd0: Extended features[62:32]:f77ef Max PASID: 0x2f DevTblSegSup = 0x3 MarcSup = 0x1 ivhd0: supported paging level:7, will use only: 4 ivhd0: device range: 0x0 - 0xffff ivhd0: PCI cap 0x190b640f@0x40 feature:19 Tested with ./vmrun.sh -c 4 -m 4096M -t tap0 -d FreeBSD-11.2-RELEASE-amd64.raw amd64 and ./vmrun.sh -c 4 -m 4096M -t tap1 -d FreeBSD-11.2-RELEASE-i386.raw i386 I sent a jpg of the crash in a separate image as it was rejected from the mailing list. ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 x203 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada