From owner-freebsd-virtualization@freebsd.org Mon Oct 15 16:23:49 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 242D910C1336 for ; Mon, 15 Oct 2018 16:23:49 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A9CCA7ACDF for ; Mon, 15 Oct 2018 16:23:48 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro-2.local (ralph.baldwin.cx [66.234.199.215]) by mail.baldwin.cx (Postfix) with ESMTPSA id 79F6810B476; Mon, 15 Oct 2018 12:23:47 -0400 (EDT) Subject: Re: CFT: Reload LDTR after #VMEXIT on AMD-v in bhyve To: Mike Tancsa , freebsd-virtualization@freebsd.org References: <8282ebc8-7287-04fa-42d7-2c8501c4b979@sentex.net> From: John Baldwin Message-ID: Date: Mon, 15 Oct 2018 09:23:46 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <8282ebc8-7287-04fa-42d7-2c8501c4b979@sentex.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Mon, 15 Oct 2018 12:23:47 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Oct 2018 16:23:49 -0000 On 10/11/18 1:04 PM, Mike Tancsa wrote: > On 10/11/2018 3:02 PM, John Baldwin wrote: >> Can someone using bhyve on an AMD host test this patch? Just booting a >> guest to multiuser is probably sufficient testing: >> >> https://github.com/bsdjhb/freebsd/commit/97323364e196900548f5293ac97bfb22b8a2ba72.patch > > well, if I let the box boot fully and then load the vmm.ko, all is good. > But if I load it from /boot/loader.conf, I get a panic at boot up (img > attached) > > But other than that, it works fine. That panic doesn't really make sense. :( The patch only changes behavior when you are actually running a guest, it doesn't affect anything in the vmm.ko initialization. > > > 0{ryzenbsd12}# fetch -o p > https://github.com/bsdjhb/freebsd/commit/97323364e196900548f5293ac97bfb22b8a2ba72.patch > fetch: > https://github.com/bsdjhb/freebsd/commit/97323364e196900548f5293ac97bfb22b8a2ba72.patch: > size of remote file is not known > p                                                     1618  B   15 > MBps    00s > 0{ryzenbsd12}# patch < p > Hmm...  Looks like a unified diff to me... > The text leading up to this was: > -------------------------- > |From 97323364e196900548f5293ac97bfb22b8a2ba72 Mon Sep 17 00:00:00 2001 > |From: John Baldwin > |Date: Tue, 9 Oct 2018 14:49:37 -0700 > |Subject: [PATCH] Reload the LDT selector after an AMD-v #VMEXIT. > | > |cpu_switch() always reloads the LDT, so this can only affect the > |hypervisor process itself.  Fix this by explicitly reloading the host > |LDT selector after each #VMEXIT.  The stock bhyve process on FreeBSD > |never uses a custom LDT, so this change is cosmetic. > |--- > | sys/amd64/vmm/amd/svm.c | 13 +++++++++++++ > | 1 file changed, 13 insertions(+) > | > |diff --git a/sys/amd64/vmm/amd/svm.c b/sys/amd64/vmm/amd/svm.c > |index 2597bf9775706..c420db550bc7e 100644 > |--- a/sys/amd64/vmm/amd/svm.c > |+++ b/sys/amd64/vmm/amd/svm.c > -------------------------- > Patching file sys/amd64/vmm/amd/svm.c using Plan A... > Hunk #1 succeeded at 1940. > Hunk #2 succeeded at 2025. > Hunk #3 succeeded at 2064. > done > 0{ryzenbsd12}# > > I confirmed prior to the patch I could run stock 11.2R  i386 and amd64 > guest images on 12.0-ALPHA9 FreeBSD 12.0-ALPHA9 r339287 as the hypervisor > > CPU: AMD Ryzen 5 1600X Six-Core Processor            (3593.34-MHz > K8-class CPU) >   Origin="AuthenticAMD"  Id=0x800f11  Family=0x17  Model=0x1  Stepping=1 >   > Features=0x178bfbff >   > Features2=0x7ed8320b >   AMD Features=0x2e500800 >   AMD > Features2=0x35c233ff >   Structured Extended > Features=0x209c01a9 >   XSAVE Features=0xf >   AMD Extended Feature Extensions ID EBX=0x1007 >   SVM: NP,NRIP,VClean,AFlush,DAssist,NAsids=32768 >   TSC: P-state invariant, performance statistics > > > > > with the patched version > > ivhd0: on acpi0 > ivhd0: Flag:b0 > ivhd0: Features(type:0x11) MsiNumPPR = 0 PNBanks= 2 PNCounters= 0 > ivhd0: Extended features[31:0]:22294ada HATS = > 0x2 GATS = 0x0 GLXSup = 0x1 SmiFSup = 0x1 SmiFRC = 0x2 GAMSup = 0x1 > DualPortLogSup = 0x2 DualEventLogSup = 0x2 > ivhd0: Extended features[62:32]:f77ef Max PASID: 0x2f > DevTblSegSup = 0x3 MarcSup = 0x1 > ivhd0: supported paging level:7, will use only: 4 > ivhd0: device range: 0x0 - 0xffff > ivhd0: PCI cap 0x190b640f@0x40 feature:19 Are these messages (ivhd0) new with the patched version and not present with the old one? These shouldn't change due to the patch as these are about the AMD I/O MMU only used by bhyve when doing PCI pass through. -- John Baldwin