From owner-freebsd-virtualization@freebsd.org Thu Aug 16 16:28:09 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 88DDC106CECE for ; Thu, 16 Aug 2018 16:28:09 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 03E868B774 for ; Thu, 16 Aug 2018 16:28:08 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w7GGS6xi054506; Thu, 16 Aug 2018 09:28:06 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w7GGS52P054505; Thu, 16 Aug 2018 09:28:05 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201808161628.w7GGS52P054505@pdx.rh.CN85.dnsmgr.net> Subject: Re: Checking bhyve supported features (sysctls) In-Reply-To: <201808161613.w7GGDaNB054438@pdx.rh.CN85.dnsmgr.net> To: "Rodney W. Grimes" Date: Thu, 16 Aug 2018 09:28:05 -0700 (PDT) CC: Matt Churchyard , "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 16:28:09 -0000 > > Text manually wrapped to 80, any broken quoting is my fault - rwg > > > > Hello, > > > > > > I'm looking for better ways to check for bhyve support / available > > > features without trying to scan through dmesg output. > > > > >Yes, it would be very good to remove that, as it usually tries > > >to grep a non-existent file /var/run/dmesg.boot that is not > > >created until after vm_bhyve has been called from /usr/local/etc/rc.d > > >when you have things set to autostartup >in /etc/rc.conf > > > > > > > > > > I notice that the following 2 sysctl's appear to be set to 1 as soon > > > as the vmm module is loaded > > > > > > hw.vmm.vmx.initialized: 1 > > > hw.vmm.vmx.cap.unrestricted_guest: 1 > > > > > > Will these be available on both Intel & AMD processors as a way > > > to determine if the module has loaded successfully and can run guests? > > > > > > I also see the below sysctl related to iommu. > > > > > > hw.vmm.iommu.initialized > > > > > > Again, will this be set to 1 as soon as the module is loaded if > > > iommu is supported, or only when it is used? > > > There also seems to be a vmm.amdvi.enable sysctl. > > > Would both these need checking or is vmm.iommu enough to > > > determine support on any processor. > > > > >Probalby the safest way for a shell script to decide if bhyve is > > >up and running is to stat /dev/vmm, if that exists then the modules > > >have loaded and initialized and bhyve should be ready to process guests. > > > > Hmm, I don't get /dev/vmm unless I actually have running guests. > > I'll investigate that, I was pretty sure that you should get this > as soon as the vmm.ko module is finished initialzing, but you might > be right in that it takes a first vm to cause its creation. > Confirmed, /dev/vmm does not exist until the first vm > is created. > > > > > >sysctl's mentiond above would be a poor way to make this determination. > > > > It would be nice if sysctls were better documented. > > Agreed. > > > If vmx.initialized is set once vmm has successfully loaded, I can't see a better way of checking for bhyve support (assuming it's not Intel specific). This entry definitely exists and is set to 0 if you load the module on a non-supported system, and set to 1 as soon as vmm loads on my Intel test system. > > Given its undocumented status you would be relying on an > undocumented feature that could change in either name or > behavior, and that is not desirable. > > Let me see if I can come up with something else. I looked at the code for bhyvectl, bhyveload and byhve. They do not actually try to decide if vmm is supported or not, they simply process the error from a vm_create() or vm_open() call and exit with an error code if they can not handle it (some of the code can handle a vm_create failure if infact we are trying to create a vm that already exists). If you want to maintain full compatibility a similiar stratergy may be in order. Why is it that vm-bhyve specifically needs to know if the kernel has vmm support or not? Cant it just be written to handle the errors returned if the supported functions do not exist? -- Rod Grimes rgrimes@freebsd.org