From owner-freebsd-virtualization@freebsd.org Thu Aug 16 18:55:07 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8C331072E5E for ; Thu, 16 Aug 2018 18:55:06 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com [IPv6:2a00:1450:4864:20::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3947E73541; Thu, 16 Aug 2018 18:55:06 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by mail-lj1-x22a.google.com with SMTP id 203-v6so4457856ljj.13; Thu, 16 Aug 2018 11:55:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:reply-to:in-reply-to:references:from:date:message-id :subject:to:cc; bh=/RvhSJq3lADw1B1HkN3axcYAu0VLrBLeSJa4DjdeRXM=; b=bKgTKqRgKQro9y9xShBbdPxze0qAnH1aiY+XTvL6pMzFU0a47y4YsZWCBd5He+qOH2 8+I3K9fCxeU/op8EuTE1hCPNAlf+MzBfx+DGranU8BC33m6mKdDNdNTnx1rADUPO9dRt P3GEpgh20ZWsB+nmJ3aOpWxpbMAbxcKiu8JSpHl4d4EjV3Y3Z55n/qjr/d5foiaRzG7s rXGHyoYqxsIELcX7P5n26kFAlejE+ChNITCAp7ustOuRmfBugRrws6+e7p/8P8+zOl7I hCHdWC4sMe5YqgswIpvrams7XbqD3RhwoM50ggK3VJ71DHF+FJLDPjd0fU0RyxK+DFUH 5D6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=/RvhSJq3lADw1B1HkN3axcYAu0VLrBLeSJa4DjdeRXM=; b=f7kl/puE4pDZtv1NdeRqyueQYesp6QXjgE44tvQV/eOJnF0GmlEmyRLeheJHBSQ/Ln s6wp8dyrkvvMJMgDVTZ2Hr/yZVdkCeBeQ9dKiDX821vM0QZ2hIX3kcQwt/crIfQT/WQ4 lwDyq/sWQ3fgEIzNf6Oc2dg4RsnsSMtPUjhG1w/JHkTXkxYRKNYJlLxT5TrMfNbkszxr n68NyFDKJDCCQuFGnN8rn6eWSqhPz2Zsl4U/QkRwgApiMmcuRTGzLt3ZJuvCzURMe0zH yvTEFaKkw5nJSitsb6GLhr+GRe4VymNuLCjq+j8IORssyuHHs1SmXJmVZA+H7h07vo2k S5Mg== X-Gm-Message-State: AOUpUlFsHikyt1SAB4PKLPCnTPfnEMwI2Xjoh9x9VQQrJ958HdFUMSS+ RU5VDAJrkAMwGkARiZQFYiN+uz6gLS96wrf31WbWtOQd X-Google-Smtp-Source: AA+uWPw/poCk6S+6FXwt3YStXwTFKEeSCm9+vQ+RsfcvoMhOsJ7Ce2HW/2gZ5t3PIX8bPDUObbp2TpbR1/Q17Fbw0pE= X-Received: by 2002:a2e:88d0:: with SMTP id a16-v6mr12097128ljk.63.1534445703665; Thu, 16 Aug 2018 11:55:03 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a19:1f4c:0:0:0:0:0 with HTTP; Thu, 16 Aug 2018 11:55:02 -0700 (PDT) Reply-To: araujo@freebsd.org In-Reply-To: References: <201808161628.w7GGS52P054505@pdx.rh.CN85.dnsmgr.net> From: Marcelo Araujo Date: Fri, 17 Aug 2018 02:55:02 +0800 Message-ID: Subject: Re: Checking bhyve supported features (sysctls) To: Allan Jude Cc: freebsd-virtualization@freebsd.org, "Rodney W. Grimes" , Matt Churchyard Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 18:55:07 -0000 2018-08-17 0:53 GMT+08:00 Allan Jude : > On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" < > freebsd-rwg@pdx.rh.CN85.dnsmgr.net> wrote: > >> > >> Text manually wrapped to 80, any broken quoting is my fault - rwg > >> > >> > > Hello, > >> > > > >> > > I'm looking for better ways to check for bhyve support / > >available > >> > > features without trying to scan through dmesg output. > >> > > >> > >Yes, it would be very good to remove that, as it usually tries > >> > >to grep a non-existent file /var/run/dmesg.boot that is not > >> > >created until after vm_bhyve has been called from > >/usr/local/etc/rc.d > >> > >when you have things set to autostartup >in /etc/rc.conf > >> > > >> > > >> > > > >> > > I notice that the following 2 sysctl's appear to be set to 1 as > >soon > >> > > as the vmm module is loaded > >> > > > >> > > hw.vmm.vmx.initialized: 1 > >> > > hw.vmm.vmx.cap.unrestricted_guest: 1 > >> > > > >> > > Will these be available on both Intel & AMD processors as a way > >> > > to determine if the module has loaded successfully and can run > >guests? > >> > > > >> > > I also see the below sysctl related to iommu. > >> > > > >> > > hw.vmm.iommu.initialized > >> > > > >> > > Again, will this be set to 1 as soon as the module is loaded if > >> > > iommu is supported, or only when it is used? > >> > > There also seems to be a vmm.amdvi.enable sysctl. > >> > > Would both these need checking or is vmm.iommu enough to > >> > > determine support on any processor. > >> > > >> > >Probalby the safest way for a shell script to decide if bhyve is > >> > >up and running is to stat /dev/vmm, if that exists then the > >modules > >> > >have loaded and initialized and bhyve should be ready to process > >guests. > >> > > >> > Hmm, I don't get /dev/vmm unless I actually have running guests. > >> > >> I'll investigate that, I was pretty sure that you should get this > >> as soon as the vmm.ko module is finished initialzing, but you might > >> be right in that it takes a first vm to cause its creation. > >> Confirmed, /dev/vmm does not exist until the first vm > >> is created. > >> > >> > > >> > >sysctl's mentiond above would be a poor way to make this > >determination. > >> > > >> > It would be nice if sysctls were better documented. > >> > >> Agreed. > >> > >> > If vmx.initialized is set once vmm has successfully loaded, I can't > >see a better way of checking for bhyve support (assuming it's not Intel > >specific). This entry definitely exists and is set to 0 if you load the > >module on a non-supported system, and set to 1 as soon as vmm loads on > >my Intel test system. > >> > >> Given its undocumented status you would be relying on an > >> undocumented feature that could change in either name or > >> behavior, and that is not desirable. > >> > >> Let me see if I can come up with something else. > > > >I looked at the code for bhyvectl, bhyveload and > >byhve. They do not actually try to decide if vmm > >is supported or not, they simply process the error > >from a vm_create() or vm_open() call and exit > >with an error code if they can not handle it > >(some of the code can handle a vm_create failure > >if infact we are trying to create a vm that > >already exists). > > > >If you want to maintain full compatibility a similiar > >stratergy may be in order. > > > >Why is it that vm-bhyve specifically needs to know > >if the kernel has vmm support or not? > >Cant it just be written to handle the errors returned > >if the supported functions do not exist? > > I think the question vm-bhyve wants to answer is: does the CPU have the > required features to run a multicore VM. > > These or similar sysctls do seem to be the correct way to communicate that > support. > You are correct! The question in case as I understood was about CPU feature supported, actually vmm(8) knows all this information! Some examples such like CPU with VMX unrestricted mode support (UG) that is necessary for guest VMs running with multiple vCPU or like VT-d necessary for PCI device passthrough. I have a patch that exposes a sysctl saying what bhyve(8) is capable to run, however it needs to be polished a bit more to be more informative. I think for third part software like vm-bhyve these information are crucial as these software can get advantage of these information prior to run a certain set that will end up in a fail because of a partial CPU support. Best, > -- > Allan Jude > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization- > unsubscribe@freebsd.org" > -- -- Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_)