From owner-freebsd-virtualization@freebsd.org Sun Aug 12 05:20:33 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 381F810648D8 for ; Sun, 12 Aug 2018 05:20:33 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from relay2.tomsk.ru (mail.sibptus.tomsk.ru [212.73.124.5]) by mx1.freebsd.org (Postfix) with ESMTP id 8F3BE791F2 for ; Sun, 12 Aug 2018 05:20:32 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) X-Virus-Scanned: by clamd daemon 0.100.1 for FreeBSD at relay.sibptus.ru Received: from [212.73.125.240] (HELO admin.sibptus.transneft.ru) by relay2.tomsk.ru (CommuniGate Pro SMTP 5.1.16) with ESMTPS id 40075711; Sun, 12 Aug 2018 12:20:30 +0700 Received: from admin.sibptus.transneft.ru (sudakov@localhost [127.0.0.1]) by admin.sibptus.transneft.ru (8.15.2/8.15.2) with ESMTP id w7C5KUnH073194; Sun, 12 Aug 2018 12:20:30 +0700 (+07) (envelope-from vas@mpeks.tomsk.su) Received: (from sudakov@localhost) by admin.sibptus.transneft.ru (8.15.2/8.15.2/Submit) id w7C5KP9b073191; Sun, 12 Aug 2018 12:20:25 +0700 (+07) (envelope-from vas@mpeks.tomsk.su) X-Authentication-Warning: admin.sibptus.transneft.ru: sudakov set sender to vas@mpeks.tomsk.su using -f Date: Sun, 12 Aug 2018 12:20:25 +0700 From: Victor Sudakov To: "Rodney W. Grimes" Cc: Shawn Webb , freebsd-virtualization@freebsd.org Subject: Re: Curent Centos 7 and bhyve Message-ID: <20180812052025.GB73103@admin.sibptus.transneft.ru> References: <20180811162214.GA67623@admin.sibptus.transneft.ru> <201808111634.w7BGY1J3028598@pdx.rh.CN85.dnsmgr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201808111634.w7BGY1J3028598@pdx.rh.CN85.dnsmgr.net> Organization: AO "Svyaztransneft", SibPTUS X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.9.5 (2018-04-13) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Aug 2018 05:20:33 -0000 Rodney W. Grimes wrote: > > Rodney W. Grimes wrote: > > > > > > > > > Are there issues with Current CEntos and bhyve? > > > > > > > > > > > > > > > > Sure there are, please look at > > > > > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230453 > > > > > > > > > > > > > > Booting in UEFI mode works. > > > > > > > > > > > > This means we need an update to /usr/local/share/examples/vm-bhyve/centos7.conf ? > > > > > > It says 'loader="grub"' for the present. > > > > > > > > > > > > Do you have a vm config to boot centos7 in UEFI mode you could share? > > > > > > > > > > I just use /usr/share/examples/bhyve/vmrun.sh. I don't use any > > > > > third-party utility to manage bhyve VMs. vmrun.sh is pretty > > > > > straight-forward. > > > > > > > > Thanks for replying. However, I highly recommend vm-bhyve, maybe you > > > > should give it a try. You will love the ease of VM creation and > > > > provisioning, network management, ZFS integration (VM snapshots and > > > > cloning), console and datastore management etc. > > > > > > Though it has a lot of features, it also has some short comings, > > > like you can not spec a vm to be wired in memory, which IMHO is > > > the only way to insure consistent VM performance. > > > > Well, we have "bhyve_options" configuration option in the vm config, > > why not put "-S" there, is that what you mean by wiring the vm in > > memory? > > I believe that fails as that only adds the -S to bhyve, and > you must specify it both on bhyveload and bhyve for it to > work. I think it is totally doable becase vm-bhyve is nothing but a suit of scripts. A PR with a feature request would be appropriate. What about VM that don't use bhyveload, but some other kind of loader like grub2-bhyve? > > > > > > Its artificial restriction of 16 character VM names is also > > > a fair bit annoying. > > > > Maybe. > > Maybe? No, factually. I migrated a number of ESXi VM's and > had to patch vm-bhyve to not have this restriction, so it is > annoying. Did you send your patches upstream? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 From owner-freebsd-virtualization@freebsd.org Sun Aug 12 09:16:50 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D3DB6106A235 for ; Sun, 12 Aug 2018 09:16:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 70CF1802E4 for ; Sun, 12 Aug 2018 09:16:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 35DFF106A232; Sun, 12 Aug 2018 09:16:50 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 249D9106A231 for ; Sun, 12 Aug 2018 09:16:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A4C89802DD for ; Sun, 12 Aug 2018 09:16:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 06F791CA0D for ; Sun, 12 Aug 2018 09:16:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7C9GmqE004127 for ; Sun, 12 Aug 2018 09:16:48 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7C9Gm6M004115 for virtualization@FreeBSD.org; Sun, 12 Aug 2018 09:16:48 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 184046] bhyve(4) manpage references non-existant manpages bhyvectl(8), vmm(4) Date: Sun, 12 Aug 2018 09:16:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Documentation X-Bugzilla-Component: Manual Pages X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: 0mp@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Aug 2018 09:16:51 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D184046 Mateusz Piotrowski <0mp@FreeBSD.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |0mp@FreeBSD.org Resolution|--- |FIXED Status|In Progress |Closed --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Sun Aug 12 14:29:22 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A0071071CE8 for ; Sun, 12 Aug 2018 14:29:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id ABBBF8A541 for ; Sun, 12 Aug 2018 14:29:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 6D7BD1071CE6; Sun, 12 Aug 2018 14:29:21 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C5C21071CE5 for ; Sun, 12 Aug 2018 14:29:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EFBCC8A53F for ; Sun, 12 Aug 2018 14:29:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 381821F53A for ; Sun, 12 Aug 2018 14:29:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7CETKP2000988 for ; Sun, 12 Aug 2018 14:29:20 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7CETKLM000987 for virtualization@FreeBSD.org; Sun, 12 Aug 2018 14:29:20 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 230402] With buildworld, the system can not use swap Date: Sun, 12 Aug 2018 14:29:19 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: component Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Aug 2018 14:29:22 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230402 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Component|bin |misc --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-virtualization@freebsd.org Sun Aug 12 14:59:39 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A648410731F3 for ; Sun, 12 Aug 2018 14:59:39 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0D88B8BB65 for ; Sun, 12 Aug 2018 14:59:38 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w7CExUrh032889; Sun, 12 Aug 2018 07:59:30 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w7CExSWX032888; Sun, 12 Aug 2018 07:59:28 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201808121459.w7CExSWX032888@pdx.rh.CN85.dnsmgr.net> Subject: Re: Curent Centos 7 and bhyve In-Reply-To: <20180812052025.GB73103@admin.sibptus.transneft.ru> To: Victor Sudakov Date: Sun, 12 Aug 2018 07:59:28 -0700 (PDT) CC: Shawn Webb , freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Aug 2018 14:59:39 -0000 > Rodney W. Grimes wrote: > > > Rodney W. Grimes wrote: > > > > > > > > > > Are there issues with Current CEntos and bhyve? > > > > > > > > > > > > > > > > > > Sure there are, please look at > > > > > > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230453 > > > > > > > > > > > > > > > > Booting in UEFI mode works. > > > > > > > > > > > > > > This means we need an update to /usr/local/share/examples/vm-bhyve/centos7.conf ? > > > > > > > It says 'loader="grub"' for the present. > > > > > > > > > > > > > > Do you have a vm config to boot centos7 in UEFI mode you could share? > > > > > > > > > > > > I just use /usr/share/examples/bhyve/vmrun.sh. I don't use any > > > > > > third-party utility to manage bhyve VMs. vmrun.sh is pretty > > > > > > straight-forward. > > > > > > > > > > Thanks for replying. However, I highly recommend vm-bhyve, maybe you > > > > > should give it a try. You will love the ease of VM creation and > > > > > provisioning, network management, ZFS integration (VM snapshots and > > > > > cloning), console and datastore management etc. > > > > > > > > Though it has a lot of features, it also has some short comings, > > > > like you can not spec a vm to be wired in memory, which IMHO is > > > > the only way to insure consistent VM performance. > > > > > > Well, we have "bhyve_options" configuration option in the vm config, > > > why not put "-S" there, is that what you mean by wiring the vm in > > > memory? > > > > I believe that fails as that only adds the -S to bhyve, and > > you must specify it both on bhyveload and bhyve for it to > > work. > > I think it is totally doable becase vm-bhyve is nothing but a suit of > scripts. A PR with a feature request would be appropriate. I made several attempts to contact the author at the email address provided at the git hub while making other bhyve changes to try and coordinate with him. I got no response after 3 attempts, so have stopped trying to contact them. (This was while I was adding the -c cpu topology modifications.) > > What about VM that don't use bhyveload, but some other kind of loader > like grub2-bhyve? I am not sure how vm-bhyve deals with that as I have none of those type VM's. > > > > > > > > > Its artificial restriction of 16 character VM names is also > > > > a fair bit annoying. > > > > > > Maybe. > > > > Maybe? No, factually. I migrated a number of ESXi VM's and > > had to patch vm-bhyve to not have this restriction, so it is > > annoying. > > Did you send your patches upstream? Could not even get an email ack from upstream. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Sun Aug 12 17:27:21 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 241991076F9C for ; Sun, 12 Aug 2018 17:27:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id B683E71041 for ; Sun, 12 Aug 2018 17:27:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 781D31076F9A; Sun, 12 Aug 2018 17:27:20 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 670031076F99 for ; Sun, 12 Aug 2018 17:27:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 08E2F7103D for ; Sun, 12 Aug 2018 17:27:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 3C9D120EE8 for ; Sun, 12 Aug 2018 17:27:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7CHRJR8069828 for ; Sun, 12 Aug 2018 17:27:19 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7CHRJ6r069826 for virtualization@FreeBSD.org; Sun, 12 Aug 2018 17:27:19 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 230402] With buildworld, the system can not use swap Date: Sun, 12 Aug 2018 17:27:19 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: marklmi26-fbsd@yahoo.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Aug 2018 17:27:21 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230402 --- Comment #8 from Mark Millard --- (In reply to Mark Millard from comment #7) [This is extracted from another context that involved the Pine64+ 2GB.] As of updating to -r337400 the Pine64+ 2GB no longer will boot from the e.MMC on the microsd adapter card. (I switched to tracking fully modern dts use, u-boot, etc.) So I tried a build via a USB SSD as the root file system and swap partition. As reported in: https://lists.freebsd.org/pipermail/freebsd-arm/2018-August/018605.html it failed with an OOM kill. This should have avoided I/O latency problems being involved. (That message is part of a long on-going thread tied to OOM kills, most of the reports involving large I/O latencies being involved.) I can not change the "Afects Only Me" status. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-virtualization@freebsd.org Sun Aug 12 17:51:24 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86955107813A for ; Sun, 12 Aug 2018 17:51:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 23B7A736A9 for ; Sun, 12 Aug 2018 17:51:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id DCEE91078138; Sun, 12 Aug 2018 17:51:23 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CBCFA1078137 for ; Sun, 12 Aug 2018 17:51:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6DD17736A2 for ; Sun, 12 Aug 2018 17:51:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id BBFF2212F5 for ; Sun, 12 Aug 2018 17:51:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7CHpML6020422 for ; Sun, 12 Aug 2018 17:51:22 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7CHpM7p020421 for virtualization@FreeBSD.org; Sun, 12 Aug 2018 17:51:22 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 230402] With buildworld, the system can not use swap Date: Sun, 12 Aug 2018 17:51:23 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: marklmi26-fbsd@yahoo.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Aug 2018 17:51:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230402 --- Comment #9 from Mark Millard --- (In reply to Mark Millard from comment #8) Other bugzilla's are: 227609 230454. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-virtualization@freebsd.org Mon Aug 13 01:24:10 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 51830105F1B6 for ; Mon, 13 Aug 2018 01:24:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id E24DA82BCF for ; Mon, 13 Aug 2018 01:24:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id A7422105F1B0; Mon, 13 Aug 2018 01:24:09 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 96095105F1AF for ; Mon, 13 Aug 2018 01:24:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 37ACC82BC7 for ; Mon, 13 Aug 2018 01:24:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 963FF252E2 for ; Mon, 13 Aug 2018 01:24:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7D1O8Co035827 for ; Mon, 13 Aug 2018 01:24:08 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7D1O80S035826 for virtualization@FreeBSD.org; Mon, 13 Aug 2018 01:24:08 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 230402] With buildworld, the system can not use swap Date: Mon, 13 Aug 2018 01:24:07 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: see_also Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 01:24:10 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230402 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.freebsd.org/bu | |gzilla/show_bug.cgi?id=3D2= 276 | |09 --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-virtualization@freebsd.org Mon Aug 13 01:25:11 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A1F4D105F2A7 for ; Mon, 13 Aug 2018 01:25:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 3BA3C82CBF for ; Mon, 13 Aug 2018 01:25:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 00987105F2A3; Mon, 13 Aug 2018 01:25:11 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E3AC7105F2A0 for ; Mon, 13 Aug 2018 01:25:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6AE2782CB7 for ; Mon, 13 Aug 2018 01:25:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id B9341252EE for ; Mon, 13 Aug 2018 01:25:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7D1P9kD036890 for ; Mon, 13 Aug 2018 01:25:09 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7D1P9lV036889 for virtualization@FreeBSD.org; Mon, 13 Aug 2018 01:25:09 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 230402] With buildworld, the system can not use swap Date: Mon, 13 Aug 2018 01:25:09 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: see_also bug_severity Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 01:25:11 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230402 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.freebsd.org/bu | |gzilla/show_bug.cgi?id=3D2= 304 | |54 Severity|Affects Only Me |Affects Some People --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-virtualization@freebsd.org Mon Aug 13 02:04:22 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5B21D1061AD8 for ; Mon, 13 Aug 2018 02:04:22 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from relay2.tomsk.ru (mail.sibptus.tomsk.ru [212.73.124.5]) by mx1.freebsd.org (Postfix) with ESMTP id AF066843FC for ; Mon, 13 Aug 2018 02:04:20 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) X-Virus-Scanned: by clamd daemon 0.100.1 for FreeBSD at relay.sibptus.ru Received: from [212.73.125.240] (HELO admin.sibptus.transneft.ru) by relay2.tomsk.ru (CommuniGate Pro SMTP 5.1.16) with ESMTPS id 40076190; Mon, 13 Aug 2018 09:04:18 +0700 Received: from admin.sibptus.transneft.ru (sudakov@localhost [127.0.0.1]) by admin.sibptus.transneft.ru (8.15.2/8.15.2) with ESMTP id w7D24GsC083651; Mon, 13 Aug 2018 09:04:18 +0700 (+07) (envelope-from vas@mpeks.tomsk.su) Received: (from sudakov@localhost) by admin.sibptus.transneft.ru (8.15.2/8.15.2/Submit) id w7D24CBL083647; Mon, 13 Aug 2018 09:04:12 +0700 (+07) (envelope-from vas@mpeks.tomsk.su) X-Authentication-Warning: admin.sibptus.transneft.ru: sudakov set sender to vas@mpeks.tomsk.su using -f Date: Mon, 13 Aug 2018 09:04:12 +0700 From: Victor Sudakov To: "Rodney W. Grimes" Cc: Shawn Webb , freebsd-virtualization@freebsd.org Subject: Re: Curent Centos 7 and bhyve Message-ID: <20180813020412.GA83165@admin.sibptus.transneft.ru> References: <20180812052025.GB73103@admin.sibptus.transneft.ru> <201808121459.w7CExSWX032888@pdx.rh.CN85.dnsmgr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201808121459.w7CExSWX032888@pdx.rh.CN85.dnsmgr.net> Organization: AO "Svyaztransneft", SibPTUS X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.9.5 (2018-04-13) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 02:04:22 -0000 Rodney W. Grimes wrote: [dd] > > > > > > > > > > Though it has a lot of features, it also has some short comings, > > > > > like you can not spec a vm to be wired in memory, which IMHO is > > > > > the only way to insure consistent VM performance. > > > > > > > > Well, we have "bhyve_options" configuration option in the vm config, > > > > why not put "-S" there, is that what you mean by wiring the vm in > > > > memory? > > > > > > I believe that fails as that only adds the -S to bhyve, and > > > you must specify it both on bhyveload and bhyve for it to > > > work. > > > > I think it is totally doable becase vm-bhyve is nothing but a suit of > > scripts. A PR with a feature request would be appropriate. > > I made several attempts to contact the author at the email address > provided at the git hub while making other bhyve changes to try > and coordinate with him. I got no response after 3 attempts, > so have stopped trying to contact them. (This was while I was > adding the -c cpu topology modifications.) You can add yourself to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230580 maybe something useful comes out of it. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 From owner-freebsd-virtualization@freebsd.org Mon Aug 13 08:15:10 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C2DF01069E9F for ; Mon, 13 Aug 2018 08:15:10 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-a.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44D518F86C for ; Mon, 13 Aug 2018 08:15:09 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-a.userve.net (Postfix) with ESMTPS id 14343C80; Mon, 13 Aug 2018 09:15:01 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1534148101; bh=bTqHqQ6IB5TdqjDaIwCvbJNPE/osXzkGj/lqWwezmhg=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=Liee9ETF8tZftBtRgeDqFoyFbFSqtdTMO0MTuwOPAsaxH2QjI+3V39ZHCkZZNTpW7 ZF7VHgUgGvIUH52Z5BnBJPOw/jHuqm2dNYirjnS/sYgvIMMVJc00XJ8592Gx7LDwbj ecdw9tjo/WKMUlHM7gs4EtPkPKtpTn9ANqnRyGNU= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Mon, 13 Aug 2018 09:15:00 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Mon, 13 Aug 2018 09:15:00 +0100 From: Matt Churchyard To: Victor Sudakov , "Rodney W. Grimes" CC: "freebsd-virtualization@freebsd.org" Subject: RE: Curent Centos 7 and bhyve Thread-Topic: Curent Centos 7 and bhyve Thread-Index: AQHUMQRwp/VmOgvRrEWW0WbxakcQ6aS5/FiAgACE6YCAAAzbAIAAANiAgAAG4oCAAAHfgIAAFGsAgAADS4CAANYhgIAAockAgAC5ugCAAHVP0A== Date: Mon, 13 Aug 2018 08:14:59 +0000 Message-ID: <1b484d237c4a4f4dabcbd7b0eece7675@SERVER.ad.usd-group.com> References: <20180812052025.GB73103@admin.sibptus.transneft.ru> <201808121459.w7CExSWX032888@pdx.rh.CN85.dnsmgr.net> <20180813020412.GA83165@admin.sibptus.transneft.ru> In-Reply-To: <20180813020412.GA83165@admin.sibptus.transneft.ru> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 08:15:11 -0000 > Rodney W. Grimes wrote: [dd] > > > > >=20 > > > > > Though it has a lot of features, it also has some short=20 > > > > > comings, like you can not spec a vm to be wired in memory,=20 > > > > > which IMHO is the only way to insure consistent VM performance. > > > >=20 > > > > Well, we have "bhyve_options" configuration option in the vm=20 > > > > config, why not put "-S" there, is that what you mean by wiring=20 > > > > the vm in memory? > > >=20 > > > I believe that fails as that only adds the -S to bhyve, and you=20 > > > must specify it both on bhyveload and bhyve for it to work. > >=20 > > I think it is totally doable becase vm-bhyve is nothing but a suit=20 > > of scripts. A PR with a feature request would be appropriate. >=20 > I made several attempts to contact the author at the email address=20 > provided at the git hub while making other bhyve changes to try and=20 > coordinate with him. I got no response after 3 attempts, > so have stopped trying to contact them. (This was while I was > adding the -c cpu topology modifications.) > You can add yourself to > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230580 > maybe something useful comes out of it. I've already commented on the bug, although I'll reply here as well. If "-S" is found in bhyve_options it does currently affect both commands. I= have decided that a specific wired_memory option is useful though and will= add this to the next release. The name limit has been increased to 32 since v1.2 I didn't realize the changes for cpu topology had actually made it into hea= d, although I don't believe it's actually in a release yet? I will plan to = support configuration and display of these before 12 release. Matt > -- > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > AS43859 > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org= /mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@free= bsd.org" From owner-freebsd-virtualization@freebsd.org Mon Aug 13 09:53:10 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22E01106CAAE for ; Mon, 13 Aug 2018 09:53:10 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from relay2.tomsk.ru (mail.sibptus.tomsk.ru [212.73.124.5]) by mx1.freebsd.org (Postfix) with ESMTP id 65BF3729B1 for ; Mon, 13 Aug 2018 09:53:09 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) X-Virus-Scanned: by clamd daemon 0.100.1 for FreeBSD at relay.sibptus.ru Received: from [212.73.125.240] (HELO admin.sibptus.transneft.ru) by relay2.tomsk.ru (CommuniGate Pro SMTP 5.1.16) with ESMTPS id 40076403; Mon, 13 Aug 2018 16:53:07 +0700 Received: from admin.sibptus.transneft.ru (sudakov@localhost [127.0.0.1]) by admin.sibptus.transneft.ru (8.15.2/8.15.2) with ESMTP id w7D9r7dC099936; Mon, 13 Aug 2018 16:53:07 +0700 (+07) (envelope-from vas@mpeks.tomsk.su) Received: (from sudakov@localhost) by admin.sibptus.transneft.ru (8.15.2/8.15.2/Submit) id w7D9r21H099933; Mon, 13 Aug 2018 16:53:02 +0700 (+07) (envelope-from vas@mpeks.tomsk.su) X-Authentication-Warning: admin.sibptus.transneft.ru: sudakov set sender to vas@mpeks.tomsk.su using -f Date: Mon, 13 Aug 2018 16:53:01 +0700 From: Victor Sudakov To: Matt Churchyard Cc: "Rodney W. Grimes" , "freebsd-virtualization@freebsd.org" Subject: Re: Curent Centos 7 and bhyve Message-ID: <20180813095301.GA99455@admin.sibptus.transneft.ru> References: <20180812052025.GB73103@admin.sibptus.transneft.ru> <201808121459.w7CExSWX032888@pdx.rh.CN85.dnsmgr.net> <20180813020412.GA83165@admin.sibptus.transneft.ru> <1b484d237c4a4f4dabcbd7b0eece7675@SERVER.ad.usd-group.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1b484d237c4a4f4dabcbd7b0eece7675@SERVER.ad.usd-group.com> Organization: AO "Svyaztransneft", SibPTUS X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.9.5 (2018-04-13) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 09:53:10 -0000 Matt Churchyard wrote: > > Rodney W. Grimes wrote: > > [dd] > > > I've already commented on the bug, although I'll reply here as well. > If "-S" is found in bhyve_options it does currently affect both > commands. Sorry, this is not working for me (vm-bhyve 1.1.8_2 from official packages): Aug 13 16:43:09: bhyve exited with status 1 Aug 13 16:43:09: destroying network device tap0 Aug 13 16:43:09: stopped Aug 13 16:43:12: initialising Aug 13 16:43:12: [loader: bhyveload] Aug 13 16:43:12: [uefi: no] Aug 13 16:43:12: [cpu: 1] Aug 13 16:43:12: [memory: 1G] Aug 13 16:43:12: [hostbridge: standard] Aug 13 16:43:12: [com ports: com1] Aug 13 16:43:12: [uuid: e148afdb-d4af-11e6-9a94-5404a6b49a66] Aug 13 16:43:12: [utctime: yes] Aug 13 16:43:12: [debug mode: no] Aug 13 16:43:12: [primary disk: disk0.img] Aug 13 16:43:12: [primary disk dev: file] Aug 13 16:43:12: initialising network device tap0 Aug 13 16:43:12: adding tap0 -> bridge0 (main) Aug 13 16:43:12: booting Aug 13 16:43:12: bhyveload -c /dev/nmdm0A -m 1G -e autoboot_delay=3 -d /d02/vm/fido/disk0.img fido Aug 13 16:43:17: [bhyve options: -c 1 -m 1G -AHP -S -U e148afdb-d4af-11e6-9a94-5404a6b49a66 -u] Aug 13 16:43:17: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/d02/vm/fido/disk0.img -s 5:0,virtio-net,tap0,mac=58:9c:fc:01:f6:ac] Aug 13 16:43:17: [bhyve console: -l com1,/dev/nmdm0A] Aug 13 16:43:17: starting bhyve (run 1) Aug 13 16:43:17: bhyve exited with status 1 Aug 13 16:43:17: destroying network device tap0 Aug 13 16:43:17: stopped Here is my complete config: guest="freebsd" loader="bhyveload" cpu=1 memory=1G network0_type="virtio-net" network0_switch="main" disk0_type="virtio-blk" disk0_name="disk0.img" utctime="yes" uuid="e148afdb-d4af-11e6-9a94-5404a6b49a66" network0_mac="58:9c:fc:01:f6:ac" bhyve_options="-S" -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 From owner-freebsd-virtualization@freebsd.org Mon Aug 13 10:00:52 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B09D2106CC56 for ; Mon, 13 Aug 2018 10:00:52 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-b.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4E9B172C48 for ; Mon, 13 Aug 2018 10:00:52 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-b.userve.net (Postfix) with ESMTPS id 16AC2F7D; Mon, 13 Aug 2018 11:00:51 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1534154451; bh=/DqXDX25/AcD68Sedaot+FA0zekptYFpWaTFW3wb7a4=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=c4y0wThG9JICj9l9OvLWW1579BbmVitC1HLWFS0gwJBuNAeGM2I4K0LqaNA2DRaZA 2iRIDTOPKJ9P+dwGt1c7dTAD80DMFHHgXkRZtLRIYVns2+hxlYcQDiXAd8mpThyQSf 6U+euiUB0gHnYDZpu2JX5PpnD2+1v3epM0w/urgQ= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Mon, 13 Aug 2018 11:00:50 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Mon, 13 Aug 2018 11:00:50 +0100 From: Matt Churchyard To: Victor Sudakov CC: "Rodney W. Grimes" , "freebsd-virtualization@freebsd.org" Subject: RE: Curent Centos 7 and bhyve Thread-Topic: Curent Centos 7 and bhyve Thread-Index: AQHUMQRwp/VmOgvRrEWW0WbxakcQ6aS5/FiAgACE6YCAAAzbAIAAANiAgAAG4oCAAAHfgIAAFGsAgAADS4CAANYhgIAAockAgAC5ugCAAHVP0IAADa2AgAAR4PA= Date: Mon, 13 Aug 2018 10:00:50 +0000 Message-ID: <826973c6d017464fa87c13310a773ec6@SERVER.ad.usd-group.com> References: <20180812052025.GB73103@admin.sibptus.transneft.ru> <201808121459.w7CExSWX032888@pdx.rh.CN85.dnsmgr.net> <20180813020412.GA83165@admin.sibptus.transneft.ru> <1b484d237c4a4f4dabcbd7b0eece7675@SERVER.ad.usd-group.com> <20180813095301.GA99455@admin.sibptus.transneft.ru> In-Reply-To: <20180813095301.GA99455@admin.sibptus.transneft.ru> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 10:00:53 -0000 -----Original Message----- From: owner-freebsd-virtualization@freebsd.org On Behalf Of Victor Sudakov Sent: 13 August 2018 10:53 To: Matt Churchyard Cc: Rodney W. Grimes ; freebsd-virtuali= zation@freebsd.org Subject: Re: Curent Centos 7 and bhyve Matt Churchyard wrote: > > Rodney W. Grimes wrote: >=20 > [dd] >=20 >=20 > I've already commented on the bug, although I'll reply here as well. > If "-S" is found in bhyve_options it does currently affect both=20 > commands. >Sorry, this is not working for me (vm-bhyve 1.1.8_2 from official packages= ): >Aug 13 16:43:09: bhyve exited with status 1 Aug 13 16:43:09: destroying ne= twork device tap0 Aug 13 16:43:09: stopped Aug 13 16:43:12: initialising Au= g 13 16:43:12: [loader: bhyveload] Aug 13 16:43:12: [uefi: no] Aug 13 16:= 43:12: [cpu: 1] Aug 13 16:43:12: [memory: 1G] Aug 13 16:43:12: [hostbrid= ge: standard] Aug 13 16:43:12: [com ports: com1] Aug 13 16:43:12: [uuid: = e148afdb-d4af-11e6-9a94-5404a6b49a66] >Aug 13 16:43:12: [utctime: yes] >Aug 13 16:43:12: [debug mode: no] >Aug 13 16:43:12: [primary disk: disk0.img] Aug 13 16:43:12: [primary dis= k dev: file] Aug 13 16:43:12: initialising network device tap0 Aug 13 16:43= :12: adding tap0 -> bridge0 (main) Aug 13 16:43:12: booting Aug 13 16:43:12= : bhyveload -c /dev/nmdm0A -m 1G -e autoboot_delay=3D3 -d /d02/vm/fido/disk= 0.img fido Aug 13 16:43:17: [bhyve options: -c 1 -m 1G -AHP -S -U e148afdb= -d4af-11e6-9a94-5404a6b49a66 -u] Aug 13 16:43:17: [bhyve devices: -s 0,hos= tbridge -s 31,lpc -s 4:0,virtio-blk,/d02/vm/fido/disk0.img -s 5:0,virtio-ne= t,tap0,mac=3D58:9c:fc:01:f6:ac] >Aug 13 16:43:17: [bhyve console: -l com1,/dev/nmdm0A] Aug 13 16:43:17: st= arting bhyve (run 1) Aug 13 16:43:17: bhyve exited with status 1 Aug 13 16:= 43:17: destroying network device tap0 Aug 13 16:43:17: stopped Sorry, it looks like the change didn't make it into 1.1. You'd need to upgr= ade to 1.2, which is in ports but looks like it hasn't made it into pkg yet= . portsmon.freebsd.org is still broken also so I can't easily confirm wheth= er it's made it into any of the quarterly/latest pkg repos. (It was in port= s over a month ago so should at least be in latest) Matt >Here is my complete config: >guest=3D"freebsd" >loader=3D"bhyveload" >cpu=3D1 >memory=3D1G >network0_type=3D"virtio-net" >network0_switch=3D"main" >disk0_type=3D"virtio-blk" >disk0_name=3D"disk0.img" >utctime=3D"yes" >uuid=3D"e148afdb-d4af-11e6-9a94-5404a6b49a66" >network0_mac=3D"58:9c:fc:01:f6:ac" >bhyve_options=3D"-S" >-- >Victor Sudakov, VAS4-RIPE, VAS47-RIPN >AS43859 _______________________________________________ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/m= ailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebs= d.org" From owner-freebsd-virtualization@freebsd.org Mon Aug 13 11:00:37 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6E376106E8A5 for ; Mon, 13 Aug 2018 11:00:37 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from relay2.tomsk.ru (mail.sibptus.tomsk.ru [212.73.124.5]) by mx1.freebsd.org (Postfix) with ESMTP id C04BF744DE for ; Mon, 13 Aug 2018 11:00:36 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) X-Virus-Scanned: by clamd daemon 0.100.1 for FreeBSD at relay.sibptus.ru Received: from [212.73.125.240] (HELO admin.sibptus.transneft.ru) by relay2.tomsk.ru (CommuniGate Pro SMTP 5.1.16) with ESMTPS id 40076448; Mon, 13 Aug 2018 18:00:35 +0700 Received: from admin.sibptus.transneft.ru (sudakov@localhost [127.0.0.1]) by admin.sibptus.transneft.ru (8.15.2/8.15.2) with ESMTP id w7DB0Zdx002751; Mon, 13 Aug 2018 18:00:35 +0700 (+07) (envelope-from vas@mpeks.tomsk.su) Received: (from sudakov@localhost) by admin.sibptus.transneft.ru (8.15.2/8.15.2/Submit) id w7DB0VgJ002748; Mon, 13 Aug 2018 18:00:31 +0700 (+07) (envelope-from vas@mpeks.tomsk.su) X-Authentication-Warning: admin.sibptus.transneft.ru: sudakov set sender to vas@mpeks.tomsk.su using -f Date: Mon, 13 Aug 2018 18:00:31 +0700 From: Victor Sudakov To: Matt Churchyard Cc: "Rodney W. Grimes" , "freebsd-virtualization@freebsd.org" Subject: Re: Curent Centos 7 and bhyve Message-ID: <20180813110031.GA2623@admin.sibptus.transneft.ru> References: <20180812052025.GB73103@admin.sibptus.transneft.ru> <201808121459.w7CExSWX032888@pdx.rh.CN85.dnsmgr.net> <20180813020412.GA83165@admin.sibptus.transneft.ru> <1b484d237c4a4f4dabcbd7b0eece7675@SERVER.ad.usd-group.com> <20180813095301.GA99455@admin.sibptus.transneft.ru> <826973c6d017464fa87c13310a773ec6@SERVER.ad.usd-group.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <826973c6d017464fa87c13310a773ec6@SERVER.ad.usd-group.com> Organization: AO "Svyaztransneft", SibPTUS X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.9.5 (2018-04-13) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 11:00:37 -0000 Matt Churchyard wrote: > wired_memory option has been added to next release. Thanks a lot, Matt! May I ask a question: how is the "-S" option supposed to work if the loader is not bhyveload, but grub2-bhyve or UEFI? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 From owner-freebsd-virtualization@freebsd.org Mon Aug 13 11:14:42 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3127A106F143 for ; Mon, 13 Aug 2018 11:14:42 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-a.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C045E74EAA for ; Mon, 13 Aug 2018 11:14:41 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-a.userve.net (Postfix) with ESMTPS id 8E562385; Mon, 13 Aug 2018 12:14:40 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1534158880; bh=TGrXEaU2AO1iI8HvtG6fYV/nyYAKUwIgqihJlHrt1Uk=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=Mkdgy0EDVa0uxJ+LPRbCslCPqw2V8iEHGiX0FIAJJIGNm6u8IVBAuUko46fn0HMZ1 0n1yxQ/beCvgSVF8c5NKEEzSWZO+h4nGEQUBDsc67lZFJeYAoKyPSEBpPWR6rbt3Se bszhfPiRXj/3imFBlkmttLZMvvSzYSZbjA5fpcOQ= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Mon, 13 Aug 2018 12:14:40 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Mon, 13 Aug 2018 12:14:39 +0100 From: Matt Churchyard To: Victor Sudakov CC: "Rodney W. Grimes" , "freebsd-virtualization@freebsd.org" Subject: RE: Curent Centos 7 and bhyve Thread-Topic: Curent Centos 7 and bhyve Thread-Index: AQHUMQRwp/VmOgvRrEWW0WbxakcQ6aS5/FiAgACE6YCAAAzbAIAAANiAgAAG4oCAAAHfgIAAFGsAgAADS4CAANYhgIAAockAgAC5ugCAAHVP0IAADa2AgAAR4PCAAAD8gIAAElSg Date: Mon, 13 Aug 2018 11:14:39 +0000 Message-ID: <9920963a9c414fe7bc4df8e921efda27@SERVER.ad.usd-group.com> References: <20180812052025.GB73103@admin.sibptus.transneft.ru> <201808121459.w7CExSWX032888@pdx.rh.CN85.dnsmgr.net> <20180813020412.GA83165@admin.sibptus.transneft.ru> <1b484d237c4a4f4dabcbd7b0eece7675@SERVER.ad.usd-group.com> <20180813095301.GA99455@admin.sibptus.transneft.ru> <826973c6d017464fa87c13310a773ec6@SERVER.ad.usd-group.com> <20180813110031.GA2623@admin.sibptus.transneft.ru> In-Reply-To: <20180813110031.GA2623@admin.sibptus.transneft.ru> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 11:14:42 -0000 > Matt Churchyard wrote: > wired_memory option has been added to next release. > Thanks a lot, Matt! May I ask a question: how is the "-S" option supposed= to work if the loader is not bhyveload, but grub2-bhyve or UEFI? In 1.2, if "-S" is found in bhyve_options, it adds this argument to the gue= st loader. This is done before choosing between bhyveload/grub2-bhyve and w= ill be supplied to either. In 1.3 (GitHub only), which has a new wired_memory option, you will need to= use this, as the slightly hacky grep to scan bhyve_options for -S and push= this through to the loader has been removed. As far as I'm aware this is not an issue with UEFI as there is now just a s= ingle bhyve process (which gets all of bhyve_options) and no separate loade= r. Matt -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 _______________________________________________ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/m= ailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebs= d.org" From owner-freebsd-virtualization@freebsd.org Mon Aug 13 15:30:08 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E181B1075BA7 for ; Mon, 13 Aug 2018 15:30:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 7AAA57E86E for ; Mon, 13 Aug 2018 15:30:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 3BC9C1075BA6; Mon, 13 Aug 2018 15:30:08 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A8251075BA5 for ; Mon, 13 Aug 2018 15:30:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BF7607E86D for ; Mon, 13 Aug 2018 15:30:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 10CE6CA62 for ; Mon, 13 Aug 2018 15:30:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7DFU6Cn001565 for ; Mon, 13 Aug 2018 15:30:06 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7DFU60h001561 for virtualization@FreeBSD.org; Mon, 13 Aug 2018 15:30:06 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 230580] sysutils/vm-bhyve: Add support for guest memory wiring Date: Mon, 13 Aug 2018 15:30:06 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: feature, needs-patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rgrimes@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 15:30:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230580 Rodney W. Grimes changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |virtualization@FreeBSD.org --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-virtualization@freebsd.org Mon Aug 13 15:32:53 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EBAE41075D5D for ; Mon, 13 Aug 2018 15:32:52 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 73C237F07A for ; Mon, 13 Aug 2018 15:32:52 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w7DFWf24037669; Mon, 13 Aug 2018 08:32:41 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w7DFWeK0037668; Mon, 13 Aug 2018 08:32:40 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201808131532.w7DFWeK0037668@pdx.rh.CN85.dnsmgr.net> Subject: Re: Curent Centos 7 and bhyve In-Reply-To: <20180813020412.GA83165@admin.sibptus.transneft.ru> To: Victor Sudakov Date: Mon, 13 Aug 2018 08:32:40 -0700 (PDT) CC: Shawn Webb , freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 15:32:53 -0000 > Rodney W. Grimes wrote: > > [dd] > > > > > > > > > > > > > Though it has a lot of features, it also has some short comings, > > > > > > like you can not spec a vm to be wired in memory, which IMHO is > > > > > > the only way to insure consistent VM performance. > > > > > > > > > > Well, we have "bhyve_options" configuration option in the vm config, > > > > > why not put "-S" there, is that what you mean by wiring the vm in > > > > > memory? > > > > > > > > I believe that fails as that only adds the -S to bhyve, and > > > > you must specify it both on bhyveload and bhyve for it to > > > > work. > > > > > > I think it is totally doable becase vm-bhyve is nothing but a suit of > > > scripts. A PR with a feature request would be appropriate. > > > > I made several attempts to contact the author at the email address > > provided at the git hub while making other bhyve changes to try > > and coordinate with him. I got no response after 3 attempts, > > so have stopped trying to contact them. (This was while I was > > adding the -c cpu topology modifications.) > > You can add yourself to > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230580 > maybe something useful comes out of it. I added virtualization@ to the PR, though it looks as if wired_memory= has already been added as an option, which is what I did locally. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Mon Aug 13 16:27:23 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 554A31076FCF for ; Mon, 13 Aug 2018 16:27:23 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B930C80AE7 for ; Mon, 13 Aug 2018 16:27:22 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w7DGRKDg037841; Mon, 13 Aug 2018 09:27:20 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w7DGRKKT037840; Mon, 13 Aug 2018 09:27:20 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201808131627.w7DGRKKT037840@pdx.rh.CN85.dnsmgr.net> Subject: Re: Curent Centos 7 and bhyve In-Reply-To: <1b484d237c4a4f4dabcbd7b0eece7675@SERVER.ad.usd-group.com> To: Matt Churchyard Date: Mon, 13 Aug 2018 09:27:20 -0700 (PDT) CC: Victor Sudakov , "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 16:27:23 -0000 > > Rodney W. Grimes wrote: > > [dd] > > > > > > > > > > > > > Though it has a lot of features, it also has some short > > > > > > comings, like you can not spec a vm to be wired in memory, > > > > > > which IMHO is the only way to insure consistent VM performance. > > > > > > > > > > Well, we have "bhyve_options" configuration option in the vm > > > > > config, why not put "-S" there, is that what you mean by wiring > > > > > the vm in memory? > > > > > > > > I believe that fails as that only adds the -S to bhyve, and you > > > > must specify it both on bhyveload and bhyve for it to work. > > > > > > I think it is totally doable becase vm-bhyve is nothing but a suit > > > of scripts. A PR with a feature request would be appropriate. > > > > I made several attempts to contact the author at the email address > > provided at the git hub while making other bhyve changes to try and > > coordinate with him. I got no response after 3 attempts, > > so have stopped trying to contact them. (This was while I was > > adding the -c cpu topology modifications.) > > > You can add yourself to > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230580 > > maybe something useful comes out of it. > > I've already commented on the bug, although I'll reply here as well. > If "-S" is found in bhyve_options it does currently affect both commands. I have decided that a specific wired_memory option is useful though and will add this to the next release. > > The name limit has been increased to 32 since v1.2 This is better, but still an artificial limit, the implementation of bhyve allows this string to be any valid filename, the scripts should be designed to allow for that as the valid limit. I modified the script so that the vm name is the last column, and removed the length check all togeather, this allows for the string to be what ever length and not mess with column widths. root@x230a:# vm list DATASTORE LOADER CPU MEMORY VNC AUTOSTART STATE NAME default bhyveload 1 128M - No Stopped fb-bld-10-amd64 default bhyveload 4 2048M - No Stopped fb-bld-11-amd64 default bhyveload 4 1024M - No Stopped fb-bld-11-i386 default bhyveload 1 128M - No Stopped fb-bld-11.0-p1-amd64 default bhyveload 1 128M - No Stopped fb-bld-11.0-p1-i386 default bhyveload 4 512M - No Stopped fb-bld-11.1-amd64 default bhyveload 4 512M - No Stopped fb-bld-11.1-i386 > I didn't realize the changes for cpu topology had actually made it > into head, although I don't believe it's actually in a release yet? Yes, they are in head, the MFC has been delayed for other reasons. It is in the 12.0-ALPHA1 snapshot, and many before that. > I will plan to support configuration and display of these > before 12 release. Thanks. I think mostly just extract the NCPU's from the topology string. The code actually works now, but due to fixed column width assumptions the output looks bad. > > Matt > > > -- > > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > > AS43859 > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > > -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Tue Aug 14 01:34:32 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D50E105FF70 for ; Tue, 14 Aug 2018 01:34:32 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from relay2.tomsk.ru (mail.sibptus.tomsk.ru [212.73.124.5]) by mx1.freebsd.org (Postfix) with ESMTP id EF5E176D90 for ; Tue, 14 Aug 2018 01:34:30 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) X-Virus-Scanned: by clamd daemon 0.100.1 for FreeBSD at relay.sibptus.ru Received: from [212.73.125.240] (HELO admin.sibptus.transneft.ru) by relay2.tomsk.ru (CommuniGate Pro SMTP 5.1.16) with ESMTPS id 40076935; Tue, 14 Aug 2018 08:34:29 +0700 Received: from admin.sibptus.transneft.ru (sudakov@localhost [127.0.0.1]) by admin.sibptus.transneft.ru (8.15.2/8.15.2) with ESMTP id w7E1YSE1009796; Tue, 14 Aug 2018 08:34:28 +0700 (+07) (envelope-from vas@mpeks.tomsk.su) Received: (from sudakov@localhost) by admin.sibptus.transneft.ru (8.15.2/8.15.2/Submit) id w7E1YOHv009795; Tue, 14 Aug 2018 08:34:24 +0700 (+07) (envelope-from vas@mpeks.tomsk.su) X-Authentication-Warning: admin.sibptus.transneft.ru: sudakov set sender to vas@mpeks.tomsk.su using -f Date: Tue, 14 Aug 2018 08:34:24 +0700 From: Victor Sudakov To: Matt Churchyard Cc: "Rodney W. Grimes" , "freebsd-virtualization@freebsd.org" Subject: Re: Curent Centos 7 and bhyve Message-ID: <20180814013424.GA9706@admin.sibptus.transneft.ru> References: <20180812052025.GB73103@admin.sibptus.transneft.ru> <201808121459.w7CExSWX032888@pdx.rh.CN85.dnsmgr.net> <20180813020412.GA83165@admin.sibptus.transneft.ru> <1b484d237c4a4f4dabcbd7b0eece7675@SERVER.ad.usd-group.com> <20180813095301.GA99455@admin.sibptus.transneft.ru> <826973c6d017464fa87c13310a773ec6@SERVER.ad.usd-group.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <826973c6d017464fa87c13310a773ec6@SERVER.ad.usd-group.com> Organization: AO "Svyaztransneft", SibPTUS X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.9.5 (2018-04-13) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Aug 2018 01:34:32 -0000 Matt Churchyard wrote: > > Sorry, it looks like the change didn't make it into 1.1. You'd need > to upgrade to 1.2, which is in ports but looks like it hasn't made > it into pkg yet. portsmon.freebsd.org is still broken also so I > can't easily confirm whether it's made it into any of the > quarterly/latest pkg repos. (It was in ports over a month ago so > should at least be in latest) I see. My own poudriere has just built vm-bhyve-1.2.3, but I don't use my repo on that bhyve host, I use the official repo there. Does anyone know if I can install just one (or some) package from a different repo, the official one still remaining the default? -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN AS43859 From owner-freebsd-virtualization@freebsd.org Tue Aug 14 02:48:35 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C82A810668CC for ; Tue, 14 Aug 2018 02:48:35 +0000 (UTC) (envelope-from steven.e.friedrich@gmail.com) Received: from mail-it0-x22d.google.com (mail-it0-x22d.google.com [IPv6:2607:f8b0:4001:c0b::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5D6FB7A382 for ; Tue, 14 Aug 2018 02:48:35 +0000 (UTC) (envelope-from steven.e.friedrich@gmail.com) Received: by mail-it0-x22d.google.com with SMTP id 72-v6so16259882itw.3 for ; Mon, 13 Aug 2018 19:48:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:subject:message-id:from:to:mime-version :content-transfer-encoding; bh=iR0bmMDSsf9MHfgprdxQ/+WyA9v4nDhhibIDkv7QveE=; b=LvmwZJO0/5lA1I/7PgXrIhy8gIYaMxBTiBGd3RhxATjoNR9KJ4OoWqng0dpNPIfJ1o nAqmhF/QMZ9pfY0MBgEkRCEZH8W2NxNG329XASzMiLe7KMfVnFm2/wgEOeR3GbH6l1BI DVW1oIncJgosFJfBVabWvBndgnqzgcBV3eh/qUKzmKUpI+JIqfugPS5a/jif2ryLwXOS 8flP/PMkM+2cqRRupRjR3bRVeDo/CwiIwFHyNTQAw669Cyp6epcVpa+49lE6acWWQrDA ieijyeP5oql+zVmGMIMKF6jDyXXucOktKTicxuapBkcEhbWxHIFcBTThDh2T2AVGFdOv 56Uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:subject:message-id:from:to:mime-version :content-transfer-encoding; bh=iR0bmMDSsf9MHfgprdxQ/+WyA9v4nDhhibIDkv7QveE=; b=bYvvF0eUZ2cEB0roXwpPthMrUvx6NO16MzRs/P/rHpjjAxGQUOl2DW8/SGlpqbpSkF dJ2giR3Ea6uMmMSLkUYPRMaMELyVyNGYI2b8yyPWUPBxutTxa1EwDcDwAiupxG6wHl0a 33ZwkEpjznr+iMuc68EJjx12gwkpdijvLxR61nRvLwAqtJVOggKSfNO1kxHVoKQuJbXx UbtfmI9of7QF5XhdmJ3S0fvAGXX4ZyE5R8OzkvVEnyQCH0mTi+tSWazDLc0RUv/AH+0M o0P/lnuaX4e414dWHp4ZQcBi3T/h9CDK5Yzd0S53BW/dus2UQFCUwoAvozNZ7UnkjBod 4eaA== X-Gm-Message-State: AOUpUlHWTT8Znhd5DjqLBG9CC2Gb6z8/VpQlN0eTlQ3UKUyTdfcP81zF 6dI0iyYcBIzt0lhubTOh7mHa6vaZ X-Google-Smtp-Source: AA+uWPwoxhWJ6FZokW3yy0bZcUotdIjwv/nLO9mcvk6oWP1SR4IXOBN9UsD8uIpP87ubG6ME/UrdNw== X-Received: by 2002:a02:952a:: with SMTP id y39-v6mr17897770jah.60.1534214914327; Mon, 13 Aug 2018 19:48:34 -0700 (PDT) Received: from [192.168.1.4] (cpe-74-132-25-214.kya.res.rr.com. [74.132.25.214]) by smtp.gmail.com with ESMTPSA id j199-v6sm8749831itb.43.2018.08.13.19.48.33 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Aug 2018 19:48:33 -0700 (PDT) Date: Mon, 13 Aug 2018 22:48:31 -0400 Subject: Xen host on FreeBSD 11.2-RELEASE Message-ID: From: Steven Friedrich To: "freebsd-virtualization@freebsd.org" MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Aug 2018 02:48:35 -0000 SSBoYXZlIGEgbmV3IEhQIFNsaW1saW5lIERlc2t0b3AgMjkwLXAwMDE0IHdoaWNoIGZlYXR1cmVz IGFuIEludGVsIGk3LTg3MDAgYW5kIG5vIGNvbSBwb3J0cy4KSSB3YXMgaG9waW5nIHRvIGNyZWF0 ZSBhIEZyZWVCU0QgWGVuIGhvc3QgdG8gZXhwbG9yZSB2YXJpb3VzIExpbnV4IGRpc3Ryb3MuCkkg d2FzIGZvbGxvd2luZyBhbG9uZyB3aXRoIGNoYXB0ZXIgMjEuOCBGcmVlQlNEIGFzIGEgWGVuIC1I b3N0LgpJIGRpZCBhIGZyZXNoIGluc3RhbGwsIHBrZyBpbnN0YWxsIHhlbiBhbmQgd2hlbiBJIHJl Ym9vdGVkIGl0IGRpc3BsYXllZCBpbiBibHVlIEJvb3RpbmcsIGJ1dCB0aGVuIGl0IHR1cm5lZCBy ZWQuCkkgaGF2ZSBubyBzZXJpYWwgY29uc29sZSBiZWNhdXNlIEkgaGF2ZSBubyBjb20gcG9ydC4K Q2FuIGFueW9uZSBpbnZvbHZlZCB3aXRoIFhlbiB0cnkgdGhpcyBhbmQgcmV2ZWFsIGFueXRoaW5n IG5vdCBkaXNjbG9zZWQgaW4gdGhlIEhhbmRib29rPwpJIGFtIHVzaW5nIGhhbmRib29rIHJldiA1 MjExMyBtb2RpZmllZCBvbiAyMDE4LTA4LTEyIDA4OjUwOjIwIGJ5IGVhZGxlci4= From owner-freebsd-virtualization@freebsd.org Tue Aug 14 05:09:40 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4C981106BEB2 for ; Tue, 14 Aug 2018 05:09:40 +0000 (UTC) (envelope-from pratiy0100@gmail.com) Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C68E67E97A for ; Tue, 14 Aug 2018 05:09:39 +0000 (UTC) (envelope-from pratiy0100@gmail.com) Received: by mail-ed1-f43.google.com with SMTP id f23-v6so9418210edr.11 for ; Mon, 13 Aug 2018 22:09:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=nIIHSLccH4WiW5/t41ZCQgsDRCCOpiWhzpr8oaYWbpk=; b=uXATAGUJXNrr+pilp51Fvn11DbhETZxneR91UrdybKgYejne/miIqe7EGHNX3DIkGc ZGUh+QpA6rd4l80eCOq22kpIbcolbdFvtpQeaSsxofb6jY+4jqU9c8KNX4IAknUaae3h qaqkmuuva6XZZGsvnINkQcEZfIOu31qPQCLY4PtnfhNfmg2g8zZb0+M6MR9iUDUB/iDD YoVRrQgKHfsEIWmT1mURArKhyp+yTGwNxSn8pQOJNFz0OU0vIFJNJhx6TY7pwd+b8r+N Lhr9KrTLP11kyukAGiWtmGCPolc38DPp1fVLAHTn8bhBzpANU5zpEr0SY9GarRqjlAPN QgqQ== X-Gm-Message-State: AOUpUlEBX3vFm4bRKCq8Ypj/klnGimTKfnvozh/b3+eK44Z0O0y5hlAt UpKnEE20FPOzCGw3XASHlDYdTGVFTY0= X-Google-Smtp-Source: AA+uWPy8RorWWeaDbNSj6/2DjHtia8mwS4hboP8L6nMMNXNlxCiU322dMBCnIzbZijndFsRjuOey7A== X-Received: by 2002:a50:d54a:: with SMTP id f10-v6mr7325782edj.151.1534223378072; Mon, 13 Aug 2018 22:09:38 -0700 (PDT) Received: from mail-ed1-f50.google.com (mail-ed1-f50.google.com. [209.85.208.50]) by smtp.gmail.com with ESMTPSA id q5-v6sm6918840edn.83.2018.08.13.22.09.37 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Aug 2018 22:09:37 -0700 (PDT) Received: by mail-ed1-f50.google.com with SMTP id f23-v6so9418200edr.11 for ; Mon, 13 Aug 2018 22:09:37 -0700 (PDT) X-Received: by 2002:a50:b178:: with SMTP id l53-v6mr25380198edd.306.1534223377496; Mon, 13 Aug 2018 22:09:37 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Pratyush Yadav Date: Tue, 14 Aug 2018 10:39:01 +0530 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Xen host on FreeBSD 11.2-RELEASE To: steven.e.friedrich@gmail.com Cc: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Aug 2018 05:09:40 -0000 On Tue, Aug 14, 2018 at 8:19 AM Steven Friedrich wrote: > > I have a new HP Slimline Desktop 290-p0014 which features an Intel i7-8700 and no com ports. > I was hoping to create a FreeBSD Xen host to explore various Linux distros. > I was following along with chapter 21.8 FreeBSD as a Xen -Host. > I did a fresh install, pkg install xen and when I rebooted it displayed in blue Booting, but then it turned red. > I have no serial console because I have no com port. > Can anyone involved with Xen try this and reveal anything not disclosed in the Handbook? > I am using handbook rev 52113 modified on 2018-08-12 08:50:20 by eadler. I'm not quite sure I understand your question, but I assume your screen turns blank after booting, and stays like that. If so, then that is the default behavior. Xen by default does not keep using the vga console. Add this to your command line parameters (the xen_cmdline option in your /boot/loader.conf): vga=current,keep So if you are using the default Xen command line options specified in the handbook page, your xen_cmdline should look something like: xen_cmdline="dom0_mem=8192M dom0_max_vcpus=4 dom0pvh=1 console=com1,vga com1=115200,8n1 guest_loglvl=all loglvl=all vga=current,keep" If this does not work, try doing vga=ask,keep and then play around with the different vga mode options (try the text- options first). Check out this document for more details on the Xen command line options [0]. Also, freebsd-xen@freebsd.org would be a better place for Xen related questions. Regards, Pratyush Yadav [0] https://xenbits.xen.org/docs/4.6-testing/misc/xen-command-line.html From owner-freebsd-virtualization@freebsd.org Tue Aug 14 08:26:13 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0A0BB1070761 for ; Tue, 14 Aug 2018 08:26:13 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-b.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 945C584305 for ; Tue, 14 Aug 2018 08:26:12 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-b.userve.net (Postfix) with ESMTPS id E11F492B; Tue, 14 Aug 2018 09:26:04 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1534235164; bh=1gT5xK7wOEcjMxmM/QaP8e66zXTI6hUnpJTyLMRuT/M=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=Nbk1nkNb7lXWaLIEqDmA0gqm4nQwT9sH4UZDy7fsjBTJot3HzSiKe+B2SHr2JZdZz DgnP4yd2bDUSGjI3wUsUyLe4MQXFyoui6CidrDR6Ltl4e+aSdpu0i/CO0jeofpl6tj wJmbJpBBPWi86fls4V8Kl3bEctRL4Kblq77xGDzk= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Tue, 14 Aug 2018 09:26:04 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Tue, 14 Aug 2018 09:26:03 +0100 From: Matt Churchyard To: "Rodney W. Grimes" CC: "freebsd-virtualization@freebsd.org" Subject: RE: Curent Centos 7 and bhyve Thread-Topic: Curent Centos 7 and bhyve Thread-Index: AQHUMQRwp/VmOgvRrEWW0WbxakcQ6aS5/FiAgACE6YCAAAzbAIAAANiAgAAG4oCAAAHfgIAAFGsAgAADS4CAANYhgIAAockAgAC5ugCAAHVP0IAAe9kAgAEaXQA= Date: Tue, 14 Aug 2018 08:26:02 +0000 Message-ID: References: <1b484d237c4a4f4dabcbd7b0eece7675@SERVER.ad.usd-group.com> <201808131627.w7DGRKKT037840@pdx.rh.CN85.dnsmgr.net> In-Reply-To: <201808131627.w7DGRKKT037840@pdx.rh.CN85.dnsmgr.net> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Aug 2018 08:26:13 -0000 > > Rodney W. Grimes wrote: >=20 > [dd] >=20 > > > > > >=20 > > > > > > Though it has a lot of features, it also has some short=20 > > > > > > comings, like you can not spec a vm to be wired in memory,=20 > > > > > > which IMHO is the only way to insure consistent VM performance. > > > > >=20 > > > > > Well, we have "bhyve_options" configuration option in the vm=20 > > > > > config, why not put "-S" there, is that what you mean by=20 > > > > > wiring the vm in memory? > > > >=20 > > > > I believe that fails as that only adds the -S to bhyve, and you=20 > > > > must specify it both on bhyveload and bhyve for it to work. > > >=20 > > > I think it is totally doable becase vm-bhyve is nothing but a suit=20 > > > of scripts. A PR with a feature request would be appropriate. > >=20 > > I made several attempts to contact the author at the email address=20 > > provided at the git hub while making other bhyve changes to try and=20 > > coordinate with him. I got no response after 3 attempts, > > so have stopped trying to contact them. (This was while I was > > adding the -c cpu topology modifications.) >=20 > > You can add yourself to > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230580 > > maybe something useful comes out of it. >=20 > I've already commented on the bug, although I'll reply here as well. > If "-S" is found in bhyve_options it does currently affect both commands.= I have decided that a specific wired_memory option is useful though and wi= ll add this to the next release. >=20 > The name limit has been increased to 32 since v1.2 >This is better, but still an artificial limit, the implementation of bhyve= allows this string to be any valid filename, the scripts should be designe= d to allow for that as the valid limit. As far as I'm aware bhyve is currently limited to 32 characters, although t= his obviously may change. We use the name in various situations (interface = descriptions, console names, etc) so it's beneficial to make sure it's not = excessively long, or contains characters that could cause problems when use= d in other places. if(name =3D=3D NULL || strlen(name) >=3D VM_MAX_NAMELEN) return (EINVAL); #define VM_MAX_NAMELEN 32 >I modified the script so that the vm name is the last column, and removed = the length check all togeather, this allows for the string to be what ever = length and not mess with column widths. >root@x230a:# vm list >DATASTORE LOADER CPU MEMORY VNC AUTOSTAR= T STATE NAME >default bhyveload 1 128M - No = Stopped fb-bld-10-amd64 >default bhyveload 4 2048M - No = Stopped fb-bld-11-amd64 >default bhyveload 4 1024M - No = Stopped fb-bld-11-i386 >default bhyveload 1 128M - No = Stopped fb-bld-11.0-p1-amd64 >default bhyveload 1 128M - No = Stopped fb-bld-11.0-p1-i386 >default bhyveload 4 512M - No = Stopped fb-bld-11.1-amd64 >default bhyveload 4 512M - No = Stopped fb-bld-11.1-i386 >> I didn't realize the changes for cpu topology had actually made it=20 >> into head, although I don't believe it's actually in a release yet? >Yes, they are in head, the MFC has been delayed for other reasons. >It is in the 12.0-ALPHA1 snapshot, and many before that. =20 >> I will plan to support configuration and display of these before 12=20 >> release. >Thanks. I think mostly just extract the NCPU's from the topology string. = The code actually works now, but due to fixed column width assumptions the= output looks bad. I've added some additional cpu_ settings to control the new cpu fields. "vm= list" just shows the original cpu setting so this remains unmodified. "vm = info" shows full settings. >=20 > Matt >=20 > > -- > > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > > AS43859 > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list=20 > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@fr= eebsd.org" >=20 >=20 --=20 Rod Grimes rgrimes@freebsd.= org _______________________________________________ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/m= ailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebs= d.org" From owner-freebsd-virtualization@freebsd.org Thu Aug 16 07:25:41 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C99C10803F1; Thu, 16 Aug 2018 07:25:41 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8C29572016; Thu, 16 Aug 2018 07:25:40 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from hugo10.ka.punkt.de (hugo10.ka.punkt.de [217.29.44.10]) by gate1.intern.punkt.de with ESMTP id w7G7Pc0R046541; Thu, 16 Aug 2018 09:25:38 +0200 (CEST) Received: from [217.29.44.49] ([217.29.44.49]) by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id w7G7PbWu069056; Thu, 16 Aug 2018 09:25:37 +0200 (CEST) (envelope-from hausen@punkt.de) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Why can't I dtrace processes running in a jail from the host? From: "Patrick M. Hausen" In-Reply-To: <20180810183419.GA52302@raichu> Date: Thu, 16 Aug 2018 09:25:37 +0200 Cc: David Powers via freebsd-virtualization , freebsd-dtrace@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <5BD4792C-D8AB-4598-BE7A-9D63A5757392@punkt.de> References: <20180809145258.GA68459@raichu> <8B1BDE9F-BDAD-4CEB-B7A2-8052497F50EA@punkt.de> <20180810183419.GA52302@raichu> To: Mark Johnston X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 07:25:41 -0000 Good morning, I did some further investigation and with help from Mark was finally able to get it working. Took quite some effort - documentation on the PHP side - NULL :-/ BTW: has anyone ever successfully subscribed to one of the PHP mailing lists? Where is the community of the people developing that stuff? OK, back to topic, in fact I had two issues, one FreeBSD, one PHP related. 1. DTrace'ing jailed userland probes requires /dev/dtrace/* to be visible inside the jail. Hence: [devfsrules_proserver=3D100] add include $devfsrules_jail add path dtrace/* unhide iocage set devfs_ruleset=3D100 vpro0069 Voila - dtrace on the host, watch userland probes in the jail. 2. PHP > 5.6 needs the environment variable USE_ZEND_DTRACE to be set to register it's probes. Turned out that it was not sufficient to *configure* that into the PHP FPM worker but you need to set (and export) the variable on the shell before you start the FPM master daemon. Then everything works as = expected. What I regularly do in such a case is sh -x /usr/local/etc/rc.d/php-fpm start to find out what command is actually executed in the end. Then call that = directly after setting the environment. Result: setenv USE_ZEND_DTRACE 1 limits -C daemon /usr/local/sbin/php-fpm Bingo! Surprisingly enough it is *not* necessary to configure clear_env =3D no in PHP FPM ... DTrace is active as soon as the master daemon sees that environment variable. Kind regards Patrick --=20 punkt.de GmbH Internet - Dienstleistungen - Beratung Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100 76133 Karlsruhe info@punkt.de http://punkt.de AG Mannheim 108285 Gf: Juergen Egeling From owner-freebsd-virtualization@freebsd.org Thu Aug 16 10:17:01 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B2CC108525A for ; Thu, 16 Aug 2018 10:17:01 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-a.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 94A587AAC0 for ; Thu, 16 Aug 2018 10:17:00 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-a.userve.net (Postfix) with ESMTPS id 137F985D for ; Thu, 16 Aug 2018 11:16:53 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1534414613; bh=ND2VL0a8jUvQst+Z9H5LL0mdlxpvBvihTxfeZXIGQBM=; h=From:To:Subject:Date; b=Pzn2wQWJTR78v5DtUka+c8VhEa0OouBPNjPccNT3XrUzXM4pvd4bdVVMHj2DB4hyb L6Y2VnErAcAL48QwB/eImtbnIN8BKoay4O+ZofHIh/0SpPEZHJxU1H/2wNPMEePdiC kgiNwL2ic/BPW48wzgEY3EOdhd3sTHO41vTIfk1w= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Thu, 16 Aug 2018 11:16:52 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Thu, 16 Aug 2018 11:16:52 +0100 From: Matt Churchyard To: "freebsd-virtualization@freebsd.org" Subject: Checking bhyve supported features (sysctls) Thread-Topic: Checking bhyve supported features (sysctls) Thread-Index: AdQ1Sj/JzUfq4S8/RYq4DRXDNWTckQ== Date: Thu, 16 Aug 2018 10:16:52 +0000 Message-ID: <3393f8f3d32a4f0890aab87185fbed01@SERVER.ad.usd-group.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 10:17:01 -0000 Hello, I'm looking for better ways to check for bhyve support / available features= without trying to scan through dmesg output. I notice that the following 2 sysctl's appear to be set to 1 as soon as the= vmm module is loaded hw.vmm.vmx.initialized: 1 hw.vmm.vmx.cap.unrestricted_guest: 1 Will these be available on both Intel & AMD processors as a way to determin= e if the module has loaded successfully and can run guests? I also see the below sysctl related to iommu. hw.vmm.iommu.initialized Again, will this be set to 1 as soon as the module is loaded if iommu is su= pported, or only when it is used? There also seems to be a vmm.amdvi.enable sysctl. Would both these need che= cking or is vmm.iommu enough to determine support on any processor. Matt From owner-freebsd-virtualization@freebsd.org Thu Aug 16 15:16:59 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4E1F1106A584 for ; Thu, 16 Aug 2018 15:16:59 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CD52E87041 for ; Thu, 16 Aug 2018 15:16:58 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w7GFGt9p054011; Thu, 16 Aug 2018 08:16:55 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w7GFGtN1054010; Thu, 16 Aug 2018 08:16:55 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201808161516.w7GFGtN1054010@pdx.rh.CN85.dnsmgr.net> Subject: Re: Checking bhyve supported features (sysctls) In-Reply-To: <3393f8f3d32a4f0890aab87185fbed01@SERVER.ad.usd-group.com> To: Matt Churchyard Date: Thu, 16 Aug 2018 08:16:55 -0700 (PDT) CC: "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 15:16:59 -0000 > Hello, > > I'm looking for better ways to check for bhyve support / available features without trying to scan through dmesg output. Yes, it would be very good to remove that, as it usually tries to grep a non-existent file /var/run/dmesg.boot that is not created until after vm_bhyve has been called from /usr/local/etc/rc.d when you have things set to autostartup in /etc/rc.conf > > I notice that the following 2 sysctl's appear to be set to 1 as soon as the vmm module is loaded > > hw.vmm.vmx.initialized: 1 > hw.vmm.vmx.cap.unrestricted_guest: 1 > > Will these be available on both Intel & AMD processors as a way to determine if the module has loaded successfully and can run guests? > > I also see the below sysctl related to iommu. > > hw.vmm.iommu.initialized > > Again, will this be set to 1 as soon as the module is loaded if iommu is supported, or only when it is used? > There also seems to be a vmm.amdvi.enable sysctl. Would both these need checking or is vmm.iommu enough to determine support on any processor. Probalby the safest way for a shell script to decide if bhyve is up and running is to stat /dev/vmm, if that exists then the modules have loaded and initialized and bhyve should be ready to process guests. sysctl's mentiond above would be a poor way to make this determination. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Thu Aug 16 16:01:28 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 87E69106BF61 for ; Thu, 16 Aug 2018 16:01:28 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-a.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 291B78A0B5 for ; Thu, 16 Aug 2018 16:01:28 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-a.userve.net (Postfix) with ESMTPS id B927AA8; Thu, 16 Aug 2018 17:01:26 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1534435286; bh=CfdaF1e1ERpds6/QEDhPq+giq2RYP7T8lyrLeLT9eDk=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=HXu5153AVe59TFPUa6/isQva1jOMOwDLkydSbq+KNNPmpsSoe7jRr0ZD/zaim8p42 AhfDqEh2C5P2pL0J8K1czHpkM6EGWC3IdqR4R7zuLY0EDHOp4cBd2hW06Lxw0gQxH3 AQcHWOwtEhiptIRYtKWV0gy3hO4Ib1KZ8LeHXwOk= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Thu, 16 Aug 2018 17:01:26 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Thu, 16 Aug 2018 17:01:26 +0100 From: Matt Churchyard To: "Rodney W. Grimes" CC: "freebsd-virtualization@freebsd.org" Subject: RE: Checking bhyve supported features (sysctls) Thread-Topic: Checking bhyve supported features (sysctls) Thread-Index: AdQ1Sj/JzUfq4S8/RYq4DRXDNWTckQAIYkuAAANnXRA= Date: Thu, 16 Aug 2018 16:01:25 +0000 Message-ID: <60d0e18d1ba34632aa6045c85c023bbd@SERVER.ad.usd-group.com> References: <3393f8f3d32a4f0890aab87185fbed01@SERVER.ad.usd-group.com> <201808161516.w7GFGtN1054010@pdx.rh.CN85.dnsmgr.net> In-Reply-To: <201808161516.w7GFGtN1054010@pdx.rh.CN85.dnsmgr.net> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 16:01:28 -0000 > Hello, >=20 > I'm looking for better ways to check for bhyve support / available featur= es without trying to scan through dmesg output. >Yes, it would be very good to remove that, as it usually tries to grep a n= on-existent file /var/run/dmesg.boot that is not created until after vm_bhy= ve has been called from /usr/local/etc/rc.d when you have things set to aut= ostartup >in /etc/rc.conf >=20 > I notice that the following 2 sysctl's appear to be set to 1 as soon=20 > as the vmm module is loaded >=20 > hw.vmm.vmx.initialized: 1 > hw.vmm.vmx.cap.unrestricted_guest: 1 >=20 > Will these be available on both Intel & AMD processors as a way to determ= ine if the module has loaded successfully and can run guests? >=20 > I also see the below sysctl related to iommu. >=20 > hw.vmm.iommu.initialized >=20 > Again, will this be set to 1 as soon as the module is loaded if iommu is = supported, or only when it is used? > There also seems to be a vmm.amdvi.enable sysctl. Would both these need c= hecking or is vmm.iommu enough to determine support on any processor. >Probalby the safest way for a shell script to decide if bhyve is up and ru= nning is to stat /dev/vmm, if that exists then the modules have loaded and = initialized and bhyve should be ready to process guests. Hmm, I don't get /dev/vmm unless I actually have running guests. >sysctl's mentiond above would be a poor way to make this determination. It would be nice if sysctls were better documented. If vmx.initialized is s= et once vmm has successfully loaded, I can't see a better way of checking f= or bhyve support (assuming it's not Intel specific). This entry definitely = exists and is set to 0 if you load the module on a non-supported system, an= d set to 1 as soon as vmm loads on my Intel test system. Matt --=20 Rod Grimes rgrimes@freebsd.= org From owner-freebsd-virtualization@freebsd.org Thu Aug 16 16:07:59 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D04ED106C398 for ; Thu, 16 Aug 2018 16:07:59 +0000 (UTC) (envelope-from fabian.freyer@physik.tu-berlin.de) Received: from mail.physik.tu-berlin.de (mail.physik-pool.tu-berlin.de [130.149.50.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 654148A6C0; Thu, 16 Aug 2018 16:07:58 +0000 (UTC) (envelope-from fabian.freyer@physik.tu-berlin.de) Received: from [192.168.43.253] (x2f7f27d.dyn.telefonica.de [2.247.242.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.physik.tu-berlin.de (Postfix) with ESMTPSA id 6AF7161FDD; Thu, 16 Aug 2018 16:07:50 +0000 (UTC) From: "Fabian Freyer" To: "Matt Churchyard" Cc: freebsd-virtualization@freebsd.org, novel@FreeBSD.org Subject: Re: Checking bhyve supported features (sysctls) Date: Thu, 16 Aug 2018 18:07:38 +0200 X-Mailer: MailMate (1.11.3r5509) Message-ID: <77D2D821-9132-4AD1-95C1-D2B825E91A6A@physik.tu-berlin.de> In-Reply-To: <3393f8f3d32a4f0890aab87185fbed01@SERVER.ad.usd-group.com> References: <3393f8f3d32a4f0890aab87185fbed01@SERVER.ad.usd-group.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 16:08:00 -0000 On 16 Aug 2018, at 12:16, Matt Churchyard wrote: > I'm looking for better ways to check for bhyve support / available = > features without trying to scan through dmesg output. There=E2=80=99s a few patches floating around for checking features [1,2]= - = however none of these are yet committed. For [2], there=E2=80=99s also an RFC mailing list thread [3]. CC=E2=80=99ing novel@, as he wrote those patches. The hackish way libvirt checks for features right now is to parse error = messages from invalid invocations of bhyve(8). [1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210111 [2] https://reviews.freebsd.org/D15992 [3] = https://lists.freebsd.org/pipermail/freebsd-virtualization/2018-June/0065= 36.html From owner-freebsd-virtualization@freebsd.org Thu Aug 16 16:13:39 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3524B106C70C for ; Thu, 16 Aug 2018 16:13:39 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9F6DC8AC9A for ; Thu, 16 Aug 2018 16:13:38 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w7GGDaSw054439; Thu, 16 Aug 2018 09:13:36 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w7GGDaNB054438; Thu, 16 Aug 2018 09:13:36 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201808161613.w7GGDaNB054438@pdx.rh.CN85.dnsmgr.net> Subject: Re: Checking bhyve supported features (sysctls) In-Reply-To: <60d0e18d1ba34632aa6045c85c023bbd@SERVER.ad.usd-group.com> To: Matt Churchyard Date: Thu, 16 Aug 2018 09:13:36 -0700 (PDT) CC: "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 16:13:39 -0000 Text manually wrapped to 80, any broken quoting is my fault - rwg > > Hello, > > > > I'm looking for better ways to check for bhyve support / available > > features without trying to scan through dmesg output. > > >Yes, it would be very good to remove that, as it usually tries > >to grep a non-existent file /var/run/dmesg.boot that is not > >created until after vm_bhyve has been called from /usr/local/etc/rc.d > >when you have things set to autostartup >in /etc/rc.conf > > > > > > I notice that the following 2 sysctl's appear to be set to 1 as soon > > as the vmm module is loaded > > > > hw.vmm.vmx.initialized: 1 > > hw.vmm.vmx.cap.unrestricted_guest: 1 > > > > Will these be available on both Intel & AMD processors as a way > > to determine if the module has loaded successfully and can run guests? > > > > I also see the below sysctl related to iommu. > > > > hw.vmm.iommu.initialized > > > > Again, will this be set to 1 as soon as the module is loaded if > > iommu is supported, or only when it is used? > > There also seems to be a vmm.amdvi.enable sysctl. > > Would both these need checking or is vmm.iommu enough to > > determine support on any processor. > > >Probalby the safest way for a shell script to decide if bhyve is > >up and running is to stat /dev/vmm, if that exists then the modules > >have loaded and initialized and bhyve should be ready to process guests. > > Hmm, I don't get /dev/vmm unless I actually have running guests. I'll investigate that, I was pretty sure that you should get this as soon as the vmm.ko module is finished initialzing, but you might be right in that it takes a first vm to cause its creation. Confirmed, /dev/vmm does not exist until the first vm is created. > > >sysctl's mentiond above would be a poor way to make this determination. > > It would be nice if sysctls were better documented. Agreed. > If vmx.initialized is set once vmm has successfully loaded, I can't see a better way of checking for bhyve support (assuming it's not Intel specific). This entry definitely exists and is set to 0 if you load the module on a non-supported system, and set to 1 as soon as vmm loads on my Intel test system. Given its undocumented status you would be relying on an undocumented feature that could change in either name or behavior, and that is not desirable. Let me see if I can come up with something else. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Thu Aug 16 16:28:09 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 88DDC106CECE for ; Thu, 16 Aug 2018 16:28:09 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 03E868B774 for ; Thu, 16 Aug 2018 16:28:08 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w7GGS6xi054506; Thu, 16 Aug 2018 09:28:06 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w7GGS52P054505; Thu, 16 Aug 2018 09:28:05 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201808161628.w7GGS52P054505@pdx.rh.CN85.dnsmgr.net> Subject: Re: Checking bhyve supported features (sysctls) In-Reply-To: <201808161613.w7GGDaNB054438@pdx.rh.CN85.dnsmgr.net> To: "Rodney W. Grimes" Date: Thu, 16 Aug 2018 09:28:05 -0700 (PDT) CC: Matt Churchyard , "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 16:28:09 -0000 > > Text manually wrapped to 80, any broken quoting is my fault - rwg > > > > Hello, > > > > > > I'm looking for better ways to check for bhyve support / available > > > features without trying to scan through dmesg output. > > > > >Yes, it would be very good to remove that, as it usually tries > > >to grep a non-existent file /var/run/dmesg.boot that is not > > >created until after vm_bhyve has been called from /usr/local/etc/rc.d > > >when you have things set to autostartup >in /etc/rc.conf > > > > > > > > > > I notice that the following 2 sysctl's appear to be set to 1 as soon > > > as the vmm module is loaded > > > > > > hw.vmm.vmx.initialized: 1 > > > hw.vmm.vmx.cap.unrestricted_guest: 1 > > > > > > Will these be available on both Intel & AMD processors as a way > > > to determine if the module has loaded successfully and can run guests? > > > > > > I also see the below sysctl related to iommu. > > > > > > hw.vmm.iommu.initialized > > > > > > Again, will this be set to 1 as soon as the module is loaded if > > > iommu is supported, or only when it is used? > > > There also seems to be a vmm.amdvi.enable sysctl. > > > Would both these need checking or is vmm.iommu enough to > > > determine support on any processor. > > > > >Probalby the safest way for a shell script to decide if bhyve is > > >up and running is to stat /dev/vmm, if that exists then the modules > > >have loaded and initialized and bhyve should be ready to process guests. > > > > Hmm, I don't get /dev/vmm unless I actually have running guests. > > I'll investigate that, I was pretty sure that you should get this > as soon as the vmm.ko module is finished initialzing, but you might > be right in that it takes a first vm to cause its creation. > Confirmed, /dev/vmm does not exist until the first vm > is created. > > > > > >sysctl's mentiond above would be a poor way to make this determination. > > > > It would be nice if sysctls were better documented. > > Agreed. > > > If vmx.initialized is set once vmm has successfully loaded, I can't see a better way of checking for bhyve support (assuming it's not Intel specific). This entry definitely exists and is set to 0 if you load the module on a non-supported system, and set to 1 as soon as vmm loads on my Intel test system. > > Given its undocumented status you would be relying on an > undocumented feature that could change in either name or > behavior, and that is not desirable. > > Let me see if I can come up with something else. I looked at the code for bhyvectl, bhyveload and byhve. They do not actually try to decide if vmm is supported or not, they simply process the error from a vm_create() or vm_open() call and exit with an error code if they can not handle it (some of the code can handle a vm_create failure if infact we are trying to create a vm that already exists). If you want to maintain full compatibility a similiar stratergy may be in order. Why is it that vm-bhyve specifically needs to know if the kernel has vmm support or not? Cant it just be written to handle the errors returned if the supported functions do not exist? -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Thu Aug 16 16:53:47 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 27C4B106E453 for ; Thu, 16 Aug 2018 16:53:47 +0000 (UTC) (envelope-from allanjude@FreeBSD.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C02808D52B for ; Thu, 16 Aug 2018 16:53:46 +0000 (UTC) (envelope-from allanjude@FreeBSD.org) Received: from dhcp-10-241-160-237.cp.wireless.private.cam.ac.uk (global-5-53.nat-1.net.cam.ac.uk [131.111.5.53]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id C07AA18D47; Thu, 16 Aug 2018 16:53:38 +0000 (UTC) Date: Thu, 16 Aug 2018 17:53:36 +0100 User-Agent: K-9 Mail for Android In-Reply-To: <201808161628.w7GGS52P054505@pdx.rh.CN85.dnsmgr.net> References: <201808161628.w7GGS52P054505@pdx.rh.CN85.dnsmgr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: Checking bhyve supported features (sysctls) To: freebsd-virtualization@freebsd.org, "Rodney W. Grimes" CC: Matt Churchyard , "freebsd-virtualization@freebsd.org" From: Allan Jude Message-ID: X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 16:53:47 -0000 On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W=2E Grimes" wrote: >>=20 >> Text manually wrapped to 80, any broken quoting is my fault - rwg >>=20 >> > > Hello, >> > >=20 >> > > I'm looking for better ways to check for bhyve support / >available >> > > features without trying to scan through dmesg output=2E >> >=20 >> > >Yes, it would be very good to remove that, as it usually tries >> > >to grep a non-existent file /var/run/dmesg=2Eboot that is not >> > >created until after vm_bhyve has been called from >/usr/local/etc/rc=2Ed >> > >when you have things set to autostartup >in /etc/rc=2Econf >> >=20 >> >=20 >> > >=20 >> > > I notice that the following 2 sysctl's appear to be set to 1 as >soon=20 >> > > as the vmm module is loaded >> > >=20 >> > > hw=2Evmm=2Evmx=2Einitialized: 1 >> > > hw=2Evmm=2Evmx=2Ecap=2Eunrestricted_guest: 1 >> > >=20 >> > > Will these be available on both Intel & AMD processors as a way >> > > to determine if the module has loaded successfully and can run >guests? >> > >=20 >> > > I also see the below sysctl related to iommu=2E >> > >=20 >> > > hw=2Evmm=2Eiommu=2Einitialized >> > >=20 >> > > Again, will this be set to 1 as soon as the module is loaded if >> > > iommu is supported, or only when it is used? >> > > There also seems to be a vmm=2Eamdvi=2Eenable sysctl=2E >> > > Would both these need checking or is vmm=2Eiommu enough to >> > > determine support on any processor=2E >> >=20 >> > >Probalby the safest way for a shell script to decide if bhyve is >> > >up and running is to stat /dev/vmm, if that exists then the >modules >> > >have loaded and initialized and bhyve should be ready to process >guests=2E >> >=20 >> > Hmm, I don't get /dev/vmm unless I actually have running guests=2E >>=20 >> I'll investigate that, I was pretty sure that you should get this >> as soon as the vmm=2Eko module is finished initialzing, but you might >> be right in that it takes a first vm to cause its creation=2E >> Confirmed, /dev/vmm does not exist until the first vm >> is created=2E >>=20 >> >=20 >> > >sysctl's mentiond above would be a poor way to make this >determination=2E >> >=20 >> > It would be nice if sysctls were better documented=2E >>=20 >> Agreed=2E >>=20 >> > If vmx=2Einitialized is set once vmm has successfully loaded, I can't >see a better way of checking for bhyve support (assuming it's not Intel >specific)=2E This entry definitely exists and is set to 0 if you load the >module on a non-supported system, and set to 1 as soon as vmm loads on >my Intel test system=2E >>=20 >> Given its undocumented status you would be relying on an >> undocumented feature that could change in either name or >> behavior, and that is not desirable=2E >>=20 >> Let me see if I can come up with something else=2E > >I looked at the code for bhyvectl, bhyveload and >byhve=2E They do not actually try to decide if vmm >is supported or not, they simply process the error >from a vm_create() or vm_open() call and exit >with an error code if they can not handle it >(some of the code can handle a vm_create failure >if infact we are trying to create a vm that >already exists)=2E > >If you want to maintain full compatibility a similiar >stratergy may be in order=2E > >Why is it that vm-bhyve specifically needs to know >if the kernel has vmm support or not? >Cant it just be written to handle the errors returned >if the supported functions do not exist? I think the question vm-bhyve wants to answer is: does the CPU have the re= quired features to run a multicore VM=2E These or similar sysctls do seem to be the correct way to communicate that= support=2E --=20 Allan Jude From owner-freebsd-virtualization@freebsd.org Thu Aug 16 17:30:43 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A92A106FBF2 for ; Thu, 16 Aug 2018 17:30:43 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 024198EE55; Thu, 16 Aug 2018 17:30:39 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w7GHUawf054789; Thu, 16 Aug 2018 10:30:36 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w7GHUaWv054788; Thu, 16 Aug 2018 10:30:36 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201808161730.w7GHUaWv054788@pdx.rh.CN85.dnsmgr.net> Subject: Re: Checking bhyve supported features (sysctls) In-Reply-To: To: Allan Jude Date: Thu, 16 Aug 2018 10:30:36 -0700 (PDT) CC: "freebsd-virtualization@freebsd.org" , Matt Churchyard X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 17:30:43 -0000 > On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" wrote: > >> > >> Text manually wrapped to 80, any broken quoting is my fault - rwg > >> > >> > > Hello, > >> > > > >> > > I'm looking for better ways to check for bhyve support / > >available > >> > > features without trying to scan through dmesg output. > >> > > >> > >Yes, it would be very good to remove that, as it usually tries > >> > >to grep a non-existent file /var/run/dmesg.boot that is not > >> > >created until after vm_bhyve has been called from > >/usr/local/etc/rc.d > >> > >when you have things set to autostartup >in /etc/rc.conf > >> > > >> > > >> > > > >> > > I notice that the following 2 sysctl's appear to be set to 1 as > >soon > >> > > as the vmm module is loaded > >> > > > >> > > hw.vmm.vmx.initialized: 1 > >> > > hw.vmm.vmx.cap.unrestricted_guest: 1 > >> > > > >> > > Will these be available on both Intel & AMD processors as a way > >> > > to determine if the module has loaded successfully and can run > >guests? > >> > > > >> > > I also see the below sysctl related to iommu. > >> > > > >> > > hw.vmm.iommu.initialized > >> > > > >> > > Again, will this be set to 1 as soon as the module is loaded if > >> > > iommu is supported, or only when it is used? > >> > > There also seems to be a vmm.amdvi.enable sysctl. > >> > > Would both these need checking or is vmm.iommu enough to > >> > > determine support on any processor. > >> > > >> > >Probalby the safest way for a shell script to decide if bhyve is > >> > >up and running is to stat /dev/vmm, if that exists then the > >modules > >> > >have loaded and initialized and bhyve should be ready to process > >guests. > >> > > >> > Hmm, I don't get /dev/vmm unless I actually have running guests. > >> > >> I'll investigate that, I was pretty sure that you should get this > >> as soon as the vmm.ko module is finished initialzing, but you might > >> be right in that it takes a first vm to cause its creation. > >> Confirmed, /dev/vmm does not exist until the first vm > >> is created. > >> > >> > > >> > >sysctl's mentiond above would be a poor way to make this > >determination. > >> > > >> > It would be nice if sysctls were better documented. > >> > >> Agreed. > >> > >> > If vmx.initialized is set once vmm has successfully loaded, I can't > >see a better way of checking for bhyve support (assuming it's not Intel > >specific). This entry definitely exists and is set to 0 if you load the > >module on a non-supported system, and set to 1 as soon as vmm loads on > >my Intel test system. > >> > >> Given its undocumented status you would be relying on an > >> undocumented feature that could change in either name or > >> behavior, and that is not desirable. > >> > >> Let me see if I can come up with something else. > > > >I looked at the code for bhyvectl, bhyveload and > >byhve. They do not actually try to decide if vmm > >is supported or not, they simply process the error > >from a vm_create() or vm_open() call and exit > >with an error code if they can not handle it > >(some of the code can handle a vm_create failure > >if infact we are trying to create a vm that > >already exists). > > > >If you want to maintain full compatibility a similiar > >stratergy may be in order. > > > >Why is it that vm-bhyve specifically needs to know > >if the kernel has vmm support or not? > >Cant it just be written to handle the errors returned > >if the supported functions do not exist? > > I think the question vm-bhyve wants to answer is: does > the CPU have the required features to run a multicore VM. Why does it need to know that? If it tries to start a multicore/thread VM and the system can not support it an error is returned and the bhyve command fails. Actually determining that specific issue is non-trivial iirc as some vmm supported CPU's can only run a single VM with a single thread in that VM (early core cpu's). > > These or similar sysctls do seem to be the correct way > to communicate that support. I do not believe any of those sysctl's communicate that your on a broken cpu or to what extent you can run vm's with multiple threads. I went and looked at why vm-bhyve is groveling around in /var/run/dmesg.boot and found that it is simply trying to determine if the host CPU is vmm capable, specifically: util::check_bhyve_support(){ ... # get features2 line which should include popcnt _mesg=$(grep -E '^[ ]+Features2' /var/run/dmesg.boot | tail -n 1) # only check if we found it if [ -n "${_mesg}" ]; then # look for pop cnt _result=$(echo "${_mesg}" |grep "POPCNT") [ -z "${_result}" ] && util::err "it doesn't look like your cpu supports bhyve (missing POPCNT)" fi # check ept for intel _mesg=$(grep -E '^[ ]+VT-x' /var/run/dmesg.boot | tail -n 1) if [ -n "${_mesg}" ]; then # look for ept _result=$(echo "${_mesg}" |grep "EPT") [ -z "${_result}" ] && util::err "it doesn't look like your cpu supports bhyve (missing EPT)" # look for unrestricted guest _result=$(echo "${_mesg}" |grep ",UG") [ -z "${_result}" ] && VM_NO_UG="1" fi These checks are already built into the kernel. This can all go in the bit bucket, if you try to start a VM on an unsupported system an error is returned, recoding this in shell is just setting yourself up for "future" bugs. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Thu Aug 16 18:55:07 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8C331072E5E for ; Thu, 16 Aug 2018 18:55:06 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com [IPv6:2a00:1450:4864:20::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3947E73541; Thu, 16 Aug 2018 18:55:06 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by mail-lj1-x22a.google.com with SMTP id 203-v6so4457856ljj.13; Thu, 16 Aug 2018 11:55:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:reply-to:in-reply-to:references:from:date:message-id :subject:to:cc; bh=/RvhSJq3lADw1B1HkN3axcYAu0VLrBLeSJa4DjdeRXM=; b=bKgTKqRgKQro9y9xShBbdPxze0qAnH1aiY+XTvL6pMzFU0a47y4YsZWCBd5He+qOH2 8+I3K9fCxeU/op8EuTE1hCPNAlf+MzBfx+DGranU8BC33m6mKdDNdNTnx1rADUPO9dRt P3GEpgh20ZWsB+nmJ3aOpWxpbMAbxcKiu8JSpHl4d4EjV3Y3Z55n/qjr/d5foiaRzG7s rXGHyoYqxsIELcX7P5n26kFAlejE+ChNITCAp7ustOuRmfBugRrws6+e7p/8P8+zOl7I hCHdWC4sMe5YqgswIpvrams7XbqD3RhwoM50ggK3VJ71DHF+FJLDPjd0fU0RyxK+DFUH 5D6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=/RvhSJq3lADw1B1HkN3axcYAu0VLrBLeSJa4DjdeRXM=; b=f7kl/puE4pDZtv1NdeRqyueQYesp6QXjgE44tvQV/eOJnF0GmlEmyRLeheJHBSQ/Ln s6wp8dyrkvvMJMgDVTZ2Hr/yZVdkCeBeQ9dKiDX821vM0QZ2hIX3kcQwt/crIfQT/WQ4 lwDyq/sWQ3fgEIzNf6Oc2dg4RsnsSMtPUjhG1w/JHkTXkxYRKNYJlLxT5TrMfNbkszxr n68NyFDKJDCCQuFGnN8rn6eWSqhPz2Zsl4U/QkRwgApiMmcuRTGzLt3ZJuvCzURMe0zH yvTEFaKkw5nJSitsb6GLhr+GRe4VymNuLCjq+j8IORssyuHHs1SmXJmVZA+H7h07vo2k S5Mg== X-Gm-Message-State: AOUpUlFsHikyt1SAB4PKLPCnTPfnEMwI2Xjoh9x9VQQrJ958HdFUMSS+ RU5VDAJrkAMwGkARiZQFYiN+uz6gLS96wrf31WbWtOQd X-Google-Smtp-Source: AA+uWPw/poCk6S+6FXwt3YStXwTFKEeSCm9+vQ+RsfcvoMhOsJ7Ce2HW/2gZ5t3PIX8bPDUObbp2TpbR1/Q17Fbw0pE= X-Received: by 2002:a2e:88d0:: with SMTP id a16-v6mr12097128ljk.63.1534445703665; Thu, 16 Aug 2018 11:55:03 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a19:1f4c:0:0:0:0:0 with HTTP; Thu, 16 Aug 2018 11:55:02 -0700 (PDT) Reply-To: araujo@freebsd.org In-Reply-To: References: <201808161628.w7GGS52P054505@pdx.rh.CN85.dnsmgr.net> From: Marcelo Araujo Date: Fri, 17 Aug 2018 02:55:02 +0800 Message-ID: Subject: Re: Checking bhyve supported features (sysctls) To: Allan Jude Cc: freebsd-virtualization@freebsd.org, "Rodney W. Grimes" , Matt Churchyard Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 18:55:07 -0000 2018-08-17 0:53 GMT+08:00 Allan Jude : > On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" < > freebsd-rwg@pdx.rh.CN85.dnsmgr.net> wrote: > >> > >> Text manually wrapped to 80, any broken quoting is my fault - rwg > >> > >> > > Hello, > >> > > > >> > > I'm looking for better ways to check for bhyve support / > >available > >> > > features without trying to scan through dmesg output. > >> > > >> > >Yes, it would be very good to remove that, as it usually tries > >> > >to grep a non-existent file /var/run/dmesg.boot that is not > >> > >created until after vm_bhyve has been called from > >/usr/local/etc/rc.d > >> > >when you have things set to autostartup >in /etc/rc.conf > >> > > >> > > >> > > > >> > > I notice that the following 2 sysctl's appear to be set to 1 as > >soon > >> > > as the vmm module is loaded > >> > > > >> > > hw.vmm.vmx.initialized: 1 > >> > > hw.vmm.vmx.cap.unrestricted_guest: 1 > >> > > > >> > > Will these be available on both Intel & AMD processors as a way > >> > > to determine if the module has loaded successfully and can run > >guests? > >> > > > >> > > I also see the below sysctl related to iommu. > >> > > > >> > > hw.vmm.iommu.initialized > >> > > > >> > > Again, will this be set to 1 as soon as the module is loaded if > >> > > iommu is supported, or only when it is used? > >> > > There also seems to be a vmm.amdvi.enable sysctl. > >> > > Would both these need checking or is vmm.iommu enough to > >> > > determine support on any processor. > >> > > >> > >Probalby the safest way for a shell script to decide if bhyve is > >> > >up and running is to stat /dev/vmm, if that exists then the > >modules > >> > >have loaded and initialized and bhyve should be ready to process > >guests. > >> > > >> > Hmm, I don't get /dev/vmm unless I actually have running guests. > >> > >> I'll investigate that, I was pretty sure that you should get this > >> as soon as the vmm.ko module is finished initialzing, but you might > >> be right in that it takes a first vm to cause its creation. > >> Confirmed, /dev/vmm does not exist until the first vm > >> is created. > >> > >> > > >> > >sysctl's mentiond above would be a poor way to make this > >determination. > >> > > >> > It would be nice if sysctls were better documented. > >> > >> Agreed. > >> > >> > If vmx.initialized is set once vmm has successfully loaded, I can't > >see a better way of checking for bhyve support (assuming it's not Intel > >specific). This entry definitely exists and is set to 0 if you load the > >module on a non-supported system, and set to 1 as soon as vmm loads on > >my Intel test system. > >> > >> Given its undocumented status you would be relying on an > >> undocumented feature that could change in either name or > >> behavior, and that is not desirable. > >> > >> Let me see if I can come up with something else. > > > >I looked at the code for bhyvectl, bhyveload and > >byhve. They do not actually try to decide if vmm > >is supported or not, they simply process the error > >from a vm_create() or vm_open() call and exit > >with an error code if they can not handle it > >(some of the code can handle a vm_create failure > >if infact we are trying to create a vm that > >already exists). > > > >If you want to maintain full compatibility a similiar > >stratergy may be in order. > > > >Why is it that vm-bhyve specifically needs to know > >if the kernel has vmm support or not? > >Cant it just be written to handle the errors returned > >if the supported functions do not exist? > > I think the question vm-bhyve wants to answer is: does the CPU have the > required features to run a multicore VM. > > These or similar sysctls do seem to be the correct way to communicate that > support. > You are correct! The question in case as I understood was about CPU feature supported, actually vmm(8) knows all this information! Some examples such like CPU with VMX unrestricted mode support (UG) that is necessary for guest VMs running with multiple vCPU or like VT-d necessary for PCI device passthrough. I have a patch that exposes a sysctl saying what bhyve(8) is capable to run, however it needs to be polished a bit more to be more informative. I think for third part software like vm-bhyve these information are crucial as these software can get advantage of these information prior to run a certain set that will end up in a fail because of a partial CPU support. Best, > -- > Allan Jude > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization- > unsubscribe@freebsd.org" > -- -- Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_) From owner-freebsd-virtualization@freebsd.org Thu Aug 16 20:43:56 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 72FE01076611 for ; Thu, 16 Aug 2018 20:43:56 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-b.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D81E278ED4; Thu, 16 Aug 2018 20:43:55 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-b.userve.net (Postfix) with ESMTPS id 928281E7; Thu, 16 Aug 2018 21:43:54 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1534452234; bh=jJiRLLAEn87nToRDzh8nFZSzd6xPWYcOrJ32RXquS4Q=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=qfEvLnmZX3S6rip6zaW66ag8HvS16+KIEAOqd2s1kUH/aG3DpqSmIHhLzUJj5iaEJ E6H4jo4VEJz5iYRPxU4PNxaBzyB03JtU4pFI/TplzO3MGXlhCJYXaDJOtahW4cdTsa pJyhxzQYqD69nikeOlMHkqbfTgw1Jcujb+Z2nNs8= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Thu, 16 Aug 2018 21:43:53 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Thu, 16 Aug 2018 21:43:53 +0100 From: Matt Churchyard To: "araujo@freebsd.org" CC: Allan Jude , "freebsd-virtualization@freebsd.org" , "Rodney W. Grimes" Subject: Re: Checking bhyve supported features (sysctls) Thread-Topic: Checking bhyve supported features (sysctls) Thread-Index: AdQ1Sj/JzUfq4S8/RYq4DRXDNWTckQAIYkuAAANnXRD///SbAIAABAyAgAAHIQCAACHuAIAALy0w Date: Thu, 16 Aug 2018 20:43:53 +0000 Message-ID: <12D64664-2135-4D60-B534-5DACCB839A08@userve.net> References: <201808161628.w7GGS52P054505@pdx.rh.CN85.dnsmgr.net> , In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 20:43:56 -0000 On 16 Aug 2018, at 19:55, Marcelo Araujo > wrote: 2018-08-17 0:53 GMT+08:00 Allan Jude >: On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" > wrote: >> >> Text manually wrapped to 80, any broken quoting is my fault - rwg >> >> > > Hello, >> > > >> > > I'm looking for better ways to check for bhyve support / >available >> > > features without trying to scan through dmesg output. >> > >> > >Yes, it would be very good to remove that, as it usually tries >> > >to grep a non-existent file /var/run/dmesg.boot that is not >> > >created until after vm_bhyve has been called from >/usr/local/etc/rc.d >> > >when you have things set to autostartup >in /etc/rc.conf >> > >> > >> > > >> > > I notice that the following 2 sysctl's appear to be set to 1 as >soon >> > > as the vmm module is loaded >> > > >> > > hw.vmm.vmx.initialized: 1 >> > > hw.vmm.vmx.cap.unrestricted_guest: 1 >> > > >> > > Will these be available on both Intel & AMD processors as a way >> > > to determine if the module has loaded successfully and can run >guests? >> > > >> > > I also see the below sysctl related to iommu. >> > > >> > > hw.vmm.iommu.initialized >> > > >> > > Again, will this be set to 1 as soon as the module is loaded if >> > > iommu is supported, or only when it is used? >> > > There also seems to be a vmm.amdvi.enable sysctl. >> > > Would both these need checking or is vmm.iommu enough to >> > > determine support on any processor. >> > >> > >Probalby the safest way for a shell script to decide if bhyve is >> > >up and running is to stat /dev/vmm, if that exists then the >modules >> > >have loaded and initialized and bhyve should be ready to process >guests. >> > >> > Hmm, I don't get /dev/vmm unless I actually have running guests. >> >> I'll investigate that, I was pretty sure that you should get this >> as soon as the vmm.ko module is finished initialzing, but you might >> be right in that it takes a first vm to cause its creation. >> Confirmed, /dev/vmm does not exist until the first vm >> is created. >> >> > >> > >sysctl's mentiond above would be a poor way to make this >determination. >> > >> > It would be nice if sysctls were better documented. >> >> Agreed. >> >> > If vmx.initialized is set once vmm has successfully loaded, I can't >see a better way of checking for bhyve support (assuming it's not Intel >specific). This entry definitely exists and is set to 0 if you load the >module on a non-supported system, and set to 1 as soon as vmm loads on >my Intel test system. >> >> Given its undocumented status you would be relying on an >> undocumented feature that could change in either name or >> behavior, and that is not desirable. >> >> Let me see if I can come up with something else. > >I looked at the code for bhyvectl, bhyveload and >byhve. They do not actually try to decide if vmm >is supported or not, they simply process the error >from a vm_create() or vm_open() call and exit >with an error code if they can not handle it >(some of the code can handle a vm_create failure >if infact we are trying to create a vm that >already exists). > >If you want to maintain full compatibility a similiar >stratergy may be in order. > >Why is it that vm-bhyve specifically needs to know >if the kernel has vmm support or not? >Cant it just be written to handle the errors returned >if the supported functions do not exist? I think the question vm-bhyve wants to answer is: does the CPU have the req= uired features to run a multicore VM. These or similar sysctls do seem to be the correct way to communicate that = support. You are correct! The question in case as I understood was about CPU feature supported, actua= lly vmm(8) knows all this information! Some examples such like CPU with VMX= unrestricted mode support (UG) that is necessary for guest VMs running wit= h multiple vCPU or like VT-d necessary for PCI device passthrough. I have a patch that exposes a sysctl saying what bhyve(8) is capable to run= , however it needs to be polished a bit more to be more informative. I think for third part software like vm-bhyve these information are crucial= as these software can get advantage of these information prior to run a ce= rtain set that will end up in a fail because of a partial CPU support. Best, As mentioned in my first email, it does seem like some of these exist alrea= dy in the way of vmm.vmx.cap.* sysctls. We could look at bhyve output and try to process that, but that seems more = messy if there are sysctls that expose support, especially as the vmm module does seem to = know what features the cpu/hardware supports. vm-bhyve has already forked into the ba= ckground by the time bhyve runs so can't easily provide feedback to the caller other= than through the log file. We do also try and take action in some cases, such as reducing cpu count to= 1 if UG support isn't found, rather than just having bhyve fail. (Of course you could argue= we should just exit with an error and let the user decide if they want to drop the cpu cou= nt to 1. We could just do nothing, let bhyve run and if it falls over people can use= debug mode and see the bhyve output themselves in the log. Just seems useful to be able to= tell users that their hardware doesn't support the features they are trying to use up = front. It still seems to me that vmx.initialised is a reasonable indicator that vm= m has loaded without issue, but it would be useful to have some documented way of checki= ng exactly what virt features the system supports, without just running someth= ing and seeing if it falls over. Matt -- Allan Jude _______________________________________________ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebs= d.org" -- -- Marcelo Araujo (__) araujo@FreeBSD.org \\\'',) http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_) From owner-freebsd-virtualization@freebsd.org Fri Aug 17 08:25:04 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 33F0010867BB for ; Fri, 17 Aug 2018 08:25:04 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-a.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C52FD7225C; Fri, 17 Aug 2018 08:25:03 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-a.userve.net (Postfix) with ESMTPS id 29EA38DD; Fri, 17 Aug 2018 09:25:01 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1534494301; bh=LawhPjb+exT33HXKCzjt2YhJ6KSe9MRMHUVnX0qqhpc=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=dc+KYB0VFfq6W2inIU9kx4MjZ8yqJ4uYIzaB91FX+/YOMXqmtnKqPs7sG0z72o9Ok Yo7RYq59Pw1b7+37lOOQi8NywrCOw0H2G0ZPf5qfrc72JxAFTPVtPzf9KABFT5ua8/ 0M9Km4J/Xh05fCED1B7yqzBTEvuUAvyLOQEkCw4Q= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Fri, 17 Aug 2018 09:25:00 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Fri, 17 Aug 2018 09:25:00 +0100 From: Matt Churchyard To: "Rodney W. Grimes" , Allan Jude CC: "freebsd-virtualization@freebsd.org" Subject: RE: Checking bhyve supported features (sysctls) Thread-Topic: Checking bhyve supported features (sysctls) Thread-Index: AdQ1Sj/JzUfq4S8/RYq4DRXDNWTckQAIYkuAAANnXRD///SbAIAABAyAgAAHIQCAAApWAP/++Rig Date: Fri, 17 Aug 2018 08:25:00 +0000 Message-ID: References: <201808161730.w7GHUaWv054788@pdx.rh.CN85.dnsmgr.net> In-Reply-To: <201808161730.w7GHUaWv054788@pdx.rh.CN85.dnsmgr.net> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Aug 2018 08:25:04 -0000 -----Original Message----- From: owner-freebsd-virtualization@freebsd.org On Behalf Of Rodney W. Grimes Sent: 16 August 2018 18:31 To: Allan Jude Cc: Matt Churchyard ; freebsd-virtualization@fr= eebsd.org Subject: Re: Checking bhyve supported features (sysctls) > On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" wrote: > >>=20 > >> Text manually wrapped to 80, any broken quoting is my fault - rwg > >>=20 > >> > > Hello, > >> > >=20 > >> > > I'm looking for better ways to check for bhyve support / > >available > >> > > features without trying to scan through dmesg output. > >> >=20 > >> > >Yes, it would be very good to remove that, as it usually tries=20 > >> > >to grep a non-existent file /var/run/dmesg.boot that is not=20 > >> > >created until after vm_bhyve has been called from > >/usr/local/etc/rc.d > >> > >when you have things set to autostartup >in /etc/rc.conf > >> >=20 > >> >=20 > >> > >=20 > >> > > I notice that the following 2 sysctl's appear to be set to 1 as > >soon > >> > > as the vmm module is loaded > >> > >=20 > >> > > hw.vmm.vmx.initialized: 1 > >> > > hw.vmm.vmx.cap.unrestricted_guest: 1 > >> > >=20 > >> > > Will these be available on both Intel & AMD processors as a way=20 > >> > > to determine if the module has loaded successfully and can run > >guests? > >> > >=20 > >> > > I also see the below sysctl related to iommu. > >> > >=20 > >> > > hw.vmm.iommu.initialized > >> > >=20 > >> > > Again, will this be set to 1 as soon as the module is loaded if=20 > >> > > iommu is supported, or only when it is used? > >> > > There also seems to be a vmm.amdvi.enable sysctl. > >> > > Would both these need checking or is vmm.iommu enough to=20 > >> > > determine support on any processor. > >> >=20 > >> > >Probalby the safest way for a shell script to decide if bhyve is=20 > >> > >up and running is to stat /dev/vmm, if that exists then the > >modules > >> > >have loaded and initialized and bhyve should be ready to process > >guests. > >> >=20 > >> > Hmm, I don't get /dev/vmm unless I actually have running guests. > >>=20 > >> I'll investigate that, I was pretty sure that you should get this=20 > >> as soon as the vmm.ko module is finished initialzing, but you might=20 > >> be right in that it takes a first vm to cause its creation. > >> Confirmed, /dev/vmm does not exist until the first vm is created. > >>=20 > >> >=20 > >> > >sysctl's mentiond above would be a poor way to make this > >determination. > >> >=20 > >> > It would be nice if sysctls were better documented. > >>=20 > >> Agreed. > >>=20 > >> > If vmx.initialized is set once vmm has successfully loaded, I=20 > >> > can't > >see a better way of checking for bhyve support (assuming it's not=20 > >Intel specific). This entry definitely exists and is set to 0 if you=20 > >load the module on a non-supported system, and set to 1 as soon as=20 > >vmm loads on my Intel test system. > >>=20 > >> Given its undocumented status you would be relying on an=20 > >> undocumented feature that could change in either name or behavior,=20 > >> and that is not desirable. > >>=20 > >> Let me see if I can come up with something else. > > > >I looked at the code for bhyvectl, bhyveload and byhve. They do not=20 > >actually try to decide if vmm is supported or not, they simply=20 > >process the error from a vm_create() or vm_open() call and exit with=20 > >an error code if they can not handle it (some of the code can handle=20 > >a vm_create failure if infact we are trying to create a vm that=20 > >already exists). > > > >If you want to maintain full compatibility a similiar stratergy may=20 > >be in order. > > > >Why is it that vm-bhyve specifically needs to know if the kernel has=20 > >vmm support or not? > >Cant it just be written to handle the errors returned if the=20 > >supported functions do not exist? >=20 > I think the question vm-bhyve wants to answer is: does the CPU have=20 > the required features to run a multicore VM. >Why does it need to know that? If it tries to start a multicore/thread VM= and the system can not support it an error is returned and the bhyve comma= nd fails. >Actually determining that specific issue is non-trivial iirc as some vmm s= upported CPU's can only run a single VM with a single thread in that VM (ea= rly core cpu's). >=20 > These or similar sysctls do seem to be the correct way to communicate=20 > that support. >I do not believe any of those sysctl's communicate that your on a broken c= pu or to what extent you can run vm's with multiple threads. So cap.unrestricted_guest from the vmm "capabilities" set of sysctls is not= a valid way to determine if the host has unrestricted guest support (requi= red for non-freebsd or multicore freebsd guests, and as you say missing fro= m some early VT-x capable processors)? >I went and looked at why vm-bhyve is groveling around in /var/run/dmesg.bo= ot and found that it is simply trying to determine if the host CPU is vmm c= apable, >specifically: >util::check_bhyve_support(){ >... >These checks are already built into the kernel. >This can all go in the bit bucket, if you try to start a VM on an unsuppor= ted system an error is returned, recoding this in shell is just setting you= rself up for "future" bugs. The kernel knows what features are supported but does not expose these, so = all I can do is similar to libvirt and run bhyve with different options to = see what errors pop up. I think I'll just remove all checking for now and let users discover the is= sue for themselves if bhyve won't run. Hopefully the vmx.initialized / cap.= * sysctls will at some point become a defined way of seeing if vmm is ready= / testing for vmm features, as apparently these serve no purpose at the mo= ment. Matt From owner-freebsd-virtualization@freebsd.org Fri Aug 17 08:29:12 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 569B91086961 for ; Fri, 17 Aug 2018 08:29:12 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-lj1-x22f.google.com (mail-lj1-x22f.google.com [IPv6:2a00:1450:4864:20::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B37C472431; Fri, 17 Aug 2018 08:29:11 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by mail-lj1-x22f.google.com with SMTP id p10-v6so5752134ljg.2; Fri, 17 Aug 2018 01:29:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:reply-to:in-reply-to:references:from:date:message-id :subject:to:cc; bh=2YKwfy3ER2K8WarJ79tmxNyklL8tHMjzNF1z/1WRCeQ=; b=PE+z2T/Ia6KxqUWT8mSssYFJCb6j5ziEJoE8sxfTc13Kk9Q7XkrMQ81IcJS7hS6Eb7 NzU4LYZV7A9HQ3MMx7S9f7NsxVSOEPVrvHeC4ELWpANHvJ6dJQnqAqUYemUMTVe/1zB1 2HD6KQZvooBA8njcEKbFfz0M2czhs9Xo5d+L6LS5AGWJ2ubzsd+R7Y0U28swBQfxLGhR q+qy2urYjjhJwfbPLVSRtwR1COnetZzhabHE7nTfRyjuHylDrytcOkXaL+KpNyJXteJO xiYTwexmyRHi7+v2UoRQgN9s0qjwGyXDrA336JPPxVwr8GNU73Td17EQyP9DMzvQURF4 BddQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=2YKwfy3ER2K8WarJ79tmxNyklL8tHMjzNF1z/1WRCeQ=; b=KSJVnVtk272XVOPbG8ZWkvneY2F/lp1DcfmJSVRBTdTcof1BjdN4Gy/WypPnMPOpde eQd/HTMJBh/DjZdC4cGul+bkOVnoROkbbCu4djw4BLkVFGDW9N401QGmrHDSaTdaihgA SD9Ajl2SaFxcuqwKHkeOBlNk00SLFuSY1y/4iPoJGZnJpk0g5lQI92lz3/Khi99vRIkl RswQsn+9cK9JzNlmRNV7H7ZlIKoOV9a4HzAAifyquTzROj3jz5BCpQO8CeCW2jM+9gAc JWltuD/G+6kP7pUSBu7efauOXEzSMa7a9TrOMGhc3jeCkccAWQMNTZs4W/pIVbnLdUBE VVzw== X-Gm-Message-State: AOUpUlFrJ2n5NspIwfyUuZmbaWisDmDZ/2sz4exyZ7oohPMig1qQWgvw gGD3zitMUJoXYaIVB+337ufNsXj98F2mcKaCcxHmKw== X-Google-Smtp-Source: AA+uWPxskA3D7MbK4pd8QAQpFKOq9znZSOO6fiAl1aLrujrc/15SKwbQL0L2O3mIzwaZA5wsJrLyC+3NKoNkhGPkAUE= X-Received: by 2002:a2e:9a95:: with SMTP id p21-v6mr23374994lji.60.1534494550318; Fri, 17 Aug 2018 01:29:10 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a19:1f4c:0:0:0:0:0 with HTTP; Fri, 17 Aug 2018 01:29:09 -0700 (PDT) Reply-To: araujo@freebsd.org In-Reply-To: References: <201808161730.w7GHUaWv054788@pdx.rh.CN85.dnsmgr.net> From: Marcelo Araujo Date: Fri, 17 Aug 2018 16:29:09 +0800 Message-ID: Subject: Re: Checking bhyve supported features (sysctls) To: Matt Churchyard Cc: "Rodney W. Grimes" , Allan Jude , "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Aug 2018 08:29:12 -0000 2018-08-17 16:25 GMT+08:00 Matt Churchyard : > -----Original Message----- > From: owner-freebsd-virtualization@freebsd.org < > owner-freebsd-virtualization@freebsd.org> On Behalf Of Rodney W. Grimes > Sent: 16 August 2018 18:31 > To: Allan Jude > Cc: Matt Churchyard ; freebsd-virtualization@ > freebsd.org > Subject: Re: Checking bhyve supported features (sysctls) > > > On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" < > freebsd-rwg@pdx.rh.CN85.dnsmgr.net> wrote: > > >> > > >> Text manually wrapped to 80, any broken quoting is my fault - rwg > > >> > > >> > > Hello, > > >> > > > > >> > > I'm looking for better ways to check for bhyve support / > > >available > > >> > > features without trying to scan through dmesg output. > > >> > > > >> > >Yes, it would be very good to remove that, as it usually tries > > >> > >to grep a non-existent file /var/run/dmesg.boot that is not > > >> > >created until after vm_bhyve has been called from > > >/usr/local/etc/rc.d > > >> > >when you have things set to autostartup >in /etc/rc.conf > > >> > > > >> > > > >> > > > > >> > > I notice that the following 2 sysctl's appear to be set to 1 as > > >soon > > >> > > as the vmm module is loaded > > >> > > > > >> > > hw.vmm.vmx.initialized: 1 > > >> > > hw.vmm.vmx.cap.unrestricted_guest: 1 > > >> > > > > >> > > Will these be available on both Intel & AMD processors as a way > > >> > > to determine if the module has loaded successfully and can run > > >guests? > > >> > > > > >> > > I also see the below sysctl related to iommu. > > >> > > > > >> > > hw.vmm.iommu.initialized > > >> > > > > >> > > Again, will this be set to 1 as soon as the module is loaded if > > >> > > iommu is supported, or only when it is used? > > >> > > There also seems to be a vmm.amdvi.enable sysctl. > > >> > > Would both these need checking or is vmm.iommu enough to > > >> > > determine support on any processor. > > >> > > > >> > >Probalby the safest way for a shell script to decide if bhyve is > > >> > >up and running is to stat /dev/vmm, if that exists then the > > >modules > > >> > >have loaded and initialized and bhyve should be ready to process > > >guests. > > >> > > > >> > Hmm, I don't get /dev/vmm unless I actually have running guests. > > >> > > >> I'll investigate that, I was pretty sure that you should get this > > >> as soon as the vmm.ko module is finished initialzing, but you might > > >> be right in that it takes a first vm to cause its creation. > > >> Confirmed, /dev/vmm does not exist until the first vm is created. > > >> > > >> > > > >> > >sysctl's mentiond above would be a poor way to make this > > >determination. > > >> > > > >> > It would be nice if sysctls were better documented. > > >> > > >> Agreed. > > >> > > >> > If vmx.initialized is set once vmm has successfully loaded, I > > >> > can't > > >see a better way of checking for bhyve support (assuming it's not > > >Intel specific). This entry definitely exists and is set to 0 if you > > >load the module on a non-supported system, and set to 1 as soon as > > >vmm loads on my Intel test system. > > >> > > >> Given its undocumented status you would be relying on an > > >> undocumented feature that could change in either name or behavior, > > >> and that is not desirable. > > >> > > >> Let me see if I can come up with something else. > > > > > >I looked at the code for bhyvectl, bhyveload and byhve. They do not > > >actually try to decide if vmm is supported or not, they simply > > >process the error from a vm_create() or vm_open() call and exit with > > >an error code if they can not handle it (some of the code can handle > > >a vm_create failure if infact we are trying to create a vm that > > >already exists). > > > > > >If you want to maintain full compatibility a similiar stratergy may > > >be in order. > > > > > >Why is it that vm-bhyve specifically needs to know if the kernel has > > >vmm support or not? > > >Cant it just be written to handle the errors returned if the > > >supported functions do not exist? > > > > I think the question vm-bhyve wants to answer is: does the CPU have > > the required features to run a multicore VM. > > >Why does it need to know that? If it tries to start a multicore/thread > VM and the system can not support it an error is returned and the bhyve > command fails. > > >Actually determining that specific issue is non-trivial iirc as some vmm > supported CPU's can only run a single VM with a single thread in that VM > (early core cpu's). > > > > > These or similar sysctls do seem to be the correct way to communicate > > that support. > > >I do not believe any of those sysctl's communicate that your on a broken > cpu or to what extent you can run vm's with multiple threads. > > So cap.unrestricted_guest from the vmm "capabilities" set of sysctls is > not a valid way to determine if the host has unrestricted guest support > (required for non-freebsd or multicore freebsd guests, and as you say > missing from some early VT-x capable processors)? > > >I went and looked at why vm-bhyve is groveling around in > /var/run/dmesg.boot and found that it is simply trying to determine if the > host CPU is vmm capable, > >specifically: > >util::check_bhyve_support(){ > >... > > >These checks are already built into the kernel. > >This can all go in the bit bucket, if you try to start a VM on an > unsupported system an error is returned, recoding this in shell is just > setting yourself up for "future" bugs. > > The kernel knows what features are supported but does not expose these, so > all I can do is similar to libvirt and run bhyve with different options to > see what errors pop up. > I think I'll just remove all checking for now and let users discover the > issue for themselves if bhyve won't run. Hopefully the vmx.initialized / > cap.* sysctls will at some point become a defined way of seeing if vmm is > ready / testing for vmm features, as apparently these serve no purpose at > the moment. > > Matt > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization- > unsubscribe@freebsd.org" > This is something that is in my long to-do list, I will try to get back to this in couple weeks. I think the way how you are dealing with it nowadays is the best way to try to discover the CPU capabilities, better in this way than let the users blind. Best, -- -- Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_) From owner-freebsd-virtualization@freebsd.org Fri Aug 17 08:54:13 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95E971087287 for ; Fri, 17 Aug 2018 08:54:13 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-a.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D21F730EE; Fri, 17 Aug 2018 08:54:12 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-a.userve.net (Postfix) with ESMTPS id B13D597C; Fri, 17 Aug 2018 09:54:11 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1534496051; bh=6SrGTRdrAd4rip3uxf9t/keN1kfXtkw0NEGTI5RRf2Y=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=K+xZauMaXr5AbyxBmiug1feJOPNkfnsUVgLjZFdm4REH1LAzmyjJGYZqcPsoXnt4x 4pqv0pwrwARBmrz5KfJgkm0DW6JhbktQBz7RxlcwQ+phGWjmPvEHyD81+WwMvfWipi F8kLDxYrhkh27xbCP89leI8FNPtHb/X9fwRk4gBU= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Fri, 17 Aug 2018 09:54:11 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Fri, 17 Aug 2018 09:54:11 +0100 From: Matt Churchyard To: "araujo@freebsd.org" CC: Allan Jude , "freebsd-virtualization@freebsd.org" Subject: RE: Checking bhyve supported features (sysctls) Thread-Topic: Checking bhyve supported features (sysctls) Thread-Index: AdQ1Sj/JzUfq4S8/RYq4DRXDNWTckQAIYkuAAANnXRD///SbAIAABAyAgAAHIQCAAApWAP/++RiggAIB9oD//+0swA== Date: Fri, 17 Aug 2018 08:54:10 +0000 Message-ID: <8fad62d974804195bf4518a88b7c53a8@SERVER.ad.usd-group.com> References: <201808161730.w7GHUaWv054788@pdx.rh.CN85.dnsmgr.net> In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Aug 2018 08:54:14 -0000 DQoNCjIwMTgtMDgtMTcgMTY6MjUgR01UKzA4OjAwIE1hdHQgQ2h1cmNoeWFyZCA8bWF0dC5jaHVy Y2h5YXJkQHVzZXJ2ZS5uZXQ8bWFpbHRvOm1hdHQuY2h1cmNoeWFyZEB1c2VydmUubmV0Pj46DQot LS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogb3duZXItZnJlZWJzZC12aXJ0dWFsaXph dGlvbkBmcmVlYnNkLm9yZzxtYWlsdG86b3duZXItZnJlZWJzZC12aXJ0dWFsaXphdGlvbkBmcmVl YnNkLm9yZz4gPG93bmVyLWZyZWVic2QtdmlydHVhbGl6YXRpb25AZnJlZWJzZC5vcmc8bWFpbHRv Om93bmVyLWZyZWVic2QtdmlydHVhbGl6YXRpb25AZnJlZWJzZC5vcmc+PiBPbiBCZWhhbGYgT2Yg Um9kbmV5IFcuIEdyaW1lcw0KU2VudDogMTYgQXVndXN0IDIwMTggMTg6MzENClRvOiBBbGxhbiBK dWRlIDxhbGxhbmp1ZGVARnJlZUJTRC5vcmc8bWFpbHRvOmFsbGFuanVkZUBGcmVlQlNELm9yZz4+ DQpDYzogTWF0dCBDaHVyY2h5YXJkIDxtYXR0LmNodXJjaHlhcmRAdXNlcnZlLm5ldDxtYWlsdG86 bWF0dC5jaHVyY2h5YXJkQHVzZXJ2ZS5uZXQ+PjsgZnJlZWJzZC12aXJ0dWFsaXphdGlvbkBmcmVl YnNkLm9yZzxtYWlsdG86ZnJlZWJzZC12aXJ0dWFsaXphdGlvbkBmcmVlYnNkLm9yZz4NClN1Ympl Y3Q6IFJlOiBDaGVja2luZyBiaHl2ZSBzdXBwb3J0ZWQgZmVhdHVyZXMgKHN5c2N0bHMpDQoNCj4g T24gQXVndXN0IDE2LCAyMDE4IDU6Mjg6MDUgUE0gR01UKzAxOjAwLCAiUm9kbmV5IFcuIEdyaW1l cyIgPGZyZWVic2QtcndnQHBkeC5yaC5DTjg1LmRuc21nci5uZXQ8bWFpbHRvOmZyZWVic2Qtcndn QHBkeC5yaC5DTjg1LmRuc21nci5uZXQ+PiB3cm90ZToNCj4gPj4NCj4gPj4gVGV4dCBtYW51YWxs eSB3cmFwcGVkIHRvIDgwLCBhbnkgYnJva2VuIHF1b3RpbmcgaXMgbXkgZmF1bHQgLSByd2cNCj4g Pj4NCj4gPj4gPiA+IEhlbGxvLA0KPiA+PiA+ID4NCj4gPj4gPiA+IEknbSBsb29raW5nIGZvciBi ZXR0ZXIgd2F5cyB0byBjaGVjayBmb3IgIGJoeXZlIHN1cHBvcnQgLw0KPiA+YXZhaWxhYmxlDQo+ ID4+ID4gPiBmZWF0dXJlcyB3aXRob3V0IHRyeWluZyB0byBzY2FuIHRocm91Z2ggZG1lc2cgb3V0 cHV0Lg0KPiA+PiA+DQo+ID4+ID4gPlllcywgaXQgd291bGQgYmUgdmVyeSBnb29kIHRvIHJlbW92 ZSB0aGF0LCBhcyBpdCB1c3VhbGx5IHRyaWVzDQo+ID4+ID4gPnRvIGdyZXAgYSBub24tZXhpc3Rl bnQgZmlsZSAvdmFyL3J1bi9kbWVzZy5ib290IHRoYXQgaXMgbm90DQo+ID4+ID4gPmNyZWF0ZWQg dW50aWwgYWZ0ZXIgdm1fYmh5dmUgaGFzIGJlZW4gY2FsbGVkIGZyb20NCj4gPi91c3IvbG9jYWwv ZXRjL3JjLmQNCj4gPj4gPiA+d2hlbiB5b3UgaGF2ZSB0aGluZ3Mgc2V0IHRvIGF1dG9zdGFydHVw ID5pbiAvZXRjL3JjLmNvbmYNCj4gPj4gPg0KPiA+PiA+DQo+ID4+ID4gPg0KPiA+PiA+ID4gSSBu b3RpY2UgdGhhdCB0aGUgZm9sbG93aW5nIDIgc3lzY3RsJ3MgYXBwZWFyIHRvIGJlIHNldCB0byAx IGFzDQo+ID5zb29uDQo+ID4+ID4gPiBhcyB0aGUgdm1tIG1vZHVsZSBpcyBsb2FkZWQNCj4gPj4g PiA+DQo+ID4+ID4gPiBody52bW0udm14LmluaXRpYWxpemVkOiAxDQo+ID4+ID4gPiBody52bW0u dm14LmNhcC51bnJlc3RyaWN0ZWRfZ3Vlc3Q6IDENCj4gPj4gPiA+DQo+ID4+ID4gPiBXaWxsIHRo ZXNlIGJlIGF2YWlsYWJsZSBvbiBib3RoIEludGVsICYgQU1EIHByb2Nlc3NvcnMgYXMgYSB3YXkN Cj4gPj4gPiA+IHRvIGRldGVybWluZSBpZiB0aGUgbW9kdWxlIGhhcyBsb2FkZWQgc3VjY2Vzc2Z1 bGx5IGFuZCBjYW4gcnVuDQo+ID5ndWVzdHM/DQo+ID4+ID4gPg0KPiA+PiA+ID4gSSBhbHNvIHNl ZSB0aGUgYmVsb3cgc3lzY3RsIHJlbGF0ZWQgdG8gaW9tbXUuDQo+ID4+ID4gPg0KPiA+PiA+ID4g aHcudm1tLmlvbW11LmluaXRpYWxpemVkDQo+ID4+ID4gPg0KPiA+PiA+ID4gQWdhaW4sIHdpbGwg dGhpcyBiZSBzZXQgdG8gMSBhcyBzb29uIGFzIHRoZSBtb2R1bGUgaXMgbG9hZGVkIGlmDQo+ID4+ ID4gPiBpb21tdSBpcyBzdXBwb3J0ZWQsIG9yIG9ubHkgd2hlbiBpdCBpcyB1c2VkPw0KPiA+PiA+ ID4gVGhlcmUgYWxzbyBzZWVtcyB0byBiZSBhIHZtbS5hbWR2aS5lbmFibGUgc3lzY3RsLg0KPiA+ PiA+ID4gV291bGQgYm90aCB0aGVzZSBuZWVkIGNoZWNraW5nIG9yIGlzIHZtbS5pb21tdSBlbm91 Z2ggdG8NCj4gPj4gPiA+IGRldGVybWluZSBzdXBwb3J0IG9uIGFueSBwcm9jZXNzb3IuDQo+ID4+ ID4NCj4gPj4gPiA+UHJvYmFsYnkgdGhlIHNhZmVzdCB3YXkgZm9yIGEgc2hlbGwgc2NyaXB0IHRv IGRlY2lkZSBpZiBiaHl2ZSBpcw0KPiA+PiA+ID51cCBhbmQgcnVubmluZyBpcyB0byBzdGF0IC9k ZXYvdm1tLCBpZiB0aGF0IGV4aXN0cyB0aGVuIHRoZQ0KPiA+bW9kdWxlcw0KPiA+PiA+ID5oYXZl IGxvYWRlZCBhbmQgaW5pdGlhbGl6ZWQgYW5kIGJoeXZlIHNob3VsZCBiZSByZWFkeSB0byBwcm9j ZXNzDQo+ID5ndWVzdHMuDQo+ID4+ID4NCj4gPj4gPiBIbW0sIEkgZG9uJ3QgZ2V0IC9kZXYvdm1t IHVubGVzcyBJIGFjdHVhbGx5IGhhdmUgcnVubmluZyBndWVzdHMuDQo+ID4+DQo+ID4+IEknbGwg aW52ZXN0aWdhdGUgdGhhdCwgSSB3YXMgcHJldHR5IHN1cmUgdGhhdCB5b3Ugc2hvdWxkIGdldCB0 aGlzDQo+ID4+IGFzIHNvb24gYXMgdGhlIHZtbS5rbyBtb2R1bGUgaXMgZmluaXNoZWQgaW5pdGlh bHppbmcsIGJ1dCB5b3UgbWlnaHQNCj4gPj4gYmUgcmlnaHQgaW4gdGhhdCBpdCB0YWtlcyBhIGZp cnN0IHZtIHRvIGNhdXNlIGl0cyBjcmVhdGlvbi4NCj4gPj4gQ29uZmlybWVkLCAvZGV2L3ZtbSBk b2VzIG5vdCBleGlzdCB1bnRpbCB0aGUgZmlyc3Qgdm0gaXMgY3JlYXRlZC4NCj4gPj4NCj4gPj4g Pg0KPiA+PiA+ID5zeXNjdGwncyBtZW50aW9uZCBhYm92ZSB3b3VsZCBiZSBhIHBvb3Igd2F5IHRv IG1ha2UgdGhpcw0KPiA+ZGV0ZXJtaW5hdGlvbi4NCj4gPj4gPg0KPiA+PiA+IEl0IHdvdWxkIGJl IG5pY2UgaWYgc3lzY3RscyB3ZXJlIGJldHRlciBkb2N1bWVudGVkLg0KPiA+Pg0KPiA+PiBBZ3Jl ZWQuDQo+ID4+DQo+ID4+ID4gSWYgdm14LmluaXRpYWxpemVkIGlzIHNldCBvbmNlIHZtbSBoYXMg c3VjY2Vzc2Z1bGx5IGxvYWRlZCwgSQ0KPiA+PiA+IGNhbid0DQo+ID5zZWUgYSBiZXR0ZXIgd2F5 IG9mIGNoZWNraW5nIGZvciBiaHl2ZSBzdXBwb3J0IChhc3N1bWluZyBpdCdzIG5vdA0KPiA+SW50 ZWwgc3BlY2lmaWMpLiBUaGlzIGVudHJ5IGRlZmluaXRlbHkgZXhpc3RzIGFuZCBpcyBzZXQgdG8g MCBpZiB5b3UNCj4gPmxvYWQgdGhlIG1vZHVsZSBvbiBhIG5vbi1zdXBwb3J0ZWQgc3lzdGVtLCBh bmQgc2V0IHRvIDEgYXMgc29vbiBhcw0KPiA+dm1tIGxvYWRzIG9uIG15IEludGVsIHRlc3Qgc3lz dGVtLg0KPiA+Pg0KPiA+PiBHaXZlbiBpdHMgdW5kb2N1bWVudGVkIHN0YXR1cyB5b3Ugd291bGQg YmUgcmVseWluZyBvbiBhbg0KPiA+PiB1bmRvY3VtZW50ZWQgZmVhdHVyZSB0aGF0IGNvdWxkIGNo YW5nZSBpbiBlaXRoZXIgbmFtZSBvciBiZWhhdmlvciwNCj4gPj4gYW5kIHRoYXQgaXMgbm90IGRl c2lyYWJsZS4NCj4gPj4NCj4gPj4gTGV0IG1lIHNlZSBpZiBJIGNhbiBjb21lIHVwIHdpdGggc29t ZXRoaW5nIGVsc2UuDQo+ID4NCj4gPkkgbG9va2VkIGF0IHRoZSBjb2RlIGZvciBiaHl2ZWN0bCwg Ymh5dmVsb2FkIGFuZCBieWh2ZS4gIFRoZXkgZG8gbm90DQo+ID5hY3R1YWxseSB0cnkgdG8gZGVj aWRlIGlmIHZtbSBpcyBzdXBwb3J0ZWQgb3Igbm90LCB0aGV5IHNpbXBseQ0KPiA+cHJvY2VzcyB0 aGUgZXJyb3IgZnJvbSBhIHZtX2NyZWF0ZSgpIG9yIHZtX29wZW4oKSBjYWxsIGFuZCBleGl0IHdp dGgNCj4gPmFuIGVycm9yIGNvZGUgaWYgdGhleSBjYW4gbm90IGhhbmRsZSBpdCAoc29tZSBvZiB0 aGUgY29kZSBjYW4gaGFuZGxlDQo+ID5hIHZtX2NyZWF0ZSBmYWlsdXJlIGlmIGluZmFjdCB3ZSBh cmUgdHJ5aW5nIHRvIGNyZWF0ZSBhIHZtIHRoYXQNCj4gPmFscmVhZHkgZXhpc3RzKS4NCj4gPg0K PiA+SWYgeW91IHdhbnQgdG8gbWFpbnRhaW4gZnVsbCBjb21wYXRpYmlsaXR5IGEgc2ltaWxpYXIg c3RyYXRlcmd5IG1heQ0KPiA+YmUgaW4gb3JkZXIuDQo+ID4NCj4gPldoeSBpcyBpdCB0aGF0IHZt LWJoeXZlIHNwZWNpZmljYWxseSBuZWVkcyB0byBrbm93IGlmIHRoZSBrZXJuZWwgaGFzDQo+ID52 bW0gc3VwcG9ydCBvciBub3Q/DQo+ID5DYW50IGl0IGp1c3QgYmUgd3JpdHRlbiB0byBoYW5kbGUg dGhlIGVycm9ycyByZXR1cm5lZCBpZiB0aGUNCj4gPnN1cHBvcnRlZCBmdW5jdGlvbnMgZG8gbm90 IGV4aXN0Pw0KPg0KPiBJIHRoaW5rIHRoZSBxdWVzdGlvbiB2bS1iaHl2ZSB3YW50cyB0byBhbnN3 ZXIgaXM6IGRvZXMgdGhlIENQVSBoYXZlDQo+IHRoZSByZXF1aXJlZCBmZWF0dXJlcyB0byBydW4g YSBtdWx0aWNvcmUgVk0uDQoNCj5XaHkgZG9lcyBpdCBuZWVkIHRvIGtub3cgdGhhdD8gIElmIGl0 IHRyaWVzIHRvIHN0YXJ0IGEgbXVsdGljb3JlL3RocmVhZCBWTSBhbmQgdGhlIHN5c3RlbSBjYW4g bm90IHN1cHBvcnQgaXQgYW4gZXJyb3IgaXMgcmV0dXJuZWQgYW5kIHRoZSBiaHl2ZSBjb21tYW5k IGZhaWxzLg0KDQo+QWN0dWFsbHkgZGV0ZXJtaW5pbmcgdGhhdCBzcGVjaWZpYyBpc3N1ZSBpcyBu b24tdHJpdmlhbCBpaXJjIGFzIHNvbWUgdm1tIHN1cHBvcnRlZCBDUFUncyBjYW4gb25seSBydW4g YSBzaW5nbGUgVk0gd2l0aCBhIHNpbmdsZSB0aHJlYWQgaW4gdGhhdCBWTSAoZWFybHkgY29yZSBj cHUncykuDQoNCj4NCj4gVGhlc2Ugb3Igc2ltaWxhciBzeXNjdGxzIGRvIHNlZW0gdG8gYmUgdGhl IGNvcnJlY3Qgd2F5IHRvIGNvbW11bmljYXRlDQo+IHRoYXQgc3VwcG9ydC4NCg0KPkkgZG8gbm90 IGJlbGlldmUgYW55IG9mIHRob3NlIHN5c2N0bCdzIGNvbW11bmljYXRlIHRoYXQgeW91ciBvbiBh IGJyb2tlbiBjcHUgb3IgdG8gd2hhdCBleHRlbnQgeW91IGNhbiBydW4gdm0ncyB3aXRoIG11bHRp cGxlIHRocmVhZHMuDQpTbyBjYXAudW5yZXN0cmljdGVkX2d1ZXN0IGZyb20gdGhlIHZtbSAiY2Fw YWJpbGl0aWVzIiBzZXQgb2Ygc3lzY3RscyBpcyBub3QgYSB2YWxpZCB3YXkgdG8gZGV0ZXJtaW5l IGlmIHRoZSBob3N0IGhhcyB1bnJlc3RyaWN0ZWQgZ3Vlc3Qgc3VwcG9ydCAocmVxdWlyZWQgZm9y IG5vbi1mcmVlYnNkIG9yIG11bHRpY29yZSBmcmVlYnNkIGd1ZXN0cywgYW5kIGFzIHlvdSBzYXkg bWlzc2luZyBmcm9tIHNvbWUgZWFybHkgVlQteCBjYXBhYmxlIHByb2Nlc3NvcnMpPw0KDQo+SSB3 ZW50IGFuZCBsb29rZWQgYXQgd2h5IHZtLWJoeXZlIGlzIGdyb3ZlbGluZyBhcm91bmQgaW4gL3Zh ci9ydW4vZG1lc2cuYm9vdCBhbmQgZm91bmQgdGhhdCBpdCBpcyBzaW1wbHkgdHJ5aW5nIHRvIGRl dGVybWluZSBpZiB0aGUgaG9zdCBDUFUgaXMgdm1tIGNhcGFibGUsDQo+c3BlY2lmaWNhbGx5Og0K PnV0aWw6OmNoZWNrX2JoeXZlX3N1cHBvcnQoKXsNCj4uLi4NCg0KPlRoZXNlIGNoZWNrcyBhcmUg YWxyZWFkeSBidWlsdCBpbnRvIHRoZSBrZXJuZWwuDQo+VGhpcyBjYW4gYWxsIGdvIGluIHRoZSBi aXQgYnVja2V0LCBpZiB5b3UgdHJ5IHRvIHN0YXJ0IGEgVk0gb24gYW4gdW5zdXBwb3J0ZWQgc3lz dGVtIGFuIGVycm9yIGlzIHJldHVybmVkLCByZWNvZGluZyB0aGlzIGluIHNoZWxsIGlzIGp1c3Qg c2V0dGluZyB5b3Vyc2VsZiB1cCBmb3IgImZ1dHVyZSIgYnVncy4NCg0KVGhlIGtlcm5lbCBrbm93 cyB3aGF0IGZlYXR1cmVzIGFyZSBzdXBwb3J0ZWQgYnV0IGRvZXMgbm90IGV4cG9zZSB0aGVzZSwg c28gYWxsIEkgY2FuIGRvIGlzIHNpbWlsYXIgdG8gbGlidmlydCBhbmQgcnVuIGJoeXZlIHdpdGgg ZGlmZmVyZW50IG9wdGlvbnMgdG8gc2VlIHdoYXQgZXJyb3JzIHBvcCB1cC4NCkkgdGhpbmsgSSds bCBqdXN0IHJlbW92ZSBhbGwgY2hlY2tpbmcgZm9yIG5vdyBhbmQgbGV0IHVzZXJzIGRpc2NvdmVy IHRoZSBpc3N1ZSBmb3IgdGhlbXNlbHZlcyBpZiBiaHl2ZSB3b24ndCBydW4uIEhvcGVmdWxseSB0 aGUgdm14LmluaXRpYWxpemVkIC8gY2FwLiogc3lzY3RscyB3aWxsIGF0IHNvbWUgcG9pbnQgYmVj b21lIGEgZGVmaW5lZCB3YXkgb2Ygc2VlaW5nIGlmIHZtbSBpcyByZWFkeSAvIHRlc3RpbmcgZm9y IHZtbSBmZWF0dXJlcywgYXMgYXBwYXJlbnRseSB0aGVzZSBzZXJ2ZSBubyBwdXJwb3NlIGF0IHRo ZSBtb21lbnQuDQoNCk1hdHQNCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fDQpmcmVlYnNkLXZpcnR1YWxpemF0aW9uQGZyZWVic2Qub3JnPG1haWx0bzpmcmVl YnNkLXZpcnR1YWxpemF0aW9uQGZyZWVic2Qub3JnPiBtYWlsaW5nIGxpc3QNCmh0dHBzOi8vbGlz dHMuZnJlZWJzZC5vcmcvbWFpbG1hbi9saXN0aW5mby9mcmVlYnNkLXZpcnR1YWxpemF0aW9uDQpU byB1bnN1YnNjcmliZSwgc2VuZCBhbnkgbWFpbCB0byAiZnJlZWJzZC12aXJ0dWFsaXphdGlvbi11 bnN1YnNjcmliZUBmcmVlYnNkLm9yZzxtYWlsdG86ZnJlZWJzZC12aXJ0dWFsaXphdGlvbi11bnN1 YnNjcmliZUBmcmVlYnNkLm9yZz4iDQoNCj5UaGlzIGlzIHNvbWV0aGluZyB0aGF0IGlzIGluIG15 IGxvbmcgdG8tZG8gbGlzdCwgSSB3aWxsIHRyeSB0byBnZXQgYmFjayB0byB0aGlzIGluIGNvdXBs ZSB3ZWVrcy4NCj5JIHRoaW5rIHRoZSB3YXkgaG93IHlvdSBhcmUgZGVhbGluZyB3aXRoIGl0IG5v d2FkYXlzIGlzIHRoZSBiZXN0IHdheSB0byB0cnkgdG8gZGlzY292ZXIgdGhlIENQVSBjYXBhYmls aXRpZXMsIGJldHRlciBpbiB0aGlzIHdheSB0aGFuIGxldCB0aGUgdXNlcnMgYmxpbmQuDQo+QmVz dCwNClRoYW5rcyBNYXJjZWxvLA0KDQpJdOKAmXMgbm90IGV4YWN0bHkgY3JpdGljYWwuIEkganVz dCBzYXcgYW5vdGhlciBwcm9qZWN0IHVzaW5nIHZteC5pbml0aWFsaXplZCBhbmQgaXQgc2VlbWVk IGEgbXVjaCBjbGVhbmVyIHdheSB0byBkZXRlcm1pbmUgc3VwcG9ydCBieSBzZWVpbmcgaWYgdm1t IGxvYWRzIHJhdGhlciB0aGFuIHByb2JpbmcgbG9nIGZpbGVzLiBJIGFza2VkIGhlcmUgZXhwZWN0 aW5nIHNvbWVvbmUgdG8ganVzdCBzYXkg4oCceWVzIGlmIHRoYXTigJlzIDEgdGhlbiB0aGUgaG9z dCB3aWxsIHJ1biBndWVzdHPigJ0sIG9yIOKAnG5vIHlvdSBjYW7igJl0IHRlbGwgaWYgZ3Vlc3Rz IHdpbGwgcnVuIGp1c3QgYnkgdGhhdCBzeXNjdGzigJ0uDQoNCk1hdHQNCg== From owner-freebsd-virtualization@freebsd.org Fri Aug 17 09:14:31 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1ECEF1087C8E for ; Fri, 17 Aug 2018 09:14:31 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-lj1-x22b.google.com (mail-lj1-x22b.google.com [IPv6:2a00:1450:4864:20::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6EA4F73C00; Fri, 17 Aug 2018 09:14:30 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by mail-lj1-x22b.google.com with SMTP id l15-v6so5825903lji.6; Fri, 17 Aug 2018 02:14:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:reply-to:in-reply-to:references:from:date:message-id :subject:to:cc; bh=jOfF9FEfUfcrohxnRAfyR+IcwCjhr0qpJrO3q9GkM+U=; b=exqqkCsbi0tljWk6bHNRMPyumS0l8YGLBdUDdNz4ZX2VsAk21HXwFnMPpista88AwA UPkB68G03IEgVAD+aCyc0AdQZYZKReMhxec3rzYqgkGH38Q7Bxn3qotg1IRF72Iml7lq vbbub/e/DiYT0/sWQgm3aT4/0mPDHMn4dpG2dnbVDnKBsHm5bGKseLnfjreKMp3iRufv g2QZRGT1+Eeo82pGOhC2/N1HtD+S/LSsRo7kW4SWBOufXwzMHR8i6YkfRAtQpvP1jTB+ VdsHaGW6R0W5kt2e74FXH/2SgzIgyeFggCZWw3dwz043nTKsSnFyUEcGU+6FJw8J3arm 6/DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=jOfF9FEfUfcrohxnRAfyR+IcwCjhr0qpJrO3q9GkM+U=; b=sMfjegc21iGDt37OVc5cXDNtzHmOxngP+SplIipx+geJjSwtfsBhgIcO5LzKB/Nfd5 VLZtBBDuWhTnx0XMxThxqwvHSTKlec/22sdaOIreSBpV5FZmP5J8fQCsBPvyn1E/VjDi ZlOrjxUFMYEDqgmesshmYNYgYeo28WRdeQ7ZOnZ1XqpGl+BaBWVKuHU3utCo6bcS42es Mw/KU3rwTKfHN38mFDk3OxQDnOsBawNCinh1gvpH7auh4oQJ6DSQ4mdLp2BpDhjV/G9t UkW0F6ohV/JVm5N5qrNhgFvzIgA5ydxRaS49lm2+RbcRE6GA1Y4Q/nINmEv3act6UWwx Og9Q== X-Gm-Message-State: AOUpUlE6+KOdhlJLlKh/+1TjdBagbTHTlMlS7QeBbw5g/mq58lEBhD34 P5YfVCuqg0RH9H+7No9T3Y4eNnBVrnSUasEbDrqLYA== X-Google-Smtp-Source: AA+uWPyQl50naROas8ThV7G/Q1SFqPPxoaApOWKcKADMK2TCP7v9cClD8vX8Cn/uLL3lfJdkFehYwi+f2z7IQMO2QfU= X-Received: by 2002:a2e:9599:: with SMTP id w25-v6mr1141290ljh.6.1534497268289; Fri, 17 Aug 2018 02:14:28 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a19:1f4c:0:0:0:0:0 with HTTP; Fri, 17 Aug 2018 02:14:27 -0700 (PDT) Reply-To: araujo@freebsd.org In-Reply-To: <8fad62d974804195bf4518a88b7c53a8@SERVER.ad.usd-group.com> References: <201808161730.w7GHUaWv054788@pdx.rh.CN85.dnsmgr.net> <8fad62d974804195bf4518a88b7c53a8@SERVER.ad.usd-group.com> From: Marcelo Araujo Date: Fri, 17 Aug 2018 17:14:27 +0800 Message-ID: Subject: Re: Checking bhyve supported features (sysctls) To: Matt Churchyard Cc: Allan Jude , "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Aug 2018 09:14:31 -0000 2018-08-17 16:54 GMT+08:00 Matt Churchyard : > > > > > 2018-08-17 16:25 GMT+08:00 Matt Churchyard : > > -----Original Message----- > From: owner-freebsd-virtualization@freebsd.org < > owner-freebsd-virtualization@freebsd.org> On Behalf Of Rodney W. Grimes > Sent: 16 August 2018 18:31 > To: Allan Jude > Cc: Matt Churchyard ; freebsd-virtualization@ > freebsd.org > Subject: Re: Checking bhyve supported features (sysctls) > > > On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" < > freebsd-rwg@pdx.rh.CN85.dnsmgr.net> wrote: > > >> > > >> Text manually wrapped to 80, any broken quoting is my fault - rwg > > >> > > >> > > Hello, > > >> > > > > >> > > I'm looking for better ways to check for bhyve support / > > >available > > >> > > features without trying to scan through dmesg output. > > >> > > > >> > >Yes, it would be very good to remove that, as it usually tries > > >> > >to grep a non-existent file /var/run/dmesg.boot that is not > > >> > >created until after vm_bhyve has been called from > > >/usr/local/etc/rc.d > > >> > >when you have things set to autostartup >in /etc/rc.conf > > >> > > > >> > > > >> > > > > >> > > I notice that the following 2 sysctl's appear to be set to 1 as > > >soon > > >> > > as the vmm module is loaded > > >> > > > > >> > > hw.vmm.vmx.initialized: 1 > > >> > > hw.vmm.vmx.cap.unrestricted_guest: 1 > > >> > > > > >> > > Will these be available on both Intel & AMD processors as a way > > >> > > to determine if the module has loaded successfully and can run > > >guests? > > >> > > > > >> > > I also see the below sysctl related to iommu. > > >> > > > > >> > > hw.vmm.iommu.initialized > > >> > > > > >> > > Again, will this be set to 1 as soon as the module is loaded if > > >> > > iommu is supported, or only when it is used? > > >> > > There also seems to be a vmm.amdvi.enable sysctl. > > >> > > Would both these need checking or is vmm.iommu enough to > > >> > > determine support on any processor. > > >> > > > >> > >Probalby the safest way for a shell script to decide if bhyve is > > >> > >up and running is to stat /dev/vmm, if that exists then the > > >modules > > >> > >have loaded and initialized and bhyve should be ready to process > > >guests. > > >> > > > >> > Hmm, I don't get /dev/vmm unless I actually have running guests. > > >> > > >> I'll investigate that, I was pretty sure that you should get this > > >> as soon as the vmm.ko module is finished initialzing, but you might > > >> be right in that it takes a first vm to cause its creation. > > >> Confirmed, /dev/vmm does not exist until the first vm is created. > > >> > > >> > > > >> > >sysctl's mentiond above would be a poor way to make this > > >determination. > > >> > > > >> > It would be nice if sysctls were better documented. > > >> > > >> Agreed. > > >> > > >> > If vmx.initialized is set once vmm has successfully loaded, I > > >> > can't > > >see a better way of checking for bhyve support (assuming it's not > > >Intel specific). This entry definitely exists and is set to 0 if you > > >load the module on a non-supported system, and set to 1 as soon as > > >vmm loads on my Intel test system. > > >> > > >> Given its undocumented status you would be relying on an > > >> undocumented feature that could change in either name or behavior, > > >> and that is not desirable. > > >> > > >> Let me see if I can come up with something else. > > > > > >I looked at the code for bhyvectl, bhyveload and byhve. They do not > > >actually try to decide if vmm is supported or not, they simply > > >process the error from a vm_create() or vm_open() call and exit with > > >an error code if they can not handle it (some of the code can handle > > >a vm_create failure if infact we are trying to create a vm that > > >already exists). > > > > > >If you want to maintain full compatibility a similiar stratergy may > > >be in order. > > > > > >Why is it that vm-bhyve specifically needs to know if the kernel has > > >vmm support or not? > > >Cant it just be written to handle the errors returned if the > > >supported functions do not exist? > > > > I think the question vm-bhyve wants to answer is: does the CPU have > > the required features to run a multicore VM. > > >Why does it need to know that? If it tries to start a multicore/thread > VM and the system can not support it an error is returned and the bhyve > command fails. > > >Actually determining that specific issue is non-trivial iirc as some vmm > supported CPU's can only run a single VM with a single thread in that VM > (early core cpu's). > > > > > These or similar sysctls do seem to be the correct way to communicate > > that support. > > >I do not believe any of those sysctl's communicate that your on a broken > cpu or to what extent you can run vm's with multiple threads. > > So cap.unrestricted_guest from the vmm "capabilities" set of sysctls is > not a valid way to determine if the host has unrestricted guest support > (required for non-freebsd or multicore freebsd guests, and as you say > missing from some early VT-x capable processors)? > > >I went and looked at why vm-bhyve is groveling around in > /var/run/dmesg.boot and found that it is simply trying to determine if th= e > host CPU is vmm capable, > >specifically: > >util::check_bhyve_support(){ > >... > > >These checks are already built into the kernel. > >This can all go in the bit bucket, if you try to start a VM on an > unsupported system an error is returned, recoding this in shell is just > setting yourself up for "future" bugs. > > The kernel knows what features are supported but does not expose these, s= o > all I can do is similar to libvirt and run bhyve with different options t= o > see what errors pop up. > I think I'll just remove all checking for now and let users discover the > issue for themselves if bhyve won't run. Hopefully the vmx.initialized / > cap.* sysctls will at some point become a defined way of seeing if vmm is > ready / testing for vmm features, as apparently these serve no purpose at > the moment. > > Matt > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization- > unsubscribe@freebsd.org" > > > > >This is something that is in my long to-do list, I will try to get back > to this in couple weeks. > > >I think the way how you are dealing with it nowadays is the best way to > try to discover the CPU capabilities, better in this way than let the use= rs > blind. > > >Best, > > Thanks Marcelo, > > > > It=E2=80=99s not exactly critical. I just saw another project using > vmx.initialized and it seemed a much cleaner way to determine support by > seeing if vmm loads rather than probing log files. I asked here expecting > someone to just say =E2=80=9Cyes if that=E2=80=99s 1 then the host will r= un guests=E2=80=9D, or =E2=80=9Cno > you can=E2=80=99t tell if guests will run just by that sysctl=E2=80=9D. > > > > Matt > Could you share with me what project is using vmx.initialized? Best, --=20 --=20 Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_) From owner-freebsd-virtualization@freebsd.org Fri Aug 17 12:10:37 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6627106B868 for ; Fri, 17 Aug 2018 12:10:36 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-b.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48EFA79876; Fri, 17 Aug 2018 12:10:35 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-b.userve.net (Postfix) with ESMTPS id 97882A81; Fri, 17 Aug 2018 13:10:29 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1534507829; bh=/SitbP9PF5EBYBDykwRIf9HfmRiyyiTZAaQVre2VvNI=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=EBZYahEC03gAPB0Kxh6L1xihq6jdH8CGnHrgNEFjSB9eJ+FP893nAnrqhRh8IRI1i ZI/d2o0sTGL9SS095Kk//6leh3mTaJWLE1fKLt2SMXZNJA32ZK8GY1XEax5qVVHarD o1IA+dYNHdXbiyivFlPjS0yQws2o1vPojmljCgjc= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Fri, 17 Aug 2018 13:10:29 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Fri, 17 Aug 2018 13:10:28 +0100 From: Matt Churchyard To: "araujo@freebsd.org" CC: "freebsd-virtualization@freebsd.org" Subject: RE: Checking bhyve supported features (sysctls) Thread-Topic: Checking bhyve supported features (sysctls) Thread-Index: AdQ1Sj/JzUfq4S8/RYq4DRXDNWTckQAIYkuAAANnXRD///SbAIAABAyAgAAHIQCAAApWAP/++RiggAIB9oD//+0swIAAH3yA//+/4MA= Date: Fri, 17 Aug 2018 12:10:28 +0000 Message-ID: <80e2360b6e6440719f2d6a69ac2fe9ed@SERVER.ad.usd-group.com> References: <201808161730.w7GHUaWv054788@pdx.rh.CN85.dnsmgr.net> <8fad62d974804195bf4518a88b7c53a8@SERVER.ad.usd-group.com> In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Aug 2018 12:10:37 -0000 LS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCkZyb206IG93bmVyLWZyZWVic2QtdmlydHVhbGl6 YXRpb25AZnJlZWJzZC5vcmcgPG93bmVyLWZyZWVic2QtdmlydHVhbGl6YXRpb25AZnJlZWJzZC5v cmc+IE9uIEJlaGFsZiBPZiBNYXJjZWxvIEFyYXVqbw0KU2VudDogMTcgQXVndXN0IDIwMTggMTA6 MTQNClRvOiBNYXR0IENodXJjaHlhcmQgPG1hdHQuY2h1cmNoeWFyZEB1c2VydmUubmV0Pg0KQ2M6 IGZyZWVic2QtdmlydHVhbGl6YXRpb25AZnJlZWJzZC5vcmcNClN1YmplY3Q6IFJlOiBDaGVja2lu ZyBiaHl2ZSBzdXBwb3J0ZWQgZmVhdHVyZXMgKHN5c2N0bHMpDQoNCjIwMTgtMDgtMTcgMTY6NTQg R01UKzA4OjAwIE1hdHQgQ2h1cmNoeWFyZCA8bWF0dC5jaHVyY2h5YXJkQHVzZXJ2ZS5uZXQ+Og0K DQo+DQo+DQo+DQo+DQo+IDIwMTgtMDgtMTcgMTY6MjUgR01UKzA4OjAwIE1hdHQgQ2h1cmNoeWFy ZCA8bWF0dC5jaHVyY2h5YXJkQHVzZXJ2ZS5uZXQ+Og0KPg0KPiAtLS0tLU9yaWdpbmFsIE1lc3Nh Z2UtLS0tLQ0KPiBGcm9tOiBvd25lci1mcmVlYnNkLXZpcnR1YWxpemF0aW9uQGZyZWVic2Qub3Jn IDwgDQo+IG93bmVyLWZyZWVic2QtdmlydHVhbGl6YXRpb25AZnJlZWJzZC5vcmc+IE9uIEJlaGFs ZiBPZiBSb2RuZXkgVy4gDQo+IEdyaW1lcw0KPiBTZW50OiAxNiBBdWd1c3QgMjAxOCAxODozMQ0K PiBUbzogQWxsYW4gSnVkZSA8YWxsYW5qdWRlQEZyZWVCU0Qub3JnPg0KPiBDYzogTWF0dCBDaHVy Y2h5YXJkIDxtYXR0LmNodXJjaHlhcmRAdXNlcnZlLm5ldD47IA0KPiBmcmVlYnNkLXZpcnR1YWxp emF0aW9uQCBmcmVlYnNkLm9yZw0KPiBTdWJqZWN0OiBSZTogQ2hlY2tpbmcgYmh5dmUgc3VwcG9y dGVkIGZlYXR1cmVzIChzeXNjdGxzKQ0KPg0KPiA+IE9uIEF1Z3VzdCAxNiwgMjAxOCA1OjI4OjA1 IFBNIEdNVCswMTowMCwgIlJvZG5leSBXLiBHcmltZXMiIDwNCj4gZnJlZWJzZC1yd2dAcGR4LnJo LkNOODUuZG5zbWdyLm5ldD4gd3JvdGU6DQo+ID4gPj4NCj4gPiA+PiBUZXh0IG1hbnVhbGx5IHdy YXBwZWQgdG8gODAsIGFueSBicm9rZW4gcXVvdGluZyBpcyBteSBmYXVsdCAtIHJ3Zw0KPiA+ID4+ DQo+ID4gPj4gPiA+IEhlbGxvLA0KPiA+ID4+ID4gPg0KPiA+ID4+ID4gPiBJJ20gbG9va2luZyBm b3IgYmV0dGVyIHdheXMgdG8gY2hlY2sgZm9yICBiaHl2ZSBzdXBwb3J0IC8NCj4gPiA+YXZhaWxh YmxlDQo+ID4gPj4gPiA+IGZlYXR1cmVzIHdpdGhvdXQgdHJ5aW5nIHRvIHNjYW4gdGhyb3VnaCBk bWVzZyBvdXRwdXQuDQo+ID4gPj4gPg0KPiA+ID4+ID4gPlllcywgaXQgd291bGQgYmUgdmVyeSBn b29kIHRvIHJlbW92ZSB0aGF0LCBhcyBpdCB1c3VhbGx5IHRyaWVzIA0KPiA+ID4+ID4gPnRvIGdy ZXAgYSBub24tZXhpc3RlbnQgZmlsZSAvdmFyL3J1bi9kbWVzZy5ib290IHRoYXQgaXMgbm90IA0K PiA+ID4+ID4gPmNyZWF0ZWQgdW50aWwgYWZ0ZXIgdm1fYmh5dmUgaGFzIGJlZW4gY2FsbGVkIGZy b20NCj4gPiA+L3Vzci9sb2NhbC9ldGMvcmMuZA0KPiA+ID4+ID4gPndoZW4geW91IGhhdmUgdGhp bmdzIHNldCB0byBhdXRvc3RhcnR1cCA+aW4gL2V0Yy9yYy5jb25mDQo+ID4gPj4gPg0KPiA+ID4+ ID4NCj4gPiA+PiA+ID4NCj4gPiA+PiA+ID4gSSBub3RpY2UgdGhhdCB0aGUgZm9sbG93aW5nIDIg c3lzY3RsJ3MgYXBwZWFyIHRvIGJlIHNldCB0byAxIA0KPiA+ID4+ID4gPiBhcw0KPiA+ID5zb29u DQo+ID4gPj4gPiA+IGFzIHRoZSB2bW0gbW9kdWxlIGlzIGxvYWRlZA0KPiA+ID4+ID4gPg0KPiA+ ID4+ID4gPiBody52bW0udm14LmluaXRpYWxpemVkOiAxDQo+ID4gPj4gPiA+IGh3LnZtbS52bXgu Y2FwLnVucmVzdHJpY3RlZF9ndWVzdDogMQ0KPiA+ID4+ID4gPg0KPiA+ID4+ID4gPiBXaWxsIHRo ZXNlIGJlIGF2YWlsYWJsZSBvbiBib3RoIEludGVsICYgQU1EIHByb2Nlc3NvcnMgYXMgYSANCj4g PiA+PiA+ID4gd2F5IHRvIGRldGVybWluZSBpZiB0aGUgbW9kdWxlIGhhcyBsb2FkZWQgc3VjY2Vz c2Z1bGx5IGFuZCANCj4gPiA+PiA+ID4gY2FuIHJ1bg0KPiA+ID5ndWVzdHM/DQo+ID4gPj4gPiA+ DQo+ID4gPj4gPiA+IEkgYWxzbyBzZWUgdGhlIGJlbG93IHN5c2N0bCByZWxhdGVkIHRvIGlvbW11 Lg0KPiA+ID4+ID4gPg0KPiA+ID4+ID4gPiBody52bW0uaW9tbXUuaW5pdGlhbGl6ZWQNCj4gPiA+ PiA+ID4NCj4gPiA+PiA+ID4gQWdhaW4sIHdpbGwgdGhpcyBiZSBzZXQgdG8gMSBhcyBzb29uIGFz IHRoZSBtb2R1bGUgaXMgbG9hZGVkIA0KPiA+ID4+ID4gPiBpZiBpb21tdSBpcyBzdXBwb3J0ZWQs IG9yIG9ubHkgd2hlbiBpdCBpcyB1c2VkPw0KPiA+ID4+ID4gPiBUaGVyZSBhbHNvIHNlZW1zIHRv IGJlIGEgdm1tLmFtZHZpLmVuYWJsZSBzeXNjdGwuDQo+ID4gPj4gPiA+IFdvdWxkIGJvdGggdGhl c2UgbmVlZCBjaGVja2luZyBvciBpcyB2bW0uaW9tbXUgZW5vdWdoIHRvIA0KPiA+ID4+ID4gPiBk ZXRlcm1pbmUgc3VwcG9ydCBvbiBhbnkgcHJvY2Vzc29yLg0KPiA+ID4+ID4NCj4gPiA+PiA+ID5Q cm9iYWxieSB0aGUgc2FmZXN0IHdheSBmb3IgYSBzaGVsbCBzY3JpcHQgdG8gZGVjaWRlIGlmIGJo eXZlIA0KPiA+ID4+ID4gPmlzIHVwIGFuZCBydW5uaW5nIGlzIHRvIHN0YXQgL2Rldi92bW0sIGlm IHRoYXQgZXhpc3RzIHRoZW4gdGhlDQo+ID4gPm1vZHVsZXMNCj4gPiA+PiA+ID5oYXZlIGxvYWRl ZCBhbmQgaW5pdGlhbGl6ZWQgYW5kIGJoeXZlIHNob3VsZCBiZSByZWFkeSB0byANCj4gPiA+PiA+ ID5wcm9jZXNzDQo+ID4gPmd1ZXN0cy4NCj4gPiA+PiA+DQo+ID4gPj4gPiBIbW0sIEkgZG9uJ3Qg Z2V0IC9kZXYvdm1tIHVubGVzcyBJIGFjdHVhbGx5IGhhdmUgcnVubmluZyBndWVzdHMuDQo+ID4g Pj4NCj4gPiA+PiBJJ2xsIGludmVzdGlnYXRlIHRoYXQsIEkgd2FzIHByZXR0eSBzdXJlIHRoYXQg eW91IHNob3VsZCBnZXQgdGhpcyANCj4gPiA+PiBhcyBzb29uIGFzIHRoZSB2bW0ua28gbW9kdWxl IGlzIGZpbmlzaGVkIGluaXRpYWx6aW5nLCBidXQgeW91IA0KPiA+ID4+IG1pZ2h0IGJlIHJpZ2h0 IGluIHRoYXQgaXQgdGFrZXMgYSBmaXJzdCB2bSB0byBjYXVzZSBpdHMgY3JlYXRpb24uDQo+ID4g Pj4gQ29uZmlybWVkLCAvZGV2L3ZtbSBkb2VzIG5vdCBleGlzdCB1bnRpbCB0aGUgZmlyc3Qgdm0g aXMgY3JlYXRlZC4NCj4gPiA+Pg0KPiA+ID4+ID4NCj4gPiA+PiA+ID5zeXNjdGwncyBtZW50aW9u ZCBhYm92ZSB3b3VsZCBiZSBhIHBvb3Igd2F5IHRvIG1ha2UgdGhpcw0KPiA+ID5kZXRlcm1pbmF0 aW9uLg0KPiA+ID4+ID4NCj4gPiA+PiA+IEl0IHdvdWxkIGJlIG5pY2UgaWYgc3lzY3RscyB3ZXJl IGJldHRlciBkb2N1bWVudGVkLg0KPiA+ID4+DQo+ID4gPj4gQWdyZWVkLg0KPiA+ID4+DQo+ID4g Pj4gPiBJZiB2bXguaW5pdGlhbGl6ZWQgaXMgc2V0IG9uY2Ugdm1tIGhhcyBzdWNjZXNzZnVsbHkg bG9hZGVkLCBJIA0KPiA+ID4+ID4gY2FuJ3QNCj4gPiA+c2VlIGEgYmV0dGVyIHdheSBvZiBjaGVj a2luZyBmb3IgYmh5dmUgc3VwcG9ydCAoYXNzdW1pbmcgaXQncyBub3QgDQo+ID4gPkludGVsIHNw ZWNpZmljKS4gVGhpcyBlbnRyeSBkZWZpbml0ZWx5IGV4aXN0cyBhbmQgaXMgc2V0IHRvIDAgaWYg DQo+ID4gPnlvdSBsb2FkIHRoZSBtb2R1bGUgb24gYSBub24tc3VwcG9ydGVkIHN5c3RlbSwgYW5k IHNldCB0byAxIGFzIHNvb24gDQo+ID4gPmFzIHZtbSBsb2FkcyBvbiBteSBJbnRlbCB0ZXN0IHN5 c3RlbS4NCj4gPiA+Pg0KPiA+ID4+IEdpdmVuIGl0cyB1bmRvY3VtZW50ZWQgc3RhdHVzIHlvdSB3 b3VsZCBiZSByZWx5aW5nIG9uIGFuIA0KPiA+ID4+IHVuZG9jdW1lbnRlZCBmZWF0dXJlIHRoYXQg Y291bGQgY2hhbmdlIGluIGVpdGhlciBuYW1lIG9yIA0KPiA+ID4+IGJlaGF2aW9yLCBhbmQgdGhh dCBpcyBub3QgZGVzaXJhYmxlLg0KPiA+ID4+DQo+ID4gPj4gTGV0IG1lIHNlZSBpZiBJIGNhbiBj b21lIHVwIHdpdGggc29tZXRoaW5nIGVsc2UuDQo+ID4gPg0KPiA+ID5JIGxvb2tlZCBhdCB0aGUg Y29kZSBmb3IgYmh5dmVjdGwsIGJoeXZlbG9hZCBhbmQgYnlodmUuICBUaGV5IGRvIA0KPiA+ID5u b3QgYWN0dWFsbHkgdHJ5IHRvIGRlY2lkZSBpZiB2bW0gaXMgc3VwcG9ydGVkIG9yIG5vdCwgdGhl eSBzaW1wbHkgDQo+ID4gPnByb2Nlc3MgdGhlIGVycm9yIGZyb20gYSB2bV9jcmVhdGUoKSBvciB2 bV9vcGVuKCkgY2FsbCBhbmQgZXhpdCANCj4gPiA+d2l0aCBhbiBlcnJvciBjb2RlIGlmIHRoZXkg Y2FuIG5vdCBoYW5kbGUgaXQgKHNvbWUgb2YgdGhlIGNvZGUgY2FuIA0KPiA+ID5oYW5kbGUgYSB2 bV9jcmVhdGUgZmFpbHVyZSBpZiBpbmZhY3Qgd2UgYXJlIHRyeWluZyB0byBjcmVhdGUgYSB2bSAN Cj4gPiA+dGhhdCBhbHJlYWR5IGV4aXN0cykuDQo+ID4gPg0KPiA+ID5JZiB5b3Ugd2FudCB0byBt YWludGFpbiBmdWxsIGNvbXBhdGliaWxpdHkgYSBzaW1pbGlhciBzdHJhdGVyZ3kgbWF5IA0KPiA+ ID5iZSBpbiBvcmRlci4NCj4gPiA+DQo+ID4gPldoeSBpcyBpdCB0aGF0IHZtLWJoeXZlIHNwZWNp ZmljYWxseSBuZWVkcyB0byBrbm93IGlmIHRoZSBrZXJuZWwgDQo+ID4gPmhhcyB2bW0gc3VwcG9y dCBvciBub3Q/DQo+ID4gPkNhbnQgaXQganVzdCBiZSB3cml0dGVuIHRvIGhhbmRsZSB0aGUgZXJy b3JzIHJldHVybmVkIGlmIHRoZSANCj4gPiA+c3VwcG9ydGVkIGZ1bmN0aW9ucyBkbyBub3QgZXhp c3Q/DQo+ID4NCj4gPiBJIHRoaW5rIHRoZSBxdWVzdGlvbiB2bS1iaHl2ZSB3YW50cyB0byBhbnN3 ZXIgaXM6IGRvZXMgdGhlIENQVSBoYXZlIA0KPiA+IHRoZSByZXF1aXJlZCBmZWF0dXJlcyB0byBy dW4gYSBtdWx0aWNvcmUgVk0uDQo+DQo+ID5XaHkgZG9lcyBpdCBuZWVkIHRvIGtub3cgdGhhdD8g IElmIGl0IHRyaWVzIHRvIHN0YXJ0IGEgDQo+ID5tdWx0aWNvcmUvdGhyZWFkDQo+IFZNIGFuZCB0 aGUgc3lzdGVtIGNhbiBub3Qgc3VwcG9ydCBpdCBhbiBlcnJvciBpcyByZXR1cm5lZCBhbmQgdGhl IA0KPiBiaHl2ZSBjb21tYW5kIGZhaWxzLg0KPg0KPiA+QWN0dWFsbHkgZGV0ZXJtaW5pbmcgdGhh dCBzcGVjaWZpYyBpc3N1ZSBpcyBub24tdHJpdmlhbCBpaXJjIGFzIHNvbWUgDQo+ID52bW0NCj4g c3VwcG9ydGVkIENQVSdzIGNhbiBvbmx5IHJ1biBhIHNpbmdsZSBWTSB3aXRoIGEgc2luZ2xlIHRo cmVhZCBpbiB0aGF0IA0KPiBWTSAoZWFybHkgY29yZSBjcHUncykuDQo+DQo+ID4NCj4gPiBUaGVz ZSBvciBzaW1pbGFyIHN5c2N0bHMgZG8gc2VlbSB0byBiZSB0aGUgY29ycmVjdCB3YXkgdG8gDQo+ ID4gY29tbXVuaWNhdGUgdGhhdCBzdXBwb3J0Lg0KPg0KPiA+SSBkbyBub3QgYmVsaWV2ZSBhbnkg b2YgdGhvc2Ugc3lzY3RsJ3MgY29tbXVuaWNhdGUgdGhhdCB5b3VyIG9uIGEgDQo+ID5icm9rZW4N Cj4gY3B1IG9yIHRvIHdoYXQgZXh0ZW50IHlvdSBjYW4gcnVuIHZtJ3Mgd2l0aCBtdWx0aXBsZSB0 aHJlYWRzLg0KPg0KPiBTbyBjYXAudW5yZXN0cmljdGVkX2d1ZXN0IGZyb20gdGhlIHZtbSAiY2Fw YWJpbGl0aWVzIiBzZXQgb2Ygc3lzY3RscyANCj4gaXMgbm90IGEgdmFsaWQgd2F5IHRvIGRldGVy bWluZSBpZiB0aGUgaG9zdCBoYXMgdW5yZXN0cmljdGVkIGd1ZXN0IA0KPiBzdXBwb3J0IChyZXF1 aXJlZCBmb3Igbm9uLWZyZWVic2Qgb3IgbXVsdGljb3JlIGZyZWVic2QgZ3Vlc3RzLCBhbmQgYXMg DQo+IHlvdSBzYXkgbWlzc2luZyBmcm9tIHNvbWUgZWFybHkgVlQteCBjYXBhYmxlIHByb2Nlc3Nv cnMpPw0KPg0KPiA+SSB3ZW50IGFuZCBsb29rZWQgYXQgd2h5IHZtLWJoeXZlIGlzIGdyb3ZlbGlu ZyBhcm91bmQgaW4NCj4gL3Zhci9ydW4vZG1lc2cuYm9vdCBhbmQgZm91bmQgdGhhdCBpdCBpcyBz aW1wbHkgdHJ5aW5nIHRvIGRldGVybWluZSBpZiANCj4gdGhlIGhvc3QgQ1BVIGlzIHZtbSBjYXBh YmxlLA0KPiA+c3BlY2lmaWNhbGx5Og0KPiA+dXRpbDo6Y2hlY2tfYmh5dmVfc3VwcG9ydCgpew0K PiA+Li4uDQo+DQo+ID5UaGVzZSBjaGVja3MgYXJlIGFscmVhZHkgYnVpbHQgaW50byB0aGUga2Vy bmVsLg0KPiA+VGhpcyBjYW4gYWxsIGdvIGluIHRoZSBiaXQgYnVja2V0LCBpZiB5b3UgdHJ5IHRv IHN0YXJ0IGEgVk0gb24gYW4NCj4gdW5zdXBwb3J0ZWQgc3lzdGVtIGFuIGVycm9yIGlzIHJldHVy bmVkLCByZWNvZGluZyB0aGlzIGluIHNoZWxsIGlzIA0KPiBqdXN0IHNldHRpbmcgeW91cnNlbGYg dXAgZm9yICJmdXR1cmUiIGJ1Z3MuDQo+DQo+IFRoZSBrZXJuZWwga25vd3Mgd2hhdCBmZWF0dXJl cyBhcmUgc3VwcG9ydGVkIGJ1dCBkb2VzIG5vdCBleHBvc2UgDQo+IHRoZXNlLCBzbyBhbGwgSSBj YW4gZG8gaXMgc2ltaWxhciB0byBsaWJ2aXJ0IGFuZCBydW4gYmh5dmUgd2l0aCANCj4gZGlmZmVy ZW50IG9wdGlvbnMgdG8gc2VlIHdoYXQgZXJyb3JzIHBvcCB1cC4NCj4gSSB0aGluayBJJ2xsIGp1 c3QgcmVtb3ZlIGFsbCBjaGVja2luZyBmb3Igbm93IGFuZCBsZXQgdXNlcnMgZGlzY292ZXIgDQo+ IHRoZSBpc3N1ZSBmb3IgdGhlbXNlbHZlcyBpZiBiaHl2ZSB3b24ndCBydW4uIEhvcGVmdWxseSB0 aGUgDQo+IHZteC5pbml0aWFsaXplZCAvDQo+IGNhcC4qIHN5c2N0bHMgd2lsbCBhdCBzb21lIHBv aW50IGJlY29tZSBhIGRlZmluZWQgd2F5IG9mIHNlZWluZyBpZiB2bW0gDQo+IGlzIHJlYWR5IC8g dGVzdGluZyBmb3Igdm1tIGZlYXR1cmVzLCBhcyBhcHBhcmVudGx5IHRoZXNlIHNlcnZlIG5vIA0K PiBwdXJwb3NlIGF0IHRoZSBtb21lbnQuDQo+DQo+IE1hdHQNCj4NCj4gX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gZnJlZWJzZC12aXJ0dWFsaXphdGlv bkBmcmVlYnNkLm9yZyBtYWlsaW5nIGxpc3QgDQo+IGh0dHBzOi8vbGlzdHMuZnJlZWJzZC5vcmcv bWFpbG1hbi9saXN0aW5mby9mcmVlYnNkLXZpcnR1YWxpemF0aW9uDQo+IFRvIHVuc3Vic2NyaWJl LCBzZW5kIGFueSBtYWlsIHRvICJmcmVlYnNkLXZpcnR1YWxpemF0aW9uLSANCj4gdW5zdWJzY3Jp YmVAZnJlZWJzZC5vcmciDQo+DQo+DQo+DQo+ID5UaGlzIGlzIHNvbWV0aGluZyB0aGF0IGlzIGlu IG15IGxvbmcgdG8tZG8gbGlzdCwgSSB3aWxsIHRyeSB0byBnZXQgDQo+ID5iYWNrDQo+IHRvIHRo aXMgaW4gY291cGxlIHdlZWtzLg0KPg0KPiA+SSB0aGluayB0aGUgd2F5IGhvdyB5b3UgYXJlIGRl YWxpbmcgd2l0aCBpdCBub3dhZGF5cyBpcyB0aGUgYmVzdCB3YXkgDQo+ID50bw0KPiB0cnkgdG8g ZGlzY292ZXIgdGhlIENQVSBjYXBhYmlsaXRpZXMsIGJldHRlciBpbiB0aGlzIHdheSB0aGFuIGxl dCB0aGUgDQo+IHVzZXJzIGJsaW5kLg0KPg0KPiA+QmVzdCwNCj4NCj4gVGhhbmtzIE1hcmNlbG8s DQo+DQo+DQo+DQo+IEl04oCZcyBub3QgZXhhY3RseSBjcml0aWNhbC4gSSBqdXN0IHNhdyBhbm90 aGVyIHByb2plY3QgdXNpbmcgDQo+IHZteC5pbml0aWFsaXplZCBhbmQgaXQgc2VlbWVkIGEgbXVj aCBjbGVhbmVyIHdheSB0byBkZXRlcm1pbmUgc3VwcG9ydCANCj4gYnkgc2VlaW5nIGlmIHZtbSBs b2FkcyByYXRoZXIgdGhhbiBwcm9iaW5nIGxvZyBmaWxlcy4gSSBhc2tlZCBoZXJlIA0KPiBleHBl Y3Rpbmcgc29tZW9uZSB0byBqdXN0IHNheSDigJx5ZXMgaWYgdGhhdOKAmXMgMSB0aGVuIHRoZSBo b3N0IHdpbGwgcnVuIA0KPiBndWVzdHPigJ0sIG9yIOKAnG5vIHlvdSBjYW7igJl0IHRlbGwgaWYg Z3Vlc3RzIHdpbGwgcnVuIGp1c3QgYnkgdGhhdCBzeXNjdGzigJ0uDQo+DQo+DQo+DQo+IE1hdHQN Cj4NCg0KPiBDb3VsZCB5b3Ugc2hhcmUgd2l0aCBtZSB3aGF0IHByb2plY3QgaXMgdXNpbmcgdm14 LmluaXRpYWxpemVkPw0KDQpodHRwczovL2dpdGh1Yi5jb20vYmlnZHJhZ29uc29mdC9idm0NCnNy Yy92bS5jIGxpbmUgMjc5MA0KDQoNCj4gQmVzdCwNCg0KLS0gDQo= From owner-freebsd-virtualization@freebsd.org Fri Aug 17 14:53:47 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8131E10714AD for ; Fri, 17 Aug 2018 14:53:47 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CB34A817D5; Fri, 17 Aug 2018 14:53:46 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w7HErim0059595; Fri, 17 Aug 2018 07:53:44 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w7HEriNK059594; Fri, 17 Aug 2018 07:53:44 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201808171453.w7HEriNK059594@pdx.rh.CN85.dnsmgr.net> Subject: Re: Checking bhyve supported features (sysctls) In-Reply-To: To: Matt Churchyard Date: Fri, 17 Aug 2018 07:53:44 -0700 (PDT) CC: Allan Jude , "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Aug 2018 14:53:47 -0000 > From: owner-freebsd-virtualization@freebsd.org On Behalf Of Rodney W. Grimes > Sent: 16 August 2018 18:31 > To: Allan Jude > Cc: Matt Churchyard ; freebsd-virtualization@freebsd.org > Subject: Re: Checking bhyve supported features (sysctls) > > > On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" wrote: > > >> > > >> Text manually wrapped to 80, any broken quoting is my fault - rwg > > >> > > >> > > Hello, > > >> > > > > >> > > I'm looking for better ways to check for bhyve support / > > >available > > >> > > features without trying to scan through dmesg output. > > >> > > > >> > >Yes, it would be very good to remove that, as it usually tries > > >> > >to grep a non-existent file /var/run/dmesg.boot that is not > > >> > >created until after vm_bhyve has been called from > > >/usr/local/etc/rc.d > > >> > >when you have things set to autostartup >in /etc/rc.conf > > >> > > > >> > > > >> > > > > >> > > I notice that the following 2 sysctl's appear to be set to 1 as > > >soon > > >> > > as the vmm module is loaded > > >> > > > > >> > > hw.vmm.vmx.initialized: 1 > > >> > > hw.vmm.vmx.cap.unrestricted_guest: 1 > > >> > > > > >> > > Will these be available on both Intel & AMD processors as a way > > >> > > to determine if the module has loaded successfully and can run > > >guests? > > >> > > > > >> > > I also see the below sysctl related to iommu. > > >> > > > > >> > > hw.vmm.iommu.initialized > > >> > > > > >> > > Again, will this be set to 1 as soon as the module is loaded if > > >> > > iommu is supported, or only when it is used? > > >> > > There also seems to be a vmm.amdvi.enable sysctl. > > >> > > Would both these need checking or is vmm.iommu enough to > > >> > > determine support on any processor. > > >> > > > >> > >Probalby the safest way for a shell script to decide if bhyve is > > >> > >up and running is to stat /dev/vmm, if that exists then the > > >modules > > >> > >have loaded and initialized and bhyve should be ready to process > > >guests. > > >> > > > >> > Hmm, I don't get /dev/vmm unless I actually have running guests. > > >> > > >> I'll investigate that, I was pretty sure that you should get this > > >> as soon as the vmm.ko module is finished initialzing, but you might > > >> be right in that it takes a first vm to cause its creation. > > >> Confirmed, /dev/vmm does not exist until the first vm is created. > > >> > > >> > > > >> > >sysctl's mentiond above would be a poor way to make this > > >determination. > > >> > > > >> > It would be nice if sysctls were better documented. > > >> > > >> Agreed. > > >> > > >> > If vmx.initialized is set once vmm has successfully loaded, I > > >> > can't > > >see a better way of checking for bhyve support (assuming it's not > > >Intel specific). This entry definitely exists and is set to 0 if you > > >load the module on a non-supported system, and set to 1 as soon as > > >vmm loads on my Intel test system. > > >> > > >> Given its undocumented status you would be relying on an > > >> undocumented feature that could change in either name or behavior, > > >> and that is not desirable. > > >> > > >> Let me see if I can come up with something else. > > > > > >I looked at the code for bhyvectl, bhyveload and byhve. They do not > > >actually try to decide if vmm is supported or not, they simply > > >process the error from a vm_create() or vm_open() call and exit with > > >an error code if they can not handle it (some of the code can handle > > >a vm_create failure if infact we are trying to create a vm that > > >already exists). > > > > > >If you want to maintain full compatibility a similiar stratergy may > > >be in order. > > > > > >Why is it that vm-bhyve specifically needs to know if the kernel has > > >vmm support or not? > > >Cant it just be written to handle the errors returned if the > > >supported functions do not exist? > > > > I think the question vm-bhyve wants to answer is: does the CPU have > > the required features to run a multicore VM. > > >Why does it need to know that? If it tries to start a multicore/thread VM and the system can not support it an error is returned and the bhyve command fails. > > >Actually determining that specific issue is non-trivial iirc as some vmm supported CPU's can only run a single VM with a single thread in that VM (early core cpu's). > > > > > These or similar sysctls do seem to be the correct way to communicate > > that support. > > >I do not believe any of those sysctl's communicate that your on a broken cpu or to what extent you can run vm's with multiple threads. > > So cap.unrestricted_guest from the vmm "capabilities" set of sysctls is not a valid way to determine if the host has unrestricted guest support (required for non-freebsd or multicore freebsd guests, and as you say missing from some early VT-x capable processors)? I overlooked that one, that could be used, and is probably fine to use, though it is an undocumented sysctl, there is one in tree consumer, bhyve(8), so this is unlikely to change in the future. root {1245}# find . -type f | grep -v .svn | xargs grep unrestricted_guest ./sys/amd64/vmm/intel/vmx.c:static int cap_unrestricted_guest; ./sys/amd64/vmm/intel/vmx.c:SYSCTL_INT(_hw_vmm_vmx_cap, OID_AUTO, unrestricted_guest, CTLFLAG_RD, ./sys/amd64/vmm/intel/vmx.c: &cap_unrestricted_guest, 0, "Unrestricted guests"); ./sys/amd64/vmm/intel/vmx.c: cap_unrestricted_guest = (vmx_set_ctlreg(MSR_VMX_PROCBASED_CTLS2, ./sys/amd64/vmm/intel/vmx.c: if (cap_unrestricted_guest) ./sys/amd64/vmm/intel/vmx.c: if (cap_unrestricted_guest) ./sys/amd64/vmm/intel/vmx.c: if (cap_unrestricted_guest) { ./lib/libvmmapi/vmmapi.c: { "unrestricted_guest", VM_CAP_UNRESTRICTED_GUEST }, :root {1246}# find . -type f | grep -v .svn | xargs grep VM_CAP_UNRESTRICTED_GUEST ./usr.sbin/bhyve/spinup_ap.c: error = vm_set_capability(ctx, newcpu, VM_CAP_UNRESTRICTED_GUEST, 1); ./usr.sbin/bhyve/bhyverun.c: error = vm_get_capability(ctx, BSP, VM_CAP_UNRESTRICTED_GUEST, &tmp); ./usr.sbin/bhyve/bhyverun.c: if (vm_set_capability(ctx, BSP, VM_CAP_UNRESTRICTED_GUEST, 1)) { ./sys/amd64/vmm/intel/vmx.c: case VM_CAP_UNRESTRICTED_GUEST: ./sys/amd64/vmm/intel/vmx.c: case VM_CAP_UNRESTRICTED_GUEST: ./sys/amd64/vmm/amd/svm.c: case VM_CAP_UNRESTRICTED_GUEST: ./sys/amd64/vmm/amd/svm.c: case VM_CAP_UNRESTRICTED_GUEST: ./sys/amd64/include/vmm.h: VM_CAP_UNRESTRICTED_GUEST, ./lib/libvmmapi/vmmapi_freebsd.c: error = vm_get_capability(vmctx, vcpu, VM_CAP_UNRESTRICTED_GUEST, &tmp); ./lib/libvmmapi/vmmapi_freebsd.c: error = vm_set_capability(vmctx, vcpu, VM_CAP_UNRESTRICTED_GUEST, 1); ./lib/libvmmapi/vmmapi.c: { "unrestricted_guest", VM_CAP_UNRESTRICTED_GUEST }, FYI, the prefered and by design way to interact with vmm.ko is via the vmmapi. > >I went and looked at why vm-bhyve is groveling around in /var/run/dmesg.boot and found that it is simply trying to determine if the host CPU is vmm capable, > >specifically: > >util::check_bhyve_support(){ > >... > > >These checks are already built into the kernel. > >This can all go in the bit bucket, if you try to start a VM on an unsupported system an error is returned, recoding this in shell is just setting yourself up for "future" bugs. > > The kernel knows what features are supported but does not expose these, so all I can do is similar to libvirt and run bhyve with different options to see what errors pop up. > I think I'll just remove all checking for now and let users discover the issue for themselves if bhyve won't run. Hopefully the vmx.initialized / cap.* sysctls will at some point become a defined way of seeing if vmm is ready / testing for vmm features, as apparently these serve no purpose at the moment. I would, again, strongly encorage removal of the grep /var/run/dmesg.boot, for me it causes boot time error messages. -- Rod Grimes rgrimes@freebsd.org