From owner-freebsd-virtualization@freebsd.org Mon Apr 27 09:34:30 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F30402B0D52 for ; Mon, 27 Apr 2020 09:34:30 +0000 (UTC) (envelope-from info@os-plus.org) Received: from mailserv.regfish.com (mailserv.regfish.com [52.57.101.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailserv.regfish.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 499fkG23Nnz3FFw for ; Mon, 27 Apr 2020 09:34:30 +0000 (UTC) (envelope-from info@os-plus.org) Received: by mailserv.regfish.com (Postfix, from userid 706) id 3450961397; Mon, 27 Apr 2020 09:34:29 +0000 (UTC) X-Original-To: freebsd-virtualization@freebsd.org Received: from mbp2016.i.os-plus.de (p2E5E8F7A.dip0.t-ipconnect.de [46.94.143.122]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: 40815-0001) by mailserv.regfish.com (Postfix) with ESMTPSA id F1CC761357; Mon, 27 Apr 2020 09:34:28 +0000 (UTC) Subject: Re: bhyve with AMD Ryzen 3900X and MSI MEG ACE X570 To: Aryeh Friedman , Anish Cc: "freebsd-virtualization@freebsd.org" References: From: Christian Jeannot Message-ID: <91b223a6-62d7-023f-d268-d6a2e7ded2e8@os-plus.org> Date: Mon, 27 Apr 2020 11:34:27 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 499fkG23Nnz3FFw X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of info@os-plus.org has no SPF policy when checking 52.57.101.90) smtp.mailfrom=info@os-plus.org X-Spamd-Result: default: False [-1.10 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; IP_SCORE(-0.10)[ipnet: 52.57.0.0/16(0.53), asn: 16509(-0.96), country: US(-0.05)]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[os-plus.org]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-0.97)[-0.969,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[90.101.57.52.list.dnswl.org : 127.0.10.0]; NEURAL_HAM_MEDIUM(-0.94)[-0.939,0]; R_SPF_NA(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16509, ipnet:52.57.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Apr 2020 09:34:31 -0000 Hello Aryeh, Hello @All, I ´was able to switch the Expert mode in the Overclocking/CPU Features section from the BIOS. After this I was abel to see the real settings from IOMMU and SVM. IOMMU is on Auto SVM is disabled. I have enabled SVM, saved the settings and booted the system. vmm is loaded with no error message. I even do not see any entry from vmm in dmesg. When I run kldstat I can see that vmm is loaded. Executing cpucontrol -m 0xc0010114 /dev/cpuctl0 or cpucontrol -m 0xc0010114 /dev/cpuctl1 gives now MSR 0xc0010114: 0x00000000 0x00000008 From my point this looks good and should be solved now so I can continue. I say many thanks to this list and to all people who assisted. Best regards --Christian Am 27.04.20 um 09:21 schrieb Aryeh Friedman: > > > On Mon, Apr 27, 2020 at 2:58 AM Anish > wrote: > > Hi Christian, > > This looks like BIOS issue, BIOS must not set Bit4/SVMDIS in VM_CR MSR > (0xC001_0114) if SVM is enabled. Bhyve checks for this bit before > enabling SVM. You can read it using the following steps: > $kldload cpuctl > > > $ cpucontrol -m 0xc0010114 /dev/cpuctlX <- Read VM_CR MSR > MSR 0xc0010114: 0x00000000 0x00000008 > > X is any CPU core number. > > > Again it is likely not a BIOS issues since the OP and me have nearly > identical setups (same MB manufacturer/same product line for both the > CPU and MB as well the same BIOS in most likely hood) and I get the > following (which works): > > root@neomarx:~ # kldload cpuctl > root@neomarx:~ # cpucontrol -m 0xc0010114 /dev/cpuctl0 > MSR 0xc0010114: 0x00000000 0x00000008 > > My BIOS: 7C02v1D 2019-11-11 > OP BIOS: 7C35v18 2020-01-16 > > So if anything his should work and mine not work. > > @OP do you have IOMMU support on or to auto (that is the one possible > setting you didn't state in your initial post -- mine is auto) > > > On my AMD/Ryzen system, bit4 is clear and I can load bhyve. I have > attached > a screenshot from AMD SDM Vol2, which you may find useful. > > [image: Screen Shot 2020-04-26 at 12.01.14 PM.png] > > -Anish > > > > On Sun, Apr 26, 2020 at 6:27 AM Christian Jeannot > wrote: > > > I have checked MSI support page. The latest BIOS version is already > > installed. > > > > Regards > > > > --Christian > > > > Am 26.04.20 um 15:11 schrieb M. Eisenhardt: > > > I would suggest to update your bios. > > > Kind regards > > > Michael > > > > > >> Am 26.04.2020 um 14:00 schrieb > > freebsd-virtualization-request@freebsd.org > : > > >> > > >> Send freebsd-virtualization mailing list submissions to > > >> freebsd-virtualization@freebsd.org > > > >> > > >> To subscribe or unsubscribe via the World Wide Web, visit > > >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > >> or, via email, send a message with subject or body 'help' to > > >> freebsd-virtualization-request@freebsd.org > > > >> > > >> You can reach the person managing the list at > > >> freebsd-virtualization-owner@freebsd.org > > > >> > > >> When replying, please edit your Subject line so it is more > specific > > >> than "Re: Contents of freebsd-virtualization digest..." > > >> > > >> > > >> Today's Topics: > > >> > > >>    1. bhyve with AMD Ryzen 3900X and MSI MEG ACE X570 > > >>       (Christian Jeannot) > > >>    2. Re: bhyve with AMD Ryzen 3900X and MSI MEG ACE X570 > > >>       (Aryeh Friedman) > > >>    3. Re: bhyve with AMD Ryzen 3900X and MSI MEG ACE X570 > > >>       (Christian Jeannot) > > >> > > >> > > >> > ---------------------------------------------------------------------- > > >> > > >> Message: 1 > > >> Date: Sun, 26 Apr 2020 10:57:50 +0200 > > >> From: Christian Jeannot > > > >> To: freebsd-virtualization@freebsd.org > > > >> Subject: bhyve with AMD Ryzen 3900X and MSI MEG ACE X570 > > >> Message-ID: <2535c7f2-ef59-467e-779b-6c9c6db82005@os-plus.org > > > > >> Content-Type: text/plain; charset=utf-8; format=flowed > > >> > > >> Hi Community, > > >> > > >> I am testing bhyve from FreeBSD 12.1 RELEASE. > > >> > > >> My environment: > > >> - CPU: AMD Ryze 3900X > > >> - Mainboard: MSI MEG ACE X570, Default settings, latest BIOS > 7C35v18 > > >> - RAM: G.Skill RipJaws V DIMM Kit 64GB, DDR4-3200, CL16-18-18-38 > > >> - GPU: Sapphire Nitro RX 5700XT > > >> > > >> I checked the settings in the BIOS. Virtualization options are > enabled. > > >> SVM is enabled. > > >> > > >> I follow the instruction from the FreeBSD handbook ?21.7. > FreeBSD as a > > >> Host with bhyve?. > > >> > > >> When I run > > >>      kldload vmm > > >> I got an output that SVM is disabled by BIOS. When I check the > BIOS it > > >> is enabled. > > >> > > >> When I try to start the VM with the example script I got the > output > > >>      vm_create device not configured > > >> > > >> Did I miss something to configure? > > >> > > >> Does anyone has a Ryzen CPU and a actual MSI mainboard > successfully run > > >> bhyve? > > >> > > >> Best regards > > >> > > >> ?Christian > > >> -- > > >> -- > > >> christian jeannot > > >> vogelmauer 17 > > >> 86152 augsburg > > >> +49 821 81552861 > > >> info@os-plus.org > > >> > > >> > > >> ------------------------------ > > >> > > >> Message: 2 > > >> Date: Sun, 26 Apr 2020 05:32:36 -0400 > > >> From: Aryeh Friedman > > > >> To: Christian Jeannot > > > >> Cc: "freebsd-virtualization@freebsd.org > " > > >>      > > > >> Subject: Re: bhyve with AMD Ryzen 3900X and MSI MEG ACE X570 > > >> Message-ID: > > >> >   > > > > > >> Content-Type: text/plain; charset="UTF-8" > > >> > > >>> On Sun, Apr 26, 2020 at 4:57 AM Christian Jeannot > > > > wrote: > > >>> > > >>> Hi Community, > > >>> > > >>> I am testing bhyve from FreeBSD 12.1 RELEASE. > > >>> > > >>> My environment: > > >>> - CPU: AMD Ryze 3900X > > >>> - Mainboard: MSI MEG ACE X570, Default settings, latest BIOS > 7C35v18 > > >>> - RAM: G.Skill RipJaws V DIMM Kit 64GB, DDR4-3200, CL16-18-18-38 > > >>> - GPU: Sapphire Nitro RX 5700XT > > >>> > > >>> I checked the settings in the BIOS. Virtualization options > are enabled. > > >>> SVM is enabled. > > >>> > > >>> I follow the instruction from the FreeBSD handbook ?21.7. > FreeBSD as a > > >>> Host with bhyve?. > > >>> > > >>> When I run > > >>>      kldload vmm > > >>> I got an output that SVM is disabled by BIOS. When I check > the BIOS it > > >>> is enabled. > > >>> > > >> > > >> What happens if you load it from /boot/load.conf > > >> > > >> Also you shouldn't have to turn on kvm (I do have it turned on > but it > > >> should not be required). > > >> > > >> Bhyve has a upper limit of 16 vm's that it can run at once but > I don't > > >> think this is based on core count of the processor. > > >> > > >> > > >>> When I try to start the VM with the example script I got the > output > > >>>      vm_create device not configured > > >>> > > >>> Did I miss something to configure? > > >>> > > >>> Does anyone has a Ryzen CPU and a actual MSI mainboard > successfully run > > >>> bhyve? > > >>> > > >> > > >> I have two Ryzen 5 2600X/MSI machines as far I can tell > looking at the > > >> specs (and what I remember when I was deciding to get a 3900X > or 2600X) > > >> there is no practical difference, except for core count, from > the point > > of > > >> view of virtualization/bhyve.  One machine is a MSI B450 > Tomahawk and > > the > > >> other is MSI B450 Gaming Pro.   Both machines run bhyve with > out any > > >> problem (one of them is my main home office server with 3 vm's > on it > > >> [windows and 2 FreeBSD]) and the other is my primary desktop > machine and > > >> used for R&D on PetiteCloud (which is a web interface to bhyve and > > qemu). > > >> > > >> Best regards > > >>> > > >>> ?Christian > > >>> -- > > >>> -- > > >>> christian jeannot > > >>> vogelmauer 17 > > >>> 86152 augsburg > > >>> +49 821 81552861 > > >>> info@os-plus.org > > >>> _______________________________________________ > > >>> freebsd-virtualization@freebsd.org > mailing list > > >>> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > >>> To unsubscribe, send any mail to " > > >>> freebsd-virtualization-unsubscribe@freebsd.org > " > > >>> > > >> > > >> > > >> -- > > >> Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org > > >> > > >> > > >> ------------------------------ > > >> > > >> Message: 3 > > >> Date: Sun, 26 Apr 2020 11:56:39 +0200 > > >> From: Christian Jeannot > > > >> To: Aryeh Friedman > > > >> Cc: freebsd-virtualization@freebsd.org > > > >> Subject: Re: bhyve with AMD Ryzen 3900X and MSI MEG ACE X570 > > >> Message-ID: > > > >> Content-Type: text/plain; charset=utf-8; format=flowed > > >> > > >> > > >> > > >>> Am 26.04.20 um 11:32 schrieb Aryeh Friedman: > > >>> > > >>> > > >>> On Sun, Apr 26, 2020 at 4:57 AM Christian Jeannot > > > >>> >> wrote: > > >>> > > >>>     Hi Community, > > >>> > > >>>     I am testing bhyve from FreeBSD 12.1 RELEASE. > > >>> > > >>>     My environment: > > >>>     - CPU: AMD Ryze 3900X > > >>>     - Mainboard: MSI MEG ACE X570, Default settings, latest BIOS > > 7C35v18 > > >>>     - RAM: G.Skill RipJaws V DIMM Kit 64GB, DDR4-3200, > CL16-18-18-38 > > >>>     - GPU: Sapphire Nitro RX 5700XT > > >>> > > >>>     I checked the settings in the BIOS. Virtualization > options are > > enabled. > > >>>     SVM is enabled. > > >>> > > >>>     I follow the instruction from the FreeBSD handbook ?21.7. > FreeBSD > > as a > > >>>     Host with bhyve?. > > >>> > > >>>     When I run > > >>>      ? ? ?kldload vmm > > >>>     I got an output that SVM is disabled by BIOS. When I > check the > > BIOS it > > >>>     is enabled. > > >>> > > >>> > > >>> What happens if you load it from /boot/load.conf > > >> > > >> When I add > > >>     vmm_load="YES" > > >> in /boot/loader.conf I can see the following with dmesg after > a reboot > > >> > > >> SVM: disabled by BIOS. > > >> module_register_init: MOD_LOAD (vmm, 0xffffffff8264dd20, 0) > error 6 > > >> > > >>> > > >>> Also you shouldn't have to turn on kvm (I do have it turned > on but it > > >>> should not be required). > > >>> > > >>> Bhyve has a upper limit of 16 vm's that it can run at once > but I don't > > >>> think this is based on core count of the processor. > > >>> > > >>> > > >>>     When I try to start the VM with the example script I got > the output > > >>>      ? ? ?vm_create device not configured > > >>> > > >>>     Did I miss something to configure? > > >>> > > >>>     Does anyone has a Ryzen CPU and a actual MSI mainboard > > successfully run > > >>>     bhyve? > > >>> > > >>> > > >>> I have two Ryzen 5 2600X/MSI machines as far I can tell > looking at the > > >>> specs (and what I remember when I was deciding to get a 3900X > or 2600X) > > >>> there is no practical difference, except for core count, from > the point > > >>> of view of virtualization/bhyve.? One machine is a MSI B450 > Tomahawk > > and > > >>> the other is MSI B450 Gaming Pro.?? Both machines run bhyve > with out > > any > > >>> problem (one of them is my main home office server with 3 > vm's on it > > >>> [windows and 2 FreeBSD]) and the other is my primary desktop > machine > > and > > >>> used for R&D on PetiteCloud (which is a web interface to > bhyve and > > qemu). > > >> > > >> Thank you. Good to know that bhye is working with AMD CPUs, > Mainboards > > >> and chipsets. > > >> > > >> --Christian > > >> > > >> -- > > >> -- > > >> christian jeannot > > >> vogelmauer 17 > > >> 86152 augsburg > > >> +49 821 81552861 > > >> info@os-plus.org > > >> > > >> > > >> ------------------------------ > > >> > > >> Subject: Digest Footer > > >> > > >> _______________________________________________ > > >> freebsd-virtualization@freebsd.org > mailing list > > >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > >> To unsubscribe, send any mail to " > > freebsd-virtualization-unsubscribe@freebsd.org > " > > >> > > >> > > >> ------------------------------ > > >> > > >> End of freebsd-virtualization Digest, Vol 490, Issue 5 > > >> ****************************************************** > > > > > > _______________________________________________ > > > freebsd-virtualization@freebsd.org > mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > > To unsubscribe, send any mail to " > > freebsd-virtualization-unsubscribe@freebsd.org > " > > > > > > > -- > > -- > > christian jeannot > > vogelmauer 17 > > 86152 augsburg > > +49 821 81552861 > > info@os-plus.org > > _______________________________________________ > > freebsd-virtualization@freebsd.org > mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to " > > freebsd-virtualization-unsubscribe@freebsd.org > " > > > _______________________________________________ > freebsd-virtualization@freebsd.org > mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org > " > > > > -- > Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org -- -- christian jeannot vogelmauer 17 86152 augsburg +49 821 81552861 info@os-plus.org