Date: Wed, 1 Dec 2021 17:52:19 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> To: bsdlists@jld3.net, jbo@insane.engineer Cc: freebsd-virtualization@freebsd.org Subject: Re: bhyve vCPU limit Message-ID: <30e4454c-414a-833f-3829-586a450e7205@quip.cz> In-Reply-To: <4E8A7FD3-B01E-4ADE-A290-360F3B04AC0F@jld3.net> References: <PigdsByvTXmOLg46mIkWprP1GQQPuxEiHn55uKNYuSBIzBFFe-CVGYdJ2FuzYSd5OebhMlSpRGMIisaN07yzjSSaWz8JQ7LeXDeINIZg_D8=@insane.engineer> <4E8A7FD3-B01E-4ADE-A290-360F3B04AC0F@jld3.net>
index | next in thread | previous in thread | raw e-mail
On 01/12/2021 17:17, John Doherty via freebsd-virtualization wrote: > That limitation appears to still exist in FreeBSD 13.0-RELEASE: > > [root@grit] # freebsd-version -k ; grep 'VM_MAXCPU' > /usr/src/sys/amd64/include/vmm.h > 13.0-RELEASE > #define VM_MAXCPU 16 /* maximum virtual cpus */ > > I ran into this in May 2021 and with some help from folks on this list > was able to increase it. The simplest (if not minimalist) way to do that > is: > > 1. edit /usr/src/sys/amd64/include/vmm.h to increase that value: I used 48 > 2. make buildworld > 3. make installworld > > The increased value has been working fine for me since I did that. I run > a couple of VMs with 24 vCPUs each and several others with smaller > numbers all the time and have run others with as many as 48 temporarily. > No problems that I have seen. I am sorry for hijacking this thread but your information is very interesting. I was playing with VMs in VirtualBox and Bhyve and compared performance with increasing vCPU count. The more cores VM get the slower was even a simple single threaded task like loading PF rules from /etc/pf.conf. It was tested on FreeBSD 11.4 and 12.2, I tested ULE and 4BSD schedulers. Maybe it was somewhat HW related but it always shows VMs with more than 2 v CPUs significantly slower. VMs with 6+ vCPU was almost unusable (loading of PF ruleset takes about 8 seconds instead of fraction on single vCPU VM). Do you have any special tunning to have so large number of vCPU without this penalty? Kind regards Miroslav Lachmanhome | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?30e4454c-414a-833f-3829-586a450e7205>