Date: Wed, 1 Dec 2021 17:52:19 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> To: bsdlists@jld3.net, jbo@insane.engineer Cc: freebsd-virtualization@freebsd.org Subject: Re: bhyve vCPU limit Message-ID: <30e4454c-414a-833f-3829-586a450e7205@quip.cz> In-Reply-To: <4E8A7FD3-B01E-4ADE-A290-360F3B04AC0F@jld3.net> References: <PigdsByvTXmOLg46mIkWprP1GQQPuxEiHn55uKNYuSBIzBFFe-CVGYdJ2FuzYSd5OebhMlSpRGMIisaN07yzjSSaWz8JQ7LeXDeINIZg_D8=@insane.engineer> <4E8A7FD3-B01E-4ADE-A290-360F3B04AC0F@jld3.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 01/12/2021 17:17, John Doherty via freebsd-virtualization wrote: > That limitation appears to still exist in FreeBSD 13.0-RELEASE: > > [root@grit] # freebsd-version -k ; grep 'VM_MAXCPU' > /usr/src/sys/amd64/include/vmm.h > 13.0-RELEASE > #define VM_MAXCPU 16 /* maximum virtual cpus */ > > I ran into this in May 2021 and with some help from folks on this list > was able to increase it. The simplest (if not minimalist) way to do that > is: > > 1. edit /usr/src/sys/amd64/include/vmm.h to increase that value: I used 48 > 2. make buildworld > 3. make installworld > > The increased value has been working fine for me since I did that. I run > a couple of VMs with 24 vCPUs each and several others with smaller > numbers all the time and have run others with as many as 48 temporarily. > No problems that I have seen. I am sorry for hijacking this thread but your information is very interesting. I was playing with VMs in VirtualBox and Bhyve and compared performance with increasing vCPU count. The more cores VM get the slower was even a simple single threaded task like loading PF rules from /etc/pf.conf. It was tested on FreeBSD 11.4 and 12.2, I tested ULE and 4BSD schedulers. Maybe it was somewhat HW related but it always shows VMs with more than 2 v CPUs significantly slower. VMs with 6+ vCPU was almost unusable (loading of PF ruleset takes about 8 seconds instead of fraction on single vCPU VM). Do you have any special tunning to have so large number of vCPU without this penalty? Kind regards Miroslav Lachman
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?30e4454c-414a-833f-3829-586a450e7205>