Date: Wed, 1 Dec 2021 23:15:58 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> To: Oleg Ginzburg <olevole@olevole.ru>, rgrimes@freebsd.org Cc: bsdlists@jld3.net, jbo@insane.engineer, freebsd-virtualization@freebsd.org Subject: Re: bhyve vCPU limit Message-ID: <8a27cc73-4c68-a643-5809-d6ec87201eb2@quip.cz> In-Reply-To: <CAMsb%2BmbY4ZbtFvBHd15naGT-BdXAVKg6BAOTpnBNg7hNcg2Czw@mail.gmail.com> References: <PigdsByvTXmOLg46mIkWprP1GQQPuxEiHn55uKNYuSBIzBFFe-CVGYdJ2FuzYSd5OebhMlSpRGMIisaN07yzjSSaWz8JQ7LeXDeINIZg_D8=@insane.engineer> <4E8A7FD3-B01E-4ADE-A290-360F3B04AC0F@jld3.net> <30e4454c-414a-833f-3829-586a450e7205@quip.cz> <CAMsb%2BmbY4ZbtFvBHd15naGT-BdXAVKg6BAOTpnBNg7hNcg2Czw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 01/12/2021 19:13, Oleg Ginzburg wrote: > Hi > > On Wed, Dec 1, 2021 at 7:52 PM Miroslav Lachman <000.fbsd@quip.cz> wrote: >> I was playing with VMs in VirtualBox and Bhyve and compared >> performance with increasing vCPU count. The more cores VM get the slower > > How many physical cores you have on the host? This is a characteristic > behavior when overcoming. > But without overcommit I observe productivity growth: I tested it on 2 machines. One had 6 cores 12 threads, the second 4 cores 8 threads. Even if the host machine was almost idle, only one VM was running the slowness with more than 1 vCPU was so significant. Let's say 0.01 sec for 1 vCPU, 0.3 s for 2 vCPU, 0.9 s for 4 vCPU. (on machine with real 4 cores / 8 threads). With overcommit of vCPUs it was like 3 seconds with 6 vCPU and 8 seconds for 8 vCPU. Still the same task on the same idle machine. On the other hand I can start 5 VMs with 2 vCPUs each and "everything seems normal" even if the total count of vCPUs are 10 on machine with 4 physical / 8 threads. In fact I tested with total count of 15 vCPUs distributed between 5+ VMs but the performance problem was always visible on VM with more than 2 vCPU. > 1) > Single vCore benchmark via cpuminer: https://pastebin.com/mg46RvDT > TOP from host: https://pastebin.com/LvbBiyFz > > > 2) > Sixteen vCore benchmark via cpuminer: https://pastebin.com/7FQjaVM8 > TOP from host: https://pastebin.com/dLpreq9D > > I observe almost linear growth. > > In addition, the loss of performance can be associated with NUMA > (required cpuset/cpuset_setaffinity > and control of memory capacity) > > As far as I remember, it is not enough to simply change one parameter > in .h file: https://bhyvecon.org/bhyveconOttawa2019-Rodney.pdf Thank you for your data! Kind regards Miroslav Lachman
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8a27cc73-4c68-a643-5809-d6ec87201eb2>