From nobody Sun Mar 19 18:12:11 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PfmDP3WCdz40VHk for ; Sun, 19 Mar 2023 18:12:21 +0000 (UTC) (envelope-from SRS0=3sUP=7L=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PfmDN2xnLz3F9Y for ; Sun, 19 Mar 2023 18:12:20 +0000 (UTC) (envelope-from SRS0=3sUP=7L=quip.cz=000.fbsd@elsa.codelab.cz) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of "SRS0=3sUP=7L=quip.cz=000.fbsd@elsa.codelab.cz" has no SPF policy when checking 94.124.105.4) smtp.mailfrom="SRS0=3sUP=7L=quip.cz=000.fbsd@elsa.codelab.cz"; dmarc=none Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 88186D788C; Sun, 19 Mar 2023 19:12:12 +0100 (CET) Received: from [192.168.145.50] (ip-89-177-27-225.bb.vodafone.cz [89.177.27.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id D2BDCD7892; Sun, 19 Mar 2023 19:12:11 +0100 (CET) Message-ID: <343e827b-a1dc-b69b-9dcc-c6675deac070@quip.cz> Date: Sun, 19 Mar 2023 19:12:11 +0100 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Subject: Re: Bhyve process consumes way too much CPU To: =?UTF-8?Q?Julie_Koubov=c3=a1?= , freebsd-virtualization@freebsd.org References: Content-Language: cs-Cestina, en-US From: Miroslav Lachman <000.fbsd@quip.cz> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-1.76 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; AUTH_NA(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.96)[-0.958]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=3sUP=7L=quip.cz=000.fbsd@elsa.codelab.cz]; MIME_GOOD(-0.10)[text/plain]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; RCVD_TLS_LAST(0.00)[]; R_SPF_NA(0.00)[no SPF record]; R_DKIM_NA(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[quip.cz]; TO_DN_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=3sUP=7L=quip.cz=000.fbsd@elsa.codelab.cz] X-Rspamd-Queue-Id: 4PfmDN2xnLz3F9Y X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N On 18/03/2023 14:59, Julie Koubová wrote: > Hey everyone, > > I'm running Linux (Home Assistant OS) in Bhyve on FreeBSD 13.1. I use > PCI passthrough to allow the VM to access a USB card with a couple of > radio dongles. The host machine is an Intel Core i3 13100 with 64 GB of > RAM. The CPU has 4 physical cores (8 hyper-threaded). The virtual > machine is assigned four cores. > > The host load averages are 0.39 0.39 0.40 right now, which seems way too > much. The same workload was previously handled by a Raspberry Pi 4, and > the CPU usage there was under 10% when not doing anything special. > Inside the guest OS, the CPU usage is reported around 5%, which seems > reasonable. > > What's wrong? How can I start debugging this issue? I use ZFS on the > host, vm-bhyve to manage the virtual machines, and I don't have a swap > partition. I had similar problem few years ago. Never solved. Exhibited on bhyve and VirtualBox too. The problem was "the more vCPU for VM, the slower VM". Can you try to set just 1 vCPU to your VM? In my case, VM with 1 vCPU was fast, almost no overhead, 2 v CPUs slightly slower but 4 or more was slow as hell. I would also recommend not to overprovision real CPU core count to vCPU and not use multi/hyper threading cores as real cores. With your CPU, use only 4 cores to assign to all your VMs (4 VMs with 1 vCPU each, or 2 VMs with 2 vCPU each, 1 VM with 4 vCPU) Miroslav Lachman From nobody Sun Mar 19 21:00:42 2023 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pfqyg21S9z3yghP for ; Sun, 19 Mar 2023 21:00:43 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pfqyf6pD6z3r2L for ; Sun, 19 Mar 2023 21:00:42 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1679259643; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=HEnC7EaKOm3I2hZu1Y31G5RvhX59R/no/D7qAtCx6xE=; b=OcshtIGlMIGER4rMy8hrf9c2l/4xgIodJagyVqyNdBSIDv9nc6+RhgsXIS++fwqmw1CGa9 2oWIuqMnvbLRNapWZGpXUj7Lf5hX55VVnNl7IBVY0lu023YQawqXliczXoR+83bEDMQjYD Owjn5XCcWWue2VrI4UpEhE0nGpt9bHxl8bsTmrX8Ll/eVqAf7CiEMPg+WmLs7ldkvM5jRu bUY0llqh4t9AYLDG+7aXhT/CpZPFqb+TcbDmid5C0SY4YAamBT7Uhu8z1DF2nzCG2hcv6B 9mC68vSue2VBWiL82K+YzLkWRXokbwphk0u70HHCJ4nUwp4QNEqCvCpDqq2uqA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1679259643; a=rsa-sha256; cv=none; b=P8k8ly5rt9+SGtb3+eol5cMW/6mISQQ+pcBOVcLbRGdJjuFqnTihg31jGgjqiqWkcWCVCr cQxmjuMcbCDHR/Nrjwu/+Xag+AgYP7oHDBh1yd/3bMqhMEe8aliBDwjjNfQq15WLJclOAc Nr0Xcm45DHkeDAQGSVXoPDCmHTiSikorP/zm63KP1zCCmb0Vtv21sN7YDolxfEIAhH0/+L 4FksXXSJU4Hnx9Dw7lgCpdQf8ldgRzerTWUTcx4Lj+bRFtiLHXtCfXYGzqiLtGQZuvXCVu UP4yuPOtHKodn95tP//Kpn4TIcJmeu2F2GfLB3MmrTovucfwHopjIPobbj0TIQ== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Pfqyf5gnSz13d6 for ; Sun, 19 Mar 2023 21:00:42 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 32JL0gGd045856 for ; Sun, 19 Mar 2023 21:00:42 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 32JL0gUQ045855 for virtualization@FreeBSD.org; Sun, 19 Mar 2023 21:00:42 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202303192100.32JL0gUQ045855@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: virtualization@FreeBSD.org Subject: Problem reports for virtualization@FreeBSD.org that need special attention Date: Sun, 19 Mar 2023 21:00:42 +0000 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="16792596425.70D5C9f.41353" Content-Transfer-Encoding: 7bit X-ThisMailContainsUnwantedMimeParts: N --16792596425.70D5C9f.41353 Date: Sun, 19 Mar 2023 21:00:42 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- In Progress | 247208 | mpt(4): VMWare virtualized LSI controller panics New | 240945 | [hyper-v] [netvsc] hn network driver incorrectly Open | 244838 | "bectl activate -t" does not honor the -t flag in 3 problems total for which you should take action. --16792596425.70D5C9f.41353 Date: Sun, 19 Mar 2023 21:00:42 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" 
The following is a listing of current problems submitted by FreeBSD users,
which need special attention. These represent problem reports covering
all versions including experimental development code and obsolete releases.

Status      |    Bug Id | Description
------------+-----------+---------------------------------------------------
In Progress |    247208 | mpt(4): VMWare virtualized LSI controller panics 
New         |    240945 | [hyper-v] [netvsc] hn network driver incorrectly 
Open        |    244838 | "bectl activate -t" does not honor the -t flag in

3 problems total for which you should take action.
--16792596425.70D5C9f.41353-- From nobody Mon Mar 20 15:11:55 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PgJBV4wWhz3yrK0 for ; Mon, 20 Mar 2023 15:12:34 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PgJBV4Tz6z3qPY for ; Mon, 20 Mar 2023 15:12:34 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x52c.google.com with SMTP id o12so48055709edb.9 for ; Mon, 20 Mar 2023 08:12:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679325153; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=N2JEa6LhfrUkQ6EkuxRX058tUTlieW4wKZl0n2ylo84=; b=lBmfAQvPKFVQ6rpE+zrAOdlBJKpC8LGayPTUbkV+XhRpvaVadRxxE8GRsY9fAtfRNd Icpf2+6v3LYdJ1kS54gT7qk6doezZPy9Zgr629liK8x89u3cVGS7SVfe8goMJ8nnVihp r0iTnw04XftGK+D6TBvwGQ8bGOdubBRAUxre3lau5fxOWWWGBEBCN1dPMT4jSfyG3T3A CptCsTezxXWb9ZUc2tEFhLFKb6belatNJTOlAfP/LFi+w7KaKB7nwbwVvQcmNlp95gk9 HgNFovCjFp2a1x+Cf+fRKMtGtcmXrOjpKdFh00h3N6H4/+FmoKBHOMPdgPH+FFS13nzt sUlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679325153; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=N2JEa6LhfrUkQ6EkuxRX058tUTlieW4wKZl0n2ylo84=; b=oPG2+irfzHFfO4YqNFqOjw0MQ/e7gcdEYQhgt3kWt+e/8U+pdgxD+0s4Eydbvx2FuG GswT3U6b8K9ZHnKwPJy2NpGewN27uUOQ0TpqXfCGlDA6N1vZzf9FalAmwJbvBST+dkEk xDmsNOABuOKqrtQCiObYY3ku9mmKC1RM+n4hz5l5TgYkqs1ZyxnYeGdGIJ+nznRgfbkF 9DJvJeYSg/ui6Gn9/ymuCzmmI9ZpMA4UOiA5V1hGjNvscJCEnZlB+ng/4zmz9+CoF/Z1 QlU15RkPNqc2fvFhrX+VcSGe6MET5vtUsbHbWnHg1474rmFWy+aXMSE8dS6Qc8T8gbcH kp6A== X-Gm-Message-State: AO0yUKWTdh8uQCvsyD65V2Jk5E7RkykSiBuQp3i3qX9208qmLLFowM2o EV26WaFYLZtiJjcnj2k2byC+ZVdkjZYjaH1y6/V2I5GbTo4= X-Google-Smtp-Source: AK7set+RD2rKaPDExjgwEfjOgY8m4vN59XqpZFumErbQepZgbzd1Up44lbZhxlSSsHxNPcUvtOO/whvavF8FMSOd9NY= X-Received: by 2002:a50:8756:0:b0:4be:f5a0:a80f with SMTP id 22-20020a508756000000b004bef5a0a80fmr6508536edv.0.1679325153176; Mon, 20 Mar 2023 08:12:33 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: <343e827b-a1dc-b69b-9dcc-c6675deac070@quip.cz> In-Reply-To: <343e827b-a1dc-b69b-9dcc-c6675deac070@quip.cz> From: Mario Marietto Date: Mon, 20 Mar 2023 16:11:55 +0100 Message-ID: Subject: Re: Bhyve process consumes way too much CPU To: Miroslav Lachman <000.fbsd@quip.cz> Cc: =?UTF-8?Q?Julie_Koubov=C3=A1?= , freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="00000000000037af0605f7565bd6" X-Rspamd-Queue-Id: 4PgJBV4Tz6z3qPY X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --00000000000037af0605f7565bd6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello Julie. As I said some days ago,I'm trying to install homeassistant. For the moment I'm using Ubuntu 20.04 that I have installed on the Jetson nano,but later I will use Ubuntu within bhyve on FreeBSD. Unfortunately,a developer,I suppose,told me that homeassistant is not supported on ubuntu. In Fact I tried to do that following this tutorial : https://vikoky.medium.com/jetson-nano-powered-house-29ce73f11de4 but I've got a lot of errors. I've started a thread on reddit,asking for help,here : https://www.reddit.com/r/homeassistant/comments/11wglx7/homeassisant_cant_b= e_installed_on_ubuntu_2004_the/ and he/she told me that ubuntu is not supported. So,which linux distribution have you used within bhyve ? thanks. On Sun, Mar 19, 2023 at 7:12=E2=80=AFPM Miroslav Lachman <000.fbsd@quip.cz>= wrote: > On 18/03/2023 14:59, Julie Koubov=C3=A1 wrote: > > Hey everyone, > > > > I'm running Linux (Home Assistant OS) in Bhyve on FreeBSD 13.1. I use > > PCI passthrough to allow the VM to access a USB card with a couple of > > radio dongles. The host machine is an Intel Core i3 13100 with 64 GB of > > RAM. The CPU has 4 physical cores (8 hyper-threaded). The virtual > > machine is assigned four cores. > > > > The host load averages are 0.39 0.39 0.40 right now, which seems way to= o > > much. The same workload was previously handled by a Raspberry Pi 4, and > > the CPU usage there was under 10% when not doing anything special. > > Inside the guest OS, the CPU usage is reported around 5%, which seems > > reasonable. > > > > What's wrong? How can I start debugging this issue? I use ZFS on the > > host, vm-bhyve to manage the virtual machines, and I don't have a swap > > partition. > > I had similar problem few years ago. Never solved. Exhibited on bhyve > and VirtualBox too. The problem was "the more vCPU for VM, the slower VM"= . > Can you try to set just 1 vCPU to your VM? In my case, VM with 1 vCPU > was fast, almost no overhead, 2 v CPUs slightly slower but 4 or more was > slow as hell. > I would also recommend not to overprovision real CPU core count to vCPU > and not use multi/hyper threading cores as real cores. With your CPU, > use only 4 cores to assign to all your VMs (4 VMs with 1 vCPU each, or 2 > VMs with 2 vCPU each, 1 VM with 4 vCPU) > > Miroslav Lachman > > > --=20 Mario. --00000000000037af0605f7565bd6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello Julie.

As I said some = days ago,I'm trying to install homeassistant. For the moment I'm us= ing Ubuntu 20.04 that I have installed on the Jetson nano,but later I will = use Ubuntu within bhyve on FreeBSD. Unfortunately,a developer,I suppose,tol= d me that homeassistant is not supported on ubuntu. In Fact I tried to do t= hat following this tutorial :

<= div>
but I've got a lot of errors. I've started a thr= ead on reddit,asking for help,here :


and he/she told me that ubuntu is not su= pported. So,which linux distribution have you used within bhyve ? thanks.

On Sun, Mar 19, 2023 at 7:12=E2=80=AFPM Miroslav Lachman <000.fbsd@quip.cz> wr= ote:
On 18/03/20= 23 14:59, Julie Koubov=C3=A1 wrote:
> Hey everyone,
>
> I'm running Linux (Home Assistant OS) in Bhyve on FreeBSD 13.1. I = use
> PCI passthrough to allow the VM to=C2=A0access a USB card with a coupl= e of
> radio dongles. The host machine is an Intel Core i3 13100 with 64 GB o= f
> RAM. The CPU has 4 physical cores (8 hyper-threaded). The virtual
> machine is assigned four cores.
>
> The host load averages are 0.39 0.39 0.40 right now, which seems way t= oo
> much. The same workload was previously handled by a Raspberry Pi 4, an= d
> the CPU usage there was under 10% when not doing anything special. > Inside the guest OS, the CPU usage is reported around 5%, which seems =
> reasonable.
>
> What's wrong? How can I start debugging this issue? I use ZFS on t= he
> host, vm-bhyve to manage the virtual machines, and I don't have a = swap
> partition.

I had similar problem few years ago. Never solved. Exhibited on bhyve
and VirtualBox too. The problem was "the more vCPU for VM, the slower = VM".
Can you try to set just 1 vCPU to your VM? In my case, VM with 1 vCPU
was fast, almost no overhead, 2 v CPUs slightly slower but 4 or more was slow as hell.
I would also recommend not to overprovision real CPU core count to vCPU and not use multi/hyper threading cores as real cores. With your CPU,
use only 4 cores to assign to all your VMs (4 VMs with 1 vCPU each, or 2 VMs with 2 vCPU each, 1 VM with 4 vCPU)

Miroslav Lachman




--
Mario.
--00000000000037af0605f7565bd6-- From nobody Tue Mar 21 16:41:07 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pgy6S4swLz40VyR for ; Tue, 21 Mar 2023 16:41:20 +0000 (UTC) (envelope-from zeon@zeon.kiev.ua) Received: from mail-qv1-xf2e.google.com (mail-qv1-xf2e.google.com [IPv6:2607:f8b0:4864:20::f2e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pgy6R4Ft8z3p2Z for ; Tue, 21 Mar 2023 16:41:19 +0000 (UTC) (envelope-from zeon@zeon.kiev.ua) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=zeon-kiev-ua.20210112.gappssmtp.com header.s=20210112 header.b=TF6HBazJ; spf=none (mx1.freebsd.org: domain of zeon@zeon.kiev.ua has no SPF policy when checking 2607:f8b0:4864:20::f2e) smtp.mailfrom=zeon@zeon.kiev.ua; dmarc=none Received: by mail-qv1-xf2e.google.com with SMTP id m16so2860063qvi.12 for ; Tue, 21 Mar 2023 09:41:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zeon-kiev-ua.20210112.gappssmtp.com; s=20210112; t=1679416878; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=m2B9/WIeak4/XSDnfP8MaMzHOTm3a/nSkgYkUVzZFuE=; b=TF6HBazJaagAY5Ob6ctkKGT2wl0IJ3iIetVNlU8Je/Ze/jUL6q68d9U2sVI9dG0e55 76ihY63pMQ4gqFAEtXQv+QkIRU3kvAzz7NwaevZsobUeUG63fc+RnMmftPsQ6Ud87tO8 HMyuTrjpW6x3v4f+i8acc4qtvN1Lk7Rh+GECUhwfijPcl+0QDZ2OISKMq16fNJrxWxlI WMk6Uw4MJNg+5+2QT1cfMdYLJ216qIcydRB6Nk/ojbd2SaxLK+YDOH3MlQUyKrS1cAT7 LVY3xx7cBGjjw6FOHjEuXVjOh7BtqDw/NveXUJzO9Nrk+S9D5CayNsewHk3SUBBqHlHT aAEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679416878; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=m2B9/WIeak4/XSDnfP8MaMzHOTm3a/nSkgYkUVzZFuE=; b=okiDiXO6XgvkfXJuDfrv5pL0ccVbeF60iowiKF2GnuwJrBdWJ3dAawS38GSNw/8YpV YkQK53CM+j0Znfv04hn0XcCSrSrzJpxubk7Nk04Cf1M52fBoGbaEgTl7LA9Ko9ESyyTS gZb/amXWUvwlKL11G+8bj+9SL+LkMEHI/+SUZa33qHh1IImfwttFwmdpzP/sBLku/MxL A8ZQYjmWHUdZf7vME65nfnAhMoZkqP33Z+aQ+eeqQB96CjENYu229yNBFD51TKppD2Lk UcrcZLxNgWAFo0Jk/kYhuPDDO4ClyBB/Fb1EAjbdjlyFFi66FSzsLjvyGZPzXnqQlkH3 QAkA== X-Gm-Message-State: AO0yUKVaPnpElMgsHg9gjteMZ0+3wYiH+10vsQIZDIpKj+zxbN3vkFCm vJZzxZ0qET1iXLTlmHn1gGKR2XQMqlkSl1wSgfXjIg== X-Google-Smtp-Source: AK7set/0NajTZFdY5Mlw+jpFOgvXM/rCWoqckk4HVyXxuOzzCe1WkT/Uem5QkWxto3EYknQjXmOXKfkcf5YXwJAqsOY= X-Received: by 2002:ad4:4ba3:0:b0:56e:f4f0:e71d with SMTP id i3-20020ad44ba3000000b0056ef4f0e71dmr112743qvw.6.1679416877691; Tue, 21 Mar 2023 09:41:17 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: <343e827b-a1dc-b69b-9dcc-c6675deac070@quip.cz> In-Reply-To: From: Nikita Olenets Date: Tue, 21 Mar 2023 11:41:07 -0500 Message-ID: Subject: Re: Bhyve process consumes way too much CPU To: Mario Marietto Cc: =?UTF-8?Q?Julie_Koubov=C3=A1?= , Miroslav Lachman <000.fbsd@quip.cz>, freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="0000000000006cc96505f76bb6a1" X-Spamd-Result: default: False [-3.30 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[zeon-kiev-ua.20210112.gappssmtp.com:s=20210112]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::f2e:from]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[zeon-kiev-ua.20210112.gappssmtp.com:+]; R_SPF_NA(0.00)[no SPF record]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCPT_COUNT_THREE(0.00)[4]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[zeon.kiev.ua]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4Pgy6R4Ft8z3p2Z X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N --0000000000006cc96505f76bb6a1 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, Is there any specific reason you want it to be installed under Ubuntu? I=E2=80=99m asking this because I use HasOS(home assistant OS) which is bas= ed on Linux (can=E2=80=99t recall the distribution) and I=E2=80=99m running this = for three years now. No issues at all. I do have even passed through ZigBee usb stick and it works just fine. On Mon, Mar 20, 2023 at 11:12 Mario Marietto wrote= : > Hello Julie. > > As I said some days ago,I'm trying to install homeassistant. For the > moment I'm using Ubuntu 20.04 that I have installed on the Jetson nano,bu= t > later I will use Ubuntu within bhyve on FreeBSD. Unfortunately,a > developer,I suppose,told me that homeassistant is not supported on ubuntu= . > In Fact I tried to do that following this tutorial : > > https://vikoky.medium.com/jetson-nano-powered-house-29ce73f11de4 > > but I've got a lot of errors. I've started a thread on reddit,asking for > help,here : > > > https://www.reddit.com/r/homeassistant/comments/11wglx7/homeassisant_cant= _be_installed_on_ubuntu_2004_the/ > > and he/she told me that ubuntu is not supported. So,which linux > distribution have you used within bhyve ? thanks. > > On Sun, Mar 19, 2023 at 7:12=E2=80=AFPM Miroslav Lachman <000.fbsd@quip.c= z> wrote: > >> On 18/03/2023 14:59, Julie Koubov=C3=A1 wrote: >> > Hey everyone, >> > >> > I'm running Linux (Home Assistant OS) in Bhyve on FreeBSD 13.1. I use >> > PCI passthrough to allow the VM to access a USB card with a couple of >> > radio dongles. The host machine is an Intel Core i3 13100 with 64 GB o= f >> > RAM. The CPU has 4 physical cores (8 hyper-threaded). The virtual >> > machine is assigned four cores. >> > >> > The host load averages are 0.39 0.39 0.40 right now, which seems way >> too >> > much. The same workload was previously handled by a Raspberry Pi 4, an= d >> > the CPU usage there was under 10% when not doing anything special. >> > Inside the guest OS, the CPU usage is reported around 5%, which seems >> > reasonable. >> > >> > What's wrong? How can I start debugging this issue? I use ZFS on the >> > host, vm-bhyve to manage the virtual machines, and I don't have a swap >> > partition. >> >> I had similar problem few years ago. Never solved. Exhibited on bhyve >> and VirtualBox too. The problem was "the more vCPU for VM, the slower VM= ". >> Can you try to set just 1 vCPU to your VM? In my case, VM with 1 vCPU >> was fast, almost no overhead, 2 v CPUs slightly slower but 4 or more was >> slow as hell. >> I would also recommend not to overprovision real CPU core count to vCPU >> and not use multi/hyper threading cores as real cores. With your CPU, >> use only 4 cores to assign to all your VMs (4 VMs with 1 vCPU each, or 2 >> VMs with 2 vCPU each, 1 VM with 4 vCPU) >> >> Miroslav Lachman >> >> >> > > -- > Mario. > --=20 Nikita Olenets --0000000000006cc96505f76bb6a1 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

Is= there any specific reason you want it to be installed under Ubuntu?
<= div dir=3D"auto">I=E2=80=99m asking this because I use HasOS(home assistant= OS) which is based on Linux (can=E2=80=99t recall the distribution) and I= =E2=80=99m running this for three=C2=A0years now. No issues at all. I do ha= ve even passed through ZigBee usb stick and it works just fine.

On Mon, Mar 20, 2023 at 11:12 Mario Marietto <marietto2008@gmail.com> wrote:
Hello Julie.

As I said some days ago,I'm trying to install ho= meassistant. For the moment I'm using Ubuntu 20.04 that I have installe= d on the Jetson nano,but later I will use Ubuntu within bhyve on FreeBSD. U= nfortunately,a developer,I suppose,told me that homeassistant is not suppor= ted on ubuntu. In Fact I tried to do that following this tutorial :


but I've got a= lot of errors. I've started a thread on reddit,asking for help,here :<= /div>


an= d he/she told me that ubuntu is not supported. So,which linux distribution = have you used within bhyve ? thanks.

On Sun, Mar 19, 2023 at 7:12= =E2=80=AFPM Miroslav Lachman <000.fbsd@quip.cz> wrote:
On 18/03/2023 14:59, Julie Koubov=C3=A1 wrote:<= br> > Hey everyone,
>
> I'm running Linux (Home Assistant OS) in Bhyve on FreeBSD 13.1. I = use
> PCI passthrough to allow the VM to=C2=A0access a USB card with a coupl= e of
> radio dongles. The host machine is an Intel Core i3 13100 with 64 GB o= f
> RAM. The CPU has 4 physical cores (8 hyper-threaded). The virtual
> machine is assigned four cores.
>
> The host load averages are 0.39 0.39 0.40 right now, which seems way t= oo
> much. The same workload was previously handled by a Raspberry Pi 4, an= d
> the CPU usage there was under 10% when not doing anything special. > Inside the guest OS, the CPU usage is reported around 5%, which seems =
> reasonable.
>
> What's wrong? How can I start debugging this issue? I use ZFS on t= he
> host, vm-bhyve to manage the virtual machines, and I don't have a = swap
> partition.

I had similar problem few years ago. Never solved. Exhibited on bhyve
and VirtualBox too. The problem was "the more vCPU for VM, the slower = VM".
Can you try to set just 1 vCPU to your VM? In my case, VM with 1 vCPU
was fast, almost no overhead, 2 v CPUs slightly slower but 4 or more was slow as hell.
I would also recommend not to overprovision real CPU core count to vCPU and not use multi/hyper threading cores as real cores. With your CPU,
use only 4 cores to assign to all your VMs (4 VMs with 1 vCPU each, or 2 VMs with 2 vCPU each, 1 VM with 4 vCPU)

Miroslav Lachman




--
Mario.
--
Nikita Olenets
=
--0000000000006cc96505f76bb6a1-- From nobody Tue Mar 21 16:53:35 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PgyPM3Jqtz40X76 for ; Tue, 21 Mar 2023 16:54:15 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PgyPM0M7fz3rpN for ; Tue, 21 Mar 2023 16:54:15 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x533.google.com with SMTP id eg48so62174014edb.13 for ; Tue, 21 Mar 2023 09:54:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679417653; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Jj52Nyg3rYuMfo+oJMkGEE4QE4jFy3bCFCIp4hNkmPk=; b=Yjo1vbF4STAx9NPSNKzUIK0QZBTYX3XIa9rUvCOncAFad6lOasnf4wcHCHgrwtkbmm GxAd9VgKMU7iNA1e394hpLi4UeSu6PU7Pun/13Y5/J4pUxQOhmhYIjZD/lHeDhSQTcLd t1iqYga2bF3IdhEHve2LAiVBRbTbB+QvcPzb6KtT6h2mYsUjxYI+9/TSXMvteX2R9YYr UX+rjZAE//0b+fFqZMeh0QEfLYLV7GSiAcG3dIjQLqrTMrkn0nZ8qwaY/BmBcGzqaMGk p4ZXI9DubtRmIKOhwFH5ahR70rF2OowbeksG4WdwOUYFtyUPGMuNqPEAN/6sk6Ukr9St 3UYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679417653; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Jj52Nyg3rYuMfo+oJMkGEE4QE4jFy3bCFCIp4hNkmPk=; b=QEfFVTQnr+9r0ClKqk2LMZhaT9gNl6NR1FG39Bz4PApOfQS2kBWKZz46WpoeVcjTzk NxlwLbzxf3oppk6dw12gxJPnP0bZ512poiMKvoluRHEwhAWyONeydUYhG1Juv19aYKhC tmZqTF+lJiaqYaQJzyu05+LbGccPktrC7q4ldlsoCDg8DQxHFBbjkSAapw9TVDKrJ43j 3bN0CxF41cz8V6jFAx8J0yHY0d2BDUxs31c/t01jHUMb/rAMIS9Ryz0lS8LFDgrnA942 pwLZoMlnj56iUEgOjEDpj+Kxx7d1C2xXsvhBWJos9jV7tgRkTmksLQ9boqFCJ3g9gCkK g/Dg== X-Gm-Message-State: AO0yUKW4LTzMfPeGzVvH4vrTjR2NWGU/nxmesjlzlv++XWgnMOC77Ygq PVe4Ex2z6nmr4TTxbLka0mJ9YDUs6NlL4AjgSqVgcSRQ/28= X-Google-Smtp-Source: AK7set8xYG2zqVLONGpJl+It+GpuBaE3+haoWKqAfqQSe004Nmh6aL/8TF8W/nxjqoLVmjbjNoT7PHN0BlP6cb2mMjI= X-Received: by 2002:a17:907:75ea:b0:8b1:38d6:9853 with SMTP id jz10-20020a17090775ea00b008b138d69853mr1710019ejc.2.1679417653405; Tue, 21 Mar 2023 09:54:13 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: <343e827b-a1dc-b69b-9dcc-c6675deac070@quip.cz> In-Reply-To: From: Mario Marietto Date: Tue, 21 Mar 2023 17:53:35 +0100 Message-ID: Subject: Re: Bhyve process consumes way too much CPU To: Nikita Olenets Cc: =?UTF-8?Q?Julie_Koubov=C3=A1?= , Miroslav Lachman <000.fbsd@quip.cz>, freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="000000000000a92bd405f76be4df" X-Rspamd-Queue-Id: 4PgyPM0M7fz3rpN X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --000000000000a92bd405f76be4df Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Are you talking with me ? I can install Home Assistant only on the hardware that I already have. Actually the best piece of hardware that I can use for my project (installing the chat gpt module on Home Assistant + a speech recognition software to give a voice to chat gpt) is the nvidia jetson nano. And If I remember correctly,it supports only Ubuntu. I can do the same on my Workstation,but I prefer to have the ability to move the smaller nano within a plastic container. At the end of the day It will be almost the same as a vocal assistant,but uch more programmable. On Tue, Mar 21, 2023 at 5:41=E2=80=AFPM Nikita Olenets = wrote: > Hi, > > Is there any specific reason you want it to be installed under Ubuntu? > I=E2=80=99m asking this because I use HasOS(home assistant OS) which is b= ased on > Linux (can=E2=80=99t recall the distribution) and I=E2=80=99m running thi= s for three years > now. No issues at all. I do have even passed through ZigBee usb stick and > it works just fine. > > On Mon, Mar 20, 2023 at 11:12 Mario Marietto > wrote: > >> Hello Julie. >> >> As I said some days ago,I'm trying to install homeassistant. For the >> moment I'm using Ubuntu 20.04 that I have installed on the Jetson nano,b= ut >> later I will use Ubuntu within bhyve on FreeBSD. Unfortunately,a >> developer,I suppose,told me that homeassistant is not supported on ubunt= u. >> In Fact I tried to do that following this tutorial : >> >> https://vikoky.medium.com/jetson-nano-powered-house-29ce73f11de4 >> >> but I've got a lot of errors. I've started a thread on reddit,asking for >> help,here : >> >> >> https://www.reddit.com/r/homeassistant/comments/11wglx7/homeassisant_can= t_be_installed_on_ubuntu_2004_the/ >> >> and he/she told me that ubuntu is not supported. So,which linux >> distribution have you used within bhyve ? thanks. >> >> On Sun, Mar 19, 2023 at 7:12=E2=80=AFPM Miroslav Lachman <000.fbsd@quip.= cz> >> wrote: >> >>> On 18/03/2023 14:59, Julie Koubov=C3=A1 wrote: >>> > Hey everyone, >>> > >>> > I'm running Linux (Home Assistant OS) in Bhyve on FreeBSD 13.1. I use >>> > PCI passthrough to allow the VM to access a USB card with a couple of >>> > radio dongles. The host machine is an Intel Core i3 13100 with 64 GB >>> of >>> > RAM. The CPU has 4 physical cores (8 hyper-threaded). The virtual >>> > machine is assigned four cores. >>> > >>> > The host load averages are 0.39 0.39 0.40 right now, which seems way >>> too >>> > much. The same workload was previously handled by a Raspberry Pi 4, >>> and >>> > the CPU usage there was under 10% when not doing anything special. >>> > Inside the guest OS, the CPU usage is reported around 5%, which seems >>> > reasonable. >>> > >>> > What's wrong? How can I start debugging this issue? I use ZFS on the >>> > host, vm-bhyve to manage the virtual machines, and I don't have a swa= p >>> > partition. >>> >>> I had similar problem few years ago. Never solved. Exhibited on bhyve >>> and VirtualBox too. The problem was "the more vCPU for VM, the slower >>> VM". >>> Can you try to set just 1 vCPU to your VM? In my case, VM with 1 vCPU >>> was fast, almost no overhead, 2 v CPUs slightly slower but 4 or more wa= s >>> slow as hell. >>> I would also recommend not to overprovision real CPU core count to vCPU >>> and not use multi/hyper threading cores as real cores. With your CPU, >>> use only 4 cores to assign to all your VMs (4 VMs with 1 vCPU each, or = 2 >>> VMs with 2 vCPU each, 1 VM with 4 vCPU) >>> >>> Miroslav Lachman >>> >>> >>> >> >> -- >> Mario. >> > -- > Nikita Olenets > > --=20 Mario. --000000000000a92bd405f76be4df Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Are you talking with me ? I can install Home Assistant onl= y on the hardware that I already have. Actually the best piece of hardware = that I can use for my project (installing the chat gpt module on Home Assis= tant + a speech recognition software to give a voice to chat gpt) is the nv= idia jetson nano. And If I remember correctly,it supports only Ubuntu. I ca= n do the same on my Workstation,but I prefer to have the ability to move th= e smaller nano within a plastic container. At the end of the day It will be= almost the same as a vocal assistant,but uch more programmable.
=
On Tue= , Mar 21, 2023 at 5:41=E2=80=AFPM Nikita Olenets <zeon@zeon.kiev.ua> wrote:
Hi,

Is there any specific reason you want it to = be installed under Ubuntu?
I=E2=80=99m asking this b= ecause I use HasOS(home assistant OS) which is based on Linux (can=E2=80=99= t recall the distribution) and I=E2=80=99m running this for three=C2=A0year= s now. No issues at all. I do have even passed through ZigBee usb stick and= it works just fine.

On Mon, Mar 20, 2023 at 11= :12 Mario Marietto <marietto2008@gmail.com> wrote:
Hello Julie.
=
As I said some days ago,I'm trying to install homeassist= ant. For the moment I'm using Ubuntu 20.04 that I have installed on the= Jetson nano,but later I will use Ubuntu within bhyve on FreeBSD. Unfortuna= tely,a developer,I suppose,told me that homeassistant is not supported on u= buntu. In Fact I tried to do that following this tutorial :

<= /div>

but I've got a lot of = errors. I've started a thread on reddit,asking for help,here :


and he/she= told me that ubuntu is not supported. So,which linux distribution have you= used within bhyve ? thanks.

=
On Sun, Mar 19, 2023 at 7:12=E2=80=AF= PM Miroslav Lachman <000.fbsd@quip.cz> wrote:
On 18/03/2023 14:59, Julie Koubov=C3=A1 wrote:
> Hey everyone,
>
> I'm running Linux (Home Assistant OS) in Bhyve on FreeBSD 13.1. I = use
> PCI passthrough to allow the VM to=C2=A0access a USB card with a coupl= e of
> radio dongles. The host machine is an Intel Core i3 13100 with 64 GB o= f
> RAM. The CPU has 4 physical cores (8 hyper-threaded). The virtual
> machine is assigned four cores.
>
> The host load averages are 0.39 0.39 0.40 right now, which seems way t= oo
> much. The same workload was previously handled by a Raspberry Pi 4, an= d
> the CPU usage there was under 10% when not doing anything special. > Inside the guest OS, the CPU usage is reported around 5%, which seems =
> reasonable.
>
> What's wrong? How can I start debugging this issue? I use ZFS on t= he
> host, vm-bhyve to manage the virtual machines, and I don't have a = swap
> partition.

I had similar problem few years ago. Never solved. Exhibited on bhyve
and VirtualBox too. The problem was "the more vCPU for VM, the slower = VM".
Can you try to set just 1 vCPU to your VM? In my case, VM with 1 vCPU
was fast, almost no overhead, 2 v CPUs slightly slower but 4 or more was slow as hell.
I would also recommend not to overprovision real CPU core count to vCPU and not use multi/hyper threading cores as real cores. With your CPU,
use only 4 cores to assign to all your VMs (4 VMs with 1 vCPU each, or 2 VMs with 2 vCPU each, 1 VM with 4 vCPU)

Miroslav Lachman




--
Mario.
--
Niki= ta Olenets



--
Mario.
--000000000000a92bd405f76be4df-- From nobody Tue Mar 21 16:59:50 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PgyX24kCrz40XDd for ; Tue, 21 Mar 2023 17:00:02 +0000 (UTC) (envelope-from zeon@zeon.kiev.ua) Received: from mail-qt1-x833.google.com (mail-qt1-x833.google.com [IPv6:2607:f8b0:4864:20::833]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PgyX21wmQz3s8d for ; Tue, 21 Mar 2023 17:00:02 +0000 (UTC) (envelope-from zeon@zeon.kiev.ua) Authentication-Results: mx1.freebsd.org; none Received: by mail-qt1-x833.google.com with SMTP id x1so18663898qtr.7 for ; Tue, 21 Mar 2023 10:00:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zeon-kiev-ua.20210112.gappssmtp.com; s=20210112; t=1679418001; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=iRrZKdK2o4tVD8boLkjw8rGH6mI7Gg/t/cLXIj3Ya3k=; b=s2g/fUKjSIeGtThoMELMTp/L4vSDD5ElWa5NdrbpjGVcyNgs5dOLzVNI06LeP2I78v u2GbsaxMeuvACKnnf94IMtqKe8welFwkPDdZfO1igswla1TZZ5H7uM5sadxY1J9cBEGw YLooqm8TksYiC5pkfO2oG5EWnAZX1wgp7pBsw5JeWX/XsDQb0itmiKqVtrcFsRLj8uB/ w2jNqEheN4F+zS2fU+Rjp22siP1I1QmzP9f+hNd8y68v5LiVFKNMMNR+rnn/ymgXrM0A mxgrW2YVu06j82a5xuCQDMflKimihYckZjuKMG1jt5TzrAIHcd/LP9pdvMDuHFhkXbO2 RnzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679418001; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=iRrZKdK2o4tVD8boLkjw8rGH6mI7Gg/t/cLXIj3Ya3k=; b=G4oGWXZhx7v62KSCDhsw11vDxE+aZhXhjBzhS3GzQuv9R+Y8n8KNRvZ9QR+DYkXggt XSwI3rxHnAMUZ/ozmtCemRvxa0FRX069hija902LrLiYRHffOJYNhtzNn6nfdZ3H+FC2 w7dtROXAZReUNn59UwW6BQbyROop0J+vDfq3qk59Fd14pWJ3JvmvBUuSIs1d8t66R84P PB1Mn27VVjKwUzWGNn4DFVYRAdtNwBi67B267KZ1xK+5JPshoZ/2dX14kpApI713JdNZ 5VSJbfkrPXe+Ey50bYS5lhUQJqlgw08QDQ3X37uHXeLuJQBgkHxAO5NVlHlkZWHjTg9x hx2Q== X-Gm-Message-State: AO0yUKVgd/QYEHG/zPlS7c7wP4MfB5f9AOipFoppIo6v0+2bxlB06zj0 ad8ZHOP0mb0IWX0bTBwPEPL9JSXeCvSt74XAbSLw7A== X-Google-Smtp-Source: AK7set+Opa8ILXZwX+4si9UCRf4zCXXDbHsMkO3Qmi3KPXRsoc5/ekIqjfZKlFuUgs9PJzGj+uHlgJfQ8R/eiXm8h0s= X-Received: by 2002:a05:622a:18aa:b0:3de:bafb:82c1 with SMTP id v42-20020a05622a18aa00b003debafb82c1mr219099qtc.3.1679418001401; Tue, 21 Mar 2023 10:00:01 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: <343e827b-a1dc-b69b-9dcc-c6675deac070@quip.cz> In-Reply-To: From: Nikita Olenets Date: Tue, 21 Mar 2023 11:59:50 -0500 Message-ID: Subject: Re: Bhyve process consumes way too much CPU To: Mario Marietto Cc: =?UTF-8?Q?Julie_Koubov=C3=A1?= , Miroslav Lachman <000.fbsd@quip.cz>, freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="00000000000067398d05f76bf971" X-Rspamd-Queue-Id: 4PgyX21wmQz3s8d X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --00000000000067398d05f76bf971 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hey Mario, Yeah, sorry. I was thinking that you were the topic starter and you were trying to make things work under bhyve. Have a goo day. =E2=80=94 Nikita On Tue, Mar 21, 2023 at 12:54 Mario Marietto wrote= : > Are you talking with me ? I can install Home Assistant only on the > hardware that I already have. Actually the best piece of hardware that I > can use for my project (installing the chat gpt module on Home Assistant = + > a speech recognition software to give a voice to chat gpt) is the nvidia > jetson nano. And If I remember correctly,it supports only Ubuntu. I can d= o > the same on my Workstation,but I prefer to have the ability to move the > smaller nano within a plastic container. At the end of the day It will be > almost the same as a vocal assistant,but uch more programmable. > > On Tue, Mar 21, 2023 at 5:41=E2=80=AFPM Nikita Olenets wrote: > >> Hi, >> >> Is there any specific reason you want it to be installed under Ubuntu? >> I=E2=80=99m asking this because I use HasOS(home assistant OS) which is = based on >> Linux (can=E2=80=99t recall the distribution) and I=E2=80=99m running th= is for three years >> now. No issues at all. I do have even passed through ZigBee usb stick an= d >> it works just fine. >> >> On Mon, Mar 20, 2023 at 11:12 Mario Marietto >> wrote: >> >>> Hello Julie. >>> >>> As I said some days ago,I'm trying to install homeassistant. For the >>> moment I'm using Ubuntu 20.04 that I have installed on the Jetson nano,= but >>> later I will use Ubuntu within bhyve on FreeBSD. Unfortunately,a >>> developer,I suppose,told me that homeassistant is not supported on ubun= tu. >>> In Fact I tried to do that following this tutorial : >>> >>> https://vikoky.medium.com/jetson-nano-powered-house-29ce73f11de4 >>> >>> but I've got a lot of errors. I've started a thread on reddit,asking fo= r >>> help,here : >>> >>> >>> https://www.reddit.com/r/homeassistant/comments/11wglx7/homeassisant_ca= nt_be_installed_on_ubuntu_2004_the/ >>> >>> and he/she told me that ubuntu is not supported. So,which linux >>> distribution have you used within bhyve ? thanks. >>> >>> On Sun, Mar 19, 2023 at 7:12=E2=80=AFPM Miroslav Lachman <000.fbsd@quip= .cz> >>> wrote: >>> >>>> On 18/03/2023 14:59, Julie Koubov=C3=A1 wrote: >>>> > Hey everyone, >>>> > >>>> > I'm running Linux (Home Assistant OS) in Bhyve on FreeBSD 13.1. I us= e >>>> > PCI passthrough to allow the VM to access a USB card with a couple o= f >>>> > radio dongles. The host machine is an Intel Core i3 13100 with 64 GB >>>> of >>>> > RAM. The CPU has 4 physical cores (8 hyper-threaded). The virtual >>>> > machine is assigned four cores. >>>> > >>>> > The host load averages are 0.39 0.39 0.40 right now, which seems way >>>> too >>>> > much. The same workload was previously handled by a Raspberry Pi 4, >>>> and >>>> > the CPU usage there was under 10% when not doing anything special. >>>> > Inside the guest OS, the CPU usage is reported around 5%, which seem= s >>>> > reasonable. >>>> > >>>> > What's wrong? How can I start debugging this issue? I use ZFS on the >>>> > host, vm-bhyve to manage the virtual machines, and I don't have a >>>> swap >>>> > partition. >>>> >>>> I had similar problem few years ago. Never solved. Exhibited on bhyve >>>> and VirtualBox too. The problem was "the more vCPU for VM, the slower >>>> VM". >>>> Can you try to set just 1 vCPU to your VM? In my case, VM with 1 vCPU >>>> was fast, almost no overhead, 2 v CPUs slightly slower but 4 or more >>>> was >>>> slow as hell. >>>> I would also recommend not to overprovision real CPU core count to vCP= U >>>> and not use multi/hyper threading cores as real cores. With your CPU, >>>> use only 4 cores to assign to all your VMs (4 VMs with 1 vCPU each, or >>>> 2 >>>> VMs with 2 vCPU each, 1 VM with 4 vCPU) >>>> >>>> Miroslav Lachman >>>> >>>> >>>> >>> >>> -- >>> Mario. >>> >> -- >> Nikita Olenets >> >> > > -- > Mario. > --=20 Nikita Olenets --00000000000067398d05f76bf971 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hey Mario,

Yeah, sorry. I was thinking that you were the topic starter and you we= re trying to make things work under bhyve.

Have a goo day.

=E2=80=94 Nikita

On Tue, Mar 21, 2023 at 12:54 Mario Mariett= o <marietto2008@gmail.com&= gt; wrote:
Are you= talking with me ? I can install Home Assistant only on the hardware that I= already have. Actually the best piece of hardware that I can use for my pr= oject (installing the chat gpt module on Home Assistant + a speech recognit= ion software to give a voice to chat gpt) is the nvidia jetson nano. And If= I remember correctly,it supports only Ubuntu. I can do the same on my Work= station,but I prefer to have the ability to move the smaller nano within a = plastic container. At the end of the day It will be almost the same as a vo= cal assistant,but uch more programmable.

On Tue, Mar 21, 2023 at 5:41= =E2=80=AFPM Nikita Olenets <zeon@zeon.kiev.ua> wrote:
Hi,
<= br>
Is there any specific reason you want it to be i= nstalled under Ubuntu?
I=E2=80=99m asking this becau= se I use HasOS(home assistant OS) which is based on Linux (can=E2=80=99t re= call the distribution) and I=E2=80=99m running this for three=C2=A0years no= w. No issues at all. I do have even passed through ZigBee usb stick and it = works just fine.

On Mon, Mar 20, 2023 at 11:12 = Mario Marietto <marietto2008@gmail.com> wrote:
Hello Julie.

=
As I said some days ago,I'm trying to install homeassistant.= For the moment I'm using Ubuntu 20.04 that I have installed on the Jet= son nano,but later I will use Ubuntu within bhyve on FreeBSD. Unfortunately= ,a developer,I suppose,told me that homeassistant is not supported on ubunt= u. In Fact I tried to do that following this tutorial :


but I've got a lot of erro= rs. I've started a thread on reddit,asking for help,here :

and he/she tol= d me that ubuntu is not supported. So,which linux distribution have you use= d within bhyve ? thanks.

On Sun, Mar 19, 2023 at 7:12=E2=80=AFPM M= iroslav Lachman <0= 00.fbsd@quip.cz> wrote:
On 18/03/2023 14:59, Julie Koubov=C3=A1 wrote:
> Hey everyone,
>
> I'm running Linux (Home Assistant OS) in Bhyve on FreeBSD 13.1. I = use
> PCI passthrough to allow the VM to=C2=A0access a USB card with a coupl= e of
> radio dongles. The host machine is an Intel Core i3 13100 with 64 GB o= f
> RAM. The CPU has 4 physical cores (8 hyper-threaded). The virtual
> machine is assigned four cores.
>
> The host load averages are 0.39 0.39 0.40 right now, which seems way t= oo
> much. The same workload was previously handled by a Raspberry Pi 4, an= d
> the CPU usage there was under 10% when not doing anything special. > Inside the guest OS, the CPU usage is reported around 5%, which seems =
> reasonable.
>
> What's wrong? How can I start debugging this issue? I use ZFS on t= he
> host, vm-bhyve to manage the virtual machines, and I don't have a = swap
> partition.

I had similar problem few years ago. Never solved. Exhibited on bhyve
and VirtualBox too. The problem was "the more vCPU for VM, the slower = VM".
Can you try to set just 1 vCPU to your VM? In my case, VM with 1 vCPU
was fast, almost no overhead, 2 v CPUs slightly slower but 4 or more was slow as hell.
I would also recommend not to overprovision real CPU core count to vCPU and not use multi/hyper threading cores as real cores. With your CPU,
use only 4 cores to assign to all your VMs (4 VMs with 1 vCPU each, or 2 VMs with 2 vCPU each, 1 VM with 4 vCPU)

Miroslav Lachman




--
Mario.
--
Niki= ta Olenets



--
Mario.
--
Nikita Olenets
=
--00000000000067398d05f76bf971--