From nobody Fri Jul 1 11:20:56 2022 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id BB53E8AEB94 for ; Fri, 1 Jul 2022 11:21:05 +0000 (UTC) (envelope-from nbe@vkf-renzel.de) Received: from mx1.renzel.net (mx1.renzel.net [195.243.213.130]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.renzel.net", Issuer "DigiCert TLS RSA SHA256 2020 CA1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LZCSK0c48z3lNh for ; Fri, 1 Jul 2022 11:21:04 +0000 (UTC) (envelope-from nbe@vkf-renzel.de) Received: from dublin.vkf.isb.de.renzel.net (unknown [62.159.136.5]) by mx1.renzel.net (smtpd) with ESMTP id E8A5CDB581; Fri, 1 Jul 2022 13:20:56 +0200 (CEST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.104.2 at clamav-milter.renzel.net Received: from [172.18.96.1] (asbach.renzel.de.isb [172.18.96.1]) by dublin.vkf.isb.de.renzel.net (Postfix) with ESMTPA id DE6BB1948CB; Fri, 1 Jul 2022 13:20:56 +0200 (CEST) Message-ID: Date: Fri, 1 Jul 2022 13:20:56 +0200 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Thunderbird/91.10.0 Subject: Re: bhyve: slow network throughput between guest VM and host (and vice versa)? Content-Language: en-US To: John-Mark Gurney Cc: freebsd-virtualization@FreeBSD.org References: <20220701024708.GC88842@funkthat.com> From: Nils Beyer Organization: VKF Renzel GmbH In-Reply-To: <20220701024708.GC88842@funkthat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1005.1 required=10.0 tests=ALL_TRUSTED=-1000, BAYES_00=-5.1,NICE_REPLY_A=-0.001 autolearn=ham autolearn_force=yes (ALL_TRUSTED) version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on clamav-milter.renzel.net X-Rspamd-Queue-Id: 4LZCSK0c48z3lNh X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of nbe@vkf-renzel.de designates 195.243.213.130 as permitted sender) smtp.mailfrom=nbe@vkf-renzel.de X-Spamd-Result: default: False [0.15 / 15.00]; RCVD_TLS_LAST(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[vkf-renzel.de]; NEURAL_HAM_MEDIUM(-0.10)[-0.098]; RWL_MAILSPIKE_GOOD(0.00)[195.243.213.130:from]; NEURAL_SPAM_SHORT(0.74)[0.744]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_MED(-0.20)[195.243.213.130:from]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-virtualization]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3320, ipnet:195.243.0.0/16, country:DE]; SUBJECT_ENDS_QUESTION(1.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On 01.07.22 04:47, John-Mark Gurney wrote: >> [...] >> Is that expected that I'm not able to get more throughput within the bhyve network >> stack (guest <-> host)? I was expecting way more then 10Gbit/s... > > What does CPU usage look like on the host and in the VM? This sounds > like hitting a CPU bandwidth limit in that you're limited by pps... 1) CPU load on the host is not very high (host server is a dual AMD EPYC 7351 16-Core Processor system): 3333 root 86 0 4230M 1202M CPU8 8 0:33 49.75% bhyve{vtnet-5:0 tx} 3333 root 87 0 4230M 1202M CPU0 0 1:03 48.67% bhyve{vcpu 0} 3333 root 85 0 4230M 1202M CPU6 6 0:29 45.52% bhyve{vcpu 5} 3333 root 84 0 4230M 1202M CPU5 5 0:47 38.64% bhyve{vcpu 2} 3390 root 4 0 17M 6472K RUN 3 0:19 38.55% iperf3 3333 root 81 0 4230M 1202M CPU4 4 0:26 32.00% bhyve{mevent} 3333 root 30 0 4230M 1202M CPU10 10 0:25 13.44% bhyve{vcpu 1} 3333 root 23 0 4230M 1202M RUN 9 0:08 5.44% bhyve{vcpu 4} 3333 root 23 0 4230M 1202M CPU7 7 0:17 5.26% bhyve{vcpu 3} 2) CPU load in the VM at around 30% So there should be room for improvements...