Date: Wed, 22 Oct 2025 14:27:33 -0400 From: Cheng Cui <cc@freebsd.org> To: void <void@f-m.fm> Cc: freebsd-net@freebsd.org Subject: Re: vtnet is still slow Message-ID: <51300B68-3D16-434E-8A65-696EC48FA98C@freebsd.org> In-Reply-To: <aPjkGDgl6MCF-unb@int21h> References: <aPjkGDgl6MCF-unb@int21h>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] Interesting, my Bhyve host node "n1" and my MacbookProM1 laptop are wired to the same 1Gbps home switch. The "n1" runs FreeBSD stable/14-9e40b536cb58 and Bhyve, with ethernet port re0. The "n1fbsd15x" VM runing FreeBSD stable/15-01a653205313 is a guest in "n1" with vtnet0 port. All sysctl values in default. The iperf3 "iperf3 -c laptop -i1 -t10" traffic sent from the "n1fbsd15x" VM to my laptop: [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.08 sec 89.5 MBytes 695 Mbits/sec 37 501 KBytes [ 5] 1.08-2.08 sec 112 MBytes 946 Mbits/sec 0 759 KBytes [ 5] 2.08-3.08 sec 113 MBytes 946 Mbits/sec 0 949 KBytes [ 5] 3.08-4.07 sec 111 MBytes 943 Mbits/sec 6 5.66 KBytes [ 5] 4.07-5.07 sec 112 MBytes 941 Mbits/sec 0 705 KBytes [ 5] 5.07-6.01 sec 106 MBytes 941 Mbits/sec 0 817 KBytes [ 5] 6.01-7.07 sec 119 MBytes 942 Mbits/sec 0 915 KBytes [ 5] 7.07-8.07 sec 112 MBytes 942 Mbits/sec 0 984 KBytes [ 5] 8.07-9.07 sec 112 MBytes 942 Mbits/sec 0 1.01 MBytes [ 5] 9.07-10.07 sec 113 MBytes 941 Mbits/sec 0 1.04 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.07 sec 1.07 GBytes 916 Mbits/sec 43 sender [ 5] 0.00-10.08 sec 1.07 GBytes 914 Mbits/sec receiver iperf Done. My Bhyve setup: https://wiki.freebsd.org/chengcui/install_FreeBSD13_VM_via_bhyve > On Oct 22, 2025, at 10:03, void <void@f-m.fm> wrote: > > Hi, > > I wondered if any work was happening to address the long-standing issue > of slow vtnet? (in my case, this leads to slow freebsd guest performance > in bhyve compared with every linux guest tested and OpenBSD). > > I thought maybe the recent bridge work may have ameliorated the problem some, but testing, after configuring a (recent) stable/15 server to use the ip-configured-on-bridge-method, showed no improvement using > a (recent) stable/14 guest > > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-60.03 sec 3.17 GBytes 454 Mbits/sec 0 sender > [ 5] 0.00-60.03 sec 3.17 GBytes 454 Mbits/sec receiver > > for comparison, an ubuntu 25.04 guest: > > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-60.00 sec 6.42 GBytes 919 Mbits/sec 5427 sender > [ 5] 0.00-60.00 sec 6.42 GBytes 919 Mbits/sec receiver > > the baremetal host > > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-60.01 sec 6.58 GBytes 941 Mbits/sec 0 sender > [ 5] 0.00-60.01 sec 6.58 GBytes 941 Mbits/sec receiver > > Tests done a year and a half ago on similar hardware showing > the same patterns http://void.f-m.fm.user.fm/bhyve-virtio-testing.html > -- > Best Regards, Cheng Cui [-- Attachment #2 --] <html aria-label="message body"><head><meta http-equiv="content-type" content="text/html; charset=us-ascii"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">Interesting, my Bhyve host node "n1" and my MacbookProM1 laptop are wired to the same 1Gbps home switch.<div><br></div><div>The "n1" runs FreeBSD <span style="color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);">stable/14-9e40b536cb58 and Bhyve, with ethernet port </span><span style="color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);">re0</span><span style="color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);">. The "</span><span style="color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);">n1fbsd15x" VM runing FreeBSD </span><span style="color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);">stable/15-01a653205313 </span><span style="color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);">is a guest in "n1" with </span><span style="color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);">vtnet0 port. All sysctl values in default.</span></div><div><span style="color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><br></span></div><div><span style="color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);">The iperf3 "</span><span style="color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);">iperf3 -c laptop -i1 -t10</span><span style="color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);">" traffic sent from the </span><span style="color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);">"</span><span style="color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);">n1fbsd15x" VM to my laptop:</span></div><div><p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">[ ID] Interval Transfer Bitrate Retr Cwnd</span></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">[ 5] 0.00-1.08 sec 89.5 MBytes 695 Mbits/sec 37 501 KBytes </span></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">[ 5] 1.08-2.08 sec 112 MBytes 946 Mbits/sec 0 759 KBytes </span></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">[ 5] 2.08-3.08 sec 113 MBytes 946 Mbits/sec 0 949 KBytes </span></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">[ 5] 3.08-4.07 sec 111 MBytes 943 Mbits/sec 6 5.66 KBytes </span></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">[ 5] 4.07-5.07 sec 112 MBytes 941 Mbits/sec 0 705 KBytes </span></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">[ 5] 5.07-6.01 sec 106 MBytes 941 Mbits/sec 0 817 KBytes </span></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">[ 5] 6.01-7.07 sec 119 MBytes 942 Mbits/sec 0 915 KBytes </span></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">[ 5] 7.07-8.07 sec 112 MBytes 942 Mbits/sec 0 984 KBytes </span></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">[ 5] 8.07-9.07 sec 112 MBytes 942 Mbits/sec 0 1.01 MBytes </span></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">[ 5] 9.07-10.07 sec 113 MBytes 941 Mbits/sec 0 1.04 MBytes </span></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">- - - - - - - - - - - - - - - - - - - - - - - - -</span></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">[ ID] Interval Transfer Bitrate Retr</span></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">[ 5] 0.00-10.07 sec 1.07 GBytes 916 Mbits/sec 43 sender</span></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">[ 5] 0.00-10.08 sec 1.07 GBytes 914 Mbits/sec receiver</span></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255); min-height: 17px;"><span style="font-variant-ligatures: no-common-ligatures"></span><br></p> <p style="margin: 0px; font-width: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);"><span style="font-variant-ligatures: no-common-ligatures">iperf Done.</span></p></div><div><div><br></div><div>My Bhyve setup: https://wiki.freebsd.org/chengcui/install_FreeBSD13_VM_via_bhyve</div><div><br><blockquote type="cite"><div>On Oct 22, 2025, at 10:03, void <void@f-m.fm> wrote:</div><br class="Apple-interchange-newline"><div><div>Hi,<br><br>I wondered if any work was happening to address the long-standing issue<br>of slow vtnet? (in my case, this leads to slow freebsd guest performance<br>in bhyve compared with every linux guest tested and OpenBSD).<br><br>I thought maybe the recent bridge work may have ameliorated the problem some, but testing, after configuring a (recent) stable/15 server to use the ip-configured-on-bridge-method, showed no improvement using<br>a (recent) stable/14 guest<br><br>[ ID] Interval Transfer Bitrate Retr<br>[ 5] 0.00-60.03 sec 3.17 GBytes 454 Mbits/sec 0 sender<br>[ 5] 0.00-60.03 sec 3.17 GBytes 454 Mbits/sec receiver<br><br>for comparison, an ubuntu 25.04 guest:<br><br>[ ID] Interval Transfer Bitrate Retr<br>[ 5] 0.00-60.00 sec 6.42 GBytes 919 Mbits/sec 5427 sender<br>[ 5] 0.00-60.00 sec 6.42 GBytes 919 Mbits/sec receiver<br><br>the baremetal host<br><br>[ ID] Interval Transfer Bitrate Retr<br>[ 5] 0.00-60.01 sec 6.58 GBytes 941 Mbits/sec 0 sender<br>[ 5] 0.00-60.01 sec 6.58 GBytes 941 Mbits/sec receiver<br><br>Tests done a year and a half ago on similar hardware showing<br>the same patterns http://void.f-m.fm.user.fm/bhyve-virtio-testing.html<br>-- <br><br></div></div></blockquote></div><br><div> <div>Best Regards,<br>Cheng Cui</div><div><br></div><br class="Apple-interchange-newline"> </div> <br></div></body></html>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?51300B68-3D16-434E-8A65-696EC48FA98C>