From nobody Thu Nov 20 23:22:36 2025 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4dCDsY0VzHz6J61d for ; Thu, 20 Nov 2025 23:22:41 +0000 (UTC) (envelope-from void@f-m.fm) Received: from fout-b4-smtp.messagingengine.com (fout-b4-smtp.messagingengine.com [202.12.124.147]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4dCDsX0tWRz4LXc for ; Thu, 20 Nov 2025 23:22:40 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=TRMcpxD1; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=ZkowvIvJ; dmarc=pass (policy=none) header.from=f-m.fm; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 202.12.124.147 as permitted sender) smtp.mailfrom=void@f-m.fm Received: from phl-compute-06.internal (phl-compute-06.internal [10.202.2.46]) by mailfout.stl.internal (Postfix) with ESMTP id DECF81D001C0 for ; Thu, 20 Nov 2025 18:22:38 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-06.internal (MEProxy); Thu, 20 Nov 2025 18:22:38 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1763680958; x=1763767358; bh=SH0E3HsB/z vQrpacteQgjwz8O8yk0zNwLTis2LZoymI=; b=TRMcpxD1ccGXHkiJC2uk67bqdE 5+VXjj1tz/z0fBmd5pJS9y3x+wl+XrX7CN0eI1r/qS9k6I4YKiDUUUNo2z9jCjEZ t8jXLrWIm2V4Zanguo61zIKffo1+16X4sgBmddWyuXlqwjZhACwccQIHX6o5k84s uCqSKsV9YWWUnD6c7/NP1hM+k08x+DSdVeKM2z33/tR+pVPN7nPa32sAJLrnm+qV LeWUUnL3NBhwMitNwdeFV0C4a1G0ohXk+5zWPMjEUDtu82XQOswcawi1vQ1XLz4b hbBT6zmBEsMmeLiI5rx3cZXPJ4hQl7HXs7j7Ff4M9hIo2XxmxQCzYYtVNTtQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1763680958; x=1763767358; bh=SH0E3HsB/zvQrpacteQgjwz8O8yk0zNwLTi s2LZoymI=; b=ZkowvIvJipgjOFT54SMyOLkqqpwKqW3Gb255P4STM4k6xzMNJDk fvZDOnAB2MzRx1yzJ5aG0AegDXvNtWHV+5qa57rOiLv0ESQ4e8OorcfRR6ZTz2tO zVRkh4u+hNKPOFrvPyS0QQPPlu+bC3V4hjbimdoL+rEUxUexFmMqBjfGlIPQQvn5 MTDQxm+8FZK2ShegUg3bhmTtObd/x4HWVOOSs2IGBPaRza9QKZ9W67T2jjrPDRK7 HCha64DFlR3tfwI2d4NX8ufGA2jz0SJrUPlOLdiABBkM0YRx46p6E5CGoKJ0RZhK YNmjr75/zEcGbgUrq7hyk5tJVAYTWFBzJUA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggddvvdekfeelucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfurfetoffkrfgpnffqhgenuceu rghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesthdtredttd dtvdenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhmrdhfmheqnecuggftrfgrthht vghrnhepkeeluddvlefhieelfefggffhffektdehleelgfdugfdvgeekjeejuddtheehgf eunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepvhho ihgusehfqdhmrdhfmhdpnhgspghrtghpthhtohepuddpmhhouggvpehsmhhtphhouhhtpd hrtghpthhtohepfhhrvggvsghsugdqnhgvthesfhhrvggvsghsugdrohhrgh X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Thu, 20 Nov 2025 18:22:38 -0500 (EST) Date: Thu, 20 Nov 2025 23:22:36 +0000 From: void To: freebsd-net@freebsd.org Subject: Re: vtnet is much improved in 16:current (nodebug) Message-ID: References: List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.60 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; NEURAL_HAM_SHORT(-1.00)[-0.998]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:202.12.124.128/27]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[202.12.124.147:from]; FREEMAIL_FROM(0.00)[f-m.fm]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_THREE(0.00)[3]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; ASN(0.00)[asn:151847, ipnet:202.12.124.0/24, country:AU]; RCVD_VIA_SMTP_AUTH(0.00)[]; MISSING_XM_UA(0.00)[]; DWL_DNSWL_NONE(0.00)[messagingengine.com:dkim] X-Rspamd-Queue-Id: 4dCDsX0tWRz4LXc On Wed, Nov 19, 2025 at 11:24:01AM -0500, Cheng Cui wrote: >That's good news! :) I did some very general tests: first ones were one vm upgraded over several iterations. This vm being enc zfs mounted as a zvol on a raidz1 zpool (spinning rust). Nothing changed apart from the OS version The tl;dr is that 16.0-CURRENT ufs2 nodebug kernel & userland is the clear winner in terms of throughput; it is almost equal (941 Mbits/sec) to line level. freebsd 14.3-p3 =============== iperf3 -B 192.168.1.121 -c 192.168.1.230 -l 1M -t 30 -i 2 ; netstat -nCRp tcp | egrep "Active|Proto|5201" - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.01 sec 1.38 GBytes 396 Mbits/sec 0 sender [ 5] 0.00-30.01 sec 1.38 GBytes 396 Mbits/sec receiver stable/15-n281161-fc22812b6685 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.03 sec 1.57 GBytes 450 Mbits/sec 0 sender [ 5] 0.00-30.03 sec 1.57 GBytes 450 Mbits/sec receiver 16.0-CURRENT - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.13 sec 1.77 GBytes 506 Mbits/sec 0 sender [ 5] 0.00-30.13 sec 1.77 GBytes 506 Mbits/sec receiver ==================================================== These next few tests exploring differences whether vm on zpool host was ufs backed, zfs backed, zvol backed or zvol with compression whether these factors influenced throughput. all 16.0-current 16.0-CURRENT ufs2 nodebug kernel & userland - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.05 sec 3.28 GBytes 939 Mbits/sec 0 sender [ 5] 0.00-30.05 sec 3.28 GBytes 938 Mbits/sec receiver 16-current ufs2 installed the new way - debug kernel & userland - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.07 sec 1.88 GBytes 537 Mbits/sec 0 sender [ 5] 0.00-30.09 sec 1.88 GBytes 537 Mbits/sec receiver as above, -rxcsum -txcsum -tso -tso6 -lro - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.16 sec 1.46 GBytes 417 Mbits/sec 0 sender [ 5] 0.00-30.15 sec 1.46 GBytes 417 Mbits/sec receiver as above, nodebug kernel - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.13 sec 3.01 GBytes 858 Mbits/sec 0 sender [ 5] 0.00-30.15 sec 3.01 GBytes 858 Mbits/sec receiver as above, nodebug kernel & world - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.01 sec 3.08 GBytes 881 Mbits/sec 0 sender [ 5] 0.00-30.02 sec 3.08 GBytes 881 Mbits/sec receiver 16-current installed the traditional way - zfs & debug kernel - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.03 sec 2.70 GBytes 772 Mbits/sec 0 sender [ 5] 0.00-30.03 sec 2.70 GBytes 771 Mbits/sec receiver as above, -rxcsum -txcsum -tso -tso6 -lro - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.05 sec 2.68 GBytes 765 Mbits/sec 0 sender [ 5] 0.00-30.05 sec 2.68 GBytes 765 Mbits/sec receiver as above, nodebug kernel - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.02 sec 2.92 GBytes 835 Mbits/sec 0 sender [ 5] 0.00-30.03 sec 2.92 GBytes 835 Mbits/sec receiver as above, nodebug kernel & userland - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.04 sec 2.96 GBytes 846 Mbits/sec 0 sender [ 5] 0.00-30.04 sec 2.96 GBytes 846 Mbits/sec receiver 16-current installed the traditional way - zfs & debug kernel - fileimage, encrypted zfs - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.04 sec 1.90 GBytes 544 Mbits/sec 0 sender [ 5] 0.00-30.04 sec 1.90 GBytes 544 Mbits/sec receiver as above, -rxcsum -txcsum -tso -tso6 -lro - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.11 sec 1.79 GBytes 510 Mbits/sec 0 sender [ 5] 0.00-30.12 sec 1.79 GBytes 509 Mbits/sec receiver nodebug kernel -rxcsum -txcsum -tso -tso6 -lro - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.01 sec 2.89 GBytes 826 Mbits/sec 0 sender [ 5] 0.00-30.02 sec 2.89 GBytes 826 Mbits/sec receiver as above, offloading enabled - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.01 sec 2.88 GBytes 824 Mbits/sec 0 sender [ 5] 0.00-30.02 sec 2.88 GBytes 824 Mbits/sec receiver nodebug enccrypted on zvol - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.09 sec 1.86 GBytes 530 Mbits/sec 0 sender [ 5] 0.00-30.10 sec 1.86 GBytes 530 Mbits/sec receiver as above, -rxcsum -txcsum -tso -tso6 -lro - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.08 sec 1.60 GBytes 456 Mbits/sec 0 sender [ 5] 0.00-30.08 sec 1.60 GBytes 456 Mbits/sec receiver as above, offloading enabled - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.02 sec 2.26 GBytes 646 Mbits/sec 0 sender [ 5] 0.00-30.03 sec 2.26 GBytes 646 Mbits/sec receiver --