From nobody Sun Sep 15 17:56:54 2024 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4X6G2d5mDcz5WWpD for ; Sun, 15 Sep 2024 17:56:57 +0000 (UTC) (envelope-from cryintothebluesky@gmail.com) Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4X6G2d3z25z4m4c; Sun, 15 Sep 2024 17:56:57 +0000 (UTC) (envelope-from cryintothebluesky@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-wm1-x32b.google.com with SMTP id 5b1f17b1804b1-42cba8340beso29394435e9.1; Sun, 15 Sep 2024 10:56:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726423016; x=1727027816; darn=freebsd.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=dA6LzmCxBV/8elNsi//MgmiyvV9RKKTFmerxKxOoMvI=; b=UCQto/R+uruu8JCexGBLjEX6+KM3LoG3oEA4crIkbWgXoV4UHfDZL5tmqKOqj3bf2+ BJxUWrSghKNWxfwmW4VP5G8PpbShUzfjqhutsk03g10zPO1qLgCZk1a8gPPYB9bVkGXK BlSkVbhJlE5ICjwVPmVi2mnTALMJZtBa4gu8HMeEC4H3xQLhhVxD+TDVTgcJWUgj7sVL +eIfDKYqbtdHNkUY1C3AMGveffpneMUo/GXLR0YRrhFtuXFOK9G50Sv/oVRr8asB8IdC NnmXRsTEf+l6XNdx/ahM6C4WpNiYP3YWNdaDECJd+Gz2rnpmpe37Yak+l5Dsx3QwYfbd LqSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726423016; x=1727027816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dA6LzmCxBV/8elNsi//MgmiyvV9RKKTFmerxKxOoMvI=; b=iXHgmnIx6ECCEyFfcIiibbFwmKJmQf3m3LhyPbv1tosXH+9FvQ8aE/Z/4FBaE0H67l dr7ZGKHNBLiPp5knxKpileey0zAqtZ0IW5yhUk2hVIQeJtmP2qEFHSTs5AkTj+1X+6BY In4kKxfuGXDfcPTpW5dJtP9PNIAu40ISHNYupdQA1bZujLcgLtAiFvrtCA5qRd9sMqV8 SxOW/VgMUi+reoIXsRvk3xNvcoaGiF48ntq/TPxwenc8TeodXFW/5vZozgE7d9m/tYRi 6zsvnXHjSEeFPZIDG2FiXT4NV8L1S0zA5oT/tg7D7CsoTiNpRs4Xu6kukO4PfVE8V167 Qs6w== X-Forwarded-Encrypted: i=1; AJvYcCXug7ieGEMVFSe7qpPIL5c8KL7waGKf/GHKiZGPIKYDnFe0wnrobQX4huZSriVHgMnCRZvuIpRZy2h9wg==@freebsd.org X-Gm-Message-State: AOJu0YxFwbpgjyqEKc+GOdDGbLzgKqIumuqQEhM72Sorq/BD5pg7mgZo 4erhu8f4Yq38a/Y720GStP8q+so+xKa6nOyQXhk9xCx/DQ7O2MdzLqckOw== X-Google-Smtp-Source: AGHT+IFj3q1mslPmunZIfzULqM8ox8w2G1G5uzTtRkJKQVxxaTunSD/mxzWwP+nhI7vyn5+74gPC2w== X-Received: by 2002:a05:6000:20c3:b0:378:c6d5:e2b3 with SMTP id ffacd0b85a97d-378c6d5e608mr7844277f8f.23.1726423015782; Sun, 15 Sep 2024 10:56:55 -0700 (PDT) Received: from z600.home.lan (28.129.159.143.dyn.plus.net. [143.159.129.28]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42d9b189d0dsm88389275e9.33.2024.09.15.10.56.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Sep 2024 10:56:55 -0700 (PDT) Date: Sun, 15 Sep 2024 18:56:54 +0100 From: Sad Clouds To: Doug Rabson Cc: Zhenlei Huang , Mark Saad , FreeBSD Net Subject: Re: Performance issues with vnet jails + epair + bridge Message-Id: <20240915185654.b51cfec5aa2520e5b801cc87@gmail.com> In-Reply-To: References: <20240913100938.3eac55c9fbd976fa72d58bb5@gmail.com> <39B2C95D-1E4F-4133-8923-AD305DFA9435@longcount.org> <20240913155439.1e171a88bd01ce9b97558a90@gmail.com> <20240914112516.cfb31bae68ab90b83ca7ad4b@gmail.com> X-Mailer: Sylpheed 3.8.0beta1 (GTK+ 2.24.33; x86_64-pc-linux-gnu) List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4X6G2d3z25z4m4c On Sun, 15 Sep 2024 18:01:07 +0100 Doug Rabson wrote: > I just did a throughput test with iperf3 client on a FreeBSD 14.1 host with > an intel 10GB nic connecting to an iperf3 server running in a vnet jail on > a truenas host (13.something) also with an intel 10GB nic and I get full > 10GB throughput in this setup. In the past, I had to disable LRO on the > truenas host for this to work properly. > > Doug. Hello Doug, can you please confirm that you are NOT using if_epair(4)? I imagine you dedicate one of the Intel 10Gb ports to a jail. This is not an option for some of us, so a virtual NIC of some sort is the only option with vnet jails. Other people also mentioned that vnet by itself is not an issue and your test confirms this, however I'm observing poor scalability specifically with the epair virtual NIC. I will be trying netgraph when I have some more time. If there are other alternatives to if_epair then I would be interested to learn about them.