From nobody Mon Sep 16 23:12:50 2024 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4X71101WTGz5WN9w for ; Mon, 16 Sep 2024 23:13:08 +0000 (UTC) (envelope-from bsd-lists@bsdforge.com) Received: from udns.ultimatedns.net (udns.ultimatedns.net [24.113.41.81]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "ultimatedns.net", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4X710z57jyz4XGx; Mon, 16 Sep 2024 23:13:07 +0000 (UTC) (envelope-from bsd-lists@bsdforge.com) Authentication-Results: mx1.freebsd.org; none Received: from ultimatedns.net (localhost [127.0.0.1]) by udns.ultimatedns.net (8.16.1/8.16.1) with ESMTP id 48GNCoVi026100; Mon, 16 Sep 2024 16:12:57 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ultimatedns.net; s=mx99; t=1726528385; x=1726528985; r=y; bh=rFKAiqQZEmTmPWUP0kMhJHZmQM+7rhqy3kSNp65DqnQ=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=vH2e8I3IzWCsaNwKHaZ0PMqJAVp3Ov9vTALe/4USuH+Ao3erI4acFXvJH4arSLoCf ps3burISN1wBIIob+2ZQ2Vi23Q4+/cSG8al4QYtw0h7MjZTKnQsPMVsdKu533Ntnm4 nW9W5x/ypsFgstv//2W6h/MkRbXPwsbP2rjibtolMclmZo4XWFHdQLvLOMLVnAa2pZ 3EGmpHrmb0hxnGv0OYOW8yUOS9ZonlhHWtrPdS6fG9Q7sAsQpTx6CXJBpv8p7PReP0 uc/2NPaTX00R16FCPN7oUU7aGFWMXcEIixZeBuENQClfxOmqAOO2NH/YjXGLzAho24 YVG3TrhWjwHsg== List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org MIME-Version: 1.0 Date: Mon, 16 Sep 2024 16:12:50 -0700 From: Chris To: Miroslav Lachman <000.fbsd@quip.cz> Cc: Sad Clouds , Zhenlei Huang , Mark Saad , FreeBSD Net Subject: Re: Performance issues with vnet jails + epair + bridge In-Reply-To: References: <20240913100938.3eac55c9fbd976fa72d58bb5@gmail.com> <39B2C95D-1E4F-4133-8923-AD305DFA9435@longcount.org> <20240913155439.1e171a88bd01ce9b97558a90@gmail.com> <20240914112516.cfb31bae68ab90b83ca7ad4b@gmail.com> <20240915185654.b51cfec5aa2520e5b801cc87@gmail.com> User-Agent: UDNSMS/17.0 Message-ID: <4a30743b7df5c5ca45386e69bebdbbe3@bsdforge.com> X-Sender: bsd-lists@bsdforge.com Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:11404, ipnet:24.113.0.0/16, country:US] X-Rspamd-Queue-Id: 4X710z57jyz4XGx On 2024-09-16 07:32, Miroslav Lachman wrote: > On 15/09/2024 19:56, Sad Clouds wrote: >> On Sun, 15 Sep 2024 18:01:07 +0100 >> Doug Rabson wrote: >> >>> I just did a throughput test with iperf3 client on a FreeBSD 14.1 host >>> with >>> an intel 10GB nic connecting to an iperf3 server running in a vnet jail on >>> a truenas host (13.something) also with an intel 10GB nic and I get full >>> 10GB throughput in this setup. In the past, I had to disable LRO on the >>> truenas host for this to work properly. >>> >>> Doug. >> >> Hello Doug, can you please confirm that you are NOT using if_epair(4)? I >> imagine you dedicate one of the Intel 10Gb ports to a jail. This is not >> an option for some of us, so a virtual NIC of some sort is the only >> option with vnet jails. Other people also mentioned that vnet by itself >> is not an issue and your test confirms this, however I'm observing poor >> scalability specifically with the epair virtual NIC. >> >> I will be trying netgraph when I have some more time. If there are >> other alternatives to if_epair then I would be interested to learn >> about them. > > Try ngbuddy, it will help you with configuring netgraph. > > Or you can create second loopback interface, for example lo1 and put your > jails on > it. I don't know what your scenario is, but I used lo1 in the past with > private > IPs (10.x.x.x or 172.16.x.x) for jails not facing the outside world, just > communicating with the host) > > You can put something similar to rc.conf: > > cloned_interfaces="lo1" > # interface where all jails will bind > ifconfig_lo1="inet 172.16.55.22 netmask 255.255.255.0" I was going to suggest something like this as well. I often employ this with jails. You can also facilitate exchanging traffic with the outside via pf(4) (host) rc.conf(5): cloned_interfaces="lo1" pf_enable="YES" pflog_enable="YES" (host) pf.conf: EXT_ADDR="192.168.1.2" set skip on { lo0, lo1 } nat pass on wlan0 from { lo1 } to any -> $EXT_ADDR rdr pass on wlan0 proto tcp from any to { lo1 } -> $EXT_ADDR Exchanging the EXT_ADDR value with your hosts NIC address. I use the loopback interface as 120.0.0.1-N as needed. --Chris > > Then run: service netif cloneup > > Kind regards > Miroslav Lachman -- sent from a device written from and running on FreeBSD