Date: Mon, 16 Sep 2024 16:12:50 -0700 From: Chris <bsd-lists@bsdforge.com> To: Miroslav Lachman <000.fbsd@quip.cz> Cc: Sad Clouds <cryintothebluesky@gmail.com>, Zhenlei Huang <zlei@freebsd.org>, Mark Saad <nonesuch@longcount.org>, FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: Performance issues with vnet jails + epair + bridge Message-ID: <4a30743b7df5c5ca45386e69bebdbbe3@bsdforge.com> In-Reply-To: <cd68d032-36fd-4b10-a087-5bbc79b09cdb@quip.cz> References: <20240913100938.3eac55c9fbd976fa72d58bb5@gmail.com> <39B2C95D-1E4F-4133-8923-AD305DFA9435@longcount.org> <20240913155439.1e171a88bd01ce9b97558a90@gmail.com> <A95066A8-F5FC-451B-85CE-C463952ABADE@FreeBSD.org> <20240914112516.cfb31bae68ab90b83ca7ad4b@gmail.com> <CACA0VUjE43FUTaqAtXTur-4akQybKytv-oc1rHxwoUUXM3VQ=Q@mail.gmail.com> <20240915185654.b51cfec5aa2520e5b801cc87@gmail.com> <cd68d032-36fd-4b10-a087-5bbc79b09cdb@quip.cz>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2024-09-16 07:32, Miroslav Lachman wrote:
> On 15/09/2024 19:56, Sad Clouds wrote:
>> On Sun, 15 Sep 2024 18:01:07 +0100
>> Doug Rabson <dfr@rabson.org> wrote:
>>
>>> I just did a throughput test with iperf3 client on a FreeBSD 14.1 host
>>> with
>>> an intel 10GB nic connecting to an iperf3 server running in a vnet jail on
>>> a truenas host (13.something) also with an intel 10GB nic and I get full
>>> 10GB throughput in this setup. In the past, I had to disable LRO on the
>>> truenas host for this to work properly.
>>>
>>> Doug.
>>
>> Hello Doug, can you please confirm that you are NOT using if_epair(4)? I
>> imagine you dedicate one of the Intel 10Gb ports to a jail. This is not
>> an option for some of us, so a virtual NIC of some sort is the only
>> option with vnet jails. Other people also mentioned that vnet by itself
>> is not an issue and your test confirms this, however I'm observing poor
>> scalability specifically with the epair virtual NIC.
>>
>> I will be trying netgraph when I have some more time. If there are
>> other alternatives to if_epair then I would be interested to learn
>> about them.
>
> Try ngbuddy, it will help you with configuring netgraph.
>
> Or you can create second loopback interface, for example lo1 and put your
> jails on
> it. I don't know what your scenario is, but I used lo1 in the past with
> private
> IPs (10.x.x.x or 172.16.x.x) for jails not facing the outside world, just
> communicating with the host)
>
> You can put something similar to rc.conf:
>
> cloned_interfaces="lo1"
> # interface where all jails will bind
> ifconfig_lo1="inet 172.16.55.22 netmask 255.255.255.0"
I was going to suggest something like this as well. I often employ this with
jails. You can also facilitate exchanging traffic with the outside via pf(4)
(host) rc.conf(5):
cloned_interfaces="lo1"
pf_enable="YES"
pflog_enable="YES"
(host) pf.conf:
EXT_ADDR="192.168.1.2"
set skip on { lo0, lo1 }
nat pass on wlan0 from { lo1 } to any -> $EXT_ADDR
rdr pass on wlan0 proto tcp from any to { lo1 } -> $EXT_ADDR
Exchanging the EXT_ADDR value with your hosts NIC address. I use
the loopback interface as 120.0.0.1-N as needed.
--Chris
>
> Then run: service netif cloneup
>
> Kind regards
> Miroslav Lachman
--
sent from a device written from and running on FreeBSD
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4a30743b7df5c5ca45386e69bebdbbe3>