Date: Sat, 14 Sep 2024 10:45:03 +0800 From: Zhenlei Huang <zlei@FreeBSD.org> To: Sad Clouds <cryintothebluesky@gmail.com> Cc: Mark Saad <nonesuch@longcount.org>, FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: Performance issues with vnet jails + epair + bridge Message-ID: <A95066A8-F5FC-451B-85CE-C463952ABADE@FreeBSD.org> In-Reply-To: <20240913155439.1e171a88bd01ce9b97558a90@gmail.com> References: <20240913100938.3eac55c9fbd976fa72d58bb5@gmail.com> <39B2C95D-1E4F-4133-8923-AD305DFA9435@longcount.org> <20240913155439.1e171a88bd01ce9b97558a90@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Sep 13, 2024, at 10:54 PM, Sad Clouds <cryintothebluesky@gmail.com> = wrote: >=20 > On Fri, 13 Sep 2024 08:08:02 -0400 > Mark Saad <nonesuch@longcount.org> wrote: >=20 >> Sad >> Can you go back a bit you mentioned there is a RPi in the mix ? = Some of the raspberries have their nic usb attached under the covers . = Which will kill the total speed of things.=20 >>=20 >> Can you cobble together a diagram of what you have on either end ? >=20 > Hello, I'm not sending data across the network, only between the host > and the jails. I'm trying to evaluate how FreeBSD handles TCP data > locally within a single host. When you take vnet into account, the **locally** traffic should within on single vnet jail. If you want traffic across vnet jails, if_epair or = netgraph hooks should be employed, and it of course will introduce some overhead. >=20 > I understand that vnet jails will have more overhead, compared to a > shared TCP/IP stack via localhost. So I'm trying to measure it and see > where the bottlenecks are. The overhead of vnet jail should be neglectable, compared to legacy jail or no-jail. Bare in mind when VIMAGE option is enabled, there is a = default vnet 0. It is not visible via jls and can not be destroyed. So when you = see bottlenecks, for example this case, it is mostly caused by other = components such as if_epair, but not the vnet jail itself. >=20 > The Raspberry Pi 4 host has a single vnet jail, exchanging data with > the host via epair(4) and if_bridge(4) interfaces. I don't really know > what topology FreeBSD is using to represent all this so can't draw any > diagrams, but I think all data flows through the kernel internally and > never leaves the physical network interface. For vnet jails, when you try to describe the network topology, you can treat them as VM / physical boxes. I have one box with dozens of vnet jails. Each of them has very single responsibility, e.g. DHCP, LADP, pf firewall, OOB access. The topology = looks quite clear and it is easy to maintenance. The only overhead is too much hops between the vnet jail instances. For my use case the performance is not critical and it works great for years. >=20 Best regards, Zhenlei
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A95066A8-F5FC-451B-85CE-C463952ABADE>