Date: Wed, 29 Dec 2021 00:29:47 +0100 From: Marko Zec <zec@fer.hr> To: Gleb Smirnoff <glebius@freebsd.org> Cc: Kristof Provost <kp@freebsd.org>, src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: eb93b99d6986 - main - in_pcb: delay crfree() down into UMA dtor Message-ID: <20211229002947.7b7d5e94@x23> In-Reply-To: <YcpefigN9veLp0Jl@FreeBSD.org> References: <202112051847.1B5Il2GP030287@gitrepo.freebsd.org> <28AE53F1-2B22-444B-B1EC-1600FA741FE2@FreeBSD.org> <YbmEIsjf7RZBOOAJ@FreeBSD.org> <Yb19E4rQQ9%2BYxf48@FreeBSD.org> <YcpefigN9veLp0Jl@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 27 Dec 2021 16:46:54 -0800 Gleb Smirnoff <glebius@freebsd.org> wrote: > On Fri, Dec 17, 2021 at 10:17:55PM -0800, Gleb Smirnoff wrote: > T> T> K> The jails created by that test never go away. It=E2=80=99s as if= =20 > T> T> K> `crfree(inp->inp_cred);` doesn=E2=80=99t actually get called. An= d > T> T> K> indeed, it looks like inpcb_dtor() does not get called at > T> T> K> all. =20 > T> T>=20 > T> T> Yes, I faced this problem today, too. :( > T> T>=20 > T> T> My radical opinion is that per-VNET pcb zones should just be > T> T> eliminated. The only thing they serve is imposing maxsockets > T> T> limit separately for each VNET. But we already have the > T> T> maxsocket limit on the socket zone, which is _global_! > T> T>=20 > T> T> Anybody to explain me the sense of the per-VNET per-pcb zone > T> T> limit set to the same maxsockets value? You can't create a pcb > T> T> without a socket, which is guaranteed by the in_pcballoc() > T> T> prototype. Of course I understand that pcbs may outlive the > T> T> socket. But those pcbs that outlive a socket, are eventually > T> T> garbage collected as their lifetime is finite. Anyway jail/VNET > T> T> was never declared as a resource management framework anyway! > T> T>=20 > T> T> So, for this particular problem I would suggest just eliminate > T> T> per-VNET pcb zones, but in general the fact that idle SMR zone > T> T> may never purge its cache sucks and needs improvement. =20 > T>=20 > T> I have created a patch that would mitigate that problem. Once the > T> zones are global, the jails will eventually die if there is some > T> pcb zone traffic. > T>=20 > T> https://reviews.freebsd.org/D33542 =20 >=20 > Despite I still believe that PCB zones belong to global state rather > than to a VNET, the patch doesn't help to mitigate massive memory > leaks with vnet jails on a machine that is dedicated solely to run a > test suite. If machine does nothing except a test suite, there is > almost zero pcb traffic. If there is no pcb zone traffic, the SMR > caches stay, and thus destroyed jails will also stay. Our vnet jail > "weights" a lot! Even with the global PCB zone patch applied, each > vnet jail creates 33 UMA zones! >=20 > I think we need a KPI to purge the SMR caches, and we also need to put > vnet jails on a diet. These are two independent problems, of course. +1 for nuking all per-vnet PCB zones and the alike! At the time I V_irtualized them during the early stages of VNET implementaion, the focus was on correctnes and tracking of inter-vnet resource leaks. Once that step was reasonably completed (circa 15 years ago!), per-VNET zones became a pure waste of memory, amplified with per-CPU local free pools for each zone, not to mention the PITA with VNET cleanups... If memory still serves me well, a few folks asserted that per-VNET zones could be useful for hypothetical VNET snapshots / live migration to another machine, a project I heard about on several occasions but have never seen it... So, by all means go ahead and devirtualize them all... Cheers, Marko
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20211229002947.7b7d5e94>