Date: Fri, 1 Nov 2024 12:42:37 +0200 From: Andriy Gapon <avg@FreeBSD.org> To: freebsd-fs <fs@FreeBSD.org> Subject: Re: vn_alloc_cyclecount is always zero? Message-ID: <7b6475ca-ef66-474e-9084-2bc61301c31b@FreeBSD.org> In-Reply-To: <1181f483-7e88-4333-9b22-c76146ec82b2@FreeBSD.org> References: <1181f483-7e88-4333-9b22-c76146ec82b2@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 01/11/2024 12:24, Andriy Gapon wrote: > > It seems that after commit https://cgit.freebsd.org/src/commit/?id=8733bc277 > vn_alloc_cyclecount is always zero as the only code that modifies the variable > is under a condition that the variable is already non-zero while its initial > value is zero. > > That does not look right because there is some consequential code that it is > conditional on vn_alloc_cyclecount. > E.g.: > - vstir is never set; > - vn_alloc_hard never reaches 'goto alloc'. > > I checked compiled code just in case and it looks that the compiler agrees with > me: vn_alloc_cyclecount is optimized out as well as all code depending on it not > being zero. > > The vnode recycling code (with its many parameters and paths) has never been > easy for me to understand, so I am not sure about all implications but there is > one real-world problem that prompted me to look into the code. Not to be unnecessarily cryptic, let me describe the problem. vnode_list has hundreds of thousands of vnodes (close to desiredvnodes), all of them either being held (v_holdcnt > 0) or being already freed (dead vnops, v_holdcnt == VHOLD_NO_SMR). Then, there is a thread looping over the list in vnlru_free_impl, fruitlessly, while holding and never dropping vnode_list_mtx. Then, there is a multitude of other threads blocked on vnode_list_mtx in various places, mostly different locations in vn_alloc_hard. Among those threads is vnlru_proc thread which is blocked on vnode_list_mtx in vlrureclaim. There are a lot of vnodes whose use count is zero (held only by the name cache?) but vnlru_proc rarely gets a chance to run because of all other threads that eventually directly call into vnlru_free_impl and grab the lock for a long time. It does not help that vlrureclaim is super nice to others by periodically dropping the lock and yielding. So whatever freeable vnodes it creates get quickly consumed. Overall, there is a super long delay for each vnode allocation, so the system feels totally unresponsive. I may have misunderstood some details of the problem and the code, but hope that my description paints a good picture for those who know the code. I believe that vn_alloc_hard never reaching 'goto alloc' is a big part of the problem. -- Andriy Gapon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7b6475ca-ef66-474e-9084-2bc61301c31b>