Date: Mon, 4 Jan 2021 01:11:26 +0000 From: Rick Macklem <rmacklem@uoguelph.ca> To: Konstantin Belousov <kostikbel@gmail.com> Cc: J David <j.david.lists@gmail.com>, "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org> Subject: Re: Major issues with nfsv4 Message-ID: <YQXPR0101MB0968836D68FBBB0C19ACC2B4DDD20@YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM> In-Reply-To: <X%2BitPGhDz6Vldqh%2B@kib.kiev.ua> References: <YQXPR0101MB0968B17010B3B36C8C41FDE1DDC90@YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM> <X9Q9GAhNHbXGbKy7@kib.kiev.ua> <YQXPR0101MB0968C7629D57CA21319E50C2DDC90@YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM> <X9UDArKjUqJVS035@kib.kiev.ua> <CABXB=RRNnW9nNhFCJS1evNUTEX9LNnzyf2gOmZHHGkzAoQxbPw@mail.gmail.com> <YQXPR0101MB0968B120A417AF69CEBB6A12DDC80@YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM> <X9aGwshgh7Cwiv8p@kib.kiev.ua> <CABXB=RTFSAEZvp%2BmoiF%2BrE9vpEjJVacLYa6G=yP641f9oHJ1zw@mail.gmail.com> <YQXPR0101MB09681D2CB8FBD5DDE907D5A5DDC40@YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM> <YQXPR0101MB096897CA4344DFDC8D22DFE9DDDB0@YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM>, <X%2BitPGhDz6Vldqh%2B@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
Kostik wrote:=0A=
>Rick Macklem wrote:=0A=
[stuff snipped]=0A=
>> I see vfs_deferred_inactive() in sys/kern/vfs_subr.c, but I do not=0A=
>> know when/how it gets called?=0A=
>Right, vfs_deferred_inactive() is one way which tries to handle missed=0A=
>inactivations. If upon vput() the lock is only shared and upgrade=0A=
>failed, vnode is marked as VI_OWEINACT and put onto 'lazy' list,=0A=
>processed by vfs_sync(MNT_LAZY). It is typically called from syncer,=0A=
>which means each 60 secs. There, if the vnode is still unreferenced, it=0A=
>is inactivated.=0A=
If I read the code correctly vfs_deferred_inactive() gets called with=0A=
LK_EXCLUSIVE | LK_NOWAIT | LK_INTERLOCK.=0A=
Then there is this code in nfs_lock():=0A=
vp =3D ap->a_vp;=0A=
332 lktype =3D ap->a_flags & LK_TYPE_MASK;=0A=
333 error =3D VOP_LOCK1_APV(&default_vnodeops, ap);=0A=
334 if (error !=3D 0 || vp->v_op !=3D &newnfs_vnodeops)=0A=
335 return (error);=0A=
336 np =3D VTONFS(vp);=0A=
337 if (np =3D=3D NULL)=0A=
338 return (0);=0A=
339 NFSLOCKNODE(np);=0A=
340 if ((np->n_flag & NVNSETSZSKIP) =3D=3D 0 || (lktype !=3D LK_SHA=
RED &&=0A=
341 lktype !=3D LK_EXCLUSIVE && lktype !=3D LK_UPGRADE &&=0A=
342 lktype !=3D LK_TRYUPGRADE)) {=0A=
343 NFSUNLOCKNODE(np);=0A=
344 return (0);=0A=
345 }=0A=
346 onfault =3D (ap->a_flags & LK_EATTR_MASK) =3D=3D LK_NOWAIT &&=
=0A=
347 (ap->a_flags & LK_INIT_MASK) =3D=3D LK_CANRECURSE &&=0A=
348 (lktype =3D=3D LK_SHARED || lktype =3D=3D LK_EXCLUSIVE);=0A=
349 if (onfault && vp->v_vnlock->lk_recurse =3D=3D 0) {=0A=
350 /*=0A=
351 * Force retry in vm_fault(), to make the lock request=
=0A=
352 * sleepable, which allows us to piggy-back the=0A=
353 * sleepable call to vnode_pager_setsize().=0A=
354 */=0A=
355 NFSUNLOCKNODE(np);=0A=
356 VOP_UNLOCK(vp);=0A=
357 return (EBUSY);=0A=
358 }=0A=
Would the above code result in an EBUSY reply when the vn_lock(LK_NOWAIT |=
=0A=
LK_EXCLUSIVE) is made in vfs_deferred_inactive()?=0A=
=0A=
If so, it looks like vfs_periodic_inactive()->vfs_deferred_inactive() will =
just=0A=
call vdefer_inactive_unlocked()->vdefer_inactive() and it will end up=0A=
on the lazy list with VI_DEFINACT set again, if I am reading the=0A=
code correctly?=0A=
=0A=
>Another place where inactivation can occur is reclamation. There in=0A=
>vgonel(), we call VOP_INACTIVE() if VI_OWEINACT is set. In principle,=0A=
>this is redundand because correct filesystem must do the same cleanup=0A=
>(and more) at reclamation as at the inactivation. But we also call=0A=
>VOP_CLOSE(FNONBLOCK) before VOP_RECLAIM().=0A=
And this would make all the NFSv4 Opens go away (get closed)=0A=
when the nullfs mounts are unmounted, as reported by J. David.=0A=
=0A=
>Looking at this from another angle, if inactivation for NFSv4 vnodes=0A=
>is not called longer than 2 minutes, perhaps there is a reference leak.=0A=
>It is not due to VFS forgetting about due VOP_INACTIVE() call.=0A=
Unless the nfs_lock(LK_NOWAIT) call keeps failing with EBUSY,=0A=
I think?=0A=
=0A=
rick=0A=
ps: I need to find a way to reproduce this.=0A=
=0A=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?YQXPR0101MB0968836D68FBBB0C19ACC2B4DDD20>