Date: Tue, 6 Apr 2010 12:24:29 +0300 From: Kostik Belousov <kostikbel@gmail.com> To: Petr Salinger <Petr.Salinger@seznam.cz> Cc: freebsd-hackers@freebsd.org Subject: Re: leak of the vnodes Message-ID: <20100406092429.GV2415@deviant.kiev.zoral.com.ua> In-Reply-To: <Pine.LNX.4.62.1004061011470.24982@sci.felk.cvut.cz> References: <Pine.LNX.4.62.1004022337010.16354@sci.felk.cvut.cz> <Pine.LNX.4.62.1004030915440.17349@sci.felk.cvut.cz> <20100403165238.GO2415@deviant.kiev.zoral.com.ua> <Pine.LNX.4.62.1004032106480.18491@sci.felk.cvut.cz> <20100403171945.GQ2415@deviant.kiev.zoral.com.ua> <Pine.LNX.4.62.1004032318210.18783@sci.felk.cvut.cz> <20100403210829.GW2415@deviant.kiev.zoral.com.ua> <Pine.LNX.4.62.1004052223210.23732@sci.felk.cvut.cz> <20100405185924.GS2415@deviant.kiev.zoral.com.ua> <Pine.LNX.4.62.1004061011470.24982@sci.felk.cvut.cz>
next in thread | previous in thread | raw e-mail | index | archive | help
--xaJmculcd5PV2weY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 06, 2010 at 11:56:28AM +0200, Petr Salinger wrote: > >>I would expect that sum of mnt_nvnodelistsize should be vfs.numvnodes. > >>The sum is at about 3400, but the vfs.numvnodes is at about 38000. > >>Is my expectation correct ? > >Not quite, reclaimed vnode is removed from mp list. >=20 > Are they in some other list ? No. Some code path does not properly release hold count on the vnode, most likely. Vnode is added to the free list when its hold count reaches zero and vnode is not doomed, doomed vnode is just freed. >=20 > >Can you check > >that vmstat -z | grep VNODE output coincides with sysctl vfs.numvnodes ? >=20 > I would say yes: >=20 > kern.maxvnodes: 31000 > kern.minvnodes: 2000 > vfs.freevnodes: 21 > vfs.wantfreevnodes: 2000 > vfs.numvnodes: 31011 > debug.vnlru_nowhere: 261 >=20 > ITEM SIZE LIMIT USED FREE REQUESTS =20 > FAILURES > VNODE: 480, 0, 30999, 777, 248903, = =20 > 0 > VNODEPOLL: 112, 0, 80, 85, 80, = =20 > 0 >=20 > But still: >=20 > 0xffffff0003939be0 /dev/ad0s2a on / (ufs) > mnt_nvnodelistsize =3D 570 >=20 > 0xffffff000393a000 devfs on /dev (devfs) > mnt_nvnodelistsize =3D 41 >=20 > 0xffffff0003939000 /dev/ad0s2d on /opt/sid/build (ufs) > mnt_nvnodelistsize =3D 52 >=20 > There is 30000 vnodes somewhere. Right. The leak is there. >=20 > >Also, can you unmount linprocfs before the test and see whether it > >leaks as before ? > >Another thing to try is set sysctl debug.disablefullpath to 1. >=20 > Done both before this test, still "hangs/leaks": >=20 > vfs.cache.numfullpathfound: 29075 > vfs.cache.numfullpathfail4: 0 > vfs.cache.numfullpathfail2: 0 > vfs.cache.numfullpathfail1: 0 > vfs.cache.numfullpathcalls: 29075 > debug.disablefullpath: 1 Can you try to narrow down the sequence of operations that is needed to reproduce the leak ? If not, you may add vn_printf() + ddb_backtrace to vgonel(), but this definitely cause immense amount of spam on the console. I am not sure how easy would it be to spot the leaking vnodes. --xaJmculcd5PV2weY Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAku6/cwACgkQC3+MBN1Mb4gHWgCcCkLZCD1xOODcLwhDWO7Bp3/4 iDAAoNnK3q/zSRVbaFFx7zmHQ+1OtnFv =CMvx -----END PGP SIGNATURE----- --xaJmculcd5PV2weY--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100406092429.GV2415>