From owner-freebsd-stable@FreeBSD.ORG Tue May 7 22:36:50 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 4CE9EE5A; Tue, 7 May 2013 22:36:50 +0000 (UTC) (envelope-from dewayne.geraghty@heuristicsystems.com.au) Received: from nschwmtas03p.mx.bigpond.com (nschwmtas03p.mx.bigpond.com [61.9.189.143]) by mx1.freebsd.org (Postfix) with ESMTP id 8B9DB74B; Tue, 7 May 2013 22:36:49 +0000 (UTC) Received: from nschwcmgw05p ([61.9.190.165]) by nschwmtas03p.mx.bigpond.com with ESMTP id <20130507223641.IOP2008.nschwmtas03p.mx.bigpond.com@nschwcmgw05p>; Tue, 7 May 2013 22:36:41 +0000 Received: from hermes.heuristicsystems.com.au ([58.172.113.247]) by nschwcmgw05p with BigPond Outbound id ZAcg1l00R5LKYmq01AcgPw; Tue, 07 May 2013 22:36:41 +0000 X-Authority-Analysis: v=2.0 cv=QJDqt33L c=1 sm=1 a=YibVxx38Z+cwdCKSMcELyg==:17 a=k4yzAXmc-yEA:10 a=8nJEP1OIZ-IA:10 a=GHIR_BbyAAAA:8 a=RmBvt3o0NWIA:10 a=6I5d2MoRAAAA:8 a=J8tqD64_rZ3NdKCdztYA:9 a=wPNLvfGTeEIA:10 a=SV7veod9ZcQA:10 a=uhHikAKd3b9-GwYN:21 a=3gnV2ZkxXiNlYijp:21 a=YibVxx38Z+cwdCKSMcELyg==:117 Received: from black (black.hs [10.0.5.1]) (authenticated bits=0) by hermes.heuristicsystems.com.au (8.14.5/8.13.6) with ESMTP id r47MZSg0091180 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NOT); Wed, 8 May 2013 08:35:29 +1000 (EST) (envelope-from dewayne.geraghty@heuristicsystems.com.au) From: "Dewayne Geraghty" To: "'Mikolaj Golub'" , "=?iso-8859-1?Q?'G=F6ran_L=F6wkrantz'?=" References: <20130507204149.GA3267@gmail.com> Subject: RE: Nullfs leaks i-nodes Date: Wed, 8 May 2013 08:35:18 +1000 Message-ID: <56EF269F84824D8DB413D289BB8CBE19@as.lan> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 11 In-Reply-To: <20130507204149.GA3267@gmail.com> Thread-Index: Ac5LY1ryw14pMFB/RPi2FDMaUDaRngAC9cCw X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-stable@freebsd.org, 'Kostik Belousov' X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 May 2013 22:36:50 -0000 > -----Original Message----- > From: owner-freebsd-stable@freebsd.org=20 > [mailto:owner-freebsd-stable@freebsd.org] On Behalf Of Mikolaj Golub > Sent: Wednesday, 8 May 2013 6:42 AM > To: G=F6ran L=F6wkrantz > Cc: Kostik Belousov; freebsd-stable@freebsd.org > Subject: Re: Nullfs leaks i-nodes >=20 > On Tue, May 07, 2013 at 08:30:06AM +0200, G=F6ran L=F6wkrantz wrote: > > I created a PR, kern/178238, on this but would like to know=20 > if anyone has=20 > > any ideas or patches? > >=20 > > Have updated the system where I see this to FreeBSD=20 > 9.1-STABLE #0 r250229=20 > > and still have the problem. >=20 > I am observing an effect that might look like inode leak, which I > think is due free nullfs vnodes caching, recently added by kib > (r240285): free inode number does not increase after unlink; but if I > purge the free vnodes cache (temporary setting vfs.wantfreevnodes to 0 > and observing vfs.freevnodes decreasing to 0) the inode number grows > back. >=20 > You have only about 1000 inodes available on your underlying fs, while > vfs.wantfreevnodes I think is much higher, resulting in running out of > i-nodes. >=20 > If it is really your case you can disable caching, mounting nullfs > with nocache (it looks like caching is not important in your case). >=20 > --=20 > Mikolaj Golub > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to=20 > "freebsd-stable-unsubscribe@freebsd.org" >=20 Hi Goran, After I included Kib's vnode caching patch the performance on my "port = builder" machine, decreased significantly. The "port builder" is one of many jails and nullfs is used extensively. I was = starving the system of vnodes. Increasing the kern.maxvnodes, resulted in better performance than the original system configuration = without vnode caching. Thanks Kib :) I don't think you'll run out of vnodes as it is self adjusting (that was = my concern too) I changed kern.maxvnode to approx 3 times what it wanted and tuned for = my needs. Try it and keep an eye on: sysctl vfs.numvnodes vfs.wantfreevnodes vfs.freevnodes = vm.stats.vm.v_vnodepgsout vm.stats.vm.v_vnodepgsin Regards, Dewayne