From owner-freebsd-stable@FreeBSD.ORG Sat Feb 27 19:21:16 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 066461065670; Sat, 27 Feb 2010 19:21:16 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id 7A01E8FC1B; Sat, 27 Feb 2010 19:21:09 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.2/8.14.2) with ESMTP id o1RJL5oM028046; Sat, 27 Feb 2010 20:21:07 +0100 Received: from pmp.uni-hannover.de (theq.pmp.uni-hannover.de [130.75.117.4]) by www.pmp.uni-hannover.de (Postfix) with SMTP id DF62624; Sat, 27 Feb 2010 20:21:05 +0100 (CET) Date: Sat, 27 Feb 2010 20:21:05 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Willem Jan Withagen Message-Id: <20100227202105.f31cbef7.gerrit@pmp.uni-hannover.de> In-Reply-To: <4B892918.4080701@digiware.nl> References: <4B86F384.3010308@digiware.nl> <2a41acea1002251459v40e8c6ddxd0437decbada4594@mail.gmail.com> <4B8795B1.4020006@digiware.nl> <20100226120339.GB17798@icarus.home.lan> <20100226133138.d47dd080.gerrit@pmp.uni-hannover.de> <20100226134429.041ea6f2.gerrit@pmp.uni-hannover.de> <20100226141754.86ae5a3f.gerrit@pmp.uni-hannover.de> <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080220.ac6a2e4d.gerrit@pmp.uni-hannover.de> <4B892918.4080701@digiware.nl> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 2.4.2 (GTK+ 2.10.12; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PMX-Version: 5.5.9.388399, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2010.2.27.190629 Cc: freebsd-fs@freebsd.org, stable@freebsd.org, Jeremy Chadwick Subject: Re: mbuf leakage with nfs/zfs? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 19:21:16 -0000 On Sat, 27 Feb 2010 15:15:52 +0100 Willem Jan Withagen wrote about Re: mbuf leakage with nfs/zfs?: WJW> > 81492/2613/84105 mbufs in use (current/cache/total) WJW> > 80467/2235/82702/128000 mbuf clusters in use WJW> > (current/cache/total/max) 80458/822 mbuf+clusters out of packet WJW> > secondary zone in use (current/cache) WJW> Over the night I only had rsync and FreeBSD nfs traffic. WJW> WJW> 45337/2828/48165 mbufs in use (current/cache/total) WJW> 44708/1902/46610/262144 mbuf clusters in use (current/cache/total/max) WJW> 44040/888 mbuf+clusters out of packet secondary zone in use WJW> (current/cache) After about 24h I now have 128320/2630/130950 mbufs in use (current/cache/total) 127294/1200/128494/512000 mbuf clusters in use (current/cache/total/max) 127294/834 mbuf+clusters out of packet secondary zone in use (current/cache) WJW> I only have one Linux box runing Kubuntu 8.10, mounted UDP: WJW> (rw,udp,nolock,rsize=32768,wsize=32768,intr) Hm, are you able to narrow this down? Does a single Linux client with tcp mount cause the same trouble? Or a FreeBSD client with udp? If it was only Linux clients with udp mounts or something like this, I could understand why it took some time to pop up. WJW> But running something like 'find openembedded | xarg cat > /dev/null' WJW> Shows a steadily growing number of mbufs, and letting the system sit WJW> for 5 min. doesn't decrease the used mbufs I still have several udp and tcp mounts by Linux clients on my Server, though most of them are probably stale now after the upgrade; and my buffers keep draining... WJW> Doing this on another FreeBSD 7.2 client runs the mbufs up(max inc WJW> about 2000 mbuf), but within a few secs after the last file was WJW> fetched, the mbuf tab runs down to around to what is was before the WJW> command. FreeBSD client with udp mount? Then it is either Linux client with udp or all Linux clients triggering this leakage. I doubt that this is the case with all Linux clients, this would have caused more trouble earlier. WJW> Not shure where to go from here? I'm certainly not fluent enough in WJW> NFS to start interpreting a wireshark trace. Nor do I. I already wrote Rick Macklem an Email on Friday, but so far only got back an automated reply stating he is on "permanent vacation". I guess we need him or one of the other nfs guys to get this fixed. Could you try a single Linux client with tcp mount in the meantime? This would tell us if Linux clients as such are causing the issue, or if it is only Linux with udp mount. cu Gerrit P.S.: I cc'ed freebsd-fs because my PR went there.