From owner-freebsd-fs@FreeBSD.ORG Wed Mar 3 08:57:43 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 51C8E106566C; Wed, 3 Mar 2010 08:57:43 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.16.84]) by mx1.freebsd.org (Postfix) with ESMTP id E508A8FC0C; Wed, 3 Mar 2010 08:57:42 +0000 (UTC) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by kabab.cs.huji.ac.il with esmtp id 1NmkOe-000PSY-JT; Wed, 03 Mar 2010 10:57:36 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: Rick Macklem In-reply-to: References: <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080220.ac6a2e4d.gerrit@pmp.uni-hannover.de> <4B892918.4080701@digiware.nl> <20100227202105.f31cbef7.gerrit@pmp.uni-hannover.de> <20100227193819.GA60576@icarus.home.lan> <4B89943C.70704@digiware.nl> <20100227220310.GA65110@icarus.home.lan> Comments: In-reply-to Rick Macklem message dated "Tue, 02 Mar 2010 19:52:40 -0500." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 03 Mar 2010 10:57:35 +0200 From: Daniel Braniss Message-ID: Cc: stable@freebsd.org, freebsd-fs@freebsd.org, Willem Jan Withagen , =?utf-8?B?RWlyaWsgw5h2ZXJieQ==?= , rwatson@freebsd.org, Jeremy Chadwick Subject: Re: mbuf leakage with nfs/udp (was mbuf leakage with nfs/zfs?) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2010 08:57:43 -0000 > > > On Tue, 2 Mar 2010, Daniel Braniss wrote: > > > runing with the experimental nfs server all is ok! > > (at least I can't see any mbuf leakage :-) > > > > so now that we can assume that the problem is in NFS/UDP writes via > > classic nfsserver, where to look? > > > > It might also be the krpc reply cache, since the experimental server > isn't using it (nfsv4 requires a rather twisted reply cache and it was > easier to just use that one for nfsv2,3 for the experimental server, > as well). > > >> If it doesn't go away, the problem is more likely in the krpc or the > >> generic udp code. (When I looked at svc_dg.c, I could only spot one > >> possible leak and you've already determined that patch doesn't help. > >> The other big difference when using udp on the FreeBSD8 krpc is the > >> reply cache code. I seem to recall it's an lru cache with a fixed upper > >> bound, but it might be broken and leaking. > >> > >> If you change the server to set sp_rcache = NULL in the initialization > >> function in sys/nfsserver/nfs_srvkrpc.c, I think that disables the replay > >> cache. You wouldn't want to run this way in production, but it would > >> determine if the leak is in it. > >> > >> Change the 3 lines in nfsrv_init() to: > >> nfsrv_pool->sp_rcache = NULL; > >> nfsrv_pool->sp_assign = NULL; > >> nfsrv_pool->sp_done = NULL; > >> > >> and I think the krpc replay cache will be disabled. > >> > > If someone gets a chance to try the above (not in production mode:-), > it will determine if the problem is in the reply cache or the nfs server's > write code. > >> Good luck with it and please report back if you get to try the above. > >> > > Thanks for trying the experimental server. It is getting narrowed down, > due to everyone's work on it. > disabling the krpc reply cache does it, no visible damage. Somehow this reminds me of my old 1970 beetle, parts would fall off but it would continue working :-) where to go from here? danny