Date: Sun, 28 Feb 2010 14:11:42 +0200 From: Daniel Braniss <danny@cs.huji.ac.il> To: Jeremy Chadwick <freebsd@jdc.parodius.com> Cc: stable@freebsd.org, freebsd-fs@freebsd.org, Willem Jan Withagen <wjw@digiware.nl>, Gerrit =?iso-8859-1?Q?K=FChn?= <gerrit@pmp.uni-hannover.de>, Eirik =?iso-8859-1?Q?=D8verby?= <ltning@anduin.net>, rwatson@freebsd.org Subject: Re: mbuf leakage with nfs/zfs? Message-ID: <E1Nlhzr-0004ci-3W@kabab.cs.huji.ac.il> In-Reply-To: <20100227220310.GA65110@icarus.home.lan> References: <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <E1Nl6VA-000557-D9@kabab.cs.huji.ac.il> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080220.ac6a2e4d.gerrit@pmp.uni-hannover.de> <4B892918.4080701@digiware.nl> <20100227202105.f31cbef7.gerrit@pmp.uni-hannover.de> <20100227193819.GA60576@icarus.home.lan> <BD8AC9F6-DF96-41F9-8E92-48A4E5606DC7@anduin.net> <4B89943C.70704@digiware.nl> <20100227220310.GA65110@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Sat, Feb 27, 2010 at 10:53:00PM +0100, Willem Jan Withagen wrote: > > On 27-2-2010 21:32, Eirik Øverby wrote: > > >I've had a discussion with some folks on this for a while. I can easil=y > > >reproduce this situation by mounting a FreeBSD ZFS filesystem via > > >NFS-UDP from an OpenBSD machine. Telling the OpenBSD machine to use TC=P > > >instead of UDP makes the problem go away. > > > > > >Other FreeBSD systems mounting the same share, either using UDP or TCP=, > > >does not cause the problem to show up. > > > > > >A patch was suggested by Rick Macklem, but that did not solve the issu=e: > > >http://lists.freebsd.org/pipermail/freebsd-current/2009-December/01418=1.html>; > > > I concur. > > Everything in my network is now on TCP, and there is no mbuf leakage. > > I just don't get over the 5500 mark, no matter what I throw at it. > > > > I do feel that TCP is not as well performing on a local net with Linux, > > hence the choice for UDP. But TCP is workable as next best. > > I'm pulling in Robert Watson, who has some familiarity with the UDP > stack/code in FreeBSD. I'm not sure he'll be a sufficient source of > knowledge for this specific issue since it appears (?) to be specific to > NFS; Rick Macklem would be a better choice, but as reported, he's MIA. > > Robert, are you aware of any changes or implementation issues which > might cause excessive (read: leaking) mbuf use under UDP-based NFS? Do > you know of a way folks could determine the source of the leak, either > via DDB or while the system is live? I have been runing some tests in a controlled environment. server and client are both 64bit Xeon/X5550 @ 2.67GHz with 16Gb of memory FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 SMT threads the client is runing latest 8.0 stable the load is created by runing 'make -j32 buildworld' and sleeping 150 sec. in between runs, this is the straight line you will see in the graphs. Both the src and obj directories are NFS mounted from the server, regular UFS. when server is running 7.2-stable no leakage is seen. see ftp://ftp.cs.huji.ac.il/users/danny/freebsd/mbufs/{tcp,udp}-7.2.ps when server is runing 8.0-stable see ftp://ftp.cs.huji.ac.il/users/danny/freebsd/mbufs/{tcp,udp}-8.0.ps you can see that udp is leaking! cheers, danny ps: I think the subject should be changed again, removing zfs ...
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E1Nlhzr-0004ci-3W>