From owner-freebsd-fs@FreeBSD.ORG Tue Feb 8 09:58:25 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67790106564A for ; Tue, 8 Feb 2011 09:58:25 +0000 (UTC) (envelope-from admin@kkip.pl) Received: from mainframe.kkip.pl (kkip.pl [87.105.164.78]) by mx1.freebsd.org (Postfix) with ESMTP id 17DAE8FC1D for ; Tue, 8 Feb 2011 09:58:24 +0000 (UTC) Received: from static-78-8-144-74.ssp.dialog.net.pl ([78.8.144.74] helo=[192.168.0.2]) by mainframe.kkip.pl with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.73 (FreeBSD)) (envelope-from ) id 1Pmjny-000Nrw-P0; Tue, 08 Feb 2011 10:24:22 +0100 Message-ID: <4D510BBB.1060708@kkip.pl> Date: Tue, 08 Feb 2011 10:24:11 +0100 From: Bartosz Stec User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; pl; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: Emil Muratov References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Authenticated-User: admin@kkip.pl X-Authenticator: plain X-Sender-Verify: SUCCEEDED (sender exists & accepts mail) X-Spam-Score: -8.5 X-Spam-Score-Int: -84 X-Exim-Version: 4.73 (build at 10-Jan-2011 16:29:01) X-Date: 2011-02-08 10:24:22 X-Connected-IP: 78.8.144.74:54351 X-Message-Linecount: 66 X-Body-Linecount: 53 X-Message-Size: 2711 X-Body-Size: 2099 X-Received-Count: 1 X-Recipient-Count: 3 X-Local-Recipient-Count: 3 X-Local-Recipient-Defer-Count: 0 X-Local-Recipient-Fail-Count: 0 Cc: freebsd-fs@freebsd.org, pjd@freebsd.org Subject: Re: Memory leak in ZFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2011 09:58:25 -0000 W dniu 2011-02-07 22:37, Emil Muratov pisze: > >> For the past few weeks, I noticed that the amount of memory reported >> in top >> (sum of active, inact, wired, cache buf and free) keeps decreasing as >> the >> uptime increases. I can't pinpoint to when I first noticed this, as I >> have >> updated the system a few times just in case this has been fixed. > > Yes, I have the same issue on my home file storage. My system is 8.1 > amd64, 2G ram, zfs on root raidz with 4x1,5T drives. > After updating to stable a couple of days ago I noticed that the > system leaks memory very fast. Checking here and there I found that > the issue concerns sendfile (yep, again!). > > How to reproduce: > Configure samba with aio and sendfile (mine is version 3.5.6) > > smb.conf > [global] > use sendfile=true > aio read size = 16384 > > Download a couple of large samba shared files (8-10 gigs). > > > While downloading files I can see that memory decreazes to nowhere > very-very fast, several MBs per second! First it drains free mem, than > active and inactive, than comes wired until the whole system commits > suicide suffocating itself to the death. > The only way to free memory is to reboot the system. I can't unload > zfs module like PJD suggested to do, 'cause my root is on zfs :( > I'll try to make a bootable flash and move root to the flash to try to > unload module and what will happen. > > Everything was OK in stable before the new year, sendfile used to pump > free and wired memory to inactive than slowly reclaiming it back. But > it seems something was changed after NY holydays? I'm glad someone else finally picked that problem, so there's appareantly no memory-eating ghost in my machine ;) Here's my thread on stable list about this issue: http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/061247.html And in fact, PC reported in thread above is also SAMBA server with aio/sendfile enabled and ZFS. I would be happy testing some patches if necessary, because until now I need to monitor memory and reboot this server before it dies. -- Bartosz Stec