From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 3 09:34:54 2009 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD799106568B for ; Thu, 3 Sep 2009 09:34:54 +0000 (UTC) (envelope-from kraduk@googlemail.com) Received: from mail-bw0-f206.google.com (mail-bw0-f206.google.com [209.85.218.206]) by mx1.freebsd.org (Postfix) with ESMTP id 10DFA8FC23 for ; Thu, 3 Sep 2009 09:34:53 +0000 (UTC) Received: by bwz2 with SMTP id 2so1320174bwz.43 for ; Thu, 03 Sep 2009 02:34:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=uVIxYSvYT6oXFgS/3tfsY3GTzyI87uVt2PTFTskxkfk=; b=tTdikVX72//mzgCcsHIlgLmPjMybzu1MBnv/1LY64ZHw+fQ3KDfq1js1MKWgb8PCev YFIHrrptILcMYfQ7VFNOgib6z9RtiXj91wjzyCQEKbOnHYlGdKxXsh7269WCUorqQBxY cRgYIQu7UKNsBEINIUwTaroDB9fy+ud5vHV90= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=Gfap3XvhvzDbWjfvv295bmHK5REL41oWBPgosenRU+s9pkxpqNFRpeK2v0jstyWYcE YuTCauSjLxJBpwHdh0cWp3db+vXBQbavfSZ2qx7rt9YWrqWXbm93yteVfEffxkQHMSLY QhXF9xp7TrPRF+iBqkx6Oo91FyvCNR+5Fp/R0= MIME-Version: 1.0 Received: by 10.103.81.38 with SMTP id i38mr4032681mul.92.1251968688146; Thu, 03 Sep 2009 02:04:48 -0700 (PDT) In-Reply-To: References: <20090902193059.M336@Robert-Eckardt.de> <78cb3d3f0909030009y505a30by769052258576bfeb@mail.gmail.com> <20090903071913.M84990@Robert-Eckardt.de> <20090903080558.M13772@Robert-Eckardt.de> Date: Thu, 3 Sep 2009 10:04:48 +0100 Message-ID: From: krad To: Robert Eckardt X-Mailman-Approved-At: Thu, 03 Sep 2009 11:25:03 +0000 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: hackers@freebsd.org, Adrian Penisoara Subject: Re: Fw: Re: ZFS continuously growing X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Sep 2009 09:34:54 -0000 2009/9/3 krad > > > 2009/9/3 Robert Eckardt > > On Thu, 3 Sep 2009 09:09:19 +0200, Adrian Penisoara wrote >> > Hi, >> > >> > On Wed, Sep 2, 2009 at 10:22 PM, Robert Eckardt >> > wrote: >> > > Do I have to be worried? >> > > Is there a memory leak in the current ZFS implementation? >> > > Why is used space growing slower than free space is shrinking? >> > > Is there some garbage collection needed in ZFS? >> > > >> > > Besides, although the backup server has 3 GB RAM I had to tune arc_max >> > > to 150MB to copy the backed-up data from an 2.8TB ZFS (v6) to the >> > > 4.5 TB ZFS (v13) by "zfs send|zfs recv" without kmalloc panic. >> > > (I.e., the defaults algorithm was not sufficient.) >> > >> > Do I take you are using ZFS snapshots in between rsync'ing >> > (send/recv requires snapshots) ? Could you please post the "zfs >> > list" output after subsequent runs to clarify ? >> > >> > Regards, >> > Adrian >> > EnterpriseBSD >> >> Hi Adrian, >> >> no I'm not using snapshots. Just seperate directories, where identical >> files are hardlinked by rsync to the version one day older. >> The send|recv was neccessary when I increased the raidz of the backup-fs. >> (Copying everthing to two 1.5TB HDDs and after adding disks back again. >> I used s.th. like "zfs send bigpool/big@backup | zfs recv big/big".) >> >> Here the zfs list of the last five days: >> Thu Sep 3 09:36:12 CEST 2009 (Today add. 2GB of data were transfered.) >> big 1861882752 0 1861882752 0% 5 14545959 >> 0% /big >> big/big 4676727168 2814844416 1861882752 60% 43137409 14545959 >> 75% /big/big >> NAME USED AVAIL REFER MOUNTPOINT >> big 2.72T 1.73T 31.5K /big >> big/big 2.72T 1.73T 2.62T /big/big >> >> Wed Sep 2 09:36:24 CEST 2009 >> big 1869058944 128 1869058816 0% 5 14602022 >> 0% /big >> big/big 4679698688 2810639872 1869058816 60% 43226966 14602022 >> 75% /big/big >> NAME USED AVAIL REFER MOUNTPOINT >> big 2.72T 1.74T 31.5K /big >> big/big 2.72T 1.74T 2.62T /big/big >> >> Tue Sep 1 09:36:33 CEST 2009 >> big 1875352064 0 1875352064 0% 5 14651188 >> 0% /big >> big/big 4683241856 2807889792 1875352064 60% 43316454 14651188 >> 75% /big/big >> NAME USED AVAIL REFER MOUNTPOINT >> big 2.71T 1.75T 31.5K /big >> big/big 2.71T 1.75T 2.62T /big/big >> >> Mon Aug 31 09:45:26 CEST 2009 >> big 1881967616 128 1881967488 0% 5 14702871 >> 0% /big >> big/big 4686380928 2804413440 1881967488 60% 43406044 14702871 >> 75% /big/big >> NAME USED AVAIL REFER MOUNTPOINT >> big 2.71T 1.75T 31.5K /big >> big/big 2.71T 1.75T 2.61T /big/big >> >> Sun Aug 30 09:39:31 CEST 2009 >> big 1891064192 0 1891064192 0% 5 14773939 >> 0% /big >> big/big 4694821376 2803757184 1891064192 60% 43496712 14773939 >> 75% /big/big >> NAME USED AVAIL REFER MOUNTPOINT >> big 2.70T 1.76T 31.5K /big >> big/big 2.70T 1.76T 2.61T /big/big >> >> Regards, >> Robert >> >> -- >> Dr. Robert Eckardt --- Robert.Eckardt@Robert-Eckardt.de >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org >> " >> > > do a " zfs list -t all" > > you will see all snapshots and zvols then as well > > also might be worth doing a "zfs get all| grep copies" just to make sure you haven't got multiple copies assigned to any fs It sounds like for some reason the free blocks are getting put back into the free block table/map Also try scrubbing the zpool for good measure. Its worth croning it once a week