From owner-freebsd-stable@FreeBSD.ORG Tue Oct 26 10:02:24 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 210831065670 for ; Tue, 26 Oct 2010 10:02:24 +0000 (UTC) (envelope-from erik@tefre.com) Received: from mta1-filtered.netlife.no (mail.netlife.no [62.92.26.226]) by mx1.freebsd.org (Postfix) with ESMTP id CCED48FC18 for ; Tue, 26 Oct 2010 10:02:23 +0000 (UTC) Received: from amavis2.netlife.no (unknown [10.115.1.12]) by mta1-filtered.netlife.no (Postfix) with ESMTP id AE7632EA541; Tue, 26 Oct 2010 12:02:21 +0200 (CEST) X-Virus-Scanned: amavisd-new at netlife.no Received: from mta1-submission.netlife.no ([62.92.26.226]) by amavis2.netlife.no (amavis2.netlife.no [10.115.1.12]) (amavisd-new, port 10026) with ESMTP id TdpgtwpJ1nPe; Tue, 26 Oct 2010 12:02:21 +0200 (CEST) Received: from baviandesktop.netlife.no (kontor.netlife.no [217.13.28.50]) (Authenticated sender: erik_pop@netlife.no) by mta1-submission.netlife.no (Postfix) with ESMTPA id 48F7B2EA53B; Tue, 26 Oct 2010 12:02:21 +0200 (CEST) Message-ID: <4CC6A72C.80406@tefre.com> Date: Tue, 26 Oct 2010 12:02:20 +0200 From: Erik Stian Tefre User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.12) Gecko/20100910 Thunderbird/3.0.7 MIME-Version: 1.0 To: Giulio Ferro References: <4CC18BB9.2030200@zirakzigil.org> In-Reply-To: <4CC18BB9.2030200@zirakzigil.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: ZFS wrong size stats with amavis X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Oct 2010 10:02:24 -0000 On 10/22/10 15:03, Giulio Ferro wrote: > I've seen people discuss about this and it happened to me as well. > > Freebsd 8.1 stable 25th september amd64 > This server has one single boot disk with ufs + 1 array (hardware, 3ware) > which hold 2,7TB data. I've formatted the latter with ZFS. > > Things seemed to work ok until I upgraded the system. One day the server > jails didn't boot anymore. I checked and the array occupation was 100% with > 0 byte free. > > This sounded strange, so I found a hidden file under > /var/amavis/.spamassassin. > I removed the dir, started again, but after a while it grew again until > the system became unusable. > > This is how I solved: I moved the amavis partition under UFS, then > mounted with nullfs that dir under /var in the zfs jail. > > I solved in the sense that it didn't grow anymore, but the fs occupation > stayed very near 100%, with only 1GB free space. > > If I launch du -s under /zfs (host machine, not jail) I get a total space > of about 750GB, but df -h always turns up with 2,7TB space occupied. > > I tried a zpool upgrade zfs and zpool scrub zfs, but to no avail. > > What should I do. short of moving the data, destroying the pool and > creating it again (which I can't very easily do)? I had the same problem with extremely large files in /var/amavis/.spamassassin after upgrading from 7.x to 8.1 (zpool mirror, no compression, zpool upgraded to v14, zfs filesystems upgraded to v3). This worked for me as a fix/workaround: zfs create pool/newamavisjail rm /pool/oldamavisjail/var/amavis/.spamassassin/[hugefiles] rsync ... /pool/oldamavisjail /pool/newamavisjail portsnap fetch update; portupgrade -af (in newamavisjail) restart newamavisjail I don't think I permanently lost disk space from the pool because of this. I guess you have checked that there are no snapshots referring to the huge files? (zfs list and zfs list -t snapshot) -- Erik