Date: Wed, 29 Sep 2010 12:25:34 -0700 From: Jeremy Chadwick <freebsd@jdc.parodius.com> To: Torbjorn Kristoffersen <torbjoern@gmail.com> Cc: freebsd-fs@freebsd.org Subject: Re: Strange ZFS problem, filesystem claims to be full when clearly not full Message-ID: <20100929192534.GA97031@icarus.home.lan> In-Reply-To: <AANLkTimRmBi=th1oia5ZuKcEtLR%2BYjK04KNYeZhu931A@mail.gmail.com> References: <AANLkTimRmBi=th1oia5ZuKcEtLR%2BYjK04KNYeZhu931A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Sep 29, 2010 at 08:46:38PM +0200, Torbjorn Kristoffersen wrote: > I have a ZFS "tank" called tpool, the server runs a couple of jails (each > with a zfs filesystem). There is a problem with one of these filesystems. > First, its disk usage as shown in ``df -h'': > ... > tpool/rb.org 100G 95G 4.6G 95% /jails/rb.org > ... > > The command ``zfs list'' shows the same: > .. > tpool/rb.org 95.4G 4.56G 95.4G /jails/rb.org > .. > > However, there is a very mysterious problem somewhere. > Something inside this jail is eating diskspace, but we can't find any > directories that is actually taking the diskspace. We first suspected either > fetchmail or spamassassin of causing a lot of space to be used, since some > of their directories were huge. (These were later deleted, and which is why > you see that 4.6GB is now available, before that 0GB was available). > > However, we can't find *any trace* of an actual directory or file that is > taking all the spac.e > > Take this for instance: > > outsidejail# du -sh rb.org > 43G rb.org > > How can this be? df and zfs are showing that the entire drive is nearly > full, yet I can't find any directory that is actually taking all this space. > I've carefully looked through every single directory within the jail trying > to find something that's taking all that space, but to no avail. > > ---- > My system stats: > # uname -a > FreeBSD grim 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:36:49 UTC > 2010 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 > # zpool get version tpool > NAME PROPERTY VALUE SOURCE > tpool version 14 default > # zpool status > pool: tpool > state: ONLINE > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > tpool ONLINE 0 0 0 > mirror ONLINE 0 0 0 > ad4s1d ONLINE 0 0 0 > ad6s1d ONLINE 0 0 0 > > errors: No known data errors > > [ Note that I've also done a scrub recently ] 1) Have you checked using fstat to ensure that no file descriptors remain open on any of your ZFS filesystems (not pools)? 2) Are you using compression on any of your ZFS filesystems? -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100929192534.GA97031>