From owner-freebsd-fs@freebsd.org Tue Jul 14 09:56:03 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06AD799B30D for ; Tue, 14 Jul 2015 09:56:03 +0000 (UTC) (envelope-from accounts-ag@netlabs.org) Received: from r2-d2.netlabs.org (r2-d2.netlabs.org [213.238.45.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 599C8AE7 for ; Tue, 14 Jul 2015 09:56:01 +0000 (UTC) (envelope-from accounts-ag@netlabs.org) Received: (qmail 24499 invoked by uid 89); 14 Jul 2015 09:49:17 -0000 Received: from unknown (HELO eternal-3.metropolis.netlabs.org) (accounts-ag@netlabs.org@213.144.156.18) by 0 with ESMTPA; 14 Jul 2015 09:49:17 -0000 To: freebsd-fs@freebsd.org From: accounts-ag Subject: ZFS volume does not seem to free space for deleted files Message-ID: <55A4DB1D.7000404@netlabs.org> Date: Tue, 14 Jul 2015 11:49:17 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.0.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Jul 2015 09:56:03 -0000 Hi, I'm trying to clean up a zfs volume used by a FreeBSD jail: NAME USED AVAIL REFER MOUNTPOINT tank/jail/www 116G 68.3G 4.57G /tank/jail/www The USED space was probably accurate before I freed a lot of space on this device by cleaning up /var/tmp and some (a lot...) system mails which accumulated over years. According to "du" this at least freed +50GB. However, somehow ZFS does not free that yet. In the stopped jail starting from /tank/jail/www, "du" (or ncdu to be precise) reports: Total disk usage: 4.6 GiB Apparent size: 4.6 GiB Items: 76516 This seems to correspond with what is listed as REFER. So I checked space usage over everything on tank/jail/www: # zfs list -t all -o space -r tank/jail/www NAME AVAIL USED USEDSNAP USEDDS tank/jail/www 68.3G 116G 112G 4.57G tank/jail/www@monthly-2015-07-01_00.00.00--3m - 29.0M - - tank/jail/www@daily-2015-07-01_00.00.00--2w - 27.8M - - tank/jail/www@daily-2015-07-02_00.00.00--2w - 1.22G - - tank/jail/www@daily-2015-07-03_00.00.00--2w - 1.27G - - tank/jail/www@daily-2015-07-04_00.00.00--2w - 1.52G - - tank/jail/www@daily-2015-07-05_00.00.00--2w - 62.6M - - tank/jail/www@weekly-2015-07-05_00.00.00--1m - 62.5M - - tank/jail/www@daily-2015-07-06_00.00.00--2w - 1.51G - - tank/jail/www@daily-2015-07-07_00.00.00--2w - 1.38G - - tank/jail/www@daily-2015-07-08_00.00.00--2w - 1.39G - - tank/jail/www@daily-2015-07-09_00.00.00--2w - 1.49G - - tank/jail/www@daily-2015-07-10_00.00.00--2w - 1.17G - - tank/jail/www@daily-2015-07-11_00.00.00--2w - 1.33G - - tank/jail/www@weekly-2015-07-12_00.00.00--1m - 119M - - tank/jail/www@daily-2015-07-12_00.00.00--2w - 117M - - tank/jail/www@hourly-2015-07-12_04.04.00--2d - 255M - - tank/jail/www@hourly-2015-07-12_10.04.00--2d - 265M - - tank/jail/www@hourly-2015-07-12_16.04.00--2d - 267M - - tank/jail/www@hourly-2015-07-12_22.04.00--2d - 204M - - tank/jail/www@daily-2015-07-13_00.00.00--2w - 257M - - tank/jail/www@hourly-2015-07-13_04.04.00--2d - 257M - - tank/jail/www@hourly-2015-07-13_10.04.00--2d - 260M - - tank/jail/www@hourly-2015-07-13_16.04.00--2d - 1.70G - - tank/jail/www@hourly-2015-07-13_22.04.00--2d - 48.4M - - tank/jail/www@daily-2015-07-14_00.00.00--2w - 32.4M - - tank/jail/www@hourly-2015-07-14_04.04.00--2d - 20.0M - - tank/jail/www@hourly-2015-07-14_10.04.00--2d - 18.9M - - So according to my calculation it is using less than 20GB of space for snapshots. Which does not make much sense to me as I clearly do expect the deleted files to show up somewhere (I did it on 2015-07-13). So how can the big difference to USED be explained which reports 116G in total? Would I have to delete all snapshots to properly free the disk space? I was unmounting the volume meanwhile and remounting it but that didn't change anything. zpool info: # zpool list -o name,size,allocated,free,freeing NAME SIZE ALLOC FREE FREEING tank 295G 222G 72.9G 0 thanks regards Adrian