From owner-freebsd-fs@FreeBSD.ORG Thu Jun 23 15:23:10 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (unknown [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D6B5106566B for ; Thu, 23 Jun 2011 15:23:10 +0000 (UTC) (envelope-from hans@beastielabs.net) Received: from mail.beastielabs.net (beasties.demon.nl [82.161.3.114]) by mx1.freebsd.org (Postfix) with ESMTP id B8BA08FC0C for ; Thu, 23 Jun 2011 15:23:08 +0000 (UTC) Received: from testsoekris.hotsoft.nl (localhost [127.0.0.1]) by mail.beastielabs.net (8.14.4/8.14.4) with ESMTP id p5NEjehU021251; Thu, 23 Jun 2011 16:45:40 +0200 (CEST) (envelope-from hans@testsoekris.hotsoft.nl) Received: (from hans@localhost) by testsoekris.hotsoft.nl (8.14.4/8.14.4/Submit) id p5NEjcIP021250; Thu, 23 Jun 2011 16:45:38 +0200 (CEST) (envelope-from hans) Date: Thu, 23 Jun 2011 16:45:38 +0200 From: Hans Ottevanger To: Peter Jeremy Message-ID: <20110623144538.GA20990@testsoekris.hotsoft.nl> References: <20110617153415.GA92803@testsoekris.hotsoft.nl> <201106171842.p5HIgQjn018296@chez.mckusick.com> <20110621084645.GA68018@server.vk2pj.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110621084645.GA68018@server.vk2pj.dyndns.org> User-Agent: Mutt/1.4.2.3i Cc: Kirk McKusick , freebsd-fs@freebsd.org, Jeff Roberson Subject: Re: SU+J: negative used diskspace (for a while) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jun 2011 15:23:10 -0000 On Tue, Jun 21, 2011 at 06:46:45PM +1000, Peter Jeremy wrote: > On 2011-Jun-17 11:42:26 -0700, Kirk McKusick wrote: > >> Date: Fri, 17 Jun 2011 17:34:15 +0200 > >> From: Hans Ottevanger > >> So it takes more than a minute before the disk space is back to "normal" > >> values. > > >We used to account for deleted blocks at the instant that they were > >removed. This accounting was rather complex, so as part of doing > >SU+J, Jeff simplified it. Under the simplification, the removal is > >not accounted for until part way through the removal process. The > >result is that you now get these false negative block counts until > >the blocks have been partially reclaimed. If this behavior causes > >enough trouble, Jeff might be convinced that the more accurate block > >accounting is necessary. > > Negative values may also impact NFS clients - though just limiting the > reported used space to 0 should avoid them getting too upset. > It appears that negative values occur only when the filesystem is almost empty (like in my tests). Otherwise the values are just way too low for a while. > That said, whilst I haven't seen negative used values, ZFS and > Solaris UFS also take an extended period before 'df' reports > correct values (several minutes for Solaris UFS). In the case > of ZFS, even 'du' can report incorrect information for a while. > If I remember well, in "the old days", when Soft Updates was fairly new, it also had such issues in the first release, but it was repaired soon afterwards. Of course it is a real advantage to see the results immediately when you are cleaning out filesystems, especially when you are in a hurry. I just verified that the current issue was quite recently introduced with commit r222958. Before that 'df' reported changes promptly and correctly, even with Journaling enabled. As an answer to my original message to -current@ Peter Holm has assured me (http://lists.freebsd.org/pipermail/freebsd-current/2011-June/025318.html) that this is a known issue that is on Jeff's TODO list. All I can say to the UFS+J guys is: keep up the good work. Journaling is a very cool addition to UFS, amongst others a real time saver at boot time after system crashes. Just a few years ago I saw people writing that adding journaling to UFS was not really possible and it would probably never happen. Well, here it is, though I understand that it was a bit more difficult than foreseen and still needs a lot of work 8-). Thanks, guys. Kind regards, Hans Ottevanger