From owner-freebsd-fs@FreeBSD.ORG Fri Sep 27 15:00:31 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3C94C175 for ; Fri, 27 Sep 2013 15:00:31 +0000 (UTC) (envelope-from lkchen@k-state.edu) Received: from ksu-out.merit.edu (ksu-out.merit.edu [207.75.117.133]) by mx1.freebsd.org (Postfix) with ESMTP id CA8A6275E for ; Fri, 27 Sep 2013 15:00:30 +0000 (UTC) X-Merit-ExtLoop1: 1 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgIFAFOcRVLPS3TT/2dsb2JhbABagwc4UoMpvj4WdIIlAQEFI2IPGAICDRkCWRmIBgyZDY8EiROJDwSBKYxxBIFQglSBNgObVY4ig0CBTUE X-IronPort-AV: E=Sophos;i="4.90,993,1371096000"; d="scan'208";a="166652002" X-MERIT-SOURCE: KSU Received: from ksu-sfpop-mailstore02.merit.edu ([207.75.116.211]) by sfpop-ironport01.merit.edu with ESMTP; 27 Sep 2013 10:57:54 -0400 Date: Fri, 27 Sep 2013 10:56:49 -0400 (EDT) From: "Lawrence K. Chen, P.Eng." To: freebsd-fs@freebsd.org Message-ID: <378297647.87296597.1380293809297.JavaMail.root@k-state.edu> In-Reply-To: Subject: Re: zfs: the exponential file system from hell MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [70.179.133.134] X-Mailer: Zimbra 7.2.2_GA_2852 (ZimbraWebClient - GC29 ([unknown])/7.2.2_GA_2852) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 15:00:31 -0000 ----- Original Message ----- > On Fri, 27 Sep 2013 14:29:38 +0200, Attila Nagy wrote: > > > Hi, > > > > Did anyone try to fill a zpool with multiple zfs in it and graph > > the > > space accounted by df and zpool list? > > If not, here it is: > > https://picasaweb.google.com/104147045962330059540/FreeBSDZfsVsDf#5928271443977601554 > > > > The zpool in question: > > NAME STATE READ WRITE CKSUM > > mnt ONLINE 0 0 0 > > raidz2-0 ONLINE 0 0 0 > > da1 ONLINE 0 0 0 > > da2 ONLINE 0 0 0 > > da3 ONLINE 0 0 0 > > da4 ONLINE 0 0 0 > > raidz2-1 ONLINE 0 0 0 > > da5 ONLINE 0 0 0 > > da6 ONLINE 0 0 0 > > da7 ONLINE 0 0 0 > > da8 ONLINE 0 0 0 > > raidz2-2 ONLINE 0 0 0 > > da9 ONLINE 0 0 0 > > da10 ONLINE 0 0 0 > > da11 ONLINE 0 0 0 > > da13 ONLINE 0 0 0 > > > > And the zfs-es on it: > > NAME USED AVAIL REFER MOUNTPOINT > > mnt 23.3G 0 53.8K /mnt > > mnt/p1 3.89G 0 3.89G /mnt/p1 > > mnt/p2 3.89G 0 3.89G /mnt/p2 > > mnt/p3 3.89G 0 3.89G /mnt/p3 > > mnt/p4 3.89G 0 3.89G /mnt/p4 > > mnt/p5 3.89G 0 3.89G /mnt/p5 > > mnt/p6 3.89G 0 3.89G /mnt/p6 > > > > I have evenly filled the six zfs in 1/100th percents of the full > > space > > available, and graphed the results of zpool list's capacity and > > df's > > capacity. > > The x scale is the real space usage in percents. > > > > It's quite annoying when df says the file systems are 20% full, > > while in > > reality, they are at 60%. > > > > Any chance that it will be solved? > > So, if I understand you correctly, you compare the output of free > space of > zpool list (of the whole pool) to the output of df of one of size > zfs's. > So the conclusion is that zpool list shows you how much space there > is in > the pool and df shows the > amount-of-space-of-one-zfs-if-you-don't-do-change-other-zfs's-in-the-pool-in-the-meantime. > > This is not going to be fixed. It is by design. Choose the best tool > to > measure what you want to know. > But I understand it is a bit confusing if you come from traditional > fixed-size filesystems. > You can emulate a fixed-size filesystem by using quota's. Give all > zfs's > 1/6 of the space in quota and your df will work perfectly. > > NB: I'm not a ZFS developer so I don't have an authoritative opinion > about > all this. > > Regards, > Ronald. zpool list gives raw storage space. Whereas zfs list and df would deal in terms of usable fs space. And, a given block set, for raidz2, is N data blocks, plus 2 parity blocks, where N+2 <= total disks in vdev. Which can further complicate the conversion of raw storage to usable. (plus there's additional metadata if you make changes to only some of the data blocks in a block set...and more overhead since the old block set has to stay around until all its data blocks are orphaned.) -- Who: Lawrence K. Chen, P.Eng. - W0LKC - Senior Unix Systems Administrator For: Enterprise Server Technologies (EST) -- & SafeZone Ally