Date: Wed, 22 Feb 2017 18:49:43 -0600 From: "Eric A. Borisch" <eborisch@gmail.com> To: Wiktor Niesiobedzki <bsd@vink.pl> Cc: "Eugene M. Zheganin" <emz@norma.perm.ru>, "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org> Subject: Re: zfs raidz overhead Message-ID: <CAASnNnoLP32REdLMPcm0G6jR=wYLNH9V6uqRH=u-ukrE2G7RZA@mail.gmail.com> In-Reply-To: <CAH17caWPRtJVpTQNrqaabtYt7xR%2Boc-eL87tvea=pXjG12oEJg@mail.gmail.com> References: <1b54a2fe35407a95edca1f992fa08a71@norman-vivat.ru> <CAASnNnpB7NFWUbBLxKidXzsDMAwzcJzRc_f4R-9JG_=BZ9fA%2BA@mail.gmail.com> <CAH17caWPRtJVpTQNrqaabtYt7xR%2Boc-eL87tvea=pXjG12oEJg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Feb 22, 2017 at 3:50 PM, Wiktor Niesiobedzki <bsd@vink.pl> wrote: > I can add to this, that this is not only seen on raidz, but also on > mirror pools, such as this: > # zpool status tank > pool: tank > state: ONLINE > scan: scrub repaired 0 in 3h22m with 0 errors on Thu Feb 9 06:47:07 2017 > config: > > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > gpt/tank1.eli ONLINE 0 0 0 > gpt/tank2.eli ONLINE 0 0 0 > > errors: No known data errors > > > When I createted test zvols: > # zfs create -V10gb -o volblocksize=8k tank/tst-8k > # zfs create -V10gb -o volblocksize=16k tank/tst-16k > # zfs create -V10gb -o volblocksize=32k tank/tst-32k > # zfs create -V10gb -o volblocksize=64k tank/tst-64k > # zfs create -V10gb -o volblocksize=128k tank/tst-128k > > # zfs get used tank/tst-8k > NAME PROPERTY VALUE SOURCE > tank/tst-8k used 10.3G - > root@kadlubek:~ # zfs get used tank/tst-16k > NAME PROPERTY VALUE SOURCE > tank/tst-16k used 10.2G - > root@kadlubek:~ # zfs get used tank/tst-32k > NAME PROPERTY VALUE SOURCE > tank/tst-32k used 10.1G - > root@kadlubek:~ # zfs get used tank/tst-64k > NAME PROPERTY VALUE SOURCE > tank/tst-64k used 10.0G - > root@kadlubek:~ # zfs get used tank/tst-128k > NAME PROPERTY VALUE SOURCE > tank/tst-128k used 10.0G - Nope, that all looks correct. There is space reserved for metadata (checksum for example) when you create a zvol. Since checksums are computed by block, it makes sense that more metadata is required for the 8k volblocksize. > I also noted, that snapshots impact used stats far much, than usedbysnapshot value: > zfs get volsize,used,referenced,compressratio,volblocksize, > usedbysnapshots,usedbydataset,usedbychildren > tank/dkr-thinpool > NAME PROPERTY VALUE SOURCE > tank/dkr-thinpool volsize 10G local > tank/dkr-thinpool used 12.0G - > tank/dkr-thinpool referenced 1.87G - > tank/dkr-thinpool compressratio 1.91x - > tank/dkr-thinpool volblocksize 64K - > tank/dkr-thinpool usedbysnapshots 90.4M - > tank/dkr-thinpool usedbydataset 1.87G - > tank/dkr-thinpool usedbychildren 0 - > > > On a 10G volume, filled with 2G of data, and 90M used by snapshosts, > used is 2G. When I destroy the snapshots, used will drop to 10.0G. > It makes sense when you just remember that 'used' is enforcing a contract to the user that 10G of new data can be written to the zvol (refreservation, I'm sure it's set to 10G here), and still have all of the current snapshots will still be available. Breaking that down, usedbysnapshots refers to the data that is no longer referenced by the dataset (overwritten, in the case of a zvol.) Used is the space the the dataset is consuming (i.e. not available for other datasets). This means that 10G (refreservation)+ the referenced property of your earliest snapshot + the written value of all but your latest snapshot + the aforementioned metadata overhead needs to be reserved, or 'used' by the dataset. (I think I got that calculation right.) So even though usedbysnapshots is 90M (90M of blocks have been overwritten), the referenced and written properties of the snapshot accumulate to the space you're seeing here. Once all the snapshots are gone, used drops to (almost) the volsize. Hope that helps. If you want thin provisioning, you can unset the refreservation, or just create the volume with the '-s' option. - Eric
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAASnNnoLP32REdLMPcm0G6jR=wYLNH9V6uqRH=u-ukrE2G7RZA>