Date: Mon, 21 Aug 2023 16:15:39 +0200 From: freebsd@vanderzwan.org To: Kaya Saman <kayasaman@optiplex-networks.com> Cc: questions@freebsd.org Subject: Re: ZFS Root size keeps going down after upgrade to 13.2-release Message-ID: <005285FF-8BCB-4C65-AFCF-61A06265C530@vanderzwan.org> In-Reply-To: <f68aaedf-9c53-5e45-78ae-0c2e0abd7b95@optiplex-networks.com> References: <8f3b445d-3f55-f359-ce5e-0d86c57755fe@optiplex-networks.com> <20230821130132.50aeb96b51b2b71efa14cb94@sohara.org> <8dbe0d99-ca94-998d-0c4e-27642db1b11a@optiplex-networks.com> <362941E9-C354-4325-A691-6DC44E5F5AB6@vanderzwan.org> <ff0c28e6-154e-ca74-e8f0-1f36019f8617@optiplex-networks.com> <0178E106-F136-44F6-BB36-D94ABEE4BEDE@vanderzwan.org> <3caba337-1d4c-4555-d3ba-7d36278a4c9b@optiplex-networks.com> <0E0B068F-C160-445F-BCED-828694122C8C@vanderzwan.org> <f3894f18-2338-a8ee-65f0-5d53ef181993@optiplex-networks.com> <2842F607-E62B-4D83-95AE-B44AE769B77F@vanderzwan.org> <f68aaedf-9c53-5e45-78ae-0c2e0abd7b95@optiplex-networks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 21 Aug 2023, at 15:52, Kaya Saman <kayasaman@optiplex-networks.com> =
wrote:
>=20
>=20
> On 8/21/23 14:49, freebsd@vanderzwan.org wrote:
>>=20
>>> On 21 Aug 2023, at 15:41, Kaya Saman =
<kayasaman@optiplex-networks.com> wrote:
>>>=20
>>>=20
>>> On 8/21/23 14:38, freebsd@vanderzwan.org wrote:
>>>>> On 21 Aug 2023, at 15:34, Kaya Saman =
<kayasaman@optiplex-networks.com> wrote:
>>>>>=20
>>>>>=20
>>>>> On 8/21/23 14:23, freebsd@vanderzwan.org wrote:
>>>>>>> On 21 Aug 2023, at 15:14, Kaya Saman =
<kayasaman@optiplex-networks.com> wrote:
>>>>>>>=20
>>>>>>>=20
>>>>>>> On 8/21/23 14:07, freebsd@vanderzwan.org wrote:
>>>>>>>>> On 21 Aug 2023, at 14:21, Kaya Saman =
<kayasaman@optiplex-networks.com> wrote:
>>>>>>>>> # zfs list -ro space zroot/ROOT
>>>>>>>>> NAME AVAIL USED =
USEDSNAP USEDDS USEDREFRESERV USEDCHILD
>>>>>>>>> zroot/ROOT 720M 88.5G 0B =
88K 0B 88.5G
>>>>>>>>> zroot/ROOT/13.1-RELEASE-p5_2023-08-20_220010 720M 8K 0B =
8K 0B 0B
>>>>>>>>> zroot/ROOT/13.2-RELEASE-p2_2023-08-20_224534 720M 8K 0B =
8K 0B 0B
>>>>>>>>> zroot/ROOT/default 720M 88.5G =
52.0G 36.5G 0B 0B
>>>>>>>>>=20
>>>>>>>> That last line seems to indicate there is 52.0 G used in =
snapshots of zroot/ROOT/default and 36.5G in the dataset itself.
>>>>>>>> Output of zfs list -o space -t snapshot |grep =
zroot/ROOT/default should give a list of the snapshots and the sizes.
>>>>>>>>=20
>>>>>>>> Paul
>>>>>>>>=20
>>>>>>>>=20
>>>>>>> Thanks so much Paul.... I feel like I'm spinning around in =
circles currently...
>>>>>>>=20
>>>>>>>=20
>>>>>>> There doesn't seem to be anything indicative of that space being =
used up?
>>>>>>>=20
>>>>>>>=20
>>>>>>> zfs list -o space -t snapshot |grep zroot/ROOT/defaul
>>>>>>> zroot/ROOT/default@2022-11-28-06:08:26-0 - 583M - - =
- -
>>>>>>> zroot/ROOT/default@2022-11-28-06:38:48-0 - 11.4M - - =
- -
>>>>>>> zroot/ROOT/default@2022-11-28-07:26:20-0 - 9.41M - - =
- -
>>>>>>> zroot/ROOT/default@2023-08-20-22:00:10-0 - 34.5M - - =
- -
>>>>>>> zroot/ROOT/default@2023-08-20-22:45:34-0 - 9.56M - - =
- -
>>>>>>>=20
>>>>>>>=20
>>>>>>> certainly not 52GB?
>>>>>>>=20
>>>>>>>=20
>>>>>> That=E2=80=99s definitely confusing.
>>>>>> What=E2=80=99s the full output of 'zfs list -o space -r zroot=E2=80=
=99 and of 'zpool list -v zroot=E2=80=99
>>>>>>=20
>>>>>> Paul
>>>>>>=20
>>>>>>=20
>>>>> Here are the outputs:
>>>>>=20
>>>>>=20
>>>>> zfs list -o space -r zroot
>>>>> NAME AVAIL USED =
USEDSNAP USEDDS USEDREFRESERV USEDCHILD
>>>>> zroot 720M 107G 0B =
2.20G 0B 105G
>>>>> zroot/ROOT 720M 88.5G 0B =
88K 0B 88.5G
>>>>> zroot/ROOT/13.1-RELEASE-p5_2023-08-20_220010 720M 8K 0B =
8K 0B 0B
>>>>> zroot/ROOT/13.2-RELEASE-p2_2023-08-20_224534 720M 8K 0B =
8K 0B 0B
>>>>> zroot/ROOT/default 720M 88.5G 52.0G =
36.5G 0B 0B
>>>>> zroot/swap 3.78G 6.19G 0B =
3.11G 3.08G 0B
>>>>> zroot/tmp 720M 121M 0B =
121M 0B 0B
>>>>> zroot/usr 720M 7.09G 0B =
88K 0B 7.09G
>>>>> zroot/usr/home 720M 88K 0B =
88K 0B 0B
>>>>> zroot/usr/obj 720M 4.75G 0B =
4.75G 0B 0B
>>>>> zroot/usr/ports 720M 432K 0B =
176K 0B 256K
>>>>> zroot/usr/ports/distfiles 720M 168K 0B =
168K 0B 0B
>>>>> zroot/usr/ports/packages 720M 88K 0B =
88K 0B 0B
>>>>> zroot/usr/src 720M 2.35G 0B =
2.35G 0B 0B
>>>>> zroot/var 720M 2.60G 0B =
88K 0B 2.60G
>>>>> zroot/var/audit 720M 88K 0B =
88K 0B 0B
>>>>> zroot/var/backups 720M 141M 0B =
141M 0B 0B
>>>>> zroot/var/crash 720M 88K 0B =
88K 0B 0B
>>>>> zroot/var/db 720M 1.69G 0B =
1.59G 0B 94.8M
>>>>> zroot/var/db/pkg 720M 94.8M 0B =
94.8M 0B 0B
>>>>> zroot/var/empty 720M 88K 0B =
88K 0B 0B
>>>>> zroot/var/log 720M 792M 0B =
792M 0B 0B
>>>>> zroot/var/mail 720M 3.96M 0B =
3.96M 0B 0B
>>>>> zroot/var/run 720M 248K 0B =
248K 0B 0B
>>>>> zroot/var/tmp 720M 672K 0B =
672K 0B 0B
>>>>>=20
>>>>>=20
>>>>> zpool list -v zroot
>>>>> NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP =
DEDUP HEALTH ALTROOT
>>>>> zroot 111G 104G 7.22G - - 82% 93% =
1.00x ONLINE -
>>>>> mirror-0 111G 104G 7.22G - - 82% 93.5% =
- ONLINE
>>>>> ada0p3 111G - - - - - - - =
ONLINE
>>>>> ada1p3 111G - - - - - - - =
ONLINE
>>>>>=20
>>>> Those look quite normal. What about 'zfs list -r -t all -o space =
zroot/ROOT/default=E2=80=99 ?
>>>>=20
>>>> Paul
>>>>=20
>>>>=20
>>>>=20
>>> zfs list -r -t all -o space zroot/ROOT/default
>>> NAME AVAIL USED USEDSNAP =
USEDDS USEDREFRESERV USEDCHILD
>>> zroot/ROOT/default 720M 88.5G 52.0G =
36.5G 0B 0B
>>> zroot/ROOT/default@2022-11-28-06:08:26-0 - 583M - - =
- -
>>> zroot/ROOT/default@2022-11-28-06:38:48-0 - 11.4M - - =
- -
>>> zroot/ROOT/default@2022-11-28-07:26:20-0 - 9.41M - - =
- -
>>> zroot/ROOT/default@2023-08-20-22:00:10-0 - 34.5M - - =
- -
>>> zroot/ROOT/default@2023-08-20-22:45:34-0 - 9.56M - - =
- -
>>>=20
>>>=20
>>> Strange! It shows a 52GB 'claimed' snapshot but where is it?
>>>=20
>>>=20
>> Can you give output of 'zfs get all zroot/ROOT/default=E2=80=99 as =
well ? Maybe that gives a hint.
>>=20
>> Paul
>>=20
>=20
> Sure:
>=20
>=20
> zfs get all zroot/ROOT/default
> NAME PROPERTY VALUE SOURCE
> zroot/ROOT/default type filesystem -
> zroot/ROOT/default creation Tue Aug 7 2:53 2018 -
> zroot/ROOT/default used 88.5G -
> zroot/ROOT/default available 720M -
> zroot/ROOT/default referenced 36.5G -
> zroot/ROOT/default compressratio 1.29x -
> zroot/ROOT/default mounted yes -
> zroot/ROOT/default quota none default
> zroot/ROOT/default reservation none default
> zroot/ROOT/default recordsize 128K default
> zroot/ROOT/default mountpoint / local
> zroot/ROOT/default sharenfs off default
> zroot/ROOT/default checksum on default
> zroot/ROOT/default compression lz4 inherited from zroot
> zroot/ROOT/default atime on default
> zroot/ROOT/default devices on default
> zroot/ROOT/default exec on default
> zroot/ROOT/default setuid on default
> zroot/ROOT/default readonly off default
> zroot/ROOT/default jailed off default
> zroot/ROOT/default snapdir hidden default
> zroot/ROOT/default aclmode discard default
> zroot/ROOT/default aclinherit restricted default
> zroot/ROOT/default createtxg 93 -
> zroot/ROOT/default canmount noauto local
> zroot/ROOT/default xattr on default
> zroot/ROOT/default copies 1 default
> zroot/ROOT/default version 5 -
> zroot/ROOT/default utf8only off -
> zroot/ROOT/default normalization none -
> zroot/ROOT/default casesensitivity sensitive -
> zroot/ROOT/default vscan off default
> zroot/ROOT/default nbmand off default
> zroot/ROOT/default sharesmb off default
> zroot/ROOT/default refquota none default
> zroot/ROOT/default refreservation none default
> zroot/ROOT/default guid 15509736917917171623 -
> zroot/ROOT/default primarycache all default
> zroot/ROOT/default secondarycache all default
> zroot/ROOT/default usedbysnapshots 52.0G -
> zroot/ROOT/default usedbydataset 36.5G -
> zroot/ROOT/default usedbychildren 0B -
> zroot/ROOT/default usedbyrefreservation 0B -
> zroot/ROOT/default logbias latency default
> zroot/ROOT/default objsetid 58 -
> zroot/ROOT/default dedup off default
> zroot/ROOT/default mlslabel none default
> zroot/ROOT/default sync standard default
> zroot/ROOT/default dnodesize legacy default
> zroot/ROOT/default refcompressratio 1.32x -
> zroot/ROOT/default written 1.53G -
> zroot/ROOT/default logicalused 108G -
> zroot/ROOT/default logicalreferenced 46.0G -
> zroot/ROOT/default volmode default default
> zroot/ROOT/default filesystem_limit none default
> zroot/ROOT/default snapshot_limit none default
> zroot/ROOT/default filesystem_count none default
> zroot/ROOT/default snapshot_count none default
> zroot/ROOT/default snapdev hidden default
> zroot/ROOT/default acltype nfsv4 default
> zroot/ROOT/default context none default
> zroot/ROOT/default fscontext none default
> zroot/ROOT/default defcontext none default
> zroot/ROOT/default rootcontext none default
> zroot/ROOT/default relatime off default
> zroot/ROOT/default redundant_metadata all default
> zroot/ROOT/default overlay on default
> zroot/ROOT/default encryption off default
> zroot/ROOT/default keylocation none default
> zroot/ROOT/default keyformat none default
> zroot/ROOT/default pbkdf2iters 0 default
> zroot/ROOT/default special_small_blocks 0 default
>=20
>=20
That looks consistent with the rest of the data.
Man page for zfsprops mentions this for snapshots;
The used space of a snapshot (see the =
Snapshots
section of zfsconcepts(7)) is space that is
referenced exclusively by this snapshot. If =
this
snapshot is destroyed, the amount of used =
space
will be freed. Space that is shared by =
multiple
snapshots isn't accounted for in this metric. =
When
a snapshot is destroyed, space that was =
previously
shared with this snapshot can become unique =
to
snapshots adjacent to it, thus changing the =
used
space of those snapshots. The used space of =
the
latest snapshot can also be affected by =
changes in
the file system. Note that the used space of =
a
snapshot is a subset of the written space of =
the
snapshot.
Note : Space that is shared by multiple snapshots isn't accounted for =
in this metric.
My theory is that the snapshots/boot environments share about 52G of =
with each other but not with the parent dataset.
As there is no data unique to one snapshot it does not show
What you could do is delete all except one bootenvironment.
The snapshot data used should then be unique to that last snapshot and =
appear in the lists.
Paul
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?005285FF-8BCB-4C65-AFCF-61A06265C530>