Date: Sat, 12 Nov 2022 10:55:53 -0500 From: Rich <rincebrain@gmail.com> To: kaycee gb <kisscoolandthegangbang@hotmail.fr> Cc: freebsd-fs@freebsd.org Subject: Re: Odd behaviour of two identical ZFS servers mirroring via rsync Message-ID: <CAOeNLupyb92p2K83VL4%2BR5Y20zpPNsWw=3ouDKN=CNy36hfH4w@mail.gmail.com> In-Reply-To: <HE1PR0402MB3452B8E5387361B90C407B1EA0039@HE1PR0402MB3452.eurprd04.prod.outlook.com> References: <alpine.BSF.2.22.395.2211111709230.29479@mail0.time-domain.net> <HE1PR0402MB3452B8E5387361B90C407B1EA0039@HE1PR0402MB3452.eurprd04.prod.outlook.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --]
Hi everyone,
If you have an example file that's claiming to take up more space on one
side than the other, you could use zdb to see what it's doing on disk.
e.g.
$ ls -i /workspace/mirrors/centos/CentOS-8.1.1911-x86_64-dvd1.iso
441 /workspace/mirrors/centos/CentOS-8.1.1911-x86_64-dvd1.iso
$ sudo zdb -dbdbdbdbdbdb workspace/mirrors/centos 441
Dataset workspace/mirrors/centos [ZPL], ID 1069, cr_txg 33536418, 501G,
359422 objects, rootbp DVA[0]=<3:7023204000:1000>
DVA[1]=<3:7421125000:1000> [L0 DMU objset] skein uncompressed unencrypted
LE contiguous unique double size=1000L/1000P birth=39305123L/39305123P
fill=359422
cksum=1a2c0618fec098ea:27ad9c57dd26336a:a79b9e5413f126d7:98eb32d7beb1b658
Object lvl iblk dblk dsize dnsize lsize %full type
441 3 128K 128K 6.83G 512 7.04G 99.99 ZFS plain file
(K=inherit) (Z=inherit=zstd-unknown)
288 bonus System
attributes
dnode flags: USED_BYTES USERUSED_ACCOUNTED
dnode maxblkid: 57639
path /CentOS-8.1.1911-x86_64-dvd1.iso
uid 1002
gid 1002
atime Sat Mar 7 19:34:54 2020
mtime Sat Feb 22 15:58:48 2020
ctime Wed Apr 8 23:11:42 2020
crtime Wed Apr 8 23:11:02 2020
gen 24292265
mode 100764
size 7554990080
parent 4
links 1
pflags 40800000004
SA xattrs: 112 bytes, 1 entries
user.DOSATTRIB =
0x20\000\000\003\000\003\000\000\000\021\000\000\000
\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\020\006
c\341\364\325\001\000\000\000\000\000\000\000\000
Indirect blocks:
0 L2 DVA[0]=<2:399307d000:1000>
DVA[1]=<0:6c92a673000:1000> [L2 ZFS plain file] skein lz4 unencrypted LE
contiguous unique double size=20000L/1000P birth=33536573L/33536573P
fill=57636
cksum=4c5a8422b0199ec:8bda69b65610ddec:8ba4cc6c09a562b7:f5320ee2c5db878d
0 L1 DVA[0]=<1:57316ff4000:b000>
DVA[1]=<2:54335f23000:b000> [L1 ZFS plain file] skein lz4 unencrypted LE
contiguous unique double size=20000L/b000P birth=33536565L/33536565P
fill=1022
cksum=23bc863675aeedbe:1ce9e654a1463229:cd17146a117928cd:cfb1524c72123546
0 L0 DVA[0]=<2:4f75e232000:4000> [L0 ZFS plain file] skein
zstd unencrypted LE contiguous unique single size=20000L/4000P
birth=33536565L/33536565P fill=1
cksum=910008c8d6d0acfb:a5c6a7ee6f8d39de:8a5fcf7b14323a94:e5a9b7cfb00a4e98
[...]
And you can compare the block entries it prints and see why it might be
taking more space on one copy than the other...
- Rich
On Sat, Nov 12, 2022 at 4:37 AM kaycee gb <kisscoolandthegangbang@hotmail.fr>
wrote:
> Le Fri, 11 Nov 2022 17:42:44 +0000 (GMT),
> andy thomas <andy@time-domain.co.uk> a écrit :
>
> > I have two identical servers, called clustor2 and clustor-backup, each
> > with a ZFS RAIDZ-1 pool containing 9 SAS hard disks plus one spare and
> two
> > SSDs for the ZIL and ARC functions. clustor2 stores user data from a
> > HPC while clustor2-backup uses rsync to mirrors all the data from
> clustor2
> > every 24 hours.
> >
>
> Hi,
>
> For the mirroring part I would give zfs send/recv a try. I like rsync but
> I'm
> sure in this case zfs send/recv would be more efficient and faster.
>
> K.
>
>
>
[-- Attachment #2 --]
<div dir="ltr">Hi everyone,<div>If you have an example file that's claiming to take up more space on one side than the other, you could use zdb to see what it's doing on disk.</div><div><br></div><div>e.g.</div><div>$ ls -i /workspace/mirrors/centos/CentOS-8.1.1911-x86_64-dvd1.iso<br>441 /workspace/mirrors/centos/CentOS-8.1.1911-x86_64-dvd1.iso<br>$ sudo zdb -dbdbdbdbdbdb workspace/mirrors/centos 441<br>Dataset workspace/mirrors/centos [ZPL], ID 1069, cr_txg 33536418, 501G, 359422 objects, rootbp DVA[0]=<3:7023204000:1000> DVA[1]=<3:7421125000:1000> [L0 DMU objset] skein uncompressed unencrypted LE contiguous unique double size=1000L/1000P birth=39305123L/39305123P fill=359422 cksum=1a2c0618fec098ea:27ad9c57dd26336a:a79b9e5413f126d7:98eb32d7beb1b658<br><br> Object lvl iblk dblk dsize dnsize lsize %full type<br> 441 3 128K 128K 6.83G 512 7.04G 99.99 ZFS plain file (K=inherit) (Z=inherit=zstd-unknown)<br> 288 bonus System attributes<br> dnode flags: USED_BYTES USERUSED_ACCOUNTED<br> dnode maxblkid: 57639<br> path /CentOS-8.1.1911-x86_64-dvd1.iso<br> uid 1002<br> gid 1002<br> atime Sat Mar 7 19:34:54 2020<br> mtime Sat Feb 22 15:58:48 2020<br> ctime Wed Apr 8 23:11:42 2020<br> crtime Wed Apr 8 23:11:02 2020<br> gen 24292265<br> mode 100764<br> size 7554990080<br> parent 4<br> links 1<br> pflags 40800000004<br> SA xattrs: 112 bytes, 1 entries<br><br> user.DOSATTRIB = 0x20\000\000\003\000\003\000\000\000\021\000\000\000 \000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\020\006 c\341\364\325\001\000\000\000\000\000\000\000\000<br>Indirect blocks:<br> 0 L2 DVA[0]=<2:399307d000:1000> DVA[1]=<0:6c92a673000:1000> [L2 ZFS plain file] skein lz4 unencrypted LE contiguous unique double size=20000L/1000P birth=33536573L/33536573P fill=57636 cksum=4c5a8422b0199ec:8bda69b65610ddec:8ba4cc6c09a562b7:f5320ee2c5db878d<br> 0 L1 DVA[0]=<1:57316ff4000:b000> DVA[1]=<2:54335f23000:b000> [L1 ZFS plain file] skein lz4 unencrypted LE contiguous unique double size=20000L/b000P birth=33536565L/33536565P fill=1022 cksum=23bc863675aeedbe:1ce9e654a1463229:cd17146a117928cd:cfb1524c72123546<br> 0 L0 DVA[0]=<2:4f75e232000:4000> [L0 ZFS plain file] skein zstd unencrypted LE contiguous unique single size=20000L/4000P birth=33536565L/33536565P fill=1 cksum=910008c8d6d0acfb:a5c6a7ee6f8d39de:8a5fcf7b14323a94:e5a9b7cfb00a4e98<br></div><div>[...]</div><div><br></div><div>And you can compare the block entries it prints and see why it might be taking more space on one copy than the other...</div><div><br></div><div>- Rich</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Nov 12, 2022 at 4:37 AM kaycee gb <<a href="mailto:kisscoolandthegangbang@hotmail.fr">kisscoolandthegangbang@hotmail.fr</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Le Fri, 11 Nov 2022 17:42:44 +0000 (GMT),<br>
andy thomas <<a href="mailto:andy@time-domain.co.uk" target="_blank">andy@time-domain.co.uk</a>> a écrit :<br>
<br>
> I have two identical servers, called clustor2 and clustor-backup, each <br>
> with a ZFS RAIDZ-1 pool containing 9 SAS hard disks plus one spare and two <br>
> SSDs for the ZIL and ARC functions. clustor2 stores user data from a <br>
> HPC while clustor2-backup uses rsync to mirrors all the data from clustor2 <br>
> every 24 hours.<br>
><br>
<br>
Hi,<br>
<br>
For the mirroring part I would give zfs send/recv a try. I like rsync but I'm<br>
sure in this case zfs send/recv would be more efficient and faster. <br>
<br>
K.<br>
<br>
<br>
</blockquote></div>
home |
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOeNLupyb92p2K83VL4%2BR5Y20zpPNsWw=3ouDKN=CNy36hfH4w>