Date: Fri, 11 Nov 2022 23:02:19 +0300 From: Mehmet Erol Sanliturk <m.e.sanliturk@gmail.com> To: andy thomas <andy@time-domain.co.uk> Cc: freebsd-fs@freebsd.org Subject: Re: Odd behaviour of two identical ZFS servers mirroring via rsync Message-ID: <CAOgwaMuoLQ9Er67Y%2B=q%2Bf9724v2WT3L5v5TZaRVXq%2B=1vEyJ%2BA@mail.gmail.com> In-Reply-To: <alpine.BSF.2.22.395.2211111709230.29479@mail0.time-domain.net> References: <alpine.BSF.2.22.395.2211111709230.29479@mail0.time-domain.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--00000000000026b43905ed376002 Content-Type: text/plain; charset="UTF-8" , Nov 11, 2022 at 8:42 PM andy thomas <andy@time-domain.co.uk> wrote: > I have two identical servers, called clustor2 and clustor-backup, each > with a ZFS RAIDZ-1 pool containing 9 SAS hard disks plus one spare and two > SSDs for the ZIL and ARC functions. clustor2 stores user data from a > HPC while clustor2-backup uses rsync to mirrors all the data from clustor2 > every 24 hours. > > However, the disk usage on the mirror server is considerably more than on > the other server - attached is a screenshot showing the two servers side > by side, with the mirror server on the right, and displaying the contents > of the same subdirectory choen at random (named 'ratio_10.0' in this > instance); as you can see, the sizes of the files within each of the > folders are identical but 'du' reports very different > space usages for each folder and 'zpool list' also reports a significant > difference in ZFS pool size. > > I'm not sure if this is relevant but both servers have ZFS pools with no > compression although lz4 compression is enabled on the ZFS filesystems & > both run FreeBSD 11.3 with ZFS version 5. > > Perhaps using zfs send/receive instead of rsync for mirroring might solve > this disparity? > > Thanks in advance for any suggestions, > > Andy Your question I am understanding the following points . I am using rsync in Fedora Linux . There are parameters of rsync such as --delete to delete files from the destination drive when they do not exist in the source drive . Please carefully scan rsync parameters and use suitable ones for your application . If a parameter like --delete is not used , rsync copies new files from the source drive and it does not delete any files from the destination drive . With my best wishes for all . Mehmet Erol Sanliturk --00000000000026b43905ed376002 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div class=3D"gmail_default" style=3D"fon= t-family:tahoma,sans-serif;font-size:large"><br></div></div><div dir=3D"ltr= "><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">, Nov 11= , 2022 at 8:42 PM andy thomas <<a href=3D"mailto:andy@time-domain.co.uk"= target=3D"_blank">andy@time-domain.co.uk</a>> wrote:<br></div><blockquo= te class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px = solid rgb(204,204,204);padding-left:1ex">I have two identical servers, call= ed clustor2 and clustor-backup, each <br> with a ZFS RAIDZ-1 pool containing 9 SAS hard disks plus one spare and two = <br> SSDs for the ZIL and ARC functions. clustor2 stores user data from a <br> HPC while clustor2-backup uses rsync to mirrors all the data from clustor2 = <br> every 24 hours.<br> <br> However, the disk usage on the mirror server is considerably more than on <= br> the other server - attached is a screenshot showing the two servers side <b= r> by side, with the mirror server on the right, and displaying the contents <= br> of the same subdirectory choen at random (named 'ratio_10.0' in thi= s <br> instance); as you can see, the sizes of the files within each of the <br> folders are identical but 'du' reports very different <br> space usages for each folder and 'zpool list' also reports a signif= icant <br> difference in ZFS pool size.<br> <br> I'm not sure if this is relevant but both servers have ZFS pools with n= o <br> compression although lz4 compression is enabled on the ZFS filesystems &= ; <br> both run FreeBSD 11.3 with ZFS version 5.<br> <br> Perhaps using zfs send/receive instead of rsync for mirroring might solve <= br> this disparity?<br> <br> Thanks in advance for any suggestions,<br> <br> Andy</blockquote><div><br></div><div><br></div><div><br></div><div><div sty= le=3D"font-family:tahoma,sans-serif;font-size:large" class=3D"gmail_default= ">Your question I am understanding the following points .</div><br></div><d= iv><br></div><div><br></div><div><div style=3D"font-family:tahoma,sans-seri= f;font-size:large" class=3D"gmail_default">I am using=C2=A0 rsync=C2=A0 in = Fedora Linux .</div><div style=3D"font-family:tahoma,sans-serif;font-size:l= arge" class=3D"gmail_default"><br></div><div style=3D"font-family:tahoma,sa= ns-serif;font-size:large" class=3D"gmail_default">There are=C2=A0 parameter= s of=C2=A0 rsync=C2=A0 such as <br></div><div style=3D"font-family:tahoma,s= ans-serif;font-size:large" class=3D"gmail_default"></div><div style=3D"font= -family:tahoma,sans-serif;font-size:large" class=3D"gmail_default"><br></di= v><div style=3D"font-family:tahoma,sans-serif;font-size:large" class=3D"gma= il_default">=C2=A0--delete <br></div><div style=3D"font-family:tahoma,sans-= serif;font-size:large" class=3D"gmail_default"><br></div><div style=3D"font= -family:tahoma,sans-serif;font-size:large" class=3D"gmail_default">to delet= e files from the destination drive when they do not exist in the source dri= ve .</div><div style=3D"font-family:tahoma,sans-serif;font-size:large" clas= s=3D"gmail_default"><br></div><div style=3D"font-family:tahoma,sans-serif;f= ont-size:large" class=3D"gmail_default"><br></div><div style=3D"font-family= :tahoma,sans-serif;font-size:large" class=3D"gmail_default">Please carefull= y scan=C2=A0 rsync=C2=A0 parameters and use suitable ones for your applicat= ion .</div><div style=3D"font-family:tahoma,sans-serif;font-size:large" cla= ss=3D"gmail_default"><br></div><div style=3D"font-family:tahoma,sans-serif;= font-size:large" class=3D"gmail_default"><br></div><div style=3D"font-famil= y:tahoma,sans-serif;font-size:large" class=3D"gmail_default">If=C2=A0 a par= ameter like=C2=A0 --delete=C2=A0 is not used , rsync=C2=A0 copies new files= from the source drive and</div><div style=3D"font-family:tahoma,sans-serif= ;font-size:large" class=3D"gmail_default">it does not delete any files from= the destination drive .</div><div style=3D"font-family:tahoma,sans-serif;f= ont-size:large" class=3D"gmail_default"><br></div><div style=3D"font-family= :tahoma,sans-serif;font-size:large" class=3D"gmail_default"><br></div><div = style=3D"font-family:tahoma,sans-serif;font-size:large" class=3D"gmail_defa= ult">With my best wishes for all .</div><div style=3D"font-family:tahoma,sa= ns-serif;font-size:large" class=3D"gmail_default"><br></div><div style=3D"f= ont-family:tahoma,sans-serif;font-size:large" class=3D"gmail_default"><br><= /div><div style=3D"font-family:tahoma,sans-serif;font-size:large" class=3D"= gmail_default">Mehmet Erol Sanliturk<br></div><div style=3D"font-family:tah= oma,sans-serif;font-size:large" class=3D"gmail_default"><br></div><div styl= e=3D"font-family:tahoma,sans-serif;font-size:large" class=3D"gmail_default"= ><br></div><div style=3D"font-family:tahoma,sans-serif;font-size:large" cla= ss=3D"gmail_default"><br></div><div style=3D"font-family:tahoma,sans-serif;= font-size:large" class=3D"gmail_default"><br></div><div style=3D"font-famil= y:tahoma,sans-serif;font-size:large" class=3D"gmail_default"><br></div><div= style=3D"font-family:tahoma,sans-serif;font-size:large" class=3D"gmail_def= ault"></div><br></div><div>=C2=A0</div></div></div> </div> --00000000000026b43905ed376002--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOgwaMuoLQ9Er67Y%2B=q%2Bf9724v2WT3L5v5TZaRVXq%2B=1vEyJ%2BA>