Date: Fri, 11 Nov 2022 20:20:37 +0000 (GMT) From: andy thomas <andy@time-domain.co.uk> To: Mehmet Erol Sanliturk <m.e.sanliturk@gmail.com> Cc: andy thomas <andy@time-domain.co.uk>, freebsd-fs@freebsd.org Subject: Re: Odd behaviour of two identical ZFS servers mirroring via rsync Message-ID: <alpine.BSF.2.22.395.2211112008220.30520@mail0.time-domain.net> In-Reply-To: <CAOgwaMuoLQ9Er67Y%2B=q%2Bf9724v2WT3L5v5TZaRVXq%2B=1vEyJ%2BA@mail.gmail.com> References: <alpine.BSF.2.22.395.2211111709230.29479@mail0.time-domain.net> <CAOgwaMuoLQ9Er67Y%2B=q%2Bf9724v2WT3L5v5TZaRVXq%2B=1vEyJ%2BA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --951801389-1876059679-1668198037=:30520 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8BIT Yes, I can confirm the rsync --delete option is being used and in fact, 'du' reports some of the mirrored folders as having identical sizes on both servers, mainly those containing only small amounts of data. It seems almost as if ZFS is not freeing up blocks when rsync has deleted or shrank files, leaving unwanted blocks lurking around in the folder that 'du' then discovers and adds to its tally when it works out the space usage of that folder! I suppose I could always destroy the zfs dataset on the mirror server & resync the whole thing but will take days to complete even over a 10 Gbit/s network link (the servers ought to be upgraded to FBSD 13.1 as well). Andy On Fri, 11 Nov 2022, Mehmet Erol Sanliturk wrote: > > , Nov 11, 2022 at 8:42 PM andy thomas <andy@time-domain.co.uk> wrote: > I have two identical servers, called clustor2 and > clustor-backup, each > with a ZFS RAIDZ-1 pool containing 9 SAS hard disks plus one > spare and two > SSDs for the ZIL and ARC functions. clustor2 stores user data > from a > HPC while clustor2-backup uses rsync to mirrors all the data > from clustor2 > every 24 hours. > > However, the disk usage on the mirror server is considerably > more than on > the other server - attached is a screenshot showing the two > servers side > by side, with the mirror server on the right, and displaying the > contents > of the same subdirectory choen at random (named 'ratio_10.0' in > this > instance); as you can see, the sizes of the files within each of > the > folders are identical but 'du' reports very different > space usages for each folder and 'zpool list' also reports a > significant > difference in ZFS pool size. > > I'm not sure if this is relevant but both servers have ZFS pools > with no > compression although lz4 compression is enabled on the ZFS > filesystems & > both run FreeBSD 11.3 with ZFS version 5. > > Perhaps using zfs send/receive instead of rsync for mirroring > might solve > this disparity? > > Thanks in advance for any suggestions, > > Andy > > > > > Your question I am understanding the following points . > > > > I am using rsync in Fedora Linux . > > There are parameters of rsync such as > > --delete > > to delete files from the destination drive when they do not exist in the > source drive . > > > Please carefully scan rsync parameters and use suitable ones for your > application . > > > If a parameter like --delete is not used , rsync copies new files from > the source drive and > it does not delete any files from the destination drive . > > > With my best wishes for all . > > > Mehmet Erol Sanliturk > > > > > > > > > ---------------------------- Andy Thomas, Time Domain Systems Tel: +44 (0)7866 556626 http://www.time-domain.co.uk --951801389-1876059679-1668198037=:30520--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.22.395.2211112008220.30520>