Date: Sat, 12 Nov 2022 08:24:27 +0000 (GMT) From: andy thomas <andy@time-domain.co.uk> To: Bob Friesenhahn <bfriesen@simple.dallas.tx.us> Cc: andy thomas <andy@time-domain.co.uk>, freebsd-fs@FreeBSD.org Subject: Re: Odd behaviour of two identical ZFS servers mirroring via rsync Message-ID: <alpine.BSF.2.22.395.2211120803420.32449@mail0.time-domain.net> In-Reply-To: <alpine.GSO.2.20.2211111444320.16889@scrappy.simplesystems.org> References: <alpine.BSF.2.22.395.2211111709230.29479@mail0.time-domain.net> <CAOgwaMuoLQ9Er67Y%2B=q%2Bf9724v2WT3L5v5TZaRVXq%2B=1vEyJ%2BA@mail.gmail.com> <alpine.BSF.2.22.395.2211112008220.30520@mail0.time-domain.net> <alpine.GSO.2.20.2211111444320.16889@scrappy.simplesystems.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Thank you for the suggestions, I'll set up a pair of test servers and experiment with adjusting the rsync block size to match the 128k ZFS record size, noting disk usage on both for varying file sizes. Buffering could well be an issue here with data on the server being mirrrored contantly changing within the HPC it supports (30 Linux compute nodes with up to 700 simultaneous user jobs) and this might be something I will just have to live with. Andy On Fri, 11 Nov 2022, Bob Friesenhahn wrote: > On Fri, 11 Nov 2022, andy thomas wrote: >> >> It seems almost as if ZFS is not freeing up blocks when rsync has deleted >> or shrank files, leaving unwanted blocks lurking around in the folder that >> 'du' then discovers and adds to its tally when it works out the space usage >> of that folder! > > This would be completely expected behavior if zfs snapshots are used. > > The rsync block sizes can be adjusted to be a better match for zfs block > sizes (e.g. 128k). For example, zfs will do a 'sync' to write new data to > disk and it will help if all of the data in an new/updated zfs block is > provided at the time of the 'sync' (rather than 1/4 or 1/2 of the block). > > Network buffering can also be a factor since it effects the timing of data > delivery to the backup server. If the sending side tends to stall, then add > more buffering. > > Bob > -- > Bob Friesenhahn > bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ > GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ > Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt > > ---------------------------- Andy Thomas, Time Domain Systems Tel: +44 (0)7866 556626 http://www.time-domain.co.uk
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.22.395.2211120803420.32449>