Date: Thu, 4 Apr 2013 23:06:50 +0200 From: Joar Jegleim <joar.jegleim@gmail.com> To: Terje Elde <terje@elde.net>, freebsd-questions <freebsd-questions@freebsd.org> Subject: Re: Regarding zfs send / receive Message-ID: <CAFfb-hoDYEdZGo5gfv=PbyCUKuDC6N0ECn=27YzUYEW=C%2BeaLA@mail.gmail.com> In-Reply-To: <E019EA56-708A-481D-9FD8-5EB66D6B98AA@elde.net> References: <CAFfb-hqTFH0oK9rOpWHo6wrodzuOm5oRbetqY3RSXvF7Gsa6PA@mail.gmail.com> <E019EA56-708A-481D-9FD8-5EB66D6B98AA@elde.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Terje ! sorry for late reply, I've been checking my mail, forgetting that all my mailing list mail are sorted into their own folders skipping inbox :p the zfs sync setup is a huge advantage over rsync simply because incremental rsync of the volume takes ~12 hours, while the zfs differential snapshot's usually take less than a minute . Though it's only ~1TB of data, it's more than 2 million jpegs which rsync have to stat ... I'm guessing my predecessor who chose this setup, over for instance HAST, didn't feel confident enough regarding HAST in production ( I'm looking into that for a future solution) . There's no legacy stuff on the receiving end, old pools are deleted for every sync. I haven't got my script here but google pointed me too https://github.com/hoopty/zfs-sync/blob/master/zfs-sync which look like a script very similar to the one I'm using . In fact, I'm gonna take a closer look at that script and see what differs from my script (apart from it being much prettier :p ) I didn't know about zpool.cache, gonna check that tomorrow, thanks. -- ---------------------- Joar Jegleim Homepage: http://cosmicb.no Linkedin: http://no.linkedin.com/in/joarjegleim fb: http://www.facebook.com/joar.jegleim AKA: CosmicB @Freenode ---------------------- On 2 April 2013 14:40, Terje Elde <terje@elde.net> wrote: > On 2. apr. 2013, at 13.44, Joar Jegleim wrote: > > So my question(s) to the list would be: > > In my setup have I taken the use case for zfs send / receive too far > > (?) as in, it's not meant for this kind of syncing and this often, so > > there's actually nothing 'wrong'. > > I'm not sure if you've taken it too far, but I'm not entirely sure if > you're getting any advantage over using rsync or similar for this kind of > thing. > > First two things that spring to mind: > > Do you have any legacy stuff on the receiving machine? Things like > physically removed old zpools, that are still in zpool.cache, seems to slow > down various operations, including creation of new stuffs (such as the > snapshots you receive). > > Also, you don't mention if you're deleting old snapshots on the receiving > end? If you're doing an incremental run every 15 minutes, that's something > like 3000 snapshots pr. month, pr. filesystem. > > Terje > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFfb-hoDYEdZGo5gfv=PbyCUKuDC6N0ECn=27YzUYEW=C%2BeaLA>