Date: Thu, 4 Nov 2010 14:25:05 +0000 From: krad <kraduk@gmail.com> To: Daan Vreeken <Daan@vehosting.nl> Cc: cronfy <cronfy@gmail.com>, freebsd-hackers@freebsd.org Subject: Re: Slow disk access while rsync - what should I tune? Message-ID: <AANLkTinvcS4_PVO0NtpYu8ADNL=q90WQ6AVKzEiJRqc1@mail.gmail.com> In-Reply-To: <201011011511.38244.Daan@vehosting.nl> References: <AANLkTikzZvZn=vNNRtcSViWq8ty7b8qOooQ4NbHiJH5q@mail.gmail.com> <AANLkTikpk4O-q_Omh9eAGZB474J1BVu2YJ7OKWvhZm7v@mail.gmail.com> <201011011511.38244.Daan@vehosting.nl>
next in thread | previous in thread | raw e-mail | index | archive | help
On 1 November 2010 14:11, Daan Vreeken <Daan@vehosting.nl> wrote: > Hi Cronfy, > > On Saturday 30 October 2010 23:48:45 cronfy wrote: > > Hello. > > > > > Every time backup starts server slows down significantly, disk > > > operations become very slow. It may take up to 10 seconds to stat() a > > > file that is not in filesystem cache. At the same time, rsync on > > > remote server does not affect disk load much, server works without > > > slowdown. > > > > Thank you all for the answers. > ... > > Also, is it possible to limit disk operations for rm -rf somehow? The > > only idea I have at the moment is to replace rm -rf with 'find | > > slow_down_script | xargs rm' (or use similar patch as for rsync)... > > Yes there is. You could use the same 'trick' I've added to rsync and limit > the > amount of I/O-creating system calls an application creates. > You could even create a small wrapper library that does this for a specific > application, without having to recompile or change the application. > > You can find a working proof of concept in "slowdown.c" here : > http://vehosting.nl/pub_diffs/ > > The library can be compiled with : > gcc -Wall -fPIC -shared -o slowdown.so slowdown.c > > Then start the application you want to I/O-limit with something like : > ( > export LD_PRELOAD=slowdown.so > export LD_LIBRARY_PATH=.:${LD_LIBRARY_PATH} > ls -R /a/random/huge/directory/ > ) > > (Assuming you start the application from withing the directory > where "slowdown.so" resides.) > This should work with rsync, ls and rm "out of the box", without changing > the > source of the applications. > > > Regards, > -- > Daan Vreeken > VEHosting > http://VEHosting.nl > tel: +31-(0)40-7113050 / +31-(0)6-46210825 > KvK nr: 17174380 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > Just a wild punt here, but are you using zfs on both systems? If you are look at doing incremental zfs sends as an alternative.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTinvcS4_PVO0NtpYu8ADNL=q90WQ6AVKzEiJRqc1>