Date: Mon, 1 Nov 2010 15:11:38 +0100 From: Daan Vreeken <Daan@vehosting.nl> To: cronfy <cronfy@gmail.com> Cc: freebsd-hackers@freebsd.org Subject: Re: Slow disk access while rsync - what should I tune? Message-ID: <201011011511.38244.Daan@vehosting.nl> In-Reply-To: <AANLkTikpk4O-q_Omh9eAGZB474J1BVu2YJ7OKWvhZm7v@mail.gmail.com> References: <AANLkTikzZvZn=vNNRtcSViWq8ty7b8qOooQ4NbHiJH5q@mail.gmail.com> <AANLkTikpk4O-q_Omh9eAGZB474J1BVu2YJ7OKWvhZm7v@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Cronfy, On Saturday 30 October 2010 23:48:45 cronfy wrote: > Hello. > > > Every time backup starts server slows down significantly, disk > > operations become very slow. It may take up to 10 seconds to stat() a > > file that is not in filesystem cache. At the same time, rsync on > > remote server does not affect disk load much, server works without > > slowdown. > > Thank you all for the answers. ... > Also, is it possible to limit disk operations for rm -rf somehow? The > only idea I have at the moment is to replace rm -rf with 'find | > slow_down_script | xargs rm' (or use similar patch as for rsync)... Yes there is. You could use the same 'trick' I've added to rsync and limit the amount of I/O-creating system calls an application creates. You could even create a small wrapper library that does this for a specific application, without having to recompile or change the application. You can find a working proof of concept in "slowdown.c" here : http://vehosting.nl/pub_diffs/ The library can be compiled with : gcc -Wall -fPIC -shared -o slowdown.so slowdown.c Then start the application you want to I/O-limit with something like : ( export LD_PRELOAD=slowdown.so export LD_LIBRARY_PATH=.:${LD_LIBRARY_PATH} ls -R /a/random/huge/directory/ ) (Assuming you start the application from withing the directory where "slowdown.so" resides.) This should work with rsync, ls and rm "out of the box", without changing the source of the applications. Regards, -- Daan Vreeken VEHosting http://VEHosting.nl tel: +31-(0)40-7113050 / +31-(0)6-46210825 KvK nr: 17174380
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201011011511.38244.Daan>