Date: Mon, 2 Jun 2003 13:25:23 -0400 From: David Gilbert <dgilbert@velocet.ca> To: damir@voljatel.si Cc: freebsd-performance@freebsd.org Subject: copy 150G over 100Mbit Message-ID: <16091.34947.952653.880037@canoe.velocet.net> In-Reply-To: <200306021623.17927.damir@voljatel.si> References: <200306021623.17927.damir@voljatel.si>
next in thread | previous in thread | raw e-mail | index | archive | help
>>>>> "Damir" == Damir Horvat <damir@voljatel.si> writes: Damir> Hi! I need to copy ~150Gbytes over switched 100Mbit network to Damir> new machine (in max 7-8 hours). Damir> Source machine (IDE RAID 5) is in production, iostat shows Damir> average transfer of 8-9Mbytes/sec. Destination box is FBC RAID Damir> 10. Damir> I've tryed rsync and cp over NFS. Both came down roughly to Damir> 2Gig's per hour, which is unacceptable. The only option left Damir> (as I see it) is to try with cross-over cable. Damir> I don't care for security as all transfers are in private Damir> network. Damir> Any ideas, how to approach this problem? Here are a few notes. As others have said, read and write speed are big factors. If you already have an 8 to 9 Mb/sec read rate on the old system, you may not be able to achieve another 12Mb/sec to fill the link. Also, many motherboards are going to have an overall limit to performance which is non linear (depends on number of cards and how much bandwidth each one does). NFS and rsync are not your best tools. They have no streaming. You want buffers at every point to eliminate bottlenecks. On one machine you might do: tar -cvf - . | team 1m 16 | nc two 6464 On the other: nc -l 6464 | team 1m 16 | tar -xvf - Someone also mentioned cpio being faster... I don't know if that will be true with enough buffering. Team (a port) is a good friend to have around. Softupdates will likely save your but. Make sure it's on for the target. You might even try the following sequence: touch FOOBAR do the backup wait for final day do the backup with files newer than FOOBAR most utilities will have a -newer switch. Dave. -- ============================================================================ |David Gilbert, Velocet Communications. | Two things can only be | |Mail: dgilbert@velocet.net | equal if and only if they | |http://daveg.ca | are precisely opposite. | =========================================================GLO================
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?16091.34947.952653.880037>