From owner-freebsd-performance@FreeBSD.ORG Mon Jun 2 08:05:25 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D343E37B401 for ; Mon, 2 Jun 2003 08:05:25 -0700 (PDT) Received: from svaha.com (svaha.com [64.46.156.67]) by mx1.FreeBSD.org (Postfix) with ESMTP id DC5A143F85 for ; Mon, 2 Jun 2003 08:05:24 -0700 (PDT) (envelope-from meconlen@obfuscated.net) Received: from obfuscated.net ([64.156.25.5]) (AUTH: LOGIN meconlen, TLS: TLSv1/SSLv3,128bits,RC4-MD5) by svaha.com with esmtp; Mon, 02 Jun 2003 11:05:23 -0400 Message-ID: <3EDB67B1.2040609@obfuscated.net> Date: Mon, 02 Jun 2003 11:05:21 -0400 From: Michael Conlen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Achim Patzner References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-performance@freebsd.org cc: damir@voljatel.si Subject: Re: copy 150G over 100Mbit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jun 2003 15:05:26 -0000 12.5MBytes/sec = 750 MBytes/min = 4500 MBytes/hour. Two problems, protocol overhead and disk speed. How fast can you read off that IDE system? Try tarring the filesystem and dumping it to /dev/null. If you can't get 12.5MBytes/sec then don't look to the network first.You should consider the size of your files. Sure your disks can move data fast, however if you have lots and lots of really small files your disks are going to be seeking all the time and reading very little of it. If your moving one large file there's less seek and more read. Also with lots of small files whatever protocol you use will have more overhead to eat more of your network bandwith. The difference can be an order of magnitude in disk throughput on reads. Over the network you will end up with all kinds of overhead for various protocols. NFS isn't the fastest thing in the world. Rsync has it's uses, but I don't think this is one of them. Consider using cpio over the network. Quick tests show that cpio gets better performance than tar by a long shot. Another consideration might be to move the entire filesystem. If you create identical sized partitions on each system, instead of mounting them just copy the whole filesystem across using ssh or rsh. I haven't tried this on FreeBSD but it shouldn't be to hard to get working. The idea being that you read from /dev/ad0s1f on one box and write to /dev/ad0s1f on the other. Something like dd if=/dev/ad0s1f | ssh hostname 'dd of=/dev/ad0s1f' You can try playing with the bs parameter to get optimal read/write speeds. A quick test on my system shows that the disk throughput increases as the block size increases up to about 8k from 4.5MBytes/sec at 512 to 12MBytes/sec at 8k. On a RAID setup it's likely to be larger. You may be able to dump to a larger partition on the destination system though the extra space would go unused. I don't have a good way to test this here. -- Michael Conlen Achim Patzner wrote: >Am 02.06.2003 16:23 Uhr schrieb "Damir Horvat" unter : > > >>I need to copy ~150Gbytes over switched 100Mbit network to new machine (in max >>7-8 hours). >> >>Source machine (IDE RAID 5) is in production, iostat shows average transfer of >>8-9Mbytes/sec. Destination box is FBC RAID 10. >> >> > >Excuse me, but what would you expect using a 100 M*bit*/s network >connection? > > >Achim > >_______________________________________________ >freebsd-performance@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-performance >To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > >