From owner-freebsd-isp@FreeBSD.ORG Wed Jun 25 03:21:35 2003 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F24A537B401 for ; Wed, 25 Jun 2003 03:21:34 -0700 (PDT) Received: from sl.pt (mail.sl.pt [212.55.140.13]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6E79743F75 for ; Wed, 25 Jun 2003 03:21:33 -0700 (PDT) (envelope-from japc@co.sapo.pt) Received: (qmail 8096 invoked by uid 0); 25 Jun 2003 10:21:27 -0000 Received: from unknown (HELO morgoth.sl.pt) ([172.28.229.134]) (envelope-sender ) by mail.sl.pt (qmail-ldap-1.03) with SMTP for ; 25 Jun 2003 10:21:27 -0000 Received: (qmail 1144 invoked by uid 500); 25 Jun 2003 10:19:52 -0000 Date: Wed, 25 Jun 2003 11:19:52 +0100 From: Jose Celestino To: freebsd-isp@freebsd.org Message-ID: <20030625101952.GB973@co.sapo.pt> Mail-Followup-To: freebsd-isp@freebsd.org References: <200306181340.06730.etienne@unix.za.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200306181340.06730.etienne@unix.za.org> User-Agent: Mutt/1.5.3i Subject: Re: What would be the best way to copy lots of files from one server to another. X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2003 10:21:35 -0000 Words by Etienne Ledoux [Fri, Jun 20, 2003 at 02:21:01PM +0200]: > Greetings, > > I need to copy lots of directories/files from one server to another. Approx > 45Gb . These files are mailboxes stored in maildir format. What would be the > best/safest/quickest way to do this ? > > e. > I've done this about 3 months ago when migrating our mailserver from the old (almost pre-historic) machine it was on to a new one. We had about 55Gb of data, all in maildir format, lots of small files. I left an rsync running over the night, syncing the /maildirs/ from the old, yet still production machine at that time, to the new one. I don't know how much it took for this initial sync, but I guess it took long, I/O on the old machine was awful and there's wasn't that much ram also. Next day, sometime before the switchover, I did another rsync, it took some (about 5) minutes. I then stopped everything except qmail (I stopped tcpserver of course) on the old machine, ALRMed qmail-send to force the immediate delivery of all messages... well, I only really cared about the local ones and indeed some remote messages remained to be delivered but that could be taken care afterwards. There was about 500 messages in queue at that time, about 22 of them where local ones. After no local deliveries pending I stopped qmail and did a last rsync (it took about a couple of minutes) and switched IPs on the machines. On the old one I routed all mail deliveries to the new one (so that I could catch the bounces of any failing remote messages) and started qmail so that it could stay trying to deliver the remotes until queuelifetime. All in all there was about 5 minutes downtime (there was really more as I took the chance to upgrade courier-imap and missed something on the init file that completelly fscked the authentications, but that was only imap). http://rsync.samba.org/ If you want sheer speed and do not want to mess with screwdrivers you should use the nc aproach: on the destination side: nc -l -p 6660 | tar xf - on the origin side: time tar cvf - * | nc DESTINATION_SERVER 6660 > > _______________________________________________ > freebsd-isp@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-isp > To unsubscribe, send any mail to "freebsd-isp-unsubscribe@freebsd.org" > -- Jose Celestino | http://xpto.org/~japc/files/japc-pgpkey.asc ---------------------------------------------------------------- "Quis Custodiet Ipsos Custodes" - "Who will guard the guards?" -- Juvenal's Satires, circa 120 AD