Date: Sun, 06 May 2001 14:38:12 -0700 From: Peter Wemm <peter@wemm.org> To: hubs@FreeBSD.ORG Cc: will@physics.purdue.edu Subject: Re: ftp-master method Message-ID: <20010506213812.3BAFE380E@overcee.netplex.com.au> In-Reply-To: <200105041644.f44GiuO54477@vashon.polstra.com>
next in thread | previous in thread | raw e-mail | index | archive | help
John Polstra wrote: > In article <20010502161330.B5017@casimir.physics.purdue.edu>, Will > Andrews <will@physics.purdue.edu> wrote: > > > OK, so people have problems with either mirror/spegla+ftp and rsync. > > How about cvsup? > > I've suggested this a few times myself. I haven't heard any arguments > against it, but nobody ever seems to consider it seriously for this > application. It wouldn't be hard to try it and find out how well it > works. > > > It uses the rsync algorithm with some optimizations, > > Well, to be fair, I doubt it compresses quite as well as rsync. > (Rsync is tuned better.) But CVSup probably streams better, and I > wouldn't be surprised if it was faster overall in terms of elapsed > time. One gripe I have with rsync itself is that it seems to build its file lists right from the start and holds them in memory.. It then streams the updates. This means that rsyncd consumes about 50-60MB of ram on the server at all times (for an update). It does its delete pass right at then end, if nothing went wrong. Does cvsupd do better? (ie: transfer the file list incrementally?) One thing does bother me though, is that cvsup tends to ignore files that didn't exist. Having cvsup 'take over' maintainence of an existing (out of date) tree seems likely to leave behind lots of leftovers. You have script to clean this up, right? Or does cvsup have an even stronger "mirror" mode that will get all this stuff? > > although it does hurt when it comes to cpu. > > Hmm -- maybe, maybe not. It uses a lot of CPU for dealing with RCS > files, because it parses them and edits them and generally spends a > lot of time grokking them. The rsync algorithm is much simpler, so > CVSup's CPU usage should be better for that case. The archive has got 5 or 6 checked out trees in it, including several source checkouts, mailing list archives and the ncvs tree. cvsupd can use the checkouts.cvs file to load the metadata for the ,v files quickly, right? (pub/FreeBSD/development/FreeBSD-CVS/*, including a sup/ dir). Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hubs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20010506213812.3BAFE380E>