Date: Sun, 6 May 2001 19:31:48 -0700 (PDT) From: John Polstra <jdp@polstra.com> To: peter@wemm.org Cc: hubs@freebsd.org Subject: Re: ftp-master method Message-ID: <200105070231.f472Vmq62912@vashon.polstra.com> In-Reply-To: <20010506213812.3BAFE380E@overcee.netplex.com.au> References: <20010506213812.3BAFE380E@overcee.netplex.com.au>
next in thread | previous in thread | raw e-mail | index | archive | help
In article <20010506213812.3BAFE380E@overcee.netplex.com.au>,
Peter Wemm <peter@wemm.org> wrote:
>
> One gripe I have with rsync itself is that it seems to build its file
> lists right from the start and holds them in memory.. It then streams
> the updates. This means that rsyncd consumes about 50-60MB of ram on the
> server at all times (for an update). It does its delete pass right at then
> end, if nothing went wrong.
>
> Does cvsupd do better? (ie: transfer the file list incrementally?)
Yes, it does. When doing rsync-style updates it still uses the
streaming model shown in the diagram at
http://www.polstra.com/projects/freeware/CVSup/howsofast.html
Updates for the first files will be coming in while the file list is
still streaming out in the other direction.
> One thing does bother me though, is that cvsup tends to ignore files
> that didn't exist. Having cvsup 'take over' maintainence of an
> existing (out of date) tree seems likely to leave behind lots of
> leftovers.
You're right. I hadn't thought of that problem with respect to the
FTP tree. We certainly don't want to make people repopulate their
zillion-GB FTP trees.
> You have script to clean this up, right?
Yes, there is a script "cvsupchk" which does exactly that in the
contrib subdirectory of the CVSup distribution (thanks to Joseph
Koshy).
> Or does cvsup have an even stronger "mirror" mode that will get all
> this stuff?
Not currently. But this makes it a higher priority. It's not that
hard in principle. Hmm ...
> The archive has got 5 or 6 checked out trees in it, including
> several source checkouts, mailing list archives and the ncvs tree.
> cvsupd can use the checkouts.cvs file to load the metadata for the
> ,v files quickly, right? (pub/FreeBSD/development/FreeBSD-CVS/*,
> including a sup/ dir).
Yes, definitely. It will use its RCS file algorithm on RCS files
and rsync on other files.
John
--
John Polstra jdp@polstra.com
John D. Polstra & Co., Inc. Seattle, Washington USA
"Disappointment is a good sign of basic intelligence." -- Chögyam Trungpa
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hubs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200105070231.f472Vmq62912>