From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 9 11:26:52 2004 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 97B8E16A4CE for ; Mon, 9 Feb 2004 11:26:52 -0800 (PST) Received: from ion.gank.org (ion.gank.org [69.55.238.164]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7519F43D2F for ; Mon, 9 Feb 2004 11:26:52 -0800 (PST) (envelope-from craig@tobuj.gank.org) Received: from localhost (ion.gank.org [69.55.238.164]) by ion.gank.org (mail) with ESMTP id 024702B4E9; Mon, 9 Feb 2004 13:26:50 -0600 (CST) Received: from ion.gank.org ([69.55.238.164]) by localhost (ion.gank.org [69.55.238.164]) (amavisd-new, port 10024) with LMTP id 16126-02-2; Mon, 9 Feb 2004 13:26:49 -0600 (CST) Received: from owen1492.uf.corelab.com (pix.corelab.com [12.45.169.2]) by ion.gank.org (mail) with ESMTP id E63F42B22B; Mon, 9 Feb 2004 13:26:48 -0600 (CST) From: Craig Boston To: Stijn Hoop Date: Mon, 9 Feb 2004 13:26:45 -0600 User-Agent: KMail/1.6 References: <200402091130.05656.craig@tobuj.gank.org> <20040209175314.GU2803@pcwin002.win.tue.nl> In-Reply-To: <20040209175314.GU2803@pcwin002.win.tue.nl> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200402091326.45172.craig@tobuj.gank.org> X-Virus-Scanned: by amavisd-new at gank.org cc: freebsd-hackers@freebsd.org Subject: Re: Subversion/CVS experiment summary X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Feb 2004 19:26:52 -0000 On Monday 09 February 2004 11:53 am, Stijn Hoop wrote: > Did you have to modify the script, or pass unusual options? I'd like to > reproduce this, but I didn't get very far when I tried a few days ago with > the 0.37.0 version of the tool. No, I used the script as-is. The version I have is LastChangedRevision: 8527, with a date of Jan. 29. It looks like that version is slightly newer than the one included with 0.37 (Rev 8512). One thing that may have made a difference is that so far I've been importing things in chunks rather than trying to do the whole repo at once. > but although it looks like it handles things much better (even vendor > branches etc), it loads EVERYTHING into memory -- which means that it > eventually grew to 1G of memory/swap at which point my memory was > exhausted, and this was at pass 2 of 7... Does the Python version do the same thing? I didn't think to look at memory usage very closely while it was running :-/ > > Comments on importing: It's SLOOOOOOOOOOW. It took 43.9 hours just for > > src/sys, and this is a relatively speedy system! It starts out at a > > pretty good pace, but the more commits it processes, the slower each one > > seems to take. > > That doesn't bode well. Is committing in the new SVN repository also slow? No, creating a new branch and committing new and changed files to it seems to be just as quick as with an empty repository. I haven't delved into the script enough to know for sure, so this is a wild guess, but I think the speed issue has to do with the script itself. I'm guessing that the method it uses to track the status/branch/etc. of the RCS files is subject to a linear slowdown. I'm going to attempt to verify this by doing a dump / load cycle on the repo that everything was imported into. If it goes quickly then we can assume it's the conversion script. If not, then there are bigger problems... > My thoughts were now going to do something like Tom Lord proposed for an > Arch gateway -- just import a CVS working copy into SVN at a certain > cut-off date, and setup a bi-directional gateway between the two. If I'm reading that right, it sounds similar to a thought I had about just routinely checking out snapshots and committing them on a vendor branch. Of course you'd have to do that separately for each branch you're interested in. IMO, I find it immensely useful to have the entire history of a file at hand. > Actually, would a sort of access control wrapper that is now also used with > the FreeBSD CVS repository not work? I do agree that it would be nice to > have per-directory access control with the svnserve method. Yes, I think the same sort of access hooks (pre-commit?) can be used. The Subversion manual even mentions that, I just forgot about it... That method has always seemed a little... hackish to me. You still need write access (file permissions) to almost the entire repo so it does nothing against editing the files directly -- though with SVN this is a little more difficult as it's all bdb files rather than plain text. Maybe there's a more secure way to do it with a restricted shell that I just don't know about. > > [ repeated merges ] > Which is a shame, this would be a major selling point. On the other hand, > considering the amount of work done and the fact that it really works quite > well already (at least for my small repository) should make people want to > switch :) In the release notes for 0.37 there is a brief blurb about "'svn merge' now notices ancestry by default". I'm not sure exactly what that means or if it's related... > > It also looks as if Subversion 0.37 (aka 1.0-RC) has just been released. > > I'll have to take a look at it and see if any of the problems noted above > > have been resolved. > > Please let me know the results! Will do! My local Subversion server (one running Apache, not the machine I've been doing the tests on) had just finished upgrading the port. I don't think it needs a dump/load cycle but I'm doing one anyway just to be safe... Craig