Date: Tue, 15 Apr 2014 21:14:08 +1000 (EST) From: Ian Smith <smithi@nimnet.asn.au> To: Arthur Chance <freebsd@qeng-ho.org> Cc: John Mehr <jmehr@umn.edu>, Erich Dollansky <erichsfreebsdlist@alogt.com>, freebsd-questions@freebsd.org Subject: Re: size of source tree Message-ID: <20140415194933.Y55844@sola.nimnet.asn.au> In-Reply-To: <mailman.57.1397390401.89040.freebsd-questions@freebsd.org> References: <mailman.57.1397390401.89040.freebsd-questions@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
In freebsd-questions Digest, Vol 514, Issue 11, Message: 1
On Sat, 12 Apr 2014 13:04:09 +0100 Arthur Chance <freebsd@qeng-ho.org> wrote:
> On 12/04/2014 12:50, Matthew Seaman wrote:
> > On 12/04/2014 10:09, Erich Dollansky wrote:
> >> Hi,
> >>
> >> On Sat, 12 Apr 2014 09:26:23 +0100
> >> Matthew Seaman <m.seaman@infracaninophile.co.uk> wrote:
> >>
> >>> On 12/04/2014 04:54, Erich Dollansky wrote:
> >>>> The source tree dated 26.03.14 has a size of some 899GM while
> >>>> today's source tree has some 1.8GB.
> >>>>
> >>>> Why is it suddenly so huge?
> >>>>
> >>>
> >>> How were these source trees obtained? A checkout from SVN will have a
> >>> .svn directory containing pristine copies of all of the files, which
> >>> will pretty much double the space requirement.
> >>>
> >> both have been obtained with svn. The smaller one was started last year
> >> with the then current version of svn.
> >>
> >> Is there an option to turn this behaviour of keeping a copy off?
> >
> > Not if you want to use SVN. You can use freebsd-update to get system
> > sources -- but only from a release branch.
>
> Wasn't svnup (${PORTS}/net/svnup) designed for this? It fetches from svn
> respositories but doesn't use the .svn directory.
Indeed. I did some early testing (c. v0.56 IIRC) of svnup and sent John
Mehr (cc'd) some observations and a few suggestions on a then 9.1 system
that subsequently died. Early versions only used svn: protocol and were
pretty slow (and CPU-intensive) from here, before John rewrote the tree
indexing code and added http: and https: protocols.
So the other night I installed it from 9.2-R ports, getting v1.0 (since
upgraded to fix a bug with 10.0 I see). I fetched the 'release' target
to a work area so I could compare it with my virgin 9.2-R sources but
silly me hadn't noticed that target fetched base/releng/9.2 rather than
base/release/9.2.0, so all the ID headers were different of course :)
So I blew that away and did it again with release/9.2.0, after which
diff -r /usr/src /tmp/release returned nothing, well passing that test.
Both times took about two hours to pull the sources (using https:) which
still seemed kinda slow, with svnup never using over ~7% CPU, mostly
half that, so I don't really understand why it takes tat long, unless
the RTT to svn0.us-west.freebsd.org of ~215ms might be a factor?
I haven't tried svn for comparison but would expect to get more than
~170KB/s out of my 8-10Mbps otherwise idle link. Maybe someone could
comment on that, though I notice in the very next digest message:
> From: Lena@lena.kiev.ua
> [..]
> rm -rf /usr/src
> svn export svn://svn0.us-east.FreeBSD.org/base/releng/8.4 /usr/src
>
> Takes about 30 minutes.
To which Erich later responded:
> At your place, yes, not where I am most of the time.
So Erich, how does an initial two hours sound to you, to save 900MB?
You might do better from Europe to us-east than from here, dunno.
However a subsequent repeat run (updating nothing) took only a bit over
2 minutes, confirming earlier experience that the initial whole-tree
fetch is slow but null or small incremental updates are quite fast
enough for its intended use - ie a cvsup replacement for non-developers
who want to upgrade their sources, track stable and such without the
need for pushing patches (ie committing); for designed use I think it's
a useful and very lightweight tool; I'd recommend having a play with it.
As for overheads, svnup just keeps a full index of the src repository
(which for 9.2.0 is 800MiB) in
-rw-r--r-- 1 root wheel 3.5M Apr 14 02:37 /var/tmp/svnup/release
which is seriously light-weight vs the size doubling of /usr/src/.svn
I haven't tried svnup for ports; portsnap works well and quickly for me.
cheers, Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140415194933.Y55844>