Date: Mon, 14 Mar 2016 11:41:39 +0000 From: "Montgomery-Smith, Stephen" <stephen@missouri.edu> To: "ctm-users@freebsd.org" <ctm-users@freebsd.org> Subject: Re: Space problems with svn-cur Message-ID: <56E6A373.2040901@missouri.edu> In-Reply-To: <56E58CEA.1000009@missouri.edu> References: <201603120959.u2C9x6jU014905@fire.js.berklix.net> <56E4BBE1.9080007@missouri.edu> <56E5135F.70302@inse.ru> <56E58CEA.1000009@missouri.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On 03/13/2016 10:53 AM, Montgomery-Smith, Stephen wrote: > On 03/13/2016 01:14 AM, Roman Kurakin wrote: >> Hi, >> >> On 03/13/2016 04:01 AM, Montgomery-Smith, Stephen wrote: >>> On 03/12/2016 03:59 AM, Julian H. Stacey wrote: >>>> "Montgomery-Smith, Stephen" wrote: >>>>> The svn repository is now getting so large that the CTM generating >>>>> computer is running out of disk space when it creates the xEmpty delt= as >>>>> for svn-cur. The easist band-aid fix is for me to simply stop >>>>> generating these xEmpty deltas. Since CTM is no longer FreeBSD >>>>> approved, I'm going to assume that no-one new is going to join, and >>>>> hence this is not a problem. >>>>> >>>>> Comments? >>>> Hi Stephen >>>> There's plenty of space on ctm.berklix.org >>> The issue is that there isn't enough space to create the xEmpty deltas. >>> There is plenty of space to store the deltas. >>> >>> The svn repositories add up to about 35GB. Fully compressed, they >>> shrink to about 4GB. >>> >>> I'll look into other solutions. But I also get a feeling I am putting = a >>> lot of effort into something that is only used by a very few people. >> The xEmpty is the best way to start clean new copy on a new host. >> The solution could be a big deltas relative the last xEmpty. >=20 > This is the problem. Internally the svn xEmpty deltas look like this: >=20 > CTMTR <big number> > <huge tar file of the entire repository> >=20 > where the big number is the size of the following huge tar file. What I > have been doing is to create the huge tar file in a tmp directory, and > then find its size, and then copy this file onto the end of the xEmpty > file. This meant that tmp had to be able to take 25GB (the size of the > ports repository). >=20 > Having big deltas relative to the last xEmpty is not a solution, because > I still have to store the repository of every 100th delta. That's 25GB > whichever way you look at it. >=20 > What I am trying now is this. I create the tar file twice - once to get > the size, and the second time to append it to the xEmpty file. This > relies on tar creating the same size file both times. Experiments I > recently tried suggest this should be OK. This seems to be working. I created an xEmpty for 4406. I also found out it takes about 5 hours to create an xEmpty for svn-cur. Almost all that time is spent by xz doing the compression.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?56E6A373.2040901>